Fix L2 agent does not remove unused ipset set

The patch fixes L2 agent does not remove unused ipset set when security
group contains rules for only IPv4 or IPv6.

Change-Id: I375b1683cd763c0a33dc935558c637874d36ffa1
Closes-bug: #1373287
Merge "Fix L2 agent does not remove unused ipset set"