Don't break if ipset contains/missing ip to add/del