Remove port from vlan correctly