# Patent

## Title
Maintaining point of presence at access switch for roaming clients in distributed wireless controller system

## Abstract

Techniques are provided to enable support of roaming wireless devices in a network such that the wireless devices can keep their Internet Protocol (IP) addresses as they roam across mobility sub-domains. Traffic for a wireless device that roams is tunneled back to the access switch that serves the IP subnet which includes an IP address for the wireless device. Traffic is tunneled back to that access switch for the wireless device when the wireless device roams to another access switch which does not serve the IP subnet for the wireless device in the same mobility sub-domain and when the wireless device roams to a different mobility sub-domain, in which case the traffic is tunneled between tunneling endpoints in the respective mobility sub-domains.

## Background
RELATED APPLICATIONS
This application is related to the following commonly assigned co-pending U.S. Patent Applications filed on May 4, 2010:
U.S. patent application Ser. No. 12/773,360, entitled “Maintaining Point Of Presence At Tunneling Endpoint For Roaming Clients In Distributed Wireless Controller System.”
U.S. patent application Ser. No. 12/773,355, entitled “Routing To The Access Layer To Support Mobility Of Internet Protocol Devices.”
TECHNICAL FIELD
The present disclosure relates to networking techniques capable of supporting mobility of a network device.
BACKGROUND
Networked services to wired and wireless devices are supported by equipment that makes up what may be referred to as the “infrastructure” of the network. Examples of equipment in the network infrastructure include routers, access switches and control computers or servers that are used to store data pertaining to the status of devices that connect to the network. Some access switches have routing capabilities and in this regard are also referred to as “forwarders” because they forward packets from one access switch to another.
A device with networking capability, referred to herein as a “client device” or “station”, may connect to the network at one access switch and then physically move, i.e., roam, such that it connects to a different access switch in the network. This roaming capability is prevalent with client devices that have wireless capabilities and can connect to a wired network at a different access switch by establishing a wireless connection, such as a wireless local area network (WLAN) connection with a wireless access point (AP) device.
Wireless is one of the many services being integrated within an access switch. The wireless service ensures that the access layer terminates the data plane, to facilitate the delivery of a converged architecture. Convergence implies that regardless of whether the traffic originates from a wired or wireless station the same set of features can be applied. There will be unification at the management and configuration level for the end customer, and wired and wireless will become an access method into the organization. Unification implies that mobility services are provided to both wireless and wired stations.



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is an example of a diagram of a network infrastructure architecture.
FIG. 2 is an example of a block diagram of an access switch that is part of the network infrastructure architecture shown in FIG. 1.
FIG. 3 is an example of a block diagram of mobility controller apparatus that is part of the network infrastructure architecture shown in FIG. 1.
FIG. 4 is an example of a block diagram of a mobility tunnel endpoint apparatus that is part of the network infrastructure architecture shown in FIG. 1.
FIG. 5 is an example of a block diagram of a mobility oracle apparatus that is part of the network infrastructure shown in FIG. 1.
FIG. 6 is an example of a diagram showing part of the network infrastructure depicted in FIG. 1 and further illustrating a device roaming between access switches in a mobility sub-domain.
FIG. 7 is an example of a flow diagram illustrating control signaling between entities for the roaming scenario depicted in FIG. 6.
FIG. 8 is an example of a diagram showing part of the network infrastructure depicted in FIG. 1 and further illustrating a device roaming between access switches across mobility sub-domains.
FIG. 9 is an example of a flow diagram illustrating control signaling between entities for the roaming scenario depicted in FIG. 8.



DESCRIPTION OF EXAMPLE EMBODIMENTS
Overview
Techniques are provided herein to enable support for roaming of wireless client devices in a network such that the wireless devices can keep their Internet Protocol (IP) addresses as they roam within and across mobility sub-domains. Traffic for a wireless device that roams is tunneled back to an access switch where it was connected to the network and which is responsible for handling traffic for the group of IP addresses (IP subnet) that includes an IP address of the wireless device. Traffic is tunneled back to that access switch for the wireless device when the wireless device roams to another access switch which does not serve that IP subnet for the wireless device in the same mobility sub-domain and when the wireless device roams to a different mobility sub-domain, in which case the traffic is tunneled between tunneling endpoints in the respective mobility sub-domains.
In order to provide topology transparency to stations/devices as they roam within the network, the network infrastructure equipment needs to provide for seamless roaming. From the station's perspective, any mobility event should not change its IP address, its default router or Dynamic Host Configuration Protocol (DHCP) server. This means that as stations roam, they send an Address Resolution Protocol (ARP) request to their default router, or transmit a DHCP request to the server that had previously assigned its address. A hierarchical mobility architecture is provided that supports seamless mobility in networks where wireless services are terminated at the access switch. The point of presence of the roamed station stays at an access switch that serves the IP subnet for the station after it has roam to another access switch.
Example Embodiments
Reference is first made to FIG. 1. FIG. 1 illustrates a diagram depicting a network infrastructure architecture that is configured to support mobility of wireless client devices. The network architecture 5 comprises a mobility domain shown at reference numeral 10. A mobility domain is a geographical region for which roaming services are to be provided. Contiguous coverage is intended to be provided in this geographical region by the network architecture 5. The network architecture 5 provides better scaling properties over existing systems in that it breaks down the traditional mobility group into multiple mobility sub-domains. Thus, a mobility domain includes one or more mobility sub-domains. For simplicity, FIG. 1 shows two sub-domains 20(1) and 20(2) and labeled Mobility Sub-Domain 1 and Mobility Sub-Domain 2, respectively. For instance, a mobility sub-domain could consist of a single building within a campus. A sub-domain is more of a representation of the network topology than the physical walls of a building, so it is also possible for a mobility sub-domain to span multiple buildings in a campus, for example.
The network architecture 5 further comprises a mobility controller and a mobility oracle. In one form, each mobility sub-domain includes one or more mobility controllers (MCs) and mobility tunnel endpoint (MTE) pairs. While more than a single MC-MTE pair may be present in a sub-domain, only one may be active at any given time. The provision of multiple pairs in a sub-domain is for resilience and failure back up. In another form, a single MC is provided for the entire mobility domain. Said another way, for networks that consist of more than a single mobility sub-domain, the mobility oracle is provided. The mobility oracle is a centralized database that includes information on each of the stations in the mobility domain, the home mobility sub-domain for each station and the current foreign sub-domain providing service to each station. The mobility oracle is consulted by the individual MCs in order to facilitate inter mobility sub-domain mobility events.
FIG. 1 shows that in mobility sub-domain 20(1) there is a mobility controller 30(1) paired with an MTE 32(1) and a backup mobility controller 30(1)′ paired with a backup MTE 32(1)′. Similarly, in mobility sub-domain 20(2) there is a mobility controller 30(2) paired with an MTE 32(2) and a backup mobility controller 30(2)′ paired with a backup MTE 32(2)′. The MTE is a tunneling endpoint apparatus and its functions are described further hereinafter.
In the example architecture shown in FIG. 1, the functions of the MTEs in each sub-domain may be incorporated or integrated with other network equipment. For example, in sub-domain 20(1), the MTEs 32(1) and 32(1)′ may be incorporated into a distribution switch and further connected to distribution/core switches 33(1) and 33(1)′, respectively. The distribution/core switches 33(1) and 33(1)′ are in turn connected to a core network 40 that represents a Layer 3 or “core” portion of the network architecture 5. In mobility sub-domain 20(2), the MTEs 32(2) and 32(2)′ may be integrated into respective distribution/core switches that are in turn connected to the core network 40. In this case, there are distribution switches 37(1) and 37(1)′ connected to MTEs 32(2) and 32(2)′ in sub-domain 20(2).
A mobility controller provides the mobility control plane operations, facilitating mobility events that occur both within a mobility sub-domain, as well as across sub-domains. To this end, an entity called the mobility oracle 50 is provided. The mobility oracle 50 is a centralized control apparatus that comprises database that includes information on each of the client devices in the mobility domain, their home mobility sub-domain and the current foreign sub-domain providing service. The mobility oracle 50 is consulted by the individual mobility controllers in order to facilitate inter sub-domain mobility events. The mobility oracle 50 is shown coupled to the core network 40, but it may also be connected at the sub-domain level to any of the mobility sub-domains. As with the mobility sub-domain's mobility controller, more than one mobility oracle may be deployed for redundancy purposes, although only one would be active at any given time for the mobility domain.
Within each mobility sub-domain are access switches that provide the access layer connectivity to client devices operating in the mobility domain 10. For example, mobility sub-domain 20(1) has access switches 60(1)-60(4) and mobility sub-domain 20(2) has access switches 62(1)-62(4). Each access switch is capable of serving one or more IP subnets. An IP subnet comprises a plurality of IP addresses. An IP address for a client device can be assigned by a DHCP server, it can be statically configured on the client device or the client device can auto generate the IP address from a subnet served by its home access switch. It is possible that two or more access switches may serve the same IP subnet(s).
A client device obtains an IP address the first time it connects to the network, but it may also obtain another IP address thereafter. For example, an IPv6 client device can obtain a new IP address at any time, and thus can have multiple IP addresses. In general, the client device determines when to obtain an IP address.
Access switches within a mobility sub-domain may be grouped together in what is referred to herein as switch peer group. A switch peer group is statically configured by the MC, based on static information or information that is dynamically learned. Within a switch peer group, every switch has to have the same view of the membership of the group. A switch peer group does not span across mobility sub-domains or routing boundaries. A mobility sub-domain may have one or more switch peer groups. A purpose of the switch peer group is to allow access switches, between which client devices roam, to interact directly during handoff events. This permits a network to scale by eliminating the need for the MC to be involved in each and every handoff within a mobility sub-domain. Tunnels optionally directly connect the access switches in a switch peer group. If present, these tunnels allow data traffic to be directly sent from one access switch to another in the same switch peer group, without involving the MTE.
As explained hereinafter, client devices associate to an access switch, either by a wired network connection, or a wireless network connection (through a wireless access point device). FIG. 1 shows wireless access point (AP) devices at reference numerals 70(1)-70(n). The AP devices support the Control and Provisioning of Wireless Access Points (CAPWAP) protocol. As the CAPWAP architecture specifies, the APs perform the physical (PHY) layer and real-time IEEE 802.11 MAC functions, which includes IEEE 802.11 encryption. The AP establishes a tunnel to the access switch to tunnel client devices' wireless traffic. Thus, an AP is said to be “on” an access switch when it is configured to tunnel traffic for a wireless device to the access switch.
The APs encrypt all CAPWAP control traffic using the Datagram Transport Layer Security (DTLS) protocol. If the AP supports Cisco TrustSec (CTS) or IEEE 802.1AE (MacSec) encryption, then a link between the switch and the AP may be protected by Layer 2 CTS, in which case both CAPWAP control messages and CAPWAP traffic will get encrypted. If CTS is not supported, then the CAPWAP data traffic is unencrypted. In one possible form, CAPWAP data traffic can also be DTLS encrypted as an option.
Each MTE provides mobility services on the data plane, ensuring that a client device's point of presence on the Layer 3 network remains constant across mobility events. An MTE's involvement in a routing scenario for a client device is optional in that the functions of the MTE are only utilized when tunneling is employed.
FIG. 1 shows the MTE function as being located in either the distribution or the distribution/core switch. The location of the MTE is shown in this way purely for illustrative purposes as it could reside in any number of devices, integrated in switches/routers or in stand-alone appliances. The actual embodiment of the MTE may depend upon the switches, routers and appliances supporting a tunneling process described herein. The MTE can have two different roles depending on the availability of the subnets for the roamed client device at the MTE. If the subnet of the roamed client device is available at the MTE, the MTE could become the point of presence; otherwise the MTE functions as a tunnel switching entity that connects the roamed client device to the point of presence, which could be an access switch. The MTE may be integrated in a border router in each sub-domain. In this example, the MTEs may be integrated with border or edge routers of their respective sub-domain. The MTE functions serves as a tunneling endpoint apparatus with respect to traffic for devices by tunneling the traffic to access switches in its mobility sub-domain, and in cases where a device roams to a different mobility sub-domain, by way tunneling traffic to another MTE in the foreign mobility sub-domain to which the device roams. This is described further hereinafter in connection with FIGS. 6-9.
FIG. 1 shows the MCs and the MTEs as co-located entities. Again, the MC handles the mobility control logic, while the MTE provides the data plane operations. The MC and MTE functions may be encompassed in a single physical entity. When integrated in a single entity or unit, the MC configures its data plane, the MTE function, through a set of application programming interfaces (APIs) or commands. Thus, in the MC/MTE single unit integrated configuration, the MTE is the data path of the MC. However, when the MC and MTE functions are embodied in separate entities, some additional signaling for the commands is necessary between the MC and the MTE. This would involve the MC forwarding portions of the signaling it had received from an access switch to configure the forwarding tables stored at the MTE. The separation of these functions makes it possible to deploy a network that does not make use of tunneling. Such a network would still require the mobility control plane, provided by the MC, but would not require the functions of the MTE.
Reference is now made to FIG. 2 for a description of an example block diagram of an access switch. This diagram is meant to representative of a block diagram for any of the access switches 60(1)-60(4) and 62(1)-62(4) shown in FIG. 1, and in general for any access switch in any mobility sub-domain. The access switch comprises a processor 64, a switch and router unit 66 that may be in the form of an Application Specific Integrated Circuit (ASIC), a network interface unit 67, a system bus 68 and a memory 70. The switch and router unit 66 provides the packet forwarding (routing) and switching functions that are well known for a network access switch. The network interface unit 67 processes packets for transmission over the network and processes packets received from the network For example, the network interface unit 67 is an Ethernet card or similar device. The access switch is also referred to herein as a “forwarder” because it forwards packets to and from a client device. Instructions for access switch control logic 100 are stored in the memory 69 for execution by the processor 64. As explained above, an access switch is configured to serve IP addresses in one or more IP subnets.
The processor 64 may be a programmable processor or a fixed-logic processor. In the case of a programmable processor, the memory 69 is any type of tangible processor readable memory (e.g., random access, read-only, etc.) that is encoded with or stores instructions that, when executed by the processor 64, cause the processor to perform a variety of functions including the functions of the access switch control logic 100 described herein. Alternatively, the processor 64 may a fixed-logic processing device, such as an ASIC or digital signal processor or a network processor or a general purpose processor, that is configured with firmware comprised of instructions that cause the processor(s) 64 to perform the functions described herein. Thus, instructions for the logic 100 may take any of a variety of forms, so as to be encoded in one or more tangible media for execution, such as with fixed logic or programmable logic (e.g., software/computer instructions executed by a processor) and the processor(s) 64 may be a programmable processor, programmable digital logic (e.g., field programmable gate array) or an ASIC that comprises fixed digital logic, or a combination thereof.
Examples of functions of the access switch control logic 100 are described hereinafter in connection with FIGS. 6-9. These functions include “mobility agent” functions and datapath functions. The mobility agent functions are responsible for handling mobility events on the access switch, configuring the datapath elements on the switch for mobility and communicating with the MC. The datapath functions include terminating the CAPWAP tunnels which encapsulate IEEE 802.11 traffic sourced by wireless client devices, allowing the access switch to treat wired and wireless traffic in a uniform fashion.
More specifically, the functions of the mobility agent in the access switch are as follows. The mobility agent is responsible for responding in a timely manner to mobility control protocol messages sent by the various entities in the network, ensuring that a roaming budget time period is maintained for client devices. If the wireless subnets are not available at the MC/MTE, then the mobility agent assumes the role of the point of presence for roamed client devices that were originally associated with it. When the network is configured in a Layer 2 mode, the mobility agent is responsible for advertising reachability for the client devices connected to it. If tunneling is employed, an Address Resolution Protocol (ARP) request would be transmitted on behalf of the client device through the tunnel, which the point of presence (access switch) would bridge onto its uplink interface. The mobility agent is responsible for subscribing to multicast groups on behalf of a client device after a roaming event has occurred. This information is passed as part of the context to the new access switch to ensure that the multicast flows follow the client device as it roams. When the access switch is connected to a Layer 3 access network, the mobility agent is responsible for injecting routes for the client devices that are associated with it for which tunneling is not provided. The mobility agent performs an 802.1X authenticator function for both wired and wireless client devices. Finally, when a station successfully authenticates to the network, the mobility agent forwards the Pairwise Master Key (PMK) to the MC, and the MC is responsible for flooding the PMK to all of the access switches in the mobility sub-domain.
Turning now to FIG. 3, an example block diagram of an MC is now described. An MC is a control apparatus that may be embodied by a computing apparatus comprising a processor 34, a network interface unit 35 and a memory 36. Examples of specific embodiments of the processor 34 and the network interface unit 35 are described above in connection with FIG. 2. The memory 36 stores MC control process logic that, when executed by the processor 34, cause the processor 34 to perform the MC functions described herein. In addition, the memory 36 stores a stations database 205 and a switch database 210.
The stations database 205 maintains a database of all wireless client devices that are being served within the local sub-domain or the entire mobility domain (if the MC is configured to serve the entire mobility domain). This database may not store full client device context and may only include information indicating whether the client device currently considers the local sub-domain as its home, and is in many ways very similar to the function provided by the mobility oracle, although with a more limited scope, i.e., only for client devices local to the sub-domain. The database may include additional information such as the client device's credentials, which could be in the form of the user's identity, or a common name in a certificate, as well as the IP Address assigned to the device, if one has already been assigned to it by the network.
The switch database 210 maintains a database of all access switches within the mobility sub-domain, and updates all of the access switches, in real-time, as changes to the database occur (e.g., addition or removal of a switch from the network).
Other functions of the MC are summarized as follows. The MC is responsible for responding in a timely manner to mobility control protocol messages from other entities to ensure that the system achieves the desired roaming budget. The MC acts as a gateway between the access switches and the mobility oracle. When the MC does not find a match in its local database, it forwards the request to the mobility oracle, which is responsible for the entire mobility domain. However, there are deployment scenarios where the MC is responsible for the entire mobility domain. When tunneling is employed for a client device, its point of presence on the network could be the MTE if the wireless subnets are available at the MTE. Therefore, in these cases, the MC will respond to any ARP requests received for the client devices it is responsible for. When the MC is connected to a Layer 3 network, it is responsible for injecting routes into the network for the client devices it provides service for via a tunnel. The MC is the control point for the access switches for all mobility management related requests. When a change in a client device's point of attachment occurs, the MC is responsible for configuring the proper forwarding policy on the MTE, which may be co-located with the MC. If the MC and the MTEs are physically separate, the MC is responsible for initiating the signaling to the MTE to enforce changes in the client device's point of attachment in the network. The MC is capable of handling unsolicited commands from the Remote Authentication Dial-in User Service (RADIUS) infrastructure. These messages can be received by an access switch and forwarded to the MC to clear out or update the client key cache entries. It is also the responsibility of the MC to forward these messages to other MCs in mobility domain if a message is received from access switch. Finally, the MC optionally acts as an Network Time Server to the access switches to allow all access switches within a mobility sub-domain to have their clocks synchronized. The MC in turn synchronizes its clock off the mobility oracle.
Turning to FIG. 4, an example block diagram of an MTE is now described. The MTE is a routing apparatus that may also have processing functions. The MTE comprises a processor 42, a memory 44 and a network interface unit 46. The MTE may be integrated into a distribution switch or router and to this end FIG. 4 shows basic switching components including a switch and router unit 47 and a system bus 48. Instructions are stored in the memory 44 for MTE control logic 300. The processor 42 executes the instructions for the MTE control logic 300 to perform the various MTE functions described herein.
The MTE handles the mobility data plane. The role of the MTE is different depending on whether or not it is serving as the point of presence for client devices in the sub-domain. If the wireless subnets are not available at the MTE, then the point of presence for roamed client devices is at the home access switch. In this scenario, the MTE serves as a tunnel switching entity that connects the foreign access switch (point of attachment) to the home access switch (point of presence). If the wireless subnets are available at the MTE, then the MTE serves as the point of presence.
The functions of the MTE are generally as follows. The MTE terminates “mobility” tunnels from the access switches in its mobility sub-domain. Thus, there are pre-established tunnels between the MTE and each access switch in a given mobility sub-domain. Traffic to and from the roamed client device is sent to the foreign access switch via the mobility tunnel. An MTE-MTE tunnel is used to tunnel traffic between mobility sub-domains. The MTE has an interface the MC uses to configure the MTEs forwarding tables to reflect mobility events. When the MC and MTE are collocated, this is simply an API. If both functions are not collocated, this is a protocol.
As explained herein, the MC and MTE functions may be implemented by separate physical entities. In the case where they are implemented in a single entity, the MTE does not actually act as a router, and therefore does not inject routes into the network. The MC is responsible for advertising routes. However, the interfaces on which the routes are injected are considered part of the MTE. In the unlikely event that the MTE is decoupled from the MC, it is responsible for transmitting certain packets on behalf of the MC. For instance, the MC will provide Proxy ARP and routing services, yet these packets are transmitted on the MTEs interfaces. For networks that do not make use of tunneling, the MTE is not a necessary function.
Reference is now made to FIG. 5 for a description a block diagram of the mobility oracle 50. The mobility oracle 50 is a central controller for the mobility domain and contains a centralized database that includes information on each of the client devices in the network, their home mobility sub-domain and the current foreign sub-domain providing service. The mobility oracle 50 is a computing apparatus with network connectivity that oversees the entire mobility domain, but it does not necessarily perform any routing or packet forwarding functions. The mobility oracle 50 comprises a processor 52, a network interface unit 54 to provide network connectivity with the MCs and MTEs in the mobility domain, and a memory 56 that stores mobility oracle control logic 400 and a station database 405. The station database 405 maintains a database of all stations that are being provided service within the mobility domain. This station database 405 is populated through interactions the mobility oracle has with all of the MCs in all of the mobility sub-domains it supports. The station database includes each station's MAC address, its current home mobility sub-domain, and if roaming, its current foreign mobility sub-domain. When the mobility oracle 50 receives a request from an MC, it is responsible for performing the station lookup, and forwarding the request to the proper MC. The mobility oracle 50 acts as a Network Time Protocol (NTP) server to the MCs to allow all of the controllers within the mobility domain to have their clocks synchronized. The functions of the mobility oracle control logic 400 are described hereinafter in connection with FIGS. 8-11.
The following terms are defined for convenience in connection with the descriptions herein.
Foreign Mobility Controller: The MC providing mobility management service for the client device in a foreign mobility sub-domain. The foreign MC acts as a liaison between access switches in the foreign sub-domain and the MC in the home sub-domain.
Foreign Mobility Sub-Domain: The mobility sub-domain, controlled by a MC, supporting a client device which is anchored in (its IP address is part of an IP subnet of) another mobility sub-domain.
Foreign Switch: The access switch in the foreign mobility sub-domain currently providing service to the client device.
Home Mobility Controller: The MC providing the single point of control and mobility management service for client devices in their home mobility sub-domain.
Home Mobility Sub-Domain: The mobility sub-domain, controlled by a MC, for a client device where its IP address was assigned.
Home Access Switch: The access switch in the home mobility sub-domain that serves the IP subnet that includes the IP address of the client device and that last provided service to that client device. The Home Access Switch may not be the access switch where the wireless device initially joined the mobility domain.
Mobility Domain: A collection of mobility sub-domains across which mobility needs to be supported.
Mobility Sub-Domain: The mobility sub-domain is an autonomous component of the overall mobility domain network. A sub-domain generally connects into the core network, and includes one or more MC functions, and optionally their associated MTEs. A mobility sub-domain is the set of devices managed by the active Mobility Controller. A mobility sub-domain comprises of a set of access switches, and associated APs, across which fast roaming is desired. A mobility sub-domain is equivalent to an 802.11r key domain. The mobility sub-domain may also be referred to as an IP Everywhere (IPe) sub-domain. A mobility sub-domain and an mobility sub-domain are terms that are used interchangeably herein.
Point of Attachment: A client device's point of attachment is where the client is currently associated to the wireless network. This could either be the access switch that is currently providing service to the AP where the client device is associated, or the WLAN controller in the case of a legacy deployment. Thus, a wireless client device may roam from one AP on a first access switch to another AP on a second access switch and thereby become “attached” at or on the second access switch.
Point of Presence: A client device's point of presence is the place in the network where the client device is being advertised. For instance, if an access switch is advertising reachability to the client device via a routing protocol, the interface on which the route is being advertised is considered the client device's point of presence.
Station: A client device that connects to and requests service from the network. The device may have a wired, wireless or both interfaces. The term station may be used interchangeably with the term client device.
The MTE can have two different roles depending on the availability of the subnets for the roamed station at the MTE. If the subnet of the roamed station is available at the MTE, the MTE could become the point of presence; other it functions as a tunnel switching entity that connects the roamed station to the point of presence, which could be an access switch. In the following examples, the tunneling architecture is used and the wireless subnets are not available at the MTE.
When the wireless VLANs or IP subnets are not available at the MTE, then the point of presence for an Layer 3 roamed station is maintained on the original or anchor access switch—the access switch to which the station initially (for the first) joined (associated in) the network. In this case the MC/MTE becomes the tunnel switch point, where the MTE serves as a tunneling endpoint apparatus and switches the traffic from the tunnel from the foreign access switch to another tunnel to an anchor access switch and vice versa. This scenario can happen if the MC/MTE is not located at the distribution layer, but is deeper in the network, at the core. In fact, this is a very likely scenario during the transition from an overlay WLAN controller architecture to the architecture disclosed herein, since the WLAN controller which will serve as the controller for legacy switches and MC/MTE for the access switches is often at the core of the network.
In the scenarios described hereinafter in connection with FIGS. 6-9, a wireless station roams from one access switch to another access switch by associating with an AP on a different access switch. For simplicity, a wireless station is said to roam from access switch to access switch but it is understood that this involves the wireless station leaving the service of one AP and associating to an AP on a different access switch.
FIG. 6 illustrates a scenario where a wireless station roams within a mobility sub-domain from one access switch to another access switch, and across switch peer groups. In this scenario, the access switch to which the wireless station was previously attached is an access switch that serves the IP subnet that includes an IP address of the wireless station and the access switch to which the wireless station roams is an access switch that does not serve the IP subnet for the wireless station. Furthermore, the “previous” access switch as indicated in FIG. 6 need not be the same access switch where the wireless station initially connected to the network since multiple access switches can serve multiple (and common) IP subnets. FIG. 7 illustrates a flow diagram depicting control signal flows for the scenario of FIG. 6. FIG. 8 illustrates a scenario where a wireless station roams across mobility sub-domains and FIG. 9 illustrates the control signaling flow for the scenario of FIG. 8.
FIGS. 7 and 9 illustrate control signaling flows for two roaming scenarios. In FIGS. 7 and 9, reference numerals in the 100's refer to operations of an access switch, reference numerals in the 200's refer to operations of an MC, reference numerals in the 300's refer to operations of an MTE, and reference numerals in the 400's refer to operations of the mobility oracle.
Referring now to FIG. 6, a block diagram is shown of a portion of the mobility domain shown in FIG. 1, and in particular access switches within a single mobility sub-domain, e.g., sub-domain 20(1). In this example, access switches 60(1)-60(3) of sub-domain 20(1) are shown, where access switches 60(1) and 60(2) are part of the same switch peer group and access switch 60(3) is part of a different switch peer group. A wireless station 80 is shown that initially joins the network by associating to an AP 70(2) that is on access switch 60(2). Accordingly, the station 80 is given an IP address from a subnet available on, or served by, the access switch 60(2). Therefore, while the station 80 is associated to an AP on access switch 60(2), its point of presence and point of attachment are the same: access switch 60(2). As shown at the arrow 82, traffic for station 80 is natively bridged at the access switch 60(2) to the external network cloud shown at reference numeral 90. No tunneling is required for the station's traffic. For Layer 3 roamed stations, the point of presence remains at the last access switch with which the station was associated where the station's subnet was available (access switch 60(2) in this example) while its point of attachment moves where the station roams.
When station 80 roams to an access switch where its IP subnet is not available (not served by that different access switch), this triggers a handoff process. For example, the station 80 roams to and associates with an AP 70(3) on access switch 60(3), which in this example, is in a different switch peer group than access switches 60(1) and 60(2). Through exchanges of messages described hereinafter in connection with FIG. 7, the MTE 32(1) is configured to send traffic for the station 80 to and receive traffic from the access switch 60(3) in a tunnel 92(1) between the MTE 32(1) and the access switch 60(3), and to and from the access switch 60(2) in a tunnel 92(2). The access switch 60(2) is the point of presence for the station 80 because it is the last access switch to which station 80 was associated where its IP subnet is available. Thus, as shown by the dashed arrow at reference numeral 84, traffic for station 80 (after it has roamed) to and from the network cloud 90 passes through the access switch 60(2) even when station 80 is associated to a different access switch, e.g., access switch 60(3). The MTE 32(1) serves as a switching point for tunnels between the point of presence switch and the point of attachment switch. When station 80 sends outbound traffic to a wired host (in the network cloud 90), the traffic is tunneled in tunnel 92(1) from access switch 60(3) to the MTE 32(1) which forwards the traffic in tunnel 92(2) to the access switch 60(2), from which it is forwarded natively. Similarly, since the point of presence is at access switch 60(2), the inbound traffic from a wired host (in the network cloud 90) intended for the station 80 comes to access switch 60(2), is sent in tunnel 92(2) to MTE 32(1), and then in tunnel 92(1) to access switch 60(3) where the station 80 is currently attached.
Reference is now made to FIG. 7 for a description of the control message/signaling flow between the various entities to facilitate the intra-sub-domain roaming scenario shown in FIG. 6. Again, this control signal flow is for the situation when a station roams from one access switch in one switch peer group to another access switch in another switch peer group (or more generally between access switches in a deployment that does not use switch peer groups) in the same mobility sub-domain. One example is shown in FIG. 6 where the station roams from access switch 60(2), where it originally joined the network (and is also in this example the last access switch to which the station was associated where its IP subnet is available) to another access switch 60(3). When the access switch to which the station roams detects the association of the station (by way of an AP on that access switch) and after the station is authenticated, it sends a Mobile Announce message to the mobility controller for that mobility sub-domain. This is shown at reference numeral 110. The Mobile Announce message comprises information identifying the roamed station, e.g., MAC address of the roamed station of the roamed station and an identifier of the access switch to which the station has roamed. The mobility controller for that mobility sub-domain receives the Mobile Announce message and does a look up in its station database to determine the access switch where the station was last attached where its IP subnet is available, which may, but not in all cases, be the access switch where the station initially joined to the network. After making this determination, the mobility controller, 215, sends the Mobile Announce message to the access switch where the station was previously attached and also serves the IP subnet for the IP address of that station. The access switch that receives the Mobile Announce message obtains the identity of the original source of the Mobile Announce message (the access switch where the station has roamed) and the identity of the station that has roamed, and at 115 sends a Handoff message to the access switch to which the station has roamed. The Handoff message contains context information about the station, that is information identifying the station, including the IP address of the station and MAC address of the station and any policies associated with the station. At this time, as indicated at reference numeral 120, the access switch that serves the IP subnet for the station where the station was previously attached starts tunneling traffic to the MTE in the mobility sub-domain.
The access switch where the station is currently attached receives the Handoff message and generates a Handoff Complete message that it sends, at 125, to the mobility controller for the mobility sub-domain. The Handoff Complete message confirms that the station is connected at that access switch and also contains the identity of that access switch. In response to the mobility controller receiving and processing the Handoff Complete message, it configures the MTE for the mobility sub-domain to start tunneling traffic to the access switch where the station was previously attached and which serves the IP subnet for the station, as shown at 220. At 130, the access switch where the station is currently attached sends a Handoff Notification message to other access switches in its switch peer group to update the location of the station among the switches in its switch peer group (if a switch peer group implementation is deployed).
At 225, the mobility controller sends to the access switch where the station is currently attached an acknowledgment (ACK) message for the Handoff Complete message. In response to receiving the ACK message, the access switch where the station is currently located starts sending traffic over a tunnel between it and the MTE as shown at reference numeral 135. At 140, the access switch from which the station has roamed sends out a message to its switch peer group members to notify them that the station has left the switch peer group.
While the example shown in FIGS. 6 and 7 depict a scenario where the access switch from which the station has roamed is the access switch where the station initially joined the network, this is only an example. The station may have roamed to several other switches after leaving that original access switch before reaching the current access switch where it is attached. In this case, the mobility controller will determine, from its station database, which of the access switches has the IP subnet for the station (which in some cases will be the access switch where the station initially joined the network) and will cause that access switch to tunnel traffic to the MTE. The access switch where the station is currently located will also tunnel traffic to the MTE, and the MTE will bridge the two tunnels, and all of the intermediately access switches where the station had been associated are not involved in the tunneling of traffic for the station. Only the access switch that serves the IP subnet for the station and the access switch where the station is currently attached are involved in the tunneling.
The scenario depicted in FIGS. 6 and 7 may be summarized as follows. At a controller apparatus in a mobility sub-domain of a network (that may or may not comprise a plurality of mobility sub-domains), where the mobility sub-domain comprises a plurality of access switches each serving a one or more IP subnets comprising a plurality of IP addresses, determining that a wireless device that previously associated to a wireless access point on a first access switch has roamed to and connected with a wireless access point on a second access switch in the mobility sub-domain; at the controller apparatus, sending a message to the first access switch to notify it of attachment of the wireless device connected at the second access switch and to command the first access switch to send traffic for the wireless device in a tunnel to a tunneling endpoint apparatus in the mobility sub-domain controlled the controller apparatus (mobility controller); and at the controller apparatus, configuring the tunneling endpoint apparatus to send traffic for the wireless device to the first access switch in a tunnel between the tunneling endpoint apparatus and the first access switch. The mobility controller may determine that the wireless device has roamed to the second access switch based on reception from the second access switch of a mobile announce message indicating that the wireless device has associated to a wireless access point on the second access switch and by determining, based on stored data at the controller apparatus, that the wireless device was previously associated to a wireless access point on the first access switch that serves the IP subnet which includes the IP address of the wireless device. The mobility controller sends a message to the second access switch to cause the second access switch to send traffic from the wireless device in a tunnel to the tunneling endpoint apparatus such that outbound traffic from the wireless device passes through the tunneling endpoint apparatus between the second access switch and the tunneling endpoint apparatus and in the tunnel between the tunneling endpoint apparatus and the first access switch, and such that inbound traffic to the wireless device passes through the first access switch, to the tunneling endpoint apparatus in the tunnel between the first access switch and the tunneling endpoint apparatus and to the second access in the tunnel between the tunneling endpoint apparatus and the second access switch.
From the perspective of a mobility controller configured to support the scenario depicted in FIGS. 6 and 7, such a mobility controller may be summarized as comprising a network interface unit configured to enable communications over a network and a processor configured to be coupled to the network interface unit. The processor is configured to determine that a wireless device that was previously associated to a wireless access point on a first access switch in a mobility sub-domain comprising a plurality of access switches each serving one or more IP subnets that comprises a plurality of IP addresses, has roamed to and associated with a wireless access point on a second access switch in the mobility sub-domain; generate a message to be sent to the first access switch to notify the first access switch of attachment of the wireless device at the second access switch and to command the first access switch to send traffic for the device in a tunnel to a tunneling endpoint apparatus in the mobility sub-domain; and to configure the tunneling endpoint apparatus to send traffic for the wireless device to the first access switch in a tunnel between the tunneling endpoint apparatus and the first access switch.
The operations described above in connection with FIGS. 6 and 7 for a mobility controller may be also embodied as instructions stored in a processor readable medium that, when executed by a processor, cause the processor to perform those operations.
As explained herein, the MTE is a tunneling endpoint apparatus and it may be integrated in a single unit with the MC for a given mobility sub-domain and in each of the plurality of mobility sub-domains. In this case, then the operation 220 involves configuring a switching unit (e.g., switch and router unit 47 shown in FIG. 4) to direct traffic for the station in the tunnel to the first access switch and in the tunnel to the second access switch. On the other hand, when the MTE is separate from the MC, this operation may involve sending a command to the MTE from the MC to configure the MTE to direct traffic for the device in the tunnel to the first access switch and in the tunnel to the second access switch.
Reference is now made to FIG. 8. FIG. 8 shows a scenario where a station roams across mobility sub-domains to an access switch that is in a different mobility sub-domain from the mobility sub-domain of the access switch where it was previously attached. In this example, the station 80 is previously attached at access switch 60(2) in mobility sub-domain 20(1). Stations 80 roams to another mobility sub-domain, e.g., mobility sub-domain 20(2) where it associates with an AP 70(6) on access switch 62(1). (There are other access switches in mobility sub-domain 20(1) but for simplicity they are not shown in FIG. 8). FIG. 8 shows the native (home) routing of traffic for station 80 at reference numeral 82. In this roaming example, the traffic for the roamed station 80 is shown by dashed arrow 86. The home MTE, MTE 32(1), tunnels traffic for the station 80 to the previous access switch 60(2) in tunnel 92(2). Traffic is sent in a tunnel 92(3) between MTEs 32(1) and 32(2), and in a tunnel 92(4) between the MTE 32(2) in the mobility sub-domain 20(2) to the access switch 62(1) where the station 80 is currently attached. Only the tunnel between the access switches and the MTE in the foreign mobility sub-domain is affected by mobility events in foreign mobility sub-domains. The previous access switch, e.g., access switch 60(2), for the station that serves the IP subnet for the station 80 in its home mobility sub-domain continues to be responsible for advertising reachability of the station 80 and remains the point of presence for the roamed station 80.
Reference is now made to FIG. 9 for an example of the control signaling flow between entities to support the inter mobility sub-domain roaming scenario such as that shown in FIG. 8. When a station roams across mobility sub-domains, at 145 the access switch to which the station roams in, e.g., mobility sub-domain 2, generates a Mobile Announce message that it sends to the mobility controller for that sub-domain. This is particularly the case when it is the first time that the station appears in this mobility sub-domain. The access switch to which the station 80 attaches does not know the station 80 in this case. The mobility controller for the foreign mobility sub-domain, mobility sub-domain 2, also may not know the station 80 if it is the first time that the station is moving to this mobility sub-domain. Therefore, at 230, the mobility controller forwards the Mobile Announce message to the mobility oracle.
The mobility oracle determines, from its stations database, the mobility controller for the home mobility sub-domain for station 80 based on information contained in the Mobile Announce message that identifies the station 80. Using this information, at 420 the mobility oracle forwards the Mobile Announce message to the mobility controller for the mobility sub-domain 1. The mobility controller for the home mobility sub-domain of the station, mobility sub-domain 1, upon receiving the Mobile Announce message, at 235 forwards it to the access switch where the station was previously attached in mobility sub-domain 1 and which access switch serves the IP subnet for the station 80. This access switch, in response to receiving the Mobile Announce message, learns that the station 80 has roamed to a particular access switch in mobility sub-domain 2 from the content of the Mobile Announce message, and in response at 150 sends a Handoff message to the access switch in mobility sub-domain 2 where the station is currently attached. Also, in response to receiving the Mobile Announce message, the access switch in mobility sub-domain 1 starts sending traffic for the station 80 to the MTE in mobility sub-domain 1 as indicated at reference numeral 155. At 160, the access switch in mobility sub-domain 1 sends a message to other switches in its switch peer group (assuming a switch peer group implementation is used) to notify those access switches that the station has left the switch peer group.
At 165, in response to receiving the Handoff message, the access switch in mobility sub-domain 2 where the station is currently attached sends a Handoff Complete message to the mobility controller in mobility sub-domain 1. The access switch in mobility sub-domain 2 also starts tunneling traffic for the station to the MTE in mobility sub-domain 2. In response to receiving the Handoff Complete message from the access switch where the station is currently attached, the mobility controller in mobility sub-domain 2 updates its stations database accordingly and at 240, configures the MTE in mobility sub-domain 2 to start tunneling traffic for the station to the MTE in mobility sub-domain 1. Also, at 245 the mobility controller in mobility sub-domain 2 sends the Handoff Complete message to the mobility oracle.
At 170, the access switch where the station is currently attached sends a Handoff Notification message to other switches in its switch peer group in mobility sub-domain 2 (assuming that a switch peer group implementation is used). Finally, at 250, the mobility controller in mobility sub-domain 2 sends a Handoff Complete message to the mobility controller in mobility sub-domain 1. At 255, the mobility controller in mobility sub-domain 1 updates its database and configures the MTE in mobility sub-domain 1 to start to tunnel traffic for the station to the access switch in mobility sub-domain 1.
As depicted in FIGS. 8 and 9, the tunneling will go between the MTEs of the two mobility sub-domains and back to the point of presence access switch in the home mobility sub-domain. Likewise, the station may further roam to other access switches in the foreign mobility sub-domain and the MC in that foreign mobility sub-domain will configure the MTE in that foreign mobility sub-domain to point the tunneling to the current point of attachment access switch. If the station roams to yet another foreign mobility sub-domain, the intervening mobility sub-domain where the station was previously located, is not involved and traffic is tunneled between the MTE of the home mobility sub-domain and the most current foreign mobility sub-domain.
Thus, the scenario depicted in FIGS. 8 and 9 may be summarized as follows. At a controller apparatus in a second mobility sub-domain of a network comprising a plurality of mobility sub-domains, each mobility sub-domain comprising one or more access switches that serve one or more IP subnets that comprise a plurality of IP address, receiving from an access switch in the second mobility sub-domain a mobile announce message indicating that a wireless device has roamed to and associated with a wireless access point on the access switch in the second mobility sub-domain; forwarding the mobile announce message to a central controller apparatus for the plurality of mobility sub-domains, which central controller apparatus determines that the wireless device roamed from a first mobility sub-domain where it was previously associated to a wireless access point on an access switch in the first mobility sub-domain, and which access switch in the first mobility sub-domain serves the IP subnet which includes the IP address of the wireless device; and forwarding the message from the central controller apparatus to a controller apparatus in the first mobility sub-domain; at the controller apparatus in the first mobility sub-domain, forwarding the mobile announce message to the access switch in the first mobility sub-domain that serves the IP subnet which includes the IP address of the wireless device; sending traffic for the wireless device in a tunnel from the access switch in the first mobility sub-domain to a tunneling endpoint apparatus in the first mobility sub-domain; and sending traffic for the wireless device in a tunnel between the tunneling endpoint apparatus in the first mobility sub-domain and a tunneling endpoint apparatus in the second mobility sub-domain, and between the tunneling endpoint apparatus in the second mobility sub-domain and the access switch wherein the wireless device is currently attached in the second mobility sub-domain.
When the controller apparatus and tunneling endpoint apparatus are integrated into a single unit and which further comprises a switching unit (e.g., switch and router 47 shown in FIG. 4), and the configuring of the tunneling endpoint apparatus may comprise supplying a command to a switching unit in the respective single units (combined MC/MTE) to configure the switching unit to select traffic for the device in the respective tunnels and to direct outbound traffic from the wireless device from the tunneling endpoint apparatus in the second mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain and in the tunnel from the tunneling endpoint apparatus in the first mobility sub-domain to the access switch in the first mobility sub-domain and to direct inbound traffic to the wireless device from the access switch in the first mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain, from the tunneling endpoint apparatus in the first mobility sub-domain to the tunneling endpoint apparatus in the second mobility sub-domain and from the tunneling endpoint apparatus in the second mobility sub-domain to the access switch in the second mobility sub-domain.
From the perspective of an MC configured to support the scenario depicted in FIGS. 8 and 9, an apparatus is provided comprising a network interface unit configured to enable communications over a network and a processor configured to be coupled to the network interface unit. The processor is configured to receive a message indicating that a wireless device that previously associated to a wireless access point on an access switch in a first mobility sub-domain of a mobility domain comprising a plurality of mobility sub-domains, each mobility sub-domain comprises one or more access switches that serve one or more IP subnets comprising a plurality of IP addresses, has roamed and associated to a wireless access point on an access switch in a second mobility sub-domain; configure a tunneling endpoint apparatus in the second mobility sub-domain to start sending traffic for the wireless device in a tunnel to a tunneling endpoint apparatus in the first mobility sub-domain; and generate a handoff complete message to be sent to a controller apparatus in the first mobility sub-domain to cause the controller apparatus in the first mobility sub-domain to configure a tunneling endpoint apparatus in the first mobility sub-domain to start sending traffic for the wireless device in a tunnel to the access switch in the first mobility sub-domain.
In the case where the MC and MTE is integrated into a single unit, then the processor configures the MTE by configuring a switching unit contained therein to direct traffic for the wireless device in the tunnel to the access switch in the second mobility sub-domain and in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain such that outbound traffic from the wireless device travels from the tunneling endpoint apparatus in the second mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain and in the tunnel from the tunneling endpoint apparatus in the first mobility sub-domain to the access switch in the first mobility sub-domain and inbound traffic to the wireless device travels from the access switch in the first mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain, from the tunneling endpoint apparatus in the first mobility sub-domain to the tunneling endpoint apparatus in the second mobility sub-domain and from the tunneling endpoint apparatus in the second mobility sub-domain to the access switch in the second mobility sub-domain.
The operations for the MC summarized above for the scenario of FIGS. 8 and 9 may be embodied as instructions stored in a processor readable medium that, when executed by a processor, cause the processor to perform those operations.
The techniques described herein have several advantages. First, both wired and wireless clients can be present on the same IP subnet, allowing for uniform handling of wired and wireless clients, uniform policy application and uniform management of the clients. Second, this hierarchical architecture allows the network to scale much better and support more APs and clients, compared to current architectures. This is achieved primarily by controlling the cost of handoffs on the control path, and by distributing the data path to the access switches. Third, by configuring the point of presence for a station to be on access switch that serves the IP subnet for that station (the access switch that the station was mostly recently attached to that served its IP subnet), a flexible deployment of the MCs/MTEs may be made. Customers who have their current WLAN controllers in the core (past the first routing boundary) can easily migrate to this architecture, while existing WLAN controllers can continue to operate in the network as MCs/MTEs.
The above description is by way of example only.

## Claims

1. A method comprising:
providing a plurality of mobility sub-domains for a network, each mobility sub-domain comprising one or more access switches that serve one or more Internet Protocol (IP) subnets each of which comprises a plurality of IP addresses;
providing a controller apparatus in a first mobility sub-domain, a controller apparatus in a second mobility sub-domain and a central controller apparatus for the plurality of mobility sub-domains;
receiving at the central controller apparatus from the controller apparatus in the second mobility sub-domain a mobile announce message indicating that a wireless device has roamed to and associated with a wireless access point on an access switch in the second mobility sub-domain;
determining at the central controller apparatus that the wireless device roamed from a first mobility sub-domain where it was previously associated to a wireless access point on an access switch in the first mobility sub-domain, and which access switch in the first mobility sub-domain serves the IP subnet that includes an IP address of the wireless device;
forwarding the mobile announce message from the central controller apparatus to the controller apparatus in the first mobility sub-domain;
forwarding the mobile announce message from the controller apparatus in the first mobility sub-domain to the access switch in the first mobility sub-domain that serves the IP subnet that includes an IP address of the wireless device;
sending traffic for the wireless device in a tunnel from the access switch in the first mobility sub-domain to a tunneling endpoint apparatus in the first mobility sub-domain; and
sending traffic for the wireless device in a tunnel between the tunneling endpoint apparatus in the first mobility sub-domain and a tunneling endpoint apparatus in the second mobility sub-domain, and between the tunneling endpoint apparatus in the second mobility sub-domain and the access switch where the wireless device is currently attached in the second mobility sub-domain.

2. The method of claim 1, wherein determining that the wireless device has roamed isbased on data stored at the central controller apparatus.

3. The method of claim 1, wherein the controller apparatus and the tunneling endpoint apparatus in each of the respective first mobility sub-domain and second mobility sub-domain are integrated in a single unit, and further comprising configuring a switching unit in the single unit to direct traffic for the wireless device in the tunnel in the respective mobility sub-domains.

4. The method of claim 1, wherein the tunneling endpoint apparatus is separate from the controller apparatus in each of the respective first and second mobility sub-domains, and further comprising sending a command to the tunneling endpoint apparatus from the controller apparatus in each of the respective first and second mobility sub-domains to configure the tunneling endpoint apparatus to direct traffic for the device in the tunnel.

5. A method comprising:
at a controller apparatus in a second mobility sub-domain of a network comprising a plurality of mobility sub-domains, each mobility sub-domain comprising one or more access switches that serve one or more Internet Protocol (IP) subnets that comprise a plurality of IP addresses, receiving from an access switch in the second mobility sub-domain a mobile announce message indicating that a wireless device has roamed to and associated with a wireless access point on an access switch in the second mobility sub-domain;
forwarding the mobile announce message to a central controller apparatus for the plurality of mobility sub-domains, which central controller apparatus determines that the wireless device roamed from a first mobility sub-domain where it was previously associated to a wireless access point on an access switch in the first mobility sub-domain, and which access switch in the first mobility sub-domain serves the IP subnet which includes an IP address of the wireless device;
forwarding the mobile announce message from the central controller apparatus to a controller apparatus in the first mobility sub-domain;
at the controller apparatus in the first mobility sub-domain, forwarding the mobile announce message to the access switch in the first mobility sub-domain that serves the IP subnet which includes an IP address of the wireless device;
sending traffic for the wireless device in a tunnel from the access switch in the first mobility sub-domain to a tunneling endpoint apparatus in the first mobility sub-domain; and
sending traffic for the wireless device in a tunnel between the tunneling endpoint apparatus in the first mobility sub-domain and a tunneling endpoint apparatus in the second mobility sub-domain, and between the tunneling endpoint apparatus in the second mobility sub-domain and the access switch where the wireless device is currently attached in the second mobility sub-domain.

6. The method of claim 5, and further comprising sending a handoff message from the access switch in the first mobility sub-domain that serves the IP subnet which includes the IP address of the wireless device to the access switch in the second mobility sub-domain where the wireless device is currently attached; and sending a handoff complete message from the access switch in the second mobility sub-domain to the controller apparatus in the second mobility sub-domain.

7. The method of claim 6, and further comprising at the controller apparatus in the second mobility sub-domain, in response to receiving the handoff complete message from the access switch in the second mobility sub-domain, updating a database of information indicating mobility sub-domain locations of devices and configuring the tunneling endpoint apparatus in the second mobility sub-domain to start sending traffic in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain.

8. The method of claim 7, and further comprising sending the handoff complete message from the controller apparatus in the second mobility sub-domain to the controller apparatus in the first mobility sub-domain, and at the controller apparatus in the first mobility sub-domain, in response to receiving the handoff complete message from the controller apparatus in the second mobility sub-domain, configuring the tunneling endpoint apparatus in the first mobility sub-domain to start sending traffic for the wireless device in the tunnel to the access switch in the first mobility sub-domain.

9. The method of claim 8, wherein the controller apparatus and the tunneling endpoint apparatus for the first mobility sub-domain are integrated into a single unit and the controller apparatus and the tunneling endpoint apparatus for the second mobility sub-domain are integrated into a single unit, and wherein configuring the respective tunneling endpoint apparatus in the first and second mobility sub-domains comprises supplying a command to a switching unit in the respective single units to configure the switching unit to select traffic for the wireless device in the respective tunnels and to direct outbound traffic from the wireless device from the tunneling endpoint apparatus in the second mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain and in the tunnel from the tunneling endpoint apparatus in the first mobility sub-domain to the access switch in the first mobility sub-domain and to direct inbound traffic to the wireless device from the access switch in the first mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain, from the tunneling endpoint apparatus in the first mobility sub-domain to the tunneling endpoint apparatus in the second mobility sub-domain and from the tunneling endpoint apparatus in the second mobility sub-domain to the access switch where the wireless device is currently attached in the second mobility sub-domain.

10. A system comprising:
a tunneling endpoint apparatus in a first mobility sub-domain of a mobility domain comprising a plurality of mobility sub-domains, each mobility sub-domain comprising one or more access switches that serve one or more Internet Protocol (IP) subnets each comprising a plurality of IP addresses;
a tunneling endpoint apparatus in a second mobility sub-domain of the plurality of mobility sub-domains;
controller in the second mobility sub-domain configured to:
receive a message indicating that a wireless device that previously associated to a wireless access point on an access switch in the first mobility sub-domain has roamed and associated to a wireless access point on an access switch in the second mobility sub-domain;
configure the tunneling endpoint apparatus in the second mobility sub-domain to start sending traffic for the wireless device in a tunnel to the tunneling endpoint apparatus in the first mobility sub-domain; and
generate a handoff complete message to be sent to a controller in the first mobility sub-domain to cause the controller in the first mobility sub-domain to configure the tunneling endpoint apparatus in the first mobility sub-domain to start sending traffic for the wireless device in a tunnel to the access switch in the first mobility sub-domain;
wherein the controller in the second mobility sub-domain configures the tunneling endpoint apparatus in the second mobility sub-domain by configuring a switching unit in the tunneling endpoint apparatus to direct traffic for the wireless device in the tunnel to the access switch in the second mobility sub-domain and in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain such that outbound traffic from the wireless device travels from the tunneling endpoint apparatus in the second mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain and in the tunnel from the tunneling endpoint apparatus in the first mobility sub-domain to the access switch in the first mobility sub-domain and inbound traffic to the wireless device travels from the access switch in the first mobility sub-domain in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain, from the tunneling endpoint apparatus in the first mobility sub-domain to the tunneling endpoint apparatus in the second mobility sub-domain and from the tunneling endpoint apparatus in the second mobility sub-domain to the access switch in the second mobility sub-domain.

11. The system of claim 10, wherein the controller is further configured to receive from the access switch in the second mobility sub-domain a mobile announce message indicating the wireless device has roamed to and associated with a wireless access point on the access switch in the second mobility domain.

12. The system of claim 11, and further comprising a central controller for the plurality of mobility sub-domains, wherein the controller in the second mobility sub-domain is further configured to forward the mobile announce message to the central controller, and the central controller apparatus is configured to determine that the wireless device roamed from the first mobility sub-domain where it was previously associated to a wireless access point on an access switch in the first mobility sub-domain, and which access switch in the first mobility sub-domain serves the IP subnet which includes an IP address of the wireless device.

13. The system of claim 12, wherein the central controller is configured to forward the mobile announce message to the controller in the first mobility sub-domain, and the controller in the first mobility sub-domain is configured to forward the mobile announce message to the access switch in the first mobility sub-domain that serves the IP subnet which includes the IP address of the wireless device.

14. The system of claim 10, wherein the access switch in the first mobility sub-domain that serves the IP subnet which includes the IP address of the wireless device is configured to send a handoff message to the access switch in the second mobility sub-domain where the wireless device is currently attached, and the access switch in the second mobility sub-domain is configured to send a handoff complete message to the controller in the second mobility sub-domain.

15. The system of claim 14, wherein the controller in the second mobility sub-domain is configured, in response to receiving the handoff complete message from the access switch in the second mobility sub-domain, to update a database of information indicating mobility sub-domain locations of devices and to configure the tunneling endpoint apparatus in the second mobility sub-domain to start sending traffic in the tunnel to the tunneling endpoint apparatus in the first mobility sub-domain.

16. The system of claim 15, wherein the controller in the second mobility sub-domain is configured to send the handoff complete message to the controller in the first mobility sub-domain, and the controller in the first mobility sub-domain is configured to, in response to receiving the handoff complete message, configure the tunneling endpoint apparatus in the first mobility sub-domain to start sending traffic for the wireless device in the tunnel to the access switch in the first mobility sub-domain.