Efficient convergence in network events

One embodiment of the present invention provides a switch. The switch includes a storage device, one or more line cards, and a control card. A respective line card includes one or more ports and forwarding hardware. The control card determines routing and forwarding tables for the switch, and comprises processing circuitry and a management module. The management module manage a tunnel or virtual network at the switch. During operation, the control card determine an event associated with layer-2 operations of the switch. The control card refrains from notifying the management module regarding the event and notifies a first line card in the one or more line cards regarding the event. The first line card then updates a layer-3 forwarding entry in the corresponding forwarding hardware based on the notification.

BACKGROUND

Field

The present disclosure relates to communication networks. More specifically, the present disclosure relates to a system and a method for facilitating efficient convergence to a switch in a network event.

Related Art

The exponential growth of the internet has made it a popular delivery medium for a variety of applications running on physical and virtual devices. Such applications have brought with them an increasing demand for bandwidth. As a result, equipment vendors race to build larger and faster switches with versatile capabilities, such as efficient management of control messages of a switch. However, the capabilities of a switch cannot grow infinitely. Those capabilities are limited by physical space, power consumption, and design complexity, to name a few factors. Furthermore, switches with higher capabilities are usually more complex and expensive. As a result, increasing efficiency in existing capabilities of a switch adds significant value proposition.

To meet with increasing traffic demand, a switch can support overlay tunnels. In an overlay network that facilitates a virtualized layer-2 network, a layer-2 network is extended across a layer-3 network using the overlay tunnels. Typically, when the switch forwards a packet via the tunnel, an outer layer-2 header is added to the tunnel encapsulation header. The switch then forwards the encapsulated packet to the next-hop switch based on the outer layer-2 header. However, a layer-2 event (e.g., a media access control (MAC) address movement or a failover) can trigger a change in how the switch forwards the tunnel-encapsulated packet to the next hop. For example, the layer-2 event can affect the next-hop entry for the tunnel. For a large number of tunnels, such a layer-2 event can trigger a large number of changes.

While the overlay tunnels bring many desirable features in a switch, some issues remain unsolved in facilitating efficient management of the changes associated with a layer-2 event.

SUMMARY

One embodiment of the present invention provides a switch. The switch includes a storage device, one or more line cards, and a control card. A respective line card includes one or more ports and forwarding hardware. The control card determines routing and forwarding tables for the switch, and comprises processing circuitry and a management module. The management module manage a tunnel or virtual network at the switch. During operation, the control card determine an event associated with layer-2 operations of the switch. The control card refrains from notifying the management module regarding the event and notifies a first line card in the one or more line cards regarding the event. The first line card then updates a layer-3 forwarding entry in the corresponding forwarding hardware based on the notification.

In a variation on this embodiment, the control card determines the event by determining an unavailability of a first port of the switch, obtaining an Address Resolution Protocol (ARP) response via a second port of the switch, and associating the second port with a mapping between an Internet Protocol (IP) address and a media access control (MAC) address obtained from the ARP response.

In a variation on this embodiment, the layer-3 forwarding entry is associated with a layer-3 next-hop switch for a remote tunnel endpoint of a tunnel affected by the event.

In a further variation, the layer-3 forwarding entry corresponds to an outer layer-2 header associated with the layer-3 next-hop switch.

In a variation on this embodiment, the control card also includes a multi-destination module, which registers to receive information of the event and updates a second layer-3 forwarding entry for multi-destination traffic in the forwarding hardware of the first line card.

In a further variation, the multi-destination module maintains a list of tunnels and virtual networks of the switch. The list indicates a mapping between a tunnel or virtual network to a corresponding line card.

In a further variation, the multi-destination module identifies a tunnel or virtual network, which is associated with the first line card and affected by the event, from the list. Here, the second layer-3 forwarding entry corresponds to the identified tunnel or virtual network.

One embodiment of the present invention provides a switch. The switch includes a storage device, one or more line cards, and a control card. A respective line card includes one or more ports and forwarding hardware. The control card determines routing and forwarding tables for the switch, and comprises processing circuitry, a management module, and a multi-destination module. The management module manages one or more tunnels or virtual networks at the switch. The multi-destination module configures layer-3 forwarding entries for multi-destination traffic in the forwarding hardware of the one or more line cards. During operation, the management module determines an event associated with layer-3 operations of the switch and sends a probe message to the multi-destination module to determine whether the multi-destination module is available. If the management module receives a probe response message from the multi-destination module, the management module sends, to the multi-destination module, a notification message comprising information associated with a first tunnel or virtual network affected by the event.

In a variation on this embodiment, if the management module does not receive a probe response message within a period of time, the management module determines a state for the first tunnel or virtual network. The state indicates that an update in a forwarding entry is needed for the first tunnel or virtual network.

In a further variation, if the management module receives the probe response message, the management module identifies the state for the first tunnel or virtual network and inserts information associated with the first tunnel or virtual network into a buffer.

In a further variation, if the buffer reaches a threshold, the management module includes the information of the buffer in the notification message.

In a variation on this embodiment, the management module maintains a list of the one or more tunnels or virtual networks managed by the management module. The list includes an entry for the first tunnel or virtual network.

In a variation on this embodiment, if the management module detects an unsuccessful transmission of the notification message, the management module sends a second probe message to the multi-destination module to determine whether the multi-destination module is available.

In a variation on this embodiment, whether the multi-destination module is available is determined based on a global state maintained for the multi-destination module. The global state is accessible by the management module and a second module of the control card.

DETAILED DESCRIPTION

Overview

In embodiments of the present invention, the problem of efficiently managing control messages associated with a respective overlay tunnel instance in a layer-2 event is solved by (i) propagating a control instruction between respective layer-3 modules of a control card and a line card; (ii) propagating a control instruction between a layer-3 module and a multi-destination traffic module of a control card; and (iii) facilitating intelligent buffering and adaptive synchronization with a multi-destination traffic module.

A control card of a switch is a switch card, which can also be part of a back plane, of a switch that controls the switch. The control card implements routing protocols and establishes routes. A layer-3 module of the control card is responsible for managing forwarding information associated with a respective layer-3 forwarding entry. On the other hand, a layer-3 module of a line card of the switch is responsible for incorporating the managing forwarding information associated with a respective layer-3 forwarding entry into the forwarding hardware (e.g., in ternary content-addressable memory (TCAM)) of the line card. The multi-destination traffic module of the control card is responsible for managing the forwarding information associated with multi-destination traffic for overlay tunnels in a respective line card.

Typically, a tunnel instance is established between two tunnel endpoints. A tunnel endpoint can be a switch (or any computing device) capable of originating or terminating a tunnel encapsulation header. To forward a packet via the tunnel, the tunnel endpoint encapsulates the packet with an encapsulation header associated with a corresponding tunneling protocol (e.g., a layer-3 encapsulation header over an inner layer-2 header). The source and destination addresses in the encapsulation header correspond to the tunnel endpoints of the tunnel. In addition, to forward the encapsulated packet to the next-hop corresponding to the destination address, an outer layer-2 header is added to the encapsulation header. This outer layer-2 header is determined based on the forwarding information associated with the tunnel instance.

Examples of a tunneling protocol include, but are not limited to, virtual extensible LAN (VXLAN), generic routing encapsulation (GRE), network virtualization using GRE (NVGRE), layer-2 tunneling protocol (L2TP), and multi-protocol label switching (MPLS). Different virtual local area networks (VLANs) are mapped to different corresponding virtual network identifiers for a tunnel. A tunnel endpoint can include the virtual network identifier in the encapsulation header associated with the tunnel. For example, if the tunneling protocol is VXLAN, the tunnel endpoint can be a virtual tunnel endpoint (VTEP), which maps a VXLAN network identifier (VNI) to a corresponding VLAN. In some embodiments, the tunnel endpoint is in a distributed tunnel endpoint, which includes a plurality of tunnel endpoints operating based on virtual router redundancy protocol (VRRP).

A switch can support a large number of tunnel instances (e.g., a large number of VXLAN tunnel instances). As a result, a layer-2 event can affect the layer-2 forwarding information associated with these tunnel instances, leading to a large number of changes to the forwarding information. Furthermore, each tunnel can support a large number of virtual networks (e.g., Virtual Private LAN Service (VPLS)). A tunnel instance or a virtual network instance in a switch can be referred to as a communication instance of the switch. With existing technologies, a layer-2 module of the control card of the switch detects the layer-2 event and notifies the layer-3 module of the control card. The layer-3 module then issues an Address Resolution Protocol (ARP) request for the Internet Protocol (IP) address of the next-hop layer-3 switch and obtains the corresponding media access control (MAC) address. The layer-3 module determines the port from which the layer-2 module has learned the MAC address and notifies a respective tunnel and/or virtual network instances regarding the information associated with the layer-2 event.

Each instance of a tunnel and/or a virtual network, in turn, can generate a control message associated with the changes in the corresponding layer-2 forwarding information (e.g., a new forwarding port for the next-hop layer-3 switch). The instance then sends the control message to the corresponding instance of a line card for updating the forwarding hardware of the line card. As a result, a large number of control messages are generated and propagated within the switch. This can cause a longer convergence time for both unicast and multi-destination traffic in the switch. Multi-destination traffic typically includes broadcast, unknown unicast, and multicast (BUM) traffic.

To solve this problem, embodiments of the present invention facilitate efficient management of the control messages. Instead of notifying each instance of a tunnel and/or a virtual network, the layer-3 module of the control card of the switch generates a control message for the layer-3 module of a respective line card. The layer-3 module of the line card then notifies a respective tunnel and/or virtual network instances to update the hardware of the line card with the updated forwarding information. In addition, instead of each instance of a tunnel and/or a virtual network notifying the multi-destination traffic module to update the corresponding forwarding entry in the hardware of a respective line card, the multi-destination traffic module receives a notification message from the layer-3 module, and updates the forwarding entries for a respective tunnel/virtual network instance for multi-destination traffic. In this way, the control card reduces the notifications across the control card as well as the control messages within the switch.

One or more tunnel/virtual network instances may send a control instruction to the multi-destination traffic module of the control card of a switch. For example, the layer-3 module of a switch may detect an event associated with one or more tunnels. Examples of such an event include, but are not limited to, a configuration event, an operational event, a tunnel activation/deactivation event, a layer-3 next-hop change, an IP address change, and a tunnel key change. As a result, the tunnels and the virtual networks operating on the tunnels may require reconfiguration. The corresponding tunnel/virtual network instances may send control messages to the multi-destination traffic module. With existing technologies, the multi-destination traffic module can be in an occupied state executing some of these instructions when the later instructions arrive. As a result, the multi-destination traffic module may not be able to receive the later instructions (e.g., due to an unsuccessful communication). This leads to repeated resending of these instructions, which causes additional processing in the switch and delay in convergence.

To solve this problem, embodiments of the present invention allow a corresponding tunnel/virtual network module (e.g., the VXLAN module or the MPLS module) to maintain a local state for a respective tunnel/virtual network. The state can indicate whether the event has affected the tunnel/virtual network. The modules also maintain a global state indicating whether the multi-destination traffic module is available (e.g., whether the multi-destination traffic module can spare processing capacity). A tunnel module, which manages one or more tunnels affected by the event, periodically sends a probe message to determine the availability of the multi-destination traffic module. If the multi-destination traffic module sends a response message, the tunnel module checks the local states of a respective tunnel and generates a control instruction for a respective affected tunnel based on the local states. The tunnel module then stores these control instructions in a buffer. If the buffer reaches a size (e.g., reaches a preconfigured length), or a timer of the buffer expires, the tunnel module includes the control instructions in a control message and sends the control message to the multi-destination traffic module. In this way, the tunnel module sends control instructions when the multi-destination traffic module is available.

In some embodiments, the switch can be a member switch of a network of interconnected switches (e.g., a fabric switch). In a fabric switch, any number of switches coupled in an arbitrary topology can be controlled as a single logical switch. The fabric switch can be an Ethernet fabric switch or a virtual cluster switch (VCS), which can operate as a single Ethernet switch. In some embodiments, a respective switch in the fabric switch is an Internet Protocol (IP) routing-capable switch (e.g., an IP router). In some further embodiments, a respective switch in the fabric switch is a Transparent Interconnection of Lots of Links (TRILL) routing bridge (RBridge).

It should be noted that a fabric switch is not the same as conventional switch stacking. In switch stacking, multiple switches are interconnected at a common location (often within the same rack), based on a particular topology, and manually configured in a particular way. These stacked switches typically share a common address, such as an IP address, so they can be addressed as a single switch externally. Furthermore, switch stacking requires a significant amount of manual configuration of the ports and inter-switch links. The need for manual configuration prohibits switch stacking from being a viable option in building a large-scale switching system. The topology restriction imposed by switch stacking also limits the number of switches that can be stacked. This is because it is very difficult, if not impossible, to design a stack topology that allows the overall switch bandwidth to scale adequately with the number of switch units.

In contrast, a fabric switch can include an arbitrary number of switches with individual addresses, can be based on an arbitrary physical topology, and does not require extensive manual configuration. The switches can reside in the same location, or be distributed over different locations. These features overcome the inherent limitations of switch stacking and make it possible to build a large “switch farm,” which can be treated as a single, logical switch. Due to the automatic configuration capabilities of the fabric switch, an individual physical switch can dynamically join or leave the fabric switch without disrupting services to the rest of the network.

Furthermore, the automatic and dynamic configurability of the fabric switch allows a network operator to build its switching system in a distributed and “pay-as-you-grow” fashion without sacrificing scalability. The fabric switch's ability to respond to changing network conditions makes it an ideal solution in a virtual computing environment, where network loads often change with time.

It should also be noted that a fabric switch is distinct from a VLAN. A fabric switch can accommodate a plurality of VLANs. A VLAN is typically identified by a VLAN tag. In contrast, the fabric switch is identified by a fabric identifier (e.g., a cluster identifier), which is assigned to the fabric switch. Since a fabric switch can be represented as a logical chassis, the fabric identifier can also be referred to as a logical chassis identifier. A respective member switch of the fabric switch is associated with the fabric identifier. In some embodiments, a fabric switch identifier is pre-assigned to a member switch. As a result, when the switch joins a fabric switch, other member switches identify the switch as a member switch of the fabric switch.

In this disclosure, the term “fabric switch” refers to a number of interconnected physical switches which can form a single, scalable network of switches. The member switches of the fabric switch can operate as individual switches. The member switches of the fabric switch can also operate as a single logical switch for provisioning, controlling, and/or data forwarding. “Fabric switch” should not be interpreted as limiting embodiments of the present invention to a plurality of switches operating as a single, logical switch. In this disclosure, the terms “fabric switch” and “fabric” are used interchangeably.

Although the instant disclosure is presented using examples based on an encapsulation protocol, embodiments of the present invention are not limited to networks defined using one particular encapsulation protocol associated with a particular Open System Interconnection Reference Model (OSI reference model) layer. For example, embodiments of the present invention can also be applied to a multi-protocol label switching (MPLS) network. In this disclosure, the term “encapsulation” is used in a generic sense, and can refer to encapsulation in any networking layer, sub-layer, or a combination of networking layers.

The term “end host” can refer to any device external to a network (e.g., does not perform forwarding in that network). Examples of an end host include, but are not limited to, a physical or virtual machine, a conventional layer-2 switch, a layer-3 router, or any other type of network device. Additionally, an end host can be coupled to other switches or hosts further away from a layer-2 or layer-3 network. An end host can also be an aggregation point for a number of network devices to enter the network. An end host hosting one or more virtual machines can be referred to as a host machine. In this disclosure, the terms “end host” and “host machine” are used interchangeably.

The term “VLAN” is used in a generic sense, and can refer to any virtualized network. Any virtualized network comprising a segment of physical networking devices, software network resources, and network functionality can be referred to as a “VLAN.” “VLAN” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. “VLAN” can be replaced by other terminologies referring to a virtualized network or network segment, such as “Virtual Private Network (VPN),” “Virtual Private LAN Service (VPLS),” or “Easy Virtual Network (EVN).”

The term “packet” refers to a group of bits that can be transported together across a network. “Packet” should not be interpreted as limiting embodiments of the present invention to layer-3 networks. “Packet” can be replaced by other terminologies referring to a group of bits, such as “frame,” “cell,” or “datagram.”

The term “switch” is used in a generic sense, and can refer to any standalone or fabric switch operating in any network layer. “Switch” can be a physical device or software running on a computing device. “Switch” should not be interpreted as limiting embodiments of the present invention to layer-2 networks. Any device that can forward traffic to an external device or another switch can be referred to as a “switch.” Examples of a “switch” include, but are not limited to, a layer-2 switch, a layer-3 router, a TRILL RBridge, or a fabric switch comprising a plurality of similar or heterogeneous smaller physical switches.

The term “edge port” refers to a port on a network which exchanges data frames with a device outside of the network (i.e., an edge port is not used for exchanging data frames with another member switch of a network). The term “inter-switch port” refers to a port which sends/receives data frames among member switches of the network. A link between inter-switch ports is referred to as an “inter-switch link.” The terms “interface” and “port” are used interchangeably.

The term “switch identifier” refers to a group of bits that can be used to identify a switch. Examples of a switch identifier include, but are not limited to, a media access control (MAC) address, an Internet Protocol (IP) address, an RBridge identifier, or a combination thereof. In this disclosure, “switch identifier” is used as a generic term, is not limited to any bit format, and can refer to any format that can identify a switch.

The term “tunnel” refers to a data communication where one or more networking protocols are encapsulated using another networking protocol (typically, a lower-layer header is encapsulated with an upper-layer header). Although the instant disclosure is presented using examples based on a layer-3 encapsulation of a layer-2 protocol, “tunnel” should not be interpreted as limiting embodiments of the present invention to layer-2 and layer-3 protocols. A “tunnel” can be established for and using any networking layer, sub-layer, or a combination of networking layers.

Fast Convergence of Unicast Traffic

FIG. 1Aillustrates an exemplary switch facilitating fast convergence in a layer-2 event, in accordance with an embodiment of the present invention. In this example, switches102and104are coupled to a network100. Network100can be a local area network (LAN), a wide area network (WAN), or a data center network (DCN). If network100is a DCN, switch102and/or switch104can be top-of-the-rack switches. Switch102can include a plurality of hot-swappable cards. A hot-swappable card can be added to or removed from switch102while switch102remains operational. The plurality of switch cards of switch102can include one or more line cards and switch fabric cards.

Here, switch102includes a plurality of line cards112,114,116, and118. A respective line card can include one or more ports (or interfaces) that allow links to connect to switch102. For example, ports131and132of switch102couple switch102to network100and are in line card114. Switch102can also include one or more switch fabric cards (e.g., Peripheral Component Interconnect Express (PCIe) cards). These switch fabric cards facilitate switching fabric110of switch102. Switching fabric110allows internal switching of switch102(i.e., switching of packets within switch102, such as between line cards112and114). In some embodiments, a respective switch fabric card is coupled to a respective line card. As a result, a packet from a line card can be switched to any other line card by any of the switch fabric cards.

During operation, switch102and104establish and maintain a tunnel180between them. Tunnel180can be considered a tunnel instance at switch102. To forward a packet via tunnel180, switch102encapsulates the packet with an encapsulation header associated with a corresponding tunneling protocol. The encapsulation header can be a layer-3 header encapsulating the inner layer-2 header of the packet. The source and destination addresses (e.g., the IP addresses) in the encapsulation header correspond to switches102and104, respectively. In addition, to forward the encapsulated packet to the next-hop switch leading to switch104, such as switch103in network100, switch102adds an outer layer-2 header to the encapsulation header. This outer layer-2 header is determined based on the forwarding information associated with the tunnel instance of tunnel180. The source and destination addresses (e.g., the MAC addresses) in the outer layer-2 header correspond to switches102and103, respectively.

In some embodiments, switch102can include a control card120, which can also be part of a back plane of switch102or can be inserted into switch102, that controls switch102. Control card120implements routing protocols and establishes routes with other switches from switch102. Control card120includes a set of management modules122, which comprises a layer-2 module123and a layer-3 module124. Layer-2 module123manages the layer-2 operations of switch102(e.g., maintaining and updating layer-2 forwarding information). Layer-3 module124is responsible for managing forwarding information associated with a respective layer-3 forwarding entry. For example, the next-hop forwarding information associated with tunnel180is maintained by layer-3 module124.

Management modules122can also include one or more tunnel modules that are associated with a corresponding tunneling protocol. For example, management modules122can include multiprotocol label switching (MPLS) module125and Virtual Extensible Local Area Network (VXLAN) module126. If tunnel180is based on MPLS, the tunnel instance of tunnel180can be managed by MPLS module125. MPLS module125maintains a list of MPLS tunnel instances in switch102and manages information associated with the encapsulation header associated with a respective MPLS tunnel instance. This list may include information associated with an encapsulation header for a respective MPLS tunnel instance. On the other hand, if tunnel180is based on VXLAN, the tunnel instance of tunnel180can be managed by VXLAN module126. VXLAN module126also maintains a list of VXLAN tunnel instances in switch102and manages information associated with the encapsulation header associated with a respective VXLAN tunnel instance.

Information associated with an encapsulation header can include source and destination addresses of the encapsulation header and the outer layer-2 header. In addition, one or more virtual network instances, such as VPLS instances, can operate on tunnel180. Management modules122then further includes a VPLS module127. The VPLS instances can be managed by VPLS module127. VPLS module127maintains a list of VPLS instances in switch102and manages information associated with the encapsulation header associated with the corresponding VPLS instance. In this way, a respective tunnel/virtual network module maintains a list of corresponding tunnel/virtual network instances and their encapsulation information. A tunnel instance or a virtual network instance in switch102can be referred to as a communication instance of switch102.

On the other hand, a respective line card includes the corresponding modules of management modules122. For example, line card114includes a layer-2 module133, a layer-3 module134, an MPLS module135, a VXLAN module136, and a VPLS module137. Layer-2 module133is responsible for incorporating the layer-2 forwarding information (e.g., based on MAC address learning) into the forwarding hardware (e.g., in TCAM) of line card114. Similarly, layer-3 module134is responsible for incorporating layer-3 forwarding information into the forwarding hardware of line card114.

Switch102can support a large number of tunnel and virtual network instances. Suppose that switch102detects a layer-2 event190associated with switch102(denoted with an “X”). Layer-2 event190can affect the layer-2 forwarding information associated with the tunnel/virtual network instances. As a result, the forwarding information of each of the instances is updated by the corresponding module. When switch102detects layer-2 event190, switch102determines that port131is unavailable (e.g., due to a failure). As a result, layer-2 module123flushes the MAC addresses learned from port131. This causes layer-3 module124to issue an ARP request for the IP address of switch103. Layer-3 module124then receives an ARP response comprising a mapping between the IP and MAC addresses of switch103via port132.

Layer-2 module123learns the MAC address from the ARP response via port132. In some embodiments, layer-2 module123obtains learned MAC address information from layer-2 module133. Upon receiving the ARP response, layer-3 module124obtains information associated with port132from layer-2 module123(e.g., from which port layer-2 module123has learned the MAC address) via a notification message182. This message triggers a switchover, which causes layer-3 module124to select port132for forwarding traffic that was previously forwarded by port131. In some embodiments, ports131and132are part of a link aggregation, which operates as a single logical link. The link aggregation can be in a virtual link aggregation group (vLAG), which operates as a single logical link with “active-active” forwarding and couples an end device to a plurality of switches.

Based on the ARP resolution, layer-3 module124associates port132with the IP address to MAC address mapping. Since the port of the ARP resolution has changed, a respective tunnel and/or virtual network instances for which switch103is the next-hop switch needs this updated information for providing its outer layer-2 header. In some embodiments, layer-2 module123and layer-3 module124use inter-process communication (IPC) messages for exchanging notification messages. Examples of inter-process communication messages include, but are not limited to, a socket, shared memory, a message queue, message passing, and pipes.

With existing technologies, layer-3 module124notifies a respective tunnel and/or virtual network module of switch102regarding the updated information, such as that the ARP has been resolved at port132. The tunnel/virtual network module, in turn, generates a control message associated with each instance of the corresponding tunnel/virtual network notifying the corresponding module of a respective line card for updating the forwarding hardware of the line card. For example, VPLS module127of control card120notifies a corresponding VPLS module137of line card114for a respective VPLS instance. VPLS module137then updates the forwarding hardware of line card114. In the same way, VPLS module127notifies a corresponding VPLS module of line cards112,116, and118. As a result, a large number of control messages are generated and propagated within switch102, requiring extensive processing at control card120. This can cause a longer convergence time for switch102.

To solve this problem, embodiments of the present invention facilitate efficient management of the control messages in switch102. Furthermore, a respective module in a line card registers with the layer-3 module of the line card to receive notification. For example, MPLS module135, VXLAN module136, and VPLS module137of line card114register with layer-3 module134for receiving notification associated with a layer-2 event. As a result, MPLS module135, VXLAN module136, or VPLS module137receives a corresponding notification message from layer-3 module134regarding the layer-2 event. It should be noted that layer-2 module123can communicate with layer-2 module133to send or receive information associated with a layer-2 event, such as the learning of the MAC address associated with the ARP resolution.

Instead of causing a respective tunnel/virtual network module to generate a notification message for each tunnel/virtual network instance, layer-3 module124notifies layer-3 module134of line card114regarding layer-2 event190. Layer-3 module124can send a control message184via the internal communication channel (e.g., an internal control bus) of switch102to layer-3 module134. In the same way, layer-3 module124sends a control message for the layer-3 module of line cards112,116, and118. Layer-3 module134then sends one or more notification messages186to notify a respective tunnel/virtual network module in line card114. For example, layer-3 module134notifies MPLS module135regarding the updated information. MPLS module135then updates the entry associated with a respective MPLS instance in the forwarding hardware of line card114.

In the same way, a respective tunnel/virtual network module then updates the entries of a respective instance associated with the module in the forwarding hardware of line card114. Since the tunnel/virtual network module updates the forwarding hardware at line card114based on the notification from layer-3 module134, the module at the line card does not rely on a message from the corresponding module of control card120. In this way, control card120provides fast convergence to switch102upon detecting layer-2 event190by reducing the notifications across control card120as well as the control messages within switch102.

FIG. 1Billustrates an exemplary change to layer-2 forwarding information associated with overlay tunnels, in accordance with an embodiment of the present invention. In this example, switch102is coupled with an end device192, which can be a physical device or a virtual machine running on a physical host. In this example, tunnel180can be an MPLS tunnel, which supports multiple VPLS instances. Switch102maintains encapsulation information (e.g., MPLS labels, pseudo-wire labels, control words, etc.) for each VPLS instance. In addition, switches102and104can establish and maintain a VXLAN tunnel181, which can be represented by a corresponding VXLAN instance. Switch102can also maintain encapsulation information (e.g., source and destination IP addresses, User Datagram Protocol (UDP) information, VXLAN network identifier (VNI), etc.) associated with the VXLAN instance. Suppose that switch102maintains VPLS encapsulation information162and164for two VPLS instances. Similarly, switch102maintains VXLAN encapsulation information166for the VXLAN instance.

The VPLS instances and the VXLAN instance of switch102can have the same next-hop switch103. Hence, these instances may use the same layer-2 encapsulation information, which can be used to generate the outer layer-2 header for each of these instances. In this example, VPLS encapsulation information162and164, and VXLAN encapsulation information166point to the same layer-2 encapsulation information172, which is associated with port131. This allows switch102to efficiently manage the layer-2 encapsulation information. Switch102uses the encapsulation information of a tunnel/virtual network instance to generate the encapsulation header (e.g., for VXLAN, it can be the outer layer-3 header and the VXLAN header). Switch102also uses layer-2 encapsulation information172to generate the outer layer-2 header.

Upon detecting layer-2 event190, switch102initiates switchover to port132. Switch102then updates VPLS encapsulation information162and164, and VXLAN encapsulation information166to point to layer-2 encapsulation information174, which is associated with port132, in the forwarding hardware of a respective line card. In some embodiments, switch102determines layer-2 encapsulation information174based on an ARP resolution via port132. In this way, switch102facilitates a fast convergence in layer-2 event190, as described in conjunction withFIG. 1A. When switch102receives another layer-2 packet144from end device192after layer-2 event190has occurred, switch102encapsulates packet144with an encapsulation header156based on VPLS encapsulation information162. Switch also obtains layer-2 encapsulation information174, adds an outer layer-2 header158, and generates an encapsulated packet148, which includes packet144as a payload. Switch102then forwards packet148using port132via tunnel180.

Fast Convergence of Multi-Destination Traffic

FIG. 2Aillustrates an exemplary switch facilitating fast convergence for multi-destination traffic in response to a layer-2 event, in accordance with an embodiment of the present invention. To manage multi-destination traffic (e.g., BUM traffic), management modules122of switch102can include a multi-destination traffic module202, which is responsible for managing the forwarding information associated with multi-destination traffic for overlay tunnels in a respective line card. Furthermore, multi-destination traffic module202also configures the traffic manager of a respective line card for multi-destination traffic.

With existing technology, upon receiving a notification regarding layer-2 event190, a respective tunnel/virtual network module of control card120sends a notification message to multi-destination traffic module202for a respective tunnel/virtual network instance. This notification message includes the updated layer-2 information. In response, multi-destination traffic module202configures a respective line card and its traffic manager and updates the forwarding information for multi-destination traffic (e.g., updates reverse path forwarding) for a respective instance. This leads to extensive processing and message exchanges at control card120. As a result, switch102may face a longer convergence time for multi-destination traffic.

To solve this problem, embodiments of the present invention facilitate efficient management of the control messages to multi-destination traffic module202. Instead of receiving notification for each instance of a tunnel/virtual network, multi-destination traffic module202registers with layer-3 module124for receiving notification regarding layer-2 events. Hence, MPLS module125, VXLAN module126, and VPLS module127refrain from sending notifications to multi-destination traffic module202regarding layer-2 events. Upon receiving notification message182, layer-3 module124sends a notification message282to multi-destination traffic module202comprising the updated forwarding information based on the ARP resolution via port132.

In some embodiments, multi-destination traffic module202maintains a list (e.g., a linked list, an array, a database table, etc.) that lists the encapsulation information of a respective tunnel/virtual network instance that needs configuration from multi-destination traffic module202. Upon receiving notification message282, multi-destination traffic module202sends one or more control messages284to a respective line card. Multi-destination traffic module202can traverse the list and send a control message to a respective line card for a respective instance. Such a control message can include configuration information for the traffic manager of a line card and the updated forwarding information associated with a tunnel/virtual network instance. The corresponding module in the line card then updates the forwarding hardware of the line card with the updated forwarding information. In this way, multi-destination traffic module202reduces the notifications across the control card as well as the control messages within the switch.

FIG. 2Billustrates an exemplary data structure for facilitating fast convergence for multi-destination traffic in response to a layer-2 event, in accordance with an embodiment of the present invention. Multi-destination traffic module202maintains a tunnel instance list210. A respective entry in tunnel instance list210can correspond to a line card of switch102. In this example, entries212,214,216, and218correspond to line cards112,114,116, and118, respectively, of switch102. A respective entry in tunnel instance list210can point to a list of tunnel and virtual network instances for the corresponding line card.

Upon receiving the updated information from layer-3 module124, multi-destination traffic module202traverses tunnel instance list210and retrieves an entry from list210. Multi-destination traffic module202identifies the line card and the tunnel/virtual network instance associated with the entry, and configures the line card with the updated information for the tunnel/virtual network instance. Multi-destination traffic module202also updates the forwarding hardware of the line card with the updated information for the tunnel/virtual network instance.

Entry212can include a list262, which includes a plurality of tunnel/virtual network instances, such as instances222,224, and226, instantiated in line card112. Entry214can include a list264, which includes a plurality of tunnel/virtual network instances, such as instances232,234, and236, instantiated in line card114. Entry216can include a list266, which includes a plurality of tunnel/virtual network instances, such as instances242,244, and246, instantiated in line card116. Entry218can include a list268, which includes a plurality of tunnel/virtual network instances, such as instances252,254, and256, instantiated in line card118.

Here, entry212can include a pointer to a list262that includes the tunnel and virtual network instances of line card112. In this way, tunnel instance list210can be a doubly linked list or a multi-dimensional array. It should be noted that if the same tunnel/virtual network instances are instantiated in a respective line card, entries of tunnel instance list210can include the tunnel/virtual network instances. Under such circumstances, multi-destination traffic module202can use the same list of tunnel/virtual network instances for a respective line card. In some embodiments, list210can include the tunnel/virtual instances, such as instances222, and234, instead of entries212,214,216, and218. Multi-destination traffic module202then can include an indicator the tunnel/virtual network instances indicating which instance is associated with which line card.

Operations for Fast Convergence

FIG. 3Apresents a flowchart illustrating an exemplary process of a layer-3 module in a control card of a switch efficiently managing a layer-2 event, in accordance with an embodiment of the present invention. During operation, the layer-3 module receives a notification message from a local layer-2 module, which is on the same control card (operation302) and obtains ARP resolution information from the notification message (operation304). The ARP resolution information can include the port from which the layer-2 module has learned the MAC address of the ARP resolution. The layer-3 module generates a control message comprising the ARP resolution information (operation306) and sends the control message to the layer-3 module of a respective line card (operation308). In some embodiments, the layer-3 module precludes the local layer-3 module from sending a notification message comprising the ARP resolution information to a respective tunnel/virtual network module (operation310).

FIG. 3Bpresents a flowchart illustrating an exemplary process of a layer-3 module in a line card of a switch efficiently propagating changes associated with a layer-2 event, in accordance with an embodiment of the present invention. During operation, the layer-3 module receives a control message from the layer-3 module of the control card of the switch (operation332). The layer-3 module obtains the ARP resolution information from the control message (operation334). The layer-3 module generates a notification message comprising configuration information associated with the ARP resolution (operation336) and sends the notification message to a respective tunnel/virtual network instance (operation338). The configuration information can include the information needed to update a forwarding entry in the forwarding hardware of a line card based on the ARP resolution information.

FIG. 3Cpresents a flowchart illustrating an exemplary process of a tunnel/virtual network module in a line card of a switch incorporating changes associated with a layer-2 event, in accordance with an embodiment of the present invention. During operation, the module receives a notification message from the layer-3 module of the line card (operation352). The module obtains the configuration information associated with the ARP resolution from the notification message (operation354). The module identifies a respective tunnel/virtual network associated with the local module (operation356). The module updates the entry associated with the identified tunnel/virtual network instance in the forwarding hardware of the local line card based on the configuration information (operation356).

FIG. 4Apresents a flowchart illustrating an exemplary process of a multi-destination traffic module in a control card of a switch efficiently managing a layer-2 event for multi-destination traffic, in accordance with an embodiment of the present invention. Here, the multi-destination traffic module is registered with the layer-3 module of the control card. During operation, the multi-destination traffic module receives a notification message from the layer-3 module (operation402). It should be noted that the multi-destination traffic module does not receive a notification message from an instance.

The multi-destination traffic module obtains the configuration information associated with the ARP resolution from the notification message (operation404). The multi-destination traffic module identifies a respective tunnel/virtual network instance for a respective line card from a local tunnel instance list (operation406). The multi-destination traffic module generates a control message to configure multi-destination traffic forwarding for the identified tunnel/virtual network instance in the corresponding line card (operation408). The multi-destination traffic module then sends the control message to the corresponding line card (operation410).

FIG. 4Bpresents a flowchart illustrating an exemplary process of a line card of a switch incorporating changes associated with a layer-2 event for multi-destination traffic, in accordance with an embodiment of the present invention. During operation, the line card receives a control message from the multi-destination traffic module of the control card (operation452). The line card obtains, from the control message, information for configuring multi-destination traffic associated with a tunnel/virtual network instance in the traffic manager of the line card (operation454). The line card then configures the traffic manager based on the obtained information (operation456). The line card also obtains, from the control message, the multi-destination traffic forwarding entry/entries associated with the tunnel/virtual network instance (operation458). The line card then updates the local forwarding hardware based on the obtained entry/entries (operation460).

Intelligent Buffering and Adaptive Synchronization

FIG. 5Aillustrates an exemplary message exchange for facilitating intelligent buffering and adaptive synchronization for efficiently managing an event for multi-destination traffic, in accordance with an embodiment of the present invention. During operation, switch102may detect an event (e.g., a layer-3, configuration, network, or operational event). This event may change layer-3 information associated with a tunnel/virtual network, such as a change of IP address or the layer-3 next-hop switch. Such an event may cause a tunnel/virtual network module to send a notification message to multi-destination traffic module202for a respective instance associated with the module.

With existing technologies, multi-destination traffic module202can be in an occupied state when many of these notifications arrive. As a result, multi-destination traffic module202may not receive the notification message from tunnel/virtual network module500of switch102. In this example, tunnel/virtual network module500can be one of: MPLS module125, VXLAN module126, and VPLS module127. If module500detects an unsuccessful transmission of the notification message (e.g., by detecting a failure to transmit an IPC message), module500resends the notification message. This leads to repeated resending of these instructions, which causes additional processing in switch102and delay in convergence.

To solve this problem, embodiments of the present invention allow module500to utilize intelligent buffering of the updates and adaptive synchronization with multi-destination traffic module202. During operation, module500detects an event (e.g., a configuration event) and checks the availability of multi-destination traffic module202(operation502). Suppose that module500detects that multi-destination traffic module202is busy. Module500then sets a local state for a respective tunnel/virtual network instance, which is associated with module500and affected by the event, to be “updated” (operation504). In some embodiments, the “updated” state can be indicated by setting a bit (e.g., a dirty bit) for that instance. For example, if module500is VXLAN module126, module500maintains a list of the VXLAN tunnel instances in switch102and sets a local state for a VXLAN instance affected by the event as “updated” in the entry for the instance in the list.

Module500sends a probe message to multi-destination traffic module202(operation506). This probe message can be short in length and, therefore, ensures that the overhead of probe message processing in multi-destination traffic module202is nominal. If module500does not receive a response within a predetermined time, module500can resend the probe message (operation508). Upon receiving the probe message, multi-destination traffic module202can add this probe message to the processing queue. Multi-destination traffic module202eventually detects the probe message at the top of the queue (operation510), thereby determining that multi-destination traffic module202is available for processing control instructions. However, if multi-destination traffic module202is in an occupied state executing other instructions when the probe message is sent, multi-destination traffic module202may not receive the probe message. This causes the probe message to time out and module500to resend the probe message (operation508).

If multi-destination traffic module202is available, multi-destination traffic module202detects the probe message at the top of its queue (operation510) and sends a probe response message to module500(operation512). Upon receiving the probe response message, module500determines that multi-destination traffic module202is available. Module124then stores, in a buffer, configuration information of the instances marked by the set state (e.g., the “updated” state) (operation514). If module500detects a buffer threshold (e.g., the buffer has reached a predetermined size or a timer for the buffer has expired), module500generates a notification message with the configuration information in the buffer (operation516). Module500then sends the notification message (operation518).

Upon receiving the notification message, multi-destination traffic module202updates the information associated with the tunnel/virtual network instance in the line cards of switch102(operation520). For example, if the next-hop IP address has changed for an instance, multi-destination traffic module202may update the entry associated with the instance in the forwarding hardware of a respective line card. Multi-destination traffic module202may obtain the updated information from an ARP resolution. If module500detects a successful transmission, module500clears the buffer (operation522).

FIG. 5Billustrates exemplary global and local states for facilitating intelligent buffering and adaptive synchronization for efficiently managing an event for multi-destination traffic, in accordance with an embodiment of the present invention. To utilize intelligent buffering and adaptive synchronization, switch102maintains global and local states. The global states indicate the state of multi-destination traffic module202, which is referred to as probe state550, based on the probe message exchange. Global states can be accessed and modified by any of the modules participating in the probe message exchange. Furthermore, a respective local state is associated with a corresponding tunnel/virtual network instance. It should be noted that a respective module maintains its own local states, which are not accessible by other modules.

Probe state550indicates whether multi-destination traffic module202is busy. Probe state550includes a busy state552and an available state554. Initially, probe state550is at busy state552. When module500sends a probe message, probe state550remains at busy state552. If the probe message is resent, probe state550still remains at busy state552. If module500receives a probe response, probe state550transitions to available state554. If module500sends a probe message while probe state550is in available state554, probe state550transitions to busy state552.

In some embodiments, module500maintains a tunnel/virtual network instance list560, which lists a respective instance associated with module500. Suppose that tunnel/virtual network instances562,564,566, and568are associated with module500. For example, if module500is VPLS module127, instances562,564,566, and568are VPLS instances. Module500can maintain states572,574,576, and578for instances562,564,566, and568, respectively. If an event affects instances562,566, and568, and probe state550is in busy state552, module500sets states572,576, and578as “updated” instead of sending individual notification messages to multi-destination traffic module202.

If probe state550is in available state554, module500stores information associated with instances562,566, and568in a buffer and clears states572,576, and578. If the buffer reaches a predetermined size or a timer associated with the buffer expires, module500generates a notification message comprising information associated with instances562,566, and568, and sends the notification message to multi-destination traffic module202. If module500detects a successful transmission of the notification message, module500clears the buffer. In this way, module500uses local and global states to facilitate intelligent buffering and adaptive synchronization.

It should be noted that module500can update the states in list560to indicate the most relevant updates. For example, module500can detect another event that affects instances566and568, and renders the previous event irrelevant for instance562. Under such circumstances, the updates to instance562caused by the previous event may no longer be needed. Module500then clears state572(denoted with a dashed strike-though line). In this way, when probe state550is in available state554, module500inserts the most recent and relevant updates into the buffer.

Operations for Intelligent Buffering and Adaptive Synchronization

FIG. 6Apresents a flowchart illustrating an exemplary process of a module in a control card of a switch facilitating intelligent buffering for efficiently managing an event for multi-destination traffic, in accordance with an embodiment of the present invention. During operation, the module identifies the tunnel/virtual network affected by an event (operation602) and determines whether the effect is relevant (operation604). An effect can become irrelevant if another event leads to such a change that the effect may not be applicable or cause a conflict.

If the effect is relevant, the module sets the state of the respective instance affected by the event (operation606) and checks whether the previously set states are relevant (operation608). If the previously set states are not relevant, the module clears the previously set states that are no longer relevant (operation610). If the previously set states are relevant, the module may not change the previously set states. On the other hand, if the effect is not relevant (operation604), the module refrains from setting state for a respective instance for which the effect is not relevant (operation612).

FIG. 6Bpresents a flowchart illustrating an exemplary process of a module facilitating adaptive synchronization for efficiently managing an event for multi-destination traffic, in accordance with an embodiment of the present invention. During operation, the module sends a probe message to the multi-destination traffic module (operation652). The module then checks whether a probe response message is received within a predetermined time (operation654). If a probe response message is not received within a predetermined time (e.g., a timer has expired for the probe message), the module resends the probe message to the multi-destination traffic module (operation652).

On the other hand, if a probe response message is received within a predetermined time (e.g., received without the timer expiring), the module identifies a respective instance with a set state (e.g., an “updated” state) (operation656) and inserts information associated with the identified instances in the buffer (operation658). Such information can include the information needed to configure the forwarding entries of the corresponding instance for multi-destination traffic in response to the event. The module checks whether the buffer has reached the buffer threshold (operation660). If the buffer has not reached the buffer threshold, the module continues to identify a respective instance with a set state (operation656).

On the other hand, if the buffer has reached the buffer threshold, the module generates a notification message comprising the buffer (operation662) and sends the notification message to the multi-destination traffic module (operation664). The module then checks whether the notification message is successfully transmitted (operation666). If the notification message is successfully transmitted, the module clears the buffer (operation670). If the notification message is not successfully transmitted, the module transitions the probe state to “busy” (operation668) and sends a probe message to the multi-destination traffic module (operation652).

Exemplary Switch

FIG. 7illustrates an exemplary switch facilitating fast convergence in an event, in accordance with an embodiment of the present invention. In this example, a switch700includes a number of communication ports702, a packet processor710, a control card module730, and a storage device750. Switch700can also include switch modules, such as processing hardware of switch700(e.g., ASIC chips). Packet processor710extracts and processes header information from the received packets. Packet processor710can identify a switch identifier associated with the switch in the header of a packet. Switch700can also include a control processor792and a memory794.

In some embodiments, switch700maintains a membership in a fabric switch. Switch700maintains a configuration database in storage device750that maintains the configuration state of every switch within the fabric switch. Switch700maintains the state of the fabric switch, which is used to join other switches. In some embodiments, switch700can be configured to operate in conjunction with a remote switch as an Ethernet switch.

Communication ports702can include inter-switch communication channels for communication within the fabric switch. This inter-switch communication channel can be implemented via a regular communication port and based on any open or proprietary format. Communication ports702can also include one or more extension communication ports for communication between neighbor fabric switches. Communication ports702can include one or more TRILL ports capable of receiving frames encapsulated in a TRILL header. Communication ports702can also include one or more IP ports capable of receiving IP packets. An IP port is capable of receiving an IP packet and can be configured with an IP address. Packet processor710can process TRILL-encapsulated frames and/or IP packets (e.g., tunnel encapsulated packets).

Switch700can also include a switching unit760, which further includes one or more switch fabric cards (e.g., PCIe cards). Switching unit760also includes a plurality of line cards762and764. The switch fabric cards facilitate switching fabric770of switch700. Switching fabric770allows internal switching of switch700(i.e., switching of packets within switch700, such as between line cards762and764). In some embodiments, a respective switch fabric card is coupled to a respective line card. As a result, a packet from a line card can be switched to any other line card by any of the switch fabric cards.

A respective line card in line cards762and764includes one or more ports (e.g., ports in communication ports702, and forwarding hardware (e.g., a TCAM). Control card module730determines routing and forwarding tables for switch700, and comprises a control processor732(e.g., processing circuitry) and a management module734. Management module734manages one or more tunnel or virtual network instances at switch700. Management module734can be one or more of: MPLS module125, VXLAN module126, and VPLS module127.

During operation, control card module730determines an event associated with the layer-2 operations of switch700. Control card module730refrains from notifying management module734regarding the layer-2 event and notifies line card762(or764) regarding the event. Line card762then configures one or more layer-3 forwarding entries in the forwarding hardware of line card762based on the notification. Control card module730determines the event by determining an unavailability of a first port of switch700, obtaining an ARP response via a second port of switch700, and associating the second port with a mapping between an IP address and a MAC address obtained from the ARP response.

In some embodiments, control card module730also includes a multi-destination module736, which registers to receive information of the layer-2 event and configures a set of layer-3 forwarding entries for multi-destination traffic in the forwarding hardware of line cards762and764. Control card module730then precludes management module734from notifying multi-destination module736regarding the layer-2 event. Multi-destination module736can maintain a list of tunnel and virtual network instances of the switch. Multi-destination module identifies a tunnel or virtual network instance, which is associated with line card762, affected by the layer-2 event from the list.

In some embodiments, management module734determines an event associated with layer-3 operations of the switch and sends a probe message to multi-destination module736to determine whether multi-destination module736is available. If management module734receives a probe response message from multi-destination module736, management module734sends, to multi-destination module736, a notification message comprising information associated with a set of tunnel or virtual network instances affected by the layer-3 event. Examples of a layer-3 event include, but are not limited to, a network event, a switch event, a configuration event, or an operational event.

Management module734can identify the set of tunnel or virtual network instances, which are managed by the management module and affected by the layer-3 event. If management module734does not receive a probe response message within a period of time, management module734sets a state (e.g., a dirty bit indicating an “updated” state) for the identified tunnel or virtual network instances. Management module734can maintain a list of the tunnel or virtual network instances associated with the management module. The state of a tunnel or virtual network instance is then maintained in an entry of the tunnel or virtual network instance in the list.

If management module734receives the probe response message, management module734inserts information associated with the tunnel or virtual network instances with a set state into a buffer. If the buffer reaches a threshold, management module734includes the information of the buffer in the notification message. If the management module734detects an unsuccessful transmission of the notification message, management module734sends a second probe message to multi-destination module736to determine whether multi-destination module736is available. Management module734can determine whether multi-destination module736is available based on a global state, which is accessible by management module736, and one or more other modules of control card module730.

Note that the above-mentioned modules can be implemented in hardware as well as in software. In one embodiment, these modules can be embodied in computer-executable instructions stored in a memory which is coupled to one or more processors in switch700. When executed, these instructions cause the processor(s) to perform the aforementioned functions.

In summary, embodiments of the present invention provide a switch and a method that facilitate fast convergence to a switch. In one embodiment, the switch includes a storage device, one or more line cards, and a control card. A respective line card includes one or more ports and forwarding hardware. The control card determines routing and forwarding tables for the switch, and comprises processing circuitry and a management module. The management module manage a tunnel or virtual network at the switch. During operation, the control card determine an event associated with layer-2 operations of the switch. The control card refrains from notifying the management module regarding the event and notifies a first line card in the one or more line cards regarding the event. The first line card then updates a layer-3 forwarding entry in the corresponding forwarding hardware based on the notification.

In another embodiment, the switch includes a storage device, one or more line cards, and a control card. A respective line card includes one or more ports and forwarding hardware. The control card determines routing and forwarding tables for the switch, and comprises processing circuitry, a management module, and a multi-destination module. The management module manages one or more tunnels or virtual networks at the switch. The multi-destination module configures layer-3 forwarding entries for multi-destination traffic in the forwarding hardware of the one or more line cards. During operation, the management module determines an event associated with layer-3 operations of the switch and sends a probe message to the multi-destination module to determine whether the multi-destination module is available. If the management module receives a probe response message from the multi-destination module, the management module sends, to the multi-destination module, a notification message comprising information associated with a first tunnel or virtual network affected by the event.

The methods and processes described herein can be embodied as code and/or data, which can be stored in a computer-readable non-transitory storage medium. When a computer system reads and executes the code and/or data stored on the computer-readable non-transitory storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the medium.

The foregoing descriptions of embodiments of the present invention have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit this disclosure. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. The scope of the present invention is defined by the appended claims.