Patent Publication Number: US-8531991-B2

Title: Multi-chassis emulated switch

Description:
BACKGROUND OF THE INVENTION 
     1. Technical Field 
     The present invention relates to computer networking. 
     2. Description of the Related Art 
     Data management within organizations is an ever increasing concern, especially with the rise of the Internet information age. The heart of this data management function is sometimes known as a data center. Over the last decade, data centers have evolved into the strategic focus of Information Technology (IT) efforts to protect, optimize, and grow the organization. 
     Data center managers face several challenges in fulfilling these goals. Most enterprise data centers grew rapidly to meet the explosive economic growth of recent times. Consequently, applications commonly stand alone in underutilized, isolated infrastructure silos. Each infrastructure silo is designed based on the inclination of the specific application being deployed, so that a typical data center supports a broad I assortment of operating systems, computing platforms, and storage systems. The disparate infrastructures supporting different application “islands” are difficult to change or expand and expensive to manage, integrate, secure, and back up.  FIG. 1  illustrates this type of “isolated application” environment. 
     One solution to this problem is to design a data center environment that is highly scalable, resilient, secure, and able to integrate multiple applications and protocols. One such solution is known as the Data Center Network Architecture. A specific implementation of the Data Center Network Architecture is known as Data Center Ethernet (DCE). DCE allows for consolidation of input and output, and improved forwarding of communications within the network. This may be accomplished via specialized protocols and functionality operated by switches within a DCE network via network layer 2. Each of the switches within the DCE network may be a layer 2 device.  FIG. 2  illustrates a DCE network. Edge switch  200  may be connected to a server  202 . Edge switch  204  may be connected to server  206 . Edge switches  200 ,  204  may then be connected to several core switches  208 ,  210 , which then may be connected to other edge switches  212 ,  214 . Each DCE switch may be assigned a unique identifier. A routing protocol, such as Intermediate-System-to-Intermediate-system (IS-IS), may be used inside DCE. Switches using this routing protocol may append information to frames sent though the DCE. This appended information may be in the form of a MAC-in-MAC header attached to the frame. Edge switches  212 ,  214  may then each be connected to non-DCE devices, such as Classic Ethernet (CE) switches  216 . CE switches do not run the forwarding protocols supported by DCE, and do not append the MAC-in-MAC information. They run a variant of the Spanning Tree protocol. They are connected to the DCE network. 
     Rather than forwarding frames to MAC addresses, DCE switches send frames to edge switches based on the edge switch identification via the MAC-in-MAC header. The edge switch then knows which of its ports to send the frame out to arrive at the correct MAC address (for example, the port connected to switch  216 ), and strips off the MAC-in-MAC header prior to doing so. 
     The network design depicted in  FIG. 2 , however, encounters a problem during actual operation. Specifically, when two links originate from the same CE switch  216  to different DCE switches  212 ,  214 , the spanning tree protocols operated by CE switches recognize this as a spanning tree loop. The remedy for such a loop is to activate only one link at a time. This, however, eliminates the possibility of load sharing and providing redundancy across 2 (or more) DCE chassis. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates a typical “isolated application” environment. 
         FIG. 2  illustrates a typical DCE network. 
         FIG. 3  illustrates an example network of layer 2 switches including an emulated switch. 
         FIG. 4  illustrates a standard DCE hierarchical address format. 
         FIG. 5  illustrates an example method for configuring a system of devices including a spanning tree device connected to two or more edge switches in a network of layer 2 switches. 
         FIG. 6  illustrates an example method for forwarding a unicast frame from a device in a network of layer 2 switches to a spanning tree device via one of a plurality of layer 2 switches connected to the spanning tree device. 
         FIG. 7  illustrates an example method for handling a unicast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. 
         FIG. 8  illustrates an example method for handling a multicast or broadcast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. 
         FIG. 9  illustrates an example apparatus for configuring a system of devices including a spanning tree device connected to two or more edge switches in a network of layer 2 switches. 
         FIG. 10  illustrates an example apparatus for forwarding a unicast frame from a device in a network of layer 2 switches to a spanning tree device via one of a plurality of layer 2 switches connected to the spanning tree device. 
         FIG. 11  illustrates an example apparatus for handling a unicast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. 
         FIG. 12  illustrates an example apparatus for handling a multicast or broadcast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. 
     
    
    
     DESCRIPTION OF EXAMPLE EMBODIMENTS 
     Overview 
     A solution is provided wherein the interfaces between multiple chassis (e.g., edge switches) in a network of layer 2 devices and a spanning tree device are treated as a single emulated switch. This emulated switch effectively enables two different views to the two different sides. Thus, frames from the network of layer 2 switches destined to any port of the emulated switch may take any of the links (through any of the physical switches), thereby enabling effective load-balancing for frames traveling from the layer 2 network side into the spanning tree device. Meanwhile the spanning tree device does not recognize an illegal loop in its connection to two different edge switches as it views the two links as a single logical EtherChannel. 
     Example Embodiments 
     In this application, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be obvious, however, to one skilled in the art, that the present invention may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order to not obscure the present invention. 
     A solution is provided wherein the interfaces between multiple chassis (e.g., edge switches) and non-DCE devices are treated as a single emulated switch. This emulated switch effectively enables 2 different views to the 2 different sides.  FIG. 3  illustrates an example network of layer 2 switches including an emulated switch. Here, emulated switch  300  is viewed as being between edge switches  302 ,  304  and the CE device  306 . It appears to be equidistant from each of the edge switches  302 ,  304 . For the CE-side, it appears that the multiple links are actually a single port bundle (a single EtherChannel). Thus the CE switch/host would now use the multiple ports in a load-balanced fashion. For the DCE-side, it makes the multiple links appear as a single port of the emulated switch, which is physically connected to the multiple DCE switches that participate with the emulated switch. Thus, frames destined to any port of the emulated switch may take any of the links (through any of the physical DCE switches), thereby enabling effective load-balancing for frames traveling from DCE side into the CE switch/host. 
     DCE switches typically each have their own switch identifications and independently participate in DCE forwarding. In an embodiment of the present invention, the edge switches connected to the non-DCE device co-ordinate the emulated switch identification that is to be used to represent the multi-chassis etherchannels. In this embodiment, both physical switches may advertise their reachability to this emulated switch identification to the rest of the DCE network. The emulated switch is advertised as being equidistant from the edge switches. This allows the rest of the DCE network to build shortest path routes towards the emulated switch, which pass through one of the edge switches. Since the shortest path is determined based on physical proximity to the emulated switch, this effectively load balances between the edge switches connected to the emulated switch. This is in direct conflict with the prior art, which simply disabled one of the links, routing all traffic through only one of the edge switches, creating a potential bottleneck for the traffic. 
     In an embodiment of the present invention, broadcasts and/or multicasts originating from the DCE network would be allowed only through one port of this multi-chassis Etherchannel of the emulated switch. This requires coordination between the physical DCE switches. Similarly, broadcasts and/or multicasts originating from a non-DCE network could come into the DCE switches through any of the etherchannel ports. The decision as to which of the links to use may be based on some sort of load balancing algorithm. The goal would likely be to have traffic split evenly among the links. A Self-forwarding check on the Multi-Chassis EtherChannel ports may also compare the frame&#39;s hierarchical address with the port&#39;s hierarchical address. Therefore, even if the frame ingressing from one Multi-Chassis EtherChannel port tries to egress out of another Multi-Chassis EtherChannel ports (of the different physical switch), it can be dropped due to the self-forwarding check. 
     Unicast frames towards hosts belonging to the emulated switch could come to any of the edge switches connected to the emulated switch. In an embodiment of the present invention, each of these physical switches would use its local Multi-Chassis EtherChannel, i.e. locally attached ports that belong to Multi-Chassis EtherChannel, to deliver unicasts. If all of the Multi-Chassis EtherChannel ports on that switch are down, the frames may be sent to the peer switch over a link such as an Emulated Switch Link (ESL). The peer switch could then use its ports belonging to that Multi-Chassis EtherChannel to forward the frames. 
     Addressing using the emulated switch may be accomplished using a consistent hierarchical address format. This format may be the standard DCE hierarchical address format. This format is depicted in  FIG. 4 . Here, switch ID  400  may be used to store the emulated switch identification. Then, either the subswitch ID field  402  or the local ID field  404  may be used for Multi-Chassis EtherChannel. In either case, the physical DCE switches connected to the emulated switch coordinate so that all ports of the same Multi-Chassis EtherChannel get the same hierarchical address. It should be noted that there could be multiple Multi-Chassis EtherChannels within a single emulated switch-each Multi-Chassis EtherChannel gets the same emulated switch ID. 
     An emulated switch link (ESL) between the edge switches may be used to enable learning across ports belonging to the Multi-Chassis EtherChannel, i.e. to synchronize the MAC table for ports belonging to the Multi-Chassis EtherChannel. The ESL link may also be used to carry data frames belonging to the Multi-Chassis EtherChannel ports during link failures, for exchanging emulated switch control plane messages between the physical switches (LACP protocol frames, MAC learning updates, co-ordination of switch-id, LIDs, etc.), or as a regular data link. 
     It should be noted that the processes described above need not be limited to DCE and non-DCE switch combinations. Technically, the DCE network may be any network of layer 2 switches, while the non-DCE devices may be any network of devices that uses a spanning tree or similar algorithm. For purposes of this document, the term “spanning tree device” will be used to refer to any device that uses a spanning tree or similar algorithm. In one embodiment, this device may be a CE device. 
       FIG. 5  illustrates an example method for configuring a system of devices including a spanning tree device connected to two or more edge switches in a network of layer 2 switches. The spanning tree device may be a Classic Ethernet switch and the network of layer 2 switches may be a DCE network. At  500 , an emulated switch may be created between the spanning tree device and the two or more edge switches. At  502 , each of the two or more edge switches may be configured such that the emulated switch is viewed as part of the network of layer 2 switches, equidistant from each of the two or more edge switches. This may include configuring each of the two or more edge switches to, upon receipt of a unicast frame from another device in the network of layer 2 switches, check to ensure the unicast frame is for the emulated switch, and if so, examine a subswitch or local identification within the unicast frame and forward the frame to the spanning tree device based upon the subswitch identification or local identification. The configuring may also include configuring each of the two or more edge switches to, upon receipt of a broadcast or multicast frame from another device in the network of layer 2 switches, coordinate with each of the other of the two or more edge switches to forward the multicast frame from only one of the edge switches. The particular one of the edge switches may be selected based on a load balancing algorithm. At  504 , data may be periodically synchronized between each of the two or more edge switches. This data may include, for example, a MAC table. This synchronization may occur over an ESL link. 
       FIG. 6  illustrates an example method for forwarding a unicast frame from a device in a network of layer 2 switches to a spanning tree device via one of a plurality of layer 2 switches connected to the spanning tree device. The device may be a layer 2 switch. At  600 , the unicast frame may be encapsulated with a header including a switch identification equal to an emulated switch identification. At  602 , one of the plurality of layer 2 switches connected to the spanning tree device to which to send the unicast frame may be determined. This may be determined based on a shortest path algorithm where an emulated switch having the emulated switch identification is viewed as equidistant from each of the plurality of layer 2 switches connected to the spanning tree device. The shortest path algorithm may recognize links that are inactive or inoperative and determine the shortest path in light of this information. At  604 , the unicast frame may be forwarded to the layer 2 device calculated by the determining. 
       FIG. 7  illustrates an example method for handling a unicast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. The unicast frame is destined for a spanning tree device, wherein the spanning tree device is connected to two or more of the switches in the network of layer 2 switches. At  700 , it may be determined that a switch identification in the unicast frame corresponds to an emulated switch between the switch and the spanning tree device. At  702 , it may be determined if the connection between the switch the spanning tree device is inoperable. If not, then at  704 , a header may be stripped from the unicast frame, the header including the switch identification. Then at  706 , the unicast frame may be forwarded to the spanning tree device. The forwarding may include forwarding the unicast frame based on either the subswitch identification or the local identification in the header. If the connection is inoperable, then at  708 , the unicast frame may be transferred to one of the other of the two or more layer 2 switches for forwarding of the unicast frame to the spanning tree device. This transferring may occur, for example, via an ESL link. 
       FIG. 8  illustrates an example method for handling a multicast or broadcast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. At  800 , the switch may coordinate with each of the other of the two or more switches to determine one switch that will directly forward the frame to the spanning tree device. This coordinating may occur via, for example, an ESL link. At  802 , it may be determined if the switch is the one switch that will directly forward the frame to the spanning tree device. If so, then at  804 , a header may be stripped from the frame. At  806 , the frame may be forwarded to the spanning tree device. 
       FIG. 9  illustrates an example apparatus for configuring a system of devices including a spanning tree device connected to two or more edge switches in a network of layer 2 switches. The spanning tree device may be a Classic Ethernet switch and the network of layer 2 switches may be a DCE network. An emulated switch creator  900  may create an emulated switch between the spanning tree device and the two or more edge switches. An edge switch configurer  902  coupled to the emulated switch creator  900  may configure each of the two or more edge switches such that the emulated switch is viewed as part of the network of layer 2 switches, equidistant from each of the two or more edge switches. This may include configuring each of the two or more edge switches to, upon receipt of a unicast frame from another device in the network of layer 2 switches, check to ensure the unicast frame is for the emulated switch, and if so, examine a subswitch or local identification within the unicast frame and forward the frame to the spanning tree device based upon the subswitch identification or local identification. The configuring may also include configuring each of the two or more edge switches to, upon receipt of a broadcast or multicast frame from another device in the network of layer 2 switches, coordinate with each of the other of the two or more edge switches to forward the multicast frame from only one of the edge switches. The particular one of the edge switches may be selected based on a load balancing algorithm. An edge switch synchronizer  904  coupled to the edge switch configurer  902  may periodically synchronize data between each of the two or more edge switches. This synchronization may occur over an ESL link. 
       FIG. 10  illustrates an example apparatus for forwarding a unicast frame from a device in a network of layer 2 switches to a spanning tree device via one of a plurality of layer 2 switches connected to the spanning tree device. The device may be a layer 2 switch. A unicast frame header encapsulater  1000  may encapsulate the unicast frame with a header including a switch identification equal to an emulated switch identification. A layer 2 switch unicast frame route determiner  1002  coupled to the unicast frame header encapsulater  1000  may determine one of the of the plurality of layer 2 switches connected to the spanning tree device to which to send the unicast frame. This may be determined based on a shortest path algorithm where an emulated switch having the emulated switch identification is viewed as equidistant from each of the plurality of layer 2 switches connected to the spanning tree device. The shortest path algorithm may recognize links that are inactive or inoperative and determine the shortest path in light of this information. A unicast frame forwarder  1004  coupled to the layer 2 switch unicast frame route determiner  1002  may forward the unicast frame to the layer 2 device calculated by the determining. 
       FIG. 11  illustrates an example apparatus for handling a unicast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. The unicast frame may be destined for spanning tree device, wherein the spanning tree device is connected to two or more of the switches in the network of layer 2 switches. An emulated switch identification determiner  1100  may determine that a switch identification in the unicast frame corresponds to an emulated switch between the switch and the spanning tree device. A unicast frame header stripper  1102  coupled to the emulated switch identification determiner  1100  may strip a header from the unicast frame, the header including the switch identification. An inoperable spanning tree device connection determiner  1104  coupled to the unicast frame header stripper  1102  may determine if the connection between the switch the spanning tree device is inoperable. If not, then at a unicast frame spanning tree device forwarder  1106  coupled to the inoperable spanning tree device connection determiner  1104  may forward the unicast frame to the spanning tree device. The forwarding may include forwarding the unicast frame based on either the subswitch identification or the local identification in the header. If the connection is inoperable, then a unicast frame layer 2 switch transferrer  1108  coupled to the inoperable spanning tree device connection determiner  104  may transfer the unicast frame may be transferred to one of the other of the two or more layer 2 switches for forwarding of the unicast frame to the spanning tree device. This transferring may occur, for example, via an ESL link. 
       FIG. 12  illustrates an example apparatus for handling a multicast or broadcast frame received at a switch in a network of layer 2 switches from another switch in the network of layer 2 switches. The frame may be destined for spanning tree device, wherein the spanning tree device is connected to two or more of the switch in the network of layer 2 switches. A frame spanning tree device layer 2 switch coordinator  1200  may coordinate with each of the other of the two or more switches to determine one switch that will directly forward the frame to the spanning tree device. This coordinating may occur via, for example, an ESL link. A spanning tree frame forwarding switch determiner  1202  coupled to the frame spanning tree device layer 2 switch coordinator  1200  may determine if the switch is the one switch that will directly forward the frame to the spanning tree device. If so, then a frame header stripper  1204  coupled to the spanning tree forwarding switch determiner  1202  may strip a header from the frame. A spanning tree device frame forwarder  1206  coupled to the frame header stripper  1204  may forward the frame to the spanning tree device. 
     Although illustrative embodiments and applications of this invention are shown and described herein, many variations and modifications are possible which remain within the concept, scope, and spirit of the invention, and these variations would become clear to those of ordinary skill in the art after perusal of this application. Accordingly, the embodiments described are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.