Abstract:
A routing apparatus comprising: 1) a first router coupled to a first plurality of Ethernet links; and 2) a second router coupled to a second plurality of Ethernet links, wherein selected ones of the first plurality of Ethernet links are coupled to selected ones of the second plurality of Ethernet links to thereby form Ethernet trunk groups in which traffic associated with a plurality of Ethernet ports are aggregated into a single logical port. The routing apparatus further comprises a first high-speed link and a second high-speed link directly coupling the first router and the second router and forming a self-healing ring for transferring data packets between the first and second routers. In response to a failure associated with the failing one of the first and second routers, the first and second high-speed links transfer data traffic from the failing router to the non-failing router.

Description:
TECHNICAL FIELD OF THE INVENTION  
       [0001]     The present invention relates to data communication networks and, more particularly, to Ethernet networks.  
       BACKGROUND OF THE INVENTION  
       [0002]     In Ethernet systems, each Ethernet interface has its own medium access control (MAC) address, which is used as the source address for frames sent from the interface and is the destination address to which frames for the interface are sent. Typically, Ethernet LANs are inter-connected via hubs or switches. Hubs and switches do not translate MAC addresses, so all frames with the destination address of the interface must go through the designated MAC device, thus creating a single point of failure.  
         [0003]     The IEEE 802.3-2002 Standard defines Link Aggregation Groups to provide a larger aggregated bandwidth, load sharing, and link redundancy. These groups can be used to provide link redundancy, but still use a single MAC device with a single aggregator MAC address. Thus, there is still a single point of failure at the MAC device, so complete Ethernet redundancy is not provided.  
         [0004]     Ethernet LANs are interconnected using bridges. One approach to Ethernet redundancy is to use a modified bridge that is capable of translating MAC addresses upon detection of an interface failure. There are some problems with this approach. First, Ethernet bridge protocols do not support this kind of failure processing. Second, an Ethernet bridge does not have enough fidelity in its failure detection to know precisely what failed and how to fix the failure. An Ethernet bridge only knows that the MAC address is no longer reachable through the port. Ethernet bridges may be reconfigured through spanning tree protocols to find a new path, but Ethernet bridges do not support reconfiguring interfaces for MAC address translation.  
         [0005]     Another approach to avoiding the aforementioned single point of failure is to swap addresses in the MAC chip, thus having a different MAC chip serve the interface. This can lead to some problems during switch-over. If the second port is enabled before the first port is disabled, hubs and switches tend to lock up with protocol violations due to having two ports with the same MAC address. Ethernet protocols do not support removing the first link from the tables when the link failure occurs. Instead, Ethernet protocols must wait for the path to time out. These time-outs can be lengthy, thus leading to a significant amount of data loss.  
         [0006]     Also, there is a potential for looping to occur when there are duplicate MAC addresses. Looping is a problem in Ethernet bridges, since packets get replicated on all interfaces. If there is a loop, the replication may repeat until all bandwidth is consumed. Spanning Tree Protocol (STP) and its rapid reconfiguration descendant, Rapid Spanning Tree Protocol (RSTP), were developed to eliminate loops. These protocols use a subset of the physical interconnections to form a tree spanning the entire network without loops. The protocols eliminate duplicate paths, so a hot second path tends to be eliminated by spanning tree protocols.  
         [0007]     When security software sees duplicate MAC addresses, the security software sees this as a penetration by an unauthorized user. Thus, the approach of changing MAC addresses can also lead to security alerts.  
         [0008]     Therefore, there is a need in the art for improved Ethernet redundancy. In particular, there is a need for an effective way to overcome single point MAC device failures.  
       SUMMARY OF THE INVENTION  
       [0009]     The present invention provides redundancy at the Ethernet architectural level to provide Ethernet link and interface redundancy while avoiding single point MAC device failures. Advantageously, the present invention achieves this redundancy using conventional “pizza box” routers in a redundant architecture.  
         [0010]     To address the above-discussed deficiencies of the prior art, it is a primary object of the present invention to provide an improved redundant routing apparatus. According to an advantageous embodiment of the present invention, the routing apparatus comprises: 1) a first router capable of being coupled to a first plurality of Ethernet links; and 2) a second router capable of being coupled to a second plurality of Ethernet links, wherein selected ones of the first plurality of Ethernet links are coupled to selected ones of the second plurality of Ethernet links to thereby form Ethernet trunk groups in which traffic associated with a plurality of Ethernet ports are aggregated into a single logical port.  
         [0011]     According to one embodiment of the present invention, a first selected one of the first plurality of Ethernet links is coupled to a first selected one of the second plurality of Ethernet links to thereby form a first trunk group.  
         [0012]     According to another embodiment of the present invention, the first selected one of the first plurality of Ethernet links and the first selected one of the second plurality of Ethernet links are capable of carrying the first trunk group traffic simultaneously in a load-sharing manner.  
         [0013]     According to still another embodiment of the present invention, a subset of the first plurality of Ethernet links and a subset of the second plurality of Ethernet links are each capable of carrying all of the first trunk group traffic.  
         [0014]     According to yet another embodiment of the present invention, the routing apparatus further comprises a first high-speed link coupling the first router and the second router for transferring data packets between the first and second routers.  
         [0015]     According to a further embodiment of the present invention, the routing apparatus further comprises a second high-speed link coupling the first router and the second router for transferring data traffic between the first and second routers.  
         [0016]     According to a still further embodiment of the present invention, the first and second high-speed links form a self-healing ring with the first router and the second router.  
         [0017]     According to a yet further embodiment of the present invention, the first and second high-speed links transfer data traffic from a failing one of the first router and the second router to a non-failing one of the first router and the second router in response to a failure associated with the failing one of the first and second routers.  
         [0018]     In one embodiment of the present invention, the failure is associated with one of: 1) the first plurality of Ethernet links, 2) the second plurality of Ethernet links, 3) an interface coupled to a link in the first plurality of Ethernet links; and 4) an interface coupled to a link in the second plurality of Ethernet links.  
         [0019]     Before undertaking the DETAILED DESCRIPTION OF THE INVENTION below, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or,” is inclusive, meaning and/or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, such a device may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0020]     For a more complete understanding of the present invention and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:  
         [0021]      FIG. 1  illustrates an exemplary prior art Ethernet data network;  
         [0022]      FIG. 2  illustrates an exemplary Ethernet data network that contains Ethernet switch routers according to the principles of the present invention; and  
         [0023]      FIG. 3  illustrates an alternate embodiment of an Ethernet switch router according to the principles of the present invention.  
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0024]      FIGS. 1 through 3 , discussed below, and the various embodiments used to describe the principles of the present invention in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the invention. Those skilled in the art will understand that the principles of the present invention may be implemented in any suitably arranged data communication system.  
         [0025]      FIG. 1  illustrates exemplary prior art Ethernet network  100 . Prior art Ethernet network  100  comprises local area network (LAN)  110 , Ethernet switch  111 , links  112 , router  113  and other network(s)  114 . Router  113  is coupled to Ethernet switch  111 , which is in turn coupled to LAN  110 . Router  113  is also connected to another data communication network (or networks)  114 . Each link in links  112  between Ethernet switch  111  and router  113  is connected in an Ethernet trunk group, wherein multiple ports are aggregated into a single logical port. Such trunk groups are described in the IEEE 802.3-2002 standard, which refers to such trunk groups as “Link Aggregation Groups”.  
         [0026]     Each interface of Ethernet switch  111  that is associated with one of links  112  has a MAC address, but the Ethernet switch uses a single, logical MAC address for all interfaces associated with the link aggregation group. This is the MAC address assigned to the aggregator, which may be one of the MAC addresses of a component interface or may be a separate MAC address assigned to the aggregator. In this type of arrangement, a failure in any one of links  112  would result in the rest of links  112  carrying the traffic with the available remaining bandwidth. However, the traffic on all of links  112  flows through a single MAC device doing the link aggregation. Failure of this MAC device would be an example of a single point failure at the MAC device, as discussed generally above.  
         [0027]      FIG. 2  illustrates exemplary Ethernet data network  200 , which contains Ethernet switch routers  210  according to the principles of the present invention. Architectural redundancy is provided in order to overcome single point failures at the MAC device. Network  200  comprises Ethernet switch router  210   a , links  211 , links  212 , link  213 , local area network (LAN)  240 , local area network (LAN)  250 , and local area network (LAN)  260 . In the exemplary embodiment, each link in links  211  and links  212 , and link  213  carry data at a rate of 1 GBps (gigabit per second).  
         [0028]     Exemplary Ethernet switch/router (ESR)  210   a  comprises switch/router  220   a , switch/router  220   b , and switch/router  220   c . Each one of switch/routers  220   a - c  is a “pizza box” type router, so called because the size and shape of router is approximately that of a pizza box. Switch/routers  220   a ,  220   b  and  220   c  are connected in self-healing rings by link  221 , link  222 , and link  223 . In the exemplary embodiment, links  221 ,  222  and  223  are HiGig interfaces that carry data at a rate of 12 GBps.  
         [0029]     Switch/router  220   a  is coupled to links  211  and switch/router  220   b  is coupled to links  212 . Sets of links in links  211  and links  212  form trunk groups (or link aggregation groups according to in the IEEE 802.3-2002 standard). For example, link  211   a , which is coupled to switch/router  220   a , and link  212   b , which is coupled to switch/router  220   b , form trunk group  214   a , indicates by a dotted line loop. Similarly, link  211   b  and link  212   b  form trunk group  214   b . Finally, link  211   c , link  212   c , and link  213 , which is coupled to switch/router  220   c , form trunk group  214   c.    
         [0030]     Links  211   a  and  212   a  of trunk group  214   a  are coupled to Ethernet switch/router (ESR)  210   b  in LAN  240 . Links  211   b  and  212   b  of trunk group  214   b  are coupled to Ethernet switch/router (ESR)  210   c  in LAN  250 . Finally, links  211   c ,  212   c  and  213  of trunk group  214   c  are coupled to Ethernet switch/router (ESR)  210   d  in LAN  260 .  
         [0031]     Switch router  220   a  and switch router  220   b  carry traffic in a load-sharing manner on links  211   a  and  212   a  in trunk group  214   a  until something fails (e.g., a link to a pizza box router, one of the pizza box router interfaces, or a pizza box router itself) on one of the two paths. If this occurs, then all of the traffic flows over the remaining good path.  
         [0032]     In the exemplary embodiment shown in  FIG. 2 , it is assumed that ESR  210   b  in LAN  240 , ESR  210   c  in LAN  250 , and ESR  210   d  in LAN  260  are similar to ESR  210  and support this type of redundancy. However, if ESR  210   b , ESR  210   c  and ESR  210   d  are not the same as ESR  210   a , then it is assumed that ESR  210   b , ESR  210   c  and ESR  210   d  support IEEE 802.3 link aggregation at the expense of a single point of failure at the MAC device at ESR  210   b , ESR  210   c  and ESR  210   d  while still avoiding single point failures on the WAN links and in ESR  210   a . It is assumed that the WAN interfaces of LAN  240 , LAN  250  and LAN  260  are protected by redundancy, since each affects many users, whereas the connections within each LAN may not affect as many simultaneous users.  
         [0033]     Two or more links in an Ethernet trunk group may be used simultaneously in a load sharing manner to handle the traffic for the associated Ethernet trunk group. For example, if one interface or link of a redundant trunk group pair fails, then all traffic for that client (i.e., that trunk group) will be carried by the remaining redundant interface or link. Links  221 - 223  may be used to get traffic from working interfaces and links of switch/routers  220   a ,  220   b , or  220   c  with the failed interface to the other switch/router in which the paired interface is working.  
         [0034]     For example, if link  211   a  fails, but link  212   a  continues to work, then packets coming into switch/router  220   a  on link  211   b  that are destined for failed link  211   a  may be transferred from switch/router  220   a  across link  221  to switch/router  220   b  and sent out link  212   a . If switch/router  220   a  fails, then the remote end could sense the failure and direct all traffic to the remaining good switch/router  220   b . The remote end senses the lost connection with far end of a link and sends all data over the remaining good link.  
         [0035]     If entire switch/router  220   a  fails, there may not be enough function components left to send data over links  221 - 223  to the other switch/router. However, if only portions of a switch router fail, then links  221 - 223  may be used to route data between the router  220   a  with the failed interface or link and the router with the good interface and link, such as router  220   b  or  220   c . If one of links  221 ,  222  or  223  fails, then the other interface is available to carry traffic between the switch/routers as may be necessary. For example, if link  221  or its interface in switch/router  220   a  fails, switch/router  220   a  may still send data to switch/router  220   b  in two steps via links  223  and link  222 .  
         [0036]     Considering now the particular example of the redundant pair of links  211   a  and  212   a . If link  211   a  fails, then traffic for that client can be carried via links  212   a  and switch/router  220   b . Similarly, if link  212   a  fails, then traffic for that client can be carried via interface  211   a  and switch/router  220   a.    
         [0037]     Considering further exemplary failure scenarios, if switch/router  220   a  fails, then all traffic received by switch/router  220   a  may be forwarded to switch/router  220   b  via one of the redundant links  221  or  223  and  222 . Switch/router  220   b  then forwards the traffic on to other network(s). This is true if there is a partial failure, so that some of the interfaces of failed switch/router  220   a  or  220   b  still work. In case of a total failure of switch/router  220   a  or  220   b , the remote end can recognize the failure and send all traffic to the good switch/router. If link  221  fails and some interfaces of switch/router  220   b  fail, then redundant links  223  and  222  are still available to forward traffic for failed link  221  between switch/router  220   b  and switch/router  220   a.    
         [0038]      FIG. 3  illustrates Ethernet switch/router (ESR)  310  according an alternate embodiment of the present invention. ESR  310  comprises switch/router  301 - 303 , links  311 - 313 , links  321 - 323 , switch/routers  331  and  332 , links  341  and  342 , and links  351  and  352 . In the arrangement of  FIG. 3 , switch/routers  301 ,  302  and  303  provide three-way redundancy for Ethernet trunk groups or link aggregation groups formed from links coupled to switch/routers  301 - 303 , such as links  321 - 323 . By way of example, links  321 - 323  form trunk group  325 , indicated by a dotted line loop. Similarly, switch/routers  331  and  332  provide two-way redundancy for Ethernet trunk groups or link aggregation groups formed from links coupled to switch/routers  331  and  332 , such as links  351  and  352 . By way of example, links  351  and  352  form trunk group  355 , indicated by a dotted line loop.  
         [0039]     Links  321 - 323  may handle the traffic simultaneously in load-sharing fashion, but any one or two of them can handle all of the traffic, if necessary. Link  311  interconnects switch/routers  301  and  302 , link  312  interconnects switch/routers  302  and  303 , and link  313  interconnects switch/routers  301  and  303 . In the three-way redundancy arrangement provided by switch/routers  301 ,  302  and  303 , switch/router  301  is linked to switch/router  303  both by link  313 , and by links  311  and  312  in combination with switch/router  302 . Links  351  and  352  may handle the traffic simultaneously in load-sharing fashion, but either one can handle all of the traffic, if necessary. Links  341  and  342  interconnect switch/routers  331  and  332 .  
         [0040]     In some embodiments, each of the switch/routers  301 - 303 ,  331  and  332  can receive traffic from any of the ports of a given trunk group. The switch/routers can also send traffic to any port of a given trunk group using, for example, a suitable conventional software selection algorithm. The switch/routers can maintain packet sequencing using any suitable packet sequencing method, for example, the method described in co-pending U.S. patent application Ser. No. 10/655,149, entitled “APPARATUS AND METHOD FOR MAINTAINING PACKET SEQUENCING IN A PARALLEL ROUTER”, which is incorporated herein by reference.  
         [0041]     Also, the methods employed by IEEE 802.3-2002 for maintaining packet sequencing over Link Aggregation Groups can be employed. This is the preferred method if one end of the Link Aggregation Group is a standard 802.3-2002 Ethernet Switch supporting Link Aggregation Groups. This end of the link aggregation loop will be subject to single point failures at the MAC device, while the ESR end will be immune from the single point failures at the MAC device.  
         [0042]     Although the present invention has been described with an exemplary embodiment, various changes and modifications may be suggested to one skilled in the art. It is intended that the present invention encompass such changes and modifications as fall within the scope of the appended claims.