Apparatus and method for architecturally redundant ethernet

A routing apparatus comprising: 1) a first router coupled to a first plurality of Ethernet links; and 2) a second router coupled to a second plurality of Ethernet links, wherein selected ones of the first plurality of Ethernet links are coupled to selected ones of the second plurality of Ethernet links to thereby form Ethernet trunk groups in which traffic associated with a plurality of Ethernet ports are aggregated into a single logical port. The routing apparatus further comprises a first high-speed link and a second high-speed link directly coupling the first router and the second router and forming a self-healing ring for transferring data packets between the first and second routers. In response to a failure associated with the failing one of the first and second routers, the first and second high-speed links transfer data traffic from the failing router to the non-failing router.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to data communication networks and, more particularly, to Ethernet networks.

BACKGROUND OF THE INVENTION

In Ethernet systems, each Ethernet interface has its own medium access control (MAC) address, which is used as the source address for frames sent from the interface and is the destination address to which frames for the interface are sent. Typically, Ethernet LANs are inter-connected via hubs or switches. Hubs and switches do not translate MAC addresses, so all frames with the destination address of the interface must go through the designated MAC device, thus creating a single point of failure.

The IEEE 802.3-2002 Standard defines Link Aggregation Groups to provide a larger aggregated bandwidth, load sharing, and link redundancy. These groups can be used to provide link redundancy, but still use a single MAC device with a single aggregator MAC address. Thus, there is still a single point of failure at the MAC device, so complete Ethernet redundancy is not provided.

Ethernet LANs are interconnected using bridges. One approach to Ethernet redundancy is to use a modified bridge that is capable of translating MAC addresses upon detection of an interface failure. There are some problems with this approach. First, Ethernet bridge protocols do not support this kind of failure processing. Second, an Ethernet bridge does not have enough fidelity in its failure detection to know precisely what failed and how to fix the failure. An Ethernet bridge only knows that the MAC address is no longer reachable through the port. Ethernet bridges may be reconfigured through spanning tree protocols to find a new path, but Ethernet bridges do not support reconfiguring interfaces for MAC address translation.

Another approach to avoiding the aforementioned single point of failure is to swap addresses in the MAC chip, thus having a different MAC chip serve the interface. This can lead to some problems during switch-over. If the second port is enabled before the first port is disabled, hubs and switches tend to lock up with protocol violations due to having two ports with the same MAC address. Ethernet protocols do not support removing the first link from the tables when the link failure occurs. Instead, Ethernet protocols must wait for the path to time out. These time-outs can be lengthy, thus leading to a significant amount of data loss.

Also, there is a potential for looping to occur when there are duplicate MAC addresses. Looping is a problem in Ethernet bridges, since packets get replicated on all interfaces. If there is a loop, the replication may repeat until all bandwidth is consumed. Spanning Tree Protocol (STP) and its rapid reconfiguration descendant, Rapid Spanning Tree Protocol (RSTP), were developed to eliminate loops. These protocols use a subset of the physical interconnections to form a tree spanning the entire network without loops. The protocols eliminate duplicate paths, so a hot second path tends to be eliminated by spanning tree protocols.

When security software sees duplicate MAC addresses, the security software sees this as a penetration by an unauthorized user. Thus, the approach of changing MAC addresses can also lead to security alerts.

Therefore, there is a need in the art for improved Ethernet redundancy. In particular, there is a need for an effective way to overcome single point MAC device failures.

SUMMARY OF THE INVENTION

The present invention provides redundancy at the Ethernet architectural level to provide Ethernet link and interface redundancy while avoiding single point MAC device failures. Advantageously, the present invention achieves this redundancy using conventional “pizza box” routers in a redundant architecture.

To address the above-discussed deficiencies of the prior art, it is a primary object of the present invention to provide an improved redundant routing apparatus. According to an advantageous embodiment of the present invention, the routing apparatus comprises: 1) a first router capable of being coupled to a first plurality of Ethernet links; and 2) a second router capable of being coupled to a second plurality of Ethernet links, wherein selected ones of the first plurality of Ethernet links are coupled to selected ones of the second plurality of Ethernet links to thereby form Ethernet trunk groups in which traffic associated with a plurality of Ethernet ports are aggregated into a single logical port.

According to one embodiment of the present invention, a first selected one of the first plurality of Ethernet links is coupled to a first selected one of the second plurality of Ethernet links to thereby form a first trunk group.

According to another embodiment of the present invention, the first selected one of the first plurality of Ethernet links and the first selected one of the second plurality of Ethernet links are capable of carrying the first trunk group traffic simultaneously in a load-sharing manner.

According to still another embodiment of the present invention, a subset of the first plurality of Ethernet links and a subset of the second plurality of Ethernet links are each capable of carrying all of the first trunk group traffic.

According to yet another embodiment of the present invention, the routing apparatus further comprises a first high-speed link coupling the first router and the second router for transferring data packets between the first and second routers.

According to a further embodiment of the present invention, the routing apparatus further comprises a second high-speed link coupling the first router and the second router for transferring data traffic between the first and second routers.

According to a still further embodiment of the present invention, the first and second high-speed links form a self-healing ring with the first router and the second router.

According to a yet further embodiment of the present invention, the first and second high-speed links transfer data traffic from a failing one of the first router and the second router to a non-failing one of the first router and the second router in response to a failure associated with the failing one of the first and second routers.

In one embodiment of the present invention, the failure is associated with one of: 1) the first plurality of Ethernet links, 2) the second plurality of Ethernet links, 3) an interface coupled to a link in the first plurality of Ethernet links; and 4) an interface coupled to a link in the second plurality of Ethernet links.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1illustrates exemplary prior art Ethernet network100. Prior art Ethernet network100comprises local area network (LAN)110, Ethernet switch111, links112, router113and other network(s)114. Router113is coupled to Ethernet switch111, which is in turn coupled to LAN110. Router113is also connected to another data communication network (or networks)114. Each link in links112between Ethernet switch111and router113is connected in an Ethernet trunk group, wherein multiple ports are aggregated into a single logical port. Such trunk groups are described in the IEEE 802.3-2002 standard, which refers to such trunk groups as “Link Aggregation Groups”.

Each interface of Ethernet switch111that is associated with one of links112has a MAC address, but the Ethernet switch uses a single, logical MAC address for all interfaces associated with the link aggregation group. This is the MAC address assigned to the aggregator, which may be one of the MAC addresses of a component interface or may be a separate MAC address assigned to the aggregator. In this type of arrangement, a failure in any one of links112would result in the rest of links112carrying the traffic with the available remaining bandwidth. However, the traffic on all of links112flows through a single MAC device doing the link aggregation. Failure of this MAC device would be an example of a single point failure at the MAC device, as discussed generally above.

FIG. 2illustrates exemplary Ethernet data network200, which contains Ethernet switch routers210according to the principles of the present invention. Architectural redundancy is provided in order to overcome single point failures at the MAC device. Network200comprises Ethernet switch router210a, links211, links212, link213, local area network (LAN)240, local area network (LAN)250, and local area network (LAN)260. In the exemplary embodiment, each link in links211and links212, and link213carry data at a rate of 1 GBps (gigabit per second).

Exemplary Ethernet switch/router (ESR)210acomprises switch/router220a, switch/router220b, and switch/router220c. Each one of switch/routers220a-cis a “pizza box” type router, so called because the size and shape of router is approximately that of a pizza box. Switch/routers220a,220band220care connected in self-healing rings by link221, link222, and link223. In the exemplary embodiment, links221,222and223are HiGig interfaces that carry data at a rate of 12 GBps.

Switch/router220ais coupled to links211and switch/router220bis coupled to links212. Sets of links in links211and links212form trunk groups (or link aggregation groups according to in the IEEE 802.3-2002 standard). For example, link211a, which is coupled to switch/router220a, and link212b, which is coupled to switch/router220b, form trunk group214a, indicates by a dotted line loop. Similarly, link211band link212bform trunk group214b. Finally, link211c, link212c, and link213, which is coupled to switch/router220c, form trunk group214c.

Switch router220aand switch router220bcarry traffic in a load-sharing manner on links211aand212ain trunk group214auntil something fails (e.g., a link to a pizza box router, one of the pizza box router interfaces, or a pizza box router itself) on one of the two paths. If this occurs, then all of the traffic flows over the remaining good path.

In the exemplary embodiment shown inFIG. 2, it is assumed that ESR210bin LAN240, ESR210cin LAN250, and ESR210din LAN260are similar to ESR210and support this type of redundancy. However, if ESR210b, ESR210cand ESR210dare not the same as ESR210a, then it is assumed that ESR210b, ESR210cand ESR210dsupport IEEE 802.3 link aggregation at the expense of a single point of failure at the MAC device at ESR210b, ESR210cand ESR210dwhile still avoiding single point failures on the WAN links and in ESR210a. It is assumed that the WAN interfaces of LAN240, LAN250and LAN260are protected by redundancy, since each affects many users, whereas the connections within each LAN may not affect as many simultaneous users.

Two or more links in an Ethernet trunk group may be used simultaneously in a load sharing manner to handle the traffic for the associated Ethernet trunk group. For example, if one interface or link of a redundant trunk group pair fails, then all traffic for that client (i.e., that trunk group) will be carried by the remaining redundant interface or link. Links221-223may be used to get traffic from working interfaces and links of switch/routers220a,220b, or220cwith the failed interface to the other switch/router in which the paired interface is working.

For example, if link211afails, but link212acontinues to work, then packets coming into switch/router220aon link211bthat are destined for failed link211amay be transferred from switch/router220aacross link221to switch/router220band sent out link212a. If switch/router220afails, then the remote end could sense the failure and direct all traffic to the remaining good switch/router220b. The remote end senses the lost connection with far end of a link and sends all data over the remaining good link.

If entire switch/router220afails, there may not be enough function components left to send data over links221-223to the other switch/router. However, if only portions of a switch router fail, then links221-223may be used to route data between the router220awith the failed interface or link and the router with the good interface and link, such as router220bor220c. If one of links221,222or223fails, then the other interface is available to carry traffic between the switch/routers as may be necessary. For example, if link221or its interface in switch/router220afails, switch/router220amay still send data to switch/router220bin two steps via links223and link222.

Considering now the particular example of the redundant pair of links211aand212a. If link211afails, then traffic for that client can be carried via links212aand switch/router220b. Similarly, if link212afails, then traffic for that client can be carried via interface211aand switch/router220a.

Considering further exemplary failure scenarios, if switch/router220afails, then all traffic received by switch/router220amay be forwarded to switch/router220bvia one of the redundant links221or223and222. Switch/router220bthen forwards the traffic on to other network(s). This is true if there is a partial failure, so that some of the interfaces of failed switch/router220aor220bstill work. In case of a total failure of switch/router220aor220b, the remote end can recognize the failure and send all traffic to the good switch/router. If link221fails and some interfaces of switch/router220bfail, then redundant links223and222are still available to forward traffic for failed link221between switch/router220band switch/router220a.

FIG. 3illustrates Ethernet switch/router (ESR)310according an alternate embodiment of the present invention. ESR310comprises switch/router301-303, links311-313, links321-323, switch/routers331and332, links341and342, and links351and352. In the arrangement ofFIG. 3, switch/routers301,302and303provide three-way redundancy for Ethernet trunk groups or link aggregation groups formed from links coupled to switch/routers301-303, such as links321-323. By way of example, links321-323form trunk group325, indicated by a dotted line loop. Similarly, switch/routers331and332provide two-way redundancy for Ethernet trunk groups or link aggregation groups formed from links coupled to switch/routers331and332, such as links351and352. By way of example, links351and352form trunk group355, indicated by a dotted line loop.

Links321-323may handle the traffic simultaneously in load-sharing fashion, but any one or two of them can handle all of the traffic, if necessary. Link311interconnects switch/routers301and302, link312interconnects switch/routers302and303, and link313interconnects switch/routers301and303. In the three-way redundancy arrangement provided by switch/routers301,302and303, switch/router301is linked to switch/router303both by link313, and by links311and312in combination with switch/router302. Links351and352may handle the traffic simultaneously in load-sharing fashion, but either one can handle all of the traffic, if necessary. Links341and342interconnect switch/routers331and332.

In some embodiments, each of the switch/routers301-303,331and332can receive traffic from any of the ports of a given trunk group. The switch/routers can also send traffic to any port of a given trunk group using, for example, a suitable conventional software selection algorithm. The switch/routers can maintain packet sequencing using any suitable packet sequencing method, for example, the method described in co-pending U.S. patent application Ser. No. 10/655,149, entitled “APPARATUS AND METHOD FOR MAINTAINING PACKET SEQUENCING IN A PARALLEL ROUTER”, which is incorporated herein by reference.

Also, the methods employed by IEEE 802.3-2002 for maintaining packet sequencing over Link Aggregation Groups can be employed. This is the preferred method if one end of the Link Aggregation Group is a standard 802.3-2002 Ethernet Switch supporting Link Aggregation Groups. This end of the link aggregation loop will be subject to single point failures at the MAC device, while the ESR end will be immune from the single point failures at the MAC device.

Although the present invention has been described with an exemplary embodiment, various changes and modifications may be suggested to one skilled in the art. It is intended that the present invention encompass such changes and modifications as fall within the scope of the appended claims.