Abstract:
Methods and systems for aggregating Ethernet communications are disclosed. A disclosed apparatus includes a first Ethernet port to communicate with a second Ethernet port of a first device, a third Ethernet port to communicate with a fourth Ethernet port of a second device, a fifth Ethernet port to receive Ethernet frames, and a switching portion to direct nth ones of the frames to a first queue associated with the second port, direct n−1 frames preceding each of the nth ones of the frames to a second queue associated with the fourth port, and select a value of n based on a ratio of a first non-zero data rate of the first device for a first communication link in a first direction and a second non-zero data rate of the second device for a second communication link in the first direction, and based on a remaining capacity of the first queue.

Description:
RELATED APPLICATION 
     This patent arises from a continuation of U.S. patent application Ser. No. 10/636,366 filed Aug. 7, 2003 (now U.S. Pat. No. 7,630,412), entitled “Methods and Systems for Aggregating Ethernet Communications,” which was filed on Aug. 7, 2003, and which is hereby incorporated by reference in its entirety. 
    
    
     FIELD OF THE DISCLOSURE 
     The present invention is related to Ethernet communications. More particularly, the present invention is related to the aggregation of Ethernet communications. 
     BACKGROUND 
     Ethernet communications are a popular data network protocol used to carry frames of data. Ethernet communications are often used within local area networks to carry data between nearby computers as well as within wide area networks where the Ethernet may be carried through lower layer connections such as an asynchronous transfer mode (“ATM”) link. Ethernet communications allow for the transfer between end points of higher layer data packets such as Internet Protocol packets that are destined for a remote location such as a domain of the global Internet. Ethernet communications may be of various types, each with its own data rate, and the Ethernet communications may have a same data rate in both the upload and download directions of data transfer for an Ethernet capable device. 
     The local network or device of a subscriber utilizing Ethernet communications may be provided access to an Ethernet network over a separate communication device that transfers Ethernet frames of data by using the lower layers of the communications protocol, such as ATM transferred over a cable or DSL modem. Thus, the subscriber is provided a cable or DSL connectivity that carries Ethernet frames out of the local network or device. However, the upload and download speeds of the cable or DSL connectivity are typically less than that of the Ethernet communication with these communication devices, and the subscriber is ultimately limited to the bandwidth available through the cable or DSL modem. 
     Other options for achieving higher bandwidths are available such as Ethernet service directly to the subscriber, but Ethernet service often requires the addition of fiber in the plant between the service provider and the subscriber. Because the copper plant is already in place in most instances, it is more economical to provide data transfer between the subscriber and the service provider network over the existing copper lines. However, the bandwidth limitations for data services using copper lines such as DSL and cable continue to limit the available bandwidth. 
     While there is discussion and attempts at physical layer bonding, such as the physical layer bonding of DSL modems, this physical layer bonding does not allow for aggregation between a DSL line or similar transport and an existing Ethernet service. Furthermore, intermediate layer aggregation, such as aggregation at the ATM IMA level requires specialized equipment and also suffers from the same limitations as the physical layer bonding attempted for DSL modems. 
     SUMMARY 
     Embodiments of the present invention address these issues and others by providing aggregation of Ethernet communications so that multiple communication devices that utilize Ethernet communications may be used to provide data transfer between endpoints, such as between a subscriber and an Ethernet network. Accordingly, one endpoint may have the available bandwidth of multiple communication devices as opposed to the bandwidth of a single communication device. 
     One embodiment is a method of aggregating Ethernet communications. The method involves providing a first communication device having an Ethernet port and a communication port and providing a second communication device having an Ethernet port and a communication port. The method further involves providing an Ethernet switch in data communication with the Ethernet ports of the first communication device and the second communication device. The Ethernet switch includes an Ethernet port such that data communications of the Ethernet port of the Ethernet switch are provided through the communication port of the first communication device and the communication port of the second communication device. 
     Another embodiment is a system for aggregating Ethernet communications. The system includes a first communication device having an Ethernet port, a second communication device having an Ethernet port, and a network device having an Ethernet port. An Ethernet switch has a first Ethernet port in communication with the Ethernet port of the first communication device and has a second Ethernet port in communication with the Ethernet port of the second communication device. The Ethernet switch aggregates communications with the first communication device and the second communication device to provide communication through a third Ethernet port in communication with the network device. 
     Another embodiment is a system for aggregating Ethernet communications. The system includes a first communication device having an Ethernet port and a communications port and a second communication device having an Ethernet port and a communications port. The system further includes a third communication device having an Ethernet port and a communications port, and the communications port of the third communication device is in data communication with the communications port of the first communication device. The system also includes a fourth communication device having an Ethernet port and a communications port, and the communications port of the fourth communication device is in data communication with the communications port of the second communication device. A first Ethernet switch has a first Ethernet port in communication with the Ethernet port of the first communication device and has a second Ethernet port in communication with the Ethernet port of the second communication device. The first Ethernet switch aggregates communications with the first communication device and the second communication device. A second Ethernet switch has a first Ethernet port in communication with the Ethernet port of the third communication device and has a second Ethernet port in communication with the Ethernet port of the fourth communication device. The second Ethernet switch aggregates communications with the third communication device and the fourth communication device. 
     Another embodiment is an Ethernet aggregation device that includes an Ethernet switching portion linked to an Ethernet port. The device also includes a first communications portion corresponding to a first virtual Ethernet port and is linked to a first communications port such that communications carrying Ethernet frames of the first virtual Ethernet port are transferred through the first communications port. A second communications portion corresponds to a second virtual Ethernet port and is linked to a second communications port such that communications carrying Ethernet frames of the second virtual Ethernet port are transferred through the second communications port. The Ethernet switching portion aggregates Ethernet communications of the Ethernet port between the first and second virtual Ethernet ports. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a high level view of data service from a service provider network to the subscriber that may utilize embodiments of the present invention to aggregate Ethernet links. 
         FIG. 2  is a diagram of the connectivity of devices according to an embodiment of the present invention to aggregate Ethernet links. 
         FIG. 3  is a diagram of the connectivity of a symmetric DSL configuration utilizing multiple asymmetric DSL modems per service endpoint, which may utilize embodiments of the present invention to aggregate the multiple asymmetric DSL modems. 
         FIG. 4  is a diagram of an Ethernet aggregation device according to an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present invention provide for aggregated Ethernet communications thereby allowing multiple lower-speed communication links between one service end point and another, which increases the available bandwidth at a service endpoint over using a single lower-speed communication link. Therefore, subscribers of DSL or cable data connectivity to an Ethernet network may be provided with a higher bandwidth service than would otherwise be available. 
       FIG. 1  shows a high level view of an Ethernet service being provided from an Ethernet network  102  to a network device  114  of a subscriber. A service provider maintains the connectivity between the Ethernet network  102  and the network device  114  of the subscriber. The service provider, or another service provider, may also provide access to the public switched telephone network (“PSTN”)  104  for subscribers. As shown in  FIG. 1 , the Ethernet network access is distributed to the subscriber from the Ethernet network  102  in various ways where Ethernet communication is carried between the Ethernet network  102  and the subscriber over one of several available transports. 
     The Ethernet communications between the Ethernet network  102  and the network device  114  of the subscriber are aggregated at an Ethernet switch  106  of the service provider network  102  so that multiple communication devices may be used to transfer the data between the subscriber and the Ethernet network  102 . For example, a set of communication devices  108  such as a set of cable modems or a set of DSL modems contained within a DSL access multiplexer (“DSLAM”) are aggregated at the Ethernet switch  106  and are used to transport the higher level Ethernet data communications between a central office (“CO”) of the service provider and out to the location of the subscriber. The data communications are received at another set of communication devices  110  such as a set of cable or DSL modems. The data communications are then converted back to Ethernet level communications received by another Ethernet switch  112  that aggregates the multiple communication devices  110 . The Ethernet switch  112  provides communication with the network device  114  of the subscriber. 
     The Ethernet communications may be provided to the subscriber as encapsulated in the lower level of communications such as through a DSL or cable modem service such that the multiple communication devices  110  reside at the subscriber location along with the Ethernet switch  112 . As an alternative, the Ethernet communications may be provided directly as a service to the subscriber such as by including the multiple communication devices  110  within the service provider network along with the Ethernet switch  112 . In this alternative, the Ethernet connection extends from the Ethernet switch  112  of the service provider network into the location of the subscriber where the network device  114  may connect to the Ethernet service. 
     The Ethernet communications may be either symmetric or asymmetric, depending upon the configuration of the sets of communication devices  108  and  110 . For example, the communications may be made asymmetric by having asymmetric DSL (“ADSL”) modems  108  and  110  providing the asymmetry between the upload and download data transfer rates. The upload speed may be faster than the download speed or the download speed may be faster than the upload speed. For example, ADSL modems  108  may be type ATU-C and ADSL modems  110  may be ATU-R so that the download for the subscriber is faster than the upload. Alternatively, the ADSL modems  108  may be type ATU-R while ADSL modems  110  may be ATU-C so that the upload for the subscriber is faster than the download. Furthermore, as discussed with reference to  FIG. 3 , one ATU-R modem  304  and one ATU-C modem  306  may be aggregated at one end point and one ATU-R modem  310  and one ATU-C modem  308  may be aggregated at the other endpoint to achieve a symmetric data service. 
     An example system for aggregating Ethernet communication is shown in  FIG. 2 . The example of  FIG. 1  includes a communication device  210  having an Ethernet port E 1  and a communication port C 1 , and a communication device  212  having an Ethernet port E 2  and a communication port C 2 . The example system further includes an Ethernet switch  214  in data communication with the Ethernet ports E 1  and E 2  of the communication device  210  and the communication device  212  via respective Ethernet ports E 10  and E 11  of the Ethernet switch  214 . The Ethernet switch  214  includes an Ethernet port E 3  such that data communication of the Ethernet port E 3  of the Ethernet switch  214  are provided through the communication port C 1  of the communication device  210  and the communication port C 2  of the communication device  212 . 
     The example system of  FIG. 2  further includes a communication device  206  having an Ethernet port E 4  and a communication port C 3 , a communication device  208  having an Ethernet port E 5  and a communication port C 4 , and a network device  202  having an Ethernet port E 6 . An Ethernet switch  204  has an Ethernet port E 7  in communication with the Ethernet port E 4  of the communication device  206  and has an Ethernet port E 8  in communication with the Ethernet port E 5  of the communication device  208 . The Ethernet switch  204  aggregates communication with the communication device  206  and the communication device  208  to provide communication through an Ethernet port E 9  in communication with the network device  202 . 
     As shown in  FIG. 2 , the communication port C 3  of the communication device  206  is in data communication with the communication port C 1  of the communication device  210 , and the communication port C 4  of the communication device  208  is in data communication with the communication port C 2  of the second communication device  212 . 
     Providing an upload speed faster than a download speed between the ADSL modems  108  and  110  may not be permitted under a regulatory scheme where copper communications extend from the ADSL modem  108  back to the Ethernet network  102  and where the upload speed refers to an upload from the subscriber to the service provider network. However, the ADSL modems  108  may be positioned away from a CO to a point that is closer to the point of service where the ADSL modems  110  are positioned. Communications between the location of the ADSL modems  108  and the network  102  may then occur over fiber to avoid the faster upload speeds from creating copper networking issues. 
     The aggregated Ethernet communications utilize aggregated Ethernet connections so as to increase the bandwidths of the data transfer directions. The Ethernet switch  106  is positioned between the DSLAM or other service point containing modems  108  and the downstream link back to the Ethernet network  102 . The Ethernet switch  106  aggregates the communications of multiple (1−N) modems  108  that are in communication with multiple (1−N) modems  110 . The multiple modems  110  are then aggregated by an Ethernet switch  112  located at a service point such as the subscriber point of service. 
     Thus, the single Ethernet link accessible by the subscriber may benefit from the bandwidth of multiple modem connections to increase the effective bandwidth of the data service being provided to the subscriber. This aggregation of Ethernet links is described below in more detail with reference to  FIG. 2 . 
     The aggregated Ethernet communications shown in  FIG. 1  may also co-exist with other conventional services such as plain old telephone service (“POTS”). For example, the DSLAM housing DSL modems  108  provides a splitter function to interconnect the link back to the Ethernet network  102  with the DSL modems  108  for data communications while interconnecting voice links between the PSTN  104  and POTS device  116  of the subscriber. The voice links are passed through a filter-splitter on the subscriber end and also at the splitter function of the DSLAM so that the copper line pairs between the DSL modems  108  and DSL modems  110  may carry both the voice signals and the data signals while preventing the data signaling from being heard by the POTS devices  116 . 
     As shown in  FIG. 2 , the Ethernet communications of an Ethernet link may be transferred by multiple lower level communication devices  206 ,  208  such as DSL or cable modems  106  that communicate with distant multiple communication devices  210 ,  212  such as cable or DSL modems  108 . Where the data service is asymmetric, the upload speed differs from the download speed as provided from the communication devices  206 ,  208 ,  210 , and  212 . 
     The subscriber has a network device  202  that utilizes Ethernet communications, such as a router linked to a personal computer or such as a network interface card (“NIC”) of the personal computer. The Ethernet communications are exchanged between the Ethernet port E 6  of the network device  202  and the Ethernet port E 9  of an Ethernet switch  204 . The Ethernet switch  204  then communicates via the Ethernet ports E 7  and E 8  and corresponding Ethernet ports E 4  and E 5  of the two communication devices  206 ,  208  using Ethernet communications, and aggregates the two devices  206 ,  208  for use by the network device  202 . The Ethernet switch  204  may aggregate communications with the two devices  206 ,  208  in accordance with the link aggregation standard IEEE 802. 1ad. 
     The communication devices  206 ,  208  then exchange communications with the communication devices  210 ,  212  via respective pairs of the communication ports C 1 , C 3  and C 2 , C 4  over the copper lines, such as the RJ-11 phone line interface between DSL modems or the cable television coaxial interfaces between cable modems. This exchange between the communication devices  206 ,  208 ,  210  and  212  may either be symmetric or asymmetric, depending upon configuration of the communication devices  206 ,  208 ,  210 , and  212 . The communication devices  210 ,  212  also communicate with an Ethernet switch  214  that aggregates the communications of the communication devices  210 ,  212  when exchanging communications with the Ethernet network  102 . The Ethernet switch  214  may exchange communications with downstream device such as an ATM switch  216  or broadband service gateway (“BSG”)  218 . The Ethernet switching that provides for the aggregation of communication devices may alternatively be incorporated into the BSG  218  rather than utilizing an external Ethernet switch  214 . The data communications from the communication devices  210 ,  212  may be exchanged with the Ethernet switch  214  by a direct Ethernet interface for a DSLAM, by ATM interfaces through the ATM switch  216  (e.g., Ethernet over ATM encapsulation), or by interfaces from the BSG  218  (e.g., Ethernet, ATM, or Ethernet encapsulated over an Internet Protocol interface). 
     To optimize the aggregated communication devices  206 ,  208 ,  210  and  212 , the Ethernet switches  204 ,  214  may perform rate shaping and/or load balancing. The Ethernet switches  204 ,  214  may perform rate shaping by directing frames of data according to the upload speed of the communication devices  206 ,  208 ,  210  and  212  being aggregated by the Ethernet switch  204 ,  214 . Thus, the Ethernet switches  204 ,  214  may store in memory any upload speed differential that may exist between the two communication devices  206 ,  208 ,  210  and  212  connected to the ports E 7 , E 8 , E 10 , E 11  of the aggregator device  204 ,  214 . An example where the two communication devices  206 ,  208 ,  210  and  212  being aggregated by the Ethernet switch  204 ,  214  have different upload speeds is illustrated in  FIG. 3 . 
     In the example of  FIG. 3 , the communication devices  206 ,  208 ,  210  and  212  of  FIG. 2  are ADSL modems  304 ,  306 ,  308 , and  310 , respectively, where the asymmetry is flipped for the two modems  304 ,  306 ,  308 ,  310  at each endpoint so that a symmetric data transfer results. One Ethernet switch  312  aggregates an ATU-C ADSL modem  308  having a relatively faster upload speed with an ATU-R ADSL modem  310  having a, relatively faster download speed. Likewise, Ethernet switch  302  aggregates an ATU-R ADSL modem  304  having a relatively faster download speed with an ATU-C ADSL modem  306  having a relatively faster upload speed. The ATU-R modems  304 ,  310  communicate with the ATU-C modems  308 ,  306 , respectively, to establish the bi-directional data transfer. With this example ADSL configuration, the differential in upload speeds for each aggregated pair of modems may be defined as 8 megabits per second upload for one communication device and 1.5 megabits per second upload for the other communication device, which results in an example differential factor of 5.3. The Ethernet switch  302 ,  312  may then channel every sixth frame F 1 , F 6  to the modem  304 ,  306 ,  308 ,  310  with the slow upload speed while the five preceding frames F 2 -F 6 , F 7 -F 11  are channeled to the modem  304 ,  306 ,  308 ,  310  with the fast upload speed, as depicted in  FIG. 3 . 
     In addition to rate shaping, the Ethernet switches  204 ,  214 ,  302  and  312  of  FIGS. 2 and 3  may load balance between the communication devices being aggregated. The Ethernet switch  204 ,  214 ,  302 ,  312  communicates status information with the communication devices  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310  being aggregated to detect the amount of data that a communication device  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310  has queued to transfer. If one of the communication devices  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310  has overly filled its respective queue Q 1 , Q 2 , Q 4 , Q 5  relative to the other communication device  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310 , such as due to frame retransmissions or other similar reasons, then more frames are temporarily directed to the other communication device  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310  until the relative loads of each communication device  206 ,  208 ,  210 ,  212 ,  304 ,  306 ,  308 ,  310  have become balanced. 
       FIG. 4  shows an embodiment of an Ethernet aggregator device  402  that incorporates the functionality of the Ethernet switch and the communication devices into one device. This Ethernet aggregator device  402  may be utilized at the subscriber side, the service provider side, or at both sides. The device  402  includes an Ethernet switching portion  408  that has an Ethernet port for exchanging Ethernet communications. The Ethernet switching portion  408  directs the Ethernet communications via virtual port aggregation between a first communication portion  410  and a second communication portion  412 . Thus, there is a virtual port pathway  404  between the switching portion  408  and the communications portion  412  while there is a virtual port pathway  406  between the switching portion  408  and the communications portion  410 . These two communication portions correspond to integrated modems, such as DSL or cable modems. Each of these communications portions exchanges communications carrying Ethernet frames through communications ports, such as PSTN RJ-11 ports for DSL modems or coaxial ports for cable modems. 
     As discussed above, the subscriber of the Ethernet service is provided access to aggregated Ethernet communications that utilize multiple lower level communication devices to increase the bandwidth available for data transfer between the subscriber and the Ethernet network. Thus, data transports such as lines suitable for DSL or cable modem communications for an existing service provider configuration may be aggregated at the Ethernet level at both the subscriber and service provider end to provide the increased bandwidth service to the subscriber. 
     Although the present invention has been described in connection with various illustrative embodiments, those of ordinary skill in the art will understand that many modifications can be made thereto within the scope of the claims that follow. Accordingly, it is not intended that the scope of the invention in any way be limited by the above description, but instead be determined entirely by reference to the claims that follow.