Patent Description:
Regardless of the network topology utilized by such communication networks, endpoints are usually tied together virtually. A user wanting to communicate with a server, for example, uses a physical path that is composed of network links, whose path is determined by the policies of the network owner. For example, an internet service provider (ISP) may provide network access to client devices via a hub-and-spoke network, where all traffic traverses through the hub on its way to or from remote clients. Even though there may be multiple access points for the remote client or the hub, the physical path that a packet flow takes is the same. The alternate communication paths are generally for diversity. In such implementations, all traffic is physically routed through one path, which may have many links that can change due to unforeseen circumstances. The total bandwidth is limited to the smallest communication link in the chain. As another example, consider a multi-star or mesh network. Even though there may be multiple physical communication paths to reach an endpoint from another endpoint, most of the aggregate available bandwidth (i.e., across all physical links) typically goes unused, and/or or network traffic is divided based on flows or destinations.

Link aggregation or channel bonding may refer to methods for combining multiple communication links between two communication endpoints in order to increase throughput beyond that of a single connection and/or to provide redundancy in case one of the links fails.

<CIT> describes methods, systems, and devices for wireless communication. Communicating devices may format data to be transmitted into a set of data units that are allocated to a communication link based on various factors. Correspondingly, a device that receives the data packets may reorder the packets.

Implementations of the disclosure are directed to network layer channel bonding methods and systems.

According to a first aspect of the invention, there is provided a method according to claim <NUM>.

According to a second aspect of the invention, there is provided a system according to claim <NUM>.

Some optional and/or preferable features a provided in the dependent claims.

In one embodiment, a method, comprises: operating a first communication device to transmit data to a second communication device over a plurality of communication links, each of the communication links associated with a respective different communication medium; receiving, at the first communication device, an input data stream for transmission to the second communication device, the input data stream comprising a plurality of packets; determining, at the first communication device, a throughput and latency of each of the plurality of communication links; placing, using a stream scheduler, based on a virtual link defining a virtual path of the plurality of communication links from the first communication device to the second communication device, the plurality of packets of the input data stream in one or more queues, wherein the one or more queues are based on quality of service (QoS) conditions; after placing the plurality of packets in one or more queues, dividing by a stream partitioner, based on the determined throughput and latency of each of the plurality of communication links on the virtual link, the plurality of packets into multiple sets of packets, each of the sets of packets configured to be transmitted by the first communication device over a respective one of the communication links, wherein dividing the plurality of packets comprises: assigning a number of the plurality of packets to each of the sets of packets based on the throughput of the set's corresponding communication link in proportion to a sum of all of the throughputs of the physical links; and ordering, based on the determined latencies, the plurality of packets to reduce a latency variance of the plurality of communication links; adding a sequence header to each of the plurality of packets, the sequence header comprising a sequence number based on the QoS and the second communication device; and after adding the sequence header, concurrently transmitting, from the first communication device to the second communication device, the sets of packets over the communication links, each of the sets of packets transmitted over the set's respective communication link.

In some implementations, the method further comprises: prior to transmitting each of the sets of packets, placing each of the sets of packets in a respective queue associated with the communication link over which the set of packets is configured to be transmitted over. In some implementations, the method further comprises: adjusting a size of each of the queues based on the determined latencies.

In some implementations, the communication mediums comprise: a satellite communication medium and a terrestrial cable communication medium; a cellular communication medium and a terrestrial cable communication medium; two different satellite communication mediums, two different cellular communication mediums; or two different terrestrial cable communication mediums.

Other features and aspects of the disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the features in accordance with various embodiments. The summary is not intended to limit the scope of the invention, which is defined solely by the claims attached hereto.

The technology disclosed herein, in accordance with one or more embodiments, is described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict typical or example embodiments of the disclosed technology. These drawings are provided to facilitate the reader's understanding of the disclosed technology and shall not be considered limiting of the breadth, scope, or applicability thereof. It should be noted that for clarity and ease of illustration these drawings are not necessarily made to scale.

The figures are not intended to be exhaustive or to limit the invention to the precise form disclosed. It should be understood that the invention can be practiced with modification and alteration, and that the disclosed technology be limited only by the claims and the equivalents thereof.

Some present communication systems utilize one of two types of channel bonding techniques: layer <NUM> (L2) channel bonding and layer <NUM> (L3) link aggregation. L2 channel bonding usually requires the same communication medium (e.g., two LEO satellite communication mediums) for all the channels. For example, in DVB-S2x channel bonding, a stream may be transported over three satellite transponders that are treated as one logical transponder.

L3 link aggregation typically requires that the communication mediums are similar in terms of their properties and the traffic flows (e.g., Transmission Control Protocol (TCP) sessions) are not split between them. Consider the case of a communication system that implements L3 link aggregation between two endpoints, where there are two different physical link paths between the endpoints. During communication between the endpoints, each traffic flow sent between the endpoints is put onto only one of the link paths. For example, if there are seven traffic flows between the two endpoints (e.g., flow <NUM>, flow <NUM>,. flow <NUM>), traffic flows <NUM>, <NUM>, and <NUM> may be placed on the first physical link path, and traffic flows <NUM>, <NUM>, <NUM>, and <NUM> may be placed on the second physical link path. In instances where both physical link paths operate at about the same throughput (e.g., <NUM> Megabits per second (Mbps) for each of the two sets of traffic flows), the total throughput between the two endpoints could potentially be doubled (e.g., to <NUM> Mbps). On the other hand, if almost no new traffic is received on one of the communication paths (e.g., traffic flows <NUM>, <NUM>, and <NUM>), and significant traffic is received on the other communication path (e.g., traffic flows <NUM>, <NUM>, <NUM>, and <NUM>), the total throughput between the two endpoints may be limited to the maximum throughput of a single path (e.g., <NUM> Mbps) unless the communication path that is not receiving new traffic has traffic waiting (e.g., in a buffer), which itself causes latency and jitter. As the foregoing example illustrates, L3 link aggregation may cause traffic to build up on one or more of the link paths while underutilizing one or more of the other link paths.

Various implementations of the disclosure are directed to network layer channel bonding techniques that address some of the limitations of conventional channel bonding techniques. In accordance with implementations of the disclosure, data of one or more traffic flows from two or more links of a network system may be combined to allow an overall throughput that is equal to or comparable to the sum of those links. As further discussed below, packets of data flows may be combined at layer <NUM> of the Open Systems Interconnection model (OSI) model. By virtue of utilizing the channel bonding methods and systems described herein, data may be transmitted on different types of communication mediums (e.g., satellite and cellular, two different satellite communication mediums, etc.). Additionally, the channel bonding techniques described herein may permit data to be transmitted on different physical links having different speeds and/or latencies while taking advantage of the full bandwidth available on each of the physical links between two endpoints of the communication system. These and other advantages that may be realized by the present disclosure are discussed below.

<FIG> illustrates an example communication system <NUM> in which the channel bonding systems and methods described herein may be implemented. In this example, communication system <NUM> is implemented as a hub-and-spoke network whereby all network traffic traverses through endpoint <NUM> to/from endpoints <NUM>-<NUM> through <NUM>-N (individually referred to as "an endpoint <NUM>" and collectively referred to as "endpoints <NUM>"). Communication system <NUM> may correspond to a system whereby an ISP associated with endpoint <NUM> provides subscribers associated with endpoints <NUM> and end devices <NUM>-<NUM> through <NUM>-N (individually referred to as an "end device <NUM>" and collectively referred to as "end devices <NUM>") network access (e.g., Internet access) to servers <NUM> (e.g., web servers). It should be noted, however, that the channel bonding techniques described herein are not limited to hub-and-spoke network systems or network systems where an ISP provides network access to subscribers. For example, the techniques described herein may be implemented in any type of mesh network such as multistar network configurations between endpoints, in ad-hoc networks between endpoints, etc..

End devices <NUM> may be any client device configured to generate a traffic flow that is transmitted to server system <NUM> by an endpoint <NUM> over multinetwork <NUM>, further described below. For example, an end device <NUM> may be a smartphone, a laptop, a desktop computer, a tablet, a smart television, a smart speaker, a smart home device, etc. A given traffic flow may carry image, video, audio, and/or other information.

Endpoint <NUM> and endpoints <NUM> are communication devices configured to implement the channel bonding techniques described herein. In particular, as depicted by <FIG>, a given endpoint <NUM> may transmit data to endpoint <NUM> or receive data from endpoint <NUM> over a plurality of networks <NUM>-<NUM> through <NUM>-N (individually referred to as a "network <NUM>"). These communication networks may be collectively referred to as a "multinetwork" <NUM>. Each network <NUM> of multinetwork <NUM> may be associated with a respective communication medium. For example, a given network <NUM> may correspond to a satellite communication medium (e.g., geosynchronous (GEO) satellite, low earth orbit (LEO) satellite, etc.), a cellular communication medium (e.g., <NUM>, <NUM>, Long Term Evolution (LTE), <NUM>, etc.), a cable communication medium, etc. An endpoint <NUM> or <NUM> may access each network <NUM> via a given physical link. For example, endpoint <NUM> is depicted as accessing networks <NUM>-<NUM>, and <NUM>-<NUM>, and <NUM>-N via physical links <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>, respectively. Endpoint <NUM>-<NUM> is depicted as accessing networks <NUM>-<NUM> and <NUM>-<NUM> via physical links <NUM>-<NUM> and <NUM>-<NUM>, respectively. Endpoint <NUM>-<NUM> is depicted as accessing networks <NUM>-<NUM> and <NUM>-N via physical links <NUM>-<NUM> and <NUM>-<NUM>, respectively. Endpoint <NUM>-N is depicted as accessing network <NUM>-N via physical link <NUM>-<NUM>. In some implementations, an endpoint <NUM> or <NUM> may be implemented as a modem (e.g., satellite modem) or a gateway (i.e., combined modem router such as a satellite gateway). In some implementation, an endpoint <NUM> may be integrated into an end device <NUM> that generates data for transmission over multinetwork <NUM>.

By implementing the network layer channel bonding techniques describe herein, a source endpoint may transmit traffic to a destination endpoint via all networks that both endpoints are communicatively coupled to. As such, by bonding communication channels together, each endpoint may have more bandwidth than any one link. By way of particular example, consider the case where endpoint <NUM>-<NUM> is able to transmit data over two different outroutes of two different satellite links (e.g., satellite link <NUM>-<NUM> and satellite link <NUM>-<NUM>), each satellite link having a maximum throughput of <NUM> Mbps. In such a case, endpoint <NUM> may transmit up to <NUM> Mbps of traffic to endpoint <NUM>, regardless of whether a given data flow is transmitted through network <NUM>-<NUM>, network <NUM>-<NUM>, or some combination thereof. As another example, consider the case where endpoint <NUM>-<NUM> is able to transmit data to endpoint <NUM> over a satellite link (e.g., link <NUM>-<NUM> or <NUM>-<NUM>) having a maximum throughput of <NUM> Mbps and a cellular link (e.g., link <NUM>-<NUM> or <NUM>-<NUM>) having a maximum throughput of <NUM> Mbps. In this scenario, despite the different speeds and communication mediums used by each the of the links, an aggregate throughput of up to <NUM> Mbps may be achieved. As such, the foregoing system enables the use of different links at different speeds, and on different communication mediums.

Each endpoint may include the necessary hardware, software, and/or firmware to provide functions such as traffic flow routing, packet processing, performance-enhancing proxies (PEPs), network acceleration (e.g., IP spoofing, web acceleration, and other network accelerations functions), etc. Due to the presence of PEPs and network accelerators, a source endpoint may have knowledge of each destination endpoint. An endpoint may also have knowledge of the physical links or connections (e.g., <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, or <NUM>-<NUM>) that lead to a destination endpoint, as well as the characteristics and limitations of the various physical links leading to that endpoint. The physical link may have a committed information rate (CIR) allocated to use for the entire link, as well as a CIR for each of the endpoints that use that link. A given endpoint may also have the software needed to encode packets such that a modem of the endpoint may transmit the data to the remote endpoint.

Due to throughput and/or latency differences over different communication links, an algorithm may be used to parse and order the data to the physical links, in a manner that alleviates waiting for packets on slower communications links. The amount of data that is transmitted on each physical link may be proportional to the throughput on the physical link. In some implementations, a small packet header may be used to allow packets to be reordered when arriving at a destination endpoint. In some implementations, a destination endpoint may send feedback to a source endpoint to allow for adjusting bandwidth on a given link due to congestion and/or physical faults. Each endpoint may also send feedback about the latency and jitter on a given link. The traffic flows on each physical link may be configured to meet both bandwidth and QoS requirements. Moreover, excess bandwidth may be traded for providing resiliency by duplicating traffic along various links. Latency sensitive traffic may be placed on a link with less jitter/latency issues whereas traffic that is not latency sensitive could be placed on another link. For example, HTTP Gets may be placed on a lower latency link, whereas TCP acknowledgements may be placed on a higher latency link.

In various implementations, endpoints may be grouped by virtual links. For multinetwork <NUM>, a virtual link defining a virtual path <NUM> from a source endpoint to a destination endpoint may be created, and its parameters may be stored at the source endpoint and/or destination endpoint. A virtual link may be composed of different physical links with different characteristics such as latency and speed. For example, a given endpoint may have a CIR for each physical link, which may be combined to create a CIR for a given virtual link between two endpoints. The CIR for a given physical link may refer to the maximum throughput that the link can carry and/or is assigned to the endpoints.

<FIG> is a block diagram illustrating an example data flow of user traffic <NUM> from a source endpoint <NUM> to a destination endpoint <NUM>, where channel bonding is implemented in accordance with implementations of the disclosure. For simplicity of illustration, some components of source endpoint <NUM> and destination endpoint <NUM> are not shown.

The user traffic <NUM> may originate, for example, as a data stream generated by an end device <NUM> communicatively coupled to the source endpoint <NUM>. Packets of user traffic <NUM> are received by a receiver (not shown) of source endpoint <NUM> and appropriately routed. The received packets may be accelerated using one or more accelerators <NUM>. The one or more accelerators <NUM> may accelerate the packets based on the characteristics of the physical link (e.g., satellite, cellular, etc.) they are destined for. For example, PEPS and network acceleration may be applied to the received packets. In other implementations, packet acceleration may not be applied.

After packet acceleration, stream scheduler <NUM> may configured to place a stream's packets in one or more queues that are ready to be transmitted. These queues may be based on quality of service (QoS). In this instance, because source endpoint <NUM> is configured to transmit packets over different communication mediums, the packets may be placed in the queues based on a virtual link (as opposed to a single physical link) defining a virtual path of multiple physical links from source endpoint <NUM> to destination endpoint <NUM>. In particular, the stream scheduler <NUM> may schedule traffic on the virtual link based on the CIR for the endpoint, and the CIR of the destination endpoint <NUM>.

Thereafter, stream partitioner <NUM> is configured to take the stream of data and divide and order the data flow according to the different throughputs and different latencies of the various physical links on the virtual link from source endpoint <NUM> to destination endpoint <NUM>. To this end, stream partitioner <NUM> may implement an algorithm that monitors the physical links for each of their properties (e.g., throughput and latency). The number of input frames sent on each link may be proportional to a maximum throughput allowed (e.g., CIR) for each physical link.

<FIG> illustrates one particular example of a stream partitioner <NUM> dividing and ordering packets of a data stream in an implementation where source endpoint <NUM> transmits data streams to a destination endpoint <NUM> over three physical links: satellite link A, satellite link B, and a low latency LTE link. As depicted in this example, the LTE link has a throughput of 1x, satellite link B has a throughput of 5x, and satellite link A has a throughput of 10x. Based on the relative throughputs, for every packet assigned to the LTE link for transmission using transmit (TX) circuitry/modulator <NUM>-<NUM>, five packets are assigned to satellite link B for transmission using transmit (TX) circuitry/modulator <NUM>-<NUM>, and ten packets are assigned to satellite link C for transmission using transmit (TX) circuitry/modulator <NUM>-<NUM>. Additionally, the packets are ordered/scheduled based on the relative latencies of the different communication links. For example, the packets may be ordered to account for the approximate <NUM> latency delay associated with satellite transmissions, and the low latency delay (e.g., <NUM>-<NUM>) associated with the LTE link. As such, the packets may be reordered to reduce the jitter and the latency difference between the links. This may also help reduce the size of buffers on a stream combiner <NUM> of destination endpoint <NUM>, further discussed below. The stream partitioner <NUM> may place the packets in respective queues <NUM>-<NUM>, <NUM>-<NUM>, or <NUM>-<NUM>, and each queue may be implemented as a jitter buffer, whereby the size of each queue is continually adjusted to keep the overall end to end jitter low.

Referring again to <FIG>, after stream partitioner <NUM> divides and orders the data flow according to the different throughputs and different latencies of the various physical links, the partitioned data flow may be transmitted to destination endpoint <NUM> over a plurality of networks <NUM>-<NUM> to <NUM>-N using a respective plurality of transmitters <NUM>-<NUM> to <NUM>-N (individually referred to as a "transmitter <NUM>" and collectively referred to as "transmitters <NUM>"). In various implementations, the packets of the partitioned data flow may be transmitted in parallel over multiple networks of multinetwork <NUM> to maximize an aggregate transmission throughput. Although transmitters <NUM> are depicted in this example as being components of source endpoint <NUM>, in other implementations, source endpoint <NUM> may route the packets to transmitters <NUM> of one or more secondary devices for transmission.

In some implementations, a small header may be placed on each packet as it is sent in order for a stream combiner of destination endpoint <NUM> to reorder the packets. This header may contain a sequence number that is based on both the QoS and the destination endpoint <NUM>. When the stream partitioner <NUM> attempts to use multiple links for a given traffic flow (e.g., QoS plus destination endpoint), it may use the header to identify this flow. For example, if an endpoint has <NUM> QoSs, the destination endpoint may be considered twelve different endpoints. In such a scenario, the stream combiner <NUM> may reorder packets for twelve different streams or flows, each stream or flow comprising traffic going to a unique destination (i.e., QoS plus destination endpoint).

Referring now to destination endpoint <NUM>, the different partitions of the partitioned data stream transmitted over multinetwork <NUM> may be received by a plurality of receivers <NUM>-<NUM> to <NUM>-N (individually referred to as a "receiver <NUM>" and collectively referred to as "receivers <NUM>") of destination endpoint <NUM>. In particular, each receiver <NUM> may receive, from a respective network <NUM>, a signal including a set of frames encapsulating packets corresponding to a partition of the partitioned data stream. The packets may be extracted from the frames and subsequently combined using stream combiner <NUM>.

In particular, stream combiner <NUM> is configured to read a header (e.g., containing sequence number) from each of the packets to reorder the packets into a receive data stream. An appropriate receiver algorithm may be implemented to ensure that dropped packets are caught, and to ensure in-order delivery of packets. If a gap is identified, but not recognized as dropped, the packets may be placed in a re-order queue. Once the gap is cleared or recognized as a drop, the re-ordered queue may emptied to the awaiting next level of software. Optionally, destination endpoint <NUM> may establish a feedback loop with source endpoint <NUM>. This may provide feedback on traffic flow so that the source endpoint <NUM> may modify the physical links and how much traffic it can use for the source endpoint.

As also depicted in <FIG>, destination endpoint <NUM> may also include processing and/or acceleration components <NUM> to provide additional processing and/or acceleration functions to the extracted, reordered, and combined packets of the data flow.

<FIG> provides a simplified representation of the operation of a source endpoint <NUM> and a destination endpoint <NUM> in a particular implementation where IP frames are transmitted over two of three available communication links (a first satellite link, an LTE link, and a second satellite link). As depicted, five frames corresponding to five packets are reordered and transmitted as GSE frames over the first satellite communication link and as GPRS/UDP frames over the LTE link. Each set of frames is received by the destination endpoint over each respective link, and reordered and combined using a stream combiner of the destination endpoint.

<FIG> is an operational flow diagram illustrating an example method <NUM> that may be performed by a first communication device to implement network layer channel bonding when transmitting to a second communication device, in accordance with implementations of the disclosure. For example, the first communication device may correspond to source endpoint <NUM> and the second communication device may correspond to destination endpoint <NUM>. In some implementations, the operations of method <NUM> may be implemented by the first communication device in response to a processor of the first communication device executing instructions stored on a non-transitory computer readable medium of the first communication device.

At operation <NUM>, the first communication device is operated to transmit data to a second communication device over a plurality of communication links, each of the communication links associated with a respective communication medium. For example, the communication mediums may comprise one or more satellite communication mediums and one or more cellular communication mediums, two or more different satellite communication mediums, or two or more different cellular communication mediums.

In some implementations, operating the first communication device to transmit the data may include preparing the first communication device for data transmissions to the second communication device. For example, the first communication device may establish a virtual link from the first communication to the second communication device based on a plurality of physical links that include the plurality of communication links. For example, when configuring the virtual link, the first communication device may establish a CIR and other parameters associated with each of the physical links. Additionally, the first communication device may perform a handshake procedure (e.g., TCP <NUM>-way handshake) with the second communication device to initiate data communications.

At operation <NUM>, the first communication device receives an input data stream for transmission to the second communication device, the input data stream comprising a plurality of packets. For example, the input data stream may be received from an end device <NUM> as described above. After receiving the input data stream, a stream scheduler of the first communication device (e.g., stream scheduler <NUM>) may place the stream's packets in one or more queues (e.g., based on QoS conditions) that are ready to be transmitted.

At operation <NUM>, the first communication device determines a throughput and latency of each of the plurality of communication links.

At operation <NUM>, based on the determined throughput and latency of each of the plurality of communication links: the first communication device divides the plurality of packets into multiple sets of packets, each of the sets of packets configured to be transmitted by the first communication device over a respective one of the communication links. To this end, a stream partitioner <NUM> as described above may be utilized.

In some implementations, dividing the plurality of packets into the multiple sets of packets, comprises: assigning a number of the plurality of packets to each of the sets of packets based on the throughput of the set's corresponding communication link in proportion to a sum of all of the throughputs of the physical links. In such implementations, dividing the plurality of packets into the multiple sets of packets, may further comprise: ordering the plurality of packets based on the determined latencies. In particular, the packets may be ordered to reduce a latency variance of the plurality of communication links.

At operation <NUM>, the first communication device transmits, to the second communication device, each of the sets of packets over the set's respective communication link. In various implementations, each of the sets of packets may be concurrently transmitted over each of the sets of communication links. For example, by transmitting the different sets of packets in parallel, the throughput from the first communication device to the second communication device may be greater than the throughput of any individual communication link. In some implementations, prior to transmission of each of the sets of packets, each of the sets of packets is placed in a respective queue associated with the communication link over which the set of packets is configured to be transmitted over. The size of each of the queues may be periodically or continuously adjusted based on the determined latencies.

In some implementations, prior to transmission of each of the sets of packets, a sequence header is added to each of the plurality of packets. The header may contain a sequence number that is based on both the QoS and the destination endpoint <NUM>.

<FIG> is an operational flow diagram illustrating an example method <NUM> that may be implemented by a destination communication device (e.g., destination endpoint <NUM>) to provide network layer channel bonding when receiving data from a source communication device (e.g., source endpoint <NUM>), in accordance with implementations of the disclosure. In some implementations, the operations of method <NUM> may be implemented by the destination communication device in response to a processor of the destination communication device executing instructions stored on a non-transitory computer readable medium of the destination communication device.

At operation <NUM>, the destination communication device receives, at each of a plurality of receivers, a signal including a set of frames transmitted over a respective communication medium by a source communication device. At operation <NUM>, a plurality of packets are obtained by extracting a packet from each frame of each of the sets of frames. At operation <NUM>, a sequence header (e.g., sequence number) is read for each of the plurality of extracted packets. At operation <NUM>, based on the sequence headers read from the packets, the packets are reordered into a receive data stream.

Where QoS is implemented at the destination endpoint, sequence headers may be based on the QoS in addition to the destination communicative device. As such, a stream combiner of the destination communication device may reorder packets for each stream or traffic flow having its own QoS.

In some implementations, the destination communication device may be further configured to send feedback to the source communication device regarding bandwidth on a given physical link traversed by one or more of the signals. For example, the feedback may indicate traffic congestion and/or physical faults on a given physical link. In some implementations, the destination communication device may be further configured to send feedback to the source communication device regarding latency and/or jitter on a given physical link traversed by one or more of the signals. Using this feedback, data stream partitions may be adjusted by the stream partitioner of the source communication device.

<FIG> illustrates a computer system/communication device <NUM> upon which example embodiments according to the present disclosure can be implemented. Computer system <NUM> can include a bus <NUM> or other communication mechanism for communicating information, and a processor <NUM> coupled to bus <NUM> for processing information. Computer system <NUM> may also include main memory <NUM>, such as a random access memory (RAM) or other dynamic storage device, coupled to bus <NUM> for storing information and instructions to be executed by processor <NUM>. Main memory <NUM> can also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor <NUM>. Computer system <NUM> may further include a read only memory (ROM) <NUM> or other static storage device coupled to bus <NUM> for storing static information and instructions for processor <NUM>. A storage device <NUM>, such as a magnetic disk or optical disk, may additionally be coupled to bus <NUM> for storing information and instructions.

According to one embodiment of the disclosure, network layer channel bonding may be provided by computer system <NUM> in response to processor <NUM> executing an arrangement of instructions contained in main memory <NUM>. Such instructions can be read into main memory <NUM> from another computer-readable medium, such as storage device <NUM>. Execution of the arrangement of instructions contained in main memory <NUM> causes processor <NUM> to perform one or more processes described herein. One or more processors in a multi-processing arrangement may also be employed to execute the instructions contained in main memory <NUM>. In alternative embodiments, hard-wired circuitry is used in place of or in combination with software instructions to implement various embodiments. Thus, embodiments described in the present disclosure are not limited to any specific combination of hardware circuitry and software.

Computer system <NUM> may also include a communication interface <NUM> coupled to bus <NUM>. Communication interface <NUM> can provide a two-way data communication coupling to a network link <NUM> connected to a local network <NUM>. Wired and/or wireless links may be implemented. In any such implementation, communication interface <NUM> sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

Network link <NUM> may provide data communication through one or more networks to other data devices. By way of example, network link <NUM> can provide a connection through local area network <NUM> to network devices, for example including a host computer (PC) <NUM>, a smartphone <NUM>, and the like. Local area network <NUM> may both use electrical, electromagnetic, or optical signals to convey information and instructions. The signals through the various networks and the signals on network link <NUM> and through communication interface <NUM>, which communicate digital data with computer system <NUM>, are example forms of carrier waves bearing the information and instructions.

Computer system <NUM> may send messages and receive data, including program code, through the network(s), network link <NUM>, and communication interface <NUM>. In the Internet example, a server (not shown) might transmit requested code belonging to an application program for implementing an embodiment of the present disclosure through local network <NUM> and communication interface <NUM>. Processor <NUM> executes the transmitted code while being received and/or store the code in storage device <NUM>, or other non-volatile storage for later execution. In this manner, computer system <NUM> obtains application code in the form of a carrier wave.

Computer system <NUM> includes equipment for communication with an external communications network. In particular, the computer system <NUM> may include a transmit-side physical-layer device (TX PHY) <NUM>, a receive-side physical-layer device (RX PHY) <NUM>, a transmit-side media access controller (TX MAC) <NUM>, and a receive-side media access controller (RX MAC) <NUM>. Transmit packets may be provided to the TX MAC <NUM> and TX PHY <NUM>, which provide corresponding signals to the external communications network <NUM>. For example, in a satellite communications network, TX MAC may be a TX satellite link controller (SLC), and TX PHY <NUM> may provide corresponding signals to a satellite using a terrestrial antenna/dish. Signals received from an external communications network <NUM> may be received via RX PHY <NUM> and RX MAC <NUM>, from which receive packets may be obtained.

<FIG> illustrates a chip set <NUM> in which embodiments of the disclosure may be implemented. Chip set <NUM> can include, for instance, processor and memory components described with respect to <FIG> incorporated in one or more physical packages. By way of example, a physical package includes an arrangement of one or more materials, components, and/or wires on a structural assembly (e.g., a baseboard) to provide one or more characteristics such as physical strength, conservation of size, and/or limitation of electrical interaction.

In one embodiment, chip set <NUM> includes a communication mechanism such as a bus <NUM> for passing information among the components of the chip set <NUM>. A processor <NUM> has connectivity to bus <NUM> to execute instructions and process information stored in a memory <NUM>. Processor <NUM> includes one or more processing cores with each core configured to perform independently. Alternatively or in addition, processor <NUM> includes one or more microprocessors configured in tandem via bus <NUM> to enable independent execution of instructions, pipelining, and multithreading. Processor <NUM> may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP) <NUM>, and/or one or more application-specific integrated circuits (ASIC) <NUM>. DSP <NUM> can typically be configured to process real-world signals (e.g., sound) in real time independently of processor <NUM>. Similarly, ASIC <NUM> can be configured to performed specialized functions not easily performed by a general purposed processor.

Processor <NUM> and accompanying components have connectivity to the memory <NUM> via bus <NUM>. Memory <NUM> includes both dynamic memory (e.g., RAM) and static memory (e.g., ROM) for storing executable instructions that, when executed by processor <NUM>, DSP <NUM>, and/or ASIC <NUM>, perform the process of example embodiments as described herein. Memory <NUM> also stores the data associated with or generated by the execution of the process.

In this document, the terms "machine readable medium," "computer readable medium," and similar terms are used to generally refer to non-transitory mediums, volatile or non-volatile, that store data and/or instructions that cause a machine to operate in a specific fashion. Common forms of machine readable media include, for example, a hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, an optical disc or any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge, and networked versions of the same.

These and other various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processing device for execution. Such instructions embodied on the medium, are generally referred to as "instructions" or "code. " Instructions may be grouped in the form of computer programs or other groupings. When executed, such instructions may enable a processing device to perform features or functions of the present application as discussed herein.

In this document, a "processing device" may be implemented as a single processor that performs processing operations or a combination of specialized and/or general-purpose processors that perform processing operations. A processing device may include a CPU, GPU, APU, DSP, FPGA, ASIC, SOC, and/or other processing circuitry.

The various embodiments set forth herein are described in terms of exemplary block diagrams, flow charts and other illustrations. As will become apparent to one of ordinary skill in the art after reading this document, the illustrated embodiments and their various alternatives can be implemented without confinement to the illustrated examples. For example, block diagrams and their accompanying description should not be construed as mandating a particular architecture or configuration.

Each of the processes, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code components executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms may be implemented partially or wholly in application-specific circuitry. The various features and processes described above may be used independently of one another, or may be combined in various ways. Different combinations and subcombinations are intended to fall within the scope of this disclosure, and certain method or process blocks may be omitted in some implementations. Additionally, unless the context dictates otherwise, the methods and processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto can be performed in other sequences that are appropriate, or may be performed in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The performance of certain of the operations or processes may be distributed among computer systems or computers processors, not only residing within a single machine, but deployed across a number of machines.

As used herein, the term "or" may be construed in either an inclusive or exclusive sense. Moreover, the description of resources, operations, or structures in the singular shall not be read to exclude the plural. Conditional language, such as, among others, "can," "could," "might," or "may," unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or steps.

Claim 1:
A method (<NUM>), comprising:
operating (<NUM>) a first communication device (<NUM>) to transmit data to a second communication device (<NUM>) over a plurality of communication links, each of the communication links associated with a respective different communication medium;
receiving (<NUM>), at the first communication device (<NUM>), an input data stream (<NUM>) for transmission to the second communication device (<NUM>), the input data stream (<NUM>) comprising a plurality of packets;
determining (<NUM>), at the first communication device (<NUM>), a throughput and latency of each of the plurality of communication links;
placing, using a stream scheduler (<NUM>), based on a virtual link defining a virtual path (<NUM>) of the plurality of communication links from the first communication device (<NUM>) to the second communication device (<NUM>), the plurality of packets of the input data stream (<NUM>) in one or more queues, wherein the one or more queues are based on quality of service (QoS) conditions;
after placing the plurality of packets in one or more queues, dividing (<NUM>) by a stream partitioner (<NUM>), based on the determined throughput and latency of each of the plurality of communication links on the virtual link, the plurality of packets into multiple sets of packets, each of the sets of packets configured to be transmitted by the first communication device (<NUM>) over a respective one of the communication links, wherein dividing the plurality of packets comprises:
assigning a number of the plurality of packets to each of the sets of packets based on the throughput of the set's corresponding communication link in proportion to a sum of all of the throughputs of the physical links; and
ordering, based on the determined latencies, the plurality of packets to reduce a latency variance of the plurality of communication links;
adding a sequence header to each of the plurality of packets, the sequence header comprising a sequence number based on the QoS and the second communication device; and
after adding the sequence header, concurrently transmitting (<NUM>), from the first communication device (<NUM>) to the second communication device (<NUM>), the sets of packets over the communication links, each of the sets of packets transmitted over the set's respective communication link.