Patent Publication Number: US-9838318-B2

Title: TCP congestion control for large latency networks

Description:
RELATED APPLICATION 
     This application claims priority to International Application No. PCT/US2012/071839 filed Dec. 27, 2012, which claims priority to previously filed U.S. Provisional application Ser. No. 61/580,784 filed Dec. 28, 2011, which is incorporated by reference in its entirety. 
    
    
     TECHNICAL FIELD 
     The current description relates generally to data communication over a computer network, and more particularly to a method and apparatus for Transmission Control Protocol (TCP) congestion control in large latency networks. 
     BACKGROUND 
     The function of a TCP congestion control mechanism is to adjust the rate with which the protocol sends packets to the network using a congestion control window Cwnd. A good congestion control mechanism can fully utilize the bandwidth while avoiding over-driving the network and thereby creating packet losses. Since the introduction of the first widely used TCP congestion control mechanism, TCP Reno, many TCP congestion control mechanisms have been proposed. Different congestion control mechanisms attempt to address various shortcomings of other congestion control mechanisms depending upon different network characteristics. 
     One particular network type that is difficult to provide a congestion control mechanism for that will fairly and fully utilize the network bandwidth is a heterogenous network that includes both high latency links and low latency links. One attempt at improving the congestion control mechanisms for heterogenous, high latency networks is TCP-Hybla. TCP-Hybla is similar to TCP Reno, in that it adjusts a congestion window (Cwnd) size based on the receipt of ACK packets and on a timeout of receiving the ACK. The congestion window controls the number of allowable unacknowledged packets that can be in transit. In standard TCP Reno the Cwnd is increased by a maximum segment size (MSS) for each non-duplicate ACK received. When a timeout occurs and the transmitted packet has not been acknowledged, Cwnd is decreased to 1 MSS. The increase in the window size is based on the RTT, and so high latency links do not fully utilize the available bandwidth. TCP Hybla attempts to increase the bandwidth use for high latency networks by removing the dependence of the throughput on RTT. TCP-Hybla adjusts Cwnd according to a normalized round trip time used to equalize the Additive Increase and Multiplicative Decrease (AIMD) performance of the congestion control mechanism. The throughput of TCP-Hybla is: 
     
       
         
           
             
               T 
               R 
             
             = 
             
               
                 1 
                 
                   RTT 
                   0 
                 
               
               ⁢ 
               
                 
                   2 
                   
                     3 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     PLR 
                   
                 
               
             
           
         
       
     
     As can be seen, the dependence of the throughput on RTT is removed and as such, the high latency links are not penalized. 
     Although TCP-Hybla can provide good throughput for high latency links, it cannot fully utilize the bandwidth when there is high packet loss, as may be the case for wireless links. 
     An additional or alternative TCP congestion control mechanism for heterogeneous networks with a high latency link and that may exhibit packet loss is desirable. 
     SUMMARY 
     In accordance with the description there is provided a method for congestion control of a communication session over a heterogeneous network having a high latency link. The method comprises determining a round trip time (RTT) for the communication session; determining a normalized RTT (ρ) based on an equalizing round trip time (RTT 0 ); determining a number, N, of virtual communication sessions; and setting a congestion window (Cwnd) size for the communication session based on N TCP virtual communication sessions and ρ. 
     In accordance with the description there is further provided a computing device for communicating over a network. The computing device comprises a processor for executing instructions; and a memory for storing instructions, which when executed by the processor configure the computing device to perform a method for congestion control. The method comprises determining a round trip time (RTT) for the communication session; determining a normalized RTT (ρ) based on an equalizing round trip time (RTT 0 ); determining a number, N, of virtual communication sessions; and setting a congestion window (Cwnd) size for the communication session based on N TCP virtual communication sessions and ρ. 
     In accordance with the description there is further provided a non-transitory computer readable medium storing instructions for configuring a computing device to perform a method for congestion control. The method comprises determining a round trip time (RTT) for the communication session; determining a normalized RTT (ρ) based on an equalizing round trip time (RTT 0 ); determining a number, N, of virtual communication sessions; and setting a congestion window (Cwnd) size for the communication session based on N TCP virtual communication sessions and ρ. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Illustrative embodiments of the invention will now be described with reference to the following drawings in which: 
         FIG. 1  depicts in a block diagram an illustrative network in which the TCP-FIT congestion control mechanism may be used; 
         FIG. 2  depicts in a block diagram an illustrative computing device that implements an enhanced TCP-Hybla congestion control mechanism; 
         FIG. 3  depicts in a flow chart an illustrative method of congestion control using an enhanced TCP-Hybla congestion control mechanism; 
         FIG. 4  depicts in a flow chart a further illustrative embodiment of a method of congestion control using an enhanced TCP-Hybla congestion control mechanism; 
         FIG. 5  shows a throughput comparison of different TCP algorithms over a high speed and long distance networks; 
         FIG. 6  shows a throughput comparison of different algorithms over a low speed and long distance networks; and 
         FIG. 7  shows throughput speedup of different algorithms compared to TCP Reno. 
     
    
    
     DESCRIPTION 
     A TCP congestion control mechanism that can eliminate penalization of TCP over networks with high latency links, while achieving high throughput performance even in the presence of high packet loss rate and packets reorder is described herein. As described further, the congestion control mechanism controls the window size, Cwnd, based on a number of virtual Hybla connections. The number of virtual connections used to set the window size may be adjusted dynamically using a queuing delay based mechanism to keep a balance between network utilization and congestion avoidance. 
     Additive Increase and Multiplicative Decrease (AIMD) is the most widely used congestion control algorithm in TCP congestion control. In AIMD algorithm, the congestion control window Cwnd of a TCP sender is adjusted by
 
Each ACK:  Cωnd←Cωnd+a  
 
Each Loss:  Cωnd←Cωnd−b·Cωnd   (1)
 
     In the expression (1), a and b are the increase and decrease factor of AIMD. AIMD increase its window by a to probe the network in every RTT (Round-Trip Time), and decrease the window by b·cωnd, where bε(0,1), to recover the network from congestion when a packet loss signaled congestion event is detected. The average throughput of (1) is 
     
       
         
           
             
               
                 
                   T 
                   = 
                   
                     
                       1 
                       RTT 
                     
                     ⁢ 
                     
                       
                         
                           a 
                           ⁡ 
                           
                             ( 
                             
                               2 
                               - 
                               b 
                             
                             ) 
                           
                         
                         
                           2 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             PLR 
                             · 
                             b 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     As is clear from above, equation (2) is a function of network RTT and PLR (Packet Loss Rate). In standard TCP Reno algorithm, a is set to 1 and b is set to 
               1   2     .         
Therefore, the average throughput of TCP Reno is
 
     
       
         
           
             
               
                 
                   
                     T 
                     R 
                   
                   = 
                   
                     
                       1 
                       RTT 
                     
                     ⁢ 
                     
                       
                         2 
                         
                           3 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           PLR 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     From equation (3), it can be seen that networks with high-latency, and so large RTT, will seriously degrade the throughput performance of TCP Reno. 
     TCP Hybla attempts to eliminate penalization of TCP connections that incorporate a high-latency terrestrial or satellite radio link, due to their longer round trip times. In TCP Hybla, the increase factor of 1 is set as 
                     a   H     =     RTT     RTT   0               (   4   )               
where RTT 0  is a normalized round trip time used to equalize the AIMD performance. The default setting of RTT 0  is 15 ms, although other settings for RTT 0  are possible. Therefore, the throughput of TCP Hybla can be modeled as
 
     
       
         
           
             
               
                 
                   
                     T 
                     H 
                   
                   = 
                   
                     
                       1 
                       
                         RTT 
                         0 
                       
                     
                     ⁢ 
                     
                       
                         2 
                         
                           3 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           PLR 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     As can be seen from above, the negative effects of high network latency is removed. However, the throughput is dependent upon the packet loss rate, and as such will not fully utilize the bandwidth in the presence of high packet loss. 
     TCP Hybla can be enhanced to improve the performance of heterogeneous networks with high latency networks that may experience pack loss. In the enhanced TCP hybla congestion control mechanism, the congestion control window is adjusted by 
     
       
         
           
             
               
                 
                   
                     
                       Each 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       ACK 
                       ⁢ 
                       
                         : 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       C 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       ω 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       nd 
                     
                     ← 
                     
                       
                         C 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         ω 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         nd 
                       
                       + 
                       
                         N 
                         · 
                         
                           a 
                           H 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       Each 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       Loss 
                       ⁢ 
                       
                         : 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       C 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       ω 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       nd 
                     
                     ← 
                     
                       
                         C 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         ω 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         nd 
                       
                       - 
                       
                         
                           
                             2 
                             
                               
                                 3 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 N 
                               
                               + 
                               1 
                             
                           
                           · 
                           C 
                         
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         ω 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         nd 
                       
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     In other words, the increase and decrease factors of (2) for the enhanced TCP Hybla congestion control mechanism are 
                     a   =     N   ·     RTT     RTT   0           ,     
     ⁢     b   =     2       3   ⁢           ⁢   N     +   1                 (   7   )               
where N is a dynamic parameter as discussed below. According to the expression (2), the throughput function of (6) is
 
     
       
         
           
             
               
                 
                   
                     T 
                     E 
                   
                   = 
                   
                     
                       N 
                       
                         RTT 
                         0 
                       
                     
                     ⁢ 
                     
                       
                         2 
                         
                           3 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           PLR 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     Compared to the throughput function (5) it can be seen that the throughput of the enhanced Hybla congestion control mechanism is N times that of TCP Hybla under the same network conditions. 
     The number N of Hybla sessions simulated in the enhanced Hybla session can be dynamically adjusted according to 
                       N     t   +   1       :=     max   ⁢     {     1   ,       N   t     +   β   -       N   t     ·         RTT   t     -     RTT   min         α   ·     RTT   t               }         ,           (   9   )               
where N t  is the N at the updating period t, RTT t  is the average RTT for the updating period t, RTT min  is the minimal observed RTT value for a given observation period, which may correspond to t. α and β may be preset positive parameters. The values of α and β may be tuned experimentally to provide desirable performance for various networks. Illustrative values are
 
             α   =     1   2           
and β=1. For β=1 the expectation of N from (9) is
 
     
       
         
           
             
               
                 
                   
                     E 
                     ⁡ 
                     
                       [ 
                       N 
                       ] 
                     
                   
                   = 
                   
                     max 
                     ⁢ 
                     
                       
                         { 
                         
                           1 
                           , 
                           
                             
                               
                                 α 
                                 · 
                                 
                                   RTT 
                                   t 
                                 
                               
                               
                                 
                                   RTT 
                                   t 
                                 
                                 - 
                                 
                                   RTT 
                                   min 
                                 
                               
                             
                             ⁢ 
                             
                               N 
                               t 
                             
                           
                         
                         } 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
     Plugging (10) into (8), the throughput function of the enhanced Hybla congestion control mechanism for given PLR and RTT is 
     
       
         
           
             
               
                 
                   
                     
                       T 
                       F 
                     
                     = 
                     
                       max 
                       ⁢ 
                       
                         { 
                         
                           
                             1 
                             
                               RTT 
                               0 
                             
                           
                           , 
                           
                             
                               α 
                               q 
                             
                             · 
                             
                               RTT 
                               
                                 RTT 
                                 0 
                               
                             
                           
                         
                         } 
                       
                       ⁢ 
                       
                         
                           3 
                           
                             2 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             PLR 
                           
                         
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
     Where q=RTT−RTT min , and represents a network queuing delay. As shown in the expression (11), the throughput of the enhanced Hybla congestion control mechanism will increase with the increasing of network RTT, which makes the congestion control mechanism well suited to high latency links. 
       FIG. 1  depicts in a block diagram an illustrative network in which the enhanced TCP-Hybla congestion control mechanism may be used. The network  100  comprises a plurality of senders  102   a ,  102   b  (referred to collectively as senders  102 ), each of which comprises a TCP stack  104  that includes the enhanced TCP-Hybla congestion control mechanism  106 . The senders  102  send packets of information to a plurality of receivers  108   a ,  108   b  (referred to collectively as receivers  108 ). The receivers  108  each comprise a TCP stack  110 . The TCP stacks  110  of the receivers  108  are depicted without a congestion control mechanism. Although the TCP implementation of the receivers  108  may include the congestion control mechanism, it is the senders  102  that are responsible for the congestion control and as such it is omitted from the receivers  108  of  FIG. 1 . 
     The senders  102  transmit the packets to the receivers  108  over a network comprised of a plurality of links between nodes  112   a ,  112   b ,  112   c ,  112   d  (referred to collectively as nodes  112 ). One of the links between nodes is a high latency link. In  FIG. 1 , the link between nodes  122   c  and  112   d  is depicted as a high latency link provided by a satellite connection  114 . The high latency link does not need to be provided by a link between two nodes, instead the high latency may be provided through a combination of links that each individually may not be high latency but combined provide a high latency connection between a sender and receiver. 
     One of the nodes will be a bottleneck. The bottleneck link has a bandwidth that it can transmit packets at. When the number of packets arriving at the bottleneck link  112  exceeds the transmission bandwidth capacity  114 , the packets  116  cannot be transmitted right away and so are temporarily stored in a buffer of the node. If packets arrive while the buffer is full, the node is congested and some packet(s) will be dropped. The timeout timer of the sender associated with the dropped packet will expire, and the congestion control mechanism will reduce the Cwnd of the session so that packets will be transmitted more slowly. 
       FIG. 2  depicts in a block diagram an illustrative computing device that may be used to implement the enhanced TCP-Hybla congestion control mechanism described herein. The computing device  200  comprises a processing unit  202  for executing instructions and a memory unit for storing instructions. The memory unit may include volatile memory  204  and non-volatile storage  206 . The memory unit stores instructions  208  that when executed by the processing unit  202  configure the computing device  200  to provide the enhanced TCP-Hybla congestion control mechanism  210 . The computing device  200  may further comprise an input/output (I/O) interface  212  for connecting devices to the computing device  200 . 
     The enhanced TCP-Hybla congestion control mechanism  210  comprises an RTT control component  214  that that determines the round trip time of packets. The RTT control component  214  may also determine other times associated with round trip times. For example, the RTT control component  214  may further determine an average RTT, a minimum and maximum RTT. The determined RTT information is used by a session number control component  216  that adjusts the dynamic value of the number of virtual Hybla connections used for setting the size of Cwnd. A Cwnd control component  218  uses the determined number of virtual connections to set the size of the Cwnd for the connection when an ACK packet is received or a timeout timer expires, indicating a lost packet. 
       FIG. 3  depicts in a flow chart an illustrative method of congestion control using the enhanced TCP-Hybla congestion control mechanism. The method  300  determines a round trip time (RTT) for the connection ( 302 ). The RTT can be determined from a time elapsed between when a packet is transmitted and when an ACK is received for the packet. A normalized RTT (ρ) is determined from the determined RTT ( 304 ). The normalized RTT can be determined by dividing the RTT by a normalization factor RTT 0 . The normalization factor can be preset, for example at 10 ms, 15 ms, 20 ms, 25 ms or other values depending upon the network. Alternatively, the normalization factor may be determined by monitoring the RTT of network connections. As an example, RTT 0  may be set as the minimum RTT time observed from the TCP session or from other TCP sessions on the same computing device. A number, N, of virtual TCP-Hybla communication sessions may be determined ( 306 ). The number N may be determined as set forth in equation (9) above. Once the number N of TCP-Hybla sessions has been determined, the congestion window size Cwnd can be set using the normalized RTT, ρ, and N ( 308 ). The size of the congestion window can be set according to equations (6) and (7) above. 
       FIG. 4  depicts in a flow chart a further illustrative embodiment of a method of congestion control. The method  400  begins with transmitting a packet ( 402 ). The method waits ( 404 ) and checks if an ACK for the transmitted packet has been received ( 406 ). If an ACK has not been received (No at  406 ), the method determines if a timeout associated with the transmitted packet has expired ( 408 ) and if the timeout has not expired (No at  408 ), the method returns to wait for the ACK ( 404 ). If the timeout has expired (Yes at  408 ), then a packet loss has occurred. Accordingly the number of virtual sessions used in setting the size of Cwnd is updated ( 410 ) and used to update Cwnd ( 412 ). When a packet is lost, the size of Cwnd is decreased. Once the size of Cwnd is set, the method may then send another packet when appropriate. When an ACK has been received (Yes at  406 ), the RTT is updated ( 414 ) and the number of virtual sessions used in setting the size of Cwnd is updated ( 410 ) and used to update Cwnd ( 412 ). When an ACK is received, the size of Cwnd is increased. 
     An analysis of the network utilization efficiency of the enhanced TCP-Hybla was performed using a simple, but widely adopted, network scenario. In the scenario, S TCP sessions indexed by sε{1, . . . , S} share a bottleneck link at the same time. The bottleneck link has a bandwidth upper bound B, round-trip propagation delay D and inherent random packet loss rate P. When the S TCP sessions send packets through the bottleneck link, the aggregate throughput is denoted by T S =Σ sεS T S  The bottleneck link queuing delays q and congestion packet loss rate p are given by 
     
       
         
           
               
             
               
                 
                   
                     { 
                     
                       
                         
                           
                             
                               p 
                               = 
                               0 
                             
                             , 
                             
                               q 
                               = 
                               0 
                             
                             , 
                           
                         
                         
                           
                             
                               
                                 if 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   T 
                                   S 
                                 
                               
                               &lt; 
                               B 
                             
                             , 
                           
                         
                         
                           
                             ( 
                             a 
                             ) 
                           
                         
                       
                       
                         
                           
                             
                               p 
                               = 
                               
                                 f 
                                 ⁡ 
                                 
                                   ( 
                                   q 
                                   ) 
                                 
                               
                             
                             , 
                             
                               q 
                               &gt; 
                               0 
                             
                             , 
                           
                         
                         
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 T 
                                 S 
                               
                             
                             = 
                             
                               B 
                               . 
                             
                           
                         
                         
                           
                             ( 
                             b 
                             ) 
                           
                         
                       
                     
                   
                 
                 
                   
                     ( 
                     12 
                     ) 
                   
                 
               
             
           
         
       
     
     ƒ(·) is the define function of the queue management algorithm of the bottleneck link. It is assumed that ƒ(0)=0 and ƒ(·) is a non-decreasing function of q. 
     Theorem 1: If S contains an enhanced TCP-Hybla session, the bottleneck link operates in state (b) of equation (12). 
     Proof: It is obvious that for any TCP sessions, their aggregate throughput in the network satisfies Σ sεS T s ≦B since the aggregate throughput of a link cannot exceed the available bandwidth. Suppose the bottleneck link is operating in state (α) with q=0. From Equation (11) the throughput of enhanced TCP-Hybla is T F →∞. This contradicts with Σ sεS T s ≦B. Therefore, the bottleneck must be operating in state (b). 
     Theorem 1 shows that enhanced TCP-Hybla can ensure the capacity of a network bottleneck is fully utilized. It is noted that similar conclusions cannot be proven for other TCP Hybla sessions. Since p≧0 and q≧0, there is an upper bound for the throughput of a standard TCP Hybla session s for any given P 
     
       
         
           
             
               
                 
                   
                     T 
                     s 
                   
                   = 
                   
                     
                       
                         
                           1 
                           
                             RTT 
                             0 
                           
                         
                         ⁢ 
                         
                           
                             3 
                             
                               2 
                               ⁢ 
                               
                                 ( 
                                 
                                   P 
                                   + 
                                   p 
                                 
                                 ) 
                               
                             
                           
                         
                       
                       ≤ 
                       
                         
                           1 
                           
                             RTT 
                             0 
                           
                         
                         ⁢ 
                         
                           
                             3 
                             
                               2 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               P 
                             
                           
                         
                       
                     
                     = 
                     
                       
                         T 
                         s 
                         max 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
     It is obvious that, for a bottleneck link that contains only TCP Hybla sessions, if the aggregated throughput of all TCP sessions sεS is consistent with 
                           ∑     s   ∈   S               ⁢           ⁢     T   s       ≤       ∑     s   ∈   S               ⁢     T   s   max         =         ∑     s   ∈   S               ⁢       1     RTT   0       ⁢       3     2   ⁢           ⁢   P             &lt;   B       ,           (   14   )               
the bottleneck link works at the unsaturated state (α) of (12). The condition (13) only depends on the inherent properties P and B of the bottleneck link. As shown in (13), when the random packet loss rate P of the network is sufficiently high, as may be the case with paths that travel over a wireless network, TCP Hybla sessions are unable to fully utilize network capacity. As such, standard TCP Hybla cannot achieve optimal performance in wireless networks with random packet loss and false loss caused by packets reorder.
 
     A number of experiments were performed to illustrate the performance of the enhanced TCP-Hybla congestion control mechanism. In the experiments, enhanced Hybla and TCP-FIT were embedded into the Linux kernel of TCP servers.  FIG. 5  shows a throughput comparison of different TCP algorithms over a high speed and long distance networks. In the experiment, a TCP client located in Zurich, Switzerland, downloaded a 100 MB file from an HTTP Server at Los Angeles, US. The average RTT of the link was 160 ms, packet loss rate was 0%. As shown in  FIG. 5 , the throughput of enhanced Hybla was higher than other algorithms. 
       FIG. 6  shows a throughput comparison of different algorithms over a low speed and long distance networks. In the experiment, a TCP client located in Beijing, China, downloaded a 10 MB file from an HTTP Server at Los Angeles, US. The average RTT of the link was 210 ms, packet loss rate was 0.1%. As shown in  FIG. 6 , the throughput of the enhanced TCP-Hybla was close to TCP-FIT and higher than other algorithms. The results of the experiments depicted in  FIGS. 5 and 6  demonstrate the performance improvement of the enhanced TCP-Hybla over high latency networks with random packet loss. 
     The performance of enhanced TCP-Hybla over large scale networks was tested over the PlanetLab testbed. PlanetLab is a group of computers available as a testbed for computer networking and distributed systems research. The throughput of enhanced TCP-Hybla was tested on PlanetLab. The throughput was compared with standard TCP Hybla, TCP CUBIC and TCP Reno. In the tests, 153 PlanetLab nodes located in 78 cities of 32 countries were used to download video clips from HTTP servers located in Los Angeles, US. These nodes were able to represent the condition of the current Internet. In the experiments, the PlanetLab nodes were divided into several groups according to their available network bandwidth. The throughput speedup of different algorithms compared to TCP Reno was plotted by groups in  FIG. 7 . As shown in  FIG. 7 , enhanced TCP Hybla achieved improved performance compared with the other TCP algorithms. 
     The above has described an enhance TCP Hybla congestion control mechanism that sets the congestion window size based on a number N of parallel virtual Hybla connections. A further enhanced TCP Hybla, referred to as enhanced start Hybla congestion control mechanism described further below, maintains the congestion window based on a combination of virtual Hybla connections and virtual Reno connections. The enhanced start Hybla congestion control mechanism maintains the congestion window using a number, N 1 , of parallel virtual TCP Hybla connections and a number, N 2 , of parallel virtual TCP-Reno connections. 
     Similar to the congestion control mechanism described above, the enhanced start Hybla congestion control mechanism adjusts the number of virtual connections according to the network conditions. The sum of N 1  and N 2 , i.e. the number of virtual connections, is given the notation N, and is updated periodically. For example, N may be updated every 1000 milliseconds. N may be updated according to: 
                     N     t   +   1       =       N   t     +   σ   -       N   t     ·     o   ^     ·         RTT   average     -     RTT   min         RTT   average                   (   15   )               
where N t+1  is the updated number of parallel virtual connection sessions; N t  is the current number of parallel virtual connection sessions; σ and τ are selected positive parameters. σ may be set to 1 and τ may be set as ½. RTT average  is the average round trip time in a previous estimation period and RTT min  is the minimal value for the round trip time for a previous window of time.
 
     The virtual TCP-Hybla connections number, N 1 , and the virtual TCP-Reno connections number, N 2 , are proportional to N 
                         {             N   1     =     N   ·   γ                   N   2     =     N   ·     (     1   -   γ     )                       (   16   )                 
where γ is a parameter selected to be between 0 and 1. A default value of γ may be ½. The selection of γ can be used to balance the aggressiveness of the Hybla portion of the congestion control with the more stable performance of the Reno portion of the congestion control. Both N 1  and N 2  can be periodically adjusted in order to balance the aggressiveness of TCP Hybla with the stableness of TCP Reno. The values of N 1  and N 2  can be determined based on current network conditions, past network conditions, or an estimation of future network conditions.
 
     For every virtual TCP-Hybla connection, there is a normalized RTT provided by 
                     ρ   =       RTT   average         (       RTT   0     +     RTT   min       )     /   2         ,           (   17   )               
where RTT 0  is the reference round trip time as described above.
 
     The enhanced start Hybla congestion control mechanism can operate in different modes. A slow start threshold (SS thresh ) can be used to determine when to switch from one mode to another. SS thresh  may be set based on the window size when a loss even occurs. For example, SS thresh  may be set to 
             Cwnd   ,     Cwnd   2     ,       (     1   -     2     3   ⁢   N         )     ·   Cwnd           
or other values. While the congestion window is less than SS thres , the enhanced start Hybla operates in a slow start phase. In the slow start phase, the enhanced start Hybla maintains the congestion window, Cwnd, according to:
 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             
                               Cwnd 
                               
                                 i 
                                 + 
                                 1 
                               
                             
                             = 
                             
                               
                                 Cwnd 
                                 i 
                               
                               + 
                               
                                 
                                   N 
                                   1 
                                 
                                 · 
                                 
                                   ( 
                                   
                                     
                                       2 
                                       ρ 
                                     
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               
                                 N 
                                 2 
                               
                             
                           
                           , 
                         
                       
                       
                         
                           for 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           each 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           ACK 
                         
                       
                     
                     
                       
                         
                           
                             
                               Cwnd 
                               
                                 i 
                                 + 
                                 1 
                               
                             
                             = 
                             
                               
                                 Cwnd 
                                 i 
                               
                               - 
                               
                                 
                                   Cwnd 
                                   i 
                                 
                                 · 
                                 
                                   2 
                                   
                                     3 
                                     · 
                                     
                                       ( 
                                       
                                         
                                           N 
                                           1 
                                         
                                         + 
                                         
                                           N 
                                           2 
                                         
                                       
                                       ) 
                                     
                                   
                                 
                               
                             
                           
                           , 
                         
                       
                       
                         
                           for 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           each 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           LOSS 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   18 
                   ) 
                 
               
             
           
         
       
     
     Once the congestion window is larger than SS thresh , the enhanced start Hybla switches to a congestion avoidance phase. In the congestion avoidance phase, the congestion window is adjusted according to: 
     
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             
                               Cwnd 
                               
                                 i 
                                 + 
                                 1 
                               
                             
                             = 
                             
                               
                                 Cwnd 
                                 i 
                               
                               + 
                               
                                 
                                   N 
                                   1 
                                 
                                 · 
                                 
                                   
                                     ρ 
                                     2 
                                   
                                   
                                     Cwnd 
                                     i 
                                   
                                 
                               
                               + 
                               
                                 
                                   N 
                                   2 
                                 
                                 · 
                                 
                                   1 
                                   
                                     Cwnd 
                                     i 
                                   
                                 
                               
                             
                           
                           , 
                         
                       
                       
                         
                           each 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           ACK 
                         
                       
                     
                     
                       
                         
                           
                             
                               Cwnd 
                               
                                 i 
                                 + 
                                 1 
                               
                             
                             = 
                             
                               
                                 Cwnd 
                                 i 
                               
                               - 
                               
                                 
                                   Cwnd 
                                   i 
                                 
                                 · 
                                 
                                   2 
                                   
                                     3 
                                     · 
                                     
                                       ( 
                                       
                                         
                                           N 
                                           1 
                                         
                                         + 
                                         
                                           N 
                                           2 
                                         
                                       
                                       ) 
                                     
                                   
                                 
                               
                             
                           
                           , 
                         
                       
                       
                         
                           each 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           LOSS 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
           
         
       
     
     The above has described various TCP congestion control mechanisms that can improve the throughput for high latency networks that experience packet loss. It is contemplated that any or all of these hardware and software components could be embodied exclusively in hardware, exclusively in software, exclusively in firmware, or in any combination of hardware, software, and/or firmware. Accordingly, while the following describes example methods and system, persons having ordinary skill in the art will readily appreciate that the examples provided are not the only way to implement such methods and apparatus. It will be apparent to one skilled in the art that numerous modifications and departures from the specific embodiments described herein may be made without departing from the spirit and scope of the present invention.