Abstract:
A method for mitigating buffering delay in a TCP-encapsulated data stream transmitted over a network includes, at a node in the network, receiving a first packet through a first interface. The first packet includes data of the TCP-encapsulated data stream. A second packet is received through a second interface. The second packet includes an acknowledgment of segments of the TCP-encapsulated data stream. The method further includes analyzing the first packet and the second packet to estimate an amount of outstanding data, using the amount of outstanding data to determine a state of buffer occupancy, updating a TCP Window field in the second packet in function of the state of buffer occupancy, and transmitting the updated second packet through the first interface.

Description:
FIELD OF THE INVENTION 
       [0001]    The present invention pertains to the field of communication at the transport layer, more in particular to the use of Transmission Control Protocol (TCP) signaling at the network edge. 
       BACKGROUND 
       [0002]    Wireless and cellular communication channels are often limited in capacity and subject to propagation effects such as shadowing and interference results, which result in further limitation of useful bandwidth due to need to retransmit bursts of damaged packets. In an end-to-end TCP-based communication, these channels often constitute a bottleneck link. Moreover, a layer-2 retransmission mechanism is used to hide over-the-air losses, thereby making the round-trip delay for such TCP session quite variable. During these loss events, the datagram transmission buffers of the BTS are filled with new traffic, while the lost data is retransmitted. 
       SUMMARY 
       [0003]    It is an object of the present invention to provide a method for mitigating the buffering delay, in particular in wireless networks, more in particular at the cellular edge. 
         [0004]    According to a first aspect of the invention, there is provided a method for mitigating buffering delay in a TCP-encapsulated data stream transmitted over a network, the method comprising conducting the following steps at a node in said network: receiving a first packet through a first interface, said first packet comprising data of said TCP-encapsulated data stream; receiving a second packet through a second interface, said second packet comprising an acknowledgment of segments of said TCP-encapsulated data stream; analyzing said first packet and said second packet to estimate an amount of outstanding data; using said amount of outstanding data to determine a state of buffer occupancy; updating a TCP Window field in said first packet in function of said state of buffer occupancy, and transmitting said updated first packet through said second interface. 
         [0005]    It is an advantage of the method of the present invention that it provides a non-invasive form of flow control without resorting to the raster drastic measure of dropping packets, which could have undesired consequences. It is a further advantage of the method of the present invention that it can be implemented at various existing or newly provided network nodes in the path of the TCP communication in a wholly transparent way, i.e. without knowledge or active cooperation of the end points. 
         [0006]    Embodiments of the method of the present invention are based inter alia on the insight that the acknowledgments of previously transmitted TCP data segments, as appearing in the return flow of a TCP conversation, can be used to estimate an amount of outstanding data. This amount of outstanding data is indicative of the buffer occupancy conditions in the portion of the network between the sender and the receiver. 
         [0007]    In an embodiment of the method of the present invention, said analyzing comprises comparing a SEQ field of said first packet to an ACK field of said second packet. 
         [0008]    In an embodiment of the method of the present invention, said determining of a state of buffer occupancy comprises comparing said amount of outstanding data to a predetermined threshold amount. 
         [0009]    In an embodiment of the method of the present invention, one of said first interface and said second interface comprises a cellular interface. In a particular embodiment, one of said first interface and said second interface comprises a 3G wireless interface. In a particular embodiment, one of said first interface and said second interface comprises a femtocell interface. 
         [0010]    In an embodiment of the method of the present invention, one of said first interface and said second interface comprises a WLAN interface. 
         [0011]    It is an advantage of these embodiments that they resolve the excessive buffering problem of the traditional weakest link of a wireless or mobile network: the radio-based edge. Additionally, embodiments of the method of the present invention can be advantageously used to control TCP traffic when preparing a hand-over between wireless networks with different bandwidth and/or buffering characteristics, to avoid anomalous out-of-order delivery of buffered packets when the TCP conversation is being moved from one edge network to another. 
         [0012]    According to an aspect of the present invention, there is provided a computer program configured to cause a processor to perform the method as described above. 
         [0013]    According to an aspect of the present invention, there is provided an apparatus for mitigating buffering delay in a TCP-encapsulated data stream transmitted over a network, the apparatus comprising: a first interface adapted to receive a first packet comprising data of said TCP-encapsulated data stream; a second interface adapted to receive a second packet comprising an acknowledgment of a segments of said TCP-encapsulated data stream; a processor operatively connected to said first interface and said second interface, said processor being configured to analyze said first packet and said second packet to estimate an amount of outstanding data, to use said amount of outstanding data to determine a state of buffer occupancy, and to update a TCP Window field in said second packet in function of said state of buffer occupancy, and to transmit said updated second packet through said first interface. 
         [0014]    In an embodiment of the apparatus of the present invention, one of said first interface and said second interface comprises a cellular interface. 
         [0015]    In a particular embodiment, one of said first interface and said second interface comprises a  3 G wireless interface. 
         [0016]    In a particular embodiment, one of said first interface and said second interface comprises a femtocell interface. 
         [0017]    In an embodiment of the apparatus of the present invention, one of said first interface and said second interface comprises a WLAN interface. 
         [0018]    According to an aspect of the invention, there is provided a mobile node comprising the apparatus as described above. 
         [0019]    According to an aspect of the invention, there is provided a wireless BTS comprising the apparatus as described above. 
         [0020]    The advantages of the apparatus and computer programs according to the present invention are analogous to those described above in connection with the method according to the present invention. 
     
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         [0021]    Some embodiments of apparatus and/or methods in accordance with embodiments of the present invention are now described, by way of example only, and with reference to the accompanying drawings, in which: 
           [0022]      FIG. 1  provides a flow chart of a method according to an embodiment of the present invention; 
           [0023]      FIG. 2  is a schematic representation of a network in which the method and/or apparatus according to the present invention may be deployed; 
           [0024]      FIG. 3  is another schematic representation of a network in which the method and/or apparatus according to the present invention may be deployed, using NMIP proxies; 
           [0025]      FIG. 4  represents a first use scenario of a method according to the present invention; and 
           [0026]      FIG. 5  represents a second use scenario of a method according to the present invention. 
       
    
    
       [0027]    Throughout the figures, like reference signs have been used to designate the same or similar elements. 
       DETAILED DESCRIPTION OF EMBODIMENTS 
       [0028]    It is an inherent property of TCP-based communication that large amounts of data can be in transit persistently between sender and receiver when the channel has a high bandwidth×delay product. Accordingly, when a bottleneck link allows for a large amount of buffering without packet loss, the TCP infrastructure creates a large queue of transmitted undelivered datagrams inside the network. More specifically, the data will accumulate just ahead of the bottleneck link, if the sender sends at a higher rate than the bottleneck link can sustain. 
         [0029]    When this excessive buffering is combined with an aggressive retransmission policy, the network load will become unstable and undergo crippling network congestion: datagram transfer delays grow unboundedly due to queuing delay, datagram losses, and retransmission of lost datagrams. 
         [0030]    Other drawbacks of excessive buffering include a prolonged reaction time to recover from datagram loss, the need to forward large amounts of data between BTS when dealing with mobility events, and increased memory requirements for the network equipment participating in the end-to-end TCP communication. 
         [0031]    Buffer design is a delicate trade-off. On the one hand, providing sufficiently large buffers avoids draining outbound interfaces of intervening routers, while on the other hand queuing delay, which is a significant component of datagram transmission delay, grows proportionally with the aggregate buffer size. 
         [0032]    However, even an end-to-end TCP link with carefully dimensioned buffers may operate in a sub-optimal way when deployed over a layer-2 infrastructure with automatic retransmissions that are not notified to the transport layer. If the layer-2 infrastructure of, for instance, a wireless or cellular network masks packet losses via buffering and retransmission, the TCP sender is unaware of the reduced capacity of the receiver&#39;s channel: specifically, since no packets are lost, the sender&#39;s TCP stack is unable to detect the overloading of the network and will keep increasing its sender window, further adding to the buffered data at the BTS. 
         [0033]    The present invention is based inter alia on the insight that it is particularly advantageous in wireless and cellular edge networks to use the TCP receiver window to provide flow-control information about the last leg, i.e. the bottleneck link, to rate limit the end-to-end connection. In particular, the TCP receiver window may be used to signal the bandwidth in the last leg. In this way, undesired buffer growth, which would lead to increased end-to-end latency, can be avoided. 
         [0034]    TCP, as it is presently known, uses distinct mechanisms to perform congestion control and flow control, the former being intended for dealing with end-to-end bandwidth limitations, while the latter is used to signal data processing limitations at the receiver:
       Congestion control is used to ensure that network links are fairly shared between competing flows. The presence of network congestion is inferred from packet loss in a datagram flow, which occurs when intervening routers are forced to drop datagrams due to saturated buffers. Following the detection of packet loss, TCP reduces its congestion window, i.e. the maximum allowed amount of outstanding data, which tends to have the effect of cutting the sending rate to a half for each loss event.   Flow control is achieved by letting the TCP receiver inform the sender how much free space is available at its buffers via the “TCP window” parameter, whereupon the sender will stop transmitting whenever the amount of free buffer space at the destination reaches zero.       
 
         [0037]    Presently, in large delay×bandwidth product networks, the general concern is making the value of the receive window large enough so that the network can be used more efficiently by single TCP connections. Hence, it is presently quite uncommon to see TCP connections actually being rate-limited by TCP flow control as mediated by the TCP window parameter. 
         [0038]    The present invention proposes to use the TCP flow control mechanism as a means to enforce rate limiting for heavily buffered links, such as those found in wireless systems. The invention is based inter alia on the insight that it is advantageous to transparently modify the TCP window parameter in TCP acknowledgement datagrams to rate control the sender. 
         [0039]    In an embodiment, the TCP window (“WIN”) parameter of the TCP acknowledgement datagram that travels towards the data sender is adjusted with the minimum of the amount of buffer space available inside the BTS for the TCP session, and the amount of buffering space available in the recipient. This technique may be used in the recipient equipment and/or in the BTS. If this technique is used in cellular/wireless BTS when downloading data across the wireless channel and the wireless channel is the bottleneck link, the estimated channel quality towards the mobile node can be used further flow-control the sender. 
         [0040]    With reference to  FIG. 1 , an embodiment of the method of the invention will now be described, in which an exemplary network element is located between a fast connection and a slow connection. 
         [0041]    Such a situation might occur in a node  200  as illustrated in  FIG. 2 , which concentrates access or edge links  210  onto a core network  250 , such as a cellular base station, RNC, a DSLAM, or even a router aggregating slow-speed serial links. The skilled person will appreciate that the method equally applies to other types of network elements, when operating under similar circumstances. Hereinafter, data travelling from the network node  200  towards the slow link  210  will be designated “downstream” traffic. Conversely, data travelling from the slow link  210  to the node  200  will be designated “upstream” traffic. The originator of the TCP traffic under consideration is considered to be part of the network  250  that is connected to the node  200  via the fast link  240 . The TCP receiver  220  is schematically depicted in a purely exemplary way as a mobile node. 
         [0042]    Although the method is illustrated in  FIG. 1  and described below as a sequence of steps performed in a certain order, the skilled person will appreciate that this is done for illustrative purpose only, and that any of the steps may be carried out in a different order, unless it is clear from the description that a particular step can only take place after another one has been completed, or vice versa. 
         [0043]    In a first step  110 , an estimate of the amount of outstanding data is generated, preferably by comparing the TCP sequence (“SEQ”) numbers of the downstream traffic, schematically illustrated as packets in a downstream buffer  245 , with the TCP acknowledgement (“ACK”) numbers advertised in the upstream traffic, schematically illustrated as packets in an upstream buffer  215 . This value places a lower bound on the amount of data in flight between TCP sender and receiver  220 . The actual amount of data in flight also comprises data being buffered further upstream in the network  250 , beyond the view of the network element  200 . In the context of this first step  110 , the node  200  under consideration must be capable of performing per-connection accounting of the amount of un-acknowledged data, and the total amount of data in the buffer  245 . 
         [0044]    In a second step  120 , an assessment is made as to whether a buffer is overloaded. This assessment is preferably performed by comparing the estimated amount of outstanding data obtained in the first step  100  with a maximum amount of data the access or edge link can reasonably buffer without significant performance degradation. 
         [0045]    A comparison of the edge link&#39;s measured round-trip time (RTT) latency with a maximum acceptable RTT threshold value may be leveraged as an additional trigger for buffer occupancy detection in the second step  120 . This diagnostic is particularly revealing when the capacity of the radio channel is severely degraded by persistent propagation problems, in which case the layer-2 retransmission mechanisms will inflate the RTT without causing packet loss at the higher layers. 
         [0046]    In a third step  130 , the network element under consideration rewrites the TCP window (“WIN”) parameter of the TCP acknowledgement header present in all the upstream TCP packets, before relaying them in the direction of the sender, in order to implement the degree rate limitation that reduces the downstream TCP flow to a rate that can be persistently supported by the edge link. 
         [0047]    The new WIN value is preferably chosen so that the amount of outstanding data for each TCP connection does not outgrow a predetermined threshold. A particularly preferred predetermined threshold is the value defined as the buffer size at maximum RTT latency divided by the number of competing TCP connections. 
         [0048]    Even better granularity could be achieved by using more fine-grained accounting of the actual throughput of each TCP connection. 
         [0049]    In the context of this third step  130 , the node under consideration must be capable of performing a recalculation of the checksum for the TCP packets in which the WIN field has been modified. 
         [0050]    If the method of the present invention is implemented at the wireless or cellular BTS, certain cross-layer synergies may advantageously be exploited. The detailed information that is available in such nodes on the state of the wireless channel may be used to fine tune the rate of the sender to the very rate sustained by the physical network at any given time. In addition, layer-2 scheduling choices may be taken into account in executing the method according to the invention. Furthermore, application-layer information may indicate whether particular ongoing TCP conversation are latency-sensitive or not, and this information may be taken into account in the performance of the method according to the present invention. 
         [0051]    The method of the present invention can be implemented without radically increasing the requirements on the network node in terms of processing power. Devices such as BTS, which are connected at the edge of the network, usually serve limited numbers of simultaneous connections—typically less than approximately one thousand—the processing of which is not prohibitive for the deployment of the method of the present invention. 
         [0052]    It is an advantage of the method of the present invention that it provides flow control without simultaneously bringing on the unpredictable effects that tend to result from dropping datagrams. 
         [0053]    With reference to  FIG. 2 , it may be seen that the apparatus  200  depicted therein comprises a first interface  241  for interfacing with the first network link  240 , and a second interface  211  for interfacing with the second network link  210 . The skilled person will appreciate that these interfaces  241 ,  211  are only depicted schematically, and that they in fact imply all the necessary hardware and logic that enable a network node to communicate according to a certain set of protocols. The apparatus  200  further comprises a processor  230 , operatively connected to these interfaces  241 ,  211 , and more in particular to the aforementioned buffers  215 ,  245 , the actions of which correspond to those described in connection with  FIG. 1 . It shall be understood that the interconnection between the processor  230  and the data buffer  245  and the “ACK” buffer  215 , is precisely what allows the processor the carry out the process as described. 
         [0054]    Although the apparatus  200  is illustrated in  FIG. 2  as a BTS, it may in fact be implemented as a part of a BTS or a mobile node, or as a stand-alone functional network node. In case the apparatus  200  is integrated in a larger network node, it will be clear that the interfaces  241 ,  211  may be replaced by internal interfaces, which may include a physical backplane interface, a wire interface, a set of on-board microstrips, or even a logical interface implemented in software. 
         [0055]    It should also be noted that the processor  230  may be implemented as a micro-processor, or alternatively as one or more dedicated and/or programmable or configurable hardware components, including ASIC, FPGA, and similar technologies. 
         [0056]    The network depicted in  FIG. 3  is a variation of that depicted in  FIG. 2 . The mobile node  220  is connected to a BTS (not separately illustrated) via a slow link  210 . Without loss of generality, a wireless link  210  is assumed. Unlike the network of  FIG. 2 , the network of  FIG. 3  is configured to host logic representing an embodiment of the method according to the present invention at two distinct places: inside the NMIP Mobile Node Proxy (NMP)  200   a  implemented in association with the mobile node  220 , and inside the NMIP Network Proxy (NNP)  200   b  implemented in association with the BTS. 
         [0057]    All references to NMIP are to be understood in accordance with the definitions provided in US-A-2010135301, published on 3 Jun. 2010, the contents of which are hereby incorporated by reference for this purpose. 
         [0058]      FIGS. 4 and 5  illustrate uses of the invention in the context of the network depicted in  FIG. 3 . For the sake of illustration, and without loss of generality, the 3G wireless connection  210  between the mobile node (MN)  220  and the 3G BTS  200  is assumed to be the problematic link. Again without loss of generality, an alternative wireless local area network (WLAN) connection  210 ′ is assumed to be available, passing through WLAN Access Point (AP)  200 ′. 
         [0059]      FIG. 4  focuses on the case where the mobile node (MN)  220  is receiving data originating from the corresponding node (CN)  260  via the BTS  200  (downlink). 
         [0060]    A handover between 3G and WLAN is set to occur. Without the invention, the traffic is buffered at the 3G wireless link  210 ; in this case PPP is used to encapsulate traffic over the serial line towards the mobile phone  220 , at which point datagrams travel over the bottleneck wireless link  210 . 
         [0061]    The buffer associated with the 3G wireless link  210  is very large, typically more than 80 KB. Hence, as the NMIP handover is completed, more recently generated packets are delivered on the WLAN link  210 ′, concurrently with older packets still in the 3G interface&#39;s buffer. The TCP stack of the receiver  220  will designate this situation as an anomaly and reset the connection. 
         [0062]    The invention is used to control the amount of data in flight over the edge bottleneck  210  in order to limit the impact of buffering, and in particular to avoid undesirable consequences such as connection resets. 
         [0063]    When using the invention, the NNP  200   b  modifies the TCP window field of the TCP acknowledgements (ACKs) sent back from the MN  220  to the CN  260  to transparently enforce flow control on the problematic slower link  210 . As the TCP window value set by the receiver  220  is usually large (typically 2̂16 or more, as its buffer is rapidly flushed to the application), it can be reduced to reflect the amount of buffering that can be tolerated based on specified maximum desired latency. This operation is performed transparently: neither the MN  220  nor the CN  260  are aware of the datagram rewriting performed by the NNP  200   b  in the middle of the connection. 
         [0064]      FIG. 5  focuses on the case where the mobile node (MN)  220  is sending data intended for the corresponding node (CN)  260  towards the BTS  200  (uplink). 
         [0065]    Considering an analogous situation as the one described in the context of  FIG. 4 , we will assume that a handover between 3G and WLAN is set to occur. Without the invention, the traffic is buffered at the 3G wireless link  210 ; in this case the congestion prone sender buffer is situated at the MN  200 , whence datagrams travel over the bottleneck wireless link  210 . 
         [0066]    The invention is used to control the amount of data in flight over the edge bottleneck  210  in order to limit the impact of buffering. 
         [0067]    When using the invention, the NMP  200   a  modifies the TCP window field of the TCP acknowledgements (ACKs) sent back from the CN  260  to the MN  220  to transparently enforce flow control on the problematic slower link  210 . As the TCP window value set by the receiver  260  is usually large (typically 2̂16 or more, as its buffer is rapidly flushed to the application), it can be reduced to reflect the amount of buffering that can be tolerated based on specified maximum desired latency. This operation is performed transparently: neither the MN  220  nor the CN  260  are aware of the datagram rewriting performed by the NMP  200   a  in the middle of the connection. 
         [0068]    A person of skill in the art would readily recognize that steps of various above-described methods can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. 
         [0069]    The program storage devices may be, e.g., digital memories, magnetic storage media such as a magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods.