# Cascade Probability Control to Mitigate Bufferbloat under Multiple Real-World TCP Stacks

Authors: Hoang-Linh To; Thuyen Minh Thi; Won-Joo Hwang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Persistently full buffer problem, commonly known as bufferbloat, causes unnecessary additional latency and throughput degradation whenever congestion happens in Internet. Several proposed queue management schemes, with the debloat mission, are almost based on the modification of one-loop feedback control where the instability and bad transient behavior are still big challenges. In this paper, we present a cascade probability control scheme using margin optimal method to address such challenges under different kinds of real-world TCP stacks. Simulation results guarantee the measured round trip time tracking to a low value of delay (e.g., ≈180 ms under TCP Reno, and ≈130 ms under TCP Cubic) and ≈50% delay reduction in comparison to current deployed queue management schemes in network devices.


---

## Body

## 1. Introduction

Nowadays, interactive and delay-sensitive applications such as VoIP, teleconference, and gaming often perform more poorly than before. From the economic perspective, as the cost of memory has decreased over the past few years, memory with large capacity has been put into network devices such as routers. From the engineering perspective, as traditional analysis, a larger buffer results in less loss rate under congestion. However, recent studies aboutbufferbloat reveal some bad effects of a large buffer. It might destroy interactivity of transport control protocols (TCP) under load and often results in higher latency and lower throughput. Nagle [1] firstly drew attention to effects of infinite buffers in packet-switching networks. Thenbufferbloat termed by Gettys [2] firstly opens a research field to seriously reconsider the problem of large buffers and hidden “dark” buffers which can appear everywhere in Internet devices.Conceptually, bufferbloat is a phenomenon firstly realized in packet-switched networks, where excess buffering of packets might cause high latency and jitters, as well as reducing the overall network throughput. When a router device is equipped with a large buffer, it can become practically unusable for many delay-sensitive applications like voice calls, chat, or even web surfing. Traditional rule-of-thumb for setting buffer size in an Internet route is based on bandwidth-delay product (BDP), the product of a data link’s capacity and its round-trip time (RTT). The basic premise is that modern networking kit contains too much buffer memory. In [2], it is necessary to restrict bandwidth to improve latency and ping times. Despite this fundamental of sacrificing bandwidth for latency, most users keep asking how to fill their bandwidth quota and get good VoIP/games satisfaction. Long time ago and until now, researchers nonexhaustively attempt to reduce dropping packets by adding more and more buffering in routers. However, each dropped packet is essential for well-behaved functioning of TCP in congestion.Let us examine an interaction situation between TCP and bufferbloat [3]. TCP relies on timely congestion notification to adjust its transmission rate to the available bandwidth. And bufferbloat means that new arriving packets are continued to be buffered, instead of dropped due to large buffer size. It causes the queue to build up longer at the bottlenecks. Entering the dropping state, more packets are dropped than necessary which would shrink the transmission rates of the TCP sender. In particular, if several TCP applications are transmitting over the same congestion point, all flows will see drops at the same time. Therefore, all transmission rates would be reduced simultaneously, called TCP global synchronization. Certainly, suitable amount of buffering is helpful to improve the efficiency of data transmissions to smoothen bursts transmissions. Not dropping packets early enough, however, leads to increasing delays for interactive real-time communication since widely deployed congestion control algorithms only rely on packet loss as a signal for congestion. Not dropping packets in a timely fashion also prevents TCP applications from reacting properly to overload.Solutions to bufferbloat problem or debloat include queue management algorithms, which attempt to manage queue occupancy in passive or active ways. Passive queue management (PQM) such as Drop Tail is the most currently deployed queue management algorithm in Internet router devices. It drops packets when buffer queue is overflow, so because of this simplicity, it is widely used now. The weaknesses of PQM such as lock-out [4] and global synchronization [5] motivate to urgent needs of active queue management schemes (AQMs) deployment. Several AQMs have been deeply studied in the recent decade. The most popular one is random early detection (RED) [5] and its variants [6, 7] with the main idea of using a probabilistic approach to randomly dropping packets for congestion resolution. However, most of them require careful tuning for varying network conditions; otherwise they only work well under a few specific scenarios with default parameters [8]. Game theory approach, which is mainly dedicated to users (user-aware), has also been investigated to tackle this issue [9, 10]. Recently, a new AQM named controlled delay management (CoDel) [11] has been proposed to overcome weakness. Parameterless and easy deployment are two among strong points of CoDel. Even though, for larger RTTs and smaller bandwidths, CoDel has poor link utilization than RED and its variants, the next drop time of CoDel algorithm is derived by using the multiplicative decrease of square root of total number dropped packet counting, which needs more in-depth investigation for improvement.In this paper, we revisit AQM design problem from control theoretical perspective but consider cascade control to further optimize performance in user application level while considering bufferbloat phenomenon. Control theory is one of the most efficient tools for AQM to bring a better system stability due to well-developed control theory background. Several efforts have been recently put into this direction for RED [12], PID-AQM [13, 14], and controller design in state space [15, 16] or queuing modelling with the impatient customer feature [17] and so forth. Based on the fluid-flow approximation model for additive increase-multiplicative decrease (AIMD) phase of TCP, they converted dynamic equations of window size and queue length into system transfer function or state space by linearization methods. Then they designed the specific controller according to a closed-loop transfer function of the whole TCP system. We realize that these models almost design one-loop control for queue length only which creates some problem of difficult adjusting controller parameters and stability guarantee. One more loop with cascade design in control theory would improve their performance, especially the bad transient behaviour of current debloat schemes. So we consider a two-loop control scheme for AQM, which is mainly dedicated to bufferbloat issue. Dividing into an inner and an outer loop, the inner one adjusts window size based on changing of traffic and feedback window size at time t. The outer one’s mission is to adjust queue length based on feedback queue length value at time t. Each inner loop and outer loop are designed using two transfer functions which decomposed from the fluid-flow model and, therefore, have different controllers. One difficulty when considering this cascade design is the interaction in time-scale between two loops. We see that the inner loop operates in the transport layer while the outer one operates in link layer (faster than transport layer). The main motivation is that solving bufferbloat (large buffer) problem cannot be done without considering different network layers simultaneously. Therefore, one-loop control methods like Drop Tail, RED, or classical PID should not be adequate for bufferbloat mitigation. Our proposed CPC, with an inner and an outer loop, acts in both the transport layer (adjusting window size) and the link layer (adjusting queue length). To the best of our knowledge, our work is the first attempt to adapt cascade control method to bufferbloat research field. The weakness of this method is that more complexity is added because of two additional controllers. However, better performance results (shorter queuing delay and larger goodput) are achieved and we are going to see more details in Section 5. The performance metrics we evaluate consist of measured round-trip time, queue length at each instantaneous time, and goodput at TCP application layer which represents the users’ (clients) satisfaction level. The primary contributions are summarized as follows:(i)
We propose a cascade probability control (CPC) which has two control loops. The inner loop gets information from current window size and capacity of link while the outer loop is based on difference between average measured queue length and queue length reference value.(ii)
Section4 presents our proposed CPC controllers which are designed using the optimal margin method in frequency domain. The fast transient and stability in a wide frequency range can be achieved using this method.(iii)
We develop our own model to simulate CPC performance using the popular open-source software, OMNeT++. We also compare our proposal with Drop Tail and RED under three popular TCP stacks (e.g., Reno, Cubic, and FreeBSD.) using the real-world stack module network simulation cradle (NSC). Simulation results demonstrate that bufferbloat delay can be reduced significantly (≈50% compared to Drop Tail and RED) and well controlled by CPC (Section5).


## 2. Related Works

Several approaches to AQM, using control theory with the core linearized TCP model by Hollot et al. [12], were proposed and contributed a large portion to debloat field [18]. The original goal was to propose more concrete design guidelines for the RED parameters to improve stability and responsiveness; however, they also discover clearer understanding of RED’s behaviour with changing of network conditions such as round-trip times, offer load as number of TCP flows, and link capacities. Focusing on control theory viewpoint, we categorize them into three types: classical control, robust control, and fuzzy logic control.Classical PID controller based algorithms were designed as alternative AQM solutions to meet various specifications of the Internet using feedback control. Hollot et al. [12] analysed RED as I-controller and proposed two types, the proportional (P) controller and the proportional integral (PI) controller, for improving RED. The stable region of the control gain is determined for dynamic-RED (DRED) using the Routh stability test. In [19], dynamic-RED (DRED) was proposed, using a load-dependent probability to randomly discard packets when a buffer becomes congested. It maintains the average queue size close to a predetermined threshold but allows transient traffic bursts to be queued without unnecessary packet drops. Its main advantage is that we do not have to collect state information of individual flows. In [15], a feedback control model with PI controller has been recently proposed to improve link performance in wired communication networks.Robust control approach was also studied to improve classical control. The issue of large delay with large buffer in bufferbloat was addressed by DC-AQM algorithm based on internal mode compensation (IMC) principle, which is an example of robust control approach [20]. Using IMC controller derived, they tried to tune parameters Kp, Ki, and Kd of classical PID controller to reduce delay. To also contend with large delay, gain adaptive Smith predictor with PI controller (GAS-PI) in [21] was built to improve robustness. Then, in [13], a predictive PID controller is proposed for TCP/AQM. They used the generalized predictive control method to determine suitable values for Kp, Ki, and Kd so that they made the system more robust to changes in model parameters such as offered load and round-trip time.Fuzzy control RED (FCRED) was proposed in [6]. It consists of a fuzzy controller adjusting the Pmax parameter of the RED algorithm. The fuzzy controller includes three parts: the fuzzification unit followed by the fuzzy-interference engine with fuzzy-rule base and finally a defuzzification unit. The fuzzification module maps the input values to be controlled to a fuzzy set (i.e., membership functions). The fuzzy rule base provides the connection between the input signals and the appropriate output variable. Fuzzy logic rules are constructed based on trial-and-error, which needs the knowledge and experience of domain-expert in TCP congestion control. Moreover, fuzzy logic-based AQM schemes are sometimes not distributed and hard to implement.Briefly, it can be seen that, as the years progress, the main direction has been to more and more sophisticated robust control techniques, combined with some classical techniques in control theory as well. In many cases, the linear TCP model in [12] continues to be at the core of control theoretic AQM algorithms. Cascade control, however, has not been considered yet in development of AQM. In this paper, we contribute to the AQM debloat research trends by using cascade control to address large delay bufferbloat issue. Our approach uses two controllers which are connected in cascade style. Our CPC results obtained from theoretical Matlab or simulation using OMNeT++ and the integrated NSC tool are so promising in fast transient behavior and stability.


## 3. System Model

The TCP/AQM fluid-flow model described by nonlinear differential equations has been extensively studied in particular AQM routers interacting with TCP sources (e.g., [12, 13, 16, 20]). Until now, they can capture the additive increase-multiplicative decrease (AIMD) feature from TCP [22], without slow start and time-out mechanisms. However, this lacking only affects initial start-up of the system. Once the system reaches the stable point, the differential equations solver is able to track changes in the network well [23]:(1)w˙t=1Rt-wt2wt-τRt-τpt-τ;q˙t=NwtRt-Cl,where(i)
Cl is the transmission capacity of link l (packets/sec);(ii)
R is the round-trip time (sec); R(t)=Tp+q(t)/Cl with Tp being the fixed propagation delay;(iii)
N is the number of TCP flows;(iv)
τ is feedback delay (sec).The operating point(w0,q0,p0) of TCP model (1) can be derived at w˙=0 and q˙=0 as follows:(2)w02p0=2,w0=R0ClN,R0=Tp+q0Cl.Doing linearization of the above TCP queue model around operating point(w0,q0,p0) (details in [12]), with δw˙=w-w0, we have a linearized small signal model of TCP/AQM:(3)δw˙t=-δwt2NR02Cl-δpt-R0R0Cl22N2,δq˙t=δwtNR0-δqt1R0.Convert (3) using Laplace transformation as follows:(4)s×ws=-ws2NR02Cl-pse-sR0·R0Cl22N2,s×qs=wsNR0-qs1R0.So the TCP/AQM system transfer functions in Laplace domain are(5)Ps=Ptcpwins·Pqueues·e-sR0=wsps·qsws=As+B·Cs+De-sR0,where A=R0Cl2/(2N2); B=2N/(R02Cl); C=N/R0; and D=1/R0.


## 4. Cascade Probability Control (CPC)

In literature, single loop control for AQM was often studied due to less complexity and low oscillation but did not behave well in case of disturbances (e.g., bandwidth fluctuation and bursty traffic) or bad transient. Cascade control comes to rescue and achieves fast rejection of disturbance before it affects the main system model. In this section, we propose a cascade probability control (CPC) to improve dynamic of the open-loop system transfer function (5) which consists of two subsystems, window size and queue length control. The inner loop adjusts the window size by an inner controller Cwin, while the outer loop receives queue length information at time t and uses an outer controller Cqueue to maintain q(t) (Figure 1). The final goal is to reach a stable value of queue length, so that bufferbloat phenomenon can be mitigated. Moreover, the margin optimal method for controller design also brings a fast transient behavior to our system.Figure 1
Block diagram: proposed cascade probability control scheme.


### 4.1. CPC Controllers

We design two PI controllers for simplicity and for reducing number of parameters: the inner controls dropping probabilityp2 based on traffic information and the outer controls dropping probability p1 based on difference between measured average queue length and reference queue length. We outline our proposed design framework in Figure 1 and would compete with the adaptive weight PID approach in [24].


#### 4.1.1. Inner Loop

An important design requirement is that the inner loop controller should behave quickly. From (5), the inner control objective is a linear first-order type: Ptcpwin(s)=kI/(1+Ts), where kI=A/B and T=1/B. Hence, we design an integral I-controller for inner loop:(6)Cwins=1TCwins.(i)
The close-loop transfer function of the inner loop is(7)Is=wsp1s=PtcpwinsCwins1+PtcpwinsCwins=kITCwins1+Ts+kI.(ii)
TheI(s) transfer function is converted into frequency domain, with ω being frequency:(8)Ijω=kIkI-TCwinT·ω22+ω·TCwin2,⟺Ijω2=kI2kI2+TCwin2-2kI·TCwinTω2+TCwin2·T2ω4.One of quality requirements to closed-loop control system, which is represented byI(s), is that the output is the same as the input signal or the controller Cwin should bring Ijω=1, ∀ω, which can be called margin optimal method. However, due to several reasons of real system, that requirement is rarely satisfied for all frequencies ω. An acceptable design is that Ijω≈1, in a wide band of low frequencies ω. Hence, we propose choosing TCwin such that TCwin2-2kI·TCwinT=0 or TCwin=2kIT. This close-form expression of TCwin is used to make decision for controller Cwin.


#### 4.1.2. Outer Loop

(i)
The close-loop transfer function of the outer loop is(9)Os=qsqref=IsPqueuesCqueues1+IsPqueuesCqueues.
(ii)
The outer control objective into zero-pole form is(10)IsPqueues=kITCwins1+Ts+kI·Cs+D=kO1+T1Os1+T2Os1+T3Os,
wherekO=C/D; T1OT2O=TTCwin/kI; T1O+T2O=TCwin/kI; T3O=1/D.For the outer loop, the objective function is linear third-order type, due to inclusion ofI(s). Hence, we choose proportional-integral-derivative PID controller by using the same method at the inner loop design, or Ojω≈1:(11)Cqueues=kpO1+1TiOs+TdOs,with kpO=T1O+T2O/2kOT3O, TiO=T1O+T2O, TdO=T1O·T2O/T1O+T2O.


### 4.2. CPC Numerical Analysis

The proposed CPC controllers can be easily verified by using example parameters from [12] which consist of N=60, Cl=3750 (packets/sec), and R0=0.246 (sec). According to the above analysis, the inner controller Cwin=0.000291/s, while the outer controller Cqueue=0.12811+1/(3.78s)+1.89s.Figure2(a) presents the queue length output in case of 20 packets queue reference. The CPC manual tune mode uses our above designed controllers, while the CPC autotune mode usespidtune function of Matlab. Firstly, in comparison to RED which can be modeled as a single-loop I-type controller, one advantage of CPC is that fast transient behavior can be achieved. Transient behavior is a major issue of current solutions to bufferbloat. Fast transient means that we can reach the queue length reference quickly in response to the input change of dynamic systems.Figure 2
Stability analysis of CPC.
(a)
Step response performance
(b)
Bode diagram and pole-zero mapSecondly, crossover frequency is a criterion to assess a control system’s operation ability in a wide range of frequency. The higher the crossover frequency is, the better stability at which the system can operate is. Let us denote crossover frequency asωc. If we choose a frequency higher than ωc, system would be not stable anymore. The left-hand side of Figure 2(b) is Bode diagram phase-margin of the system. It informs about ωc information of our proposed CPC scheme. Specifically, CPC manual tune has ωc=2.37(rad/s) which is the highest value, while ωc=0.627(rad/s) for CPC autotune and ωc=0.261(rad/s) for RED. Therefore, CPC can operate better in a wider range of frequency.Finally, using Nyquist stability criterion, the closed-loop transfer function of the outer loopO(s) is determined by the values of its poles. It states that, for stability, the real part of all poles must be negative or the poles are in the left half-plane of pole-zero map. The right-hand side of Figure 2(b) shows us the distribution of zeros and poles for three schemes. Clearly, all the poles have the real part negative which strongly demonstrates CPC’s stability.In summary, CPC scheme can achieve fast transient, high crossover frequency and still stable. Those motivate us to conduct simulations about CPC behavior to bufferbloat under multiple real-world TCP stacks.


## 4.1. CPC Controllers

We design two PI controllers for simplicity and for reducing number of parameters: the inner controls dropping probabilityp2 based on traffic information and the outer controls dropping probability p1 based on difference between measured average queue length and reference queue length. We outline our proposed design framework in Figure 1 and would compete with the adaptive weight PID approach in [24].


### 4.1.1. Inner Loop

An important design requirement is that the inner loop controller should behave quickly. From (5), the inner control objective is a linear first-order type: Ptcpwin(s)=kI/(1+Ts), where kI=A/B and T=1/B. Hence, we design an integral I-controller for inner loop:(6)Cwins=1TCwins.(i)
The close-loop transfer function of the inner loop is(7)Is=wsp1s=PtcpwinsCwins1+PtcpwinsCwins=kITCwins1+Ts+kI.(ii)
TheI(s) transfer function is converted into frequency domain, with ω being frequency:(8)Ijω=kIkI-TCwinT·ω22+ω·TCwin2,⟺Ijω2=kI2kI2+TCwin2-2kI·TCwinTω2+TCwin2·T2ω4.One of quality requirements to closed-loop control system, which is represented byI(s), is that the output is the same as the input signal or the controller Cwin should bring Ijω=1, ∀ω, which can be called margin optimal method. However, due to several reasons of real system, that requirement is rarely satisfied for all frequencies ω. An acceptable design is that Ijω≈1, in a wide band of low frequencies ω. Hence, we propose choosing TCwin such that TCwin2-2kI·TCwinT=0 or TCwin=2kIT. This close-form expression of TCwin is used to make decision for controller Cwin.


### 4.1.2. Outer Loop

(i)
The close-loop transfer function of the outer loop is(9)Os=qsqref=IsPqueuesCqueues1+IsPqueuesCqueues.
(ii)
The outer control objective into zero-pole form is(10)IsPqueues=kITCwins1+Ts+kI·Cs+D=kO1+T1Os1+T2Os1+T3Os,
wherekO=C/D; T1OT2O=TTCwin/kI; T1O+T2O=TCwin/kI; T3O=1/D.For the outer loop, the objective function is linear third-order type, due to inclusion ofI(s). Hence, we choose proportional-integral-derivative PID controller by using the same method at the inner loop design, or Ojω≈1:(11)Cqueues=kpO1+1TiOs+TdOs,with kpO=T1O+T2O/2kOT3O, TiO=T1O+T2O, TdO=T1O·T2O/T1O+T2O.


## 4.1.1. Inner Loop

An important design requirement is that the inner loop controller should behave quickly. From (5), the inner control objective is a linear first-order type: Ptcpwin(s)=kI/(1+Ts), where kI=A/B and T=1/B. Hence, we design an integral I-controller for inner loop:(6)Cwins=1TCwins.(i)
The close-loop transfer function of the inner loop is(7)Is=wsp1s=PtcpwinsCwins1+PtcpwinsCwins=kITCwins1+Ts+kI.(ii)
TheI(s) transfer function is converted into frequency domain, with ω being frequency:(8)Ijω=kIkI-TCwinT·ω22+ω·TCwin2,⟺Ijω2=kI2kI2+TCwin2-2kI·TCwinTω2+TCwin2·T2ω4.One of quality requirements to closed-loop control system, which is represented byI(s), is that the output is the same as the input signal or the controller Cwin should bring Ijω=1, ∀ω, which can be called margin optimal method. However, due to several reasons of real system, that requirement is rarely satisfied for all frequencies ω. An acceptable design is that Ijω≈1, in a wide band of low frequencies ω. Hence, we propose choosing TCwin such that TCwin2-2kI·TCwinT=0 or TCwin=2kIT. This close-form expression of TCwin is used to make decision for controller Cwin.


## 4.1.2. Outer Loop

(i)
The close-loop transfer function of the outer loop is(9)Os=qsqref=IsPqueuesCqueues1+IsPqueuesCqueues.
(ii)
The outer control objective into zero-pole form is(10)IsPqueues=kITCwins1+Ts+kI·Cs+D=kO1+T1Os1+T2Os1+T3Os,
wherekO=C/D; T1OT2O=TTCwin/kI; T1O+T2O=TCwin/kI; T3O=1/D.For the outer loop, the objective function is linear third-order type, due to inclusion ofI(s). Hence, we choose proportional-integral-derivative PID controller by using the same method at the inner loop design, or Ojω≈1:(11)Cqueues=kpO1+1TiOs+TdOs,with kpO=T1O+T2O/2kOT3O, TiO=T1O+T2O, TdO=T1O·T2O/T1O+T2O.


## 4.2. CPC Numerical Analysis

The proposed CPC controllers can be easily verified by using example parameters from [12] which consist of N=60, Cl=3750 (packets/sec), and R0=0.246 (sec). According to the above analysis, the inner controller Cwin=0.000291/s, while the outer controller Cqueue=0.12811+1/(3.78s)+1.89s.Figure2(a) presents the queue length output in case of 20 packets queue reference. The CPC manual tune mode uses our above designed controllers, while the CPC autotune mode usespidtune function of Matlab. Firstly, in comparison to RED which can be modeled as a single-loop I-type controller, one advantage of CPC is that fast transient behavior can be achieved. Transient behavior is a major issue of current solutions to bufferbloat. Fast transient means that we can reach the queue length reference quickly in response to the input change of dynamic systems.Figure 2
Stability analysis of CPC.
(a)
Step response performance
(b)
Bode diagram and pole-zero mapSecondly, crossover frequency is a criterion to assess a control system’s operation ability in a wide range of frequency. The higher the crossover frequency is, the better stability at which the system can operate is. Let us denote crossover frequency asωc. If we choose a frequency higher than ωc, system would be not stable anymore. The left-hand side of Figure 2(b) is Bode diagram phase-margin of the system. It informs about ωc information of our proposed CPC scheme. Specifically, CPC manual tune has ωc=2.37(rad/s) which is the highest value, while ωc=0.627(rad/s) for CPC autotune and ωc=0.261(rad/s) for RED. Therefore, CPC can operate better in a wider range of frequency.Finally, using Nyquist stability criterion, the closed-loop transfer function of the outer loopO(s) is determined by the values of its poles. It states that, for stability, the real part of all poles must be negative or the poles are in the left half-plane of pole-zero map. The right-hand side of Figure 2(b) shows us the distribution of zeros and poles for three schemes. Clearly, all the poles have the real part negative which strongly demonstrates CPC’s stability.In summary, CPC scheme can achieve fast transient, high crossover frequency and still stable. Those motivate us to conduct simulations about CPC behavior to bufferbloat under multiple real-world TCP stacks.


## 5. Simulation Results

In this section, we implement the proposed scheme in simulator to show bufferbloat phenomenon and the advantages of CPC scheme in comparison to traditional AQMs. We choose the dumbbell topology according to AQM guidelines [25].


### 5.1. Simulation Setup

We develop our own simulation model to verify the proposed CPC scheme using the popular OMNeT++ framework [26]. The chosen topology in Figure 3 represents a dumbbell network which is suitable for evaluating a queue management scheme. Three clients simultaneously send 200×3 (MByte) data to servers through intermediate routers. The advertised window’s server is set up to infinity so that it does not limit the sender’s speed. To create an artificial bottleneck, we set up the “high-to-slow” link bandwidth where the bandwidth of links from clients to router 1 are high at 1000 (Mbps) and the bandwidth of the link from router 1 to others is slow at 2 (Mbps). RTT can be measured at each server, while queue size is monitored at the congestion point. Bursty traffic is generated using a generator traffic model inside each client from OMNeT++. At the first glance, we vary the buffer size parameter to show the simulation model working toward bufferbloat phenomenon existence. Then we compare the CPC scheme with two types of traditional queue discipline which are operating inside the current Internet, Drop Tail, and RED, in terms of queue size and round-trip time or latency. Some main parameters are summarized in Table 1.Table 1
Simulation parameters.
Name
Value
Unit
Maximum buffer size
500
Packets
Desired queue length
20
Packets
Target RTT delay
200
msFigure 3
Simulation topology.


### 5.2. Bufferbloat Existence

#### 5.2.1. Experiment

To demonstrate clearly bufferbloat existence, we exploit the real-time response under load (RRUL) test specification in [27]. This test puts a network under worst case conditions and then measures latency responsiveness and other relative performances of TCP and UDP streams of varying rates, while under that load. Then, Toke Hoiland-Jorgensen produces a wrapper for netperf tool to implement test cases such as HTTP, VoIP, and FTP under RRUL. In this paper, we conduct the experiment test from our computer-client located in South Korea to two servers which are mainly dedicated for bufferbloat testing, demo.tohojo.dk, and snapon.lab.bufferbloat.net, respectively. Figures 4(a) and 4(b) present our ping results when we run experiment in 300 seconds and sampling period of 0.2 second. Ping delay for both cases are approximately 300 (ms), while 100→150 (ms) is an acceptable range for toll-quality voice and delay-sensitive applications. Once again, we see that debloat solutions were not deployed in our current experiment routers.Figure 4
Bufferbloat existence: an experiment from Korea to two servers.
(a)
Server: demo.tohojo.dk
(b)
Server: snapon.lab.bufferbloat.dk


#### 5.2.2. Simulation

As mentioned before, the main reason for bufferbloat is the unmanaged large buffer at bottleneck links that could damage TCP’s congestion control/avoidance mechanisms. Hence, a big buffer is necessary to reproduce this phenomenon. In the current Internet, buffers are measured using bytes. OMNeT++ simulation framework, however, uses packets to sizing buffers. The choice of the largest buffer size is approximately 8x BDP product (packets) [11]. Let us denote buffer size as bu. With bottleneck bandwidth 2Mbps and propagation delay 300ms, we have the results when following the traditional rule-of-thumb for buffer-size:(12)bubyte=BDP=BW×Delay=2.106×300.10-3=600kbit=75kbyte.According to [28], one TCP packet size or maximum transmission unit (MTU) is 1500 (bytes):(13)bupackets=bubyteMTU=75.1031500=50packets.Therefore, in this simulation, we vary the buffer sizes such as50, 100, 300, and 500 being the largest one (packets) under different TCP congestion-avoidance algorithms with Drop Tail queuing discipline. Firstly, Figure 5(a) shows the effects on queue evolution and round-trip time, under TCP Reno, the most used in Windows and some Linux operating systems (OS). We obtain the saw-tooth result at both queue size and RTT graph because of TCP Reno’s characteristics while others do not have saw-tooth type graphs. From Figure 5(a), bufferbloat issue can also be clearly recognized. If buffer size is very large (500 packets), RTT will become pretty large (≈2.5 sec) while queue size grows until it reaches the threshold of buffer, which is the main drawback of Drop Tail. Several AQMs come to rescue at this point. The phenomenon once again appears in TCP FreeBSD, the popular algorithm in Unix OS that does not own a saw-tooth graph. Figure 5(b), which is simulated under TCP FreeBSD, also presents a long delay (≈2.8 sec) when buffer is too large. Those results confirm bufferbloat happening at different OS whenever a long buffer queue is built up.Figure 5
Bufferbloat existence and varied buffer sizes.
(a)
TCP Reno + Drop Tail
(b)
TCP Free BSD + Drop Tail


### 5.3. CPC Performance under TCP Real-World Stacks

Next, we conduct more necessary simulations to compare our CPC scheme performance with traditional popular queuing disciplines Drop Tail and RED. We monitor the queue size at the bottleneck point (router 1), the round-trip time of packets, and goodput at each destination server. Goodput is the application level throughput, that is, the number of user information bits delivered by the network to a certain destination per unit of time. For example, if a file is transferred, the goodput that the user experiences corresponds to the file size in bits divided by the file transfer time. The goodput is always lower than the throughput (the gross bit rate that is transferred physically), which generally is lower than the channel capacity or bandwidth. Goodput monitoring is necessary to evaluate the actual good throughput performance inside networks. We also do these comparisons under several different TCP versions which include Reno, Cubic, and FreeBSD. The first one is mainly implemented in Windows and some Linux Kernels. The second one is improved and deployed in newer versions of Linux Kernel until now. The last one is popularly implemented under Unix Kernel. All the following results are derived when we set the largest buffer size at congestion point,500 (packets). In the following, we summarize the background knowledge of those three TCP versions before going into detailed results of bufferbloat delay reduction and goodput stability.


#### 5.3.1. Under Reno Real-World Stack

For each connection, TCP maintains a congestion windowcwnd, limiting the total number of packets that may be transmitted at a time. TCP Reno algorithm uniqueness is the fast recovery phase. That means when a packet loss is detected by receiving three duplicate ACKs, Reno will halve cwnd and set slow start threshold ssthresh equal to the new cwnd value, perform a fast retransmit, and enter a phase called fast recovery. It also reduces congestion window to one maximum segment size (MSS) on a timeout event. Congestion-avoidance phase, as usual, exploits AIMD to control cwnd based on packet loss notification. In fact, compound TCP stack for Windows and Linux also increases AIMD window as TCP Reno does [29].In Figure6(a), queue size and RTT according to three queue management schemes are presented. With 500 packets of buffer, Reno and Drop Tail show a saw-tooth graph where the highest RTT is high up to 2.8 (sec). With the target RTT being 200ms, RED under Reno presents better performance than Drop Tail, RTT (RED) ≈400ms, by early dropping some packets to keep the queue size small. CPC under Reno, however, achieves a much better performance while keeping the lowest queue size and lowest RTT (CPC) ≈180ms, which is acceptable for delay-sensitive applications. The reason is that CPC continuously controls the next queue size value according to traffic changes and adjusts the current queue size at the outer loop so that it indirectly affects RTT value of packets.Figure 6
TCP Reno + Drop Tail/RED/CPC.
(a)
Round-trip time
(b)
GoodputsNext, Figure6(b) shows us goodput (bit per second) statistic results under TCP Reno according to four different buffer sizes. These graphs use box-and-whisker plot that can effectively depict groups of numerical data through their quartiles. For instance, the center red line of box represents the median goodput value; the width of box shows the spread degree of goodput statistic data; for example, bigger box and longer whisker mean that statistic data oscillate so much or are unstable. Regardless of different buffer size, under TCP Reno, the median of goodput is nearly the same, around 8×104 (bps). The CPC scheme, however, achieves the best stability performance compared to the others because the width of CPC boxes is smallest.


#### 5.3.2. Under Cubic Real-World Stack

Cubic is a congestion control protocol for TCP and the current default TCP algorithm in the Linux kernel [30]. It replaces the default TCP congestion control algorithm described above with a different algorithm based on a cubic function of the time since the last congestion event. Instead of adjusting the congestion window as a function of previous values as each packet is acknowledged, the Cubic algorithm recomputes the congestion window size at each step using the cubic function calibrated. This results in the congestion window responding quickly to changes in available bandwidth. The TCP Cubic is a less aggressive and more systematic derivative of binary increase congestion (BIC). We exploit NSC module to incorporate real-world TCP/IP network stacks into our simulation model. NSC owns TCP Cubic (version 2.6.26) stack which allows us to test our CPC queuing scheme performance.In Figure7(a), we can see again that CPC scheme seems to achieve the best performance, keeping lower queue size and RTT only 130ms, which is acceptable for delay-sensitive traffic. However, the RTT gaps between three schemes are not so much because of the aggression property of TCP Cubic. While RED shows large variability of RTT, the others keep RTT much more stable.Figure 7
TCP Cubic + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsGoodput performance in Cubic also reveals TCP Cubic’s aggression. When buffers are at small values (50 and 100 packets), three boxes’ widths are nearly similar. That means that CPC can achieve the same goodput as the others if we use small buffers. Moreover, increasing the buffer size to 300 and 500 packets makes CPC’s goodput more stable, with the smaller box width. These figures demonstrate that CPC can mitigate the bufferbloat—big buffers problem pretty well.


#### 5.3.3. Under FreeBSD Real-World Stack

Finally, we investigate one more popular TCP stack. FreeBSD is a free Unix-like operating system, historically standing for “Berkeley Unix.” It was chosen as a BSD-derived TCP/IP stack that is widely used and has had much development. There are several applications that are directly based on FreeBSD, an example being the famous instant messenger WhatsApp. Much of FreeBSD became an integral part of other operating systems such as Apple’s OS X. The integrated NSC module for OMNeT++ also incorporates TCP FreeBSD real-world stack so that we can turn it on for demonstrating CPC’s performance under FreeBSD. Figure8(a) presents queue size and RTT results for three disciplines. We can see that our proposal CPC achieves a better performance, keeping lower queue size and RTT (FreeBSD) around 150ms.Figure 8
TCP FreeBSD + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsSo far, the proposed CPC scheme achieved good goodput performance under TCP Reno and Cubic stacks. We, however, find another interesting fact of CPC when performing under TCP FreeBSD stack. In Figure8(b), we can conclude two things. Firstly, the width of CPC goodput’s boxes in four buffer cases is almost bigger than the other schemes. It means the CPC stability under FreeBSD stack’s environment is not good. Secondly, our CPC scheme, however, still achieves more goodput than others. Specifically, in four cases, the system’s goodput can reach 10 (kbps) using CPC while it can reach only 4 (kbps) and 2 (kbps) under either small or large buffer using Drop Tail or RED schemes, respectively.


## 5.1. Simulation Setup

We develop our own simulation model to verify the proposed CPC scheme using the popular OMNeT++ framework [26]. The chosen topology in Figure 3 represents a dumbbell network which is suitable for evaluating a queue management scheme. Three clients simultaneously send 200×3 (MByte) data to servers through intermediate routers. The advertised window’s server is set up to infinity so that it does not limit the sender’s speed. To create an artificial bottleneck, we set up the “high-to-slow” link bandwidth where the bandwidth of links from clients to router 1 are high at 1000 (Mbps) and the bandwidth of the link from router 1 to others is slow at 2 (Mbps). RTT can be measured at each server, while queue size is monitored at the congestion point. Bursty traffic is generated using a generator traffic model inside each client from OMNeT++. At the first glance, we vary the buffer size parameter to show the simulation model working toward bufferbloat phenomenon existence. Then we compare the CPC scheme with two types of traditional queue discipline which are operating inside the current Internet, Drop Tail, and RED, in terms of queue size and round-trip time or latency. Some main parameters are summarized in Table 1.Table 1
Simulation parameters.
Name
Value
Unit
Maximum buffer size
500
Packets
Desired queue length
20
Packets
Target RTT delay
200
msFigure 3
Simulation topology.


## 5.2. Bufferbloat Existence

### 5.2.1. Experiment

To demonstrate clearly bufferbloat existence, we exploit the real-time response under load (RRUL) test specification in [27]. This test puts a network under worst case conditions and then measures latency responsiveness and other relative performances of TCP and UDP streams of varying rates, while under that load. Then, Toke Hoiland-Jorgensen produces a wrapper for netperf tool to implement test cases such as HTTP, VoIP, and FTP under RRUL. In this paper, we conduct the experiment test from our computer-client located in South Korea to two servers which are mainly dedicated for bufferbloat testing, demo.tohojo.dk, and snapon.lab.bufferbloat.net, respectively. Figures 4(a) and 4(b) present our ping results when we run experiment in 300 seconds and sampling period of 0.2 second. Ping delay for both cases are approximately 300 (ms), while 100→150 (ms) is an acceptable range for toll-quality voice and delay-sensitive applications. Once again, we see that debloat solutions were not deployed in our current experiment routers.Figure 4
Bufferbloat existence: an experiment from Korea to two servers.
(a)
Server: demo.tohojo.dk
(b)
Server: snapon.lab.bufferbloat.dk


### 5.2.2. Simulation

As mentioned before, the main reason for bufferbloat is the unmanaged large buffer at bottleneck links that could damage TCP’s congestion control/avoidance mechanisms. Hence, a big buffer is necessary to reproduce this phenomenon. In the current Internet, buffers are measured using bytes. OMNeT++ simulation framework, however, uses packets to sizing buffers. The choice of the largest buffer size is approximately 8x BDP product (packets) [11]. Let us denote buffer size as bu. With bottleneck bandwidth 2Mbps and propagation delay 300ms, we have the results when following the traditional rule-of-thumb for buffer-size:(12)bubyte=BDP=BW×Delay=2.106×300.10-3=600kbit=75kbyte.According to [28], one TCP packet size or maximum transmission unit (MTU) is 1500 (bytes):(13)bupackets=bubyteMTU=75.1031500=50packets.Therefore, in this simulation, we vary the buffer sizes such as50, 100, 300, and 500 being the largest one (packets) under different TCP congestion-avoidance algorithms with Drop Tail queuing discipline. Firstly, Figure 5(a) shows the effects on queue evolution and round-trip time, under TCP Reno, the most used in Windows and some Linux operating systems (OS). We obtain the saw-tooth result at both queue size and RTT graph because of TCP Reno’s characteristics while others do not have saw-tooth type graphs. From Figure 5(a), bufferbloat issue can also be clearly recognized. If buffer size is very large (500 packets), RTT will become pretty large (≈2.5 sec) while queue size grows until it reaches the threshold of buffer, which is the main drawback of Drop Tail. Several AQMs come to rescue at this point. The phenomenon once again appears in TCP FreeBSD, the popular algorithm in Unix OS that does not own a saw-tooth graph. Figure 5(b), which is simulated under TCP FreeBSD, also presents a long delay (≈2.8 sec) when buffer is too large. Those results confirm bufferbloat happening at different OS whenever a long buffer queue is built up.Figure 5
Bufferbloat existence and varied buffer sizes.
(a)
TCP Reno + Drop Tail
(b)
TCP Free BSD + Drop Tail


## 5.2.1. Experiment

To demonstrate clearly bufferbloat existence, we exploit the real-time response under load (RRUL) test specification in [27]. This test puts a network under worst case conditions and then measures latency responsiveness and other relative performances of TCP and UDP streams of varying rates, while under that load. Then, Toke Hoiland-Jorgensen produces a wrapper for netperf tool to implement test cases such as HTTP, VoIP, and FTP under RRUL. In this paper, we conduct the experiment test from our computer-client located in South Korea to two servers which are mainly dedicated for bufferbloat testing, demo.tohojo.dk, and snapon.lab.bufferbloat.net, respectively. Figures 4(a) and 4(b) present our ping results when we run experiment in 300 seconds and sampling period of 0.2 second. Ping delay for both cases are approximately 300 (ms), while 100→150 (ms) is an acceptable range for toll-quality voice and delay-sensitive applications. Once again, we see that debloat solutions were not deployed in our current experiment routers.Figure 4
Bufferbloat existence: an experiment from Korea to two servers.
(a)
Server: demo.tohojo.dk
(b)
Server: snapon.lab.bufferbloat.dk


## 5.2.2. Simulation

As mentioned before, the main reason for bufferbloat is the unmanaged large buffer at bottleneck links that could damage TCP’s congestion control/avoidance mechanisms. Hence, a big buffer is necessary to reproduce this phenomenon. In the current Internet, buffers are measured using bytes. OMNeT++ simulation framework, however, uses packets to sizing buffers. The choice of the largest buffer size is approximately 8x BDP product (packets) [11]. Let us denote buffer size as bu. With bottleneck bandwidth 2Mbps and propagation delay 300ms, we have the results when following the traditional rule-of-thumb for buffer-size:(12)bubyte=BDP=BW×Delay=2.106×300.10-3=600kbit=75kbyte.According to [28], one TCP packet size or maximum transmission unit (MTU) is 1500 (bytes):(13)bupackets=bubyteMTU=75.1031500=50packets.Therefore, in this simulation, we vary the buffer sizes such as50, 100, 300, and 500 being the largest one (packets) under different TCP congestion-avoidance algorithms with Drop Tail queuing discipline. Firstly, Figure 5(a) shows the effects on queue evolution and round-trip time, under TCP Reno, the most used in Windows and some Linux operating systems (OS). We obtain the saw-tooth result at both queue size and RTT graph because of TCP Reno’s characteristics while others do not have saw-tooth type graphs. From Figure 5(a), bufferbloat issue can also be clearly recognized. If buffer size is very large (500 packets), RTT will become pretty large (≈2.5 sec) while queue size grows until it reaches the threshold of buffer, which is the main drawback of Drop Tail. Several AQMs come to rescue at this point. The phenomenon once again appears in TCP FreeBSD, the popular algorithm in Unix OS that does not own a saw-tooth graph. Figure 5(b), which is simulated under TCP FreeBSD, also presents a long delay (≈2.8 sec) when buffer is too large. Those results confirm bufferbloat happening at different OS whenever a long buffer queue is built up.Figure 5
Bufferbloat existence and varied buffer sizes.
(a)
TCP Reno + Drop Tail
(b)
TCP Free BSD + Drop Tail


## 5.3. CPC Performance under TCP Real-World Stacks

Next, we conduct more necessary simulations to compare our CPC scheme performance with traditional popular queuing disciplines Drop Tail and RED. We monitor the queue size at the bottleneck point (router 1), the round-trip time of packets, and goodput at each destination server. Goodput is the application level throughput, that is, the number of user information bits delivered by the network to a certain destination per unit of time. For example, if a file is transferred, the goodput that the user experiences corresponds to the file size in bits divided by the file transfer time. The goodput is always lower than the throughput (the gross bit rate that is transferred physically), which generally is lower than the channel capacity or bandwidth. Goodput monitoring is necessary to evaluate the actual good throughput performance inside networks. We also do these comparisons under several different TCP versions which include Reno, Cubic, and FreeBSD. The first one is mainly implemented in Windows and some Linux Kernels. The second one is improved and deployed in newer versions of Linux Kernel until now. The last one is popularly implemented under Unix Kernel. All the following results are derived when we set the largest buffer size at congestion point,500 (packets). In the following, we summarize the background knowledge of those three TCP versions before going into detailed results of bufferbloat delay reduction and goodput stability.


### 5.3.1. Under Reno Real-World Stack

For each connection, TCP maintains a congestion windowcwnd, limiting the total number of packets that may be transmitted at a time. TCP Reno algorithm uniqueness is the fast recovery phase. That means when a packet loss is detected by receiving three duplicate ACKs, Reno will halve cwnd and set slow start threshold ssthresh equal to the new cwnd value, perform a fast retransmit, and enter a phase called fast recovery. It also reduces congestion window to one maximum segment size (MSS) on a timeout event. Congestion-avoidance phase, as usual, exploits AIMD to control cwnd based on packet loss notification. In fact, compound TCP stack for Windows and Linux also increases AIMD window as TCP Reno does [29].In Figure6(a), queue size and RTT according to three queue management schemes are presented. With 500 packets of buffer, Reno and Drop Tail show a saw-tooth graph where the highest RTT is high up to 2.8 (sec). With the target RTT being 200ms, RED under Reno presents better performance than Drop Tail, RTT (RED) ≈400ms, by early dropping some packets to keep the queue size small. CPC under Reno, however, achieves a much better performance while keeping the lowest queue size and lowest RTT (CPC) ≈180ms, which is acceptable for delay-sensitive applications. The reason is that CPC continuously controls the next queue size value according to traffic changes and adjusts the current queue size at the outer loop so that it indirectly affects RTT value of packets.Figure 6
TCP Reno + Drop Tail/RED/CPC.
(a)
Round-trip time
(b)
GoodputsNext, Figure6(b) shows us goodput (bit per second) statistic results under TCP Reno according to four different buffer sizes. These graphs use box-and-whisker plot that can effectively depict groups of numerical data through their quartiles. For instance, the center red line of box represents the median goodput value; the width of box shows the spread degree of goodput statistic data; for example, bigger box and longer whisker mean that statistic data oscillate so much or are unstable. Regardless of different buffer size, under TCP Reno, the median of goodput is nearly the same, around 8×104 (bps). The CPC scheme, however, achieves the best stability performance compared to the others because the width of CPC boxes is smallest.


### 5.3.2. Under Cubic Real-World Stack

Cubic is a congestion control protocol for TCP and the current default TCP algorithm in the Linux kernel [30]. It replaces the default TCP congestion control algorithm described above with a different algorithm based on a cubic function of the time since the last congestion event. Instead of adjusting the congestion window as a function of previous values as each packet is acknowledged, the Cubic algorithm recomputes the congestion window size at each step using the cubic function calibrated. This results in the congestion window responding quickly to changes in available bandwidth. The TCP Cubic is a less aggressive and more systematic derivative of binary increase congestion (BIC). We exploit NSC module to incorporate real-world TCP/IP network stacks into our simulation model. NSC owns TCP Cubic (version 2.6.26) stack which allows us to test our CPC queuing scheme performance.In Figure7(a), we can see again that CPC scheme seems to achieve the best performance, keeping lower queue size and RTT only 130ms, which is acceptable for delay-sensitive traffic. However, the RTT gaps between three schemes are not so much because of the aggression property of TCP Cubic. While RED shows large variability of RTT, the others keep RTT much more stable.Figure 7
TCP Cubic + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsGoodput performance in Cubic also reveals TCP Cubic’s aggression. When buffers are at small values (50 and 100 packets), three boxes’ widths are nearly similar. That means that CPC can achieve the same goodput as the others if we use small buffers. Moreover, increasing the buffer size to 300 and 500 packets makes CPC’s goodput more stable, with the smaller box width. These figures demonstrate that CPC can mitigate the bufferbloat—big buffers problem pretty well.


### 5.3.3. Under FreeBSD Real-World Stack

Finally, we investigate one more popular TCP stack. FreeBSD is a free Unix-like operating system, historically standing for “Berkeley Unix.” It was chosen as a BSD-derived TCP/IP stack that is widely used and has had much development. There are several applications that are directly based on FreeBSD, an example being the famous instant messenger WhatsApp. Much of FreeBSD became an integral part of other operating systems such as Apple’s OS X. The integrated NSC module for OMNeT++ also incorporates TCP FreeBSD real-world stack so that we can turn it on for demonstrating CPC’s performance under FreeBSD. Figure8(a) presents queue size and RTT results for three disciplines. We can see that our proposal CPC achieves a better performance, keeping lower queue size and RTT (FreeBSD) around 150ms.Figure 8
TCP FreeBSD + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsSo far, the proposed CPC scheme achieved good goodput performance under TCP Reno and Cubic stacks. We, however, find another interesting fact of CPC when performing under TCP FreeBSD stack. In Figure8(b), we can conclude two things. Firstly, the width of CPC goodput’s boxes in four buffer cases is almost bigger than the other schemes. It means the CPC stability under FreeBSD stack’s environment is not good. Secondly, our CPC scheme, however, still achieves more goodput than others. Specifically, in four cases, the system’s goodput can reach 10 (kbps) using CPC while it can reach only 4 (kbps) and 2 (kbps) under either small or large buffer using Drop Tail or RED schemes, respectively.


## 5.3.1. Under Reno Real-World Stack

For each connection, TCP maintains a congestion windowcwnd, limiting the total number of packets that may be transmitted at a time. TCP Reno algorithm uniqueness is the fast recovery phase. That means when a packet loss is detected by receiving three duplicate ACKs, Reno will halve cwnd and set slow start threshold ssthresh equal to the new cwnd value, perform a fast retransmit, and enter a phase called fast recovery. It also reduces congestion window to one maximum segment size (MSS) on a timeout event. Congestion-avoidance phase, as usual, exploits AIMD to control cwnd based on packet loss notification. In fact, compound TCP stack for Windows and Linux also increases AIMD window as TCP Reno does [29].In Figure6(a), queue size and RTT according to three queue management schemes are presented. With 500 packets of buffer, Reno and Drop Tail show a saw-tooth graph where the highest RTT is high up to 2.8 (sec). With the target RTT being 200ms, RED under Reno presents better performance than Drop Tail, RTT (RED) ≈400ms, by early dropping some packets to keep the queue size small. CPC under Reno, however, achieves a much better performance while keeping the lowest queue size and lowest RTT (CPC) ≈180ms, which is acceptable for delay-sensitive applications. The reason is that CPC continuously controls the next queue size value according to traffic changes and adjusts the current queue size at the outer loop so that it indirectly affects RTT value of packets.Figure 6
TCP Reno + Drop Tail/RED/CPC.
(a)
Round-trip time
(b)
GoodputsNext, Figure6(b) shows us goodput (bit per second) statistic results under TCP Reno according to four different buffer sizes. These graphs use box-and-whisker plot that can effectively depict groups of numerical data through their quartiles. For instance, the center red line of box represents the median goodput value; the width of box shows the spread degree of goodput statistic data; for example, bigger box and longer whisker mean that statistic data oscillate so much or are unstable. Regardless of different buffer size, under TCP Reno, the median of goodput is nearly the same, around 8×104 (bps). The CPC scheme, however, achieves the best stability performance compared to the others because the width of CPC boxes is smallest.


## 5.3.2. Under Cubic Real-World Stack

Cubic is a congestion control protocol for TCP and the current default TCP algorithm in the Linux kernel [30]. It replaces the default TCP congestion control algorithm described above with a different algorithm based on a cubic function of the time since the last congestion event. Instead of adjusting the congestion window as a function of previous values as each packet is acknowledged, the Cubic algorithm recomputes the congestion window size at each step using the cubic function calibrated. This results in the congestion window responding quickly to changes in available bandwidth. The TCP Cubic is a less aggressive and more systematic derivative of binary increase congestion (BIC). We exploit NSC module to incorporate real-world TCP/IP network stacks into our simulation model. NSC owns TCP Cubic (version 2.6.26) stack which allows us to test our CPC queuing scheme performance.In Figure7(a), we can see again that CPC scheme seems to achieve the best performance, keeping lower queue size and RTT only 130ms, which is acceptable for delay-sensitive traffic. However, the RTT gaps between three schemes are not so much because of the aggression property of TCP Cubic. While RED shows large variability of RTT, the others keep RTT much more stable.Figure 7
TCP Cubic + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsGoodput performance in Cubic also reveals TCP Cubic’s aggression. When buffers are at small values (50 and 100 packets), three boxes’ widths are nearly similar. That means that CPC can achieve the same goodput as the others if we use small buffers. Moreover, increasing the buffer size to 300 and 500 packets makes CPC’s goodput more stable, with the smaller box width. These figures demonstrate that CPC can mitigate the bufferbloat—big buffers problem pretty well.


## 5.3.3. Under FreeBSD Real-World Stack

Finally, we investigate one more popular TCP stack. FreeBSD is a free Unix-like operating system, historically standing for “Berkeley Unix.” It was chosen as a BSD-derived TCP/IP stack that is widely used and has had much development. There are several applications that are directly based on FreeBSD, an example being the famous instant messenger WhatsApp. Much of FreeBSD became an integral part of other operating systems such as Apple’s OS X. The integrated NSC module for OMNeT++ also incorporates TCP FreeBSD real-world stack so that we can turn it on for demonstrating CPC’s performance under FreeBSD. Figure8(a) presents queue size and RTT results for three disciplines. We can see that our proposal CPC achieves a better performance, keeping lower queue size and RTT (FreeBSD) around 150ms.Figure 8
TCP FreeBSD + Drop Tail/RED/CPC.
(a)
Round-trip Time
(b)
GoodputsSo far, the proposed CPC scheme achieved good goodput performance under TCP Reno and Cubic stacks. We, however, find another interesting fact of CPC when performing under TCP FreeBSD stack. In Figure8(b), we can conclude two things. Firstly, the width of CPC goodput’s boxes in four buffer cases is almost bigger than the other schemes. It means the CPC stability under FreeBSD stack’s environment is not good. Secondly, our CPC scheme, however, still achieves more goodput than others. Specifically, in four cases, the system’s goodput can reach 10 (kbps) using CPC while it can reach only 4 (kbps) and 2 (kbps) under either small or large buffer using Drop Tail or RED schemes, respectively.


## 6. Conclusions

Bufferbloat problem, with overfilling a queue at the low-speed side, leads to long end-to-end latency because of the persistent long queuing delay. In this paper, we proposed a control scheme called cascade probability control (CPC) as an alternative way to tackle this issue and reduce queuing delay. Our scheme introduces two-loop control model which consists of queue length and window size control to keep round-trip time value around an expected value. In comparison to current deployed schemes, our scheme’s advantages are to maintain stability and noise reduction while the queuing delay can still be ensured under several different TCP versions. Further works would extend CPC on bufferbloat and queuing delay control in the cellular communications.



---

*Source: 628583-2015-10-11.xml*