Latency-based Congestion Detection and Control for Datacenters Major Advisor :

Large-scale datacenters have become an essential part of today’s online services, and performance issues within datacenter networks have received much attention in recent years. In datacenter workloads, latencysensitive flows are prevalent, so achieving low latency is important in improving end-users’ experience. Since queueing delay dominates total network delay in datacenter networks, lowering queueing delay has been a major research challenge. To solve the queueing delay problem, congestion control algorithms should be re-designed to target datacenter networks’ own needs. The nature of congestion feedback largely governs the behavior of congestion control. In datacenter networks, where RTTs are in hundreds of microseconds, accurate feedback is crucial to achieve both high utilization and low queueing delay. Previous proposals for datacenter congestion control predominantly leverage ECN or even explicit in-network feedback to minimize the queuing delay. In this dissertation, we explore latency as accurate congestion feedback and develop a new congestion control algorithm that is specifically designed to achieve low queueing delay in datacenter networks. Before developing the algorithm, we first analyze the required properties for congestion control mechanism in datacenters and derive our design choices. Our first design choice is the use of latency as a congestion feedback instead of packet loss or ECN-based feedback. Previous TCP algorithms such as TCP Vegas have already proposed to use latency feedback, but it has been known that end-to-end latency measurements are noisy and cannot represent accurate congestion level in the networks. To overcome such limitations in our algorithm, we measure end-to-end latency in driverlevel and hardware-level. We implement driver-level measurement with Intel DPDK platform and hardware-level measurement with Mellanox timestamping-enabled network interface card. We come up with additional calibration techniques on packet bursts and hardware DMA. After removing all the sources of errors, we demonstrate that latency-based feedback is up to an order of magnitude more accurate and fine-grained than ECN-based feedback; the number of packets in the network switch can be accurately inferred by our latency measurements. Our second i design choice is accurate window size calculation. DX, our congestion control algorithm, calculates the number of packets to saturate the network link without queueing every RTT. Such fine-grained calculation is only possible due to our accurate latency measurement. One of the advantages of using DX is that there is no parameter setting whatsoever. Finally, we implement DX in Linux kernel and evaluate the performance with testbed experiments and large-scale simulations. In our dumbbell topology experiments, DX achieves 5.33 times smaller queueing delay at 1Gbps link than DCTCP, the most popular ECN-based solution, and 1.57 times smaller queueing delay at 10Gbps. We believe that improvement in hardware timestamping technology will enable better performance at 10Gbps and even higher network link rates in future datacenter networks.