Enabling ECN for Datacenter Networks With RTT Variations

ECN has been widely employed in production datacenters to deliver high throughput low latency communications. Despite being successful, prior ECN-based transports have an important drawback: they adopt a fixed RTT value in calculating instantaneous ECN marking threshold while overlooking the RTT variations in practice. In this article, we reveal that the current practice of using a fixed high-percentile RTT for ECN threshold calculation can lead to persistent queue buildups, significantly increasing packet latency. On the other hand, directly adopting lower percentile RTTs results in throughput degradation. To handle the problem, we introduce ECN<inline-formula><tex-math notation="LaTeX">$^\sharp$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>♯</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="zhang-ieq1-3204988.gif"/></alternatives></inline-formula>, a simple yet effective solution to enable ECN for RTT variations. At its heart, ECN<inline-formula><tex-math notation="LaTeX">$^\sharp$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>♯</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="zhang-ieq2-3204988.gif"/></alternatives></inline-formula> inherits the current instantaneous ECN marking (based on a high-percentile RTT) to achieve high throughput and burst tolerance, while further marking packets (conservatively) upon detecting long-term queue buildups to eliminate unnecessary queueing delay without degrading throughput. We implement ECN<inline-formula><tex-math notation="LaTeX">$^\sharp$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>♯</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="zhang-ieq3-3204988.gif"/></alternatives></inline-formula> on a Barefoot Tofino switch and evaluate it through extensive testbed experiments and large-scale simulations. Our evaluation confirms that ECN<inline-formula><tex-math notation="LaTeX">$^\sharp$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>♯</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="zhang-ieq4-3204988.gif"/></alternatives></inline-formula> can effectively reduce latency without hurting throughput. For example, compared to the current practice, ECN<inline-formula><tex-math notation="LaTeX">$^\sharp$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>♯</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="zhang-ieq5-3204988.gif"/></alternatives></inline-formula> achieves up to <inline-formula><tex-math notation="LaTeX">$23.4\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>23</mml:mn><mml:mo>. </mml:mo><mml:mn>4</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhang-ieq6-3204988.gif"/></alternatives></inline-formula> (<inline-formula><tex-math notation="LaTeX">$31.2\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>31</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhang-ieq7-3204988.gif"/></alternatives></inline-formula>) lower average (99th percentile) flow completion time (FCT) for short flows while delivering similar FCT for large flows under production workloads.