HierCC:
Hierarchical RDMA Congestion Control
Jiao Zhang, Yali Zhang, Zixuan Guan, Zirui Wan, Yinben Xia
Tian Pan, Tao Huang, Dezhi Tang, Yun Lin

1

Background
Cluster-wide storage system

Machine-learning system

CC C

compute compute compute

Network IO

storageResource disaggregation

G Gnetwork

storageM• IOPS 1M• Latency<45usG• Latency<30usM

MM• Latency<3~5us

Ultra-low latency is needed!
2Hard to balance throughput and latency￮ “High throughput” expects sender to send as many packets as possible(e.g. send as line-rate)

Sender 1

Packets

Latency
SW 1

SW 2

Throughput

Receiver

Sender 23

Hard to balance throughput and latency￮ “High throughput” expects sender to send as many packets as possible(e.g. send as line-rate)￮ “Ultra-low latency” expects sender to send as few packets as possible(e.g. send as min-rate)

Sender 1

Packets

SW 1
Sender 2

SW 2

Receiver4

Hard to balance throughput and latency

Hard to balance ultra-low latency andhighPackets throughput

Sender 1

SW 1
Sender 2

SW 2

Receiver5

Today’s solutions
Three main options

Reactive congestion
control

Proactive congestion
control

Centralized control• DCQCN(SIGCOMM’15)• TIMELY(SIGCOMM’15)• HPCC(SIGCOMM’19)• ExpressPass(SIGCOMM’17)• NDP(SIGCOMM’17)• Homa(SIGCOMM’18)• FastPass(SIGCOMM’14)• Flowtune(NSDI’7)- Use congestion signal- Use credits to schedule packets- Use a central arbiter to globallyschedule packets- Always react after congestion
happens- Meet first RTT problem- Need one extra RTT to requestcredits.

6Today’s solutions

But these solutions face a dilemmain handling traffic uncertainty!

7

Two common cases with uncertain traffic(Case 1)
Case1: Short burst flows
Sender 1
2 long-lived
flows
SW 1
Sender N

SW 2
16 short burst
flows

Receiver• Using HPCC (Make control after congestion happens)• Need multiple rounds to converge to the ideal rate8

Two common cases with uncertain traffic(Case 1)
Short burst flows• Queue length of the bottleneck link and overall throughput with 25Gbps
link speed• HPCC needs 5(#maxstage) RTT to converge to the ideal rate9

Two common cases with uncertain traffic(Case 1)
Short burst
flows

Existing reactive CC mechanisms:• Most existing reactive CC mechanisms use line-rate as the initial rate.

• When short burst flows start, the existing flows that have already converged
to the stable state have to converge to the desired rate again.
• When short burst flows finish, the remaining flows need multiple rounds tooccupy the bandwidth released by the completed flows.

10Two common cases with uncertain traffic(Case 1)
Short burst
flows

Problem1：
Perform well in stable state, but the instantaneous
performance is poor with bursty traffic.
11

Two common cases with uncertain traffic(Case 2)
Case2: Incast scenario
Sender 1
The number ofsender grows
SW 1

SW 2

ReceiverSender N• Reactive congestion control can’t effectively limit thequeue length when massive flows burst.
12

Two common cases with uncertain traffic(Case 2)
Incast scenario• Queue length peak grows by the number of flows
13

Long control loop makes performance worse
Case1: Short burst flows

Case2: Incast scenario• When RTT grows, converging time changes from 30us to 60us, andqueue length peak changes from 6MB to 9MB
14

Long control loop makes performance worse
Case1: Short burst flows

Case2: Incast scenario

Problem2：
Performances become worse when RTT grows!

15

What about using proactive solutions?
Also meet the first RTT problem![1]#1 Use the first RTT time to schedule ?

#2 Traffic burst in first RTT ?

creditscheduled

Sender

1st RTT

existing flow
Receiverunscheduled
Buffer overflow

credit request

Sendernd2

SW 1

SW 2Receiver

RTTReceiver

new flow

dataWaste the first RTT time[1] Aeolus: A Building Block for Proactive Transport in Datacenters 2020 SIGCOMM

Packets lost happens
16

What about using proactive solutions?
Also meet the first RTT problem![1]#1 Use the first RTT time to schedule ?

#2 Traffic burst in first RTT ?

creditscheduled

Sender

1st RTT

existing flow
Receiverunscheduled
Buffer overflow

credit request
Problem3:
HardSenderto accurately allocate credits inSW 1 firstSW 2RTT!
Receiver2nd RTT

Receivernew flow

dataWaste the first RTT time[1] Aeolus: A Building Block for Proactive Transport in Datacenters 2020 SIGCOMM

Packets lost happens
17

Problem: Traffic uncertainty➢ Existing solutions face a dilemma in handling traffic uncertainty.
Main Problems:

Basic Ideas:

1. Existing solutions perform well in stable➢ Aggregate flows to make a long-lived flow

state, but the instantaneous performance ispoor with bursty traffic.

2. Performances become worse when RTT grows.

The aggregated flows by destination host
are tightly distributed around 1MB[1].
➢ Shorten the control loop
Directly send feedback from sender-side ToR.

3. Hard to accurately allocate credits in first RTT➢ Using a pre-credit scheme
Set initial credits for the aggregate flows.

[1] Inside the Social Network’s (datacenter) Network 2015 SIGCOMM

18

HierCC

HierCC:
Hierarchical RDMA Congestion Control
Aggregate flows + Credit-based mechanism + Short control loop

19

Layer2(inter-rack): Aggregated flows• Aggregate flows with the same destination host at ToR switches.

Packets with the same destination host go
into the same VOQ(virtual output queue)
Core1

Core2
C

ToR1

Layer2

VOQ
Scheduler

VOQ
Scheduler

VOQ1.7

ToR4

CToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8H1H3H2

H1

H7

H2H7H4H1H8H8H7H3H7CCredit

Host20Layer2(inter-rack): Aggregated flows• Aggregate flows with the same destination host at ToR switches.
• Use credit-based scheme to schedule packets between ToR switches.

Use credits to proactive schedule packets
between ToR switches
Core1Core2
CToR1

Layer2

VOQ
Scheduler

VOQ
Scheduler

VOQ1.7

ToR4

CToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8

H1

H3

H2

H1

H7H2

H7H4H1H8H8H7H3H7CCredit

Host21

Layer2(inter-rack): Aggregated flows• Aggregate flows with the same destination host at ToR switches.
• Use credit-based scheme to schedule packets between ToR switches.

• Use pre-credit scheme to alleviate the first RTT problem.
Use pre-credit scheme to alleviate the first RTT
problem in proactive CC solutions
Core1Core2
CToR1

Layer2

VOQ
Scheduler

VOQ
Scheduler

VOQ1.7

ToR4

CToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8

H1

H3

H2

H1

H7H2

H7H4

H1

H8H8H7H3H7CCredit

Host22

Layer2(inter-rack): Aggregated flows
Aggregating flows and using pre-credit mechanism to alleviate thefirst RTT problem and ensure high bandwidth utilization between ToRs.

Core1Core2CToR1

Layer2VOQScheduler

VOQScheduler

VOQ1.7

ToR4C

ToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8H1H3H2H1H7H2H7H4H1H8H8H7H3H7CCredit

Host23

Layer1(intra-rack): Short control loop
• Directly distribute rate to each individual flows.

Core1

Core2C

ToR1
VOQ
Scheduler

VOQ
Scheduler

VOQ1.7

ToR4CToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8Layer1
H1H3H2H1H7H2H7H4H1H8H8H7H3H7CCredit

HostThe bandwidth obtained by an aggregate flow will beallocated to the corresponding individual flows.
24

Layer1(intra-rack): Short control loop
• Directly distribute rate to each individual flows.
• HierCC calculate the rate based on both the VOQ arrival rate and the VOQ
length in order to limit the VOQ usage.
Core1Core2
CToR1
VOQ
Scheduler

VOQ
Scheduler

VOQ1.7

ToR4CToR2

WRR
Scheduler

VOQ2.7

WRR
Scheduler

VOQ1.8Layer1H1H3H2H1H7H2H7H4H1H8H8H7H3H7CCredit

Host25

Implementation• The sender marks one reserved bit in the data packet header
periodically. The sender-side ToR then feeds a rate allocation
packet for every marked packet.
In this way, sender-side ToRs do not need to maintain a flowtable to store the information of active senders.
• Currently, some switch chips like Arista
Jericho and Cisco Silicon One can support
100K VOQ queues.

26

Evaluation Setup• SystemC simulations and NS3 simulations• Topology• Fattree with 25Gbps link-rate• 2 Core, 4 Aggr, 4 ToR, 64 servers (SystemC)• 16 Core, 20 Aggr, 20 ToR, 320 servers (NS3)• Traffic pattern• Long-lived flow + burst flows at 1us and 3us (SystemC)• All to all and all to one ToR patterns with 0.8 workload WebSearch traffic(SystemC)• All to all Cache traffic with different workloads (NS3)28

Evaluation: burst flows

HierCC limits VOQs length and it keeps high link bandwidth
utilization at the same time.
29

Evaluation: WebSearch

HierCC reduces the FCT of both long and short flows while alsosignificantly reducing the queue length.
30

Evaluation: Cache

HierCC reduces the overall average FCT and the 99th percentile

FCT compared to other solutions.
31Conclusion• Problem: Existing solutions face a dilemma in handling traffic uncertainty.
HierCC:• Aggregated flows with the same destination at ToR switches.
➢ make long-lived flows against the traffic uncertainty• Use pre-credit scheme to schedule packets at first RTT➢ address the first RTT problem• Directly distribute rate to the host➢ Shorten the control loop32

Thank you!

33