Multipath QUIC: Design and Evaluation
Quentin De Coninck, Olivier Bonaventure
quentin.deconinck@uclouvain.be
multipath-quic.org

Outline●The QUIC protocol●

Designing Multipath for QUIC

●Experimental Design Evaluation●Ongoing Work and Conclusion

2

QUIC: watisda?

3

QUIC = Quick UDP Internet Connection●

TCP/TLS1.3 atop UDP–>7% of the Internet trafc (YouTube, Chrome,...)●

Stream multiplexing → HTTP/2 use case●

0-RTT establishment (most of the time)
HTTP/2TLSTCPIP

HTTP/2 shim

QUIC
UDPIP

4

QUIC Packet

Flags Connection ID Packet Number

Encrypted Payload...

5

QUIC Packet

Flags Connection ID Packet Number

Encrypted Payload...

Cleartext Public Header
6

QUIC Packet

Does not dependon network
4-tupleFlags Connection ID Packet Number

Encrypted Payload...

Cleartext Public Header
7

QUIC Packet

Does not dependon network
4-tupleMonotonically
Increasing

Flags Connection ID Packet Number

Encrypted Payload...

Cleartext Public Header
8

QUIC Packet

Does not dependon network
4-tupleMonotonically
Increasing
No retransmission
ambiguities

Flags Connection ID Packet Number

Encrypted Payload...

Cleartext Public Header
9

QUIC Packet

Does not dependon network
4-tupleMonotonically
Increasing

Contains control/data frames

No retransmission
ambiguities

Flags Connection ID Packet Number

Encrypted Payload...

Cleartext Public Header
10

QUIC Packet

Does not dependon network
4-tupleMonotonically
Increasing

Contains control/data frames

No retransmission
ambiguities

Flags Connection ID Packet Number

Independent ofpackets

Encrypted Payload...

Cleartext Public Header
11

QUIC Packet

Does not dependon network
4-tupleMonotonically
Increasing

Contains control/data frames

No retransmissionambiguities

Flags Connection ID Packet Number

Cleartext Public Header

Independent ofpackets

Encrypted Payload...

Easier deployment
12

QUIC Data Transfer

H1H213

QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

H214

QUIC Data Transfer

H1

Actual data
F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

H215

QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

F CID PN=19

H2

ACK(25) MAX_DATA(for stream=5): 1=24

16

QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

H2Control Frames
F CID PN=19

ACK(25) MAX_DATA(for stream=5): 1=24

17

QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

F CID PN=19

H2

ACK(25) MAX_DATA(for stream=5): 1=24

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)18QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

F CID PN=19

H2

ACK(25) MAX_DATA(for stream=5): 1=24
Multiplexing

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)19QUIC Data Transfer

H1

F CID PN=25

STREAM(id=5,of==):”Some data in my long frame”

F CID PN=19

H2

ACK(25) MAX_DATA(for stream=5): 1=24

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)F CID PN=2=ACK(26)20QUIC and Packet Losses

H1

H221

QUIC and Packet Losses

H1

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)H222

QUIC and Packet Losses

H1

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)H2F CID PN=27 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y”

23

QUIC and Packet Losses

H1

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)H2No Packet Number Reuse
F CID PN=27 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y”

24

QUIC and Packet Losses

H1

F CID PN=26 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y” ACK(19)H2No Packet Number Reuse
F CID PN=27 STREAM(id=5,of=26):”.” STREAM(id=7,of==):”Y”

F CID PN=2= ACK(Largest=27, Block=->25)

25

What about Multipath?

26

Why Multipath QUIC?
●QUIC assumes a single-path fow

27

Why Multipath QUIC?
●QUIC assumes a single-path fow

28

Why Multipath QUIC?
●QUIC assumes a single-path fow

29

Why Multipath QUIC?
●QUIC assumes a single-path fow●Multipath QUIC–Bandwidth aggregation–Seamless network handover●Can try new WiFi while keeping using LTE30

Design of Multipath QUIC●Connection is composed of a set of paths

31

Design of Multipath QUIC●

Connection is composed of a set of paths

32

Design of Multipath QUIC●Connection is composed of a set of paths?

Pkt

Performance monitoring?
Loss detection?
Path congestion control?

33

Design of Multipath QUIC●Connection is composed of a set of paths
Pkt34

Design of Multipath QUIC●Connection is composed of a set of paths
Pkt

Flags Connection ID Path ID Packet Number
Explicit path
identifcation

Encrypted Payload...

35

Design of Multipath QUIC●Connection is composed of a set of paths
PktFlags Connection ID Path ID Packet Number
Explicit path
identifcation

No path handshake

Encrypted Payload...

36

Design of Multipath QUIC●Connection is composed of a set of paths
PktFlags Connection ID Path ID Packet Number
Explicit path
identifcation

Encrypted Payload...
Per-path numbering space

No path handshake

37

Multipath QUIC Data Transfer

Servervia WiFi

Server
via LTE

Phone

Path 1: WiFi

Path 2: LTE

38

Multipath QUIC Data Transfer

Servervia WiFi

F CID 1 PN=1 STR(id=5)

Path 1: WiFi

Server
via LTEPhone

Path 2: LTE

39

Multipath QUIC Data Transfer

Servervia WiFi

F CID 1 PN=1 STR(id=5)

F CID 1 PN=1 STR(id=7,of==)Path 1: WiFi

Server
via LTEPhone

F CID 2 PN=1 STR(id=7,of=1=24)

Path 2: LTE

40

Multipath QUIC Data Transfer

Servervia WiFi

F CID 1 PN=1 STR(id=5)

F CID 1 PN=1 STR(id=7,of==)Server
via LTEPhone

F CID 2 PN=1 STR(id=7,of=1=24)

F CID 1 PN=2 ACK(pid=1,1) ACK(pid=2,1)

Path 1: WiFi

Path 2: LTE

41

Multipath QUIC Data Transfer

Servervia WiFi

F CID 1 PN=1 STR(id=5)

F CID 1 PN=1 STR(id=7,of==)Server
via LTEPhone

F CID 2 PN=1 STR(id=7,of=1=24)

F CID 1 PN=2 ACK(pid=1,1) ACK(pid=2,1)

Path 1: WiFi

Frames not constrained
to a particular path

Path 2: LTE

42

Multipath Negotiation

H1

H243

Multipath Negotiation

H1

CHELLO(MaxPathID==x5)

H244

Multipath Negotiation

H1

CHELLO(MaxPathID==x5)

H2

SHELLO(MaxPathID==x3)

45

Multipath Negotiation

H1

CHELLO(MaxPathID==x5)

H2

SHELLO(MaxPathID==x3)

Use up to 4 paths (=x=, =x1, =x2, =x3)
46

Multipath Mechanisms●Path management●Packet scheduling●Congestion control

47

Path Management●

How and when paths are established?
IP1IP3IP2IP448

Path Management●

How and when paths are established?
IP1IP3IP2IP449

Path Management●

How and when paths are established?
Initial path
IP1IP3IP2IP450

Path Management●

How and when paths are established?
Initial path●

IP1IP3IP2IP4

Fullmesh fashion

51

Path Management●

How and when paths are established?
Initial path
IP1IP3IP2IP4●Fullmesh fashion●

ADD_ADDRESS + REMOVE_ADDRESS frames
52

Packet Scheduling●Lowest-latency frst
2= ms RTT
10 ms RTT

53

Packet Scheduling●Lowest-latency frst
2= ms RTT
10 ms RTT

54

Packet Scheduling●Lowest-latency frst
2= ms RTT
10 ms RTT●

What about when starting using a new path?
2= ms RTT?

55

Packet Scheduling●Lowest-latency frst
2= ms RTT
10 ms RTT●

What about when starting using a new path?
2= ms RTT
Duplicate?

56

Packet Scheduling●Lowest-latency frst
2= ms RTT
10 ms RTT●

What about when starting using a new path?
2= ms RTT
Duplicate?

●Schedule all frames (not only STREAM)
57

Congestion Control●

Multipath = need for coupled CC–●

CUBIC would be unfair

Opportunistic Linked Increase Algorithm–MPTCP state-of-the-art

58How well does
Multipath QUIC
perform?

59

Evaluation of Multipath QUIC●●(Multipath) QUIC vs. (Multipath) TCP–

Multipath QUIC: quic-go–

Linux Multipath TCP v=.91 with default settings

Mininet environment with 2 paths

60Evaluating Bandwith Aggregation●Download of 20 MB fle–

Over a single stream–

Collect the transfer time61

Evaluating Bandwith Aggregation●●

Download of 20 MB fle–

Over a single stream–

Collect the transfer time

For a loss-free scenario

2=ms RTT, 2= Mbps

4=ms RTT, 15 Mbps

62

Evaluating Bandwith Aggregation●●

Download of 20 MB fle–

Over a single stream–

Collect the transfer time

For a loss-free scenario–

2=ms RTT, 2= Mbps

4=ms RTT, 15 Mbps

MPQUIC has 13% speedup compared to MPTCP

63

Evaluating Bandwith Aggregation●●

Download of 20 MB fle–

Over a single stream–

Collect the transfer time

For a loss-free scenario–

2=ms RTT, 2= Mbps

4=ms RTT, 15 Mbps

MPQUIC has 13% speedup compared to MPTCPMPQUIC less bursty than MPTCP●
Probably due to CC skew on initial path in MPTCP●

64

Evaluating Bandwith Aggregation●●

Download of 20 MB fle–

Over a single stream–

Collect the transfer time

For a loss-free scenario–

2=ms RTT, 2= Mbps

4=ms RTT, 15 Mbps

MPQUIC has 13% speedup compared to MPTCPMPQUIC less bursty than MPTCP●
Probably due to CC skew on initial path in MPTCP●●

But what about other topologies?
65

Evaluating Bandwidth Aggregation●Experimental design, WSP algorithm●2x253 network scenarios–●

Vary the initial path

Median over 15 runs
Factor

Minimum

Maximum

Capacity [Mbps]

0.1

100

Round-Trip-Time [ms]050

Queuing Delay [ms]0100

Random Loss [%]02.566

Large File Download – No Loss

TCP better

QUIC better67

Large File Download – No Loss

Single-path

TCP better

QUIC better68

Large File Download – No Loss69Large File Download – No LossMPQUIC betterin 85% of cases70Large File Download – No LossMPQUIC betterin 85% of cases

Our extracted scenario71

Large File Download – No LossMPQUIC betterin 85% of cases
Path 1: 27.2 ms RTT, =.14 Mbps,
34 ms queuing delay
Path 2: 46.4 ms RTT, 49.72 Mbps,
47 ms queuing delay

Path 1: 49.4 ms RTT, 18.9= Mbps,
82 ms queing delay
Path 2: 1=.6 ms RTT, =.43 Mbps,
11 ms queuing delay

Our extracted scenario72

Large File Download – Losses73

Large File Download – Losses

QUIC copes
better withlosses
74

Large File Download – Losses

QUIC copes
better withlosses

TCP SACK: 2-3 blocks

QUIC SACK: 256 blocks

75

What is the actual
beneft of Multipath toQUIC?

76

Actual Multipath Beneft●Experimental Aggregation Beneft–Multipath QUIC/TCP vs. single-path QUIC/TCP
1-10

Zero goodput= best single path

= aggregation of all paths

MP gives = Mbps

3 Mbps + 5 Mbps paths
MP gives 5 Mbps

3 Mbps + 5 Mbps paths
MP gives 8 Mbps

●Results depends on the frst path used
–Handshake latency over initial path77

Benefts of Multipath – No Loss78

Benefts of Multipath – No Loss% scenarios multipath has EAB >= =, regardless of frst path used

79

Benefts of Multipath - Losses

80What about congestionprone networks?

81

Experimental Design with High-BDP Networks

Factor

Minimum

Maximum

Capacity [Mbps]

0.1

100

Round-Trip-Time [ms]0400

Queuing Delay [ms]02000

Random Loss [%]02.582

Multipath Benefts without Losses

83Completion Time Ratio with Losses

84

What about shorttransfers?

85

Short Transfer Evaluation with Low-BDP●Download of a 256 KB fle–●Collect transfer timeMedian over 3 runs
Factor

MinimumMaximum

Capacity [Mbps]

0.1

100

Round-Trip-Time [ms]050

Queuing Delay [ms]

0100

86

Comparison QUIC vs. TCP

87

Comparison QUIC vs. TCP

Shorter QUIC
handshake

88

Multipath Not Really Useful...

89

What about network
handover?

90

Network Handover Support●Apple MPTCP deployment mainly for handover–Main use case for Siri91

Network Handover Support●Apple MPTCP deployment mainly for handover–●Main use case for Siri

Request/Response trafc–75= bytes request/responses–

Measure delay seen by client

15ms RTT, 1==% loss after 3 s

25ms RTT

92

Multipath TCP Handover93

Multipath TCP Handover94

What Happened During MPTCP Handover?

95

What Happened During MPTCP Handover?
Req

96

What Happened During MPTCP Handover?

97

What Happened During MPTCP Handover?

RTO

Req

98

What Happened During MPTCP Handover?

RTO

Req

99

What Happened During MPTCP Handover?

ResRTO

100

What Happened During MPTCP Handover?

RTO

101

What Happened During MPTCP Handover?

RTO

RTO
Res

102

What Happened During MPTCP Handover?

RTO

RTO
Res

103

And What About Multipath QUIC?

104

And What About Multipath QUIC?

105

What Happened During Handover?

106What Happened During Handover?
F CID 1 PN

STR(Req)107What Happened During Handover?

108

What Happened During Handover?

RTO
FCID 2 PN

STR(Req)

PATHS(1 lossy)

109

What Happened During Handover?

RTO

F CID 2 PN

STR(Req)

PATHS(1 lossy)

110What Happened During Handover?

RTO

F CID 2 PN

STR(Res)111

What Happened During Handover?

RTO
F CID 2 PN

STR(Res)112

What about actualnetworks?

113

QUICTester Application●

Perform tests in actual networks–

Does (MP)QUIC work in your networks?

–

Does MPQUIC provides better performances?

–Application running on iOS11●–

https://itunes.apple.com/fr/app/quictester/id1322=19644?mt=8Feel free to provide feedback :-)QUICTester114

To sum up...

115

Conclusion●

Multipath should be part of any transport
protocol–●●Most devices are multihomed

Designed and implemented Multipath QUIC
–Source code + artifacts + IETF draft available–

See multipath-quic.org

Multipath more promising with QUIC than TCP–

Also opens potential new use cases
116Thanks!
multipath-quic.org

117