Multipath TCP:Challenges & Opportunities
Matthieu Coudron

13 Feb. 2018, IIJ

OutlineI.

Multipath communications• Advantages• Challenges
II. Multipath TCP presentation
III. Augmented multipath TCP
IV. MPTCPNUMERICS: MPTCP window management
V. Conclusion

2

Multipath communications
1. Motivations
2. Challenges

3Motivations
 Reliability
easy to retransmit on alternative paths

 Fairness & Resource Poolingmake use of previously unused resources Bandwidth aggregation
sum links’ throughput

 Confidentiality

harder to capture on several paths

Resource pooling« Resource pooling means making a collection ofnetworked resources behave as though they make up a single
pooled resource »D. Wishik et al., The resource pooling principle, CCR 20185

Multipath Challenges
 Deployment concerns
Deployment has to be incremental
Compatible with the existing infrastructure Path management
How many (disjoint) paths between hosts ?
 Packet reordering
How to deal with heterogeneous paths ?

Multipath TCP
1. An ossified Internet
2. Introduction

3. Subflow management7An ossified Internet IP/TCP fields modified bymiddleboxes

Protocol field intact but some middleboxes
drop packets with unknown protocol, i.e.,
different from TCP or UDPIP Solution(s) for a new protocol

 Happy Eyeballs: Fallback on a safeprotocol
 Tunneling Look like TCP (or UDP)

TCP

Source : O. Bonaventure Cloudnet 2012

MPTCP introduction
 Defined in RFC 6824 as a TCP extension
 Emphasis on backwards compatibility Works with most middleboxes
 Congestion control fair to TCP Can send data concurrently on several subflows Single data stream transmitted at 51.8 Gbit/s.

 Available in: Linux BSD
 iOS79

MPTCP introduction
1. First acknowledges if destination is MPTCP compliant
during the 3 way handshake
2. Then creates additional TCP subflows according to pathmanagement mechanism

10

MPTCP path management
RFC 6182 states path management should be « modular », i.e., policybased●
Several subflows can originate from the same IP with different port
numbers● By default in linux 1 subflow per (IP, IPdestination)
source●Example: 2 source IPs and 2 destination IPs => 2x2 = 4 subflows●12Some (encouraging) results

Fat tree with K=4 pods

© Raiciu, et al. “Improving datacenter
performance and robustness withmultipathTCP”, ACM SIGCOMM 2011.

13

Application sends data to MPTCP send buffer :#MPTCP

Data ↓Mapping

2B

2:1

1A

1:1

« AB » forwarded
to application

MPTCP scheduler
#TCP

DATA

#TCP

B

DATA

A
1A

1

B

MPTCP
receive buffer
ACK 2 : 2

1 : 1(A)2 : 1 (B)ACK 3 : 2

14

Application sends data to MPTCP send buffer :#MPTCP

Data ↓Mapping

2B

2:1

« AB » forwarded
to application

MPTCP scheduler
#TCP

DATA

#TCP

DATA

1B

MPTCP
receive bufferACK 3 : 215

MPTCP can be worse then TCP

MPTCP with
2 subflows :(2G)MPTCP with
1 subflow

Ferlin, S. et al. « Multi-Path Transport over
Heterogeneous Wireless Networks: Does itreally pay off? »Fairness with legacy TCP flows
 A multipath flow should
1)perform at least as well as a single-path flow would on
the best of the paths available to it2)not take up any more capacity on any one of its paths
than if it was a single path flow using only that route

 MPTCP congestion control kicks induring congestion avoidance

 window is shared between subflows

OLIA : Opportunistic Linked
Increase Algorithm
 Losses handled like in TCP (Wr = Wr/2)
 For every ack ACK on flow r, add to wr

 Where α is
 > 0 if r belongs to best paths with small cwnd
 < 0 if Wr has a big window while a better path exists with a smallerwindow = 0 otherwise

18

Problem with short connections(e.g., <100KB)
 In a wifi/LTE setup, with an initial Wifi subflow, thetransfer finishes before the LTE subflow could send data Yet LTE accounts for 61 % of the energy consumed

Nikravesh, A. et al. « An in-depth understanding ofmultipath TCP on mobile devices », Mobicom 2016

Summary Deployment incremental & backwards compatibility okincreased confidentiality can be seen as a threat Path management
How many (disjoint) paths between hosts ?
When to create/reset subflows ?
 Packet reorderingIncrease buffer size but when possible
or limit subflow usage

 Pareto-optimality
New solutions/protocols should be always better and for everyone

Path management problem:
Augmented MPTCP
1. Goal & Overview

2. Presentation of LISP
3. Tesbed & Results

M. Coudron, S. Secci, G. Pujolle « Cross-layer cooperation to boost
multipath TCP performance in cloud networks », CloudNet 201321

Contribution overview
Objective●
Increase goodput between endusers and/orDataCenters via disjoint physical paths
Problem●Endhosts ignore the topology
Solution●
Ask a protocol that knows the answer ; for instance LISP=> crosslayer solution●(Assuming WAN segment is the bottleneck)22• How many subflows to create ?
• How to achieve proper forwarding ?

By default 1 subflow
100MB/s
1GB/s
100MB/s
1 IP address

1 IP address

Wouldn’t 2 subflows be better ?
Not necessarily... need to follow different physical paths

23Architecture Overview
 Use LISP protocol to enhance MPTCP LISP can give edge path diversity information
 LISP can enable multipath WAN forwarding Enforce per subflow forwarding (SDN-like) Based on TCP ports in our case24

Location/Identifier
Separation Protocol (LISP) Defined in RFC 6830
 Tunneling protocol between edge routers Allows us to get the WAN path diversity

 IPs classified in two groups Endpoint IDentifier (EID) Routing Locators (RLOCs)

 EID associated to RLOC(s) via a mapping system
control-plane25

Mapping
ServerTo reach ...

Encapsulate to...

EID B

RB1 orRB2

2/ RA retrieves RLOCs for B(i.e., location of B)
1/ A wants to contact B

RLOC RA

4/ RB1 decapsulates andforwards inner packet to B
RLOC RB1

3/ Packet from A
encapsulated & forwarded to RB1

Host EID « B »Host EID « A »RLOC RB226

Our testbed

Our guess: Number of WAN paths = Number of RLOCs
27

1/ First subflow established
2/ Retrieves number of RLOCs
3/ Creation of 2nd subflow with
Specific source port number(ΣPortNumbers) %2 = 0 and 1 (or 1 and 0 )28

Client C
Userspacedaemon+lig program

Server S

LISP routers
Specific
Kernel module

Application

CAPABLE
1: SYN/ACK + MP_layedt RLOCs
3:Request re2: Requesof EID S
to userspace
via Netlink

1: ACK

4: Sends Map-Request

5: Sends Map-Reply, i.e

6: Sends numberof RLOCs. list of RLOCs

6: relays theinformation

1: SYN + MP_CAPABLE

UDP/LISPtunnel

7: Creation of

additional subflows if needed29

Transfer time(lower is better)30

Transfer time(lower is better)31

Transfer time (20 iterations)32

Results on 20 iterations
40% improvement33

3 subflows

34Summary A-MPTCP increases throughput in certain conditions
 enough WAN paths given by LISP WAN path is the bottleneck Too many subflows can hurt the goodput
 Crosslayer architectures are complex to deploy

 PerspectiveDetect when using subflow would hurt

35

MPTCPNUMERICS :Window management for MPTCP
1. Overview

2. Presentation of LISP
3. Simulation Results

M. Coudron, D. Duy, S. Secci « On buffer and window management forMPTCP », NoF 201636

Contribution overview
Objective●Throughput is but one metric, users may want to trade (some)
goodput for better confidentiality

Problems
MPTCP advertised window is shared between subflows so anefficient subflow might starve others●
goodput-only approaches can lead to discard less efficient
paths but these paths may present an interest for the user
●Solution●Cap the congestion windows on best paths to ensure freebuffer space for less performing subflows
37

MPTCPNUMERICS: an event based simulator Custom discrete time simulator Accepts as input
 A topology configuration file (with RTTs, buffer size) Constraints on subflow contributions ; e.g., « the Wifi subflow should contribute to at least (/no morethan) 50 % of the goodput»

 to give congestion window targets under constraints

Wcoenstr
on foraint
th cee 60s lo %w ofs u tr
Libfl affimiow cto t t
h50 e% faof stgo suo d bf
pu owtNo2-subflow topology
4-subflow topology(blue barhigher is better)

Perspectives
 Go beyond the “throughput” objective: throughput
aggregation is not the only metric of interest to users●
Latency sensitive applications●
Higher confidentiality at the expense of throughput●
Monetary costs related to interfaces●

 Our tool may help the user choose strategies such as« giving up 20 % of the foreseen throughput for an increasein path diversity of 50 % »Conclusion

42Conclusion
 MPTCP is viable today but throughput can be worse than TCP

on heterogeneous paths
 Throughput aggregation is the main area of study But users may tradeoff throughput for Energy/financiary economies
 Latency improvements (packet duplication/Network coding Higher confidentiality
 Knowing the application traffic profile would help
 TAPS (Transport Area Protocol Services) proposes anabstraction to do just that

Source code available at
http://github.com/lip6-mptcp
coudron@iij.ad.jp

Want to try MPTCP ?
1. Install the MPTCP kernel (Debian/Ubuntu)
 http://multipath-tcp.org
2. Reboot
3. Go to www.amiusingmptcp.de45

Peer reviewed communications « Augmented MPTCP communications », ICNP 2013
 « Crosslayer cooperation to boost MPTCP performance in the cloud»,
ICNP 2013 « Differentiated pacing on multiple paths to improve one-way delayestimations », IM2015
 « On buffer and window management for MPTCP », NoF 2016
 « Per node clocks to simulate time desynchronization in networks », WNS32016
 « Multipath transmission for the Internet: a survey », IEEE
Communications Surveys & Tutorial vol. 18 N°4 Déc. 2016
 « Multipath TCP in NS-3 : implementation evaluation », under majorrevision, Computer Networks

47

MPTCP State Machine48

MPTCP State MachineBuffer mode
To handle a fast retransmit scenarioStandardrecommendationTo handle a Retransmission Time-Out

Such recommandation are for bulk transfer :No recommendation for other traffic patterns or for losslessscenariosBuffer requirements Formulated as an Integer Linear
Programming (ILP) model :Objective : minimize buffer size gamma
Constraints : TCP flow control

Influence of scheduling in buffer requirements

Inc/Dec/ManualAre different
scheduling
Strategies
FR=Fast Retransmit
RTO=Retransmission
Timeout

Same RTT/window
Different fowd