IPv6 EH Traversal
Edge measurements
Ana Custuraana@erg.abdn.ac.uk

06-11-2022
IEPG IETF 115

Scope of Experiments• Tests concerned with edge networks• Consumer Edge: RIPE Atlas probes• Server Edge: Web + Authoritative NSes for the Alexa Top
1M domains

SourceHop 1

Consumer Edge

Hop 2

Hop 3

Hop N

Destination

Server Edge

IEPG IETF 115

RIPE ATLAS experiments• ~5500 IPv6-enabled probes in RIPE, globally distributed• Tested traversal by sending packets to 2 target servers (in UK and Canada)• {TCP, UDP} to port 443• {DOPT, HBHOPT}• {8,16,32,40,48,56,64} B in size
• Thanks Brian!

SourceHop 1

Consumer Edge

Hop 2

Hop 3

Hop N

Destination

Server Edge

IEPG IETF 115

RIPE ATLAS experiments• ~5500 IPv6-enabled probes in RIPE, globally distributed• Tested traversal by sending packets to 2 target servers (in UK and Canada)• {TCP, UDP} to port 443• {DOPT, HBHOPT}• {8,16,32,40,48,56,64} B in size
• Thanks Brian!

SourceHop 1

Consumer Edge

Hop 2

Hop 3

Hop N

Destination

Server Edge

IEPG IETF 115At a glance - traversal• 8B PadN option• High traversal for Destination Options (DOPTs)• Some paths support Hop-by-Hop Options (HBHOPTs)• Di erence between UDP and TCP regardless of EHSUREHV(GJHQHWZRUNV

8QLYHUVLW\RI$EHUGHHQ
6RXUFH+RS

+RS

+RS

+RS1DOPTHBHOPT~92%~11%

UDP

~68%

~9%

TCP

'HVWLQDWLRQ

29+ &DQDGD ff

IEPG IETF 115

Drops by 1st hop on the path
HBHOPTSUREHV(GJHQHWZRUNVDOPT

8QLYHUVLW\RI$EHUGHHQ
6RXUFH+RS

+RS

+RS

+RS1'HVWLQDWLRQ

29+ &DQDGD • On over 55% of the paths,

HBHOPTs packets get
dropped at the local router,no protocol di erence;• For DOPTs, w/TCP 10% offffi

the paths discard packets
at the rst hop
IEPG IETF 115

Drops by 1st hop on the path
HBHOPTSUREHV(GJHQHWZRUNVDOPT

8QLYHUVLW\RI$EHUGHHQ
6RXUFH+RS

+RS

+RS

+RS1'HVWLQDWLRQ29+ &DQDGD • On over 55% of the paths,

HBHOPTs packets get
dropped at the local router,no protocol di erence;• For DOPTs, w/TCP 10% offffi

the paths discard packets
at the rst hop
IEPG IETF 115

Per-AS traversal (UK path)
DOPT
The local AS is responsible formost of the drops:• 5% for UDP• 25% for TCP

1st AS

AS1>AS2∞

DOPT UDP
8B

95.3%

93%

91.5%

DOPT TCP
8B

74.7%

70%

68.5%

1st AS

AS1>AS2

2nd AS

AS2>AS3∞HBHOPTUDP 8B

31.4%

20.1%

15%

12.2%

11.4%

HBHOPTTCP 8B

26.9%

16.3%

13.9%

9.7%

8.6%

HBHOPT
The local AS is responsible
for most of the drops:• 68% for UDP• 74% for TCP

Drops are considered to be within the AS if the next hop on a control measurement is also in that AS.
If the next hop would otherwise be in a di erent AS, then the drop is attributed to the AS boundary.

ffIEPG IETF 115

What if packets would
traverse the rst AS?

SUREHV(GJHQHWZRUNV

6RXUFH

AS1

AS1+RS

+RSAS2+RSAS
Dest

8QLYHUVLW\RI$EHUGHHQ+RS1'HVWLQDWLRQ

29+ &DQDGD • Most probes have public IPv6 addresses• Reverse traceroute on paths where drops happen in rst AS• Same protocol/port• Does the packet reach original AS?

fifi

IEPG IETF 115

What if packets would
traverse the rst AS?
DOPTs
Reverse traceroute on paths
where drops happen in rst AS(n=271 paths for UDP)

DOPT UDP(UK)
DOPT TCP(UK)
DOPT UDP(Canada)DOPT TCP(Canada)% make it back tooriginal AS%predictedtraversal

97.6%

~96%

TBD

TBD

95.3%

~96%

TBDTBD% make it back %predicted
to original AS
traversal

HBOPTs
Reverse traceroute on paths
where drops happen in rst AS(n=3150 paths for UDP)HBHOPT UDP
(UK)

10%

~17%

HBHOPT UDP(Canada)

17%

~25%

Notes
60% packets
get dropped at
LINX peering

fififiIEPG IETF 115

Traversal vs size
HBHOPTDOPT

EH length in Bytes

EH length in Bytes• TCP sees the biggest drop in traversal at 48B: 48 + 20 = 68B(108B total)• UDP sees the biggest drop at 56B: 56 + 8 = 64B (104B total)• 40B - the magic number?
IEPG IETF 115• 40B is the max for IPv4 options

PathSpider Experiments• 12 globally distributed vantage points• DNS/{UDP,TCP}/IPv6Tests:• Valid PadN
• Invalid length• Unknown option• {DOPT, HBHOPT}• IPv6 authoritative NS-es for domains in
Alexa Top 1M list (=~20,000 targets)• Test for valid DNS response!!

Source

Hop 1

Hop 2

Hop 3

Hop N

Consumer Edge

Destination

Server Edge

Not a “traversal” test! It is a “functional” test! IEPG IETF 115
10

Protocol and size:
DNS server edge
DOPT - UDP

DOPT - TCP

HBHOPT - UDP HBHOPT - TCP

8B

53%

51.16%

16.2%

15.7%

16B

9%

8.7%

2%

2%

N= 19966 DNS Servers autoritative for Alexa Top 1M domain
Test is “successful” if the server replies to a DNS Queryff• Very small di erence between TCP and UDP• DOPT results validated from 12 locations• HBHOPT results validated from 3 locations

IEPG IETF 115

Server Edge ASes not
passing DOPTs• AS 63911 NetActuate (880)• AS 8075 Microsoft Corporation (926)• AS 397238 Ultradns (4572)• AS 209453, AS 29169 Gandi (6136)• AS 16509 Amazon-02 (n=16668)• AS13335 Cloud are (n=28098 paths validated over 12 locations)
If these were transparent, E2E test success would be 87%flTest from 12 locations, for 20,000 servers (number of paths tested in brackets).

IEPG IETF 115

Which elds are inspected?
Invalid total
EH length
leads to99.9%
drops

Invalid
optionlengths leadto 99.9%
drops\x11 \x00 \x01 \xbb \x00\x00\x00\x00
Invalid EH or Opt Length makes a difference to traversal
Invalid or unknown option type does not make a difference
Payload of option does not make a differencefiIEPG IETF 115

What did we learn this time?
• DOPTs currently travel very far along a path, seen in bothconsumer and server edge networks• However, some edge paths still drop packets with DOPTs• HBHOPTs are currently dropped on many types of paths edge CPE, CDNs (Akamai, Cloud are and friends), mobile
networks and some transit networks• A diverse set of paths still support HBHOPTs• TCP sees higher drops at the consumer edge
IEPG IETF 115
fl14

QUESTION TIMEIEPG IETF 115

Experiments Overview
ProtocolTest

Targets?

Additional tests

End2End traversal tests

Traceroute-style tests

DNS/{UDP,TCP}/IPv6{UDP, TCP}/IPv6

Send a DNS query and tests Records ICMP replies fromfor valid responses
routers along the path
IPv6 authoritative NS-es fordomains in Alexa Top 1M
list (=~20,000 targets)• Valid PadN
• Invalid length• Unknown option

5000 RIPE Atlas probes toand from 2 vantage points

8, 16, 32, 48, 56, 64B

Both tests done for Hop By Hop and Destination Options
IEPG IETF 115

fiAnatomy of a HBHOPT/DOPT Ext. Header
Total lengthin 8-octets(!), notincluding
rst 8 bytes

LengthofOption: 4 8B HBHOPT or DOPT EH with a PadN Option
bytes\x11 \x00 \x01 \x04 \x00\x00\x00\x00
Next header:
UDP (17 dec,11 hex)

Payload ofoption

Type ofoption:
PadN is01

16B HBHOPT with a PadN Option and a PMTU Option\x11 \x01 \x01 \x04 \x00..\x30 \x04 \x05\xac\x00\x00
TypeLenValueTypeLenValue

2 PAD1
options

IEPG IETF 115

Comparison to RFC7872• Drops at the server edge AS have increased compared toRFC7872 (result of a few major players), but drops in ASes
other than the destination edge have decreased• Transit networks see better traversal

IEPG IETF 115