次世代IX研究会におけるルータ相互接続試験
A Report of Router
Interoperability from Distix
菊池 豊
KIKUCHI, Yutaka
yu@kikuken.org
2002.Nov.08

MPLS Japan 2002

1

Agenda• Next Generation IX Consortium– MPLS-IX Architecture– Working Groups• Router Interoperability Test
– Policy– Test Specification– Results• Discussion
2002.Nov.08

MPLS Japan 2002

2

Next Generation IX Consortium• Background– ATM and LAN Equipments has been used for IX– Limitation of Scalability (Bandwidth and Flexibility)• Goal– Propose a new IX architecture– Verification and Validation on an actual network– Sharing know-how about the new technology• We often call it Distix, but what is Distix?
– An Ideology? A model? A technology? An test bed?
– Nobody knows about it correctly. (discovered last night)
2002.Nov.08

MPLS Japan 2002

3

Our proposal• Requirement of IX– Independency from data-link architecture– Widely distribution and scalability– Easy to maintain• Let’s use MPLS for IX– Allows any L2 arch. (e.g. PoS, GbE, ATM)– Easy to extend equipments and exchange points– Behaves as one big IX machine
2002.Nov.08

MPLS Japan 2002

4

MPLS-IX Architecture• Basic architecture– Core routers as a backbone– Customer edges as MPLS edges– Peering ISPs establish LSPs
– IX handles label information only
Carrier’s router
CORE LSR

Carrier’s network

User router
EDGE LSR

Carrier’s router
CORE LSR

User router
EDGE LSR
2002.Nov.08

MPLS Japan 2002

5

MPLS-IX Architecture(routing domain)
MPLS IX

IGP (OSPF or ISIS)
Static route to A

Static route to B

Disjoint routing domains
against affects on the other sideA
ISP1
2002.Nov.08

B
MPLS Japan 2002

ISP26

MPLS-IX Architecture(Establishing LSP)
LDP
RSVP-TELDP
RSVP-TE

MPLS IXLDP
RSVP-TE

LSP
E.g. Label=16

A
ISP12002.Nov.08

LDP
RSVP-TE

B
MPLS Japan 2002

ISP27

MPLS-IX Architecture(Establishing BGP)MPLS IX

LSP
Label=16A
ISP12002.Nov.08

BGP•Multi-hop BGP•TTL = 1
BGP will shutdown
when LSP is gone
(BGP must not keep
MPLS Japan 2002by detour)

B
ISP28

MPLS-IX Architecture(TTL handling)IP
TTL=1
MPLS TTL=254

IP
TTL=1
MPLS TTL=253

IP
TTL=1
MPLS TTL=252

IP
TTL=1MPLS TTL=255•IP TTL=1 between A and B•A and B seem to be neighborA
ISP1
2002.Nov.08

B
MPLS Japan 2002

ISP29MPLS-IX Architecture(Exchanging routing information)
MPLS IX

LSP
Label=16
ISP2 : Next Hop=B

A
ISP12002.Nov.08

Routing table
Dst Label
ISP2 16MPLS Japan 2002

BGPB
ISP210MPLS-IX Architecture(IP datagram forwarding)
MPLS IX

LSP
ISP2

１６ ISP2

BGP

A
ISP2
ISP1
2002.Nov.08

経路表
Dst Label
ISP2 16B
ISP2
MPLS Japan 2002

ISP211MPLS-IX Protocol Design(summary)Label swap in the
MPLS domain

BGP4 (iBGP/eBGP)
User network

User network
LSP (Label Switched Path)LDP

User router
2002.Nov.08

LDP

IX router

LDP

IX router

MPLS Japan 2002

User router
12

MPLS-IX Protocol Design(3D version)
Data Traffic
over LSP

BGP4 over LSP

LSP between Edges
OSPF is also elabled
between Cores
Signaling protocols
CORE
EDGE-1
2002.Nov.08

CORE

CORE
CORE

MPLS Japan 2002

EDGE-2
Physical / datalink
connectivity
13

Working Groups• IX Users WG– Verify the technology by using an actual IX– Discuss issues in IX users point of view

• IX Providers WG– Verify the technology by operating an actual IX– Discuss issues in IX providers point of view• Router Interoperability WG– Verify the interoperability on multi-venders
2002.Nov.08

MPLS Japan 2002

14

An Experimental Test Bed
kochi
17521
KCN
9363
Ayame
17932

FITWeb7672

Kikuken
17690

WAG
9612

TAM
7676

QGPOP
2523
ZTV
9351
Hi-Ho
4668

Softopia
9350

HTCN
7668

CoreToyama

Core
Kochi

Shiojiri
17518

Trial#1
65501

nagoya-u
17687
Trial#2
65502

Coreu-tokyo

NORTH
7661

Japan Gigabit Network
Core
DojimaSquare
17685
Iamas
18076
Gunma
17678

BeX-J
17942

Core
K-otemachi

CoreN-otemachi

HOTnet7682
Accelia
17686

IIJ
2497Japan TelecomCarrier

2002.Nov.08

MCI WorldCom

MPLS Japan 2002

15

Router Interoperability Test
• Regular Interoperability of MPLS
– heterogenous router vendors• Special Interoperability for MPLS-IX– Basic Functions• Connectivity (LDP/RSVP-TE)• Routing (OSPF, BGP4)• TTL, Filtering, SNMP support– Extended Functions• 6PE, RFC3107, LDP over RSVP-TE• Redundancy, QoS• Performance (is not related to interoperability but)
2002.Nov.08

MPLS Japan 2002

16

Policy of the Interoperability Test
• Open– Participation under the Agreement of Distix– Publication the results by WWW• At Participants own Expense– Equipments, Transportation, Stay, and so on• Even the site is far from Tokyo– No public budget for it• Help each other, share our knowledge and•2002.Nov.08
Enjoy!
MPLS Japan 2002
17

The Test Steps• Preparation– Specify testing methods– Finding the site (difficult to find)• Test– Monday to Friday in a week
– 0900 to 1900 everyday• Summarizing the results– Classify reliable data

• Publication on the WWW
2002.Nov.08

MPLS Japan 200218

Required Site Environments• 100 m2 at least (10m2 each per vender)– Human/Machine separation are welcome• AC100V 100A at least– AC200V and DC-48V are welcome• Reachability of Internet and portable phones• Public transportation• Accommodation if outside of Tokyo area
2002.Nov.08

MPLS Japan 2002

19

History• 2001.10.15-19– TAO JGN IPv6 Center (Tokyo)• 2002.01.28-02.01– TAO Research Center (Chiba)• 2002.05.13-17– Netmarks Quality Management Center (Tokyo)• 2002.07.01-05– N+I 2002 showcase (Chiba)• 2002.09.02-06– TAO Hokuriku IT
Center (Ishikawa)
MPLS Japan 2002

2002.Nov.08

20

Participants

exp1 exp2 exp3 N+I exp4(Agilent)
AYAME ProjectCisco Systems
Digital Magic Labs.
Extreme Networks
Foundry Networks
Fujitsu
Furukawa Electric
HITACHI
Juniper Networks
NEC (CX)
NEC (IX)
Netmarks(TOYO Technica)
2002.Nov.08
Riverstone Networks

○○
○
○
○
○
○
○
○

○
○
○
○
○
○
○
○
○
○

○
○
○

○

○
○

○
○
○
○
○
○
○
○

○

○

○

MPLS Japan 2002

○○○

○
○
○

○
○
○
○
○
○
○
○
○
○
○

○
○
○
○
○
○
○
○

21

Test Items• Edge LSR (as CE)– Capability itself, Interoperability• Core LSR (for IX)– Capability itself, Interoperability• Extended Menu
– Behavior check in a redundant topology– 6PE (IPv6 forwarding with IPv4 control plane)– RFC3107 (label carry by BGP)
– LDP over RSVP-TE– Performance (label push, swap and pop)
2002.Nov.08

MPLS Japan 2002

22

Test Specification of Edge Capability(General)• NOT copy TTLs
– IP TTL to MPLS TTL at Ingress– MPLS TTL to IP TTL at Egress• If capable of non-PHP• BGP– multi-hop with IP TTL=1– BGP packets over LSP• Check packets really labeled by packet dump
• Check IP forwarding by injecting packets from theout side of Edge LSRs
2002.Nov.08MPLS Japan 2002

23

Test Specification of Edge
Capability (LDP)• Advertisement labels (≧ 16) without PHP• LSP Filter (according to FEC by signaling)• Transport address of LDP session– Loopback address and/or interface address• Advertisement FEC– Loopback address and/or interface address• SNMP capability– Traffic of LSP– Trap of LSP up/down– Trap of LDP session up/down
2002.Nov.08

MPLS Japan 2002

24

Test Specification of Edge
Capability (RSVP-TE)• Advertisement labels (≧ 16) without PHP• LSP Filter (according to FEC by signaling)• Src/dst of RSVP PATH message– Loopback address and/or interface address• SNMP capability– Traffic of LSP– Trap of LSP up/down– Trap of LDP session up/down
2002.Nov.08

MPLS Japan 2002

25

Test Specification of Edge
Interoperability (common)• Establish an LSP• Exchange BGP packets over the LSP• Exchange routing information– by BGP multi-hop with IP TTL=1• Not fragment any packets within 1500 bytes• IP TTL=n+2 at Egress if n at Ingress
2002.Nov.08

MPLS Japan 2002

26

Edge Interoperability Test
RSVP/LDP

RSVP/LDP

RSVP/LDPeBGP
Traffic

2002.Nov.08

MPLS Japan 2002

27

2002.Nov.08

MPLS Japan 2002

28

Ishihara’s Ping Engine• UDP one-way ping
– Send a UDP datagram (specified length)– Check by the reception of the datagram– Tagged VLAN capability• Implementation– Wild lab. LAMB-RT-01SP/32– FreeBSD 4.6-Release– Software constructed on hping and libnet• http://www.hping.org/• http://www.packetfactory.net/libnet/
2002.Nov.08

MPLS Japan 200229

Views of the Ping Engine

2002.Nov.08

MPLS Japan 2002

30

Ping Engine with tagged VLANs
Tagged VLAN

2002.Nov.08

Tagged VLAN

Edge

Edge

Edge

Edge

Edge

Edge

MPLS-IX

EdgeEdgeEdge

Edge
MPLS Japan 2002

31

Test Specification of Core
Capability (General)• NOT copy MPLS TTL to IP TTL– Only if capable of PHP• Advertisement interface addresses of edge
routers (/32) via OSPF

2002.Nov.08

MPLS Japan 2002

32

Test Specification of Core
Capability (LDP/RSVP-TE)• Filter for receiving LSPs
• SNMP capability– Traffic of LSP– Trap of LSP up/down

2002.Nov.08

MPLS Japan 2002

33

Test Specification of CoreInteroperability (Common)• Check at the edge LSRs
– Establish an LSP– Not fragment any packets within 1500 bytes– IP TTL=n+2 at Egress if n at Ingress

2002.Nov.08

MPLS Japan 2002

34

Sample Results• Edge capability– General, LDP, RSVP-TE• Edge interoperability– LDP (loopback, Interface)– RSVP-TE (loopback)• Snapshots by Ishihara’s ping engine– RSVP-TE (interface)• Short packets, Long packets
2002.Nov.08

MPLS Japan 2002

35

Testing View (1)2002.Nov.08

MPLS Japan 2002

36

Testing View (2)2002.Nov.08

MPLS Japan 2002

37

Testing View (3)2002.Nov.08

MPLS Japan 2002

38

Testing View (4)2002.Nov.08

MPLS Japan 2002

39

Testing View (5)2002.Nov.08

MPLS Japan 200240

Testing View (6)2002.Nov.08

MPLS Japan 2002

41

Testing View (7)2002.Nov.08

MPLS Japan 2002

42

Testing View (8)2002.Nov.08

MPLS Japan 2002

43

Testing View (9)2002.Nov.08

MPLS Japan 2002

44

Summary of the Past Results• Basically sufficient– Still some troubles may occur by design,
implementation, and versions.
– Does mostly interoperability depend on what signaling
they implement, LDP v.s. RSVP-TE?

• Insufficient in an operational point of view
– Without PHP• IX Users have a thirst for labels at egress LSRs

– SNMP support• MPLS-IX consists of many routers even at a distance2002.Nov.08

MPLS Japan 2002

50

Discussion• Verification of the interoperability for MPLS-IX• Reconfirmation requirements and design of theMPLS-IX architecture• Contribution to MPLS community– Attractive and Important Experiences• A wall of Complexity of O(n2)• Shift to new Features• A new direction will be needed– We will pass a sub goal in the next (5th) test
– Collaboration with other WGs is necessary

2002.Nov.08

MPLS Japan 2002

51

See in detail• Distix Home Page
– http://www.distix.net/• Distix Router Interoperability WG– http://www.distix.net/router-wg/• 5th Interoperability Test in Kochi– Coming up soon!
– Venders,• Do not forget sending an application of participation
2002.Nov.08

MPLS Japan 2002

52

Thank you!
• To all the participants ofthe interoperability tests• To the audience• Distix was sponsored by
TAO I121-401
2002.Nov.08

MPLS Japan 2002

53