Evolving OAM Requirements with IP-Based Tools
Monique J. Morrow,
CTO Consulting Engineer
mmorrow@cisco.com

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

1Motivation• Align MPLS OAM Requirements with IP-basedtool constructs• Comparison of various mechanisms

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

2

Agenda• Why IP-Based Tools?
• Requirements• LSP Ping
• BFD/VCCV• OAM Message Mapping• ITU-T/IETF Mechanism Overview• Future Directions• Summary
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

3

Why IP-Based Tools?

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

4

Why IP-Based Tools?
•MPLS is IP-based•All MPLS control protocols are based on
the IP protocol suite
LDP / BGP / RSVP / PIM•Facilitate evolutionary
implementation of suchmechanisms in deployed networks
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

5

ATM OAM Does NOT Equal MPLS OAM
Virtual Circuits
Bi-directionalLabel Switched Paths
Usually Uni-directional

Established via ATM Signaling or Establishment tied closely toManagementcontrol planes
Fixed hierarchy VP/VC

Variable Label Stack

Connection oriented

Can be “connectionless”

Single routeMay use ECMP

No penultimate popping

Penultimate hop popping

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

6Requirements

Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

7

Requirements (1)• Three categories of requirements gatheredfrom 1st tier PWE/MPLS Service Providers(and others).
9VC/LSP Path Verification and Tracing
9Built-in Protocol Operations
9Standard Management APIs/NMS Applications
MIBs, CLI, XML, etc…
¾Documented in:¾draft-ietf-mpls-oam-requirements-01.txt¾Must be addressed before many providers will deploy PWE3services.

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

8Requirements (2)•Control plane verification of information
Consistency check
Authentication•Data Plane Verification•Ability to trace paths from PE to PE – Global routing
table as well as VPNs
•Ability to trace paths from CE to CE within a VPN•Ability to trace LSPs with ECMP
•Ability to Trace TE tunnels
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

9LSP Ping

Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

10

LSP Ping• Similar to ICMP (IP) Ping
Sequence Number
Timestamps
Sender Identification• Full identification of FEC based the application• Variable length for MTU discovery• Support for tunnel/path tracing• Multiple-reply modes• Handles ECMP
• Reference
http://www.ietf.org/internet-drafts/draft-ietf-mpls-lsp-ping-03.txt
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

11

MPLS Ping: Operation• Ping Mode: Connectivity check of an LSP
Test if a particular "FEC" ends at the right egress
LSR• Traceroute Mode: Hop by Hop fault localization• Uses two messages
MPLS Echo Request
MPLS Echo Reply• Packet need to follow data path

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

12

MPLS Ping Message Format

012
30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Version Number|Must Be Zero|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Message Type |Reply mode | Return Code | Return Subcode|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Sender's Handle|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Sequence Number|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|TimeStamp Sent (seconds)|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|TimeStamp Sent (microseconds)|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|TimeStamp Received (seconds)|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|TimeStamp Received (microseconds)|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
TLVs ...
|
::
::
|
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

Message Type
1 Echo Request
2 Echo Reply
Reply Mode
No replyIPv4 UDP packet
IPv4 UDP packet withRouter alert
Control Plane
TLVs include
FEC to be checked
13

MPLS Ping: Packet Flow
• Ping with label for FEC=192.169.10.0/24• Label Switched at R2, R3• R3 pops label off• R4 processes packet

R3

192.168.10.0/24
R1

R2

R5R4
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

14

Packet Flow Ping Mode:
Egress node

R3R1

R2R5R4• Check Packet integrity• Check if FEC distribution protocol is associatedwith incoming interface• Check if valid egress node for the FEC• Send echo Reply according to value of Reply Mode

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

15

MPLS Traceroute: Packet Flow
• MPLS Ping Packets are sent with TTL=1,2,3• Label switched if TTL > 1• Processed where TTL expires

R3

192.168.10.0/24
R1

R2

R5R4
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

16

Trace Mode: TTL>1• Copy one Downstream Mapping (DM) TLV from EchoReply• Pick one IP Address from address in DM TLV• Send a new Echo Request with TTL+1• Repeat (if appropriated) for each DM TLV• Reply from Egress stops iteration

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

17

R3Packet Flow Trace Mode: R1

R1

R2

R5R4

MPLS Header
TTL=1,2,3,4…..
IP
IP Src: IP-R1;
IP Dst: 127.x.y.z
TTL: 1
Router Alert
UDP
Dest Port: 3503
PayloadMessage Type=1
Reply Mode=2,ReturnCode=0
Sender Handle
Sequence Number
TimeStamp Sent
TLVs
Target FEC stackDownstream mapping

MPLS Japan 2003
M.Morrow

Downstream Mapping TLV
1230 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Downstream IPv4 Router ID|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|MTU| Address Type=1|
DS Index|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Downstream Interface Address|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Hash Key Type | Depth Limit|
Multipath Length|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|IP Address or Next Label|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+:::(more IP Addresses or Next Labels):::+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|Downstream Label|Protocol|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+© 2003 Cisco Systems, Inc. All rights reserved.

18

Packet Flow Trace Mode:Transit Node

R3R1R2R5R4• Reply processing same as Ping, then
• Check for Downstream Mapping TLVDetermine nexthop routers• Add Downstream Mapping TLVs for each
Compute label stacks, address/label ranges• Return received Label Stack if requested
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

19

Bidirectional Forwarding Detection/
Virtual Circuit Connection Verification

Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

20

Bidirectional Forwarding Detection•Simple, fixed-field, hello protocol• Nodes transmit BFD packets periodically overrespective directions of a path
• If a node stops receiving BFD packets somecomponent of the bidirectional path is assumed tohave failed• Several modes of operaton
VCCV uses Asynchronous mode
www.ietf.org/internet-drafts draft-katz-ward-bfd-01.txt

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

21

BFD Control Packet
01230 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|Vers | Diag
|H|D|P|F| Rsvd | Detect Mult |Length|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|My Discriminator|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|Your Discriminator|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Desired Min TX Interval|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Required Min RX Interval|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|
Required Min Echo RX Interval|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

22

Variable detection intervals• Each node estimates how quickly it can send andreceive BFD packets• Nodes exchange the follow parameters in everycontrol packet
Desired Min TX Interval
Required Min RX Interval
Detect Multiplier• These estimates can be modified in real time in orderto adapt to unusual situations

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

23

Determining Detection Time
TX – Transmission Interval
RX – Receive Interval
Note that TX(a->b) = RX(b->a)TX(a->b) = max(Desired Min TX(a), Required MinRX(b))
TX(b->a) = max(Desired Min TX(a), Required MinRX(b))
Detection Time(b) = Detect Mult(a) x T(a->b)
TX is jittered by 25%

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

24

Diagnostics
0 -- No Diagnostic
1 -- Control Detection Time Expired (RDI)
2 -- Echo Function Failed (N/A to VCCV)
3 -- Neighbor Signaled Session Down (FDI)
4 -- Forwarding Plane Reset (Indicates localequipment failure)
5 -- Path Down (Alarm Suppression)
6 -- Concatenated Path Down (used to propagate
access link alarms)
7 -- Administratively Down

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

25

Virtual Circuit Connection Verification(VCCV)Emulated Service
Pseudo Wire
PSN Tunnel
PW1

CE1PE1Native Service

PW2

PE2

CE2Native Service• Multiple PSN Tunnel Types
MPLS, IPSEC, L2TP, GRE,…

• Motivation•One tunnel can serve many pseudo-wires.
•MPLS LSP ping is sufficient to monitor the PSN tunnel (PE-PEconnectivity), but not VCs inside of tunnel.

• www.ietf.org/internet-drafts/draft-ietf-pwe3-vccv-00.txt
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

26

VCCV Overview• Mechanism for connectivity verification of PW• Features
Works over MPLS or IP networks
In-band CV via control word flag or out-of-band option
by inserting router alert label between tunnel and PWlabels
Works with BFD, ICMP Ping and/or LSP ping
• VCCV results may drive OAM/LMI injection oncorresponding AC(s)• http://www.ietf.org/internet-drafts/draft-ietf-pwe3-vccv-00.txt

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

27

In Band VCCV Format
Control word use is signalled in LDP - Standard form:
012
30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|0 0 0 0| Flags |FRG| Length| Sequence Number|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OAM uses a different 1st nibble
01230 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|0 0 0 1|

reserved| PPP DLL Protocol Number=IPvX|+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+|IP OAM Packet: Ping / BFD / LSP Ping|||+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

28

PWE3 OAM Example:
Continuity Verification

Attachment VCs
BFD Packet
over VCCV channel

LSP Tunnel

Attachment VC• BFD provides a lightweight means of regular periodic CV
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

29

PWE3 OAM Example:
Connection Verification

Attachment VC

VCCV Packet

Two Labels

Attachment VC•Verify/Trace Path of LSP Tunnels between PEs.
•Verify/Trace Emulated services (e.g. ATM, FR) mapped to Attachment VCs•Trace/Verify packets must take same path as data packets.
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

30

Example of Operation
CV/Trace Using VCCV and LSP Ping
NMS/mgr Triggers LSP pingtrace when failure detected
NMS/mgr Triggers VCCVVCCV Packet
Is lostAttachment VC

Attachment VC

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

31

OAM Message Mapping

MPLS Japan 2003

© 2003 Cisco Systems, Inc. All rights reserved.

32

OAM Message Mapping• Provides details of how LSP ping/VCCV failures
should translate into native ATM/FR OAM messages
that PEs return to the native attachment interfaces.
• OAM Emulation
AIS and RDI Generation for ATM AAL5 over MPLS
AIS/RDI Generation for ATM upon reception of label
withdrawal and vice versa• LMI/ILMI based status notification upon label
withdrawal reception for pseudowire virtual circuits• draft-nadeau-pwe-msg-mapping-01.txt
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

33

OAM Message Mapping Example
FRCPE

ATM
CPE
FR
UNI

FR CPE

FR

FR
UNI

FR
UNI/NNIFR PVC

Q.933
messages

Q.933
messages

ATMUNI

IP/MPLSATM CPEATMATM
UNI/NNI

MPLS PSN

Q.933
messages

MPLS PING,
Traceroute

ATMUNIATM PVC

F5/F4 end-to-end AOM cells
F5/F4 segment OAM cells

Interworking Q.933

Interworking AOM cells
AISRDICC

Loopback (I.620)
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

Loopback

34

ITU-T/IETF
Mechanism Overview

MPLS Japan 2003
M.Morrow
Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

35Addressing OAM Requirements
Two Basic Approaches
ITU
Requirements

IETF

Y.1710

draft-ietf-mpls-oam-requirements-01.txt

Y.1711 (a.k.a. CV)

BFD, LSR Self Test

FEC-CV (Y.17feccv), (Y.17fw)

MPLS Ping/Trace

L3

FEC-CV (Y.17feccv),

MPLS Ping/Trace

L2

FEC-CV (Y.17feccv), Y.17fw

VCCV, OAM state mapping

Instrumentation(Y.17fw)

MIB, Syslog

RecoveryY.1720, (Y.17fw)

FRR, MPLS HA, Graceful restart

Performance

Y.MPLSperf

Detection
MPLS
L3
L2
Diagnostic
MPLS

Security
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

36

Drawbacks of Y.1711• Does not work for Penultimate hop popping, which
is very commonly used.
• Assumes a fixed path/connection-oriented view ofthe world as in G.709 which is more the exception
than the norm.
• Requirement for a reverse path
This is the exception rather than the rule formost MPLS applications• Fixed interval between CV Packets of 1 second.
Will not scale for typical networks comprised oflarge numbers of LDP signaled LSPs.
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

37

Y.1711 in a Nutshell• Y.1711 will not scale for auto-routed LDP networks
which comprise about %90 of deployed MPLS
networks!
• Revolutionary approach requiring re-spin of manypieces of hardware.

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

38

Future Directions for ITU OAM• IP-based Tools approach added to y.17fw.
• Y.17fw to be consented, Feb 2004
Will leverage existing IP-based tools toovercome many aforementioned shortcomings.
Will integrate new tools (LSP ping/trace, VCCV)
into tool box of existing operator managementtools.
Evolutionary not revolutionary approach!

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

39

Future Directions

MPLS Japan 2003
M.Morrow
Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

40

Future Directions• Bidirectional Forwarding Detection (BFD)"Bidirectional Forwarding Detection", IETF draft-katz-ward-bfd01.txt“BFD for IPv4 and IPv6 (Single Hop)”, IETF draft-katz-ward-bfdv4v6-1hop-00.txt• Working on Ethernet OAM mechanisms (IEEE)• Interworking OAM is pivotal for converged services

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

41

Summary

MPLS Japan 2003
M.Morrow
Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

42

Summary• Most current applications offer an IP related service• IP is the basis of MPLS control planes• Most important, customer requirements highlightneed for IP-based tool mechanisms• Therefore, IP-based tools makes sense andprovides natural evolution of MPLS OAM tocurrent and future services!

MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

43

Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

44

References

MPLS Japan 2003
M.Morrow
Presentation_ID© 2003 Cisco Systems, Inc. All rights reserved.

45

References (2)¾ draft-ietf-mpls-oam-requirements-01.txt¾ draft-ietf-pwe3-vccv-00.txt¾ draft-nadeau-pwe-msg-mapping-01.txt¾ draft-swallow-mpls-lsr-self-test-00.txt¾ draft-katz-ward-bfd-01.txt¾ draft-katz-ward-bfd-v4v6-1hop-00.txt• Guest Editor Special Edition IEEE Communications Magazine on
topic of OAM for MPLS-Based Networks• Scheduled Publication, October 2004
• Call For Papers:
http://www.comsoc.org/pubs/commag/cfpcommag1004.htm
MPLS Japan 2003
M.Morrow© 2003 Cisco Systems, Inc. All rights reserved.

46