MPLS Network Management
MPLS Japan 2004
Ripin Checker, Product Manager, Cisco Systems
rchecker@cisco.com© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

1

Agenda for Today• Drivers for MPLS Management• Technical Requirements• Update on IP/MPLS Tools• Summary© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 20042

Service Provider Network Operation
OSSOSS
TDM
TDM
IPOSS

FR, ATMTDM

MPLS

OSS
FR, ATMOSS
FR, ATMIPOSS• Create operational efficiencies and increase automation in ahighly technology-intensive market• Enable competitive differentiation and customer retentionthrough high-margin, bundled services• Progressively consolidate disparate networks• Sustain existing business while rolling out new services© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

3MPLS Services and Transport Network Management

MSE

Programmable
L3VPN
Interface

Scale
ATM/FR
legacy featureparityPerformance
Connection
Management
L2/L3 + Optical

L3VPN

instrumentation

Provisioning
OAM &Troubleshooting
AToM

Traffic Eng

Low End Edge
Evolution from today
Managing CPE

Multicast VPN
Traffic
Engineering

Interworking
L2TPv3L2VPN

VLAN
Management

MAC address
L2 Switch
Management
Management

Metro E

Layer 2/3 Management Essentials: IP/MPLS Routing,
QoS, TE, OAM, HA© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

4MPLS Service Provisioning Challenge
Challenges in VPN Service Provisioning
65%

Integration with OSS and Billing System

Number of Network elements involved

47%

Integrating Multiple
technologies into a single service

47%

Finding customer
self-provisioning tools that work

41%

Equipment Manufacturer management
system not designed for provisioning

41%
35%

Interoperability among products

Manual configuration of equipment

18%
10%

20%

30%

40%

50%

60%

Source: Infonetics, 2003
© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

5

Service Provider CapEx vs. OpEx
• CapEx typically follows
the economy• OpEx is consistent$9.0• Typical ratio of a Tier 1 carrier
CapEx vs, OpEx spending• OpEx efficiencies have higher
profitability and a higher ARPU
Operations
and Support
Expense$8.0

Operational
ExpenseCapitalExpenditure

CapitalExpenditure
$7.0

$6.0

$ Billions

$5.0

27%

$4.0

$3.0

73%

$2.0

$1.0

$0.0

Q1

Q2

Q3

Quarterly Spending© 2004, Cisco Systems, Inc. All rights reserved.

Q4

Source: Typical Incumbent Carrier Balanced Sheet, Frost & Sullivan
MPLS Japan 2004

6Intelligent Information Networks
Complex networks require simplification throughManagementToday’s
Network TCO

Operations
Operations(OSS(OSS && Staff)
Staff)

Operations(OSS & Staff)• Heavy applications
duplicate effort andinvestment• Can’t hire and train
enough people

Equipment
Equipment
Network
Network

Equipment
Equipment
NetworkNetwork

INTELLIGENCEINTELLIGENCE• Intelligence trappedin people andapplications

Reduce complexity through embedding
intelligenceIntelligence pushed
into the network
instrumentation,
infrastructure
Smarterapplications
leverage network
intelligence

Automate whereappropriate
Guide humanintervention

Source:The Yankee Group© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

7

A Sample of Typical SP Customer Concerns• With all the flow through provisioning the mostcomplicated part is to make sure that theconfiguration has worked• Solving MPLS VPN connectivity problems is acomplex task for CCIEs• Is QOS configuration network specific or service
specific and is complicated to manage andtroubleshoot• Troubleshooting performance degradation in MPLS/IP
networks is the most labor intensive NOC activity• How do I transition PVCs from our ATM core toIP/MPLS infrastructure and that too in a Multi-Vendor
Network• Can you help me get better use of IP/MPLS core –particularly increased use of existing network
bandwidth resource© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

8

Customer Requirements - Fault
Fault Detection and Isolation–Control Plane Verification–Consistency check–Authentication–Data Plane Verification–Ability to verify connectivity and trace

Paths from PE to PE – Global routing table as well as VPNs
Paths from CE to CE within a VPNTE tunnels
Pseudo-wires

Data plane OAM packets must follow same path they are testing!

© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

9

Customer Requirements – Fault(Continued)• Need for Fault management functions at all layers –Device, Network, Service, EMS, Partners• Provide Reactive and Proactive Fault Mgt tools• Ability to identify specific service failure to a node orline card failure• Interwork with other technology fault management
tools to provide end-to-end fault correlation – OAMInterworking© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

10

Customer Requirements –Configuration/Provisioning• Configure OAM functions on the network devices
depending on the PE, P, Managed CPE roles• Verification of Service after Configuration• Alarm Mapping - configuration• Automate OAM functionality for pro-active
monitoring• Large Scale Service Provisioning• Standards based interfaces to devices – SNMP,
Programmatic Interface© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

11

Customer Requirements – Accounting/Performance• MPLS Flow accounting – with label stack information• Flexible and extensible export format – Netflow v9• Measurement of Key end-to-end MPLS performancestatistics like – Network delay, Packet Loss, Jitter,Connectivity• Proactive Performance measurement by raising
triggers based on events like Connection
loss/Timeout, RTT threshold etc• Historical and Real time MPLS statistics© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

12

Basics: VCs & LSPs
Virtual Circuits
Bi-directionalLabel Switched Paths
Usually Uni-directional

Established via ATM
Establishment tied
Signaling or Management closely to control planes
Fixed hierarchy VP/VC

Variable Label Stack

Connection orientedCan be “connectionless”

Single routeMay use ECMP

No penultimate popping

Penultimate hop popping© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

13

IP Troubleshooting Tools• Based on ICMP Echo Request and Reply– IP Ping– IP Traceroute– VRF Ping
– VRF Traceroute• Issues with ICMP Ping for MPLS network– Different behavior based on IP or MPLS core
– Does not detect MPLS data plane failure if IP layer
works fine
– Does not provide sufficient reply data to isolate
fault to MPLS specific issue© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

14MPLS LSP Ping/Traceroute• Detect MPLS traffic black holes or misrouting

Requirement• Isolate MPLS faults• Verify data plane against the control plane• Detect MTU of MPLS LSP paths• MPLS LSP Ping for connectivity checks

Solution• MPLS LSP Traceroute for hop-by-hop fault localization• MPLS LSP Traceroute for path tracing• IPv4 LDP prefix, VPNv4 prefix

Applications• TE tunnel• MPLS PE, P connectivity for MPLS transport, MPLS
VPN, MPLS TE applications

IETF
Standards© 2004, Cisco Systems, Inc. All rights reserved.

• Draft-ietf-mpls-lsp-ping-06.txt

MPLS Japan 2004

15

LSP Ping/Traceroute Example
MPLS Echo Request
TargetrouterOriginatingrouterPing

MPLS Echo Reply

MPLS Echo Reply

TTL=11Target
router
2

Traceroute34

TTL=25TTL=3© 2004, Cisco Systems, Inc. All rights reserved.

6789

MPLS Japan 2004

16

MPLS AToM Virtual Circuit
Connection Verification ( VCCV)• Ability to provide end-to-end fault detection and diagnostics
for an emulated pseudowire service

Requirement

One tunnel can serve many pseudowires.
MPLS LSP ping is sufficient to monitor the PSN tunnel (PE-PEconnectivity), but not VCs inside of tunnel• AToM VCCV allows sending control packets in band of an
AToM pseudowire. Two components:Solution

Signaled component to communicate VCCV capabilities as partof VC label
Switching component to cause the AToM VC payload to be treatedas a control packet
Type 1: uses Protocol ID of AToM Control wordType 2: use MPLS router alert label

Applications
IETF Standards© 2004, Cisco Systems, Inc. All rights reserved.

• Layer 2 transport over MPLS
FRoMPLS, ATMoMPLS, EoMPLS• Draft-ietf-pwe3-vccv-xx.txt

MPLS Japan 200417

Troubleshooting - VRF data plane
Detection
VRF ping Fail (IP)VRF traceroute

Remote PEin trace

Possible Access/Customer
Network Problem Inform operator

Remote PE
Not in trace

VRF, IP,MPLS ?

PE-PE ping Check (IP)

Problem

Inspect Routing Configuration
Troubleshooting

OKProblem
LSP ping¥trace (IGP label)

Inspect MPLS Configuration

OKProblem
LSP ping¥trace (VRF)© 2004, Cisco Systems, Inc. All rights reserved.

Inspect VPN Configuration
MPLS Japan 2004

18

MPLS Traffic Engineering: AutoTunnel –Primary, Backup, & Mesh Groups
Requirement• Ability to protect links and nodes with no requirement of “trafficengineering”• Need to ease configuration of “increased bandwidth inventory” MPLS
TE designs such as full mesh
• Backup AutoTunnel—Enables a router to dynamically build
backup tunnels

Solution• Primary one-hop AutoTunnel—Enables a router to dynamically create
one-hop primary tunnels on all interfaces that have been enabled withMPLS TE tunnels• Mesh Group AutoTunnel – Enables automatic establishment of full- or
partial-mesh of TE tunnels• MPLS VPN with multiservice SLAs (voice, video, and data sites)Applications• MPLS AToM-based Layer 2 services with “Bandwidth Assurances”• Enhanced SLA service offerings with low packet loss during failure
condition – “Bandwidth Protection”• draft-ietf-mpls-rsvp-lsp-fastreroute-03.txt

IETF Standards• draft-ietf-ospf-cap-01.txt• draft-vasseur-mpls-ospf-te-cap-xx.txt© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

19

MPLS Traffic Engineering
AutoTunnel – Primary & Backup
Router A establishes AutoTunnels to adjacent routers –“automates” configuration of Link & Node ProtectionAutoTunnel
Primary

Router B
Backup AutoTunnel – Next
Hop – “Link Protection”

Router A

Router D

Router configured withAutoTunnel Primary & Backup
Router C© 2004, Cisco Systems, Inc. All rights reserved.

AutoTunnel Backup Next
Next Hop – “NodeProtection”

MPLS Japan 2004

20

MPLS Traffic Engineering
AutoTunnel – Mesh Groups
Routers A, B, C, D, E – defined as members of “Mesh Group 1”
Capable of building multiple meshes for DiffServ aware Traffic Engineering
Automates configuration of full mesh of TE Tunnels resulting in operational efficiencies
SoftswitchVR

Router BRouter C

Router A
Service ProviderBackbone

Router DPSTN

Router E

Partial Mesh of
Physical Connectivity© 2004, Cisco Systems, Inc. All rights reserved.

AutoTunnels belonging
to “Mesh Group 1”
MPLS Japan 2004

21

BFD for MPLS LSPs
Low-overhead, shortduration failure detection in
the path between adjacentforwarding engines
Including theinterfaces, data link(s), &forwarding planes• There is no discovery
mechanism in BFD• Need a means tobootstrap© 2004, Cisco Systems, Inc. All rights reserved.

....

Router....
....

X....
....

IP/MPLS
IP/MPLS
TunnelTunnelXRouterForwarding Plane• In some cases carriers would liketo monitor LSPs
• Accomplished with a TLV in LSP
Ping that carries the BFD
discriminator• Applications – BFD+LDP,
BFD+VCCV, BFD+TE, BFD+BGP,
BFD+IGP
MPLS Japan 2004

22MPLS Network and Services MIB Summary andConcept/ Architecture/Dependencies
MPLS

MPLS LSR MIB

Routing

MPLS-TE MIB
MPLS-TC MIBOSPF MIBs
MPLS-FTN-MIB
MPLS-LINKBUNDLING-MIB

BGP MIBs

MPLS-LDP-MIB

PWE3

Service-Specific
MIBs…

PPVPN
PPVPN-TC MIB

PW-TC MIBPW MIBPPVPN-MPLSVPN MIB

PW-CEM MIB© 2004, Cisco Systems, Inc. All rights reserved.

PW-MPLS MIB

Depends on
Depends on
RFC2233

MPLS Japan 2004

23

Putting It All Together
Media type
ATM VP

CC CP

CC DPILMI

ATM VCFR
Ethernet
last mille
Ethernetprovider bridge

LMI

Performance Traceroute

F4 (VC-3)

F4 (VC-4)-

F5 (PT 100)

F5 (PT 101)---

Keepalive
IEEE 802.3ah-

ITU-T -Draft Y.ethoam-

IEEE 802.1ag (MAC: broadcast domain)
MPLS BFD

ITU-T
LSP
PingDraft Y.17fw-LDP HelloVCCV BFDVCCV Ping--

IGP/BGP HelloBFD

IP Ping-IP TR

MPLS LDP

LDP Hello

MPLS TERSVP Hello

MPLS PW
IPv4Loopback

LSP TR•OAM Interworking for End-to-End Network and Service Level OAM•Not just Vendor Interoperability but Standards alignment as well
© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

24

MPLS Network Management Life
Cycle
OSS

ISV Partners

Fault Management
Process
NetworkAlarms

Determine
Service Impact

ServiceProvisioning

Performance• VPN Topology• Service Provisioning

Fault

Performance
Accounting

EMS
EMS

Accounting

Device/Network Provisioning

End-End OAM
Attachment VC OAM’sCEIngressPE

MPLS OAM

LSP created by LDP and/or RSVP-TE

Attachment VC OAM’sEgressPECE

PWE3 orVPN Label© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 2004

25

Summary• Cost pressures driving
convergence, consolidationand virtualization• Complexity of networked
systems and scarce expertiseare increasing Total Cost ofOwnership (TCO)• MPLS Network management
require systems andsolutions perspective

Automation Productivity

Simplification

ROI/TCOIntelligence• Standards-based open
interfaces for easier andfaster integration© 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 200426

EthernetOAM
Standards
Status© 2004,
CiscoSystems,Inc.
All rights reserved.

© 2003, 2004, Cisco Systems, Inc. All rights reserved.

MPLS Japan 200427