The Future of MPLSA Service Provider
ToolkitMatt Kolon
MPLS Japan
October 2003

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Service Provider Survival for 2004
• The market continually changes what service
providers need to offer and support.
• Maintaining service flexibility is crucial.
• Internet access is only one of these services.
• Addressing all services with a single
infrastructure is tricky.
• There is no place for technical dogma inengineering or planning.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Ethernet ccVLAN
ATM/FR
T1/E1
Ch T1/E1
Branch Office

Example Solutions: VPNs
Low
SpeedEdgeHighSpeedEdgeccATM

MPLS Corecc

T3/E3

DSL
Dial
End to End QOS
Cable
WiFi
GPRS

Subscriber Edge

Copyright © 2003 Juniper Networks, Inc.

GigE

Head OfficeConnectivity
HostedServices• Layer 3 – 2547
• Layer 2 – Martini/Kompella• VPLS• Subscriber/Site level QOS• IPSec Termination

Proprietary and Confidential

www.juniper.net

A Versatile IP/MPLS Toolkit• Versatility: ability to support multiple profitable
services via a common set of tools• Facilitate creation of new services that
supplement existing services• Reduce the cost of rolling out new services
Service providers need a toolkit that can support
expanding existing service offerings with minimum
additional effort/cost.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Overall• Simple Versatility• Have as few tools as possible…
• … but no less than needed to support the services• A single operational infrastructure and a small set ofbasic tools mean cost savings in terms of:•Educating the NOC staff•

Educating customers•

Building tools/expertise to manage services• But simplicity is not the ultimate goal, profitability is•

Simplicity≠ profitability

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS Capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Inter-provider VPNs•

Application•••Site 2
VRF PE

Allows Service Providers to partner andextend geographic reach of VPN
Standards based approach•

Explore new markets without the needto build new infrastructure•

Reach new customer base•new revenue stream
Reach - Common national orinternational networks•

Single bill•

Fewer relationships to manage

Copyright © 2003 Juniper Networks, Inc.

VRF

PEPE

Multi-hop EBGP
redistribution of VPN
IPV4 Routes

BGP/MPLS VPN
Provider (AS2)Enterprise Customer Benefits•PE

BGP/MPLS VPN
Provider (AS1)

Service Provider Benefits•

Site 2

VRFPEPE

Site 1

VRF

Site 1

Proprietary and Confidential

www.juniper.net

Other Inter-provider Work Needed!
• Billing• Accounting• QoS• Traffic Engineering• Settlement / Metering• Session Negotiation• Essentially, we are talking about one ormore new NNI definitions.
Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Multi-vendor Interoperability• What matters:• Interoperability• Time to market• A fitting solution to a well-defined problem• What does NOT matter:• IETF Standards Status• Internet Draft + working code is necessary, but alsosufficient.
• Credible vendors and providers understand this!

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

IETF Standards and time to market•

RSVP TE extensions:••

OSPF TE extensions:••Internet Draft since April 1999 – still not an IETF standard

2547 VPN (aka BGP/MPLS VPN)••

November 1998 Internet Draft - December 2001 ProposedStandard

Internet Draft since March 2000 – still not an IETF standard

draft-martini•Internet Draft since November 2000 – still not an IETF
standardthe IETF standards process takes longer andGoing throughlonger… speeding up the IETF process may take even longer…

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Class of Service
Diff-Serv – MPLS CoS Model
Subscribers
Subscribers

Policing
Policing
Shaping
Shaping

1000s
1000s ofof Flows
Flows

Content
Content

100s
100s ofof Flows
Flows

1000s
1000s ofof Flows
Flows1000s
1000s ofof Flows
FlowsBRAS
BRAS

Ingress Edge Function:
Upstream flows policed
Downstream flows shaped
Aggregate many flows &associate with a Class(marked with DSCP)
DSCP mapped to EXP bits in
MPLS header
Queue and drop accordingly

Copyright © 2003 Juniper Networks, Inc.

Core Function:

Egress Edge Function:

perform the appropriate queuingand dropping based on class(EXP bit marking)

Queue and drop accordingly
based on class (DSCP)

Simple but no guarantees
Proprietary and Confidential

www.juniper.net

Class of Service
MPLS Traffic Engineering
Subscribers
Subscribers

PolicingPolicing
Shaping
Shaping

InsufficientInsufficientunreserved
unreserved TETE
bandwidth
bandwidth

1000s
1000s ofof Flows
FlowsO
1000s
1000s ofof Flows
Flows

ContentContent

100s
100s ofof Flows
Flows

BRAS
BRAS
1000s
1000s ofof Flows
Flows

MPLS
MPLS TETE -- RSVP
RSVP

No interaction with Diff-Serv –all traffic is treated the sameuu Constraint-based
Constraint-based LSP
LSP setup
setup -- computes
computes aa path
path withwith
sufficientsufficient unreserved
unreserved TETE bandwidth
bandwidthuu Allows
Allows efficientefficient use
use ofof available
available bandwidth
bandwidthuu Eachlinkhasasingle
bandwidthpool
Each link has a single bandwidth pool available
available

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

The Next Step:
Diff-Serv Traffic Engineering
Policing
Policing
Subscribers
Subscribers
Shaping
Shaping plusplus
Admission
Admission Control
Controlperper ClassClass
1000s
1000s ofof Flows
Flows

InsufficientInsufficient unreserved
unreserved TETE
bandwidth
bandwidth forfor Premium
Premium
ClassClass Traffic
Traffic
ContentContentO

1000s
1000s ofof Flows
Flows

100s
100s ofof Flows
FlowsBRASBRAS1000s
1000s ofof Flows
Flows

Diff-Serv TE

Diff-Serv TE guarantees
bandwidth!

u Connects Diff-Serv classes with RSVP traffic engineering databaseu Admission Control per Class based bandwidth pools
u Sessions refused to maintain guaranteed bandwidth for traffic classu Constraint Based LSP setup - per Class with different bandwidth constraints
u LSP path setup now governed by unreserved bandwidth per class

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Scalability in the proper context:
RSVP as an example
Outside of TE context :••“supporting numerous
small reservations on ahigh-bandwidth link may
easily overtax the routers
and is inadvisable”(RFC2208)“the resource
requirements for running
RSVP on a routerincreases proportionallywith the number ofseparate sessions”(RFC2208)

Copyright © 2003 Juniper Networks, Inc.

In the context of TE:• We are not dealing with small
reservations, as one TE LSP
aggregates many micro flows• Total number of RSVP sessions(total number of TE LSPs) isbounded by the number of edge
routers of a service provider, andis independent of the total volumeof traffic carried by the service
providerProprietary and Confidential

www.juniper.net

More than one way to scale
Goal: limit the volume of information that asingle box must handle• By information aggregation/abstraction via
hierarchical routing– used for scaling the Internet service (CIDR)• By partitioning the problem into a set ofindependent sub-problems– used for scaling 2547 VPNs, BGP autodiscovery/signaling
for VPLS, and L2 VPNs services
Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Scalability: BGP• Excellent example of appropriate protocol re-use• Began life as an inter-domain routing protocol• Has assumed additional service control roles• 2547bis, VPLS, kompella-draft, etc.

• Leverages knowledge and infrastructure alreadypaid for by service providers.
• Keeps the toolkit manageable, without limiting
flexibility or performance.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Reliability: Toolkit High Availability• Goal: minimize service disruption due to faults• handle both control plane and data plane faults• Control Plane graceful restart and hitless
switchover for handling faults in the control plane• Requires the ability to preserve/maintain data
plane in the presence of control plane faults• MPLS Fast Reroute for handling faults in the dataplane• More than one kind of FRR, but that’s anotherstory...

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Graceful Restart
Separate controland data planesPP
PE 2

When router recovers,
neighbors sync up
without disturbingforwarding.

PE 1

If router’s control
plane fails, data
plane can keep
forwarding packets

PE 3

PNeighbors hidefailure from allothers routersin the network

Copyright © 2003 Juniper Networks, Inc.

P
Other routers
are never madeaware of failure

Proprietary and Confidential

www.juniper.net

Hitless RE Switchover•
Keep Alive •Routing Engines

Protects against Single Node
Hardware Failure
Primary (REP) and Secondary(RES) utilize keepalive process•••

REP and RES share:•Packet ForwardingEngines•Automatic failover to RES
Synchronized Configuration
Forwarding info + PFE config

REP failure does not reset PFE•••

No forwarding interruption
Only Management sessions lostAlarms, SNMP traps on failover

Not the Same As Protocol Stateful Mirroring

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

MPLS Fast Reroute
Single user commandat head end to enable FastReroute.
Detour

DetourPrimary

LSR1

Primary

LSR2

DetourPrimary

LSR3

Primary

LSR4

LSR5

• Fast reroute is signaled to each LSR in the path
• Each LSR computes and sets up a detour path thatavoids the next link and next LSR• Each LSR along the path uses the same route
constraints used by head-end LSR

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Toolkit Requirements - Details• Support for multi-AS/multi-provider operations• Multi-vendor interoperability based on openspecifications• Effective QoS capability• Internet-size Scalability• TDM-like Availability• Operational Economy• These are the “cost of entry” for nearly all
potential services.

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Economical Ops: Resource sharing•••

By competent system design
By competent software engineering
Examples:– ability to limit memory usage by individual 2547 VPN customers by
imposing an upper bound on the number of routes in a VRF– ability to limit CPU usage by individual 2547 VPN customers bylimiting the rate of control (routing) traffic on the CE-PE link
– ability to limit bandwidth usage by individual 2547 VPN customers– ability to use different routing protocol process for each service•

There is an existence proof (products on the market) that it ispossible to build multi-service platforms that provide
adequate resource sharing

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Economical Ops: Helping Staff• Introduce new operational practices carefully• Tools and software can ease new procedures• Train, train, train!
• Certification• All of these lead to better customer service

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

The Toolkit meets requirements• By taking advantage of the commonalities between
multiple services• By using tools that are general and easily extendable• By using tools that have appropriate scaling properties• By using tools that can operate across multiple
ASs/service providers• By using tools that are applicable to a wide variety ofmultiplexing and switching technologies (e.g, routers,
SONET/SDH Cross Connects, Optical Cross Connects,etc…)• By documenting specifications in Internet Drafts, and byusing multi-vendor interoperability testing events
Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Simple Versatility• Small set of tools, many possible uses:• IP/{G}MPLS data and forwarding plane• Diffserve QoS capability•••••

BGP with Multi-Protocol Extensions
OSPF/ISIS with TE extensions
RSVP-TELDP
Graceful restart for all protocols (BGP,OSPF/ISIS, RSVP-TE, LDP)

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Summary
IP/MPLS Toolkit:
Enabling providers to deliver
as many profitable services as possible
with as few tools as possible

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net

Thank you!
matt@juniper.net

Copyright © 2003 Juniper Networks, Inc.

Proprietary and Confidential

www.juniper.net