Scalability Analysis of L2 and L3 VPN Technologies
Service Provider Perspective

Alex ZININCTO IPD APAC, Alcatel-Lucent

All Rights Reserved © Alcatel-Lucent 2006, #####

What is Scalability Analysis?
Different approaches have different complexity (overhead) Computational (CPU utilization)
 Data (amount of state) Signaling (required BW)
 Management & provisioning (OPEX)
Goal of scalability analysis:
 Understand growth of overhead as complexity input parameters increase
Method--asymptotic complexity analysis
 Asymptotic behavior of f(n) == growth of f(n) as n gets larger Typically ignore small values of n
 Slower asymptotic growth – better O-notation for “order” of growth, e.g.:f(n) = n2 + 25*n - 10 = O(n2)f(n) = 100*n2

2 | MPLS Japan 2007| Oct 10, 2007

= O(n2)All Rights Reserved © Alcatel-Lucent 2007

f1()

Example: CPU overhead functions

f2()

f1()

````)
f2(

Compare overhead functions
for two algorithms f1() and f2()

3 | MPLS Japan 2007| Oct 10, 2007

Both appear linear
f2() better than f1()

All Rights Reserved © Alcatel-Lucent 2007

Example: CPU overhead functions

O (Lf1() =og x))f2((x=O)

On a different scale
f1() better than f2()

4 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Example: CPU overhead functions

OSPF

RIP

Number of links

Choice of better solution
depends on problem scope

5 | MPLS Japan 2007| Oct 10, 2007

BUT

Generally inferior solution
can be better for small problems

All Rights Reserved © Alcatel-Lucent 2007

Network Protocol Scaling Analysis
Control Plane Number of protocol sessions (adjacencies, TCP connections, etc.)
 Amount of state (LSAs, routes, etc.)
 Computational overhead (CPU load) Signaling overhead (required BW)
Data Plane Amount of state (routes, MAC addresses, etc.)
 Computational overhead (forwarding, flooding, replication, etc.)
Management Plane Provisioning overhead Troubleshooting complexity

6 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Reason for Scaling Analysis
Understand technology aspects
 Maximum number of nodes, users, services Potential bottlenecks
 Sensitive implementation areas
Understand cost functions
 Cost of initial deployment
 Cost growth curve as function of number of subs/svcs
MemMem)O(x)gxLoog x)((LOO# subs#(exp)subs
7 | MPLS Japan 2007| Oct 10, 2007

# subs#(exp)subs
All Rights Reserved © Alcatel-Lucent 2007

VPN Scaling Analysis—General Aspects
What is the scaling factor?
 Number of CPE devices?
 Number of end users?

CPECPE

CPE

SP
NetworkCPE

L2VPNs: sites + MAC addressesCPE

L3VPNs: sites + IP routes

8 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

VPN Scaling Parameters
L2 VPNs: Control PlaneL3 VPNs: Control Plane Sessions: T-LDP

 Sessions: iBGP State: PWs, MAC addresses

 State: VPN Routes

 CPU: LDP, MAC table CPU: BGP, PE-CE

 Signaling: PW setup, status,
MAC withdraw

 Signaling: route updates,withdraw

L2 VPNs: Data Plane

L3VPNs: Data Place State: MAC table State: VPN Routes

 Computation: MAC lookup,flooding, mcast replication

 CPU: FIB lookup, mcast
replication

9 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Basic Configuration
L3 VPNs: basic MPLS/BGP

L2 VPNs: flat VPLS Full mesh T-LDP Full mesh iBGP

 No multicast optimizations

 No multicast optimizations

PEPEPEPE

PLDTPEPEPE

S: number of sites
N: number of PE nodes ~O(Log S)
M: number of MAC addresses
V: number of VPN instances

10 | MPLS Japan 2007| Oct 10, 2007

GPiB

PE

PE

PE

S: number of sites
N: number of PE nodes ~O(Log S)
R: number of VPN Routes
V: number of VPN instances

All Rights Reserved © Alcatel-Lucent 2007

VPN Scaling: Basic config
L2VPN

L3VPNPEPEPE

PLDT-PE

CP: sessionsiBPEPEPEPEPEGPPE

O(N) per PE

O(N) per PE

O(N^2) totalO(N^2) total

CP: state

O(M*V)

O(R*V)

CP: CPULDP complexity

BGP complexity

CP: Signaling

O(N) PW signaling

O(R*V) route propagation

DP: state

O(M*V)

O(R*V)

DP: computationalO(Log M) lookup
O(N) per replicated packet

O(Log R) lookup

11 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Basic setup: scaling aspects

Number of PEs and protocols sessions toconfigure and bring up

Number of PW/Route labels to signalReplication overheadA

MAC address table/RIB sizeJ

MAC: A
MAC: B
MAC: C
MAC: D
MAC: E
MAC: F
MAC: G
MAC: H
MAC: I
MAC: JIAll Rights Reserved © Alcatel-Lucent 2007

CD
MAC: A
MAC: B
MAC: C
MAC: D
MAC: E
MAC: F
MAC: G
MAC: H
MAC: I
MAC: J

MAC: A
MAC: B
MAC: C
MAC: D
MAC: E
MAC: F
MAC: G
MAC: H
MAC: I
MAC: J

MAC: A
MAC: B
MAC: C
MAC: D
MAC: E
MAC: F
MAC: G
MAC: H
MAC: I
MAC: J

H

12 | MPLS Japan 2007| Oct 10, 2007

B

MAC: A
MAC: B
MAC: C
MAC: D
MAC: E
MAC: F
MAC: G
MAC: H
MAC: I
MAC: J

GF

E

Hierarchical Configuration
L3 VPNs: Hierarchical MPLS/BGP

L2 VPNs: H-VPLS
 Hub-n-spoke T-LDP RRs for iBGP

 No multicast optimizations

 No multicast optimizations
PE

PE
PE

PE

PE

PE
PE

PE
CN

PE

CN

CN

PE

CN

PE

PE
T

CN

DP-LiB

PE

PE
PE

PE

PE

CN

PE

PE
PE

PE

S: number of sites
N: number of PE nodes ~O(Log S)
C: number of H-VPLS core nodes
M: number of MAC addresses
V: total number of VPN instances
v: VPN instances at PE
13 | MPLS Japan 2007| Oct 10, 2007

CN

PE

CN

GP

S: number of sites
N: number of PE nodes ~O(Log S)
C: number of RRs
R: number of VPN Routes
V: number of VPN instances
v: VPN instances at PE
All Rights Reserved © Alcatel-Lucent 2007

VPN Scaling: hierarchy
L2VPN

L3VPN

PE
PE

PE

PE
PE

PE

CN

PE

CN

PE
PE

CN

CN
PE

PE

CN

P
LD
TCN

PE

PE

GP
iB

CN
PE

PE
PE

CP: sessions

PE

PE

CN
PE

PE

PE

PE

PE: O(1)

PE: O(1)

CN: O(C)

CN: O(C)

CP: state

PE: O(M*v)
CN: O(M*V)

PE: O(R*v)
CN: O(R*V)

CP: CPU

PE: O(LDP)
CN: O(LDP * C)

PE: O(BGP)
CN: O(BGP * C)

CP: Signaling

PE: O(1 PW)
CN: O(N PW)

PE: O(R*v) route propagation
CN: O(R*V)

DP: state

PE: O(M*v)
CN: O(M*V)

PE: O(R*v)
CN: O(R*V)

DP: computationalO(Log M) lookup
O(Log R) lookup
PE: O(1) per replicated packet
CN: O(N) per replicated packet

14 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Scaling Multicast: Default Operation
Rcv’erMcast
Mcast pktpkt
Mcast
Mcast pkt
pkt

Rcv’erMcast
Mcast pkt
pkt

GPiBPLDT-Mcast
Mcast pktpkt
Mcast
Mcast pkt
pkt

Mcast
Mcast pkt
pkt

SourceDefault multicast replication in VPLS isflooding:No multicast in MPLS/BGP VPN basicoperation

 O(N) per replicated packet
 BW inefficiency
15 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Efficient Multicast Operation
Rcv’er

Rcv’er
Rcv’er

Rcv’erMcast
Mcast pkt
pkt

Mcast
Mcast pkt
pkt

Mcast
Mcast pkt
pkt

Mcast
Mcast pkt
pkt

Rcv’erRcv’er
P
LDT-

Mcast
Mcast pkt
pkt

Mcast
Mcast pkt
pkt

Mcast
Mcast pktpkt
Mcast
Mcast pkt
pkt

VPLS multicast optimizations

MPLS/BGP multicast support:

 IGMP snooping Pre-built mcast tree (PIM-SSM)

 H-VPLS replication

 Mcast packets encapsulated in GRE

 BW suboptimalities still possible
16 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Scaling VPN services: controlling amount of network state
PEPE

PE

PE
CNPE

CNPE
P
LDT-CNiBGP

PE

CNPE

PEPEPEMAC
MAC learninglearning
controlcontrol.1
1010.1.1.0/24
10.1.1.0/24
OSPF/BGP
10.1.2.0/24
10.1.2.0/24
10.1.3.0/24
10.1.3.0/24

40/2.1.

Route
Routecontrolcontrol

10.1.2.0/24

PE

17 | MPLS Japan 2007| Oct 10, 2007

CECE
10.1.3.0/24

All Rights Reserved © Alcatel-Lucent 2007

PE

Scaling VPLS services further: MAC address challenge
PE

PE

PE
PE

PE

PE
PE
PE

PE

PE

PE
CN

PE

PE

CN

PE
PE

CN

PE

CN

P
LD
T-PE
CN

PE
CN

CN

CN

PE

PE

PE
PE

PE

PE

PE

PE

PE
PE
PE
PE
PE

PE

CN MAC Table scaling: O(M*V)
18 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Scaling VPLS services further: MAC-in-MAC for VPLS
PEPE

 PEs aggregate traffic to B-PEsPE Possible PEto use Q-in-Q

PEPE

MAC-in-MAC
MAC-in-MAC
encapsulation
encapsulation

B-PE
B-PE

PE

B-PE
PE
 B-PEs perform
MAC-in-MAC
encapsulation before H-VPLS

B-PE

S
PLV
CN
H-PE

CN

PE

B-PE

PE

B-PE

 CNs only see B-PE’s MAC addresses

CNPE

CN

PE

B-PE

B-PE
PE

PE

CN scaling substantially improved:PE

 State: O(B) MAC addresses
PEPE

B is number of B-PEs ~Log NPE

19 | MPLS Japan 2007| Oct 10, 2007

PEB-PEPE

All Rights Reserved © Alcatel-Lucent 2007

Management & Operations: scaling aspects
L2 VPNL3 VPN T-LDP: needs provisioning MP-BGP: needs provisioning MACs: need to be controlled

 VPN Routes: need to be controlled

 OAM: VPN-aware tools needed OAM: VPN-aware tools needed

Scalable provisioning platform is a must
Implementations need fine control
over MAC table population

Implementations need control overlearned and propagated VPN routes

OAM tools to look for:OAM tools to look for:

 VPLS MAC ping

 VRF ping

 VPLS MAC traceroute VRF traceroute VPLS MAC populate/purge

20 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

Summary

VPLS and MPLS/BGP VPN technologies:
 Different scaling aspects (MAC addresses vs IP routes)
 Different scaling improvement methods
 Yet similar operational impact and approach

21 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007

www.alcatel-lucent.com

22 | MPLS Japan 2007| Oct 10, 2007

All Rights Reserved © Alcatel-Lucent 2007