Advanced Platform Services on JGN2plus

Dynamic Circuit Network(DCN)
Deployment
Jin Tanaka
KDDI/NICTKOREN Workshop
Nov. 6th 20091

Outline
1. DCN Brief Review• DCN Multi-domain Control Plane• Global Dynamic Circuit• Example of DCN Utility• Deployment Status in Japan• Benefits of Using DCN

2. Expansion of DCN connection to APAN region• How to Connect to DCN? (Data Plane)• Installation of Control Plane Software (Control Plane)

3. Issues and Advices on Development of Data Plane• Efficient Construction of Layer2 Network• Taking Care of Network Performance• Challenges in Multi-domain DCN Operation

4. Introduction of NICT SC09 DCN demo

• Fenius Interface• Space Weather Forecast over dynamic provisioned networks
21.DCN Brief Review

3

DCN Multi-domain Control Plane
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC

IDC

•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsr

Network3

Network1

User

End-to-End Ethernet VLAN Path

• Better Performance • Symmetric• Secure • Shorten Provisioning time
• Efficient use of Network Resource

End SystemEnd System4

Example of DCN Utility

NICT e-VLBI Correlation over DCN at SC08
Metsähovi, FIKashima, JPDCN

Dynamic Circuit NetworkAustin, US
5

NICT SC08 Network Topology

MPLS/L2VPN6

internet27

Deployment Status in Japanof DCN systems in JGN2plus and APAN-JP• Installation
Linux based IDC and DCs- DCN Software Suite version 0.4- Establish control plane(dcn.jgn2plus.jp) and data plane on a nationwide- Connection test on multiple VLSRs in JGN2plus intra-domain- Connection test over inter-domain between JGN2plus and APAN-JP- Preparation works for the beginning of JGN2plus DCN pilot service

IDCDCFukuoka

Tsukuba
Kashima

Otemachi8

Control/Data Planes logical diagram9Benefits of Using DCN
user to create on demand end-to-end L2 VLAN path
• EnableResearchers/campus
network operators can provision VLAN path through Web Interface bythemselves- Save significant network provisioning time
- Possible use for interactive real-time application, events over the network- Provides speedy implementation of research activities and experiments

the burden of multi-domain network coordination• Reduce
No need to coordinate the path among multiple networks in each use case- Facilitate the smooth implementation of international collaboration researchs

management of finite network resources• Schedule
IDC reservation scheduler prevents the network performance degradation by cross-trafficcollision- Especially effective in stationary and huge data transmissionof network performance in US section• Improvement
Internet2 DCN controls the Layer1 level and provides dedicate circuit secured the bandwidth- Avoids a negative impact of cross-traffic in the shared-network-specificand application can use “DCN web service API” to request circuits• UsersDCN
API can be used into your applications or scripts- DCN API enables users to develop the original client software appropriate for own application- Realization of integrated client software cooperate with DCN and application
10

2.Expansion of DCN
connection to Asian region

11Expansion into to Asian Region
Internet2 ION
KOREN
CERNETCSTNET
ThaiREN

IDC
DC

SingAREN

DCN connections to NRENs within APAN
• Expand
Having a Layer2 Ethernet connection with JGN2plus/APAN-JP-

ThaiREN(Thailand), ASGC/ASCC(Taiwan),• SingAREN(Singapore),
KOREN(KOREA), CERNET/CSTNET(China)• Explore appropriate applications for global DCN connectionsin Asian region

12How to connect to DCN?
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC

IDC

•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1

User

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsr

Network3

End-to-End Ethernet VLAN PathEnd SystemEnd System
13

Connecting to DCN -1Physical Connection from National networks interconnect
Layer2 EthernetVLANs

Other NREN
DCN or Basic Layer2 VLAN
CampusJGN2plus
DCN
Other NRENin ASIA

JGN2plus

APAN-JP• NRENs within APAN region,- Physical Link to JGN2plus DCN POPs

• Tokyo/Otemachi, Kyusyu, Kashima, Tsukuba POPs
• Layer2 connection via APAN-JP- Link should be Layer2 ethernet supporting VLANs• Universities and campuses in each NRENs,- Physical Link to NRENs POPs
- Typical connection is Ethernet VLANs
14

Connecting to DCN -2Linking to DCN - Option1: Static VLAN Web Interface•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC•XML Request

IDC•XML Topology Exchange•Path Creation•Path Creation

DC/vlsrYour Network

DC/vlsr

DC/vlsr

JGN2plus DCN

DC/vlsr

DC/vlsrInternet2 or other
DCN

Static VLAN
User

• Create Static VLAN from NRENs to JGN2plus DCN- Static VLAN from user to edge of JGN2plus DCN POPs
- Dynamic VLAN from edge of JGN2plus DCN to remote

End System

DCN connected User
- Users request connection from JGN2plus IDC

End System15

Connecting to DCN -3Linking to DCN - Option2: Own DCNWeb Interface•Authentication•Schedule Reservation
IDC•Request VLAN
•Request Bandwidth•XML Request

IDC•XML Topology Exchange•XML Topology Exchange•Path Creation

IDC•XML Request•Path Creation•Path Creation

DC/vlsr
DC/vlsr

DC/vlsr

DC/vlsr

JGN2plus DCNYour NetworkUser

DC/vlsr

DC/vlsrInternet2 or otherDCN• NREN within Asian region creates and installs
own DCN- Static connection from user to your local DCN- Local DCN has its own IDC- Users request circuit over DCN from your local IDC

End System

DC/vlsrto JGN2plus IDC

End System16

Installation of Control Plane Software
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsrInternet2 or other
DCN

Network3

End-to-End Ethernet VLAN Path

User

End System
End System
17

Control Plane Software -1Domain Controller
DC(Domain Controller)• DRAGON(Dynamic Resource Allocation via Gmpls Optiacl Network)- Open source implementation of GMPLS maintained by MAX,
USC ISI EAST, and George Mason University(Virtual Label Switched Router)- VLSRZebra
PC based control plane software

VLSR•• Provides GMPLS protocol support for devices which do not support
GMPLS• OSPF-TE, RSVP-TE• Provision the Ethernet Switch and SONET/SDH Switch• Switch setting method: SNMP, CLI, TL1, other script• Provisioning request via CLI, XML

Requirements for Installation- System
DRAGON System
JGN2plus and APAN-JP hardware
HP ProLiant DL360 G5Intel Xeon X5260 3.33GHz
DualCore
DDR2-667 2GB * 2
SAS146GB*2 (RAID1)
10/100/1000 base-T * 2-•- Linux BOX- RedHat Enterprise Base (Kernel version 2.4.2 or later)- Software Requirements• DRAGON Software package (VLSR, NARB, RCE, ASTB)• Dependence-package (SSH, GNU Compiles, Net-SNMP, libxml2,zlib-1.2.3)

1818

Installation of Control Plane Software
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC

IDC

•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1

User

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsr

Network3

End-to-End Ethernet VLAN Path

End SystemEnd System19

Control Plane Software -2Inter-Domain Controller
IDC(Inter-domain Controller)•- OSCARSOpen source project maintained by Internet2 and ESnet
Accept circuit requests from users
Use IDC protocol which consist of web services as amessaging among Inter-domain
Web User Interface function for users
Book-ahead and manage the scheduling of circuits

Requirements for Installation- System
OSCARS System•

JGN2plus and APAN-JP hardware
HP ProLiant DL360 G5Intel Xeon X5260 3.33GHz
DualCore
DDR2-667 2GB * 2
SAS146GB*2 (RAID1)
10/100/1000 base-T * 2-- Linux BOX- RedHat Enterprise (Kernel version 2.4.2 or later)- OSCARS Package Software- Third-Party Library and Package Requirements• OSCARS Package Software• MySQL5.0 / JDK5.0 / Tomcat 5.5 / Axis2 1.4.1/ Rampart 1.4.1/
Ant 1.7- SMTP(sendmail) for e-mail notification of circuit activity- NTP source2020

3.Issues and Advices onDevelopment of Data Plane

21

Efficient Construction of Layer2 Network• Hybrid or Coexistence with IP production Network- Only Japan, Korea, and Thailand are currently providing Layer2 backbone services in Asian region- It it fact that most NREN networks are build up with Layer3 and transport research traffic withIP packets
are barriers:- ThereA few carriers can provide wide-area ethernet service in Asia•• Difficulty in Layer2 network operation- In principle, DCN needs point-to-point Layer2 VLAN connection
function is a good solution to provide Layer2 VLAN connection on backbone- MPLS/L2VPNrouter• Cost-effective:- There is no need to procure ethernet infrastructure besides current IP infrastructure- If your router support MPLS, you don’t need purchase ethernet backbone switch• Operation side:- Solid performance in APAN-JP/TransPAC2/CERNET/UniNet, etc.
- Since router has lager interface queue buffer than switch, MPLS brings the expected performance- Multi vender Interoperability problem remains an issue
Ethernet

MPLS Routeror L3 Switch

SONET/SDH 622M

MPLS Routeror L3 Switch
Ethernet

PacketMPLS over POS

Packet22

Take care of Network Performance• Performance of Data Plane Network- Even though we use DCN technology, it is difficult to create bandwidth guaranteed E2E
circuits on Asian region- Since DCN is partially shared network, it should be considered long-distance Layer-2
shared network- To my knowledge, it is very difficult to provide better performance over long-distance
Layer-2 network rather than Layer-3 networkof Layer-2 switch- Capability
Buffer size of interface queue•• Wire rate
• VLAN-based QoS- Though DCN system specified the bandwidth exceeding manually-set XML topology

cannot be reserved by user basically, it is expected to design data plane network without
such bottleneck as “Saturation Point (a)” and “Bandwidth Conversion Point”• n x 1G 1G X
• 10G 1G X GigE

SONET OC12(622M)Xa) Saturation Point
1GDCN

1G1Gb)Bandwidth Conversion Point

DCN

10G1G1G2323

Challenges in Multi-domain DCN Operationof Trouble-shooting of Multi-domain DCN• -Difficulty
NOC for DCN has not been deployed even in US and Euro, Global Dynamic Circuit is-operated based on DCN control plane engineers and software developers community
In this situation, the DCN end-to-end path creation between JP-US becomes failure• No problem on JGN2plus control plane and data plane• Contact the US DCN developer and then the problem was finally resolved• There are absolutely no operational information for DCN like usual Internet2 IP network• Pause to realize importance of “operational information”
Multi-domain operation for multi-domain DCN Circuit• - Network monitoring tools and systems for multi-domain environment is indispensable to-DCN operation
DCN path is created over existing multi-admin/multi-domain infrastructures, so we would
need more various operational information beside current operation data

• Operational status of underlaying data plane network --Circuit, equipment over end-to-end/multi-domain
network• DCN-specific event status on IDC and DC software• Usage / Number of DCN circuits• User accounts for authentication- Solutions:

1. Development of perfSONAR-driven operation tool for multi-domain DCN
2. Improvement of DCN control plane softwares
3. Need more stronger collaboration among R&E NOCs
24

4.Introduction of NICT SC09
DCN demo25

Fenius Interface• Interoperable Issue on multi-domain dynamic provisioning- Each country develop control plane system with an original interface
Internet2 ：IDC（Internet2、ESnet）
 Europe：Harmony（HPDMnet）/ AutoBAHN (GEANT) Korea：Dynamic KL（KISTI）
 Japan：G-lambda（NICT、AIST、KDDI、NTT）
- Interface Standardization is progressing by OGF NSI-WG- The transfer proxy for each interface is developing by GLIF community before
standardization => Fenius
Common API

Jetty
G-lambda
Translator
Common APIWeb UI
ClientCommon API
Monitoring
ClientG-lambda

Common API
Web UI
ClientCommon web
service API

Common API
IDC
Translator

IDC

Commoninternal API

Common API
Monitoring
ClientHarmony
Common API
Harmony
Translator

Harmony
Translator
Harmony

Network
Provisioning
Service26

27

Conclusions• DCN overview- Creates on demand end-to-end L2 VLAN path between devices connected to DCN- Reduce the burden of multi-domain network coordination and provides speedy
implementation of research activities and experiment- leads to the promotion of the international collaboration on R&E networks• Expansion of DCN on APAN region- NRENs needs physical data connection, and control plane access to DCN- MAY have control plane connection- NRENs can create local DCN using available open source control plane software• Issues and advices for data plane deployment- MPLS/L2VPN is a good solution to make layer2 connection over APAN network- Care about capability of Layer-2 switch• Improve interoperability of Multi-domain/Multi-Interface- Interface Standardization is progressing by OGF NSI-WG- Fenius is developed as an interim solution that translates different type of interface
to original interface of each country28

Reference URLs
DCN Software Suite• https://wiki.internet2.edu/confluence/display/DCNSS/HomeOSCARS• http://www.es.net/oscars/• OSCARS Web Services Specification

https://wiki.internet2.edu/confluence/display/CPD/OSCARS+Web+Service+Definition• DRAGON

http://dragon.maxgigapop.net/twiki/bin/view/DRAGON/WebHome

DCN Working Group Wiki
• https://spaces.internet2.edu/display/DCN/Home• Test IDC Guide

https://wiki.internet2.edu/confluence/display/DCNSS/Internet2%27s+Test+IDC• perfSONAR
http://www.perfsonar.net/• SC08
http://sc08.supercomputing.org/• SC09http://sc09.supercomputing.org/
GNI task force• GLIF
http://www.glif.is/meetings/2009/tech/chaniotakis-gni.pdf
29