Advanced Platform Services on JGN2plus

DCN Deployment & NOC
Collaboration Tool
Jin Tanaka
KDDI/APAN-JP NOC/NICT
Network Engineering Workshop
APAN 28th Meeting
July 22 2009
Kuala Lumpur Malaysia

Outline
1. DCN Brief Review• DCN Multi-domain Control Plane• Global Dynamic Circuit• Example of DCN Utility• Benefits of Using DCN

2. Expansion of DCN connection to APAN region• How to Connect to DCN?
• Installation of Control Plane Software

3. Issues and Advices on Development of Data Plane• Efficient Construction of Layer2 Network• Taking Care of Network Performance• Challenges in Multi-domain DCN Operation

4. NOC Collaboration Tool for Multi/Inter-domain R&Enetwork• Information-sharing Tool Review• Next Steps by APAN Sydney Meeting 2

DCN
DCNBrief
BriefReview
Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration Tool

DCN Multi-domain Control Plane
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC

IDC

•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsr

Network3Network1User

End-to-End Ethernet VLAN Path

• Better Performance • Symmetric• Secure • Shorten Provisioning time
• Efficient use of Network Resource
End SystemEnd System

DCN
DCNBrief
BriefReview
Review

Expansion of DCN on APAN region

Advices on Development of data plane

Example of DCN Utility

NOC Collaboration Tool

NICT e-VLBI Correlation over DCN at SC08
Metsähovi, FI
Kashima, JPDCN

Dynamic Circuit NetworkAustin, USDCN
DCNBrief
BriefReview
Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration ToolDCN
DCNBrief
BriefReview
ReviewExpansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration Tool

Benefits of Using DCN
user to create on demand end-to-end L2 VLAN path
• EnableResearchers/campus
network operators can provision VLAN path through Web Interface bythemselves- Save significant network provisioning time
- Possible use for interactive real-time application, events over the network- Provides speedy implementation of research activities and experiments

the burden of multi-domain network coordination• Reduce
No need to coordinate the path among multiple networks in each use case- Facilitate the smooth implementation of international collaboration researchs

management of finite network resources• Schedule
IDC reservation scheduler prevents the network performance degradation by cross-trafficcollision- Especially effective in stationary and huge data transmissionof network performance in US section• Improvement
Internet2 DCN controls the Layer1 level and provides dedicate circuit secured the bandwidth- Avoids a negative impact of cross-traffic in the shared-network-specificand application can use “DCN web service API” to request circuits• UsersDCN
API can be used into your applications or scripts- DCN API enables users to develop the original client software appropriate for own application- Realization of integrated client software cooperate with DCN and application

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

How to connect to DCN?
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC

IDC

•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1User

DC/vlsr

DC/vlsrDC/vlsr

Network2

DC/vlsr

Network3

End-to-End Ethernet VLAN PathEnd SystemEnd System

DCN Brief ReviewExpansion
ExpansionofofDCN
DCNto
toAPAN
APAN region
regionAdvices on Development of data plane

NOC Collaboration ToolConnecting to DCN -1-

Physical Connection from National networks interconnect
Layer2 Ethernet
VLANs

Other NREN
DCN or Basic Layer2 VLAN
CampusJGN2plus
DCN
Other NRENin ASIA

JGN2plus

APAN-JP• NRENs within APAN region,- Physical Link to JGN2plus DCN POPs

• Tokyo/Otemachi, Kyusyu, Kashima, Tsukuba POPs
• Layer2 connection via APAN-JP- Link should be Layer2 ethernet supporting VLANs• Universities and campuses in each NRENs,- Physical Link to NRENs POPs
- Typical connection is Ethernet VLANs

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Connecting to DCN -2-Linking to DCN - Option1: Static VLAN Web Interface•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDC•XML Request

IDC•XML Topology Exchange•Path Creation•Path Creation

DC/vlsrYour Network

DC/vlsr

DC/vlsr

JGN2plus DCNDC/vlsr

DC/vlsrInternet2 or other
DCN

Static VLAN
User

• Create Static VLAN from NRENs to JGN2plus DCN- Static VLAN from user to edge of JGN2plus DCN POPs
- Dynamic VLAN from edge of JGN2plus DCN to remote

End System

DCN connected User
- Users request connection from JGN2plus IDC

End System

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Connecting to DCN -3-Linking to DCN - Option2: Own DCNWeb Interface•Authentication•Schedule Reservation
IDC•Request VLAN
•Request Bandwidth•XML Request

IDC•XML Topology Exchange•XML Topology Exchange•Path Creation

IDC•XML Request•Path Creation•Path Creation

DC/vlsr
DC/vlsr

DC/vlsr

DC/vlsr

JGN2plus DCNYour NetworkUser

DC/vlsr

DC/vlsrInternet2 or otherDCN• NREN within APAN region creates and installs
own DCN- Static connection from user to your local DCN- Local DCN has its own IDC- Users request circuit over DCN from your local IDC

End System

DC/vlsrto JGN2plus IDC

End System

DCN Brief ReviewExpansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Installation of Control Plane Software
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request BandwidthIDC•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsrInternet2 or otherDCN

Network3

End-to-End Ethernet VLAN PathUser

End System
End System

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Control Plane Software -1Domain Controller

DC(Domain Controller)• DRAGON(Dynamic Resource Allocation via Gmpls Optiacl Network)- Open source implementation of GMPLS maintained by MAX,
USC ISI EAST, and George Mason University(Virtual Label Switched Router)- VLSRZebra
PC based control plane software

VLSR•• Provides GMPLS protocol support for devices which do not support
GMPLS• OSPF-TE, RSVP-TE• Provision the Ethernet Switch and SONET/SDH Switch• Switch setting method: SNMP, CLI, TL1, other script• Provisioning request via CLI, XML

Requirements for Installation- SystemDRAGON System
JGN2plus and APAN-JP hardware
HP ProLiant DL360 G5Intel Xeon X5260 3.33GHz
DualCore
DDR2-667 2GB * 2
SAS146GB*2 (RAID1)
10/100/1000 base-T * 2-•- Linux BOX- RedHat Enterprise Base (Kernel version 2.4.2 or later)- Software Requirements• DRAGON Software package (VLSR, NARB, RCE, ASTB)• Dependence-package (SSH, GNU Compiles, Net-SNMP, libxml2,zlib-1.2.3)

12

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Installation of Control Plane Software
Web Interface

IDC•XML Request•Authentication•Schedule Reservation•Request VLAN
•Request Bandwidth

IDCIDC•XML Request•XML Topology Exchange•XML Topology Exchange•Path Creation•Path Creation•Path Creation
DC/vlsr

DC/vlsr

Network1

User

DC/vlsr

DC/vlsr

DC/vlsr

Network2

DC/vlsr

Network3

End-to-End Ethernet VLAN Path

End System
End System

DCN Brief Review

Expansion
Expansion ofof DCN
DCN toonAPAN
APANregion
regionAdvices on Development of data plane

NOC Collaboration Tool

Control Plane Software -2Inter-Domain Controller

IDC(Inter-domain Controller)•- OSCARSOpen source project maintained by Internet2 and ESnet
Accept circuit requests from users
Use IDC protocol which consist of web services as amessaging among Inter-domain
Web User Interface function for users
Book-ahead and manage the scheduling of circuits

Requirements for Installation- SystemOSCARS System•

JGN2plus and APAN-JP hardware
HP ProLiant DL360 G5Intel Xeon X5260 3.33GHz
DualCore
DDR2-667 2GB * 2
SAS146GB*2 (RAID1)
10/100/1000 base-T * 2-- Linux BOX- RedHat Enterprise (Kernel version 2.4.2 or later)- OSCARS Package Software- Third-Party Library and Package Requirements• OSCARS Package Software• MySQL5.0 / JDK5.0 / Tomcat 5.5 / Axis2 1.4.1/ Rampart 1.4.1/
Ant 1.7- SMTP(sendmail) for e-mail notification of circuit activity- NTP source14

DCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration Tool

Efficient Construction of Layer2 Networkor Coexistence with IP production Network• Hybrid
Only Japan, Korea, and Thailand are currently providing Layer2 backbone services in APAN
region- It it fact that most NREN networks are build up with Layer3 and transport research trafficwith IP packets
are barriers:- ThereA few carriers can provide wide-area ethernet service in Asia•• Difficulty in Layer2 network operation- In principle, DCN needs point-to-point Layer2 VLAN connection- MPLS/L2VPN function is a good solution to provide Layer2 VLAN connection on backbonerouter• Cost-effective:
There is no need to procure ethernet infrastructure besides current IP infrastructure- If your router support MPLS, you don’t need purchase ethernet backbone switchside:• Operation
Solid performance in APAN-JP/TransPAC2/CERNET/UniNet, etc.

- Since router has lager interface queue buffer than switch, MPLS brings the expected performance- Multi vender Interoperability problem remains an issue
Ethernet

MPLS Routeror L3 Switch

SONET/SDH 622M

MPLS Routeror L3 Switch
Ethernet

PacketMPLS over POSPacket

DCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration Tool

Take care of Network Performanceof Data Plane Network• Performance
Even though we use DCN technology, it is difficult to create bandwidth guaranteed E2E circuits onAPAN region- Since DCN is partially shared network, it should be considered long-distance Layer-2 shared network- To my knowledge, it is very difficult to provide better performance over long-distance Layer-2
network rather than Layer-3 networkof Layer-2 switch- Capability
Buffer size of interface queue•• Wire rate
• VLAN-based QoS- Though DCN system specified the bandwidth exceeding manually-set XML topology cannot be

reserved by user basically, it is expected to design data plane network without such bottleneck as“Saturation Point (a)” and “Bandwidth Conversion Point”• n x 1G 1G X
• 10G 1G X GigE

SONET OC12(622M)

Xa) Saturation Point
1GDCN1G1Gb)Bandwidth Conversion PointDCN

10G1G1G16

DCN Brief ReviewExpansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration Tool

Challenges in Multi-domain DCN Operationof Trouble-shooting of Multi-domain DCN• -Difficulty
NOC for DCN has not been deployed even in US and Euro, Global Dynamic Circuit is-operated based on DCN control plane engineers and software developers community
In this situation, the DCN end-to-end path creation between JP-US becomes failure• No problem on JGN2plus control plane and data plane• Contact the US DCN developer and then the problem was finally resolved• There are absolutely no operational information for DCN like usual Internet2 IP network• Pause to realize importance of “operational information”
Multi-domain operation for multi-domain DCN Circuit• - Network monitoring tools and systems for multi-domain environment is indispensable to-DCN operation
DCN path is created over existing multi-admin/multi-domain infrastructures, so we would
need more various operational information beside current operation data

• Operational status of underlaying data plane network --Circuit, equipment over end-to-end/multi-domain network• DCN-specific event status on IDC and DC software• Usage / Number of DCN circuits• User accounts for authentication- Solutions:
1. Development of perfSONAR-driven operation tool for multi-domain DCN
2. Improvement of DCN control plane softwares
3. Need more stronger collaboration among R&E NOCsDCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data planeNOC
NOCCollaboration
CollaborationToolToolNOC Collaboration Tool
Proposal of Integrated “Web based tool” that can share thosekind of information among NOCs
Requirements•-Essential
Link to NOC own web page of each asian REN/NRENs- NOC information for understanding the network situation and research projects- Collaborative research project lists/Detailed application lists- Calendering of experiment and event schedule- Documentation management- Authentication for NOC and Network adminbased Software•-Wiki
APAN-JP NOC Proposed “Atlassian Confluence”(Enterprise wiki/Blog/CMS) at last APAN-meeting
Plug-in system makes it easy to customize and extend Confluence to suit our NOC needs
It is used in Internet2, KEK(Japan), Accenture, Sun mirosystems, U.C. Berkeley•-Management
Public web NOC and Network admin with Authentication- APAN-JP NOC install tool and compile content responsibly

DCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC
NOCCollaboration
CollaborationTool
Tool

Information-sharing Tool
- Blueprint NOC

NRENResearch
DOCs
Project List

NOC

Ongoing
Event

Operational/
Technical InfoCalenderREN

Account

Chat/
Message
Board

NOC A

SearchExternal
CalenderRSS Feed
E-mailNOC DNet
Admins A

NOC C

DCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data planeNOC
NOCCollaboration
CollaborationTool
Tool

Information for NOC
Research Project
Iperf server
NOC contact point Experiment Schedule
Routing Information

Traceroute

Network Topology

IP addresses
Network Resource

BGP neighbor

VLAN ID

Bandwidth
Documentation

Equipment
MTU size

Measurement
Data

Network Usage

Trouble-Ticket
Researchers

DCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC
NOCCollaboration
CollaborationTool
Tool

Next Steps by APAN Sydney
1. As a first step, APAN-JP NOC collects each NOC’s contact point- Your 24 x 7 NOC mailing list & phone number- Photos of operators- Your NOC manager’s cell phone number- Your organization or NOC operation web page

2. Create an account and initial page of all APAN NOCs on AtlassianConfluence- Post your portrait picture :-)- For creating your NOC’s own personal space- For learning features of wiki basics

3. Discuss on Discussion threads of Atlassian Confluence- Call for cooperation from NOCs
- Brainstorming about what information do we need for further collaboration- Assemble necessary information

4. Design and create a framework of collaboration tool

- Consolidate information and enhancing the content- Start the trial operationDCN Brief Review

Expansion of DCN on APAN region

Advices on Development of data plane

NOC Collaboration ToolConclusions• DCN overview- Creates on demand end-to-end L2 VLAN path between devices connected to DCN- Reduce the burden of multi-domain network coordination and provides speedy
implementation of research activities and experiment- leads to the promotion of the international collaboration on R&E networks• Expansion of DCN on APAN region- NRENs needs physical data connection, and control plane access to DCN- MAY have control plane connection- NRENs can create local DCN using available open source control plane software• Issues and advices for data plane deployment- MPLS/L2VPN is a good solution to make layer2 connection over APAN network- Care about capability of Layer-2 switch• Collaborated network operation- Multi-domain operation will be more important in DCN operation- Operation technology approach: Utilization and deployment of perfSONAR- NOC collaboration approach: Wiki based Information Sharing Tool

Reference URL
DCN Software Suite• https://wiki.internet2.edu/confluence/display/DCNSS/HomeOSCARS• http://www.es.net/oscars/• OSCARS Web Services Specification

If you want to see actual DCN• path creation, please contact me.
•I will show DCN demonstration!
https://wiki.internet2.edu/confluence/display/CPD/OSCARS+Web+Service+Definition• DRAGON

http://dragon.maxgigapop.net/twiki/bin/view/DRAGON/WebHome

DCN Working Group Wiki

https://spaces.internet2.edu/display/DCN/Home

Test IDC Guide

https://wiki.internet2.edu/confluence/display/DCNSS/Internet2%27s+Test+IDC• perfSONAR

http://www.perfsonar.net/• SC08

http://sc08.supercomputing.org/

Confluence• Atlassian
http://www.atlassian.com/software/confluence/