# Presentation: 854007

## DOE UltraScience Net

**NASA ONT Workshop**
**Aug 9, 2004**
**W. R. Wing and N. Rao**

## Talk Outline

**Context - both technical and DOE?**
**What are we trying to do (and why)?**
**How are we putting it together?**
**Status**

## The Well Known Problem

**DOE needs 160 - 200 Gbs by 2007**
**Typical multi-stream throughput limited to 5-6Gbs, and programming multi-steam is hard - hero efforts yield 9-10Gbs**
**Typical single-steam throughput to a single application ~1Gbs, even on a tuned network - hero effort yields 5-6+ Gbs**
**Networks will soon offer 40 Gbs channels**
**Firewalls throw it away anyway**

## Science Applications- Bit Rate Budget

- End-to-end

**Optical Backbone **

**Host **

- SONET
- (One Channel)

- Site
- Security
- Firewall

- TCP/IP
- (One Flow)

- DWDM
- (One Lambda)

**IP Network Services **

- OC-768

- Research Opportunities

- Bit-Rate

- OC-48

- OC-192

**GigE**

**Apps**

**ESnet**

## Even Without Firewalls -We Have a Technology Problem

**•    ****DOE needs 160-200 Gbs in 2007**
**•    ****This is on the “blue line”**
**•    ****This isn’t an opto-electronics**
**      ****problem - the serializers exist**
**•    ****This isn’t a TeleCo problem -**
**      ****they’d sell it if there were a market**
**•    ****It isn’t a problem routers are going**
**      ****to fix - bonded IP channels, are**
**      ****a problematic solution at best**
**•    ****Bonding at layer-2 (Ethernet)**
**      ****is also problematic**
**•    ****Solution is L1L (Layer-1 Light)**

## Given this Context: DOE’s Strategy from the Roadmap Workshop*

**Three-tiered strategy**
**General purpose production network (ESnet today)**
**High impact network (serve fewer sites with more bandwidth)**
**Research network to develop the next generation**
**Explicitly migrate research to production**
**Research Network should provide**
**Maximum flexibility**
**Minimum investment**

- ***http://www.es.net/hypertext/welcome/pr/Roadmap/index.html**

## Challenge -

**Lit-lambda Quotes from Qwest/Level(3)**
**$600k/lambda/yr ORNL-Chicago**
**$1M/lambda/yr Chicago-Sunnyvale**
**Lit-lambdas constrain our research options**
**Ideal research network should be dark fiber**
**Even DOE can’t afford coast to coast fiber**

## Solution -

**Leverage DOE, NSF, TVA, and state resources**
**ORNL buying dark fiber Atlanta-Chicago**
**Qwest: Atlanta, ORNL, Chicago**
**TVA: Knoxville, ORNL, Nashville, Memphis **
**Put up (and trade) lambdas to NLR**
**For charter-member pricing to Sunnyvale**
**For fall-back service between Atlanta and Chicago**
**For future access to JLAB and BNL**
**Leverage the way DWDM scales**
**Lit-lambda service Chicago-Sunnyvale**

## What will this let us do?

**Bypass limitations of conventional networks**
**TCP/IP**
**Firewalls**
**Small-frame technologies**
**Take advantage of the real throughput available in optical networks today**
**Develop the techniques needed within 2-3 years for DOE Peta-scale science**
**Use switched-circuits and Layer-1 Light (L1L)**

## So, What is Layer-1 Light?

**A Circuit with the Following Characteristics:**
**Zero packet re-ordering**
**Zero jitter**
**Zero drops due to Congestion**
**Known (by definition) paths and end points**
**SONET (or below) can do this - Ethernet can’t**
**Layer-1 Light can therefore:**
**Bypass firewalls**
**Carry non-IP frames (e.g., Fiber-Channel over SONET)**
**Easily (transparently) support parallel, bonded circuits**

## Why Research Layer-1 Light ?

**L1L can (by definition) be parallel**
**L1L isn’t constrained by traditional framing, routing, and transport**
**Useless for most commercial networks**
**Doesn’t aggregate for economy of scale**
**REALLY hard to scale deployment**
**But a good match for DOE high-performance science applications**

## So - Ultra Science NetResearch Testbed

**Build a sparse lambda-switching testbed**
**Connect hubs close to DOE’s largest Science users (but let the user labs pay last-mile costs)**
**Provide an evolving matrix of switching capabilities**
**Separately fund research projects (e.g., high-performance protocols, control, visualization) that will exercise the network and directly support applications at the host institutions**

## The Resources:

**Off hours capacity from Esnet**
**Expect 2 x OC48 between Sunnyvale and Chicago in the northern route (MPLS)**
**Dedicated Lambdas on NLR**
**Two 10G (OC192 SONET) lambdas between Chicago and Sunnyvale**
**At least two more in years two or three**
**Two dedicated lambdas on ORNL path to Chicago, more available (with very flexible technology)**

## Resources Cont.

**A progression of switching approaches**
**Study/compare MPLS(GMPLS), SONET, all optical**
**New all-optical technologies coming (e.g., laser tuning)**
**Local Storage**
**Logistical Storage Depot (prior to local application development)**
**Progression of experimental point-to-point transport technologies**
**Fiber channel**
**Infiniband**

## Global Architecture for DOE UltraScience Net

## UltraScience Net Circuit Switching

## Schedule

**Atlanta-Chicago link to be up this summer**
**Connecting ORNL to it may initially be via Atlanta**
**Chicago-Sunnyvale paced by NLR**
**PNNL fiber schedule will pace their connection**
**Expect first user-traffic this summer**

## Thank You

- http://www.csm.ornl.gov/ultranet