# Presentation: 576578

## ATLAS Tier 2 Paths Within ESnet

- Mike O’Connor
- ESnet Network Engineering Group
- Lawrence Berkeley National Lab
- moc@es.net

## ATLAS Tier 2 Connectivity

- Typical ATLAS tier 2 site connectivity should be via Abilene and ESnet in order to reach Brookhaven Lab.
- “ESnet’s Goal is that DOE Lab **↔** Univ. connectivity should be as good as Lab **↔** Lab and Univ. **↔** Univ.” *W. E. Johnston*

**Notes:**
Speaking to the ATLAS tier 2 sites.
Assuming that your institute has attained sufficient connectivity to the Abilene network, I will discuss how these paths through the ESnet are engineered today and the options that exist for the future.

## ATLAS Tier 2 Support

**The role of ESnet is to provide networking that supports and anticipates the needs of the Office of Science Labs and their collaborators.**
- ESnet support for ATLAS tier 2 site connectivity makes high bandwidth and fault tolerant paths between BNL and Abilene an absolute priority.
- Planning is underway to upgrade BNL’s ESnet OC48 circuit to a multi 10G lambda fault tolerant Metropolitan Area Network (MAN).    *(Targeted for  Q1 2006)*
- Today ESnet and Abilene peer in Atlanta, Chicago, New York and Sunnyvale with a combined bandwidth of 18.2Gbps.These four locations comprise a high bandwidth and fault tolerant interconnect between these two networks.

## ESnet Provides Global High-Speed Internet Connectivity forDOE Facilities and Collaborators (Spring, 2006 Configuration)

**ESnet Science Data Network (SDN) core**

**International (high speed)**
**10 Gb/s SDN core**
**10G/s IP core**
**2.5 Gb/s IP core**
**MAN rings (≥ 10 G/s)**
**OC12 ATM (622 Mb/s)**
**OC12 / GigEthernet**
**OC3  (155 Mb/s)**
**45 Mb/s and less**

**Office Of Science Sponsored (22)**

**NNSA Sponsored (12)**

**Joint Sponsored (3)**

**Other Sponsored (NSF LIGO, NOAA)**

**Laboratory Sponsored (6)**

_***42 end user sites***_

**ESnet IP core**

**CA*net4****	****   France**
**GLORIAD****  (Russia, China)****Korea (Kreonet2**

**Japan (SINet)**
**Australia (AARNet)**
**Canada (CA*net4**
**Taiwan (TANet2)**
**Singaren**

**ESnet IP core: Packet over SONET Optical Ring and Hubs**

** ****ELP**

** ****ATL**

** **

**commercial and R&E peering points**

**MAE-E**

**PAIX-PA**
**Equinix, etc.**

**PNWGPoP/****PAcificWave**

** ****SEA**

**ESnet Provides Global High-Speed Internet Connectivity for****DOE Facilities and Collaborators (Spring, 2006 Configuration)**

**ESnet core hubs**

** ****IP**

**high-speed peering points****with Internet2/Abilene**

**CERN**
**(USLHCnet****CERN+DOE funded)**

** ****NYC**

** ****SNV**

** ****SNV SDN**

** ****SDSC**

**Equinix**

** ****ALB**

** ****CHI**

**MREN**
**Netherlands**
**StarTap****Taiwan (TANet2,****   ASCC)****	**

**AU**

**international R&E peers**

**AU**

## ESnet & Abilene Peering points

**ESnet & Abilene Peering points**

***paths do not include all router hops**
****Example site logos chosen at random.**

## Abilene Routing Toward ESnet

**Abilene Interior Gateway Protocol (IGP) mileage based metrics determine the best exit path toward the ESnet. This common method also known as “hot potato routing” it will always compute an exit path to the nearest peering point.**

**Abilene Routing Toward ESnet**

## ESnet Routing Toward Abilene

**ESnet Routing Toward Abilene**

**Abilene Multi Exit Discriminators (MEDs) are used by ESnet to choose the best exit path toward Abilene. These MEDs are based on the same IGP metrics used in the interior of their network. This creates a symmetric path back to Abilene.**

## 32 Avenue Of the Americas NYC

## Chicago

## Current Load in Chicago

**Current Load in Chicago**

## Sunnyvale & Atlanta

## Future Directions

**Long Island MAN Ring**
**ESnet Lambda Infrastructure**

## ESnet LI MAN Ring

- λ2 SDN/circuit

- λ1 production IP

**10 Gb/s****optical channels**

- 2 λs (10** **G circuits) delivered as two 10 GigEther circuits
- 8 X current BNL bandwidth
- Physically diverse fiber paths forming a ring topology.
- Will be used as a10 Gb/s production IP and a10 Gb/s path for SDN circuit services
- Services switch over to the other service path during a circuit failure.

**ESnet LI MAN Ring**

**Circuits based on WDM rings **

- *SDN – Science Data Network

## ESnet 2010 Lambda Infrastructure and LHC T0-T1 Networking

**Denver**

**Seattle**

**Sunnyvale**

**LA**

**San Diego**

**Chicago**

**Pitts**

**Raleigh**

**Jacksonville**

**Atlanta**

**KC**

**Baton Rouge**

**El Paso - ****Las Cruces**

**Phoenix**

**Pensacola**

**Dallas**

**San Ant.**

**Houston**

**Albuq.**

**Tulsa**

**New York**

**Clev**

**Boise**

**CERN-1**

**GÉANT-1**

**GÉANT-2**

**Wash DC**

**CERN-2**

- Tier 1 Centers

- ESnet IP core hubs

- New hubs

- ESnet SDN/NLR hubs

- ESnet Production IP core (10-20 Gbps)
- ESnet Science Data Network core (10G/link)(incremental upgrades, 2007-2010)
- Other NLR links
- CERN/DOE supplied (10G/link)
- International IP connections (10G/link)

- Cross connects with Internet2/Abilene

- NLR PoPs

**TRIUMF**

**FNAL**

**BNL**

**CERN-3**

- CANARIE

## Conclusions

- ESnet provides an infrastructure and services that are critical to ATLAS science
- ESnet is well positioned to provide essential network services to the LHC tier 2 community
- ESnet is working on _*providing the DOE mission science networking requirements with several new initiatives and a new architecture*_