# Presentation: 168194

## Network Connectivity and Projects  – a Perspective from the UKRichard Hughes-Jones PPNCG

- SuperJANET4 Status
- Connectivity to Europe
- Connectivity to the US
- Grid Networking Projects

##  SuperJANET4: Backbone and Access links

-  SuperJANET4: Backbone and Access links

- Worldcom supplied the transmission
- UKERNA layer the IP service
- Core PoP IP router at Worldcom
- Backbone Access Router at MANs
- Access Links:
- Large MAN 2.5 Gbit -> 10-20 Gbit
- Medium MAN 622 Mbit -> 2.5 Gbit
- 4 node DWDM development net
- Deployment Status:
- Backbone Oct 00 Routers Nov 00
- All sites Mar 01
- Proved to be Stable
- Constant growth of traffic

##  SuperJANET4: ping rtt Core routers 

-  SuperJANET4: ping rtt Core routers
- Jun 01

**Notes:**
SJ4 Core Router Access Delays (1)

##  SuperJANET4: ping rtt Core routers

-  SuperJANET4: ping rtt Core routers

- Lancaster

- Glasgow

**MAN / LAN Issues**

- Bristol

- Cambridge

**Notes:**
Particle Physics sites (1).
Bristol, Lancaster, Liverpool, Cambridge.

##  London MAN Upgrade

**UDP Throughput**
**Mbit/s**

**UDP Packet loss**
**%**

**UDPmon Tests**
**Manchester – London**
**MAN was 155 Mbit ATM**

**1****st**** Oct                 Time interval in Weeks**

- Richard HJ

**Notes:**
Particle Physics sites (1).
Bristol, Lancaster, Liverpool, Cambridge.

##  Connectivity to Europe – Access links (1)

-  Connectivity to Europe – Access links (1)

- UK Access link 155 Mbit ATM
- Sustained rate 130 Mbit
- Contract to end of Nov 01
- Bad news for users !

**Notes:**
ICFAMON plot shows significant packet loss on RAL-CERN connection at peak times especially from 30th October onwards.

**Traceping Oxford to CERN for 31****st**** October 2001**

-  Connectivity to Europe – Access links (2)

**loss around **
**ten155-gw.ja.net router**

- J. Macallister

**Notes:**
Illustrates packet loss around ten155-gw.ja.net router.

##  Connectivity to Europe : Geant

**Start mid November 2001**
**UKERNA switched off TEN-155 3 Dec 2001**
**Monitoring – see later**

**Notes:**
Particle Physics sites (1).
Bristol, Lancaster, Liverpool, Cambridge.

##  Connectivity to US

**6 * 155 Mbit links**
**US link about to go to 2.5 G bit**
**IP commodity attached in London **
**Research traffic over 2.5G bit**
**Peer in Hudson St. **
**622 Mbit to Esnet**
**622 Mbit to Abilene. **

**Notes:**
Particle Physics sites (1).
Bristol, Lancaster, Liverpool, Cambridge.

##  Monitoring: US Traffic 

- UKERNA Traffic data Kbit/s. Blue Traffic from US;  Maroon Traffic to US
- 17 Jul 2000 (300kbit/s)		7 Dec 2001 (900kbit/s)
- 				peak is 88% of total BW 930 Mbit
**10 minute averages** 				 **10 minute averages**

**Last 7 days ****1 hour averages** 			 **Last 7 days** **1 hour averages**

##  Monitoring: US Traffic - TrendsJan 99 – Nov 02

-  Monitoring: US Traffic - TrendsJan 99 – Nov 02

- UKERNA – US Traffic Daily totals Mbytes

- Weekend

- Xmas 99

- Weekday

- Xmas 01

- Peak load ~88 %

- J. Macallister

## Grid Network Projects

- DataGrid
- GridPP
- Extending DataGrid to BaBar, D0, CDF,... in UK
- DataTAG
- Foci: Network resarch, Grid inter-operability
- MB-NG
- UK funded 2.5Gbit Development Network
- Optical Switching

## Network Monitoring

- Several tools in test – plugged into a coherent structure:
- PingER, RIPE one way times, iperf, UDPmon, rTPL, GridFTP, and NWS prediction engine
- continuous tests for last few months to selected sites:
-  DL Man RL UCL CERN Lyon Bologna SARA NBI SLAC ...
- The aims of monitoring for the Grid:
- to inform Grid applications, via the middleware, of the current status of the network – input for resource broker and scheduling
- to identify fault conditions in the operation of the Grid
- to understand the instantaneous, day-to-day, and month-by-month behaviour of the network – provide advice on configuration etc.
- Report written on LDAP scheme for publishing the network information
- Will be used by UK e-science sites and non HEP WPs

## UDPmon Measuring Throughput with UDP

- Send a burst of UDP frames spaced at regular intervals
- Vary the frame size and the frame transmit spacing
- Record
- The time to send and the time to receive the frames
- The number received, the number lost, number out of order
- The received inter-packet spacing

**Local Network**
**Monitoring**
**Store & Analysis**
**of Data (Access)**

**Access to current and historic data**
**and metrics via the Web, i.e. WP7**
**NM Pages, access to metric forecasts**

**Backend LDAP script to fetch metrics**
**Monitor process to push metrics**

**local**
**LDAP**
**Server**

**Grid Application access via**
**LDAP Schema to**
**- monitoring metrics; **
**- location of monitoring data.**

**PingER**
**(RIPE TTB)**
**IperfER**
**UDPmon**
**rTPL**
**NWS**
**etc**

**LDAP**
**Schema**

**Grid Apps**
**GridFTP**

- Network Monitoring Architecture

- Robin Tasker

## Network Monitoring Components 

- Network Monitoring Components

- Ping

- Netmon

- UDPmon

- iPerf

- Ripe

**Cron**
**script**

**control**

**Cron**
**script**

**control**

**Cron**
**script**

**WEB Display**

**Analysis**

**Grid Broker**

**Predictions**

**Web I/f**

**Scheduler**

**Tool**

**Clients**

**LDAP**

**raw**

**LDAP**

**raw**

## Ping & UDP throughput MAN-RAL From 20 Oct 01

**Ping & UDP throughput MAN-RAL ****From 20 Oct 01**

**PingER rtt (ms) **
**dl – RAL **
**1000 byte packet**
**Forecast**
**UDPmon Zero packet loss!**
**UDPmon throughput Mbit/s **
**man – RAL**
**300 * 1400 byte frames**

## Ping & UDP throughput MAN-CERNFrom 20 Oct 01

**Ping & UDP throughput MAN-CERN****From 20 Oct 01**

**PingER rtt (ms) **
**dl – cern **
**1000 byte packet**
**Forecast**
**UDPmon throughput Mbit/s **
**man – cern**
**300 * 1400 byte frames**

## iperf & UDP throughput MAN-CERN From 20 Oct 01

**iperf & UDP throughput MAN-CERN**** From 20 Oct 01**

**Iperf throughput **
**ucl – cern **
**262144 byte buffer**
**Forecast**
**UDPmon throughput Mbit/s **
**man – cern**
**300 * 1400 byte frames**

## iperf & UDP throughput MAN-SARA From 20 Oct 01

**iperf & UDP throughput MAN-SARA**** From 20 Oct 01**

**Iperf throughput **
**ucl – sara **
**262144 byte buffer**
**Forecast**
**UDPmon throughput Mbit/s **
**man – sara**
**300 * 1400 byte frames**

## iperf & Pinger UK-Bologna From 20 Oct 01

**iperf & Pinger UK-Bologna**** From 20 Oct 01**

**Iperf throughput **
**ucl – Bologna **
**262144 byte buffer**
**Forecast in green**
**PingER rtt (ms) **
**dl – Bologna**
**1000 byte packet**
**Forecast**

## iperf throughput UCL-SARA From 1 Nov 01 – Geant Operational

**Iperf throughput **
**ucl – sara **
**262144 byte buffer**

**Geant Enabled**

**Routing Stable**

## Brief outline of work plan

- High Performance High Throughput:
- Understanding the end system HW,
- best way to monitor traffic and protocol packets,
- Type and effect of background traffic,
- Throughput vs rtt,  throughput vs window-size,
- Throughput using multiple TCP streams, and effect on the Network
- GridFTP and BBftp at Gigabit,
- Effect of different TCP stacks,
- Use of non-TCP protocols, and effect on the Network
- Show and tell demos
- Bandwidth Reservation and QoS
- Identification of traffic classes and middleware components requiring QoS
- Study of traffic differentiation and traffic engineering techniques
- demonstration of QoS & Bandwith Reservation

## The EU DataTAG project

- EU Transatlantic Gigabit project.
- Status: approved for 4 M ECU start on 1 Dec 2001
- Partners: CERN/PPARC/INFN/UvA. IN2P3 sub-contractor
- The main foci are:
- Grid Network Research including:
- Provisioning (CERN)
- Investigations of high performance data transport (PPARC)
- End-to-end inter-domain QoS + BW / network resource reservation
- Bulk data transfer and monitoring (UvA)
- Interoperability between Grids in Europe and the US
- PPDG, GriPhyN, DTF, iVDGL (USA)** **

## DataTAG project 

**DataTAG project**** **

**Abilene**

**ESNET**

**MREN**

**STAR-TAP**

**IT**
**GARR-B**

**NewYork**

- Olivier Martin

**NL**
**SURFnet**

**STAR**-**LIGHT**

**CERN**

## Optical Switching - DataTAG

**2.5 Gbps SDH “Lambda”**
**100/1000 Mbps Ethernet**

- Amsterdam

- λ Broker

- Application

- CERN

- StarLight

- Italy

- UK

**Optical Switching - DataTAG**

- 7600

- Iwire
- Abilene
- ESnet

## MB – NG SuperJANET Testbed

**SuperJANET4**
**Production**
**Network**

**Leeds**

- RAL /
- UKERNA

- RAL /
- UKERNA

- SJ4 Dev
- C-PoP
- Reading

- SJ4 Dev
- C-PoP
- Warrington

- SJ4 Dev
- C-PoP
- London

**Gigabit Ethernet**
**2.5 Gbit POS**
**10  Gbit POS**

## E-science core project

- Project to investigate and pilot:
- end-to-end traffic engineering and management over **multiple administrative domains** – MPLS in core diffserv at the edges.
- Managed bandwidth and Quality-of-Service provision. (Robin Tasker)
- High performance high bandwidth data transfers. (Richard HJ)
- Demonstrate end-to-end network services to CERN using Dante EU-DataGrid and the US DataTAG.
- Status: approved with requested funds start on 1 Dec 2001
- Partners:CISCO, CLRC, Manchester, UCL, UKERNA plus Lancaster and Southampton (IPv6)
- A technical meeting held – draft spec. of the HW required
- Would like to use real Grid traffic – as demonstrator:
- CDF UCL-RAL
- BaBar Man-RAL

##  And Finally in case it helps:

- GridPP and DataGrid have been working on packaging Globus v2 as a .rpm kit.
- Makes installation simpler.
- Available from DataGrid WP6 web:
-  marianne.in2p3.fr/datagrid/testbed1/globus

##  Conclusion & Status

- SuperJANET4 stable working well
- Access links to Campus becoming very inportant
- Network Links and peering to US good overall
- US-UK bandwidth upgrade overdue
- Geant access to Europe OK (at the moment !)
- Considerable UK interest in the Internet2 tests
- Grid: Lots of interesting work to do ...