# Presentation: 047931

## Report to ICFAAugust 10, 1999Matthias Kasemann, FNAL

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Related URL’s

- ICFA-SCIC Homepage
- http://www.hep.net/ICFA/index.htmlCern -> Scientific Committees -> ICFA -> ICFA Standing Committee on International Connectivity
- ICFA-NTF Homepage
- http:/nicewww.cern.ch/~davidw/icfa/icfa-ntf.html
- ICFA-NTF July’98 Report
- http://nicewww.cern.ch/~davidw/icfa/July98Report.html

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## ICFA meeting, Vancouver, 1998 

- ICFA received the final report of the 	Networking Task Force (ICFA-NTF).
- Decision: create a Standing Committee on Interregional Connectivity (ICFA-SCIC).
- Committee members represent major HEP user communities and laboratories.
- Focus should be on intercontinental connectivity (see charge).

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Charge to ICFA-SCIC:

- Make recommendations to ICFA concerning the connectivity between America, Aisia and Europe.
- As part of the process of developing theserecommendations, the committee should
- monitor traffic,
- keep track of technology developments,
- periodically review forecasts of future bandwidth needs, and
- provide early warning of potential problems.
- Create subcommittees when necessary to meet the charge.
- The chair of the committee should report to ICFA once a year, at its joint meeting with laboratory directors.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## ICFA-SCIC membership:

- The **chair** is appointed directly by ICFA.	
- Each of the major user **laboratories**, CERN, DESY, FERMILAB, KEK and SLAC, should appoint one member each. 	 
**ECFA**, **DPF** jointly with **IPP**, and **ACFA**, should appoint two members each. 	 
- ICFA will appoint one member from the**Russian federation** and one member from **South America**.	

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## ICFA-SCIC membership:

- The representatives from the laboratories are:
- Manuel Delfino, (CERN),Michael Ernst (DESY),Kasemann (Fermi) (chair),Yukio Karita (KEK),Richard Mount (SLAC).
- The North American user representatives are:
- Harvey Newman (USA),Dean Karlen (Canada).
- For Russia:   Alexei Morozow (ITEP)

- ECFA has nominated:
- Frederico Ruggieri (INFN Frascati),Denis Linglin (IN2P3, Lyon).
- ACFA has nominated:
- Prof. Rongsheng Xu 	(Computer Center, IHEP China)
- Prof. HwanBae Park (Korea University)
- For South America:  Sergio F. Novaes (University de S.Paulo)

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## ICFA-SCIC meetings:

- April 15. - 16. at FNAL.
- Main topics:
- review charge to SCIC,
- review work of ICFA-NTF( a lot of it overlaps with SCIC charge),
- define priorities and projects,
- organize work (and subgroups).
- Video Conference on July 6, 1999.
- Topics:
- update on status of network connectivity
- working group plans
- action items

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Recommendations of the ICFA NTF

- 	Recommendations concerning Inter-continental links:
**ICFA should encourage the provision of some considerable extra bandwidth, especially across the Atlantic**
**ICFA participants should make concrete proposals,               (such as recommendation to  increase bandwidth across       the Atlantic, approach to QoS , co-operation with other disciplines and agencies, etc.)**
**The bandwidth to Japan needs to be upgraded**
**Integrated end-to-end connectivity is primary requirement, to be emphasized to continental ISPs, and academic and research networks**

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Topics:

- There was a discussion on the reality of QoS today:
- how to administer it
- how to take advantage of it for HEP applications
- bulk data transfer service to be done at low utilization periods,
- move away from applications that demand low latency such as telnet
- there is a need for network aware applications for:
- interactive network connections
- distinguish bulk data transfer from low latency traffic
- IP-telephony, Voice over IP
- collaborative tools

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Topics (2):

- Status & future directions of US research networks
- The network environment for HEP research in the US continues to improve in virtually all areas
- US HEP facilities work well on ESnet
- University access improving rapidly with emerging I2 networks
- Acceptable Use Policy (AUP) issues complicate things

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Canada - Germany(Desy): poor

- There are severe trans- atlantic performance problems which need to be addressed.
- They need inter- national cooperation.
- ICFA-SCIC(Michael Ernst, DESY) will discuss with DFN.

- unusable

- poor

- ~ ok

- excellent

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Canada - Germany(Desy): poor

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## S. America / Brazil

- The needs are to increase bandwidth.
- Need A&R dedicated links, even better if had just HEP links.
- For a D0 farm they need 20kbps/PC or for 30 PC need 600kbps (total available on US link is 2000kbps).
- The main HEP partners are CERN & FNAL.
- Unclear how much fiber/infrastructure is being put in place to the US.
- there are at least 2 international consortia Oxygen, Global Crossing are going into Brazil.
- Funding is the main problem and limits international connectivity.
- The telecomm industry is being privatized, expect more competition.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Germany

- Europe connectivity improved with TEN-155,
- Expect an upgrade of A&R net in Germany next year.
- Problem areas:
- N. America (2*OC3 won't help much);
- Russia now & Japan in future.
**DFN now does ICMP traffic shaping especially at International exchanges**.
- Hoping to be able to use Differentiating Services to provide managed bandwidth for improved performance to HEP sites in N. America.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Recent History of the US-CERN Link

-  **CERN/US/France/Canada/UN-WHO Consortium**
_**October 1996 - August 1997**_** **
** ****Upgraded leased digital  CERN-US line: ****2.048 Mbps **
** ****Set-up of monitoring tools and traffic control**
** ****Start Deployment of VRVS a Web-based ****	**** videoconferencing system**
_**September 1997 - April 1999**_** **
** ****Upgraded leased CERN-US line to ****2 X 2.048 Mbps;**** **** Addition of a backup and “overflow” leased line at                          ****	****2.048 Mbps**** ****(total 6 Mbps)**** to avoid saturation in Fall 1998**
** ****Production deployment of VRVS software in the US and    ****	****	****Europe (to 1000 hosts by 4/99; Now 1300). **
** ****Set-up of CERN-US consortium rack at Perryman**** to peer with ESnet and other international nets**
** ****Test of QoS features using new Cisco software and  ****	****	****   ****	****hardware**

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Current Dev. for the US-CERN Link

-     _**October 1998 - September 1999**_
**Market survey and selection of Cable&Wireless as ISP. **
**Began Collaboration in Internet2 applications and network developments. **
**Move to C&W Chicago PoP, to connect to ****STARTAP.**
**From April 1999, set-up of a ****12 Mbps**** ATM VP/VBRnrt              circuit between CERN and C&W PoP**
**9/99: Transatlantic upgrade to ****20 Mbps ****September 1st, coincident with CERN/IN2P3 link upgrade**
**7/99: Begin organized file transfer service to “mirror”              Babar DST data from SLAC to CCIN2P3/Lyon **

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Bandwidth Growth Observation/Prediction

- Technology Tracking and Cost Model will be performed  by the ICFA SCIC Committee

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Japan site report - Yukio Karita

- Issues:
- vBNS is reluctant to peer with Japan HENP at STAR-TAP
- NACSIS - Europe line is saturated:
- will be upgraded from 2 Mbps to 30 Mbps on October 1, 1999.
- 3-4 Mbps for KEK-CERN ATM PVC will be provided then.
- Both of this done with Japanese $$’s.
- European funding can increase the bandwidth.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Remote Collaborations: VRVS videoconferencing 

- Remote Collaborations: VRVS videoconferencing

- VRVS CERN-Caltech development (1995-
- 21 reflectors Running in U.S. Europe and Asia
**Switzerland**: *CERN (2)*
**Italy**: *CNAF Bologna*
**UK**: *Rutherford Lab*
**France**: *IN2P3 Lyon, Marseilles*
**Germany**: *Heidelberg Univ.*
**Finland**: *FUNET*
**Spain**: *IFCA-Univ. Cantabria*
**Russia**: *Moscow State Univ., Tver. U.*
**U.S**:
-  *Caltech, LBNL, SLAC, FNAL,*
-  ANL, BNL, *Jefferson Lab.*
**DoE HQ** Germantown

- - **Asia**: ***Academia Sinica.****** ***_**Taiwan**_
*- ***South America*****: ******CeCalcula******.***_**Venezuela**_

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## VRVS: Widespread and Strong Support from the Research and Education Communities

-   _HENP Community_. Hosts registered from: *CMS, Atlas, Alice, Lhc-b, Aleph, NA48, NA49, NA50, AMS, Aleph, Babar, RHIC, CDF, Ceres/NA45, Chorus, Delphi, DESY/ZDV, H1, CEBAF, KLOE, KTeV, L3, Minos, Soudan2, OPAL, PHENIX, STAR, SpEcTrE, WA95, WA98, ZEUS, etc ...*
-   _Strong interest from others Research Communities_
- Internet2/UCAID *(University Corporation  for Advance Internet Development)*
- Ted Hans, Director, Application Development Internet2:
*“**..The Internet2 Community sees VRVS as the model for providing a highly functional video-enabled collaboration service for research and education...”*
- I2-DV (Internet2 Digital Video) Initiative recognized that VRVS is a uniquely suitable foundation for development and deployment of its applications. (*http://i2dv.nwu.icair.org*)
- P. Galvez is a member of the I2-DV (Internet2 Digital Video) steering committee

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Example: 9 Participants, CERN(2), Caltech, FNAL(2), Bologna (IT), 

***Example: 9 Participants, CERN(2), Caltech, FNAL(2), Bologna (IT), ***
***Roma (IT), Milan (IT), Rutherford(UK)***

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## SCIC monitoring WGActive Internet Monitoring Activities

- Les Cottrell – SLAC
- For the SCIC-WG

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Overview of Mechanism

- 19 Monitor sites, 10 countries
- 1300 monitor-remote-site pairs
- 379 unique hosts, 27 countries
- Measure response, jitter, loss, reachability
- Data goes back > 4 years
- 1 Million probes of Internet/day

- Treats Internet as black box
- Uses existing infrastructure (ping)
- Low cost, well understood

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

**Notes:**
Uses for PingER:
Establish baselines
Can/how will infrastructure support new apps & services
A priori determine if will meet expectations
Use to set SLAs, choose route/ISP
Detect anomalies
	Finding, understanding problems
Determine trends
Predict future, provide engineering/planning input

Monitoring is 
End-to-end (I.e. close to what user experiences)
active: I.e. insert probes into the network
hierarchical (not full mesh), choice of remote sites made by monitoring sites based on needs
lightweight, uses existing ping infrastructure, no code to install at remote sites, small net impact 100bps/pair

Results (both raw & summarized data) available publicly on web
Code is free and publicly available, being used by other projects such as the Cross Industry Working Team (XIWT)
Site distribution reflects interest of Esnet, HENP, I.e. heavily .edu, European and .gov

## Deployment in HEP

- Over 50% of HEP collaborator sites are explicitly monitored as remote sites by PingER
- Atlas (37%), **BaBar(68%)**, Belle(23%), **CDF(73%)**, CMS (31%), **D0(60%)**, Zeus (35%), Aleph, Delphi, Opal, L3 (43%)
- Created focussed PingER pages for **BaBar, CDF, D0**
- Remainder represented by beacon sites
- Selected to represent countries/R&E nets
- About 50 beacon sites in 27 countries

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Performance Trends

**Bandwidth ****TCP**** < 1460/(RTT * sqrt(loss))**

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

**Notes:**
Use formula (from “The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm” by Mathis, Semke, Mahdavi & Ott, ACM SIGCOMM, vol 27, number 3, July 1997) together with PingER measured RTT & loss from ESnet monitoring sites  to predict the maximum TCP bandwidth.

Numbers in parentheses indicate the number of monitor-remote site pairs contributing to the points
ESnet to ESnet looks best, followed by Esnet to .edu, Canada.
Note that over an order of magnitude improvement in last 4 years for Esnet, .edu & Canada, main compoment comes from reductions in packet losses, but RTT has also improved due to faster links, increased switching (I.e. routes pass through fewer routers, and switches have lower latency)

## Problem areas

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

**Notes:**
This slide identifies some problem areas recently identified by the SCIC.

The on diagonal elements are for trafiic that stays within a top level domain, often this means it stays within a single Network Service Provider (e.g. .uk to .uk stays within Janet). The on-diagonal elements generally have better performance, a reflection that it is good to reduce the number of NSPs crossed, and more generally that peering between NSPs is a critical  issue.

## Overview & Future directions of Internet Monitoring in HEP

- Action Items:
- Need to add India and S. America to the Beacon sites. 	 
- Calibrate the formula 	 	TCP BW < (MSS/RTT)* (1/sqrt(loss)) with measurements within the HEP community, e.g. CERN-SLAC, SLAC-LBNL etc.
- HEPNRC will redouble its efforts to get more heavily involved with PingER once again.
- Increase efforts to gather and archive traceroutes between major sites

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## Network monitoring conclusions

- Performance is getting better
- ESnet is well provisioned
- Within R&E networks things are good
- Transoceanic, needs special care
- Peering is critical
- Monitoring bare network
- If one wants to go beyond ping to start to understand network aware applications and understand the barriers to higher throughput it will need to involve the applications people. This would be less global than PingER, i.e. more focussed on point-to-point particular links and applications. This is not part of the monitoring group's responsibilities.
- Report on Survey & tracking of HEP network aware applications, QoS, prototyping work in the area of:
- distributed databases, data transfer, collaborative tools

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## SCIC Summary: Working Groups 

- Requirements Analysis Working Group (Harvey Newman & Matthias Kasemann)
- Proposed action
- compare expected requirements for 2000 to reality
- comment on "network aware" applications, R&D prototyping effort
- Report to ICFA in Spring 2000
- Technology, status, cost & development/expectations for HEP (Richard Mount, Michael Ernst, David Williams)
- Keep track of technology trends:
- review networking activities and initiatives,
- report on pilot projects,
- have active participation of ICFA-SCIC members in projects

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## SCIC Summary

- Need small focussed task force to address particularly bad performance areas:
- Japan-US, Japan - Europe: some relief on the horizon
- Canada-Germany (getting Germany to STARTAP)
- Document Status of Remote regions
- get reports from India, Pakistan
- UK - N. America: invite UK person to report on connectivity.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999

## SCIC Summary

- Reports to ICFA planed:
- Summer 1999: monitoring results, short report on activity, report on short term bottlenecks
- Spring 2000 / 2002 / ... every 2 years: monitoring results, recommendations for long term situation
- Spring 2001 / 2003 / ... every 2 years: monitoring results, update report on requirements
- First report to ICFA on Feb 10,11 2000, (report ready by October, 1999)
- 3rd Meeting: November 13, 1999 CERN at which we derive recommendations for the ICFA report.

- ICFA_SCIC Report #1   Matthias Kasemann, FNAL

- August 10, 1999