# Presentation: 215225

## The GriPhyN Project
(Grid Physics Network)

**(****Gri****d ****Phy****sics ****N****etwork)**

**Paul Avery**
**University of Florida**
**http://www.phys.ufl.edu/~avery/**
**avery@phys.ufl.edu**

**SCOP @ FNAL****Oct. 24, 2000**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Motivation: Data Intensive Science

**Scientific discovery increasingly driven by IT **
**Computationally intensive analyses**
**Massive data collections**
**Rapid access to large subsets**
**Data ****distributed**** across networks of varying capability**
**Dominant factor: data growth**
**~0.5    Petabyte   in  2000 (BaBar)**
**~10     Petabytes by 2005**
**~100   Petabytes by 2010**
**~1000 Petabytes by 2015?**
**Robust IT infrastructure essential for science**
**Provide rapid turnaround**
**Coordinate, manage the ****limited**** computing, data handling and network resources effectively**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Grids as IT Infrastructure

**Grid: ****Geographically distributed IT resources****          configured to allow coordinated use**
**Physical resources & networks provide raw capability**
**“****Middleware” services tie it together**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Data Grid Hierarchy (LHC Example)

**Data Grid Hierarchy (LHC Example)**

**Tier0  CERN****Tier1  National Lab****Tier2  Regional Center at University****Tier3  University workgroup****Tier4  Workstation**

** ****GriPhyN:**
**R&D**
**Tier2 centers**
**Unify ****all**** IT resources**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Why a Data Grid: Physical

**Unified system: ****all computing resources part of grid**
**Efficient resource use (manage scarcity)**
**Resource discovery / scheduling / coordination truly possible**
**“****The whole is greater than the sum of its parts”**
**Optimal data distribution and proximity **
**Labs are close to the (raw) data they need**
**Users are close to the (subset) data they need**
**Minimize bottlenecks**
**Efficient network use**
**local > regional > national > oceanic**
**No choke points**
**Scalable growth**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Why a Data Grid: Demographic

**Central lab cannot manage / help 1000s of users**
**Easier to leverage resources, maintain control, assert priorities at regional / local level**
**Cleanly separates functionality**
**Different resource types in different Tiers**
**Organization vs. flexibility**
**Funding complementarity (NSF vs DOE), targeted initiatives**
**New IT resources can be added “naturally”**
**Matching resources at Tier 2 universities**
**Larger institutes can join, bringing their own resources**
**Tap into new resources opened by IT “revolution” **
**Broaden community of scientists and students**
**Training and education**
**Vitality of field depends on University / Lab partnership**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN = Applications + CS + Grids

**Several scientific disciplines**
**US-CMS****	****	****High Energy Physics**
**US-ATLAS****	****High Energy Physics**
**LIGO/LSC****	****Gravity wave research**
**SDSS****	****	****	****Sloan Digital Sky Survey**
**Strong partnership with computer scientists**
**Design and implement production-scale grids**
**Maximize effectiveness of large, disparate resources**
**Develop common infrastructure, tools and services**
**Build on foundations ******** PPDG, MONARC, CONDOR, ...**
**Integrate and extend existing facilities**
****** ****$70M total cost ******** NSF(?)**
**$12M****	****R&D**
**$39M****	****Tier 2 center hardware, personnel, operations**
**$19M?****	****Networking**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Particle Physics Data Grid (PPDG)

-  First Round Goal: Optimized cached read access to 10-100 Gbytes 	drawn from a total data set of 0.1 to ~1 Petabyte
-  Matchmaking, Co-Scheduling: SRB, Condor, Globus services; HRM, NWS

**PRIMARY SITE**
**Data Acquisition,**
**CPU, Disk, **
**Tape Robot**

**SECONDARY SITE**
**CPU, Disk, **
**Tape Robot**

***Site to Site Data ***
***Replication Service***
***100 Mbytes/sec***

***ANL, BNL, Caltech, FNAL, JLAB, LBNL, ******SDSC, SLAC, U.Wisc/CS***

_***Multi-Site Cached File Access Service***_

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN R&D Funded!

**NSF/ITR results announced Sep. 13**
**$11.9M from ****I****nformation ****T****echnology ****R****esearch Program**
**$  1.4M in matching from universities**
**Largest of all ITR awards**
**Excellent reviews emphasizing importance of work**
**Joint NSF oversight from ****CISE**** and ****MPS**
**Scope of ITR funding**
**Major costs for people, esp. students, postdocs**
**No hardware or professional staff for operations !**
**2/3 CS + 1/3 application science**
**Industry partnerships being developed**
**Microsoft, Intel, IBM, Sun, HP, SGI, Compaq, Cisco**

**Still require funding for implementation****and operation of Tier 2 centers**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN Institutions

**U Florida**
**U Chicago**
**Caltech**
**U Wisconsin, Madison**
**USC/ISI**
**Harvard**
**Indiana**
**Johns Hopkins**
**Northwestern**
**Stanford**
**Boston U**
**U Illinois at Chicago**
**U Penn**
**U Texas, Brownsville**
**U Wisconsin, Milwaukee**
**UC Berkeley**

**UC San Diego**
**San Diego Supercomputer Center**
**Lawrence Berkeley Lab**
**Argonne**
**Fermilab**
**Brookhaven**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Fundamental IT Challenge

**“****Scientific communities of thousands, distributed globally, and served by networks with bandwidths varying by orders of magnitude, need to extract small signals from enormous backgrounds via computationally demanding (Teraflops-Petaflops) analysis of datasets that will grow by at least 3 orders of magnitude over the next decade: from the 100 Terabyte to the 100 Petabyte scale.”**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN Research Agenda

**Virtual Data technologies**
**Derived data, calculable via algorithm (e.g., most HEP data)**
**Instantiated 0, 1, or many times**
**Fetch vs execute algorithm**
**Very complex (versions, consistency, cost calculation, etc)**
**Planning and scheduling**
**User requirements (time vs cost)**
**Global and local policies + resource availability**
**Complexity of scheduling in dynamic environment (hierarchy)**
**Optimization and ordering of multiple scenarios**
**Requires simulation tools, e.g. MONARC**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Research Agenda (cont.)

**Execution management**
**Co-allocation of resources (CPU, storage, network transfers)**
**Fault tolerance, error reporting**
**Agents (co-allocation, execution)  ******** H. Newman talk**
**Reliable event service across Grid**
**Interaction, feedback to planning**
**Performance analysis (new)**
**Instrumentation and measurement of all grid components**
**Understand and optimize grid performance**
**Virtual Data Toolkit (VDT)**
**VDT = virtual data services + virtual data tools**
**One of the primary deliverables of R&D effort**
**Ongoing activity + feedback from experiments (5 year plan)**
**Technology transfer mechanism to other scientific domains**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN: PetaScale Virtual Data Grids

***Virtual Data Tools***

**Request Planning &**

**Scheduling Tools**

**Request Execution &**

**Management Tools**

***Transforms***

**Distributed resources**

**(code, storage,**

**computers, and network****)**

- Resource

- Management

- Services

**Resource **

**Management **

**Services**

- Security and

- Policy

- Services

**Security and**

**Policy **

**Services**

- Other Grid

- Services

**Other Grid**

**Services**

***Interactive User Tools***

**Production Team**

**Individual Investigator**

**Workgroups**

**Raw data**

**source**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## LHC Vision (e.g., CMS Hierarchy)

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## SDSS Vision

**SDSS Vision**

**Three main functions:**
**  ****Main data processing (FNAL)**
**Processing of raw data on a grid**
**Rapid turnaround with multiple TB data**
**Accessible storage of all imaging data**
**  ****Fast science analysis environment (JHU)**
**Combined data access and analysis of****calibrated data**
**Shared by the whole collaboration**
**Distributed I/O layer and processing layer**
**Connected via redundant network paths**
**  ****Public data access**
**Provide the SDSS data for the NVO (National Virtual Observatory) **
**Complex query engine for the public**
**SDSS data browsing for astronomers, and outreach**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## LIGO Vision

**Main data processing (Caltech/CACR)**
**Enable computationally limited searches******** periodic sources)**
**Access to LIGO deep archive**
**Access to Observatories**
**Science analysis environment for LSC****(**_**L**_**IGO **_**S**_**cientific **_**C**_**ollaboration)**
**Tier2 centers: shared LSC resource**
**Exploratory algorithm, astrophysics research****with LIGO reduced data sets**
**Distributed I/O layer and processing layer builds on existing APIs**
**Data mining of LIGO (event) metadatabases**
**LIGO data browsing for LSC members, outreach**

**LIGO Vision**

**LIGO I Science Run:****	****2002 – 2004+****LIGO II Upgrade:****	****2005 – 20xx ****	****(MRE to NSF 10/2000)**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## GriPhyN Cost Breakdown (May 31)

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Number of Tier2 Sites vs Time (May 31)

- 

- 

- 

- 

- 

- 

- 

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Grid Program of Work (e.g., CMS)

- First proto-Tier2 center
- Build basic services
- “4 Million” event samples

- MONARC Phase 3 simulations

- Initial work with Tier1+Tier2 Simulations

- Early Grid services
- File replication
- Multi-site caching

- First set of Tier2 centers

- Early subsystem data challenges

- Tier2 partially deployed at all sites

- Production-prototype test of Grid hierarchy, with 1st elements of Tier1

- Larger data challenges

- Production scale Grid hierarchy

- Final production shakedown

- Full distributed system software and instrumentation

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## LHC Tier2 Architecture and Cost

**Linux Farm of 128 Nodes (256 CPUs + disk)      ****	****$ 350 K**
**Data Server with RAID Array****	****$ 150 K**
**Tape Library****	****$   50 K **
**Tape Media and Consumables****	****$   40 K**
**LAN Switch****	****$   60 K**
**Collaborative Tools & Infrastructure****	****$   50 K**
**Installation & Infrastructure****	****$   50 K**
**Net Connect to WAN (Abilene)****	****$ 300 K**
**Staff (Ops and System Support)****	**_**$ 200 K**__**	**_****
**Total Estimated Cost (First Year)****	****$1,250 K**
**Average Yearly Cost including evolution,****	****$  750K**** ****upgrade and operations******
****** ****1.5 – 2 FTE support required per Tier2**
****** ****Assumes ****3 year hardware replacement**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)

## Current Grid Developments

**EU “DataGrid” initiative ******** F. Gagliardi talk**
**Approved by EU in August (3 years, $9M)**
**Exploits GriPhyN and related (Globus, PPDG) R&D**
**Collaboration with GriPhyN (tools, Boards, interoperability,****some common infrastructure)**
**http://grid.web.cern.ch/grid/**
**Rapidly increasing interest in Grids**
**Nuclear physics**
**Advanced Photon Source (APS)**
**Earthquake simulations (****http://www.neesgrid.org/****)**
**Biology (genomics, proteomics, brain scans, medicine)**
**“****Virtual” Observatories (NVO, GVO, ...)**
**Simulations of epidemics (Global Virtual Population Lab)**
**GriPhyN continues to seek funds to implement vision**

- Paul Avery (The GriPhyN Project)

- SCOP@FNAL (Oct. 24, 2000)