# Presentation: 751022

## TeraGrid Applications Workshop 

- Ian Foster
- Mathematics and Computer Science Division
- Argonne National Laboratory
- and
- Department of Computer Science
- The University of Chicago
- http://www.mcs.anl.gov/~foster

## Why Are We Here?

- Major application communities building on Grid technologies

- Major service providers building production Grid infrastructures

## Desired Outcomes

- User & TG developer consensus on TG plans
- Node and Grid software infrastructure
- Grid Tools
- Operational issues
- Plans for “early adopter” TG applications
- “Challenge problems” that can validate _early_ our ability to create and apply a terascale cyberinfrastructure
- Timelines and requirements

## Three Dimensions of Grid Computing

- Grid-enabled applications
- Data-intensive, metacomputing, on-demand
- Infrastructure
- Hardware, networks, storage
- Services: authentication, resource discovery, resource scheduling
- Operational & support procedures
- Tools
- Message passing, data access, data replication, high-throughput computing, ...

## Who is Here

- Representatives of application projects
- “Traditional”: Cactus, ...
- “Non-traditional”: ATLAS, CMS, LIGO, SDSS, NVO, BIRN, ...
- Infrastructure developers
- TeraGrid sites: NCSA, SDSC, Caltech, ANL
- iVDGL sites
- Tool developers
- Condor, Globus, NMI, MPICH-G2, SRB, Cactus, ...

## The 13.6 TF TeraGrid:Computing at 40 Gb/s

*26*

*24*

*8*

*4*

- HPSS

*5*

- HPSS

- HPSS

- UniTree

- External Networks

- External Networks

- External Networks

- External Networks

- Site Resources

- Site Resources

- Site Resources

- Site Resources

**NCSA/PACI**
**8 TF**
**240 TB**

**SDSC**
**4.1 TF**
**225 TB**

**Caltech**

**Argonne**

- NCSA, SDSC, Caltech, Argonne                              www.teragrid.org

## iVDGL: A Global Grid Laboratory

- International Virtual-Data Grid Laboratory
- A global Grid lab (US, EU, South America, Asia, ...)
- A place to conduct Data Grid tests “at scale”
- A mechanism to create common Grid infrastructure
- A production facility for LHC experiments
- An experimental laboratory for other disciplines
- A focus of outreach efforts to small institutions
- Funded for $13.65M by NSF

- “We propose to create, operate and evaluate, over asustained period of time, an international researchlaboratory for data-intensive science.”
- 	*From NSF proposal, 2001*

## Initial US-iVDGL Data Grid

- UCSD

- Florida

- Wisconsin

- Fermilab

- BNL

- Indiana

- BU

- Other sites to be added in 2002

- SKC

- Brownsville

- Hampton

- PSU

- JHU

- Caltech

## iVDGL Map (2002-2003)

- DataTAG

- Surfnet

**Later**
**Brazil**
**Pakistan**
**Russia**
**China**

## U.S. GRIDS CenterNSF Middleware Infrastructure Program

- GRIDS = Grid Research, Integration, Deployment, & Support
- NSF-funded center to provide
- State-of-the-art middleware infrastructure to support national-scale collaborative science and engineering
- Integration platform for experimental middleware technologies
- ISI, NCSA, SDSC, UC, UW + commercial partners

- www.grids-center.org

## An Example Virtual Organization: CERN’s Large Hadron Collider

- 1800 Physicists, 150 Institutes, 32 Countries
-    100 PB of data by 2010; 50,000 CPUs?

## Data Intensive Science: 2000-2015

- Scientific discovery increasingly driven by IT
- Computationally intensive analyses
- Massive data collections
- Data distributed across networks of varying capability
- Geographically distributed collaboration
- Dominant factor: data growth (1 Petabyte = 1000 TB)
- 2000	~0.5 Petabyte
- 2005	~10 Petabytes
- 2010	~100 Petabytes
- 2015	~1000 Petabytes?

- How to collect, manage,
- access and interpret this
- quantity of data?

**Drives demand for “Data Grids” to handle****additional dimension of data access & movement**

## Biomedical InformaticsResearch Network (BIRN)

- Evolving reference set of brains provides essential data for developing therapies for neurological disorders (multiple sclerosis, Alzheimer’s, etc.).** **
- Today
- One lab, small patient base
- 4 TB collection
- Tomorrow
- 10s of collaborating labs
- Larger population sample
- 400 TB data collection: more brains, higher resolution
- Multiple scale data integration and analysis

## Data Grid Projects

- Particle Physics Data Grid (US, DOE)
- Data Grid applications for HENP expts.
- GriPhyN (US, NSF)
- Petascale Virtual-Data Grids
- iVDGL (US, NSF)
- Global Grid lab
- TeraGrid (US, NSF)
- Dist. supercomp. resources (13 TFlops)
- European Data Grid (EU, EC)
- Data Grid technologies, EU deployment
- CrossGrid (EU, EC)
- Data Grid technologies, EU emphasis
- DataTAG (EU, EC)
- Transatlantic network, Grid applications
- Japanese Grid Projects  (APGrid) (Japan)
- Grid deployment throughout Japan

- Collaborations of application scientists & computer scientists
- Infrastructure devel. & deployment
- Globus based

## Grid Communities & Applications:Data Grids for High Energy Physics

- www.griphyn.org            www.ppdg.net           www.eu-datagrid.org

## Early GriPhyN Challenge Problem:CMS Data Reconstruction

**NCSA Linux cluster **

**5) Secondary reports complete to master**

**Master Condor job running at Caltech**

**7) GridFTP fetches data from UniTree**

**NCSA UniTree - GridFTP-enabled FTP server**

**4) 100 data files transferred via GridFTP, ~ 1 GB each**

**Secondary Condor job on WI pool**

**3) 100 Monte Carlo jobs on Wisconsin Condor pool**

**2) Launch secondary job on WI pool; input files via Globus GASS**

**Caltech workstation**

**6) Master starts reconstruction jobs via Globus jobmanager on cluster**

**8) Processed objectivity database stored to UniTree**

**9) Reconstruction job reports complete to master**

- Early GriPhyN Challenge Problem:CMS Data Reconstruction

- Scott Koranda, Miron Livny, others

## Desired Outcomes

- User & TG developer consensus on TG plans
- Node and Grid software infrastructure
- Grid Tools
- Operational issues
- Plans for “early adopter” TG applications
- “Challenge problems” that can validate _early_ our ability to create and apply a terascale cyberinfrastructure
- Timelines and requirements