# Presentation: 691954

## US ATLAS Grid Projects

**Rob Gardner**
**Indiana University**
- Mid Year Review of US ATLAS Computing
- NSF Headquarters, Arlington VA
- June 20, 2002

**http://www.usatlas.bnl.gov/computing/grid/**

## Projects Overview

- Particle Physics Data Grid (T. Wenaus)
- Year 2 of 3-year project to deliver vertically integrated grid services to experiments
- Initial ATLAS activity: distributed data storage on the grid (Magda)  grid analysis (new)
- EDG kit evaluation and US ATLAS testbed certification (ANL);   Monitoring, information servers (BNL)
- GriPhyN (R. Gardner)
- Year 2 of 5 year project: main deliverables for ATLAS: VDT, Pacman, Grappa; Chimera toolkit: virtual data catalog and data language (VDC, VDL, VDLI)
- Virtual data prototyping effort by S. Vanaichine (ANL) P. Nevski (BNL)
-  Architecture for virtual data portal developed (Argonne-Chicago-IU project)
- iVDGL (R. Gardner), Networking (S. McKee)
- Platform on which to design, implement, integrate VDT
- Boston and Indiana University Prototype Tier2 Centers
- Forum for grid interoperability – collaborate with EU DataGrid, DataTag, etc.
- Networking: U of Michigan leads  see Internet2 working group:   http://www.internet2.edu/henp/
- US ATLAS grid applications testbed (K.De)
- Lots of contributions from various Lab and University groups!  BNL,ANL,LBL,BU,OU,UM,UTA,IU

- Rob Gardner        US Grid Projects

- June 20, 2002

## MAnager for Grid-based DAta
Designed for ‘managed production’ and  ‘chaotic end-user’ usage
Designed for rapid development of components to support users quickly, with components later replaced by Grid Toolkit elements
Deploy as an evolving production tool and as a testing ground for Grid Toolkit components
Adopted by ATLAS for 2002 ATLAS Data Challenges
Developers - T. Wenaus and W. Deng (fulltime, postdoc)

- Designed for ‘managed production’ *and*  ‘chaotic end-user’ usage
- Designed for rapid development of components to support users quickly, with components later replaced by Grid Toolkit elements
- Deploy as an evolving production tool and as a testing ground for Grid Toolkit components
- Adopted by ATLAS for 2002 ATLAS Data Challenges
- Developers - T. Wenaus and W. Deng (fulltime, postdoc)

**PPDG Magda**http://www.usatlas.bnl.gov/computing/grid/ppdg/

- Info:              http://www.usatlas.bnl.gov/magda/info
- The system: http://www.usatlas.bnl.gov/magda/dyShowMain.pl

- Rob Gardner        US Grid Projects

- June 20, 2002

## Status

- Full time developer/support  Wensheng Deng
- Development, support: ATLAS DC, US ATLAS testbed, LAr bench test DAQ, GDMP integration
- ATLAS DC production usage for cataloging (passive and active) and replication
- Improved usability, doc for replication; dynamic replication; more replication modes
- DC1 data replication from BNL to CERN to begin
- To be used for data replication among the ~18 sites of DC1 phase 0
- 260k files, ~11TB cataloged at present
- GDMP integration beginning
- Issues given to GDMP in Jan re: Magda integration fixed in V3
- David Rebatto, a CS working for Laura Perini, is helping
- GDMP3 being set up in Milan; will install Magda and do integration work there

- Torre Wenaus

- Rob Gardner        US Grid Projects

- June 20, 2002

## PPDG Year 2 & Magda

- GDMP integration
- Use GDMP to support a ‘publish/subscribe’ replication mode within Magda, targeted at ‘conventional production’ usage (what GDMP is designed for)
- Magda still ‘adds value’ with a more flexible approach to data management and replication
- Application in demos
- Build Magda outwards into distributed job management/analysis
- Distributed job management is to be the ATLAS PPDG yr 2 focus
- Together with distributed analysis?
- Apply as replica manager in hybrid store common project
- Build it outwards (or adapt it) to support more metadata? E.g. data history information; HES metadata (HEMP a teeny step in this direction)
- To be eclipsed by Alien? Examine Alien and evaluate

- Torre Wenaus

- Rob Gardner        US Grid Projects

- June 20, 2002

## Pacmanhttp://physics.bu.edu/~youssef/pacman/

- Package manager for the grid
- Saul Youssef (Boston)
- Used by VDT
- Single tool to easily manage installation and environment
-  fetch, install, configure, add to login environment, update
- Sits over top of many software packaging approaches (rpm, tar.gz, etc.)
- Uses dependency hierarchy, so one command can drive the installation of a complete environment of many packages
- Packages organized into *caches* hosted at various sites
- Distribute responsibility for support

- Rob Gardner        US Grid Projects

- June 20, 2002

## Grappa Workhttp://iuatlas.physics.indiana.edu/grappa/

- Grid user interface for Athena
- Flexible, use existing portal technology
- Submit Athena jobs to grid computing elements
- Manage JobOptions, record sessions
- File staging and output collection supported
- Registers output to MAGDA file catalog
- Packaged for general ATLAS use with Pacman
- Demonstrated on US ATLAS Grid Testbed (ATLFAST)
- Future: virtual data portal + GANGA (UK Grid) collaboration

- Rob Gardner        US Grid Projects

- June 20, 2002

## Athena Grid Job Submission

**Athena Grid Job Submission**

- Rob Gardner        US Grid Projects

- June 20, 2002

## Slide 9

- Rob Gardner        US Grid Projects

- June 20, 2002

## Slide 10

- Rob Gardner        US Grid Projects

- June 20, 2002

## Slide 11

- Rob Gardner        US Grid Projects

- June 20, 2002

## iVDGL in ATLAShttp://www.usatlas.bnl.gov/computing/grid/ivdgl/

- Main goals for ATLAS
- Integrate the GriPhyN VDT with ATLAS Core Software
- Develop two prototype Tier 2 Centers for the US ATLAS
- Integrate Tier 2 centers with US ATLAS Tier 1 Facility
- Support development of US ATLAS Grid Testbed
- Develop, integrate the US ATLAS piece of the iVDGL Laboratory
- Year 1
- Distribute, support VDT 1.1.x to US ATLAS sites
- Tier2: 80K per Center for cluster upgrades and disk for ATLAS Data Challenges

- Rob Gardner        US Grid Projects

- June 20, 2002

## Indiana Tier 2http://tier2.iu.edu/

**Indiana Tier 2****http://tier2.iu.edu/**

- Rob Gardner        US Grid Projects

- June 20, 2002

## Boston Tier 2

- Rob Gardner        US Grid Projects

- June 20, 2002

## Tier 2 Monitoringhttp://atlas.uits.iupui.edu/ganglia/index.php

- Rob Gardner        US Grid Projects

- June 20, 2002

## Virtualize ATLAS Production

- Track event data histories
- Virtual data:
- Transformation
- Executable program
- Athena service
- Derivation
- Execution of a transformation
- Data Object
- Named entity consumed by derivation
- LFN, strings
- Future: relational tables, persistent objects

- Generator

- HEPMC

- Hits

- Pileup Gen

- Merged Hits

- Simulation

- Combiner

- p-Hits

- Digitizer

- ROD Input

- ROD emulator

- Raw data
- Objects

- Reconstruction

- Event Data
- Objects

- pythia.exe

- atlsim.exe

- Athena
-  services

- Input cards

- Rob Gardner        US Grid Projects

- June 20, 2002

## Athena

- iSite

- MAGDA-GDMP

- Virtual Data Catalog

- Virtual Data Portal
- : Browser
- : Executor
- : Follower

- Abstract Planner

- Collection Catalog

- Replica  Catalog

- Concrete Planner

- Replica Mgmt.
- Services

- Executor
- :DAGMan

- VDI

- VDL

- SE

- Persistency
- Service

- Athena Grid  Execution Wrappers

- Algorithm Mgr.
- Services

- Histogram,
- Monitoring, ...
- Services

- Event Selector
- Services

- Globus GRAM, GridFTP, ...

- Core Athena Scripting Services

- Condor Pool

- Monitoring

- CAS, SAS
- Policy

- Virtual Data System

- Rob Gardner        US Grid Projects

- June 20, 2002

## Virtual Data Tools

- Virtual Data API
- A Java class hierarchy to represent transformations and derivations
- Virtual Data Language
- Textual for illustrative examples
- XML for machine-to-machine interfaces
- Virtual Data Database
- Makes the objects of a virtual data definition persistent
- Virtual Data Service
- Provides an OGSA interface to persistent objects

- Rob Gardner        US Grid Projects

- June 20, 2002

## VDT Status

- VDT 1.1.0 equivalent installed on most sites
- VDT 1.1.2 to be released after July 4
- Globus 2.0
- Condor 6.4.0
- Condor-G  6.3.2
- GDMP 3.0
- ClassAds 0.9.2
- Support has been set up
- vdt-support@ivdgl.org

- Rob Gardner        US Grid Projects

- June 20, 2002