# Presentation: 530017

## Scientific Data Management:An Incomplete Experimental HENP Perspective

- D. Olson, LBNL
- 26 March 2002SDM-ISIC MeetingGatlinburg

## Particle Physics Data Grid

- www.ppdg.net

- PI’s:
- Mount,
- Livny,
- Newman

- Coordinators:Pordes,Olson

## Contents

- Quick overview of HENP data
- Generic data flow
- Sizes, timescales
- Average physicist view
- What’s hard
- Making technology work in production
- A clear view for average physicist
- Analysis of large datasets
- Other things as well
- Today, many issues wrapped in hopes for “Data Grid”

## Experimental HENP event data

- Basic character of data is “event”
- May be few particles

## BaBar event

- http://www.slac.stanford.edu/BFROOT/

## Experimental HENP event data

- Basic character of data is “event”
- May be few particles
- May be MANY particles

## STAR event, Au + Au

- www.star.bnl.gov

## Experimental HENP event data

- Basic character of data is “event”
- May be few tracks
- May be MANY tracks
- Detector characteristics, beam types, triggers effect the type of events recorded
- Physics analysis is a statistical analysis of many (1000’s, M’s, B’s, T’s) independent events

## Generic data flow in HENP

- “Skims”,
- “microDST production”, ...

- Filtering chosen to make this a convenient size

## A collaboration of people

- $100M, 10 yr, 100 people

- Free?, 10 yr, 20 people

- Free?, 1 yr, 10 people, 5x/yr

- Free?, 1 mo, 1 person, 50x/yr

- (“Typical” example today, LHC is larger)

## Example: CMS Tiers

## List of major accelerator-based HENP experiments

| Experiment | Location | # physicists | Time scale |
| --- | --- | --- | --- |
| BaBar | SLAC | 800 | 1999 - 2010 |
| STAR | BNL / RHIC | 450 | 2000 - 2010 |
| PHENIX | BNL / RHIC | 450 | 2000 - 2010 |
| Jlab/CLAS | JLAB | 200 | 2000 - 2010 |
| CDF | FNAL | 800 | 1995 - 2010 |
| D0 | FNAL | 800 | 1995 - 2010 |
| ATLAS | CERN | 2000 | 2006 - 2016 |
| CMS | CERN | 2000 | 2006 - 2016 |
| ALICE | CERN | 1200 | 2007 - 2017 |
| Jlab Hall D | JLAB | 200 | 2008 - 2018 |

## Size / frequency of basic activities

| Item | Size (TB) / Frequency (/yr) |  |
| --- | --- | --- |
|  | Typical today
 | LHC era (>5 yr) |
| Raw data
 | 100 TB / yr | 1,000 TB / yr |
| Event Reconstruction | 3 / yr | 2 / yr |
| DST data | 1 > DST/ raw > 0.1 | 0.1 > DST/ raw > 0.02 |
| microDST production | 0.1 > microDST/DST > .001 | ? |
| Physics analysis | 10 - 100 * #physicists / year | ? |

## Average physicist view

- Mythology, culture, terminology varies a lot from one experiment to another.
- BaBar
- Object view or primary event store (Objectivity)
- Event collection objects give primary access points to data
- Event collection has list of references to all event components of interest
- With 100,000 collections, how to organize them?
- Ntuples & PAW for final data format, analysis tool
- STAR (first year data, getting started)
- A “production, trigger” is all reconstructed events for a trigger type with a certain version of code, (P00hg, central)
- Access point is list of directory path’s below which all data are stored on disk
- WZ will be setting up STACS
- ROOT for data format and analysis tool
- ...

## What’s hard I, living with technology

- Typical computer center today
- A couple STK Powderhorn tape silos, HPSS or home-grown MSS
- 1000 linux processors
- Assortment of 100/1000 Gbps network
- 50 TB disk (1000 spindles)
- Network s/w for I/O (NFS, Objy AMS, RFIO, ...)
- AFS for distributed collaboration
- Can make large RAID filesystems w/ network access
- Faults can affect many nodes
- stale NFS file handles
- AFS faults affects nodes across country, work
- Large RAID is $$$
- Desire to reduce effect of faults
- Fewer faults
- More tolerance
- ...

## What’s hard II,A clear view for average physicist

- What’s going on in this box?

## What’s hard II,A clear view for average physicist

- What data is available?
- “data” means
- List of files? (like STAR)
- Collection object w/ pointers to all events? (like BaBar)
- “available” means
- On disk? Where?
- Exists?
- Does it really have the filters and calibrations I need?
- Is it the “official” version of the data?
- ...

## What’s hard III,Analysis of large datasets

- Dataset does not fit on disk, or requires parallel processing, oris large enough operation that chance of fault is high

## What’s hard III,Analysis of large datasets

- Dataset does not fit on disk
- Needs access s/w to couple w/ processing
- SAM, STACS
- Does performance meet demand?

## SAM (Sequential data Access via Meta-data)

- http://d0db.fnal.gov/sam/

## STACS

- http://sdm.lbl.gov/projectindividual.php?ProjectID=STACS

## What’s hard III,Analysis of large datasets

- Dataset does not fit on disk
- Needs access s/w to couple w/ processing
- SAM, STACS
- Does performance meet demand?
- Needs parallel processing (not very hard)
- Can not do analysis on private/personal machine
- Schedule access to shared resource (CPU and disk)
- Operation for a single analysis is large enough that faults occur
- Need exception handling
- Need workflow management to complete failed tasks or, at least, accurately report status

## Example shared nothing cluster

- http://www.ihep.ac.cn/~chep01/paper/4-026.pdf

## PPDG

## Summary

- Faulty technology sets boundary conditions
- Fault tolerant will expand boundaries of capabilities
- Data management is coupled with processing
- Visualization (access w/o processing) is minor in HENP
- Need access to data when & where it is needed for processing
- Working on data grid as context for data management
- PPDG has SDM ISIC as one of the technology base projects