# Presentation: 476625

## ATLAS Data Challenges

- US ATLAS Physics & Computing
- ANL
- October 30th 2001
- Gilbert Poulard
- CERN EP-ATC

## Outline

- ATLAS Data Challenges & “LHC Computing Grid Project”
- Goals
- Scenarios
- Organization

- ATLAS Plenary-18 October 2001

## From CERN Computing Review

- CERN Computing Review (December 1999 - February 2001)
- Recommendations:
- organize the computing for the LHC era
- LHC Grid project
- Phase 1: Development & prototyping (2001-2004)
- Phase 2: Installation of the 1st production system (2005-2007)
- Software & Computing Committee (SC2)
- Project accepted by the CERN council (20 September)
- Ask the experiments to validate their Computing model by iterating on a set of Data Challenges of increasing complexity
- However DC’s were in our plans

- ATLAS Plenary-18 October 2001

## LHC Computing GRID projectPhase1:

-  Prototype construction
- develop Grid middleware
- acquire experience with high-speed wide-area network
- develop model for distributed analysis
- adapt LHC applications
- deploy a prototype (CERN+Tier1+Tier2)
- Software
- complete the development of the 1st version of the physics application and enable them for the distributed grid model
- develop & support common libraries, tools & frameworks
- including simulation, analysis, data management, ...
- in parallel LHC collaborations must develop and deploy the first version of their core software

- ATLAS Plenary-18 October 2001

## ATLAS Data challenges

- Goal
-  understand and validate:
- our computing model, our data model and our software
- our technology choices
- How?
- In iterating on a set of DCs of increasing complexity
- Start with data which looks like real data
- Run the filtering and reconstruction chain
- Store the output data into our database
- Run the analysis
- Produce physics results
- To study
- Performances issues, database technologies, analysis scenarios, ...
- To identify
- weaknesses, bottle necks, etc... (but also good points)

- ATLAS Plenary-18 October 2001

## ATLAS Data challenges

- But:
- Today we don’t have ‘real data’
- Needs to produce ‘simulated data’ first
- so:
- Physics Event generation
- Simulation
- Pile-up
- Detector response
- Plus reconstruction and analysis
- 	will be part of the first Data Challenges
-  we need also to “satisfy” the ATLAS communities
- HLT, Physics groups, ...

- ATLAS Plenary-18 October 2001

## ATLAS Data Challenges: DC0

- DC0 November-December 2001
- 'continuity' test through the software chain
- aim is primarily to check the state of readiness for DC1
- We plan ~100k Z+jet events, or similar
- To validate the software:
-  issues to be checked include
-  G3 simulation running with the ‘latest’ version of the geometry
- reconstruction running
- Re-analyze part of the Physics TDR data “reading from & writing to Objectivity”
- Would test the “Objy database infrastructure”
- Complementary to the “continuity test”

- ATLAS Plenary-18 October 2001

## ATLAS Data Challenges: DC1

- DC1 February-July 2002
- reconstruction & analysis on a large scale
- learn about data model; I/O performances; identify bottle necks ...
- use of GRID as and when possible and appropriate
- data management
- Use (evaluate) more than one database technology (Objectivity and ROOT I/O)
- Learn about distributed analysis
- should involve CERN & outside-CERN sites
- site planning is going on, an incomplete list already includes sites from Canada, France, Italy, Japan, UK, US, Russia
- scale 10**7** events in 10-20 days, O(1000) PC’s
- data needed by HLT & Physics groups (others?)
- Study performance of Athena and algorithms for use in HLT
- simulation & pile-up will play an important role
- checking of Geant4 versus Geant3

- ATLAS Plenary-18 October 2001

## ATLAS Data Challenges: DC1

- DC1 will have two distinct phases
- First, production of events for HLT TDR, where the primary concern is delivery of events to HLT community;
- Second, testing of software (G4, dBases, detector description,etc.) with delivery of events for physics studies
- Software will change between these two phases
- Simulation & pile-up will be of great importance
- strategy to be defined (I/O rate, number of “event” servers?)
- As we want to do it ‘world-wide’ we will ‘port’ our software to the GRID environment and use as much as possible the GRID middleware (ATLAS kit to be prepared)

- ATLAS Plenary-18 October 2001

## ATLAS Data Challenges: DC2

- DC2 Spring-Autumn 2003
- Scope will depend on what has and has not been achieved in DC0 & DC1
- At this stage the goal includes:
- Use of ‘TestBed’ which will be built in the context of the Phase 1 of the “LHC Computing Grid Project”
- Scale at a sample of 10**8** events
- System at a complexity ~50% of 2006-2007 system
- Extensive use of the GRID middleware
- Geant4 should play a major role
- Physics samples could(should) have ‘hidden’ new physics
- Calibration and alignment procedures should be tested
- May be to be synchronized with “Grid” developments

- ATLAS Plenary-18 October 2001

## DC scenario

- Production Chain:
- Event generation
- Detector Simulation
- Pile-up
- Detectors responses
- Reconstruction
- Analysis
- These steps should be as independent as possible

- ATLAS Plenary-18 October 2001

## Production stream

- “OO-db”  is used for “OO database”, it could be Objectivity, ROOT/IO, ...

- ATLAS Plenary-18 October 2001

## Slide 13

- ATLAS Plenary-18 October 2001

## Detector Simulation

- Geant3 and Geant4
- For HLT & physics studies we will use Geant3
- Continuity with past studies
- ATLAS validation of Geant4 is proceeding well but not completed
- Detector simulation in Atlsim (Zebra output)
- Some production with Geant4 too
- Goals to be defined with G4 and Physics groups
- It is important to get experience with ‘large production’ as part of G4 validation
- it is important to use the same geometry input
- In the early stage we could decide to use only part of the detector
- it would also be good to use the same sample of generated events
- Detector simulation we propose to use the FADS/Goofy framework
- Output will be ‘Hits collections’ in OO-db
- Detector responses (& pileup) has to be worked on in new framework

- ATLAS Plenary-18 October 2001

## Reconstruction

- we want to use the ‘new reconstruction’ code being run in ***Athena*** framework
- Input should be from OO-db
- Output in OO-db:
- ESD (event summary data)
- AOD (analysis object data)
- TAG (event tag)

- ATLAS Plenary-18 October 2001

## Analysis

- We are just starting to work on this but
- Analysis tools evaluation should be part of the DC
- It will be a good test of the Event Data Model
- Performance issues should be evaluated
- Analysis scenario
- It is important to know the number of analysis groups, the number of physicists per group, the number of people who want to access the data at the same time
- It is of ‘first’ importance to ‘design’ the analysis environment
- to measure the response time
- to identify the bottle necks
- for that users’ input is needed

- ATLAS Plenary-18 October 2001

## Data management

- It is a ***key***  issue
- Evaluation of more than one technology is part of  DC1
- Infrastructure has to be put in place:
- For Objectivity & ROOT I/O
- Software, hardware,  tools to manage the data
- creation, replication, distribution
- discussed in database workshop
- Tools are needed to run the production
- “bookkeeping” , “cataloguing” ...
- Run number, random number allocation, ...
- Working group now set-up
- Job submission
- Close collaboration with ATLAS Grid (validation of Release 1)

- ATLAS Plenary-18 October 2001

## DC1-HLT - CPU 

|  | Number of events | Time per event sec SI95 | Total time 
Sec 
SI95 | Total time
Hours
SI95 |
| --- | --- | --- | --- | --- |
| 
simulation | 
107 | 
3000 | 
3 * 1010 | 
107 |
| 
reconstruction | 
107 | 
 640 | 
6.4 * 109 | 
2 * 106 |

- ATLAS Plenary-18 October 2001

## DC1-HLT - data

|  | Number of events | Event size
MB | Total size 
GB | Total size
TB |
| --- | --- | --- | --- | --- |
| 
simulation | 
107 | 
   2 | 
 20000  | 
 20 |
| 
reconstruction | 
107 | 
  0.5 | 
  5000 | 
   5 |

- ATLAS Plenary-18 October 2001

## DC1-HLT data with pile-up

| L | Number of events | Event size MB | Total size GB | Total size 
TB |
| --- | --- | --- | --- | --- |
| 2 x 1033 | 1.5 x 106 | (1)  2.6
(2)  4.7  | 4000
7000 | 4
7 |
| 1034 | 1.5 x 106
 | (1)  6.5
(2) 17.5 | 10000
26000 | 10
26 |

- In addition to  ‘simulated’ data, assuming ‘filtering’ after simulation  (~14% of the events kept).
-  **(1)** keeping only ‘digits’
-  **(2)** keeping ‘digits’ and ‘hits’

- ATLAS Plenary-18 October 2001

## Ramp-up scenario @ CERN

**Week in 2002**

- ATLAS Plenary-18 October 2001

## What next

- This week:
- Have an updated list of goals & requirements prepared with
-  HLT, Physics communities
-  simulation, reconstruction, database communities
-  people working on ‘infrastructure’ activities
- bookkeeping, cataloguing, ...
- Have a list of tasks
- Some Physics oriented
- But also like testing code, running production, ...
***with ‘established’ responsibilities ***and priorities
- And working groups in place
- You can join

- ATLAS Plenary-18 October 2001

## What next

-  In parallel
- Define ATLAS validation plan for
- EU-DataGrid Release 1
- ATLAS software for DC0 and DC1
- Understand the involvement of Tier centers
- Insure that we have the necessary resources
- @ CERN and  outside CERN
- We have already some input and a ‘table’ is being prepared
***“******And turn the key”***

- ATLAS Plenary-18 October 2001