# Presentation: 009837

## Architecture and Control Framework

**David R. Quarrie**
**Lawrence Berkeley National Laboratory**
**DRQuarrie@LBL.GOV**
- DOE/NSF Review of U.S. ATLAS and CMS Computing Projects
- Brookhaven National Laboratory
- NOVEMBER 14-17, 2000

## Overview

- Progress since May
- Milestones and deliverables
- Architecture Review Status
- Ongoing activities
- December Milestone deliverables
- Future Milestones
- Personnel
- Summary
- Additional Background Material

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Some Background

- Definitions
- Architecture: General structure of a software system, comprising components, their properties and the relationships between them
- Framework: Concrete realization of an Architecture
- ATLAS Control Framework (Athena)
- Spans simulation, reconstruction, event filter, physics analysis
- Prototype based upon GAUDI Framework from LHCb
- Parallel Unified Software Development Process (USDP)
- Reverse engineer GAUDI
- Forwards engineering from user requirements
- Identify and work on missing functionality
- Significant ATLAS-developed functionality already in place
- Good collaboration with LHCb

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## May 2000 Prototype Milestone

- Essential - feature set _must_ include:
- Read Physics TDR data ****
- Execute sequence of multiple user modules ****
- Desirable - feature set _should_ include:
- Generate Event Display ****
- Dynamic loading of user modules ****
- Sequences with branches/filters ****
- Optional - feature set _may_ include:
- Rudimentary interactive user interface **
- Limited data persistence - HBOOK only ****
- Additional items not part of formal deliverables
- Event Generator framework prototype (LBNL) ****
- Liquid Argon reconstruction (BNL) ****
- Xkalman++ Inner Detector Tracking (LBNL) ****
- Two Fast Simulation efforts begun (UCL, CERN)

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Feedback from May Milestone

- Identified deficiencies in access to event data
- Event Data Model Workshop on 31st May
- Discuss issues and requirements
- Second Event Data Model Workshop on 11th July
- Present design
- Third Event Data Model Workshop on 23rd October
- Feedback from prototype and start 2nd design iteration
- Detector Description Workshop on 28th August
- Requirements for alignments, calibrations *etc*. from simulation & reconstruction *etc*.
- Review deliverables for September Milestone
- Review scheduled for 10th July
- Architecture Review Committee (ARC) formed
- Assign tasks based on updated deliverables & feedback

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Tutorials & Workshops

- Tutorials
- Two in parallel at LBNL for May milestone on 9th May (35 attendees)
- At CERN on 31st May (18 attendees)
- At CERN on 6th July (17 attendees)
- Good attendance and feedback
- Intention to repeat at regular intervals with updated contents
- Architecture Workshops
- 9-11 March 2000
- Plans and deliverables
- 17-19 April 2000
- Focus on user requirements
- 23rd Jan 2001
- Progress to date
- Identify missing functionality
- Revisit deliverables & milestones
- Encourage feedback from developers & users

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Architecture Review Committee

- Mandate
- Review the approach taken by the A-Team
- Conduct a design review
- Review priorities for milestones and deliverables
- Comment on database milestones and deliverables
- Membership
- Daniel Froidevaux (chair)	 	 Marzio Nessi
- Dario Barberis			 Pavel Nevski
- Chris Bee 			 Alan Poppleton
- Laurent Chevalier 		 Rene Brun (ALICE)
- Stephen Haywood		 Jim Kowalkowski (D0)
- Julius Hrivnac 			 David Stickland (CMS)
- Fred Luehring

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## ARC Meetings

- 10th July
- 24th August
- 25th September
- 6th-7th December
- 5th-7th March 2001
- Ongoing, detailed review
- First phase of design review completed
- Useful design feedback
*E.g.* Accelerated development of a data dictionary
- Goal is significant feedback from users as well as developers
- Hence protracted duration
- Concern about change of priorities and new deliverables as a result of ARC “real-time” feedback

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## September 2000 Milestone (1)

- Essential
- Prototype/USDP Merged [US] **
- Athena in ATLAS release [US] ****
- Event Data Model prototype [US] ****
- Access to transient event store from Algorithm
- Interactive Scripting prototype [US] ****
- Event Generator framework deployed [US] ****
- Fast Simulation prototype ****
- Wrapped FORTRAN Fast Simulation prototype ****
- Physics Analysis output to ROOT prototype [US] ****
- Ability to output histograms & ntuples to ROOT files
- [US] US Deliverables
- Prototype  Available to developers
- Deployed  Available to end users in ATLAS software release

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## September 2000 Milestone (2)

- Essential (continued)
- TileCal testbeam input from Objectivity [DB]
- G3 Digits read in from Objectivity [DB]
- Objectivity event I/O prototype [DB]
- Demonstration of ability to read & write event data (HepMC)
- ROOT event I/O prototype [DB]
- Demonstration of ability to read & write event data (HepMC)
- Desirable
- Auditors prototype [US] ****
- Cpu/memory monitors
- Physics Analysis binding to ROOT prototype [US] **
- Ability to switch to ROOT & back directly within Athena
- Partial ATLFAST event data I/O to Objectivity & ROOT [DB]
- [DB] deliverables not discussed here

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Summary of September Milestone

- Most deliverables met
- Typically missed by less than 2 weeks
- Scripting delivered but still being integrated
- Physics Analysis binding to ROOT was major missed deliverable
- Athena in ATLAS release 1.2.0 of 26th Oct 2000
- Dual Gaudi releases available
- Vanilla Gaudi 5
- Used for ATLAS release 1.2.0
- “Gaudi 5.5” release incorporating our code changes
- Used for ATLAS release 1.2.1
- ATLAS developed changes and extensions in Gaudi 6
- Released on 10th November 2000
- Joint LHCb/ATLAS release
- To be used for ATLAS release 1.2.2

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Developments not in Formal Milestone

- Improved Remote Site Support [US]
- Goal is easy installation on a laptop with no network access
- Solaris 7 support [US]
- Not yet an officially supported ATLAS platform
- Improved Sequencer for Event Filter [US]
- Explicit branching and inverted filter logic
- Data Dictionary [US]
- Automatic generation of converters & persistent classes
- Transient Store Data Browser
- Multi-language support
- Improved Service Management [US]
- Gaudi internals (USDP feedback)
- Improved documentation [US]

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Other Developments

- Many ongoing developments based on Athena
- Event Filter
- Fast simulations
- Level 2
- Liquid Argon
- Muon
- TileCal
- Xkalman++
*Etc.*
- Active developer community
- Support
- Mailing list (atlas-sw-architecture)* *
- Craig Tull (LBNL) at CERN for 12 months

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## December 2000 Milestone

- Version supporting the Trigger TDR
- Essential
- ATRECON equivalent reconstruction prototype
- Reconstruction deliverable
- Pile-up support prototype [US]
- Event Data Model deployed [US]
- ATLAS release 1.2.1 on 10th Nov 2000 ****
- Detector Description (Calibration/Alignment) prototype
- Interactive Scripting deployed [US]
- Note late delivery of prototype
- Fast Simulation deployed (but not validated)
- Physics deliverable

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## December 2000 Milestone (2)

- Desirable
- Auditors deployed [US]
- ATLAS release 1.2.1 on 10th Nov 2000 ****
- Particle Properties Service prototype [US]
- In principle from CLHEP, but concerns about timescale
- Physics Analysis output/binding to ROOT deployed [US]
- ATLAS release 1.2.1 on 10th Nov 2000 ****
- Physics Analysis output/binding to JAS prototype
- Objectivity event I/O partially deployed
- Solaris 7 support? [US]
- Driven by ATLAS Computing schedule

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## April 2001 Milestone

- Essential
- Geant4 integration prototype
- Objectivity Event I/O fully deployed
- Desirable
- Particle Properties Service deployed [US]
- Detector Description partially deployed
- Physics Analysis output/binding to JAS deployed
- Visualization prototype
- Algorithm based already exists (May 2000 prototype)
- Full interactive service
- Statistics and Monitoring prototype [US]
- Bookkeeping prototype [US]
- Java/C++ support? [US]

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## September 2001 Milestone

- Ready for Mock Data Challenge 0
- Continuity test
- Essential
- Geant4 integration deployed (but not validated)
- Detector Description deployed
- Visualization deployed
- Statistics and Monitoring Tools deployed [US]
- Desirable
- Bookkeeping deployed [US]
- Full multi-language support (services & algorithms in Java)? [US]

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## December 2001 Milestone

- First “production” version
- Ready for Mock Data Challenge 1
- Essential
- Bookkeeping deployed [US]
- Desirable
- GRID enabled prototype [US]

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Longer Term Milestones

- Driven by overall Computing Schedule
- Continue model of Athena releases every 3-4 months
- Mock Data Challenge I
- Production starts Dec 2001
- Computing TDR
- Writing starts Jun 2002
- Mock Data Challenge II
- Production starts Nov 2002
- Physics Readiness Document
- Writing starts Feb 2004
- Start commissioning
- Jul 2005

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Software Engineers (LBNL) I

- Paolo Calafiura (50%)
- Framework (general) (WBS 2.2.1.2) 25%
- EDM (WBS 2.2.1.4) 25%
- Chris Day
- USDP (WBS 2.2.1.1.3) 50%
- Framework (scripting) (WBS 2.2.1.2.4) 50%
- Charles Leggett (80%)
- Framework (ROOT I/O) (WBS 2.2.1.2.10/11) 80%
- Massimo Marino
- Framework (re-engineering) (WBS 2.2.1.2) 75%
- EDM (WBS 2.2.1.4) 25%

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Software Engineers (LBNL) II

- David Quarrie (80%)
- Architecture (WBS 2.2.1.1) 50%
- Framework (general) (WBS 2.2.1.2) 30%
- Craig Tull
- Framework (scripting, data dictionary) (WBS 2.2.1.2) 90%
- Framework (GRID) (WBS 2.2.1.2.17) 10%
- FTEs
- Framework (3.6), EDM (0.5), USDP (0.5), Architecture (0.5)
- FY01 Funding
- US-ATLAS 4.3 FTE
- LBNL 0.8 FTE
- Shortfall of 0.5 FTE in FY02

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Summary

- Met most of the September Milestone Deliverables
- Typically missed by <2 weeks
- All apart from Physics Analysis to ROOT binding delivered by mid-November
- Intense Architecture Review underway
- Very active developer community
- Trying to encourage user involvement
- Trying to maintain the momentum since May
- Athena has demonstrated support for simulation, event filter, reconstruction, physics analysis
- Much work remains to convert prototypes to production and increase the scope

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Background Material

- The following slides provide additional background material

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Prototype/USDP Merged [US]

- A-Team deliverable - Chris Day and Massimo Marino (LBNL)
- My definition of deliverable
- Reverse engineer GAUDI
- Done for Gaudi 5
- Forwards engineering from user requirements
- All available user input entered and procedures setup to accept more
- Creation of formal requirements from user input still incomplete
- Identify and work on missing functionality
- Done for scripting and sequencing (filtering & branching)
- Starting to re-engineer some of core design
- Rational product suite & web page
- http://roderick.lbl.gov/Atlas
- ARC considers this deliverable incomplete
- Nov 2000?

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Athena in an ATLAS Release [US]

- A-Team deliverable – Paolo Calafiura & David Quarrie (LBNL)
- Everything in place
- Gaudi 5 available as external package
- Developers Gaudi installation installed in public (project) area
- Only used by beta-testers of prototypes
- GaudiInterface SRT interface package
- Several Athena examples & common executable
- CVS tags are available
- Includes dynamic loading functionality
- Minimal common executable
- Run-time dynamic loading on demand
- In ATLAS software release 1.2.0 of 26th Oct 2000
- Milestone met on 4th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Event Data Model Prototype [US]

- Paolo Calafiura (LBNL), Srini Rajagopalan (BNL) et al.
- Framework level support for EDM prototype
- Access to data from Algorithms
- Base classes, smart pointers/handles
- Keys & selectors, etc.
- StoreGate SRT package available
- Design documentation available
- One change to Gaudi/Athena base code required
- Generalized service accessor for Algorithm
- Incorporated into a developers Gaudi/Athena installation
- Milestone met on 4th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Interactive Scripting Prototype [US]

- Chris Day  & Craig Tull (LBNL)
- Requirements based on USDP feedback
- Prototype functional since 24th October
- Based on PYTHON
- Best choice
- Abstractions support alternative scripting languages
- Single implementation planned
- Significant integration issues still being tackled
- Extensions to base classes
- Integration completed by mid November 2000
- Available in updated developers release

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Event Generator Framework Deployed [US]

- Ian Hinchliffe & Marge Shapiro (LBNL)
- Several generators available
- Isajet, Pythia, *etc*.
- Herwig underway - ready by end of October
- Generator to use selectable at run-time
- Properties of generator modifiable at run-time
- SRT packages and tags in place
- Dependent on *Athena in an ATLAS release* deliverable
- In ATLAS Release 1.2.0 of 26th Oct 2000
- Milestone met on 4th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Fast Simulation Prototype

- UCL (Peter Clarke et al.) deliverable
- SRT Packages all available
- Documentation available
- Generation of histograms demonstrated
- Compatibility with Generator framework demonstrated
- Physics validation started
- Testing against EDM (StoreGate) prototype started
- 2nd generation design following experience and review
- Milestone met on 4th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Wrapped FORTRAN Fast Simulation Prototype

- Elzbieta Richter-Was (CERN)
- Two variants delivered
- Wrapped FORTRAN
- ATLFAST++ (C++)
- Primary motivation is validation of new OO design
- SRT Packages available and tagged
- Timing studies available
- Documentation available
- Histogramming & Ntuples demonstrated
- Milestone met on 15th August 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Physics Analysis Output to ROOT Prototype [US]

- Charles Leggett (LBNL)
- Output of histograms and n-tuples to ROOT files
- May prototype only supported HBOOK files
- Prototype design and implementation complete
- Extension to Gaudi base code
- Changes incorporated into Gaudi CVS repository and developer installation
- Ntuple API limitations overcome
- Mapping of variables into blocks/trees
- Available 19th October
- Milestone met on 13th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Auditors Prototype [US]

- Marge Shapiro & David Quarrie (LBNL)
- Algorithm monitoring (before/after execution *etc*.)
- CPU usage and memory usage *etc*.
- All code incorporated into Gaudi CVS repository and into developer Gaudi installation
- No SRT tags since Gaudi base functionality
- Three concrete auditors delivered
- Print Name
- CPU Time
- Memory growth
- Draft documentation fed back to Gaudi User Guide
- Milestone met on 4th October 2000

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Physics Analysis Binding to ROOT Prototype [US]

- Charles Leggett (LBNL)
- Ability to switch at run-time between Athena/Gaudi & ROOT and view histograms & n-tuples
- Not yet available
- Physics Analysis Output to ROOT Prototype took longer than expected
- Milestone not yet met
- Desirable milestone
- Readjustment of priorities
- Raise ATLFAST Event I/O to ROOT above this

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Improved External Site Support [US]

- Iwona Sakrejda (LBNL) working on developing procedures
- Fred Luehring (Indiana) acting as guinea pig
- Site-specific configurations being fed back into Gaudi CVS repository
- The ATLAS GaudiInterface package has been updated to support site-specific locations
- Goal is to demonstrate stand-alone (*i.e.* network independent) laptop installation
- Currently feasible, but requires too much manual labor
- E.g. Modifications to Gaudi & ATLAS packages

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Solaris 7 Port [US]

- Early look at Gaudi/Athena and Solaris CC 6.0
- Not yet clear when ATLAS will require this as officially supported platform
- End of this year?
- Essential Kernel packages compile (with some problems) but don’t yet link
- Changes being fed back into the CVS repository
- Very low priority since we don’t have a schedule for ATLAS
- Good exercise since it’s pointed out a few inconsistencies in the core code base

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Sequencer [US]

- Enhanced sequencer supports explicit branching as well as filtering
- Sequencer.Members = { “First”, “Second”, “Third” };
- Sequencer.BranchMembers = { “Second”, “Forth” };
- Algorithm “Third” will be executed if event passes the “Second” filter, otherwise Algorithm “Forth” will be executed.
- Inverted logic
- Sequencer1.Members = { “First”, “Second”, “Third” };
- Sequencer2.Members = { “First”, “Second:invert”, “Forth”};
- Goal is better support for Event Filter
- Requirements based on USDP feedback

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Data Dictionary [US]

- Automatic generation of code for:
- Generation of persistent classes and converters
- Browsing of transient event store
- Multiple language support
- Relevant experience within ATLAS
- Priority raised as a result of feedback from ARC
- LHCb also interested in this topic
- Previous experience within US-ATLAS
- Paolo Calafiura (CDF), David Quarrie (BaBar), Craig Tull (STAR)
- Working group being formed
- Annecy group interested in this area

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000

## Gaudi Service Management [US]

- Feedback from USDP
- Reverse engineering of Gaudi 5
- Gaudi service management and event loop are too tightly coupled
- ApplicationMgr handles both
- Services are instantiated even if they aren’t needed
- Not easy to override default implementations
- Massimo Marino (LBNL) reworking this area
- Did reverse engineering last FY while funded by LBNL
- Also looking into reducing couplings

- David Quarrie       Architecture & Control Framework

- November  14-17, 2000