# Presentation: 381753

## Software Status/Plans

- Torre Wenaus, BNL/CERN
- US ATLAS Software Manager
- US ATLAS PCAP Review
- November 14, 2002

## U.S. ATLAS Software Project Overview

- Control framework and architecture
- Chief Architect, principal development role, **ATLAS LCG applications area liaison**
- Databases and data management
- Database Leader, **primary ATLAS expertise on ROOT/relational baseline**
- Software support for development and analysis
- Software librarian, quality control, software development tools, training...
**Automated build/testing system adopted by Int’l ATLAS**
- Subsystem software roles complementing hardware responsibilities
**Muon system software coordinator**
- Scope commensurate with U.S. in ATLAS: ~20% of overall effort
**Commensurate representation on steering group**
**Strong role and participation in LCG common effort**

**Recent developments in green**

## U.S. ATLAS Software Organization

## U.S. ATLAS - ATLAS Coordination

_**US roles in Int’l ATLAS software:**_** **
**D. Quarrie (LBNL), Chief Architect**
**D. Malon (ANL), Database Coordinator**
**P. Nevski (BNL), Geant3 Simulation Coordinator, Simulation production lead**
**C. Tull (LBNL), EDG WP8 Liaison**
**H. Ma (BNL), Raw Data Coordinator**
**T. Wenaus (BNL), Planning Officer**

- US

- International

**See task matrix**

**ATLAS Subsystem/Task Matrix**

|  | Offline Coordinator | Reconstruction | Simulation | Database |
| --- | --- | --- | --- | --- |
| Chair | N. McCubbin | D. Rousseau | A. Dell’Acqua | D. Malon |
| Inner Detector | D. Barberis | D. Rousseau | F. Luehring | S. Bentvelsen /
D. Calvet |
| Liquid Argon | J. Collot | S. Rajagopalan | M. Leltchouk | H. Ma |
| Tile Calorimeter | A. Solodkov | F. Merritt | V.Tsulaya | T. LeCompte |
| Muon | J.Shank | J.F. Laporte | A. Rimoldi | S. Goldfarb |
| LVL 2 Trigger/ Trigger DAQ | S. George | S. Tapprogge | M. Weilers | A. Amorim /
F. Touchard |
| Event Filter | V. Vercesi | F. Touchard |  |  |

- Computing Steering Group members/attendees: 4 of 19 from US (Malon, Quarrie, Shank, Wenaus)

**Physics Coordinator: F.Gianotti **
**     ****Chief Architect: ****D.Quarrie**

## Project Planning Status

- U.S./Int’l ATLAS WBS/PBS and schedule fully unified
- US/Int’l software planning covered by same person (TW)
- Synergies outweigh the added burden of the ATLAS Planning Officer role
- No ‘coordination layer’ between US and Int’l ATLAS planning
- Possible because of how the ATLAS Planning Officer role is currently scoped
- As pointed out by an informal ATLAS computing review in March, ATLAS would benefit from a **full FTE** Planning Officer
- I have a standing offer to the Computing Coordinator: to step aside if/when a capable person with more time is found
- Until then, I scope the job to what I have time for and what is highest priority

## ATLAS Computing Planning

- US led a comprehensive review and update of ATLAS computing schedule in the spring
- Milestone count increased by 50% to 600; many others updated
- Milestones and planning coordinated around DC schedule
- Reasonably comprehensive and detailed through 2002
- New round underway now to flesh out 2003 schedule
- US core activity scheduling in reasonable shape
- Long term milestones reworked to reflect LHC schedule, LCG
- Centered around escalating data challenges
- Weak decision making, still a problem, translates to (among other things) weak planning
- Strong recommendation of the March review to fix this
- Should be fixed in the present computing management reorganization

## Summary Major Milestones

- Green: Done      Gray: Original date      Blue: Current date

## Major Milestones

***One DC per year until startup***

## Data Challenge 1

- DC1 phase 1 (simu production for HLT TDR) executed successfully
- World-wide operation
- Phase 2, starting in the next weeks, focuses on testing of new software
- Introduction and testing of new event data model
- Intensive Geant4 usage
- Intensive Grid usage
- Data production for Physics and computing model studies
- First tests of computing model
- Analysis using Analysis Object Data (AOD)
- DC1 Phase 2 too early for LCG common persistency POOL production use
- POOL production releases in spring/summer 2003

## Software Support, Quality Control

- New releases are available in the US typically ~1-2 days after CERN
- Provided in AFS for use throughout the US
- Librarian receives help requests and queries from ~25 people in the US
- US-developed nightly build facility used throughout ATLAS
- Central tool in the day to day work of developers and the release process
- Recently expanded as framework for progressively integrating more quality control and testing
- Testing at component, package and application level
- Code checking to be integrated
- CERN support functions being transferred to new ATLAS librarian
- BNL-based nightlies recently resumed
- Much more stable build environment than CERN at the moment
- Use timely, robust nightlies to promote usage of the Tier 1 for development

## Software Support, Quality Control (2)

- Testing integrated into automated builds
- Unit tests, package tests, integration/system tests
- ATLAS has (finally!) established a dedicated support team (SIT) for software infrastructure, testing, release management etc.
- U.S. represented by the U.S. ATLAS librarian, an active team member
- SIT needs a dedicated leader (currently the rotating, and overloaded, release manager heads SIT)
- Provides a much needed context for U.S. support and QA efforts
**pacman** (Boston U) for remote software installation
- Adopted by grid projects for VDT, and a central tool in US grid testbed work

## Grid Software

- Software development within the ATLAS complements of the grid projects is being managed as an integral part of the software effort
- Grid software activities tightly integrated into ongoing core software program, for maximal relevance and return
- Grid project programs consistent with this have been developed
- And has been successful
- e.g. Distributed data manager tool (Magda) we developed was adopted ATLAS-wide for data management in the DCs
- Grid goals, schedules integrated with ATLAS (particularly DC) program
*However *we do suffer some program distortion
*e.g. *we have to limit effort on providing ATLAS with event storage capability in order to do work on longer-range, higher-level distributed data management services

## Effort Level Changes

- ANL/Chicago – loss of .5 FTE in DB
- Ed Frank departure; no resources to replace
- Another .5 FTE recently lost, will be replaced
- BNL – cancelled 1 FTE new hire in data management
- Insufficient funding in the project and the base program to sustain the bare-bones plan
- Results in transfer of DB effort to grid (PPDG) effort – because the latter pays the bills, even if it distorts our program towards lesser priorities
- LBNL – stable project-supported FTE count in architecture/framework
- But loss of base support is threatening effort level and deliverables
- Grid funding being sought to ameliorate
- DB effort hard-hit, but ameliorated by common project
- Because the work is now in the context of a broad common project, US can still sustain our major role in ATLAS DB
- A material example of common effort translating into savings (even if we wouldn’t have chosen to structure the savings this way!)

## Personnel Priorities for FY02, FY03

- This is how we are doing relative to goals...
- Sustain LBNL (4.5FTE) and ANL (3FTE) support
- This we are doing so far.
- Add FY02, FY03 1FTE increments at BNL to reach 3FTEs
**Failed** in 02; BNL hire cancelled. Should recover to 3 FTEs in FY03
- Restore the .5FTE lost at UC to ANL
- No resources
- Establish sustained presence at CERN.
- No resources, despite being a very high priority
- We rely on labs to continue base program and other lab support to sustain existing complement of developers
***And needed base program support is not there.**** *Lab base programs are being hammered.

## General and longer term priorities

- These are reflected in the software request in the research program proposal (and go back as far as our original Jan 2000 project plan)
- Priorities in order:
- Sustain existing ANL, BNL, LBNL efforts
- Complete the ramp of the lab based core developer FTEs to the long-planned levels
- ANL 3.5 FTEs, LBNL 4.5 FTEs, BNL 4 FTEs
- Establish, over and above these lab levels, presence at CERN of at least 2 core developer FTEs
- In addition to any lab people who might be located at CERN
- Establish effort at the core-subsystem interface – sited mainly at universities and possibly CERN – to
- support the translation of core developments into established software employed by end users
- better support the leadership roles held by the US with developer effort capable of translating decisions into established solutions

## SW Funding Profile Comparisons

**2000 agency guideline**

**January 2000 PMP**

**11/2001 guideline**

**‘****Compromise profile’**
**requested in 2000**

**Mid 02 bare bones**

## Concluding Remarks

- No strategic changes; program is working, but stressed by funding
- US has consolidated the leading roles in our targeted core software areas
- Involved with new LCG common efforts in all our core areas
- Architecture/framework effort level being sustained so far
- And is delivering the baseline core software of ATLAS
- Database/data mgmt effort reduced but so far preserving our key technical expertise
- Leveraging that expertise for a strong role in common project
- Cannot tolerate further reduction in a key strategic US core area
- US major contributor to software infrastructure and QA in ATLAS
- Recent emphasis: improve QA, and make the US development and production environment as effective as possible
- Soft support from the project and base programs while the emphasis on grids grows is distorting our program in a troubling way