**[4. WBS 2.2 Software Manager's Report (Torre Wenaus,
BNL)]{.underline}**

Software project efforts in the quarter have continued to focus on
infrastructure and operations development and support for Data Challenge
1 -- now into its second phase. Other activities were associated with
contributions the LCG project, in which US ATLAS is an important
contributor to software design and development in the LCG Applications
Area. The POOL persistency project of the LCG delivered its first public
prototype in December, and work on its integration into ATLAS had begun
by the end of the period. US ATLAS activities were well represented at
SuperComputing 2003 the culmination of a large preparatory effort.

### 

### WBS 2.2.1.1, 2.2.1.2, 2.2.1.4 Framework, Architecture and Event Model 

This reporting quarter saw a continuation and expansion of the
collaborative nature of our core software development efforts. Tight
collaboration with LHCb on the Gaudi Kernel while new collaborative
efforts in the context of the LCG project began. US developers began
participating in the SEAL core libraries and services project, and the
ATLAS-specific object definition language was discontinued in favor of a
common approach with CMS pursued in the context of the LCG. Promising
results were obtained in the first performance tests of event pile-up
within the Athena framework.

Gaudi/Athena: While keeping in sync with the common Gaudi repository,
several Athena specific features have been introduced: AthenaKernel is a
new \"interface repository\" package similar to GaudiKernel. Two new
Athena-specific services have been implemented: a new random number
service, which allows for multiple independent random number streams,
and ClassIDSvc an in-memory DB of CLID, type name pairs

Scripting: The scripting service for Gaudi/Athena is moving forward on
two fronts: the binding to Gaudi objects (GaudiPython), and the UI
(AthASK). The former has been completely rewritten to make use of
boost.python v2.0 and to overcome bootstrap problems. To the latter were
added: a prompt, optimized and broadcast builds, log files, 1-click
installation, and balloon-help. Further, based on feedback from a
successful demo and tutorial, the UI internals were updated to improve
automation: package selection, recognizing (and handling) broken
releases, and (ld)path management.

IOVSvc: IOVSvc was created to automatically read and keep updated the
time dependent conditions data. We have created a fully functional
prototype that uses an ASCII file base hierarchy as a database, and
demonstrated it to work aligning modules in the Pixel detector.

ADL/Data Dictionary: Three working groups were set up by ATLAS software
management to evaluate the status of the ADL project and its performance
against a test suite of classes, to investigate the relationship with
the LCG projects in this area, and revisit the physics EDM to update the
requirements. This was motivated by the continuing delays in the
deployment of ADL as the basis for the descriptions of the event data
model. Working group results were presented and discussed in an ATLAS
Computing Steering Group meeting which came to the conclusion that the
ADL project should be terminated, and the alternative approach (also
used by CMS) of using C++ header files for object descriptions should be
adopted. The C++ header files will be parsed and their object
descriptions loaded into the LGC Dictionary. Another ad hoc working
group has followed up on this, the goal being to validate this strategy
and have a detailed strategy and work plan in place by the time of the
Detector Description and Event Data Model Workshop at the end of January
2003.

SEAL: The SEAL developers group of the LCG Applications Area started its
work. The initial focus has been to define the project work plan and to
design a prototype plugin manager. Massimo Marino of the US ATLAS core
software team is participating in this collaborative effort among the
experiments, with SEAL components targeted for eventual use in Athena.

Pileup: The first batch of memory performance studies, involving the
pileup of LAr hits (the most demanding subdetector) has been completed.
The results were very promising and showed how the overall scheme can be
run in a reasonably sized machine. The study identified a couple of
hot-spots for memory usage. This has triggered the development of
ClassIDSvc and the re-engineering of DataPtr (the SG ref-counted
pointer).

StoreGate/EDM support: The first version of the ProxyProvider scheme has
been implemented. It allows any service that implements the
IAddressProvider interface to either preload or provide upon request the
address information necessary to create the data object proxies in the
transient store. StoreGate has also been ported to gcc 3.2 to be ready
to interoperate with the LCG Pool libraries. As usual a lot of effort
has been put in improving StoreGate performance, To optimize memory and
disk usage a new container of ElementLinks, ElementLinkVector, has been
introduced. The StoreGate reference counted pointer (DataPtr) has been
redesigned to reduce its memory footprint. The introduction of
ClassIDSvc allowed to save about 400KB/store and at the same time to
(fractionally) improve the CPU time needed for a StoreGate record.

### WBS 2.2.1.3 Databases and Data Management 

In this reporting quarter, the U.S. ATLAS database group delivered the
first release of collection services for the LHC-wide POOL project,
which will provide a common persistence infrastructure for the LHC
experiments. The release supports creation of and iteration over
explicit collections instantiated in a relational database, with
references pointing to data residing in a ROOT-file-based storage layer.
Collections are queryable and filterable via SQL queries on associated
metadata. The work was included in the POOL end-of-year release 0.3.0.

The U.S. group also assumed responsibility for management of MySQL and
mysql++ packages for the POOL project, as well as for configuring and
managing MySQL servers for both ATLAS and LCG. The group introduced
support in MySQL for X.509 certificates and tested connectivity with DOE
grid certificates, important initial steps toward supporting grid-based
access to MySQL databases for LHC.

At an October database workshop in Orsay, the U.S. team organized (and
now leads) a small group of people with MySQL database administration
experience, charged with identifying long-term requirements and
near-term strategies for deployment of relational database services for
LHC. An outcome of this effort was a proposal for a multi-tier
deployment model, presented to an LHC-wide audience at CERN in November.

Work was begun on an Athena conversion service based upon the December
POOL release, and on extensions to the NOVA conversion service to allow
access by means of interval-of-validity services. This work is scheduled
to be incorporated into ATLAS software releases early in the next
reporting quarter.

The U.S. group provided support for AthenaROOT-based persistence, and
for the NOVA database infrastructure that provides access to the primary
numbers that parameterize the ATLAS geometry, updating the database to
support additional muon geometry data. Infrastructure to integrate
schema creation for the primary numbers database with builds of ATLAS
software releases was developed jointly by the Orsay and U.S. groups.

Dependencies upon the commercial database product Objectivity/DB were
removed from ATLAS software releases in this reporting quarter, as an
orderly transition from Objectivity/DB to POOL as the ATLAS baseline
persistence technology continued.

While virtual data concepts had been prototyped by the U.S. database
group in earlier reporting periods, this quarter marked the first time
that prototyping could be done based upon an official release of Chimera
from the U.S. grid project known as GriPhyN. Several proof-of-principle
tests were run; Virtual Data Language (VDL) files and test results were
provided as feedback to GriPhyN developers. Virtual data catalog
prototyping by the ATLAS database group was cited in the ATLAS-wide
newsletter

http://aenews.cern.ch/aenews.php?issueno=200209.

In addition to supporting ATLAS Data Challenge 1 production, the U.S.
group delivered infrastructure and support for demonstrations at the
Supercomputing 2002 conference in Baltimore, including demonstrations of
EDG and U.S. grid interoperability, of virtual data, and of portal-based
(GRAPPA) job submission and monitoring.

A prototype code using an embedded MySQL server was developed jointly
with W. Seligman (Nevis) for use in standalone liquid argon Geant4
simulations. The embedded server allows one to run the simulation on a
laptop, or, in general, on a machine that may be isolated from networked
database servers.

A meeting was organized at CERN in December to discuss coordination of
database activities between online, offline, and ATLAS Technical
Coordination. A workshop along these lines, with a principal focus on
conditions and other time-varying data, will be held at CERN in February
2003.

**WBS 2.2.1.10 Distributed Data Management and Processing Software**

As in the last quarter the usage of Magda in the US Atlas testbed DC1
production was continuously supported. Magda itself has been improved as
more feedback from the testbed developers came in. Magda was used to
transfer files between BNL HPSS and CERN Castor. Also, for the first
time, physicists from the Physics Working Groups used Magda for physics
analysis. They ran Magda to move the DC1 data files around the net in
their analysis activities and gave very interesting feedback.

As part of the US Atlas grid testbed demo in the SuperComputing 2002
conference. Magda was presented as an integral part of the US Atlas
testbed. Presentations were made demonstrating Magda's distributed data
management architecture, how it has been used in ATLAS DC1 production,
and achievements and near term plan. See
<http://www.atlasgrid.bnl.gov/magdademo/> for further information.

The command line tool \'magda_putfile\' has been further developed
during this period. It was extended to manage the files distributed on
each node of linux farms. It was extended to work with the files on Lyon
HPSS for the ATLAS data challenge. That is the third mass storage which
Magda now manages. It was extended to support third-party transfer with
the BNL HPSS, where both source and destination can be on BNL HPSS. Also
a new feature was added to cache and defer HPSS transfers when HPSS is
down. This feature was found to be very useful and added flexibility to
the US Atlas testbed production.

The \'magda_getfile\' tool has one more parameter called \'usagehour\'.
When putting a file to a disk cache the user can specify how many hours
he expects this file to be on the cache. After that another cron script
will delete the file.

A basic authentication mechanism was developed for the users of the web
interface. Normally the Magda web pages are for viewing and querying
file information. With the web forms, users could obtain an account for
using the web interface, login as members to do editing, modify their
profiles and logout. More edit functionalities (edit replication tasks,
location, site and host) will be developed as the next step.

A Magda user guide is in preparation. It is intended to be a complete
reference. See <http://www.atlasgrid.bnl.gov/magdadoc/userguide.htm>

A new Magda server was setup for the NCG group of Stony Brook for the
Phenix experiment.

### WBS 2.2.2 Simulation and Reconstruction Software 

The ATLAS detector simulation was further supported during Data
Challenge 1. The virtual data catalogue (VDC) database deployed in the
context of the ATLAS Data Challenges has being used successfully for
event generation and detector simulation.

The VDC based production system using Magda for data access and PACMAN
for software distribution was used to produce about 100 datasets of
fully simulated events in the second phase of DC1.

ATLSIM has been upgraded to provide the functionality to run complete
simulations of the ATLAS pileup including correlated backgrounds in the
calorimeter and in the muon system:

-   optimised pileup procedure implemented allowing up to \> 1k events
    to be added on a single physics event (in a few seconds of CPU)
    important for calorimeter and muon pileup

-   a procedure for a detailed cavern background simulation has been
    developed.

**WBS 2.2.4 Software Support and QA/QC**

The newly established ATLAS software infrastructure and testing (SIT)
team actively worked on software release management, quality assurance
and testing, and software compliance with coding standards in this
quarter. On SIT request, the U.S. ATLAS Librarian, Alex Undrus, provided
a script that creates a web page with stable release build and tests
results (similar to existing web pages of nightly builds summaries).

The ATLAS nightly build facilities at CERN and BNL were monitored and
improved. The production chain testing is incorporated in the nightly
builds. In these tests the single particle events are simulated and
reconstructed and then the detector response is compared with the
expected one. The production chain test results are reflected in a
special section of build summary web pages.

An ongoing program of incorporating greater QA in the automated build
system continued. The integrated tests that were previously described in
the nightlies scripts were moved to a dedicated package (TestRelease) in
the Atlas Software tree. The SIT group worked on organization of
individual tests for ATLAS software packages with active U.S.
participation. It was decided to create a dedicated package (AtlasTest)
for core software tests. The decision on organization of non core
packages tests and component tests is expected soon.

New ATLAS software was promptly installed at the BNL Tier I Center,
usually in one to three days after CERN installation. The BNL software
support page is updated in a timely way. .

The U.S. Atlas MySQL database, web server, and cvs pserver were
maintained and administered and their services were actively used by the
U.S. Atlas community.

## Milestones and deliverables in this quarter

### WBS 2.2.1.2 

-   Reengineered ProxyProvider complete (Nov) -- milestone met

-   Results from pile-up performance studies (Jan) -- first good results
    from LAr

-   ByteStream converters in place (Dec) -- complete except for muon
    system

-   Interval of validity service fully functional (Jan) -- pending

### WBS 2.2.1.3

-   Delivery of LCG POOL collection service prototype (Dec) -- milestone
    met

-   Retirement of Objectivity/DB from releases (Jan) -- milestone met

-   Interval of validity database connection to Athena (Jan) -- pending

### WBS 2.2.1.10

-   Production application of Magda in DC1 phase 2 (Nov) -- milestone
    met

### WBS 2.2.4

-   Migration of Athena and systems tests from nightly scripts to test
    packages (Dec) -- milestone met

-   Incorporation of production chain testing into standard test suite
    (Dec) -- milestone met

# **Forthcoming milestones and deliverables**

### WBS 2.2.1.2

-   xx

### WBS 2.2.1.3

-   Athena conversion service prototype based upon POOL Release 0.3.0
    (Feb)

-   Extensions to NOVA to support interval-of-validity services (Feb)

-   Explicit collections instantiated in ROOT, and higher-level
    collection services (Mar)

-   ATLAS workshop on conditions and other time-varying data (Feb)

### WBS 2.2.1.10

-   Completion of Magda User Guide (Mar)

### WBS 2.2.4

-   Provide the NICOS nightly control system for LCG automated builds (a
    generalized version of the ATLAS tool) -- Apr 2003

-   Build ATLAS software on RH7.3/gcc 3.2 in support of POOL integration
    -- Jan 2003