U.S. ATLAS Answers to Review Questions Posed 29-November-20001

[How can the US ATLAS costs per SW FTE be lowered?]{.underline}

The high cost per FTE of US ATLAS software developers relative to US CMS
is a consequence of our development effort residing at national labs as
compared to substantial university participation in US CMS. Our
development effort is at the labs because that is where the core
software expertise resides. We have in the past surveyed US ATLAS
universities for their potential as core software sites and have not
found interest and expertise sufficient to nucleate university-based
core efforts. Given this situation, our ability to reduce costs is
limited. What we can do, and have done just recently, is review the
personnel we support and look for opportunities to reduce costs by
replacing expensive experienced personnel with inexpensive postdocs. It
was through a careful evaluation of impact vs. savings that we decided
to replace a senior LBNL software engineer with a new postdoc. Clearly,
however, this is not an approach we can apply widely. The strength of
our program lies in the expertise of the participants. Developers with
expertise tuned to our needs are much more valuable than inexperienced
recruits.

The present design and implementation stage of ATLAS software
development is an environment in need of the talents of highly
experienced software developers, and the leading edge challenges of LHC
computing make it attractive to such developers. As we move beyond
development into operations and maintenance we can expect a migration of
some experienced people away from the project, driven both by changing
needs and changing interests.

It is worth pointing out that the US ATLAS software program has so far
enjoyed very low (non-existent) turnover among our senior developers.
The benefits of continuity among these experts are substantial.

[Is the scope of the T1 facility matched to the foreseen physics
requirements or tuned to the funding profile?]{.underline}

The facility is scaled to the physics requirements to the extent allowed
by the available funding. In particular, a disk centric model much
better meets the selection and analysis needs of ATLAS but was not
permitted by the available funding in the context of the original
schedule for LHC turn on. While the facility could be further improved
with additional funding, with the transition to a disk centric model one
is making a dramatic improvement resulting in a facility, which very
adequately satisfies requirements.

[What is the cost differential between the disk-centric T1 presented
this year versus the tape-centric T1 presented last year? What are the
differential operating costs?]{.underline}

In the table below is shown the cost difference between the two Tier 1's
as a function of year. This allows one to see various summary
differentials. If one compares the integrated facility costs at the
points where each become fully operational, 2006 in the old model and
2007 in the new model, the cost differential is \$6,299K with the new
model having a higher cost. If one compares the cost at a single point
in time, 2006, when the old model facility is fully operational but the
new model facility is not yet, the differential is --\$4,339K with the
new model having a lower cost. If one compares the facility cost at the
point in time, 2007, when the new model facility is fully operational
and the old model facility has been in operation for a full year (at an
annual ongoing operations cost which was estimated to be \$7,000K) the
differential is --\$701K with the new model again having a lower cost.
We feel that this last comparison is the most informative and shows that
the change in model in conjunction with the change in delivery date
allows one to produces a facility of dramatically improved functionality
in 2007 at a cost that is less than what would have been spent on the
old model facility through 2007.

While this may seem surprising, there are actually three factors that
contributed to ameliorating the cost of the additional capacity. They
are:

1.  Inclusion of FY 2007 "*ongoing"* operations funding (which included
    some equipment upgrade/refresh funding) into facility ramp up

2.  The Moore's Law impact of funding "*spent later*", effectively
    somewhat more than a year later.

3.  ![](media/image1.wmf)The reassignment to disk/CPU of that fraction
    of the funds spent on tape bandwidth which was being use to meet the
    "*selection from tape"* requirement.

The change in architecture somewhat reduces cost by removing the need to
develop strategies for optimally using a scarce disk resource. It is
however not clear that facilities staff would have been the ones to
solve this problem. In addition the criticality of the storage system is
reduce and therefore the level of support required for it is somewhat
reduced. In total the change in architecture will probably, in a
moderately small way reduce the operating costs of the facility. In
terms of scale, the amount of robotic tape hardware is reduced and since
this is a relatively expensive type of equipment to maintain, at least
relative to the disk and CPU purchased in its stead, there is probably
some maintenance saving here. The scale of the disk system and the
processor farms is increased, which somewhat increases the cost of
managing them. These two scaling issues play against each other. Without
a very detailed analysis it is not certain if the net effect of all of
these changes is one of reduced or increase operating costs. It seems
likely that there is a small overall saving in the cost of operating
with the new model facility but a more detailed analysis is required to
decide for sure and in any case the difference is going to be relatively
small.

[Provide a table of funding requested from FY00 - FY07 versus agency
guidance.]{.underline}

  ------------------ ----------- ---------- ----------- ----------- ----------- ----------- ------------ ------------ ------------ ------------
                     **FY 99**   **FY 00**  **FY 01**   **FY 02**   **FY 03**   **FY 04**   **FY05**     **FY06**     **Total**    **FY07**

                     ** **       ** **                                                                                             

                     ** **       ** **                                                                                             

  **Req. Nov 01      ** **       ** **      ** **       **4,000**   **5,578**   **9,021**   **11,135**   **15,903**   ** **        **19,531**
  Review**                                                                                                                         

  **Req. Nov 00 Post ** **       ** **      **3,037**   **4,010**   **6,314**   **7,734**   **10,058**   **14,431**   ** **         
  Review**                                                                                                                         

  **Req. Nov 00      ** **       ** **      **3,952**   **4,941**   **6,313**   **7,734**   **10,058**   **14,426**   **47,424**    
  Review**                                                                                                                         

  ** **              ** **       ** **      ** **       ** **       ** **       ** **       ** **        ** **        ** **         

  **Budget           **410.5**   **1781**   **3,000**   **4,000**   **5,500**   **8,500**   **13,500**   **17,500**   52,000        
  Guidance**                                                                                                                       

                                                                                                                                    

   PPDG                                                   289       289           289                                               

  **Doe Guidance**   **349**     **1126**   **2,000**   **2,500**   **3,500**   **5,500**   **9,500**    **12,500**   **35,500**    

  **NSF Assumption** **61.5**    **655**    **1,000**   **1,500**   **2,000**   **3,000**   **4,000**    **5,000**    **16,500**    
  ------------------ ----------- ---------- ----------- ----------- ----------- ----------- ------------ ------------ ------------ ------------

Note: FY 02 funding request in the PMP had not included a hardware
increment to Tier 2's from iVDGL, which are now included in this table
(bringing it to 4000k). The PPDG funding is technically "off-project",
but is tracked within the project, and is not considered part of the
"request". It actually began in July 01, but for the sake of simplicity,
we're splitting it across 02, 03 and 04 in this table.

[Provide a matrix of Grid projects and their deliverables to ATLAS.
Specify timescale, manpower delivered, and level of development for
each.]{.underline}

We make several assumptions in answering this question. First, this is a
discussion of the US Atlas work, so we have not included work related to
the EDG, although it should be noted that there are strong ties from all
involved to keep PPDG, GriPhyN, and iVDGL in line with that work.
Second, we list only ATLAS-specific work, and not work that ATLAS will
simply be taking advantage of (CMS has listed the security CA work in
PPDG for example, which we don't include).

The two tables below answer separate pieces of this question. The first
shows FTE effort by project/FY and listed in terms of where that effort
will be spent. The second table more fully explains the effort listed by
area, but without cross-referencing which project is supporting which
specific area, as our approach is fully integrated. For example, MAGDA,
developed by BNL and funded by PPDG, will be used to meet the GriPhyN
Yr1 goals although no GriPhyN funding is given to this.

Below the tables are a gantt chart showing the combined project goals,
and details of these efforts, as given to the PCAP review.

+----------+----------+----------+----------+----------+-----------+
|          | Data     | Job Mgmt | Int      | Mo       | Testbed/  |
|          | Mgmt     |          | egration | nitoring |           |
|          |          |          |          |          | Infra     |
|          |          |          |          |          | structure |
+----------+----------+----------+----------+----------+-----------+
| FY01     | 1.5      |          |          | 0.5 BNL  |           |
| PPDG     | --BNL    |          |          |          |           |
|          |          |          |          |          |           |
|          | 0.8 ANL  |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+
| FY02     | 0.75 BNL | 0.75 BNL |          | 0.5 BNL  |           |
| PPDG     |          |          |          |          |           |
|          | 0.8 ANL  |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+
| FY03     | To       | To       | 1.5 FTE  | 0.5 BNL  |           |
| PPDG     | integ.   | integ.   |          |          |           |
|          |          |          |          |          |           |
|          | 0.8 ANL  |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+
| FY01     |          | 1.  IU   |          |          | 1.0 BU    |
| GriPhyN  |          |          |          |          | packaging |
|          |          | 1.0 IU   |          |          |           |
|          |          | matching |          |          |           |
+----------+----------+----------+----------+----------+-----------+
| FY02-05  |          | 1.0 IU   |          |          |           |
|          |          |          |          |          |           |
| GriPhyN  |          |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+
| FY 02-06 |          |          |          |          | 2.  BU    |
| iVDGL    |          |          |          |          |           |
|          |          |          |          |          | 2.0 IU    |
+----------+----------+----------+----------+----------+-----------+
| FY02-04  |          |          |          | 1.0 IU   |           |
| Grid     |          |          |          |          |           |
| Telem.   |          |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+
|          |          |          |          |          |           |
+----------+----------+----------+----------+----------+-----------+

<table>
<colgroup>
<col style="width: 16%" />
<col style="width: 0%" />
<col style="width: 16%" />
<col style="width: 0%" />
<col style="width: 16%" />
<col style="width: 0%" />
<col style="width: 16%" />
<col style="width: 0%" />
<col style="width: 16%" />
<col style="width: 0%" />
<col style="width: 16%" />
</colgroup>
<tbody>
<tr class="odd">
<td>Year</td>
<td colspan="2">Data Mgmt</td>
<td colspan="2">Job Mgmt</td>
<td colspan="2">Integration between cols</td>
<td colspan="2">Monitoring</td>
<td colspan="2"><p>Testbed/</p>
<p>Infrastructure</p></td>
</tr>
<tr class="even">
<td colspan="2">FY02</td>
<td colspan="2"><p>Magda prototype 12/0-1</p>
<p>-for expert use in DC1, march</p>
<p>-for all use, june</p></td>
<td colspan="2"><p>GRAPPA in prototype now</p>
<p>-Simple job submission July</p></td>
<td colspan="2">GRAPPA integrated with Magda, prototype July</td>
<td colspan="2"><p>Leadership roles in joint monitoring group</p>
<p>-definition of common req</p>
<p>-initial deployment</p></td>
<td><p>PacMan in prototype now</p>
<p>VDT 1.0 to be packaged with pacman Dec. 02</p></td>
</tr>
<tr class="odd">
<td colspan="2"></td>
<td colspan="2">Malon work – PT by June</td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2">BNL infrastructure prototype, early testbed
deployment</td>
<td>IVDGL support</td>
</tr>
<tr class="even">
<td colspan="2"></td>
<td colspan="2">Metadata exploration</td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="even">
<td colspan="2">FY 03</td>
<td colspan="2">Magda- Expanded capacity</td>
<td colspan="2">GRAPPA as submission tool, incorporation of DAGMAN,
possibly others</td>
<td colspan="2">Integration of Grid tools and Atlas SW</td>
<td colspan="2">Atlas testbed deployment of monitoring, nw tuning</td>
<td>IVDGL support</td>
</tr>
<tr class="odd">
<td colspan="2"></td>
<td colspan="2">Malon work integrated with Athena fully</td>
<td colspan="2">Review integration with MOP, SAM, other LHC tools</td>
<td colspan="2">GRAPPA integrated with Taylor monitoring work</td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="even">
<td colspan="2"></td>
<td colspan="2">Evaluation and adoption of metadata technology</td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="odd">
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="even">
<td colspan="2">FY 04</td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2">Magda- Integrated with full scheduling approach</td>
<td colspan="2">Atlas testbed deployment of monitoring, nw tuning</td>
<td>IVDGL support</td>
</tr>
<tr class="odd">
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
<tr class="even">
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
</tr>
</tbody>
</table>

Specific milestone details

NOTE: PG -- PPDG goal, GG -- GriPhyN Goal, DC -- Data Challenge

•July 01-June02 PG1 Data management using MAGDA, Tier Tier1, Tier2

•Dec02 GG0.1 VDT 1.0 deployed (basic infrastructure)

•Jan 02 GG0.2 Integration of CERN testbed node into US Atlas testbed

•Jan 02-July02 DC1 Data creation, use of MAGDA, Tier 0-2

•July02-June03 PG2 job management, grid job submission

•July02-Dec02 GG1 Serving data from DC1 to universities, simple grid job
sub.

•Dec02-Sept03 DC2 Grid resource mgmt, data usage, smarter scheduling

•Dec02-Sept03 GG2 Dataset re-creation, metadata, advanced data grid
tools

•July03-June04 PG3 Smart job submission, resource usage

Details:

PPDG goals

PG1-present-june 02: Production distributed data service deployed to
users.

Data challenge deployment

Between CERN, BNL, and US grid testbed sites

Plan draws on grid middleware development while delivering immediately
useful capability to ATLAS

Data management has received little attention in ATLAS up to now

Full deployment is consistent with phase 2 of DC1

Consistent (and overlapping) with GG-1

PD2 - July 02- june 03: Production distributed job management service

Data challenge 2 job submission (deployment)

PG3- July 03- june 04: Create 'transparent' distributed processing
capability

Integrating distributed services into ATLAS software

Smarter job management

GriPhyN/iVDGL goals

GG0 Dec01-Jan01: VDT deployment, add CERN node to us testbed

GG1- July 2002-dec 2002 Serving DC1 data to community

limited reconstruction analysis job using grid job submission interface

\*Data serving

Serve data from DC 1 using grid infrastructure

\*some kind of job submission with minimal smarts

-possible suggestion grappa as remote job submission Bramley

GG2 - Jan-sept 2003 virtual data re-creation

What are all the parameters to keep track of?

What is the metric for evaluating the success (what is good enough)? We
won't have a bit-by-bit identical result, what's close enough?)

Feb-July 02 DC1

Note purely sequential, probably not using athena

Will produce data sets that are of interest to users

Data will be generated, and full software path will be used

Resulting reconstruction data is \~5 TB (possibly bigger)

Initially all produced at CERN

Data will need to be tagged with Metadata

Jan-sept 2003 DC2-use grid middleware 10% of production

  ------------- ------ --- ---- --- ------- ------- ------- ------- ------- ------- ------- ------- ------ ---- ---
                                    ** **   ** **   ** **   ** **   ** **   ** **   ** **   ** **                

                2001                2002                            2003                            2004         

  PG1                                                                                                            

  GG0                                                                                                            

  DC1                                                                                                            

  PG2                                                                                                            

  GG1                                                                                                            

  DC2                                                                                                            

  GG2                                                                                                            

  PG3                                                                                                            

                                                                                                                 

  data                                                                                                           
  management                                                                                                    

  scheduling                                                                                                     
  ------------- ------ --- ---- --- ------- ------- ------- ------- ------- ------- ------- ------- ------ ---- ---

[How will US Mgmt deal with potential "mission creep" in
software?]{.underline}

We identify two varieties of mission creep or scope broadening:
temporary and long term (more than \~6 months).

It is very important that we have the flexibility to make carefully
considered decisions to temporarily extend our participation into areas
beyond our defined scope in order to address emergency situations in
which US productivity and deliverables are threatened by inaction. An
example of this is the high level of involvement by David Quarrie and
others in the US in expediting the migration to CMT. As the committee
heard in many presentations, productivity suffered substantially during
this migration. Had the US not aided the process the migration would
have been both longer and more painful, to the detriment of our program.
Another example is the US venture into Geant4 integration into Athena,
which we judged to be well motivated under the circumstances but which
also has always had a clear horizon on US involvement, when integrated
Geant4 is passed to the simulation group.

It is equally important that we resist and avoid long term mission
creep. Our core areas are carefully chosen for maximum benefit to US
ATLAS physics analysis and maximum leverage of US expertise. The
deliverables in these core software areas are covered by software
agreements (one signed, more expected) that delineate our scope.
Dilution of our core effort through mission creep given our resource
constraints would weaken our program. We have a demonstrated record of
resisting mission creep; our present scope conforms to our plan. We have
resisted pressures both within and outside the US to broaden our scope.
In graphics, for example, while significant capabilities exist both at
participating universities and labs, we have declined to broaden our
program to include it. We have limited our involvement in a US
university (Santa Cruz) graphics effort to brokering a software
agreement between the university and ATLAS. In another example, in code
management tools we have declined overtures from ATLAS to get directly
involved in SRT (or other tool) development and support. We have not
allowed our recent CMT involvement to become a long term commitment (we
expect our involvement to end in about a month).

The area with the greatest potential for mission creep recently has been
in software roles most naturally sited at the host laboratory. US ATLAS
continues to argue that the CERN ATLAS software group must

have a strong role in software infrastructure and support. The US and
other remote participants cannot be expected to compensate for a weak
CERN effort.

US management will continue to assess the need for temporary ventures
outside our scope, approve them where necessary, and monitor their
progress. Should the US ever decide a long term scope extension is
necessary it will be handled by our change control mechanisms, and
revised or new software agreements.

[What is the US ATLAS plan to avoid "redundant" s/w development that
could be provided by other LHC experiments (e.g., GRAPPA vs
MOP)?]{.underline}

A consequence of ATLAS\'s late migration to C++/OO software and the
relative immaturity of its grid computing program is that ATLAS does not
have a large investment and installed base in software tools developed
in-house. As a result, ATLAS policy and practice is to \'look

to the market\' for software tools rather than pursue redundant in-house
developments. Examples include the LHCb Gaudi framework adopted by ATLAS
and now developed in collaboration with LHCb and others; the CMT code
management tool which was developed outside ATLAS; the Magda distributed
data management prototype which is designed to incorporate externally
developed grid toolkit components; the LHC-standard conditions database
employed by ATLAS; and our event database strategy which has always been
founded on employing LHC-standard solutions. US ATLAS adheres to and
supports this ATLAS-wide strategy.

Regarding the specific example of \'GRAPPA vs. MOP\', GRAPPA is a
flexible, general purpose \'grid portal\' for end user job submission
and management, whereas MOP is a tool specifically developed for
distributed production management. We anticipate using or at least
evaluating both tools. GRAPPA we are already pursuing. We are interested
in evaluating and using MOP as well in the context of the ATLAS PPDG
Year 2 program in distributed job management.

[Provide an evaluation of the risk that US ATLAS s/w deliverables will
need to be re-worked in light of future LHC-wide standards.]{.underline}

The risk is very low. As discussed in the answer to the \'redundant
software development\' question, ATLAS strategy is to plan for the use
of external tools. Accordingly, ATLAS architectural decisions have
consistently been oriented towards minimizing the dependencies of
ATLAS-developed software on tools and technologies adopted from the
community. Examples include extensive use of abstract interfaces such as
the (standard) AIDA interface to histogramming employed in Athena, the
by-design transient/persistent separation in Gaudi/Athena, and the
technology-independent database architecture as expressed in the
recently released design document.

Addressing event persistency in particular, adopting an LHC wide
persistency standard will have zero impact on user algorithm code in
Athena because of the strict transient/persistent separation. The
underlying data management layer in Athena will also be readily
adaptable to any standard solution because its design does not carry any
technology bias.

We note that (US) ATLAS is currently either using or well along in
integrating and evaluating potential future LHC-wide standards such as
Gaudi and ROOT I/O.

[Provide details of computing model numbers driving the size and
configuration of the facilities and network: size and production rate of
ESD (or start w/ events), AOD production & distribution to T2 (rate of
skims, size out), simulation production & reconstruction & analysis (&
where).]{.underline}

We use the following numbers, consistent with ATLAS planning and used in
MONARC simulations:

Atlas data rate: 270 Hz (100-400 Hz range)

Atlas raw event size: 2 MB (1 -- 3 MB range)

ESD size/event: 0.5 MB

Fraction of events in reconstructed data sample: 0.578

Detector live time fraction of 0.31 (1E7 seconds/year)

This implies an event sample of 2.7E9 events, a raw data size of 5.35 PB
per year and ESD of 775 TB produced twice per year.

AOD data size is 0.01 MB per event, so a typical AOD sample is 15.5 TB

The Tier 1 disk capacity (1 PB) is sufficient to hold one copy of the
ESD plus enough space for a few versions of the AOD, a reasonable sample
of raw data for calibration studies and leave some space for users.

Tier 1 CPU capacity (500 kSI95) is sized to maintain a reasonable CPU to
disk capacity ratio and allow analysis groups to run skim or AOD
production daily.

Tier 2 disk capacity (100 TB) is sufficient to hold two AOD copies (30
TB), a 2% sample of ESD data (15 TB), a 5% staging area for Monte Carlo
data (5E7 events at 2 MB per event = 100TB) or 5 TB, 10 Monte Carlo AOD
collections cached for analysis (10 TB), 20 TB for user analysis and 20
TB miscellaneous use.

The Tier 2 CPU capacity is scoped as follows:

Worldwide MC production 1E9 events per year at 3E4 SI95 (generation,
simulation and reconstruction) seconds per event. Assume US share is
25%, or 5% per Tier 2 center would require 47.5 kSI95 per US Tier 2
center. A similar CPU capacity should be available for analysis, so 100
kSI95 per Tier 2 is appropriate.

Network transfer assumptions/parameters:

-   Tier2s will typically run 5 "analysis jobs", where an analysis job
    is defined as taking 12 hours and processing 1% of the ESD data
    which is accessed over the WAN from the Tier 1.

-   Sum of all Tier2s and the Tier1 will provide capacity for 50
    "analysis" jobs every 12 hours.

> \- Each Tier2 will have AOD collections updated (copied from Tier 1) 4
> times/month.

-   The AOD allows selection of the 1% of the ESD data which will be
    read across the network but processed at the Tier2 site.

-   Simulation jobs will generate about 100 TB of raw simulation data
    per year at each Tier2 which is moved to the Tier1 for
    reconstruction/archiving. Tier 2 archival at two of the remote sites
    may be used to relieve bandwidth pressure and increase the
    efficiency and overall capacity of the system.

Resulting Tier 2 data transfer rates:

\- AOD copying (15.5 TB, 4 times/month) gives 190 Mbits/sec

\- Simulation data movement is 3.2 Mbits/sec

\- Reading ESD from Tier1 / job is 180 Mbits/sec, which gives a total
ESD bandwidth of 900 Mbits/sec

Total Bandwidth between Tier1-Tier2 is \~1.2 Gbits/sec

Resulting Tier 1 data transfer rates:

\- Data rate required for ESD copy to Tier1 is 1.55 PB/year or 400
Mbits/sec

\- ESD will also be "served" to other Tier1s at the same rate giving 400
Mbits/sec

\- 5 Tier2s are supported @ 1.2 Gbits/sec each

Total of 6.8 Gbits/sec bandwidth required at Tier1 (Plan is 10 Gb)

[What international capacity is likely to be tested in DC2 in early
2003?]{.underline}

Nominally, this is said to be at roughly 10%, but really the concept is
to exercise the "complexity" or interconnectivity of the system in DC2.

The \'scale\' is indeed set by the LHC Computing Grid Project plans

(the famous CERN 2379 document):

*\"..the construction of a prototype of the distributed*

*environment, reaching a size of about 50% of the complexity*

*of the final Tier 0 and Tier 1 systems for one experiment at CERN,*

*with systems of similar scale at several Regional Centres. The*

*prototype would be constructed and exercised progressively*

*during Phase 1 of the project to reach its full size in 2004.\"*

*The time-scale is for about half the prototype to be available*

*in March 2003, building up to the full prototype for March 2004.*

*For example, the cpu installation AT CERN is planned to be about*

*85kSI95 in 2003, rising to about double that in 2004; total disk*

*and tape-drive storage: 350TB rising to 600TB."*

(The above quote and numbers are straight out of

CERN 2379, section 5.1)

Several major regional centres are planning an evolution roughly in
phase with CERN: UK, France, Italy, Germany, Japan. This statement needs
to be checked quantitatively.

[Define how T2\'s should take part in US part of DC2. If US T2's are not
involved in DC2, what are the plans for integrated tests?]{.underline}

We anticipate the two prototype Tier 2 centers will be used in DC2 to
validate the distributed computing model for ATLAS. The Tier 2 centers
will generate Monte Carlo data and be used for physics analysis of DC2
AOD data, emulating operating conditions at LHC startup. Raw output from
event generation and detector simulation will be centralized at the Tier
1 center for reconstruction of ESDs. Job submission and grid metadata
cataloging tools will be used to manage this process. Monte Carlo AODs
produced at the Tier 1 center will be replicated to the Tier 2 sites for
physics analysis, and Tier 2 analysis of AOD and Tier 1-based ESD
collections will exercise the distributed functionality of the Athena
framework in a data grid environment.

[What level-of-effort is being devoted to support alternative
persistency solutions and otherwise demonstrate the "accepted" Athena
framework?]{.underline}

We address the question in two parts: support for multiple persistency
technologies, which we consider essential given the uncertainties
surrounding persistency; and the demonstration of multiple-technology
persistency support in Athena motivated more by Athena validation
considerations than functionally useful persistency.

On the former, supporting multiple technologies rather than just one has
required on the order of 0.5 FTEs (average DC level for the last \~8
months) of additional effort to date. This investment is essential if
serious technology evaluation is to occur in 2002.

On the latter, providing persistence support for continually-evolving
code (e.g., Atlfast and HepMC) beyond required proof-of-principle
infrastructure demonstrations has cost the core database effort between
0.5 and 0.75 FTEs (average DC level for the last \~1 year) of effort to
date. It is well understood in ATLAS that this responsibility, while
important, must migrate to the corresponding package developers, but
this has been slow in happening.

Once the ADL-based persistency code autogeneration mechanisms are in
place and employed for Atlfast, HepMC and other data models, the
development and support burden for sustaining any persistency mechanism
equipped with an ADL back end should be negligible.

We note that the \'Athena acceptance\' debate is behind us. Even the
(minority) contingents that dislike Athena acknowledge that it has
passed its acceptance tests in ATLAS.

[Who is providing the FTE effort for running DC1? At what level is the
US participating?]{.underline}

U.S.: Pavel Nevski is doing the production development and operations
for

DC0. While Pavel is likely to participate in DC1 as well, particularly

phase 1, we plan to transition production operations to another

(non-US) person during DC1.

International: The effort will be provided at CERN and a significant
part by CERN, AND, as has been made clear all along, some effort will be
required at and from the major participating centres. We envisage - for
the kind of tasks mentioned above from the U.S., and about 3fte at CERN
(\~2 provided by CERN), and about 1 fte in each of the major regional
centres, and this effort to be required over the period of DC1,
particularly at the start.

[Do you see any issues with BNL networking and computing
infrastructure?]{.underline}

Currently BNL has an OC3 (IP over ATM) connection to the New York NAP.
Effectively this limits the maximum bandwidth to \~120 Mbits/second,
which is less than both prototype Tier 2 centers (which have OC12
\"bottlenecks\") and many of the Tier3/candidate Tier 2 centers (3 of
which also have OC12 \"bottlenecks\"). The USATLAS testbed has
recommended OC12 as a minimum for 2002, eventually upgrading to OC48 by
the time ATLAS starts data taking.

Since many of the collaborations computing resources are located at the
Tier 1 site it is important that this site have a least the connectivity
of its clients. The ability of the testbed to adequately test the ATLAS
grid computing model is dependent on the network interconnections
between grid sites. Given the high network utilization of the RHIC
project at BNL there is a significant chance that ATLAS Testbed
activities will be adversely impacted by limited bandwidth. BNL requests
to ESnet for upgrade of this connection so far have not met with
success. The delay being based on ESnet funding limitation. However this
upgrade is very important to both the RHIC and US ATLAS projects at BNL.

[Which of these component types are now present within each of the
detector subsystems:]{.underline}

1\) Languages:          e.g. FORTRAN, C, C++, JAVA, other.\
 2) Persistency:        e.g. RZ file, ASCII, ROOT, OBJECTIVITY, XML,
other.\
3) Framework:         e.g. No framework interface, GOOFY, Athena,
Other.\
 4) GEOMETRY:      e.g. ADGG, Other\
 5) Sim. engine:        e.g. G3,G4, Fastsim, Standalone g4, Other.\
 6) Vis systems:       Please specify.\
 7) User interaction and configuration.\
 8) Four vector class.\
 9) Ntuple output.\
10) Ntuple/Histogram Abstraction layers (like HepTuple)

o Languages (FTN, C, C++, JAVA, other)

All the \"old\" software is in FTN. With the exception of one
institution (using F-90), all the new software is in C++. All Java is
currently internal to the graphics framework. There is a desire to allow
for better support for algorithmic code and services in Java. Currently
low priority.

o Persistency (RZ file, ASCII, ROOT, Objy, XML, etc.

All the \"old\" software uses RZ files, and input support for this wih
the new software has been a requirement. XML is being used for all
detector geometry definitions and internally within the graphics
framework. ROOT is used for histogram & n-tuple persistency (see later).
Objy and ROOT are planned to be symmetrically supported for the ADL data
dictionary for all detectors.

o Framework No framework interface, GOOFY, Athena, other

All detector systems are using Athena for reconstruction and fast
simulation. Several have stand-alone testbeam systems for historical
reasons. Fads/Goody is being used by the inner detector and muon systems
as an interim solution for physics validation studies.

o Geometry AGDD, other

All detector systems have prototype XML representations in conjunction
with AGDD. However experience has shown that the G4 geometry is in some
cases inadequate for simulation studies. Different optimizations are
being explored. The Pixels and SCT are being used as the prototypes for
the reconstruction geometry infrastructure, although other subsystems
have their own prototypes (e.g. muon).

o Sim. engine G3, G4, Fastsim, Standalone G4

The complete detector is covered by the G3 and fast simulations. There
is an ongoing effort to provide a complete G4 simulation with efficient
geometry. Much ATLAS work has gone into the physics validation of G4.
Several teststands have stand-alone G4 simulations (Pixels, LAr,
TileCal)

o Visualization

A common visualization API is used, with multiple back-end (Atlantis,
Aravis, GraXML, WIRED).

o User interaction and configuration

Athena supports job configuration text files (job options files) and
Python scripts. The plan is to phase out the job options files and
migrate to Python scripts because they offer more flexibility.

o Four vector class

CLHEP classes are used throughout.

o N-tuple output

Currently we support writing n-tuples to PAW/HBOOK files

or ROOT files. This is a configuration switch and invisible to
application code.

o N-tuple/Histogram abstraction layers (like HepTuple)

A single abstract interface is used for each. AIDA is used for
histograms and we\'re awaiting a \"standard\" n-tuple abstraction, with
the intention of migrating to it once one is available. Behind these
abstract interfaces a single concrete transient implementation is used.