# Topics for PCAP Review

I would like to focus on two main issues:

1.  FY 02, 03 budgets and implications and priorities.

2.  Status of grid planning - are we adequately funded and staffed? How
    coherent are the grid plans?

This note consists of several parts:

a.  Overall budget guidance and assumptions

b.  Reactions to budget guidance from software and facilities

c.  Software grid planning

d.  Facilities grid planning

# Budget Outlook for FY 02,03

In FY 01, not only did we not get our desired \"compromise\" budget, but
the DOE was only able to provide \$1.5M, rather than the guidance of
\$2.0M. \$0.5 M was borrowed from construction project funds, with the
assumption that this would be paid back in FY 02. Unfortunately the FY
02 outlook is also not great, and it is unlikely that we can repay this
loan from the construction project without severely impacting our
activities.

In addition to project funds, U.S. ATLAS may receive PPDG and iVDGL
funding, directed at grid efforts, and will receive a tiny amount from
GriPhyN (most of the GriPhyN funding was directed to CS activities).
Although the budgets are not known for FY 02 or 03, I started with the
budget guidance from the DOE/NSF, and allocated PPDG and iVDGL
activities specifically from this guidance. Although PPDG and iVDGL
funds were supposed to be \"over and above\" project funds, it is my
belief that these will be counted against project funds and hence must
be subtracted off any non-grid activities.

I think it is important to make sure that GriPhyN, iVDGL and PPDG are
funding beyond our core mandated tasks (i.e. regional center and
software), in other words, not subtracting off. Since we\'re already
stretched thin, counting these programs against our \"program\" budget
would amount to an unfounded mandate.

The following table represents a breakout of the FY 02/03 budgets, based
the DOE/NSF guidance, and my estimates of a distribution among physics,
facilities, software and reserve:

Category FY 02 FY03

Physics 197 202

SW 2,450 2,871

Fac. 1,289 2,252

Support 74 140

Reserve 0 547

Total 4,010 6,314

Here is the proposed iVDGL buget:

FY02 FY03

ATLAS T2 HW 197 242

ATLAS people 256 356

Total 453 598

We can discuss how iVDGL gets allocated among SW and Facilities. As a
\"strawman\" - let\'s assume that the sum of iVDGL numbers gets divided
by the fractional budget numbers for FY 02 and 03 in the top tables,
leading to a \"debit\" to SW of \$300k and \$150k to facilities FY 02,
and 335 to SW and 263 in FY 03, leaving:

Category FY 02 FY 03

Physics 197 202

SW 2150 2536

Fac 1139 1989

Support 74 140

Reserve 0 547

Total 3560 5414

As a general comment, this breakout leaves facilities handicapped in
terms of possible hardware purchases for storage. The shortfall is
roughly \$500k in both FY 02 and 03. The software effort is ramping a
bit. It is worthwhile pointing out that some of the BNL software effort
was to be supported by base program funding, which unfortunately has not
materialized. If PPDG funding comes through, this may alleviate this
shortfall.

### Comments on Budget

[Physics]{.underline}

The requested FTE for generator interface appears in the budget. This is
as per our plan.

### Software

[FY02:]{.underline}

[- Funding in the guidance below: \$2450k]{.underline}

[- Funding requested in our compromise profile: \$2797k]{.underline}

[- Delta: funding shortfall of \$350k.]{.underline}

Shortfall is about two people. The level of funding in the guidance is
just sufficient to sustain our planned profile of US-based activity. We
should suspend plans to provide any staffing at CERN beyond the one FTE
we already have from the relocated LBNL person. Priority must be on
sustaining (at LBNL and ANL) and approaching (at BNL) critical mass at
US core software development sites.

[FY03:]{.underline}

[- Funding in the guidance below: \$2871k]{.underline}

[- Funding requested in our compromise profile: \$2917k]{.underline}

Delta is essentially zero. Our plan stands. Incremental money over the
previous year goes towards continuing the BNL build-up and proceeding
with a delayed CERN build-up. Funding sources should allow us the
flexibility to adhere to our plan. In particular, funding restrictions
should not force the effective relocation of software development effort
from CERN (or certainly from labs) to universities.

### Facilities

In FY 01, facilities received only 38% of their requested budget,
leaving them short handed in terms of both personnel and hardware.
Rather than ramping up to the planned 5.5 FTE\'s, they are at a level of
2.5 FTE\'s. The guidance listed above for FY 02 and 03 gives them 71%
and 83% respectively of the requests for those years.

In terms of capacities, the planned capacity levels for FY 02 and 03
were to achieve 2% and 5% of overall capacity, which is a ramp to 100%
in the end of FY 05. Instead, with the present profile, these numbers
would be 0.7% and 2.8% for FY 02 and 03 respectively.

The personnel ramps in our plan called for 7.5 and 10.5 FTE\'s in FY 02
and 03, whereas the profile above gives 5.5 and 7.5 FTE\'s respectively.

In summary, this budget profile delays the ramp up of the Tier 1
facility by approximately one year. Given the delay in the LHC turn on
time, one might believe that this is not a severe impact, but there are
other issues that one must consider. The major MDC is in FY 03 (MDC2).
BNL will certainly participate in this MDC, which will emphasize
complexity (meaning interconnectivity) as opposed to pure capacity.
Another issue is the function of BNL as the Tier 1 in participating in
grid tests. This may be more severe, as initial planning exercises did
not factor in the additional effort associated with grids, and the
funding from the agencies does not address this in any way (PPDG is
mainly a software implementation, iVDGL is directed at Tier 2 centers).

# Grid Planning

The funding sources for grid efforts come from GriPhyN (Grid Physics
Network), which provides toolkit releases, but only provides one FTE
postdoc to U.S. ATLAS. More significant are iVDGL (international Virtual
Data Grid Laboratory), which establishes a limited set of prototype Tier
2\'s and personnel. This funding is limited to Universities, in
particular the initial two Tier 2 sites: Indiana University and Boston
University. PPDG, the Particle Physics Data Grid is a DOE effort and
will mainly benefit BNL and ANL.

## Software Planning

A detailed WBS projection of grid activities can be found at the BNL
website, under the URL
<http://atlassw1.phy.bnl.gov/servlets/XProject/?wbs=1.3&proj=usgrid>

The workplan for grid activities, including GriPhyN and PPDG schedules
can be found under the URL:
[http://atlassw1.phy.bnl.gov/servlets/XProject?schedule=yes&keys=usgrid](http://atlassw1.phy.bnl.gov/servlets/XProject?schedule=yes&proj=usgrid&keys=usgrid)

Here are partial contents of the software workplan:

Start End Description

2001/10/1 2002/6/30 ATLAS PPDG SciDAC Yr 1: Production distributed data
service

SciDAC Year 1 program:

Principal deliverable: Production distributed data service

PPDG CS topics involved: CS-1, CS-2, CS-3, CS-4, CS-5, CS-6

The distributed data services described here are to exist between

CERN, the Tier 1 Facility at BNL, and a few other

U.S. institutes. Likely early participants are ANL, LBNL, Boston U,

Indiana U, and U Michigan. The objective is a multi-point U.S. Grid

(in addition to the CERN link) providing distributed data services as

early as possible.

\- Distributed file and replica catalogs (CS-5)

\- Cataloging files resident on disk and in mass storage. Based on

logical names in an agreed global Grid namespace. Serving read-only

files at CERN, the Tier 1 at BNL, and select U.S. institutes.

Supported by web, shell and API tools to browse, query and

manage catalogs.

\- Deployment of data transport services in production, serving the

distributed data service (CS-6)

\- Production distributed data service providing managed distribution

and replication of data files (CS-4, CS-5, CS-6)

\- User initiated data replication via web or command line.

Simple replica cache management. Service is integrated with

hierarchical mass storage: HPSS system at the BNL Tier 1;

collaboration with CERN and EU DataGrid for CERN mass store support.

\- Remote job submission service testbed (in use by developers and

\'friendly users\') (CS-1, CS-2)

\- Supports remote submission of jobs to the Tier 1 from a number

of external centers. Requires limited deployment of distributed

authentication services.

\- Grid instrumentation service: basic monitoring of distributed data

service (CS-3)

\- Monitors activity, usage, performance, and error conditions.

Web based display and reporting tools.

\- ATLAS \"data signature\" design supporting Grid requirements in
place.

Prototyping begun.

\- A specification of data set content and characteristics complete

enough to, in principle, regenerate it. Used in tests of dataset

equivalence, current validity, etc.

\- U.S. ATLAS distributed computing services architecture

\- Requirements gathered, design begun. (CS-7)

2001/10/1 2002/9/31 iVDGL Year 1 (FY02)

Principal iVDGL goal for year 1: establish the Laboratory. Experiments

will test infrastructure and conduct first application experiments.

ATLAS activities:

\- initial grid-enabled software development

\- 1% scale data challenge simulating group-based production and
individual

physics analysis chains on iVDGL

CMS activities:

\- deploy first interfaces between CARF framework (and ODBMS) and Globus
APIs

\- complete a CMS production cycle between multiple iVDGL sites with

partially grid-enabled software

2002/7/1 2003/6/30 ATLAS PPDG SciDAC Yr 2: Production distributed job
management

PPDG SciDAC Year 2 Program:

Principal deliverable: Production distributed job submission service

The major new functionality to be delivered in year 2 is a distributed

job submission service.

\- Broad deployment of user-level Grid authentication in support of

\"Grid user\" based functionality of production remote job submission

service and enhanced distributed data service. (CS-1, CS-2)

\- Distributed data service enhancements (CS-4, CS-5, CS-6)

\- Extend service to several additional U.S. institutes.

\- Integrate replica management into core database software

supporting production.

\- Add replica cache management service.

\- Incorporate user-level storage resource reservation.

\- Add cost estimation service.

\- Provide C++ and Java APIs for catalog services.

\- Automated, event-driven (e.g. \"update available\" signal)

replica updating implemented to ensure consistency across

file instances.

\- Production remote job submission service (CS-1, CS-2)

\- Extension of simple year 1 system to production use by the general

community

\- requires full deployment of user-level Grid authentication.

\- Extension of Grid instrumentation service (CS-3)

\- Full monitoring of distributed data service and basic monitoring

of job submission service.

\- Improved display and reporting tools.

2002/10/1 2003/9/31 iVDGL Year 2 (FY03)

iVDGL year 2: Demonstrate value in application experiments. In
experiments,

first large scale application runs across 15 sites and GB/s networks.

ATLAS activities:

\- continued integration of grid services with the Athena framework.

\- Data Challenge 2: 10% complexity scale involving 5-10 iVDGL sites.

\- validate the LHC computing model.

\- performance and functionality tests will be used in ATLAS computing

technical design report.

CMS activities:

\- 5% complexity data challenge using 10 iVDGL sites and \~50 users

\- use DTF to explore use of NRC class facilities

\- completion of a CMS production cycle where half the efforts are

completed using grid tools, including:

\- first pre-production set tools for task monitoring

\- optimal task assignment to sites

in addition to tools used in the previous year.

2003/7/1 2004/6/30 ATLAS PPDG SciDAC Yr 3: Transparent distributed
processing

PPDG SciDAC Year 3 Program:

Principal deliverable: Transparent distributed processing services

The principal goal in year 3 is to enhance the distributed data and

processing services with user interfaces, specification languages, and

ATLAS infrastructure integration to provide transparency to the user

of the distributed nature of processing and analysis, both in \'batch\'

and interactively.

\- Production distributed processing service (CS-1, CS-2, CS-3)

\- Extension of remote job submission service to provide transparency

(to the distributed nature of the processing) to the ATLAS offline

analysis user.

\- Distributed services integrated directly into ATLAS software

infrastructure.

\- Integrated distributed data management services

(CS-1, CS-2, CS-3, CS-4, CS-5, CS-6)

\- Integration of distributed data services with ATLAS database and

data management infrastructure to provide (policy-constrained) user

transparency to data locality.

\- Cost estimators supporting policy control integrated.

\- Run, event, and event feature (tag) metadata integrated with PPDG

catalogs

\- integration of run and event collection metadata with PPDG

metadata catalog

\- integration of event tag generation with PPDG metadata catalog

\- integration of tag-based event selection with metadata catalog

\- Register in catalog not only logical collections, but also the

attributes by which they (or elements they contain) may be

identified

\- ATLAS \"data signature\" deployed in support of coherence/consistency

of file replicas and transparency in data set requests.

2003/10/1 2004/9/31 iVDGL Year 3 (FY04)

iVDGL year 3: Couple with other national and international
infrastructures.

In experiments, conduct large scale international experiments.

ATLAS activities:

\- establish full chain tests of the grid-enabled Athena control
framework

with increasing size and complexity

\- execute Athena-grid production for the ATLAS Physics Readiness Report

(Jan-Jun 2004)

CMS activities:

\- completion of a CMS production cycle between multiple sites where by

default all of the CMS production efforts are completed using grid tools

\- system evaluation with diverse studies of \~10\^8 fully simulated and

reconstructed events

2004/10/1 2005/9/31 iVDGL Year 4 (FY05)

iVDGL year 4: Production operation on an international scale. iVDGL

resources will be used for large scale, long duration computing runs

designed to stress test the infrastructure.

ATLAS activities:

\- 20% scale full production capability realized involving 10\'s of
iVDGL

sites in the U.S., Europe and Japan

CMS activities:

\- first year of development of the Production Grid System for CMS
physics

\- large scale data productions, including runs with a level of
complexity

(number of processors, disks, tape slots) which are 50% of the LHC

production levels

2005/10/1 2006/9/31 iVDGL Year 5 (FY06)

iVDGL year 5: Expand iVDGL to other disciplines and resources.

ATLAS activities:

\- LHC startup during 2006

\- support full-scale production of Monte Carlo and detector calibration

analysis activities using the iVDGL

CMS activities:

\- deployment of the unified collaboration-wide CMS Grid system, to be
used

during LHC operations

\- final testing and development stages, with continual scaling up and

progressive integration of all CMS sites between Tier0 and Tier2 (30+
sites)

into the production grid system

\- test interfaces to an increasing number of institute servers (up to
150)

and to \~2000 desktops in CMS, by the time of the first LHC physics run,

in Summer 2006

### Facilities

Grid WBS 1.3.12 Grid administration infrastructure

*1.3.12.1 Evaluate toolkits designed to authorize, authenticate, and
grant distributed resources to ATLAS users. Includes registration of
ATLAS grid nodes in a centralized, replicated grid information database
servers.*

*1.3.12.2 Certification of Globus tools for grid security and
authentication.*

This maps to the next two tasks:

Task: Administer user account mapping, certificate exchange,
authentication configuration.

Schedule:

Maintain accounts/certificates: Ongoing

Document BNL certificate policy/contact info on web pages: May

Evaluate/estimate system/manpower requirements to operate certificate
authority at BNL: July

Coordinate/Track certificate policies with external
organizations/collaborations: Ongoing

Task: Maintain BNL resource registration with appropriate Grid
information services.

Schedule:

Initial configuration and registration of BNL resources: Done

Document existing BNL configuration on web pages: May

Maintain resource configuration and registration with Grid services and
web pages: Ongoing

Evaluate existing tools, enhance or create tools to improve site
resource management: Some progress (zeroth release of a US ATLAS
resource registration/management tool package?) by September, but task
is ongoing.

Notes: The certification of Grid tools for security could be a Tier 1
task or may be delegated to the Tier 2 groups.

# Grid WBS 1.3.14 Toolkit configuration, deployment and support

Grid WBS 1.4 Grid testbeds

These map to the next three tasks:

Task: Support ATLAS collaborators in establishing production Grid data
transfer services between BNL and remote sites. Develop user interfaces
as needed to simplify use. Work with PPDG to incorporate any new or
improved tools developed at BNL into the deliverable PPDG tool kits.

Schedule:

Functional data transfer service tools deployed: done.

ATLAS tests of Globus ftp to/from BNL: done

User-friendly tools for data transfer: September

Integrate any new development with PPDG: Ongoing

Task: Deploy replica management tools as available to integrate local
ATLAS data storage with remote collaborating sites. Integrate replica
management with data transfer services. Assist ATLAS collaborators in
establishing and maintaining catalogs as needed.

Schedule:

Deploy and test existing tools: June

Integrate with data transfer activities: September

Improve tools/interfaces as needed & integrate with PPDG: Ongoing

Task: Develop/package release kit(s) to simplify installation and
configuration of Grid services at ATLAS Tier 2, Tier 3 and Tier 4 sites.

Schedule:

First release kit prototype (for evaluation by existing ATLAS Grid
Testbed sites): July

First US ATLAS release kit available for Tier 3 testing: September

Maintenance and upgrades: Ongoing

Support remote sites in installation and debugging Grid installation:
Ongoing

# Grid WBS 1.3.3.12 Generic interfacing to Mass Stores

*This WBS item covers development and/or integration of generic, uniform
interfaces to heterogeneous mass storage management systems. A Replica
Manager manages file and metadata copies in a distributed and
hierarchical cache. Data movers transfer files from one storage system
to another one. The interfaces should encapsulate the details of the
local file system and mass storage systems such HPSS and others.
Evaluate storage managers such as SRB: Storage Resource Broker. This is
an interface which encapsulates the details of the local file system and
mass storage systems such HPSS.*

# 

# Grid WBS 1.3.11 Grid mass storage management

1.3.11.1 Evaluation of existing APIs for mass storage management

1.3.11.2 Testing, evaluation of metadata management tools for archived
data stored on HPSS systems.

*1.3.11.3 Integration of tools for use with ATLAS software.*

Task: Integrate BNL HPSS storage with Grid data transfer and replica
management tools.

Schedule:

Working quick prototype read/write via DFS: May

Integrate HPSS Info with replica management tools: September

Deploy/Evaluate LBL developed gsiftp-HPSS interface: when available

# 

Not (yet) included in our 6 month work plan:

# Grid WBS 1.3.13 Grid Workflow Management

*1.3.13.1 Define and implement a suitable architecture for scheduling
and resource management in a GRID environment.*

*1.3.13.2 Integrate grid tools which ensure optimal co-allocation of
data, CPU and network for specific grid-network-aware jobs.*

*1.3.13.3 Integration of data and/or code migration of
unscheduled/scheduled jobs.*

*1.3.13.4 Adapt uniform interfaces to various local resource managers
necessary for grid-enabled ATLAS software programs.*

*1.3.13.5 Implement tools which enforce policies on CPU, data, and
network resource usage.*