# Grid for CDF

Rick St. Denis

August 30, 2004

# Offline priorities for Summer 2004: 25% computing offsite

The priorities are

-   Pinning remote datasets (Dcaf)

-   Storing MC files to SAM (Dcaf)

-   SAM on CAF

The status is:

-   49TB of golden datasets at FNAL, 26TB offsite but 16TB not reachable
    with Dcaf

-   Core problems:

    -   users not using SAM.

    -   DFC migration not complete and cooperation difficult

-   Users will not take advantage of offsite computing until it is no
    different from using FNAL computing. This requires

    -   SAM only use on site (Policy with experience)

    -   Choice of site when user enters dataset (CAF)

## Plan for move to SAM

-   Ask users to contact DH with scripts that detect DH stress at the
    caf submission time. Users must give the dataset they use honestly.
    Get control over the tape mounting so that non-SAM use does not
    impact SAM use negatively. (in place since early August)

-   Police for users that ignore the request (in place since early
    August)

-   Get volunteers to ensure that all user tools and documentation up to
    snuff. (in progress, but could use more users)

-   Switch to sam only use

# Offline priorities for Summer 2005: 50% computing offsite

-   Deploy grid tools to enable more usage

## Issues

-   Frank Sees Grid and SAM as inefficient and unyielding monster.
    Strong language against it. Wants to use a non-grid supported way of
    getting at grid resources. Trying to work out with Ian Fisk, Ruth,
    Wyatt...

-   Manpower for SAMGrid hurt enormously by CD policy: Lost main
    developer, but he could not deliver. At the same time CDF MC group
    clamoring for his product. Again, working this out with Fisk, Ruth,
    Wyatt

-   Loss of Glasgow manpower: 1 did nothing, 1 quit, 1 wanted to quit:
    position moved to a development product with 2 more postdocs and
    student to get ability to get correct information about datasets.

# Glasgow

-   Enormous efforts in service work -- little payoff in physics

-   Will provide output of Metadata working group -- wish to exchange
    for ability to capitalize on physics. 2 new students and a new
    postdoc (already CDF). Wish to get service work for students "paid
    for" by the last 3 years of contribution from Grid and my work.

# CDF

-   Publication efforts scandalous

-   Top paper example paragraph: waste of time to send such a PRD to the
    collaboration

-   Far too much exchange of constants in Tcl files.

-   Essence of physics is reproducibility

-   Would require strict discipline on Physics coordinator's parts.

# How SAMGrid and metadata can help

## Example:MC

-   Run MC

-   Save RCP params (format of TCL to text db)

-   Upload to SAM and get a request ID

-   Send request to MC production group

-   Program at worker node: gets number of events, run number, you
    return file (and file with metadata, but that can come from file)

```{=html}
<!-- -->
```
-   This is reproducible and the parameters are browsable. Able to get a
    list of requests that worked. Can then use that as a basis for the
    next request:

```{=html}
<!-- -->
```
-   Get an RCP database from params in SAM

-   Read into AC++ and view menus

-   Make changes

-   Go to example above

```{=html}
<!-- -->
```
-   Development Needed:

    -   Upload RCP and Download (Randy says 1 day)

    -   Get RCP to set TCL params so that they can be seen in menus when
        a user downloads a set from SAM (not sure, need to talk to Liz)

-   This is also possible to do with the Production EXE.

-   This flushes out text files that should be on the calibration db.

-   Works for a static list of modules

Discipline required

-   Analysis only allowed if based on MC with request id. Would need to
    require this in CDF notes on anlaysis. No pre-blessing without this.

## Example Ntuple: First instance

-   User runs analysis program under SAM

-   Stores files in SAM

-   Files accessible to the collaboration

Development needed

-   Documentation and field testing

-   Concatenation (ntuples cannot be too small)

Discipline required

-   Analysis only allowed if ntuples made public in this manner

## Example Ntuple: Second instance

-   User treats analysis as in MC -- able to store TCL with work

Development needed

-   Need to be able to add new "modules" on the fly and new "parameters"
    on the fly as well.

-   Ability to specify the application as a set of modules with a tag in
    CVS.

Discipline required

-   Analysis only allowed if ntuples made public in this manner, and
    code tagged in CVS.

-   Second person must be able to reproduce results.