# [Work Plan for D0 Grid]{.underline} 

07 December 2001

####### Early Draft

## Introduction

D0 Grid work involves participants from the SAM-Grid team at Fermilab,
PPDG funded developers, UK GridPP funded developers, and other D0
participants at UTA, Nikhef, Prague and elsewhere who want to work with
Grid tools and help to extend the functionality of SAM.

## Major Areas of Work 

### 1. Use of standard Grid middleware to promote interoperability

Several areas have been identified where Globus or Condor tools can be
used to either replace or augment SAM services. These were identified in
general terms in a CHEP2001 paper describing SAM and the Particle
Physics Data Grid. They include use of Globus security infrastructure,
use of GridFTP, use of Condor-G and Condor as a supported batch system
of SAM.

### 2. Grid job handling, services and applications 

This work has two foci: logical and physical handling of jobs. For both
aspects, we aim for work to be common across D0 Grid, PPDG and other
grid projects, as far as possible.

The logical job handling involves providing basic Grid job services that
allow specification, submission, reliable execution, tracking,
restarting of a job, or set of jobs. A Grid job must be made a
well-defined concept described in a standard (job description) language.
This work also includes development of demonstration applications, from
Monte Carlo request jobs to various analysis jobs.

The physical job management has a long term span and involves services
for scheduling of jobs with resource management. These services use
Information Services and experiment policies to provide inputs to the
problem of how to allocate and optimise use of the resources available
to the experiment on the Grid.

### 3. Monitoring and Information Services

This work also naturally divides into two areas of focus. Monitoring
information must be produced by SAM servers and by introducing other
'Sensors' into the system -- following emerging Grid Monitoring
Architecture standards and components. Providing monitoring information
from SAM servers must be done in a non-intrusive way that does not
compromise robustness of the system nor create potential performance
problems. Monitoring clients must be built to consume, interpret and
display the monitoring information. We must understand what display
tools need to be built to answer some of the most commonly asked
questions about the status and performance of the whole SAM Grid.

Information Services, on the other hand, are provided for the core SAM
services, at present, through a SAM database server. They include
replica catalogue and other meta-data used for resource management,
replica selection, etc. In preparation for eventual replacement of some
SAM information services, new information services, based on standard
Grid components, must be developed using independent agents as producers
of information.

## D0 Grid Project Infrastructure, maintenance and operations 

### 

### 

As with any project there is a considerable amount of work that has to
be done to provide infrastructure for people to work with as well as to
coordinate and track the project. Web pages must be created and hosted
and public relations activities must be undertaken. Liaison activities
with Grid projects and with Global Grid Forum working groups will be
needed. Once certain products are deployed, even if only in a
development/test mode, they must be kept up to date.

### 

### 

### 

### 

## Breakdown of tasks in each area of work

### Use of standard Grid middleware to promote interoperability

### 

######  1.1 Use of Globus security infrastructure for Station to Station services {#use-of-globus-security-infrastructure-for-station-to-station-services .unnumbered}

######   {#section-7 .unnumbered}

###### CA policy and issues -- and user sam

-   Work with Esnet CA and European CAs

> 1.1.2 Installation of Globus toolkit at all sites
>
> 1.1.3 Globus toolkit using Kerberos instead of PKI
>
> 1.1.4 Investigation of Certificate to Kerberos gateway use for Globus

-   Must involve Fermilab security folks

###### Use of GridFTP as a supported file transfer protocol for SAM

###### Use merely in same way as bbftp - without full GSI security

###### GridFTP using certificates 

###### Investigation and use of reliable GridFTP product

###### Being developed by Globus -- built in tuning of \# of parallel streamsUse of standard middleware to promote interoperability

###### 1.3  {#section-8 .unnumbered}

######  {#section-9 .unnumbered}

######  {#section-10 .unnumbered}

###### Use of Condor-G for job submission {#use-of-condor-g-for-job-submission .unnumbered}

###### 1.3.1 Install Condor-G  {#install-condor-g .unnumbered}

2.  Understand interface between SAM and Condor-G

3.  Implement interface between SAM and Condor-G

###### 1.4 Finish work on making Condor a supported SAM batch system {#finish-work-on-making-condor-a-supported-sam-batch-system .unnumbered}

-   Rod Walker has built a batch adapter for Condor

> 1.4.1 Include in SAM release
>
> 1.4.2 Deploy and test at various institutions
>
> 1.5 Grid standard replica catalog (from PPDG and DataGrid work)
>
> 1.5.1 Install Globus or other alpha version of some catalog code
>
> 1.5.2 Write tool to publish files and file locations from SAM
>
> 1.5.3 Write web or other interface to display SAM files from replica
> catalog

### 2.0 Grid job servicesPublish availability and status of SAM station resources and services using standard Grid middleware components and services, when they emerge.

1.  

###### Basic Job Services -- for specification, submission, restart, tracking

###### Design of Job specification language

1.  Design of what a job is -- aggregations of jobs

-   Conceptualize staged data processing; include MC data generation and
    farm processing.

-   Must be able to 'run anywhere' and bring its required libraries etc
    with it if necessary

-   Rod Walker and others have done some thinking on this

    1.  Decomposition of a job

    2.  DAG for job dependencies -- generation of D0 DAGs

    3.  DAGman for execution of DAGs

    4.  Interrupted execution of a 'chain' of jobs -- restart

###### Instead of dealing with checkpointing a job

5.  Specifications for job status monitoring

6.  Convert the "sam submit" mechanism to use Grid meta scheduling
    facilities such as CondorG/DAGman.

```{=html}
<!-- -->
```
1.  Job Scheduling Services

    1.  Co-locate jobs and data. Perform basic, high-level and global
        optimization of job dispatching (among SAM stations and sites)

        1.  Replica Selection Service

###### 2.2.2 Use of Information Services to compute "costs" of jobs and optimize placement of jobs and data {#use-of-information-services-to-compute-costs-of-jobs-and-optimize-placement-of-jobs-and-data .unnumbered}

### 

### 3.0 Monitoring and Information Services

###### 3.1 Monitoring Services {#monitoring-services .unnumbered}

3.1.1 Adapt Grid Monitoring Architecture

######  {#section-12 .unnumbered}

###### 

###### 3.1.2 Contribute to PPDG and GGF working groups

> 3.1.3 Test and adapt relevant Globus tools -- e.g. GRAM, GRIS
>
> 3.1.4 Producers of Monitoring Information

-   Enable the SAM servers to produce information in the right format

> 3.1.5 Consumers of Monitoring Information

-   Analysis and display of monitoring information in several forms

###### 3.2 Information Services {#information-services .unnumbered}

1.  Interim Information Services Architecture

    1.  De-centralization of the existing SAM information services
        (Database Server, Naming Server, etc.)

2.  Specification of new information services for D0 Grid e.g.

    1.  Provision for the Replica Selection Service

    2.  Resource availability and other provision for the Job Placement
        Service

    3.  Resource usage accounting service

3.  Identification and investigation of MDS, GRAM, GRIS, etc for
    persistent storage of some information used/produced by SAM servers

4.  External Information Producers

5.  

```{=html}
<!-- -->
```
1.  
2.  
3.  

```{=html}
<!-- -->
```
2.  

### D0 Grid project infrastructure, maintenance and operations

1.  Work plan

2.  Tracking of tasks and deliverables

3.  Setup cvs packages

4.  Web pages

5.  Meeting schedule, posting talks, minutes

6.  Fabric management

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

### 

## 

## 

## 

## 

## 

## 

## 

## 

## Goals, Deliverables and Milestones

Some high level goals, deliverables and milestones need to be identified
and people will need to work in a coordinated way to meet those goals.
PPDG and GridPP each have advertised deliverables and these must be
meshed with D0 Grid deliverables.

+----------+-----------------------------------------------------------+
| Month 3  | Architecture and Use case documents complete and project  |
|          | infrastructure set up                                     |
|          |                                                           |
|          | Deployment of Globus toolkit and Condor-G, adoption of    |
|          | ClassAds and other standard Grid technologies             |
|          |                                                           |
|          | Testing and evaluation of some of the "standard"          |
|          | Monitoring and Information services, including EDG        |
+----------+-----------------------------------------------------------+
| Month 6  | Integration of Globus Security Infrastructure and GridFTP |
|          | into SAM and deployment at several UK stations and other  |
|          | European and US stations - interoperating with Fermilab   |
|          | stations.                                                 |
+----------+-----------------------------------------------------------+
|          | De-centralization of the existing SAM information         |
|          | services                                                  |
+----------+-----------------------------------------------------------+
|          | Prototype of the Information Service that provides        |
|          | information about resources' availability and tracks      |
|          | their usage                                               |
+----------+-----------------------------------------------------------+
|          | Monitoring service for trouble-shooting and performance   |
|          | of the data handling system (stations' availability,      |
|          | throughput to/from site)                                  |
+----------+-----------------------------------------------------------+
|          | Monitoring of job status changes and job progress         |
+----------+-----------------------------------------------------------+
|          | Job placement advisory service based on resource          |
|          | availability.                                             |
+----------+-----------------------------------------------------------+
|          | First demonstration of MC production system using Request |
|          | interface.                                                |
+----------+-----------------------------------------------------------+
| Month 9  | Provision for extraction of historical data on system     |
|          | performance etc (information persistent, properly         |
|          | distributed and archived)                                 |
+----------+-----------------------------------------------------------+
| Month 12 | Automated job submission with limited intelligence in job |
|          | distribution and load balancing.                          |
+----------+-----------------------------------------------------------+
|          | Fully commissioned MC Production System with reliable     |
|          | execution of jobs, splitting into sub-jobs as necessary.  |
+----------+-----------------------------------------------------------+
|          | Monitoring of some of the global (grid-wide) activities   |
|          | such as global data replication and global job            |
|          | distribution                                              |
+----------+-----------------------------------------------------------+
| Month 18 | Intelligent job distribution and load balancing, taking   |
|          | into account the economics of data movement versus job    |
|          | movement.                                                 |
+----------+-----------------------------------------------------------+
| Month 24 | A fully robust production quality system, excellent       |
|          | monitoring and interoperability with other Grid projects  |
|          | and EU DataGrid with sharing of some resources. Updated   |
|          | Fabric capable of handing the data on each experiment.    |
+----------+-----------------------------------------------------------+
|          |                                                           |
+----------+-----------------------------------------------------------+