LHC Grid Computing Project

Architectural Roadmap towards Distributed Analysis

**[An Exercise on Grid Services Domain Decomposition]{.smallcaps}**

  --------------------------- ----------------- --------------------------
                              Document          xxx-xxx-xx-xxxx
                              identifier:       

                              Date:             06/08/2003

                              Authors:          

                              Editors:          

                              Document status:  **DRAFT v0.2**
  --------------------------- ----------------- --------------------------

  -----------------------------------------------------------------------
  [Abstract]{.underline}: We decided to start from a careful analysis of
  one of the existing projects with the most complete functionality
  necessary to meet the requirements of the distributed analysis and with
  a substantial history of successful use. We have chosen the AliEn
  project for this purpose. We inspect the services provided by the AliEn
  Grid in order to derive general domain decomposition. Having done that,
  we further examine other Grid projects to see how they fit into this
  decomposition, what other complementary services are provided to
  complete the overall functionality. The aim of this analysis is to come
  to an architecture defined as a set of collaborating services with
  well-defined interfaces. The rational of this approach is that the
  architecture defined this way is based on the services with at least
  one existing and proven implementation.

  -----------------------------------------------------------------------

  ----------- ------------ -----------------------------------------------------
  **Issue**   **Date**     **Comment**

  1.0         06/08/2003   First version
  ----------- ------------ -----------------------------------------------------

**Content**

[1 Introduction 5](#introduction)

[2 Requirements for distributed analysis on the Grid
6](#requirements-for-distributed-analysis-on-the-grid)

[3 Summary of Grid projects presented to this RTAG
7](#summary-of-grid-projects-presented-to-this-rtag)

[3.1 PROOF 7](#proof)

[3.2 AliEn 7](#alien)

[3.3 CLARENS 8](#clarens)

[3.4 DIAL 9](#dial)

[3.5 GANGA 9](#ganga)

[3.6 DIRAC 9](#dirac)

[4 AliEn Description 11](#alien-description)

[4.1 Components and Services 11](#components-and-services)

[5 General decomposition 14](#general-decomposition)

[5.1 API and User Interface 14](#api-and-user-interface)

[5.1.1 API 14](#api)

[5.1.2 User Interface 20](#user-interface)

[5.2 Authentication 21](#authentication)

[5.2.1 Authentication Service 21](#authentication-service)

[5.2.2 VO Administration Service 22](#vo-administration-service)

[5.3 Authorisation Service 23](#authorisation-service)

[5.4 Auditing Service 24](#auditing-service)

[5.5 Information System 25](#information-system)

[5.6 Computing Elements 26](#computing-elements)

[5.7 Storage Elements 28](#storage-elements)

[5.8 Workload Management 30](#workload-management)

[5.9 Data Management 34](#data-management)

[5.10 File Catalogue 36](#file-catalogue)

[5.11 Metadata Catalogue 38](#metadata-catalogue)

[5.12 Job Monitoring 39](#job-monitoring)

[5.13 Job Provenance 40](#job-provenance)

[5.14 Package Manager 40](#package-manager)

[5.15 Grid Monitoring 41](#grid-monitoring)

[6 Analysis on the Grid Error! Bookmark not
defined.](#__RefHeading___Toc47956236)

[6.1 ASYNCHRONOUS MODEL **Error! Bookmark not
defined.**](#__RefHeading___Toc47956237)

[6.2 SYNCHRONOUS MODEL **Error! Bookmark not
defined.**](#__RefHeading___Toc47956238)

# Introduction

There is a growing consensus that the next generation computational
grids will be built based on the distributed Grid Services paradigm
\["Physiology of the Grid"\]. The emerging Grid Services technology
standards elaborated by the GGF Consortium may well pave the road for
these developments. But this basic technology does not define the
necessary set of services that can be used as building blocks for
creation of practical grids. In order to define the set of services, we
need a grid architecture, which will fit these services together in a
coherent way. The need for an architectural view while developing
complex computational systems is recognized by virtually everybody now.
However, the usual top-to-bottom approach in definition of architecture
failed in many cases in the past. Therefore, we decided to take another
approach.

We decided to start from a careful analysis of one of the existing
projects with the most complete functionality necessary to meet the
requirements of the distributed analysis and with a substantial history
of successful use. We have chosen the AliEn project for this purpose. We
inspect the services provided by the AliEn Grid in order to derive
general domain decomposition. Having done that, we further examine other
Grid projects to see how they fit into this decomposition, what other
complementary services are provided to complete the overall
functionality. As a result of this analysis, we come to an architecture
defined as a set of collaborating services with well-defined interfaces.
The rational of this approach is that the architecture defined this way
is based on the services with at least one existing and proven
implementation.

The further development will result in providing missing services or new
service implementations that will correspond to the existing service
specifications. This will allow assembling whole grid applications out
of toolboxes of services with competing implementations that can be
fairly compared in similar settings. We are confident that concurrent
development of similar services by different development teams within
the same service specification is the best way to achieve the ultimate
quality of the final integrated system.

In Section 4we give a detailed description of the AliEn architecture. We
provide brief descriptions of some of the projects developed for the
distributed analysis in different experiments in Section3. The results
of the general domain decomposition derived from the analysis of the
AliEn architecture are present in Section 5. In subsequent sections we
give detailed description of the identified services. We present the
results of this exercise as a list of services subdivided in several
groups. For each service we describe briefly its intended purpose, the
possible public interface, collaborating services, existing
implementations.

# Requirements for distributed analysis on the Grid

The Grid Applications Group of SC2 undertook a detailed review of the
distributed analysis requirements recently \[HEPCAL-II\]. In the
following we give a brief summary of the analysis activity that must be
supported by the functionality of the eventual Grid system. A high level
scenario of the analysis activity can be summarized as the following.

A user or a group of users have an algorithm that they want to apply to
a particular selection of input data in a given execution environment.
The input data are selected via a query to a metadata catalogue. The
selection and the algorithm are passed to a workload management system
together with the specification of the execution environment. The
algorithms are executed on one or many nodes. The user monitors the
progress of the job execution. The results are gathered together and
passed back to the owner of the job. The resulting datasets can be
published in order to be accessible for other users.

The above scenario represents the analysis activity from the user
perspective. However, some other actions are done behind the scene of
the user interface:

-   To carry out the analysis tasks users are accessing shared computing
    resources. To do so, they must be registered with their Virtual
    Organization (VO), authenticated and their actions must be
    authorized according to their roles within the VO;

-   The user specifies the necessary execution environment (software
    packages, databases, system requirements, etc) and the system
    insures it on the execution node. In particular, the necessary
    environment can be installed according to the needs of a particular
    job;

-   The execution of the user job may trigger transfers of various
    datasets between a user interface computer, execution nodes and
    storage elements. These transfers are transparent for the user.

## ASYNCHRONOUS Analysis MODEL

As the first step, the analysis framework has to extract a subset of the
datasets from the virtual file catalogue using metadata conditions
provided by the user. The next and the most difficult part is the
splitting of the tasks according to the location of data sets. A
trade-off has to be found between best use of available resources and
minimal data movements. Ideally jobs should be executed where the data
are stored. Since one cannot expect a uniform storage location
distribution for every subset of data, the analysis framework has to
negotiate with dedicated Grid services the balancing between local data
access and data replication. Once the distribution is decided, the
analysis framework spawns sub-jobs. These are submitted to the Workload
Management with precise job descriptions. The user can control the
results while and after data are processed. The framework collects and
merges available results from all terminated sub-jobs on request. An
analysis object associated with the analysis task remains persistent in
the Grid environment so the user can go offline and reload an analysis
task at a later date, check the status, merge current results or
resubmit the same task with modified analysis code.

## SYNCHRONOUS analysis MODEL 

\[*The list is not exhaustive. It should not repeat the HEPCAL document
but should help in checking the resulting architecture against these use
cases*\]

# Summary of Grid projects presented to this RTAG

\[*These are mostly the CHEP03 abstracts related to a given project, in
the order in which they were presented to this RTAG. Other contributions
and more detailed explanations are welcome*.\]

## PROOF

**The PROOF Distributed Parallel Analysis Framework based on ROOT**

The development of the Parallel ROOT Facility, PROOF, enables a
physicist to analyze and understand much larger data sets on a shorter
time scale. It makes use of the inherent parallelism in event data and
implements an architecture that optimizes I/O and CPU utilization in
heterogeneous clusters with distributed storage. The system provides
transparent and interactive access to gigabytes today. Being part of the
ROOT framework PROOF inherits the benefits of a performant object
storage system and a wealth of statistical and visualization tools. This
paper describes the key principles of the PROOF architecture and the
implementation of the system. We will illustrate its features using a
simple example and present measurements of the scalability of the
system. Finally we will discuss how PROOF can be interfaced and make use
of the different Grid solutions.

\[http://arXiv.org/abs/physics/0306110\]

## AliEn

**The AliEn system, status and perspectives**

AliEn is a production environment that implements several components of
the Grid paradigm needed to simulate, reconstruct and analyse HEP data
in a distributed way. The system is built around Open Source components,
uses the Web Services model and standard network protocols to mplement
the computing platform that is currently being used to produce and
analyse Monte Carlo data at over 30 sites on four continents. The aim o
this paper is to present the current AliEn architecture and outline its
future evelopments in the light of emerging standards.
\[http://arXiv.org/abs/cs/0306067\]

**AliEn Resource Brokers**

AliEn (ALICE Environment) is a lightweight GRID framework developed by
the Alice Collaboration. When the experiment starts running, it will
collect data at a rate of approximately 2 PB per year, producing O(109)
files per year. All these files, including all simulated events
generated during the preparation phase of the experiment, must be
accounted and reliably tracked in the GRID environment. The backbone of
AliEn is a distributed file catalogue, which associates universal
logical file name to physical file names for each dataset and provides
transparent access to datasets independently of physical location. The
file replication and transport is carried out under the control of the
File Transport Broker. In addition, the file catalogue maintains
information about every job running in the system. The jobs are
distributed by the Job Resource Broker that is implemented using a
simplified pull (as opposed to traditional push) architecture. This
paper describes the Job and File Transport Resource Brokers and shows
that a similar architecture can be applied to solve both problems.
\[http://arXiv.org/abs/cs.dc/0306068\]

**AliEnFS - a Linux File System for the AliEn Grid Services**

Among the services offered by the AliEn (ALICE Environment this http
URL) Grid framework there is a virtual file catalogue to allow
transparent access to distributed data-sets using various file transfer
protocols. alienfs (AliEn File System) integrates the AliEn file
catalogue as a new file system type into the Linux kernel using LUFS, a
hybrid user space file system framework. LUFS uses a special kernel
interface level called VFS (Virtual File System Switch) to communicate
via a generalised file system interface to the AliEn file system daemon.
The AliEn framework is used for authentication, catalogue browsing, file
registration and read/write transfer operations. A C++ API implements
the generic file system operations. The goal of AliEnFS is to allow
users easy interactive access to a worldwide distributed virtual file
system using familiar shell commands (f.e. cp,ls,rm \...) The paper
discusses general aspects of Grid File Systems, the AliEn implementation
and present and future developments for the AliEn Grid File System.
\[http://arXiv.org/abs/cs.dc/0306071\]

**AliEn - EDG Interoperability in ALICE**

AliEn (ALICE Environment) is a GRID-like system for large-scale job
submission and distributed data management developed and used in the
context of ALICE, the CERN LHC heavy-ion experiment. With the aim of
exploiting upcoming Grid resources to run AliEn-managed jobs and store
the produced data, the problem of AliEn-EDG interoperability was
addressed and an in-terface was designed. One or more EDG (European Data
Grid) User Interface machines run the AliEn software suite (Cluster
Monitor, Storage Element and Computing Element), and act as interface
nodes between the systems. An EDG Resource Broker is seen by the AliEn
server as a single Computing Element, while the EDG storage is seen by
AliEn as a single, large Storage Element; files produced in EDG sites
are registered in both the EDG Replica Catalogue and in the AliEn Data
Catalogue, thus ensuring accessibility from both worlds. In fact, both
registrations are required: the AliEn one is used for the data
management, the EDG one to guarantee the integrity and access to EDG
produced data. A prototype interface has been successfully deployed
using the ALICE AliEn Server and the EDG and DataTAG Testbeds.
\[http://arXiv.org/abs/physics/0306103\]

## CLARENS

**The Clarens web services architecture**

Clarens is a uniquely flexible web services infrastructure providing a
unified access protocol to a diverse set of functions useful to the HEP
community. It uses the standard HTTP protocol combined with application
layer, certificate based authentication to provide single sign-on to
individuals, organizations and hosts, with fine-grained access control
to services, files and virtual organization (VO) management. This
contribution describes the server functionality, while client
applications are described in a subsequent talk.
\[http://arXiv.org/abs/cs.dc/0306002\]

**Clarens Client and Server Applications**

Several applications have been implemented with access via the Clarens
web service infrastructure, including virtual organization management,
JetMET physics data analysis using relational databases, and Storage
Resource Broker (SRB) access. This functionality is accessible
transparently from Python scripts, the Root analysis framework and from
Java applications and browser applets.
\[http://arXiv.org/abs/cs.dc/0306001\]

## DIAL

**DIAL: Distributed Interactive Analysis of Large Datasets**

DIAL will enable users to analyze very large, event-based datasets using
an application that is natural to the data format. Both the dataset and
the processing may be distributed over a farm, a site (collection of
farms) or a grid (collection of sites). Here we describe the goals of
the project, the current design and implementation, and plans for future
development. DIAL is being developed within PPDG to understand the
requirements that interactive analysis places on the grid and within
ATLAS to enable distributed interactive analysis of event data.
\[http://arXiv.org/abs/hep-ph/0305093\]

## GANGA

**GANGA: a user-Grid interface for Atlas and LHCb**

The Gaudi/Athena and Grid Alliance (GANGA) is a front-end for the
configuration, submission, monitoring, bookkeeping, output collection,
and reporting of computing jobs run on a local batch system or on the
grid. In particular, GANGA handles jobs that use applications written
for the Gaudi software framework shared by the Atlas and LHCb
experiments. GANGA exploits the commonality of Gaudi-based computing
jobs, while insulating against grid-, batch- and framework-specific
technicalities, to maximize end user productivity in defining,
configuring, and executing jobs. Designed for a python-based component
architecture, GANGA has a modular underpinning and is therefore well
placed for contributing to, and benefiting from, work in related
projects. Its functionality is accessible both from a scriptable
command-line interface, for expert users and automated tasks, and
through a graphical interface, which simplifies the interaction with
GANGA for beginning and casual users.
\[http://arXiv.org/abs/cs.se/0306085\]

## DIRAC

**DIRAC - Distributed Infrastructure with Remote Agent Control**

DIRAC is the LHCb distributed MC Production system. It consists of a
number of central services and clients running on each of the LHCb
production sites. The central services include the following:

-   Production service is managing the job queue. It accepts jobs
    prepared with a web based Production Editor. It gets requests for
    jobs from Agents, checks the capabilities of the requesting site and
    serves the jobs accordingly;

-   Job monitoring service collects and visualizes the job status
    information;

-   Bookkeeping service stores and serves metadata and replica
    information for the published datasets.

The clients at the production sites called Agents monitor the status of
the local batch system. If there are slots available, Agent contacts the
Production services for the workload; gets a job and installs the
execution environment (software, databases) according to its
requirements; submits the job to the local batch system. The job is
executed with the help of the workflow executor scripts that steers the
job and updates the central Job Monitoring service with the job progress
status. After the job is executed, the Agent insures the dataset
transfers, updating the Bookkeeping Services with the metadata and
replica information for the newly produced datasets.

The DIRAC central services and Agents are implemented as XML-RPC servers
and clients. \[http://arXiv.org/abs/cs.dc/0306060\]

# AliEn Description 

ALICE has developed the AliEn \[[^1]\] (ALIce ENvironment) framework
with aim to offer user community transparent access to computing
resources distributed worldwide. The intention is to provide a
functional computing environment that fulfils the needs of the
experiment in the preparation phase and, at the same time, defines
stable interface to the end users that will remain in place for long
time, shielding the core software from inevitable changes in the
technology. As new middleware becomes available, it will be interfaced
with AliEn, allowing evaluation of its performance and functionality.
The final objective is to reduce the size of the AliEn code, integrating
more and more high-level components from the Grid middleware, while
preserving its user environment and possibly enhancing its
functionality. If this is found to be satisfactory, AliEn code can
progressively removed in favour of standard middleware. In particular,
the intention is to make AliEn services compatible with the Open Grid
Services Architecture (OGSA) that has been proposed as a common
foundation for future Grids.

## Components and Services

The system is built around Open Source components, uses Web Services
model and standard network protocols to implement the distributed
computing platform that is currently being used to carry out the
production of Monte Carlo data at over 30 sites on four continents. Only
1% (around 30k physical lines of code in perl) is native AliEn code
while 99% of the code has been imported in form of Open Source packages
and perl modules.

![](media/image1.wmf)

Figure 1: The building blocks of AliEn

AliEn Web Services play the central role in enabling AliEn as a
distributed computing environment. The user interacts with them by
exchanging SOAP messages and they constantly exchange messages between
themselves behaving like a true Web of collaborating services.

![](media/image2.wmf){width="5.295833333333333in"
height="3.1958333333333333in"}

Figure 2: AliEn Web Services and their interactions (current
implementation)

![](media/image3.wmf){width="5.45in" height="3.745138888888889in"}

Figure 3: AliEn Services and their interactions (additional components
exposed as Web Services)

In attempt to abstract the implementation details and retain manageable
granularity of components, the services are grouped together into the
logical groups, which more or less correspond to usual components found
in alternative Grid implementations (Figure 4).

> ![](media/image4.wmf)

Figure 4: AliEn logical groups of services

The backend of AliEn is a relational database or, in a more general
case, a federation of relational databases. While this, in principle,
can also be a federation of heterogeneous databases in practice we use
the MySQL databases. To avoid linking with any specific database driver
on the client side, all connections to the databases are channelled via
a Proxy Service. The application connects to the Proxy Service by means
of a special AliEnProxy driver so that the real database driver and
libraries need to be installed only in the place where an instance of
the database Proxy Service is running.

# General decomposition

![](media/image5.wmf){width="4.5472222222222225in"
height="3.223611111111111in"}

Figure 5: The interaction diagram of key Grid components for typical
analysis use case

From the analysis of the AliEn architecture presented in the previous
section we derive the decomposition in the following key services:

-   API and User Interface

-   Authentication, Authorisation and Auditing services

-   Workload and Data Management Systems

-   File and Metadata Catalogues

-   Information service

-   Grid and Job Monitoring services

-   Storage and Computing elements

-   Package Manager and Job provenance service.

In the following we give descriptions of the identified services. The
emphasis is made on the services functionality that can be exposed via
its public interface. The interface descriptions are given to better
explain the functionality rather than to be a precise specification. In
the current version of the document detailed material on the AliEn
implementations is given where possible in order to keep the discussion
practical. The further work should allow presenting the results of this
analysis in more formal way.

## API and User Interface

### API

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The File Catalogue API provides a command line interface similar to a
UNIX file system with the most common commands implemented. Similar
functionality is provided by the graphical user interface. Using these
interfaces, it is possible to access the catalogue, submit jobs and
retrieve the output. AliEn provides a web portal as an alternative user
interface where one can check the status of the current and past jobs,
submit new jobs and interact with them. The web portal offers additional
functionality to 'power' users -- Grid administrators can check the
status of all services, monitor, start and stop them while VO
administrators (production user) can submit and manipulate bulk jobs.

To maintain compatibility with EDG/LCG Grid projects, AliEn user
interface uses the Condor ClassAds \[[^2]\] as a Job Description
Language (JDL). The JDL defines the executable, its arguments and the
software packages or data that are required by the job. The Workload
Management service can modify the job's JDL entry by adding or
elaborating requirements based on the detailed information it can get
from the system like the exact location of the dataset and replicas,
client and service capabilities.

In order to gain access to Grid at the application level, users need an
API. Besides the native perl module that allows user to use, modify or
extend the interface, AliEn provides C and C++ API. In particular, the
C++ API is thread safe and we used it to implement a fully featured file
system on top of AliEn File Catalogue (alienfs).

##### Public Interface

class TAliEnAPI {

/\*

\* General information:

\* All functions returning int will return 0 on success, a negative
value

\* on failure:

\* -1 : Error in the argument list

\* -2 : The SoapCall failed

\* -3 : The called function returned an error

\*/

public:

typedef unsigned long RESULTHANDLE;

typedef struct Result_T {

string name;

string name2;

struct stat info;

};

TAliEnAPI(){

fNextHandle = 1;

}

virtual \~TAliEnAPI() {

}

/\*

\* Opens directory cname and returns HANDLE for browsing through the
directory

\* Use ReadResult, ResetResult and CloseResult for browsing

\* Returns 0 if there is an error

\*/

RESULTHANDLE OpenDir(const char \*cname);

/\*

\* Closes the directory hResult and frees the assoziated information.

\* Do not use Result_T structs after freeing a handle.

\*/

void CloseResult(const RESULTHANDLE hResult);

/\*

\* Resets the directory information hResult, so browsing with ReadResult

\* starts with the first entry.

\*/

void ResetResult(const RESULTHANDLE hResult);

/\*

\* Returns the next result of the directory hResult.

\* A pointer to a Result_T struct is returned, the values which are
filled depend

\* on the type of result. The struct can be uses until the result is
freed with CloseResult.

\* Returns NULL when reaching the end of list

\*/

const struct Result_T\* ReadResult(const RESULTHANDLE hResult);

/\*

\* Creates a directory in AliEn.

\* If makeall = true is passed, all directories in the path, which do
not exist

\* are created.

\* Returns negative value on error, zero on success.

\*/

int MkDir(const char \*dir, const bool makeall = false);

/\*

\* Removes a directory from AliEn.

\* If deleteall = true is passed all containing subdirectories and files
are deleted.

\* Otherwise deleting of a not empty directory is not permitted.

\* Returns negative value on error, zero on success.

\*/

int RmDir(const char \*dir, const bool deleteall = false);

/\*

\* Removes a file from AliEn.

\* Returns negative value on error, zero on success.

\*/

int Rm(const char \*lfn, const bool deleteall = false);

/\*

\* Copies a file from source to target.

\* Returns negative value on error, zero on success.

\*/

int Cp(const char \*source, const char \*target);

/\*

\* Moves a file from source to target.

\* Returns negative value on error, zero on success.

\*/

int Mv(const char \*source, const char \*target);

/\*

\* Add physical file to catalog and associate logical file name.

\* Returns -1 on error, like lfn, pfn already exists, etc.

\* Example:

\* lfn=\"lfn://\[alien.cern.ch\]/alice/cern.ch/user/p/psaiz/http.file\"

\* pfn=\"castor://host/path\"

\* size=12001

\* size of the file in bytes. It\'s optional. If -1, AliEn will try

\* to guess the size from the pfn.

\*/

int AddFile(const char \*lfn, const char \*pfn, int size = -1);

/\*

\* Add physical file mirror to logical file name.

\* Example:

\* lfn=\"lfn://\[alien.cern.ch\]/alice/cern.ch/user/p/psaiz/http.file\"

\* pfn=\"castor://host/path\"

\* se=\"Alice::CERN::scratch\" (NULL = no SE)

\*/

int AddFileMirror(const char \*lfn, const char \*pfn, const char \*se);

/\*

\* Register physical file to catalog and associate logical file name.

\* Example:

\* lfn=\"lfn://\[alien.cern.ch\]/alice/cern.ch/user/p/psaiz/http.file\"

\* pfn=\"castor://host/path\"

\* Storage elements can be given as a second arguement like:

\* pfn=\"myfile.root Alice::CERN::scratch\"

\*/

int RegisterFile(const char \*lfn, const char \*pfn);

/\*

\* Get physical file name associated to logical file name.

\* Returns 0 in case of error.

\* Surf through the handles by ReadResult, ResetResult and CloseResult

\*/

RESULTHANDLE GetPhysicalFileNames(const char \*lfn);

/\*

\* Add a tag to a directory.

\* Example:

\* tagName=\"standard\"

\*/

int AddTag(const char \*lfn, const char \*tagName);

/\*

\* Get tags for specified logical file name.

\* Returns empty list in case no tags

\* were found. Returned string must be freed by user.

\*/

RESULTHANDLE GetTags(const char \*lfn);

/\*

\* Add attribute to lfn, like \"run\" \"1001\".

\* Example:

\* tagName=\"standard\"

\* attrname=\"tagIndex\" //or \"value\"

\* attrval=\<int\> //or \<char(20)\>

\*/

int AddAttribute(const char \*lfn, const char \*tagName,

const char \*attrname, const char \*attrval);

/\*

\* Delete attribute associated to lfn.

\* If attrname is empty, deletes all the attributes of this tag

\* Otherwise, it deletes only the value of that attribute

\* Arguments:

\* lfn=\"lfn://\[alien.cern.ch\]/alice/cern.ch/user/p/psaiz/file\"

\* tagName=\"MonteCarloRuns\"

\* attrname=\"\";

\*/

int DeleteAttribute(const char \*lfn, const char \*tagName,

const char \*attrname);

/\*

\* Get list of attributes defined for lfn. Returns 0 in case of error.

\* Surf through the handles by ReadResult, ResetResult and CloseResult

\*/

RESULTHANDLE GetAttributes(const char \*lfn, const char \*tagName);

/\*

\* Change access mode of lfn. \<mode\> is the usual st_mode.

\* Returns -1 in case of error.

\*/

int Chmod(char\* lfn, int mode);

/\*

\* Change Owner (if \[\<group\>!=NULL\]+Group) of lfn.

\*/

int Chown(char\* lfn, char\* owner, char\* group);

/\*

\* Query for lfn.

\* It returns a list of files that match a given pattern. The pattern

\* can contain conditions on metadata (tags).

\* Format:

\* wildcard=\"lfn://\<host\>/\<path\>?\<tagname\>:\<tagcondition\>

\* Examples:

\* wildcard=\"lfn://alien.cern.ch/alice/bin/date\"

\* wildcard=\"lfn:///alice/simulation/2001-04/V0.6\*.root\"

\*
wildcard=\"lfn:///alice/simulation/2001-04\*?MonteCarloRuns:HolesPHOSRICH=1\"

\*/

//? AlienResult_t \*AlienGetFile(const char \*wildcard);

/\*

\* Download a file.

\*/

//? AlienResult_t \*AlienRetrieveFile(const char \*lfn);

/\*

\* Download a filechunk.

\* offset : where to start to read file

\* size : bytes to read

\* destinationfilename : NULL - let AliEn decide the temp-name or

\* give \<locafilename\>

\* returns the destinationfilename or 0 in result count!

\*/

//? AlienResult_t \*AlienRetrieveFileChunk(const char \*lfn, long
offset, unsigned long size, const char\* destinationfilename);

/\* Submit a Job to AliEn, give the Job Description File as an argument.

\<JDL-File\> can be a local or file (file://\<machine\>/\<path\>) or on

alien \</alice/cern.ch/\>

\*/

int SubmitJob(const char \*jdlfile);

RESULTHANDLE QueryJob(int JobId);

//Kill

//Resubmit

};

###### DIRAC

###### EDG 1.4

###### EDG 2.0

### User Interface

##### Public Interface

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

> The User Interface Service represents the SOAP server counterpart to
> C++ API described above.

###### DIRAC

> A user client application communicates directly to the Bookkeeping
> (metadata), Production (job submission) and Monitoring services via
> XML-RPC protocol.

###### EDG 1.4

###### EDG 2.0

## Authentication

### Authentication Service

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The Authentication Service is responsible for checking user's
credentials. AliEn uses the SASL \[[^3]\] protocol for authentication
and implements several SASL mechanisms (GSSAPI using Globus/GSI, AFS
password, SSH key, X509 certificates and AliEn tokens)

![](media/image6.wmf){width="3.4805555555555556in"
height="3.345833333333333in"}

Upon successful authentication a Proxy Service acquires and holds the
real database handle on behalf of a user and returns a temporary access
token which the user has to present in order to re-connect to the
database. The token remains in user possession and is valid for a
limited period of time.

![](media/image7.wmf){width="1.429861111111111in"
height="2.7576388888888888in"}

###### DIRAC

> Plain password authentication when accessing services via Web portal
> interface

###### EDG 1.4

###### EDG 2.0

### VO Administration Service

##### Public Interface

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

  AddUser                 Yes               Adds new user to the VO database

  AddRole                 Yes               Adds new role to the VO database
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

This service provides means for update and administration of the
users/roles/groups information for a given VO. This information is used
for user authentication and authorization of various user operations.
AliEn uses a hierarchical database (LDAP -- Lightweight Directory Access
Protocol) to describe the static configuration for each Virtual
Organization (VO). This includes *People, Roles, Packages, Sites* and
*Grid* *Partitions* as well as the description and configuration of all
services on remote sites. The code that is deployed on remote sites or
user workstations does not require any specific VO configuration files,
everything is retrieved from the LDAP configuration server at run time
thus allowing user to select VO dynamically.

###### DIRAC

None

###### EDG 1.4

LDAP directory service

###### EDG 2.0

VOMS

## Authorisation Service

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

###### DIRAC

###### EDG 1.4

###### EDG 2.0

## Auditing Service 

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

An Auditing Service (Logger) provides the mechanism for all services to
report their status and error conditions. This allows Grid manager to
monitor all exceptions in the system and to take corrective action.

###### DIRAC

> All the services are writing separate log files

###### EDG 1.4

###### EDG 2.0

## Information System

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The Configuration Manager is responsible for discovery and read-only
interactions with the LDAP server. It extracts all relevant
configuration parameters that apply to a particular VO in the context of
a site and a specific host. The information is kept in a cache to avoid
frequent LDAP lookups.

![](media/image8.wmf){width="1.2979166666666666in"
height="3.0118055555555556in"}

###### DIRAC

###### EDG 1.4

> MDS

###### EDG 2.0

R-GMA

## Computing Elements

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The Computing Element (CE) is an interface to the local batch system. At
present, AliEn provides interfaces to LSF, PBS, BQS, DQS, Globus and
Condor. The task of a CE is to get hold of jobs JDLs from the CPU
Server, translate them to the syntax appropriate for the local batch
system syntax and execute them. Each job is wrapped up in another Web
Service (Process Monitor) allowing users to interact with the running
job (send a signal or inspect the output). Prior to job execution, the
CE can automatically install the software packages required by the job
using the Package Manager functionality.

> ![](media/image9.wmf){width="5.99375in" height="3.8159722222222223in"}

###### DIRAC

> Agents are processes running on each production site (Computing
> Element). They are implemented in Python and have classes representing
> different batch systems. At present there exist LSF, PBS, BQS and EDG
> interfaces. Each Agent is an XML-RPC client of the central Production
> service. It gets job in a form of xml description files, optionally
> installs the necessary software, submits jobs to the local batch
> system, performs data transfers in the end of the job execution and
> updates the Bookkeeping database.

###### EDG 1.4

###### EDG 2.0

## Storage Elements

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The Storage Element (SE) is responsible for saving and retrieving the
files to and from the local storage. It manages disk space for files and
maintains the cache for temporary files.

![](media/image10.wmf){width="4.345833333333333in"
height="3.709722222222222in"}

![](media/image11.wmf){width="1.2916666666666667in"
height="1.695138888888889in"}

###### DIRAC

> Python API to remotely access bbftp and ftp servers and to use the EDG
> Replica Manager functions.

###### EDG 1.4

###### EDG 2.0

## Workload Management

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

As opposed to the traditional push architecture, AliEn workload
management system is distributed (involves services deployed on remote
sites) and it is based on a \`\`pull\'\' approach. A central service
component manages all the tasks while computing elements are defined as
\`\`remote queues\'\' and can, in principle, provide outlet to a single
machine dedicated to run a specific task, a cluster of computers, or
even an entire foreign Grid. When jobs are submitted, they are sent to
the central task queue.

The workload manager optimizes the queue taking into account job
requirements based on input files, CPU time, architecture, disk space
etc.. It then makes jobs eligible to run on one or more computing
element. The active nodes then get jobs from the queue and start their
execution. The queue system monitors the job progression and has access
to the standard output and standard error.

While the jobs or file transfer requests are waiting in the task queue,
the Job and Transfer Optimizers will inspect JDLs and try to fulfil
requests and resolve conflicts. This can result in the system triggering
file replication in order to make job eligible to run on some sites to
balance the overall load on the system. Along the same lines, one can
also implement policy monitors to enforce VO policies by altering job
priorities.

![](media/image2.wmf)

Figure 6: AliEn Workload Management

Due to weak coupling between the resources and the Resource Brokers in
the AliEn Grid model it is possible to imagine a hierarchical Grid
structure that spans multiple AliEn and "foreign" Grids but also
includes all resources under the direct control of top level Virtual
Organization.

The connectivity lines in Figure 7 represent the collaboration and trust
relationships. In this picture the entire foreign Grid can be
represented as a single Computing and Storage Element (albeit a
potentially powerful one). In this sense, we have constructed the
AliEn-EDG interface and tested the interoperability.. Along the same
lines, AliEn-AliEn interface allows creation of federation of
collaborating Grids. The resources in this picture can be still shared
between various top level Virtual Organizations according to the local
site policy so that the Grid federations can overlap at resource level.

![](media/image1.wmf)

> Figure 7: The federation of collaborating Grids

![](media/image2.wmf){width="4.836111111111111in"
height="4.205555555555556in"}

###### DIRAC

> Collaboration of the Central Production services which manages the VO
> wise job queue and Agents running on each production site. Agents are
> "pulling" the jobs from the Production service whenever there is a
> free slot in the local batch system and steer the job execution.
>
> In case of the DIRAC work with the EDG execution back-end, a job
> submitted to the Resource Broker after arriving to a worker node
> starts with the installation of a standard DIRAC agent. This turns the
> worker node to be a small DIRAC "production site". The rest is done in
> exactly the same way as with the usual production site.

###### EDG 1.4

###### EDG 2.0

## Data Management

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

This service typically runs on the same host as the Storage Element and
provides the scheduled file transfer functionality. The File Transfer
Daemons (FTD) are mutually authenticated using certificates and will
perform file transfer on user's behalf using the bbftp^ ^\[[^4]\]
protocol. File transfers are requested and scheduled in exactly the same
way as jobs, this time under the control of the File Transfer Broker.

![](media/image3.wmf){width="4.857638888888889in"
height="3.829861111111111in"}

###### DIRAC

###### EDG 1.4

###### EDG 2.0

## File Catalogue

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

Input and output associated with any job can be registered in the File
Catalogue, a virtual file system in which a logical name is assigned to
a file. Unlike *real* file systems, the File Catalogue does not own the
files; it only keeps an association between the Logical File Name (LFN)
and (possibly more than one) Physical File Names (PFN) on a real file or
mass storage system. PFNs describe the physical location of the files
and include the name of the Storage Element and the path to the local
file.

The system supports the file replication and caching and will use file
location information when it comes to scheduling jobs for execution. The
directories and files in the File Catalogue have privileges for owner,
group and the world. This means that every user can have exclusive read
and write privileges for his portion of the logical file namespace (home
directory). In order to address the problem of scalability, the AliEn
File Catalogue is designed to allow each directory node in the hierarchy
to be supported by different database engines, possibly running on
different host and, in future version, even having different internal
table structures, optimized for a particular branch (Figure 8).

![](media/image4.wmf){width="3.979861111111111in"
height="2.9659722222222222in"}

Figure 8: AliEn File Catalogue

![](media/image5.wmf){width="3.620833333333333in"
height="3.584722222222222in"}

###### DIRAC

> ORACLE database with an XML-RPC server as a front-end.

###### EDG 1.4

###### EDG 2.0

## Metadata Catalogue

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The hierarchy of files and directories in the AliEn File Catalogue
reflects the structure of the underlying database tables. In the
simplest and default case, a new table is associated with each
directory. In analogy to a file system, the directory table can contain
entries that represent the files or again subdirectories. Due to this
internal structure, it is possible to attach to a given directory table
an arbitrary number of additional tables, each one having a different
structure and possibly different access rights while containing metadata
information that further describes the content of files in a given
directory. This scheme is highly granular and allows fine access
control. Moreover, if similar files are always catalogued together in
the same directory this can substantially reduce the amount of metadata
that needs to be stored in the database. While having to search over a
potentially large number of tables may seem ineffective, the overall
search scope has been greatly reduced using the file system hierarchy
paradigm and, if data are sensibly clustered and directories are spread
over multiple database servers, we could even execute searches in
parallel and effectively gain performance while assuring scalability.

###### DIRAC

> ORACLE database with an XML-RPC server as a front-end. Web based user
> interface.

###### EDG 1.4

###### EDG 2.0

## Job Monitoring

![](media/image6.wmf){width="1.7131944444444445in"
height="2.745833333333333in"}

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

###### DIRAC

> ORACLE database with an XML-RPC server as a front-end. Web based user
> interface displaying jobs status.

###### EDG 1.4

###### EDG 2.0

## Job Provenance

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The File Catalogue is not meant to support only regular files -- we have
extended the file system paradigm and included information about running
processes in the system (in analogy with the /proc directory on Linux
systems). Each job sent to AliEn for execution gets an unique *id* and a
corresponding */proc/id* directory where it can register temporary
files, standard input and output as well as all job products. In a
typical production scenario, only after a separate process has verified
the output, the job products will be renamed and registered in their
final destination in the File Catalogue. The entries (LFNs) in the AliEn
File Catalogue have an immutable unique file id attribute that is
required to support long references (for instance in ROOT) and symbolic
links.

###### DIRAC

> Part of the ORACLE based Bookkeeping database.

###### EDG 1.4

###### EDG 2.0

## Package Manager

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

Each VO can provide the *Packages* and *Commands* that can be
subsequently executed on AliEn Grid. Once the corresponding tar files
with bundled executables and libraries are published in the File
Catalogue and registered in a LDAP directory, the AliEn Package Manager
will install them automatically as soon as a job becomes eligible to run
on a site whose policy accepts these jobs. While installing the package
in a shared package repository, the Package Manager will resolve the
dependencies on other packages and, taking into account package
versions, install them as well. This means that old versions of packages
can be safely removed from the shared repository and, if these are
needed again at some point later, they will be re-installed
automatically by the system. This provides a convenient and automated
way to distribute the experiment specific software across the Grid and
assures accountability in the long term.

![](media/image7.wmf){width="1.1777777777777778in"
height="1.4111111111111112in"}

###### DIRAC

Once the software package is properly bundled by the LHCb Software
manager and is available in the Software repository (via http protocol),
Agents running on each production site can install the packages required
by jobs. The check is done for each job, so old packages can be safely
removed.

Application Python classes which are responsible for the interpretation
of the job descriptions and steering the job executions.

###### EDG 1.4

###### EDG 2.0

## Grid Monitoring

##### Public Interface 

  ----------------------- ----------------- --------------------------------
  **Public Method**       **Needs           **Comments**
                          authorization**   

                                            

                                            
  ----------------------- ----------------- --------------------------------

##### Existing Implementations

###### AliEn

The Resource Brokers do not require overall monitoring information to
carry out job or file transfer scheduling. However, if available, this
information would be useful to various Optimizer services. We are
currently planning to deploy the MonaLisa \[[^5]\] monitoring framework
as a part of the AliEn Monitor Module. It will collect the monitoring
information and publish it via Web Service for use by AliEn Optimizers
or for visualization purposes. On the longer term, the intention is to
re-use the network simulation code originally developed for
MONARC^ ^\[[^6]\] and now part of MonaLisa and extend it to cover the
behaviour of the distributed Web of services that constitute the AliEn
Grid. With this, we should be able to optimize and understand the
performance of the system and verify its scalability.

###### DIRAC

###### EDG 1.4

###### EDG 2.

[^1]: \[?\] P. Saiz, L. Aphecetche, P. Buncic, R. Piskac, J. -E.
    Revsbech and V. Sego, AliEn -- ALICE environment on the GRID,

    Nucl. Instr. and Meth. A 502 (2003) 437-440, http://alien.cern.ch

[^2]: \[?\] Condor Classified Advertisements,
    http://www.cs.wisc.edu/condor/classad

[^3]: \[?\] http://asg.web.cmu.edu/sasl/

[^4]: \[?\] http://doc.in2p3.fr/bbftp/

[^5]: \[?\] H. B. Newman, I.C. Legrand, MonaLisa: A Distributed
    Monitoring Service Architecture, these proceedings, MOET001

[^6]: \[?\] http://asg.web.cmu.edu/sasl/