Grid Computing

Ian Foster

*Mathematics and Computer Science Division, Argonne National Laboratory*

*Department of Computer Science, The University of Chicago*

**Abstract**. The term "Grid Computing" refers to the use, for
computational purposes, of emerging distributed Grid infrastructures:
that is, network and middleware services designed to provide on-demand
and high-performance access to all important computational resources
within an organization or community. Grid computing promises to enable
both evolutionary and revolutionary changes in the practice of
computational science and engineering based on new application
modalities such as high-speed distributed analysis of large datasets,
collaborative engineering and visualization, desktop access to
computation via "science portals," rapid parameter studies and Monte
Carlo simulations that use all available resources within an
organization, and online analysis of data from scientific instruments.
In this article, I examine the status of Grid computing circa 2000,
briefly reviewing some relevant history, outlining major current Grid
research and development activities, and pointing out likely directions
for future work. I also present a number of case studies, selected to
illustrate the potential of Grid computing in various areas of science.

# Introduction {#introduction .INTRODUCTION}

The term \`\`computational Grid\'\' or simply \`\`Grid\'\' refers to a
new class of infrastructure designed to enable resource sharing among
geographically distributed, typically multi-institutional communities
\[10\]. Much as the Internet and Web have reduced barriers to the
exchange of information, Grid protocols and services are intended to
facilitate remote access to and the coupling of computers, storage
systems, display devices, and people, regardless of physical location.

Grid concepts originated in scientific and engineering computing,
motivated by the ever-increasing focus on multi-disciplinary,
collaborative research and computationally oriented approaches to
scientific problems. We are currently at an interesting juncture in the
development and application of the technology. Five years of prototyping
studies have demonstrated the promise of new Grid-based problem-solving
approaches and created an extensive knowledge base concerning tools and
techniques. Spurred by the success of these studies, significant Grid
deployment efforts have started, in which key infrastructure elements
are widely deployed, and major scientific communities have started to
retool to embrace the use of Grid technologies. The next several years
should see significant success stories as well as refinements in our
understanding of the technology.

# Historical Antecedents

We can trace the historical antecedents of today\'s Grid computing back
to the earliest days of computer networking: after all, the original
goal of work on projects such as ARPANET and Multics was to enable
remote access to computers. Distributed systems research also has a long
and distinguished history of both fundamental contributions and
practical application, and contributes much to the body of technology on
which Grids are constructed.

Interest in the more ambitious applications considered here really
developed with the deployment of high-speed experimental gigabit
testbeds in the late 1980s. For example, the CASA testbed linked
Caltech, the Jet Propulsion Laboratory, Los Alamos National Laboratory,
and the San Diego Supercomputer Center with a dedicated high-speed
network and then demonstrated that it was possible, via a combination of
specialized algorithms and protocol refinements, to achieve significant
speedups for a variety of distributed applications \[14, 15\]. However,
these and other related experiments did not attempt to put in place any
protocols or services designed to facilitate sharing: resources were
scheduled manually.

The concept of a Grid infrastructure distinct from that of the
underlying Internet emerged as a result of the I-WAY project in 1995, in
which many of the nation\'s high-speed networks and supercomputers were
interconnected to provide a powerful, although short-lived, application
testbed \[6\]. An important part of this project was the creation of a
software infrastructure that provided a uniform authentication,
scheduling, and information service \[7\]. Both the I-WAY and its
software infrastructure proved remarkably effective, supporting some 60
application groups in a wide variety of domains.

Inspired in part by the success of the I-WAY effort, a number of new
Grid-related initiatives emerged in the late 1990s, including the Globus
project \[9\] and its Globus Ubiquitous Supercomputing Testbed
Organization \[4\], the National Computational Science Alliance\'s
National Technology Grid \[17\], and most recently the NASA Ames
Information Power Grid \[11\] and DOE ASCI Grid. These and other related
efforts are engaged in prototyping and deploying a national-scale Grid
infrastructure designed to support next-generation applications.

# Grid Technologies

The experience gained during the I-WAY experiment and in subsequent
research, development, and deployment activities has resulted in a good
understanding of the technologies required to support advanced
applications. These technologies build heavily on concepts and standards
developed within the Internet community but extend their scope in three
respects, to address (1) remote access to and sharing of the end-systems
(computers, storage systems, etc.) that Grid applications must deal
with, (2) the need for extremely high performance often encountered in
Grid applications, and (3) the truly distributed, multi-party---rather
than client-server---interactions often encountered in Grid
applications. Key ideas underlying \`\`Grid architecture\'\' include the
following:

-   *Grid protocols*. Much as the Internet Protocol defines a \`\`lingua
    franca\'\' that allows disparate devices to exchange information,
    the Grid architecture defines a set of protocols for resource
    management, data access, and resource discovery. The Grid Security
    Infrastructure (GSI) defines SSL-based protocols for authentication,
    authorization, and delegation in multi-institutional settings. The
    Grid Resource Allocation and Management (GRAM) protocol builds on
    GSI to provide for authentication, authorization, job submission,
    and computation management, hence enabling secure remote job
    submission. (The Secure Shell protocol represents another access
    mechanism.) The LDAP-based Grid Resource Information Service (GRIS)
    protocol provides for both remote enquiry about resource state and
    registration with index server(s) that provide resource discovery
    services. The FTP protocol is used for data access, with a
    combination of standardized but not widely used features (e.g., GSS
    support for security) and extensions used to address specific
    concerns of Grid environments.

-   *Resource managers*. The definition of this standard protocol suite
    makes it easy to define what we mean by \`\`Grid-enabled
    resources\'\': any resource that supports the protocols just listed
    is Grid-accessible. The \`\`resource managers\'\' that implement the
    protocols can vary greatly in sophistication. For example, a simple
    \`\`storage resource manager\'\' (SRM) might be just a GSI-enabled
    version of a standard FTP server, while a more sophisticated SRM
    might provide additional space reservation, request queuing, request
    time estimation, and other functions.

-   *Grid services*. The definition of standard resource access
    protocols makes it possible to define a variety of higher-level
    services concerned with resource sharing across virtual
    organizations. For example, Grid Information Index Services (GIIS)
    leverage GRIS capabilities to provide resource discovery services
    for a collection of distributed resources, while replication
    services support the replication of data across multiple storage
    systems and network caches.

-   *Grid tools*. The broad deployment of Grid protocols and services
    makes it feasible to create Grid tools, i.e., tools designed to ease
    the development of various classes of application. Examples of such
    tools include Condor-G \[13\] and Nimrod-G \[1\], which support
    high-throughput computations involving large numbers of independent
    tasks; tools for data-intensive computing, such as replica
    management tools for creating and selecting from among data
    replicas; collaborative environment tools, for managing the sharing
    of state information by large communities; \`\`Science Portal\'\'
    tools for creating desktop gateways to Grid resources; and
    distributed computing tools such as MPICH-G for writing applications
    that exploit Grid resources.

This layered architecture enables individual resources and sites to
participate in Grid applications with relatively little effort
(resources just need to speak a few simple protocols). At the same time,
broad deployment of protocols and services greatly simplifies the task
of developing higher-level tools and applications.

# Production Grids

Until recently, a significant obstacle to Grid computing was the fact
that the protocols and services were not deployed in any persistent
manner and hence could not be taken for granted. Several programs have
been launched in the last year aimed at the creation of \`\`production
Grids:\'\' that is, Grid infrastructures that are persistent, are
supported, and that extend to a significant number of resources found
interesting by their target user community. These different efforts each
have a different focus, but all build on the core Grid services
identified in the preceding section.

The U.S. National Science Foundation (NSF)\'s Partnerships for Advanced
Computational Infrastructure (PACIs), the NCSA Alliance, headed up by
NCSA in Illinois and NPACI, headed up by SDSC in California, both have
ambitious Grid deployment activities underway, with the term
\`\`National Technology Grid\'\' being used as an umbrella term for the
target environment.

NCSA Alliance activities center around its Virtual Machine Room (VMR),
Access Grid (AG), and Science Portal projects. The VMR is intended to
link the major computational resources across the Alliance into a single
integrated computing system, with uniform access mechanisms as well as
specialized resource discovery and brokering services. At the time of
writing, a first version of the VMR is operational: selected Grid
services are in place and various support services (Help Desk,
Certificate Authority) have been established. However, much remains to
be done before the long-term goal of truly seamless access to the
Alliance\'s resources is achieved.

Work at NPACI emphasizes the deployment of Grid services on NPACI
resources and the creation of Science Portals as a means of facilitating
access. Their HotPage system is a nice example of a Web-based Portal and
in its latest incarnation makes extensive use of Grid services for
remote access and monitoring.

NASA\'s Information Power Grid (IPG) project was started in 1998 with
the ambitious goal of integrating Grid computing into the practice of
science and engineering within NASA. Based at NASA Ames, the IPG is
simultaneously creating a production Grid infrastructure and starting
new initiatives relating to tools and applications. Persistent Grid
services and associated accounting, support, and Certificate Authority
services are in place at four NASA site: NASA Ames, Glenn, Goddard, and
Langley.

The U.S. Department of Energy\'s Office of Defense Program\'s ASCI
program is developing a tri-lab Grid as part of its DISCOM project.
DISCOM builds once more on the Globus software used in the other efforts
listed here, but to comply with DOE security policies relies on Kerberos
for authentication. The switch to Kerberos is straightforward because
the Globus Toolkit uses the Generic Security Services (GSS) API for all
authentication operation.

Reseachers within the U.S. Department of Energy\'s Office of Science
have proposed the development of a DOE Science Grid that would link
major DOE Science laboratories and collaborators, providing both
standard Grid services and advanced experimental services such as
network quality of service. Some preliminary work has been done in this
area, with the creation of the Globus Advance Reservation Network
Testbed (GARNET) linking systems at ANL, LBNL, and other institutions.

# Applications

We review briefly a number of applications chosen to be representative
of the wide range of areas in which Grid concepts are being pursued.

*Collaboration*. CAVERNsoft \[12\] and Access Grid \[5\] represent two
different technologies concerned with enabling large-scale collaborative
work by geographically distributed communities. CAVERNsoft emphasizes
support for collaborative manipulation of shared virtual spaces within
immersive virtual reality environments, while Access Grid is concerned
with communication and information sharing in the context of large-scale
(wall-sized) shared display systems.

*Tele-instrumentation*. Foster et al. \[8\] describe a system for the
collaborative, online analysis of experimental data from the Advanced
Photon Source, a high-brilliance X-ray source. This system uses Grid
protocols and services to acquire dynamically the supercomputer
resources required for online reconstruction of APS data and the
advanced visualization systems needed for subsequent incremental
dissemination of reconstructed data to remote collaborators. The result
is that a batch-mode scientific instrument is transformed into an
interactive device.

*Portals*. HotPage and ECCE\' are just two examples of the many
\`\`Portals\'\' and \`\`problem solving environments\'\' that have been
developed that enable desktop access to Grid resources. In the HotPage
portal, the focus is on providing uniform Web access to the diverse
supercomputer resources of a national collaboration, the National
Partnership for Advanced Computational Infrastructure; in the ECCE\'
problem solving environment for computational chemistry, the remote
supercomputers are essentially invisible, being called upon by ECCE\'
when a user requests a computationally demanding calculation.

*Distributed computing*. A Caltech-based group has used Grid services to
assemble a total of 13 supercomputers at 11 sites that were together
used to perform a record-breaking distributed interactive simulation
(DIS) computation \[16\]. An Argonne-Iowa-Northwestern-Wisconsin group
used the Condor-G system to solve a challenging open problem in
numerical optimization, accumulating a total of 96,000 node hours during
a seven-day period, peaking at over 1000 processors at 7 sites
worldwide. In both cases, the Grid increased resources available to
researchers by an order of magnitude.

# Future Directions

We can expect the future to see rapid and sometimes startling progress
in Grid technologies, with major advances and changes occurring not only
within the scientific and engineering communities that have pioneered
many of these ideas, but also within the commercial space.

In science and engineering, one major focus for many researchers over
the next several areas will be the development of the technology base
required to support large-scale data-intensive computing in distributed
environments. One significant driving force for this work will be the
large quantities of data to be produced by frontier science experiments
(e.g., at the Large Hadron Collider at CERN) as well as major simulation
efforts (e.g., in climate). Communities of thousands of scientists,
distributed globally and served by networks of varying bandwidths, need
to extract small signals from enormous backgrounds via computationally
demanding analyses of datasets that will grow from the 100 Terabyte to
the 100 Petabyte scale over the next decade. The computing and storage
resources required will be distributed, for both technical and strategic
reasons, across national centers, regional centers, university computing
centers, and individual desktops.

Numerous research groups and projects are pursuing the design,
development, and application of so-called "Data Grid" technologies. For
example, the SDSC Storage Resource Broker (SRB) \[3\] provides an
integrated framework for metadata management, data access, and analysis;
the Globus Data Grid project is developing basic mechanisms for
high-speed transport, replica management, and other functions \[2\].
Projects such as the Earth System Grid, Particle Physics Data Grid, and
European Union Data Grid projects are developing and applying key
technologies.

The Grid Physics Network (GriPhyN) project (www.griphyn.org) is another
large effort focused on realizing the technical concept of "virtual
data." GriPhyN uses the term *virtual data grid* as a unifying concept
to describe the new technologies required to support such
next-generation data-intensive applications. We use this term to capture
the following unique characteristics:

-   A virtual data grid has *large extent*---national or worldwide---and
    *scale*, incorporating large numbers of resources on multiple
    distance scales.

-   A virtual data grid is more than a network: it layers sophisticated
    *new services* on top of local policies, mechanisms, and interfaces,
    so that geographically remote resources can be used in a coordinated
    fashion.

-   A virtual data grid provides a new degree of *transparency* in how
    data-handling and processing capabilities are integrated to deliver
    data products to end-user applications, so that requests for such
    products are easily mapped into computation and/or data access at
    multiple locations. (This transparency is needed to enable
    optimization across diverse, distributed resources, and to keep
    application development manageable.)

These characteristics combine to enable the definition and delivery of a
potentially unlimited virtual space of data products derived from other
data. In this virtual space, requests can be satisfied via direct
retrieval of materialized products and/or computation, with local and
global resource management, policy, and security constraints determining
the strategy used. The concept of *virtual data* recognizes that all
except irreproducible raw experimental data need 'exist' physically only
as the specification for how they may be derived. The grid may
instantiate zero, one, or many copies of derivable data depending on
probable demand and the relative costs of computation, storage, and
transport. In high-energy physics today, over 90% of data access is to
derived data. On a much smaller scale, this dynamic processing,
construction, and delivery of data is precisely the strategy used to
generate much, if not most, of the web content delivered in response to
queries today.

In the commercial space, the rise of the Application Service Provider
(ASP) and of Internet Computing---that is, the exploitation for
computational purposes of the millions of often idle CPUs on the
Internet---are two significant trends. ASPs seem likely to have a
revolutionary effect on many aspects of the computer industry,
transforming today\'s business relationships so that users interact
exclusively with ASPs rather than with the software and hardware vendors
they have dealt with in the past. This trend has the potential to reduce
significantly current barriers to the use of advanced simulation
technologies, if only user interface and support issues can be
addressed. Current activities in the science and engineering arena
concerned with \`\`Science Portals\'\' can be viewed as precursors of
this trend.

Finally, Internet Computing has risen to prominence as a result firstly
of the several \`\`volunteer\'\' efforts that have delivered large
number of cycles to various highly parallel problems (e.g., SETI@Home)
and more recently as a result of commercial endeavors such as
Distributed.Net and Entropia.Com. While it is too early to say what
range of applications will prove amenable to highly distributed
execution, the possibilities for a transforming impact on science as a
result of order-of-magnitude reductions in computing costs are
significant.

# Acknowledgments

I gratefully acknowledge discussions with many colleagues on these
topics, in particular Carl Kesselman, Steven Tuecke, Bill Johnston, and
Rick Stevens. This work was supported in part by the Mathematical,
Information, and Computational Sciences Division subprogram of the
Office of Advanced Scientific Computing Research, U.S. Department of
Energy, under Contract W-31-109-Eng-38; by the Defense Advanced Research
Projects Agency under contract N66001-96-C-8523; by the National Science
Foundation; and by the NASA Information Power Grid program.

# Bibliography

> 1\. Abramson, D., Sosic, R., Giddy, J. and Hall, B. Nimrod: A Tool for
> Performing Parameterised Simulations using Distributed Workstations.
> in *Proc. 4th IEEE Symp. on High Performance Distributed Computing*,
> 1995.
>
> 2\. Allcock, B., Bester, J., Chervenak, A.L., Foster, I., Kesselman,
> C., Nefedova, V., Quesnel, D. and Tuecke, S., Efficient Data Transport
> and Replica Management for High-Performance Data-Intensive Computing.
> in *Mass Storage Conference*, (2001).
>
> 3\. Baru, C., Moore, R., Rajasekar, A. and Wan, M. The SDSC Storage
> Resource Broker. in *Proceedings of CASCON\'98 Conference*, 1998.
>
> 4\. Brunett, S., Czajkowski, K., Fitzgerald, S., Foster, I., Johnson,
> A., Kesselman, C., Leigh, J. and Tuecke, S. Application Experiences
> with the Globus Toolkit. in *Proc. 7th IEEE Symp. on High Performance
> Distributed Computing*, 1998, 81-89.
>
> 5\. Childers, L., Disz, T., Olson, R., Papka, M.E., Stevens, R. and
> Udeshi, T. Access Grid: Immersive Group-to-Group Collaborative
> Visualization. in *Proceedings of the Fourth International Immersive
> Projection Technology Workshop 2000*, to appear.
>
> 6\. DeFanti, T., Foster, I., Papka, M., Stevens, R. and Kuhfuss, T.
> Overview of the I-WAY: Wide Area Visual Supercomputing. *International
> Journal of Supercomputer Applications*, *10* (2). 123-130.
>
> 7\. Foster, I., Geisler, J., Nickless, W., Smith, W. and Tuecke, S.
> Software Infrastructure for the I-WAY Metacomputing Experiment.
> *Concurrency: Practice & Experience*, *10* (7). 567\--581.
>
> 8\. Foster, I., Insley, J., Laszewski, G.v., Kesselman, C. and
> Thiebaux, M. Distance Visualization: Data Exploration on the Grid.
> *IEEE Computer*, *32* (12). 36-43.
>
> 9\. Foster, I. and Kesselman, C. Globus: A Toolkit-Based Grid
> Architecture. in *The Grid: Blueprint for a Future Computing
> Infrastructure*, 1998, 259-278.
>
> 10\. Foster, I. and Kesselman, C. (eds.). *The Grid: Blueprint for a
> Future Computing Infrastructure*, 1999.
>
> 11\. Johnston, W.E., Gannon, D. and Nitzberg, B. Grids as Production
> Computing Environments: The Engineering Aspects of NASA\'s Information
> Power Grid. in *Proc. 8th IEEE Symp. on High Performance Distributed
> Computing*, 1999.
>
> 12\. Leigh, J., Johnson, A. and DeFanti, T.A. CAVERN: A Distributed
> Architecture for Supporting Scalable Persistence and Interoperability
> in Collaborative Virtual Environments. *Virtual Reality: Research,
> Development and Applications*, *2* (2). 217-237.
>
> 13\. Litzkow, M. and Livny, M. Experience With The Condor Distributed
> Batch System. in *IEEE Workshop on Experimental Distributed Systems*,
> 1990.
>
> 14\. Lyster, P., Bergman, L., Li, P., Stanfill, D., Crippe, B., Blom,
> R., Pardo, C. and Okaya, D. CASA Gigabit Supercomputing Network:
> CALCRUST Three-dimensional Real-Time Multi-Dataset Rendering. in
> *Proc. Supercomputing \'92*, 1992.
>
> 15\. Messina, P. Distributed Supercomputing Applications. in
> Kesselman, I.F.a.C. ed. *The Grid: Blueprint for a Future Computing
> Infrastructure*, 1998, 55-73.
>
> 16\. Messina, P., Brunett, S., Davis, D., Gottschalk, T., Curkendall,
> D., Ekroot, L. and Siegel, H. Distributed Interactive Simulation for
> Synthetic Forces. in *Proceedings of the 11th International Parallel
> Processing Symposium*, 1997.
>
> 17\. Stevens, R., Woodward, P., DeFanti, T. and Catlett, C. From the
> I-WAY to the National Technology Grid. *Communications of the ACM*,
> *40* (11). 50-61.