### Simple, Scalable, Script-Based Science Processor (S4P)

Christopher Lynnes, Bruce Vollmer, Stephen Berrick, Robert Mack, Long
Pham, and Bryan Zhou

NASA Goddard Space Flight Center

#### Code 902, Greenbelt, MD, 20771

*Abstract*- The development and deployment of data processing systems to
process Earth Observing System (EOS) data has proven to be costly and
prone to technical and schedule risk. Integration of science algorithms
into a robust operational system has been difficult. The core processing
system, based on commercial tools, has demonstrated limitations at the
rates needed to produce the several terabytes per day for EOS, primarily
due to job management overhead. This has motivated an evolution in the
EOS Data Information System toward a more distributed one incorporating
Science Investigator-Led Processing Systems (SIPS). As part of this
evolution, the Goddard Earth Sciences Distributed Active Archive Center
(GES DAAC) has developed a simplified processing system to accommodate
the increased load expected with the advent of reprocessing and launch
of a second satellite. This system, the Simple, Scalable, Script-Based
Science Processor (S4P) may also serve as a resource for future SIPS.

I. INTRODUCTION

Science data processing systems range from simple sets of scripts to
large, complex systems with powerful planning and scheduling
capabilities. The Earth Observing System Data Information System
(EOSDIS) Core System (ECS), used by the Goddard Earth Sciences
Distributed Active Archive Center (GES DAAC) to process data from the
Moderate Resolution Imaging Spectroradiometer (MODIS), falls into the
latter category. It includes several hundred thousand lines of custom
code, together with a commercial planner/scheduler and relational
database. The system has been effective processing the data from the
Terra satellite, which corresponds to about 1260 jobs per day. Over the
next three years, however, this number will increase by a factor of 6 as
the Aqua satellite is launched and reprocessing is undertaken for each
satellite at double the data rate, resulting in about 7500 jobs per day.
The performance overhead of the processing system would require
expensive upgrades to handle the increased load.

Instead, the GES DAAC is in the process of replacing the ECS data
processing system with a system from the other end of the complexity
system, the Simple, Scalable Script-based Science Processor, also known
as S4P. S4P is a set of Perl scripts that implements a data-driven
system: when data arrives, processing is triggered. S4P implements a
factory assembly-line paradigm: it consists of "stations", implemented
as UNIX directories with a configuration file and monitored by a daemon
called "stationmaster". A processing job in a given station is triggered
when the stationmaster detects arrival of a "work order", actually an
ASCII file with information about input files. Output work orders are
shipped to downstream stations when processing is complete. A simple
graphical user interface (also written in Perl) can be used for
monitoring stations and jobs, as well as troubleshooting and controlling
jobs (Figure 1). S4P is already in use within the GES DAAC for
supporting data mining \[1\], Quick-Response MODIS processing, and MODIS
Direct Broadcast processing. Its most recent application is for the
reprocessing of MODIS Terra data.

II\. IMPLEMENTATION

An S4P implementation actually consists of two main parts: the S4P
infrastructure (stationmaster, station monitor, support libraries, and
scripts) common to various S4P systems and the station configuration and
scripts specific to a particular S4P system. The system to replace the
ECS data processing system is known as S4P-MODIS, or S4PM.

![](media/image1.png){width="3.3243055555555556in"
height="2.8055555555555554in"}

Fig. 1 Graphical User Interface for S4PM

Figure 1 shows the S4P monitoring graphical user interface for the S4PM
implementation. The stations are the buttons in the left column.
Clicking on a station name brings up another graphical user interface
that allows for control of the station. Jobs within a station are the
boxes. A box with the hourglass is a running job, a box with a question
mark is a pending job, and a box with a circle and a line through it
represents a failed job. More typically, colors are used to distinguish
these job types (green for running jobs, blue for pending jobs, and red
for failed jobs). Clicking on a job box indicating failure brings up a
graphical user interface that displays any debris left behind by the
failure. A 'restart' button allows one to restart the failed job once
the problem has been resolved. Other job handlers can easily be added.

III\. S4PM STATION ARCHITECTURE

For S4PM, the typical scenario is as follows:

(1) If reprocessing data, the Production Planner uses a simple graphical
    user interface to select raw (Level 0) MODIS and ancillary data for
    processing. A request is submitted to ECS for the data via the
    Request Data station. If processing current data instead of
    reprocessing, this step is unnecessary. The data are instead shipped
    automatically by subscription.

(2) The ECS stages the data to S4PM and sends a Distribution Notice (DN)
    when the data are ready.

(3) The Data Catcher station checks the metadata for the input files and
    renames them according to a common convention used within S4PM. The
    files are also registered with the Granule Tracking station. Each
    one is assigned a number of \"uses\", corresponding to the number of
    times it is to be used for any purpose, such as processing or saving
    to the archive.

(4) The Specify Data station puts together a list of all of the
    additional data needed to process the input files.

(5) The Find Granule station locates the files corresponding to that
    additional data.

(6) The Trigger Algorithm station creates a Process Control File which
    is used by the science algorithms to provide information about the
    input files, output files, and runtime parameters. It leaves,
    however, the directory names of the output files blank. These will
    be filled in by the Allocate Disk station.

(7) The Allocate Disk station allocates space for the output files and
    fills in the directory names.

(8) The Run Algorithm station actually runs the science processing
    algorithm. If successful, it sends a list of all the output files to
    the Data Catcher station, which looks for the locally produced data
    and starts the process over again for the next science algorithm in
    the chain. It also sends a list of the *input* files to the Granule
    Tracking station so that the number of uses can be decreased
    appropriately. Finally, it sends a list of the output files to the
    Export station.

(9) The Export station tells ECS to pick up the algorithm output files
    and archive them.

(10) When ECS has archived the output files, it sends back a Product
     Acceptance Notice. The Receive Product Acceptance Notice station
     sends a list of the saved files to Granule Tracking.

(11) If Granule Tracking indicates that a granule is all \"used up\",
     that is., the number of remaining uses is 0, it sends a work order
     to the Clean Data station, which removes the files from disk and
     frees up any allocated disk space.

One useful aspect of S4PM is that it can run on the same hardware as
other processing systems, so long as each system has a dedicated
allocation of resources, such as disk and CPU. S4PM has been
reprocessing MODIS Terra data since early June 2001. Once S4PM has
proven itself in this task, the near-real-time processing of Terra will
be transitioned over and by the time Aqua launches, it will be handling
all of the GES DAAC\'s processing of MODIS data.

![](media/image2.wmf)

Fig. 2 S4PM Station Architecture

REFERENCES

1.  C. Lynnes and R. Mack, "KDD Services at the Goddard Earth Sciences
    Distributed Active Archive Center," in *Data Mining for Scientific
    and Engineering Applications*, unpublished.