Multi-Sensor Distributive On-line Processing, Visualization and Analysis
System

S. Berrick, G. Leptoukh, Z. Liu, L. Pham, H. Rui, S. Shen, W. Teng, T.
Zhu

Goddard Earth Sciences Data and Information Services Center, Code 902,
NASA GSFC, Greenbelt, Maryland 20771, USA

*Abstract*--- The ability to use data stored in the current Earth
Observing System (EOS) archives for studying regional or global
phenomena is highly dependent on having a detailed understanding of the
data\'s internal structure and physical implementation. Gaining this
understanding and applying it to data reduction is a time-consuming task
that must be undertaken before the core investigation can begin. This is
an especially difficult challenge when science objectives require users
to deal with large multi-sensor data sets that are usually of different
formats, structures, and resolutions. The NASA Goddard Earth Sciences
Data and Information Services Center (GES DISC) has taken a major step
towards meeting this challenge by developing an infrastructure that
supports a Web interface that allows users to perform interactive
analysis online without downloading any data. The underlying
infrastructure that integrates all the components is called GES-DISC
Interactive Online Visualization and Analysis Infrastructure or
\"Giovanni.\" Giovanni provides interactive, online, analysis tools for
data users to facilitate their research.

Keywords-distributed data analysis; data visualizatio;, Web
applications; online

#  Introduction

Doing regional or global investigations using Earth science data sets
from multiple sources can be a daunting task. It involves locating and
then obtaining voluminous amounts of data, subsetting to extract the
parameters and spatial areas desired, and co-locating the data with
other data. This effort may include building and maintaining a data
management system, a processing system, and visualization and graph
generation tools. Thus, there is often a significant upfront investment
before the core investigation can begin. In order to ease the burden of
such investigations among the Earth science community, the Goddard Earth
Science (GES) Data and Information Services Center (DISC) has undertaken
the development of the GES-DISC Interactive Online Visualization and
Analysis Infrastructure or Giovanni. The goal of Giovanni is to relieve
the science investigator of some of the upfront data preparation work
and provide a tool for obtaining science knowledge via plots, graphs,
and tables without having to download and prepare large amounts of data.

To accomplish this goal, Giovanni provides a Web interface to data,
analysis, and data visualization tools, all without having to download
data. The graphs and plots generated by Giovanni are suitable for
publication. If additional analyses are required, the original data can
be easily retrieved. The wealth of multi-mission data at the GES DISC
makes Giovanni an ideal environment for regional and global studies.

# Features of Giovanni 

Current features of Giovanni include:

-   Access to data from multiple remote sites as well as local sites

-   Server-side temporal and spatial subsetting

-   Server-side processing

-   Support for multiple data formats including Hierarchical Data Format
    (HDF), HDF-EOS, network Common Data Form (netCDF), GRIdded Binary
    (GRIB), and binary.

-   Support for multiple plot types including area, time, Hovmoller, and
    image animation

-   Support for outputting data in ASCII format

-   Easily configurable to support customized portals for
    measurements-based projects or disciplines

Who benefits from Giovanni? Science users who are interested in global
or long-term regional studies where coarse spatial resolution is
adequate, modelers, and college professors teaching students can all
benefit from Giovanni.

# Design Goals

The principal design goal for Giovanni was to provide a quick and simple
interactive means for science data users to study various phenomena by
trying various combinations of parameters measured by different
instruments, arrive at a conclusion, and then generate graphs suitable
for a publication. Alternatively, Giovanni would provide a means to ask
relevant what-if questions and get back answers that would stimulate
further investigations. This would all be done without having to
download and preprocess large amounts of data. A secondary design goal
was for Giovanni to be easily configurable, extensible, and portable.
The GES DISC currently runs Giovanni on Linux, SGI, and Sun platforms.

Another goal of Giovanni was to off-load as much as possible the data
processing workload onto the machines hosting the data and to reduce
data transfers to a minimum.

Given the enormous amount of HDF data at the GES DISC Distributed Active
Archive Center (DAAC), it was a requirement that Giovanni support HDF,
HDF-EOS, as well as binary.

# Giovanni Data Flow

Giovanni consists of HTML templates, CGI scripts written in Perl and in
Grid Analysis and Display System (GrADS) language. In addition, there is
an image map Java applet through which a user can select a bounding box
area to process. Access to data is via one or more GrADS-DODS
(Distributed Oceanographic Data System) Servers (GDS) running on remote
machines that have GrADS readable data.The data flow is illustrated in
Fig. 1.

GrADS was chosen for its widespread use for providing easy access,
manipulation, and visualization of Earth science data. It supports a
variety of data formats such as binary, GRIB, netCDF, HDF, and HDF-EOS.
When combined with DODS, as in GDS, the result is a secure data server
that provides subsetting and analysis across the network or even the
Internet. The ability of GDS to subset data on the server drastically
reduces the amount of data that need to be transferred across the
network and improves overall performance. GDS provides spatial or
temporal subsetting of data while applying any of a number of analysis
operations including basic math function, averages, smoothing,
correlation, and regression. An equally important feature is the ability
to run GrADS data transformations on the server.

## Science Users Select Parameters

Via the Giovanni Web interface, the user selects one or more data sets,
the spatial area, the temporal extent, and the type of output. Supported
output types are listed in Table 1. The selection criteria are passed to
the CGI scripts for processing.

## GrADS Makes Request for Subsetted Data

CGI scripts process the parameters submitted by the user. GrADS scripts
are then used to formulate GDS data requests. These requests include
parameters by which the data can be subsetted and processed. The
requests are directed to the appropriate GDS machine.

## GDS Processes the Data Requests

GDS on a remote machine receives the request for data along with
subsetting and processing parameters. The requested subsetting and
processing is performed on the server and the resulting subsetted data
are sent back to the client (Giovanni).

## CGI Passes the Output Back To the User

The subsetted data are processed by the CGI scripts on the Giovanni
machine into an image (for plots or graphs) or into ASCII data that the
user can download. For plots or graphs, the results will be displayed in
the Web browser.

## ![](media/image1.png){width="3.1597222222222223in" height="4.146527777777778in"} {#section .unnumbered}

##  {#section-1 .unnumbered}

##  {#section-2 .unnumbered}

##  {#section-3 .unnumbered}

##  {#section-4 .unnumbered}

##  {#section-5 .unnumbered}

##  {#section-6 .unnumbered}

##  {#section-7 .unnumbered}

##  {#section-8 .unnumbered}

##  {#section-9 .unnumbered}

##  {#section-10 .unnumbered}

##  {#section-11 .unnumbered}

##  {#section-12 .unnumbered}

##  {#section-13 .unnumbered}

##  {#section-14 .unnumbered}

1.  Giovanni data flow diagram.

Table 1 describes the types of output currently supported by Giovanni.

I.  Output Types Supported By Giovanni

  ----------------------------------- -----------------------------------
  Output Type                         Description

  Area Plot                           Area plot averaged or accumulated
                                      over any available data period
                                      within any rectangular area

  Time Plot                           Time series averaged over any
                                      rectangular area

  Hovmoller Plots                     Longitude-time and latitude-time
                                      plots

  Animations                          Animations available for area plots

  ASCII Output                        ASCII output, available for all
                                      plot types, suitable for feeding
                                      GIS or other applications
  ----------------------------------- -----------------------------------

A Giovanni implementation is supported by a generation script (also
Perl) and a number of configuration files. The configuration files are
hierarchical in nature and they completely define a Giovanni
implementation.

The Giovanni generation script is used to generate or modify a Giovanni
deployment based upon configuration file settings. The generation script
will modify existing HTML files and create new ones as needed. The
resulting Giovanni can then be installed into a test string or into an
operational environment.

# Giovanni User Interface

From the user\'s perspective, Giovanni is a simple Web application. The
main Web page contains a title, a brief description of the site, and a
descriptive list of one or more Giovanni deployments available. The
elements on this initial Web page are defined in the Giovanni global
configuration file. A user can select either the Java or non-Java
version.

The resulting Web page is one that allows the user to select the spatial
area via the Java image map applet or, if the non-Java version was
selected, manually by entering in coordinates defining a bounding box.
The user also selects the temporal range of the data, one or more
parameters from this data set, and the output type (ASCII or one of
several plot types). For plots, several color options are also
available.

Fig. 2 shows the Web interface for the Ocean Color Giovanni running at
the GES DISC for the Sea-viewing Wide Field-of-view Sensor (SeaWiFS)
Monthly Global 9-km Chlorophyll product. It illustrates the user
interface and the available options.

Once the Giovanni Web page options are selected, the user has the option
of generating a plot or outputting the results to an ASCII file that can
be downloaded. ASCII output is useful for GIS or other user
applications. If plotting is selected, another Web browser window is
opened within which the plot is displayed. A sample area plot is shown
in Fig 3 for the case of SeaWiFS monthly global 9-km data. Links to the
data are provided so the user can download the entire data set.

Depending upon the parameters selected, the vast majority of users will
see the results in a matter of seconds. For users who choose large
amounts of data either spatially or temporally, the results may take
several minutes.

![](media/image2.png){width="3.157638888888889in"
height="3.220833333333333in"}

2.  Example Web page of Giovanni for SeaWiFS Monthly Global 9-km
    Chlorophyll

![](media/image3.png){width="3.1631944444444446in"
height="3.252083333333333in"}

3.  Example Giovanni area plot. This one shows increased productivity
    corresponding to the Tehuano (south of Mexico) and Papagayo (west of
    Nicaragua) wind jets in Central America region using Chlorophyll
    concentration measurements by SeaWiFS on January 2004.

#  Current Implementations

At the GES DISC, several instances of Giovanni are currently supporting
our data users. Access to these Giovanni instances is available from the
GES DISC Web page at: http://disc.gsfc.nasa.gov/.

## MOVAS

The MODIS (Moderate Resolution Imaging Spectroradiometer) Online
Visualization and Analysis System (MOVAS) has been operational since
September 2003 providing Giovanni analysis tools to users of the MODIS
Terra atmospheres monthly global product (MOD08_M3). Data are available
from April 2000 to the present.

MOVAS allows scientists and researchers to easily access, visualize and
analyze MODIS Level-3 atmospheric monthly products, thus helping them to
understand seasonal-to-interannual variation of atmospheric parameters
ranging from aerosol to water vapor. MOVAS can provide information at
every single point and in any rectangular area within the data domain,
which allows researchers to conduct nearly unlimited investigations. The
ASCII output option allows users to do their own customized analyses or
applications, such as, GIS.

## TOVAS

TOVAS is the TRMM Online Visualization and Analysis System (TOVAS),
based primarily on data from the Tropical Rainfall Measuring Mission
(TRMM). TOVAS has been operational since March 2001 providing Giovanni
analysis tools to users of (1) three hourly TRMM and Other Satellite
Rainfall (3B42RT) for Jan 2002-present; (2) TRMM Level-3 Daily Rainfall
(3B42) for Jan 1998-present; (3) TRMM Level-3 Monthly Rainfall (3B43)
for Jan 1998-present; (4) Willmott and Matsuura Global Precipitation for
Jan 1950-Dec 1999; and (5) Global Precipitation Climatology Center
(GPCC) Monthly Global Precipitation for Jan 1986-present.

Like MOVAS, TOVAS is another powerful tool that allows users to fully
explore data. TOVAS makes TRMM and other gridded precipitation data
available in a format that anyone can learn to use within minutes and
put to work productively in research or applications. With few mouse
clicks, one can easily obtain precipitation information around the
world. Recent applications of TOVAS include:

-   Study of coastal urban heat island effect on rainfall.

-   Additional rainfall information to supplement ground stations in Sri
    Lanka.

-   Phenology study in Africa and North America.

-   Crop yield estimates and flood watch in Africa and Asia.

-   Rainfall information for a development project in Afghanistan.

-   Fire monitoring activities in Africa.

-   Data for hydrological modeling in Africa.

-   Range prediction of American butterflies.

-   Intercomparison with other products in North America.

-   Monitoring rain events in the Balkans.

-   Investigation of the 1997-1998 El Niño/La Niña event.

-   Investigation of insect activities in the US.

## OOVAS

The Ocean Color Online Visualization and Analysis System (OOVAS)
recently became available operationally. It provides users access to
SeaWiFS and MODIS Aqua global monthly chlorophyll data from the start of
missions. It has been developed to support the Ocean-Color Time-Series
funded by the NASA. The goals of this project are to develop and
maintain a consistent multi-decadal time series of ocean color data, and
to develop and maintain simplified user access and support for the time
series that spans missions. Giovanni serves these goals perfectly as it
allows access to long time-series information without downloading data.

# Future Directions

In the near-term, Giovanni will add support for doing correlations among
parameters. An important future direction for Giovanni to take is full
support of output formats suitable for Geographic Information Systems
(GIS), for example GeoTIFF. Giovanni will also need to better support
multi-instrument analyses with smart handling of multiple grids. Other
directions include the ability to represent errors due to missing data
and data quality in meaningful ways.

##### Acknowledgment {#acknowledgment .unnumbered}

The GES DISC wishes to thank Dr. Yoram Kaufman (NASA GSFC) for his
financial and scientific support of the MODIS Online Visualization and
Analysis System (MOVAS) development effort. The Ocean Color Online
Visualization and Analysis system (OOVAS) was supported by NASA HQ and
Dr. Watson Gregg (NASA GSFC) through REASoN CAN 42-OES-01.