# Presentation: 299600

## International EOS/NPP Direct Readout Meeting

**THE MARSEC **
**STATION**

- Presented by Giuseppe Meoli
- giuseppem@marsdb.it

## Outline

**1)**  **Overview of the MARSEC Station**
*Acquisition System*
*Scientific Areas*
*Collaborations*
*Activities*

**2) Presentation on  IMAPP/FRAPPE**
*Data Processing Software*
*Hardware*
*Parallel processing*
*General Considerations*
*System Architecture*
*Implementation*
*Near Real Time web product images*
*Processing Times*
*Future Development*

## The Station

- Mediterranean Agency for Remote Sensing and Environmental Control (MARSEC)
- is a direct readout station located in the city of Benevento Italy
- (about 1hr from Naples airport Capodichino)

## The Acquisition Systems

- The station has two antennas (built by “SeaSpace”) for direct broadcast in X-Band and L-Band;

- coverage of the station includes the Mediterranean area: **Europe**, **North Africa** and part of **Middle East**.

## Location

**Location**

- MARSEC is located in a characteristic historical palace of XVII sec, Villa dei Papi, so called  because originally it was one of the Pope’s summer residences. This building, surrounded by a nice garden, was built in 1696 by Pope Innocent XI and Pope Benedict XIII.

## X-Band Receiving System

- Acquisition system: a SeaSpace X-Band antenna mounted on a 10 meters trellis to avoid the shadowing caused by a pre-existent structure (Villa dei Papi) and a SeaSpace L-Band antenna mounted on the roof of the building where the departments and the processing room of the Mediterranean Agency are located;
- X-Band receiver system (frequency range 8.0 - 8.5 GHz) has a 4.5m tracking antenna mounted  inside a protective fiberglass radome;

- X-band receiver system is a Alcatel Omnisat demodulator  charcterized by multimission versatility, data rates up to 329 Mbit/s with low  degradation; it is interfaced with a Linux pc on which the Terascan processing is executed.

## L-Band Receiving System

## Activities 

**MARSEC** is a multidisciplinary center for an integrated monitoring of the environment. Particular emphasis is given to the integration of the following activities:
-   Remote sensing activities with EOS data
-   land products;
-   atmospheric products;
-   fire detection;
-   oceanic products (next in line).
-   Other activities
-   land slide monitoring (GPS, Total Station);
-   water quality control (Buoys);
-   research on land surface deformations (SAR);
-   services with high spatial resolution visible data (EROS);
-   integrated systems for real time monitoring of electric power
-      consumption in hospitals;
-  outreach.

## Collaborations

- MARSEC has ongoing collaborations with   **SSEC** of the **University of Wisconsin-Madison** and with other institutions for:
- research;
- implementation of parallel processing of IMAPP and NASA DAAC software;
- outreach activities;
- education.

## Research Activities

**Research Activities**

- MARSEC participated  in the European AQUA Thermodynamic Experiment - Italy (**EAQUATE-IT**). This field experiment was held in September 2004 in Italy for the validation of the AIRS instrument on the EOS Aqua satellite;
- 					(_[http:](http://metresearch.net/eaquate/)__[//metresearch](http://metresearch.net/eaquate/)__[.](http://metresearch.net/eaquate/)__[net/eaquate/](http://metresearch.net/eaquate/)_)

- experiment was sponsored by NASA/Integrated Program Office (**IPO**) for NPP/NPOESS risk mitigation with pre-Metop (IASI, AMSU, HIRS). It was an international effort that involved **NASA**, **SSEC UW-MADISON**, **MARSEC**, **IMAA-CNR**, **University of Basilicata**, **University of Bologna**;
- experiments were performed September 6 through 10, 2004, using the Scaled Composites Proteus aircraft.

## International EOS/NPP Direct Readout Meeting

**M***editerranean* **A***gency* *for ***R***emote* **S***ensing* *and* **E***nvironmental* **C***ontrol*:
- development of a parallel processing environment for EOS data

- Presented by _Giuseppe Meoli_
*giuseppem@marsdb.it*

*ec*

## Processing Software Installed at MARSEC

**TERASCAN**

## Implementation of parallel processing

- IMAPP / FRAPPE

## Ingredients for our IMAPP FRAPPE_x0017_

**Ingredients for our IMAPP FRAPPE****_x0017_**

**Software**
**IMAPP**
**NASA-DAAC**

**Cluster**
**Hardware**

**Programming**
**Language**
**Python**

**JVM**
**PBS**

**Operative**
**System**
**ROCKS**

**FRAPPE**
**software**

## IMAPP 

**IMAPP**

**IMAPP**: International MODIS/AIRS Processing Package (IMAPP); is a processing package  for MODIS AIRS data developed at Univ. Wisconsin  _[http://cimss.ssec.wisc.edu/~gumley/IMAPP/IMAPP.html](http://cimss.ssec.wisc.edu/~gumley/IMAPP/IMAPP.html)_
- converts from raw data (**Level-0**) to **geolocated calibrated reflectances and radiances** (Level-1);
- highly successful standalone system to L1 and standalone code for **L0->L1B** and supports the new more frequent calibration parameter updates;
- MODIS **Level 2** geophysical products (Terra or Aqua);
**we Run it on Linux**. Both IMAPP and the released PGE codes are targeted at this platform;
- only toolkit required is **NCSA-HDF4**;
- downlinked spacecraft **ephemeris** and **attitude** data may be used for real-time geolocation.

**http://cimss.ssec.wisc.edu/**

## Cluster Hardware

**Dell cluster** with **7 nodes** PowerEdge 2850: **Dual** Intel Xeon **processors** of **2.8 GHz** with EM64T support, teamed with **2 GB DDR-2 Memory** and PCI Express with support for 32-bit applications as well as 64-bit applications, 800 MHz front  side bus, 1MB cache L2, three PCI-X slots (64-bit/133MHz), DRIVE CONTROL EMBEDDED DUAL CHANNEL ULTRA 320 SCSI, **hard drives 300 GB** (10.000 rpm), external storage Dell PowerVault SCSI and Dell/EMC fibre channel storage, dual embedded Intel Gigabit network interface card, Fedora Core 3 operative system;
**switch Gb  Ethernet** Dell 1000 BaseT;
- 8 TB of Disk Storage (**Fiber Channel**).

## ROCKS

- advantages of Rocks distribution:
**large user and development community** (http://rocks.npaci.edu/);
- capability to maintain up-to-date package by rocks-dist;
- cluster configuration stored in the database;
**web portal (node information is kept in a web-enabled MySQL database, node activities can be viewed and retrieved on web)**;
**modular installation**;
*Globus* components;
**every node has the same configuration** (greater behavioral stability);
- simplified management of the cluster: if a cluster needs to be reconfigured, or individual nodes or a set of nodes behaves unexpectedly solution is restoration of a node or group of nodes to a known stable state.

## Python

- Python is an **interpreted, object-oriented, high-level programming language** with dynamic semantics. Its simple, easy to learn, syntax emphasizes readability and therefore reduces the cost of program maintenance;
**it supports modules and packages**, which encourages program modularity and code reuse. There are interfaces to many system calls and libraries, as well as to various windowing systems (X11, Motif, Tk, Mac, MFC);
- Python interpreter and extensive standard **library are available in source or binary form without charge for all major platforms, and can be freely distributed**;
- Python provides an increased productivity. Since there is no compilation step, the edit-test-debug cycle is very fast (the debugger is written in Python itself, testifying to Python's introspective power);
- Python implementation is portable: it runs on many UNIX systems, on Windows, OS/2, Mac, Amiga, and several other platforms; (_[http://www.python.org/](http://www.python.org/)_)
- Python implementation is copyrighted but freely usable and distributable, even for commercial use.

## Ganglia is a system to monitor groups of computers by using Java Jini technology. (Cluster monitoring systems track status of various components, e.g. memory, I/O, of cluster clients);

information on computing state is written in XML form; 

Ganglia can be used to monitor in Grid because it can be linked to several Ganglia clusters; 

at each client there is a Java service, gmond, which collects data about computer components. Data are saved into an XML document and are sent to a multicast address;

Gmetad module is located in a chosen computer. Gmetad module’s task is to collect data from a chosen group of nodes. In that way, it is possible to establish monitoring of several clusters;

web portal is placed in the same computer as gmetad: Ganglia web portal allows users to show graphs of data which are collected by gmond services. 

- information on computing state is written in XML form;
- Ganglia can be used to monitor *in Grid* because it can be linked to several Ganglia clusters;
- at each client there is a Java service, _*gmond*_, which collects data about computer components. Data are saved into an XML document and are sent to a multicast address;
_*Gmetad*_ module is located in a chosen computer. Gmetad module’s task is to collect data from a chosen group of nodes. In that way, it is possible to establish monitoring of several clusters;
- web portal is placed in the same computer as gmetad: **Ganglia web portal allows users to show graphs of data which are collected by gmond services**.

## PBS

**PBS**  stands for Portable Batch System. It is a networked subsystem for submitting, monitoring, and controlling work load of  batch  jobs  on one  or  more  systems.
- PBS:
- supports an interactive batch mode where the input and output are connected to the user's terminal;
- supports grids and provides interface to the *Globus Grid tool*;
- supports multi-clustering with restrictions on the scheduler;
- claims to **scale to thousands** of **nodes**;
**is distributed as source**;
- is compliant with Posix 1003.2d "Batch Queuing Extensions for Portable Operating  System;”
**supports well known scheduling policies** (Example: FIFO, Fair share, exclusive, etc...);
- allows jobs to be launched from clusters and run on another (Inter-cluster job launching);
- PBS accounting interfaces with NASA's ACCT++ accounting analysis tool.
*For more information*: http://www.openpbs.org

## FRAPPE

- The presented work represents a first-cut implementation of a parallel clustered data processing system for IMAPP and similarly-shaped processing pipelines using inexpensive pieces and techniques;
- Specifically IMAPP has been successfully coupled with the **FRA**mework for **P**ipeline **P**rocessing **E**xperiment (**FRAPPE**) to allow for an efficient implementation of the MODIS  and AIRS processing package on a cluster;
**FRAPPE** is an experimental python library that  provides a **framework** on which it is relatively easy to manage and run **large  processing tasks** on **different systems** (single machines on up through supercomputers) without having to rethink all the logic behind the processing chain;
- in January of this year started experimentation of IMAPP and FRAPPE, both developed at the University of Wisconsin-Madison, at the Mediterranean Agency for Remote Sensing.

## MARSEC MODIS  PROCESSING

**Latest update**: Integration in IMAPP pipeline of function to generate in near real time products images of the observed scene and to update it on MARS web page (_[http:](http://readout.marsdb.it/)__[//readout](http://readout.marsdb.it/)__[.](http://readout.marsdb.it/)__[marsdb](http://readout.marsdb.it/)__[.](http://readout.marsdb.it/)__[it](http://readout.marsdb.it/)_)

**Ghost**_x0003_

**Earth**

**BVX **

**NASA DAAC**
- Serial processing_x0003_

- MODIS
**IMAPP****_x0003_**
- Parallel processing

**MARSEC DIRECT READOUT**** WEB PAGE**

**TAPE STORAGE**

- Storage of all processing output for 1 month

**FIBER CHANNELSTORAGE**

- Storage of level 0

- ASCII e-mail for FIRE

- WEB PAGE update with images of last pass products

- AIRS/AMSRE
**IMAPP****_x0003_**
- Serial processing

- RT-STPS
- ( TIME: 10:10 UTC)

- PASS of 2005 - 10 -03
- TIME: 09:50 UTC
- (this pass is 13 minutes)

- RT-STPS
- ( TIME: 10:10 UTC)

- ( TIME: 10:46 UTC)

- ( TIME: 10:46 UTC)

- ( TIME: 10:23 UTC)

- Archive 10:04 UTC

- PASSDISK
- ( TIME: 10:05 UTC)

- PASSDISK
- ( TIME: 10:05 UTC)

## NASA DAAC

**PASSDIK from bvx **

**Run RT - STPS and GBAD in AQUA case**

**Level 1B calibrated, geolocated data**

**Cloudmask**

**NDVI**

**MAIL for FIRE**

## Parallelization of IMAPP

**Parallelization of IMAPP**

- RT-STPS
- ( TIME: 10:09 UTC)

- TIME processing: 10:19 UTC
- TIME rsync: 10:23 UTC

- TIME processing: 10:24 UTC
- TIME rsync: 10:25 UTC

- TIME processing: 10:31 UTC
- TIME rsync: 10:32 UTC

- TIME processing: 10:37 UTC
- TIME rsync: 10:41 UTC

- TIME processing: 10:30 UTC
- TIME rsync: 10:31 UTC

- TIME processing: 10:39 UTC
- TIME rsync: 10:44 UTC

- TIME processing: 10:45 UTC
- TIME rsync: 10:45 UTC

## IMAPP PRODUCTS AT MARSEC

**IMAPP PRODUCTS AT MARSEC**

**Aerosol Optical depth**
**Total Precipitable Water**
**Cloud Fraction **
**Cloud Top Pressure**
**Cloud Top Temperature**
**Retrieved Moisture Profiles 20mbar**
**Retrieved Moisture Profiles 500mbar**
**Retrieved Moisture Profiles 1000mbar **
**Retrieved Temperature Profiles 20mbar**
**Retrieved Temperature Profiles 500mbar**
**Retrieved Temperature Profiles 1000mbar**
**SST**
**Truecolor (250 - 500 - 1000 m)**
**SWIR**
**Thermal**
**Thermal anomalies**

## NEAR REAL TIME WEB PRODUCT IMAGES

## PERFORMANCE TESTING

- Testing of IMAPP pipeline performance started on March 1, 2005;
- for two months, we tested daily the computing time needed to complete IMAPP real time processing
- on single passes alone;
- along with re-processing of series of different pre-existing passes;
- in both cases, load for each node of the cluster and the presence of eventual errors in the processing were monitored using ***Ganglia*** monitoring system.

##   Example of Ganglia monitoring: 

**Test results**

## Time Test Results

- Single PASS on cluster:
- 14 minutes to finish LEVEL 1 processing;
- 36 minutes to finish  LEVEL 1 and 2 processing;
- 8 archived PASSES + 4 real-time PASSES (on cluster): processing finishes in about 3 and a half hours;
- 8 archived PASSES + 4 real-time PASSES (on single node): 38 minutes X 12 (passes) = about 8 hours;

## Future development

**Improvement processing times**
- using fiber channel storage (up to 15%);
- optimizing disk I/O access (up to ?%);
- implementation of FRAPPE to **parallelize NASA DAAC** processing and **IMAPP AIRS/AMSR-E** processing;
**upgrade bandwidth** to allow for more than one external access to data and products, within a lifetime;
- installation of **OpenADDE**;
**installation **of **new combined MODIS processing** Level 1 package developed to merge NASA DAAC, IMAPP and SeaDAS software;
- installation of MODIS products for **ocean monitoring**.

## http://readout.marsdb.it

- Thanks for your attention ....