PACI DATA QUEST Expedition

**Quarterly report (October 1, 2002 -- January 1,2002)**

### Expedition Leads: 

Ian Foster, ANL <foster@mcs.anl.gov>

Robert Brunner, NCSA <rb@ncsa.uiuc.edu>

### Coordinator

Veronika Nefedova, ANL <nefedova@mcs.anl.gov>

### Site/Subproject leads

Tom DeFanti, UIC

Miron Livny, **U.Wisconsin-Madison**

Ray Plante, NCSA

Natalia Maltsev, ANL

Nagiza Samatova, ORNL

### Partners

Bob Wilhelmson, MEAD

Dennis Gannon, Portals

Rob Gardner, U. Chicago

Paul Avery, U. Florida

Rick Cavanaugh, U. Florida

Mike Wilde, ANL

Jens Voeckler, U. Chicago

OVERVIEW

**Radio astronomy (Ray Plante, NCSA)**

The target community here is the users of the VLA/EVLA and BIMA/CARMA
radio telescopes. The BIMA Image Pipeline, now being deployed at NCSA,
automatically calibrates and images radio interferometer data as it
arrives from the BIMA telescope in northern California. The initial
version was implemented to access data from the BIMA Data Archive at
NCSA and distribute the computing between a single serial platform and
the NCSA SGI Origin system. As part of this expedition, we plan to
generalize the pipeline to use general Grid services to manage data and
access compute resources. By incorporating these generic services, we
can use the Grid with other data resources and pipelines. In particular,
we plan to establish an instance of the NRAO End-to-end system at NCSA
for processing VLA data. The distribution of data and computing between
NCSA and NRAO will make a good testbed for extension to the TeraGrid. We
can make real practical use of the distributed resources and users when
the BIMA Array combines with the Caltech OVRO Array to create the CARMA
telescope in 2005.

**Bioinformatics (Natalia Maltsev, ANL; Nagiza Samatova, ORNL)**

The first and most crucial step in genome analysis is the accurate
assignment of function to the genes. This requires the capability to
perform high throughput genetic sequence analysis based on comparative
analysis of multiple genomes, and is essential for the development of
applications in functional genomics and systems biology. However, such
factors as exponential growth of the information in the public sequence
databases, and the development of sophisticated and computationally
intensive tools and algorithms for analysis of genomes dictate an urgent
need for the development of the new approaches for high-throughput
computations to support the bioinformatics effort.

In support of this work we propose to apply existing virtual data
techniques to the labor-intensive process of gathering genomic data from
multiple sources, reformatting that data, and transforming it for
insertion into the ANL sequence database, and newly developed virtual
data techniques that extend virtual data tracking into the realm of
relation data provenance, to enable the recording and tracking of
significant portions of the processes of comparative analysis of genomes
and metabolic reconstructions from sequence data.

Algorithms Development. We will develop the following tools and
algorithms that would allow increasing precision and reliability of
high-throughput analysis of the genomes:

a.  Tools and computational infrastructure for adapting advanced
    TeraGrid based technologies for the needs of high-throughput genome
    analysis

b.  Clustering algorithms for partitioning protein space into protein
    functional families

c.  Algorithms for automated assignment of functions to the

Developed algorithms will enhance analysis of the genomes in a framework
of the public genome analysis server and will be available for the use
by the wide scientific community.

**Virtual astronomy (Robert Brunner, NCSA)**

The aim of this work is to utilize Grid-based data management for
various astrophysical data archives. This effort would first work to
integrate several public astronomical datasets, including SDSS (EDR
only), DPOSS, 2MASS, NVSSS, FIRST, and ROSAT within the NCSA domain.
Once the catalog datasets have been integrated into the Grid, we will
develop and deploy several archive services, including complex spatial
queries and data federation functionality. Different commercial and open
source data-mining toolkits will be tested on the federated datasets. We
will also integrate image datasets within the NCSA domain, requiring
several tens of Terabyte of near- and on-line storage. Finally, we will
work to extend this work to the currently planned QUEST2 survey, which,
once operational, will generate a Terabyte of new data every few months.
QUEST2 requires high-speed data transfer from SDSC (where the data
arrives from the mountain) and complex, on-demand processing for rapid
variable identification.

**Mesoscale weather (Bob Wilhelmson, NCSA)**

We will work with the [MEAD
expedition](http://www.ncsa.uiuc.edu/expeditions/MEAD/) to apply PDQ
technologies to MEAD problems. In a first stage, we will apply basic
data movement and replica management technologies. In a second, we will
apply the Chimera virtual data system. Details can be found in the MEAD
expedition proposal.

**Integration of Optical Networks with Data Grid Tools (Tom DeFanti,
UC)**

**Milestones, Deliverables and Status**

+---------------------------+----------+-----------+----------+-------+
| **PACI Data Quest**       | **Tue    | **Tue     | **       | **    |
|                           | 1        | 9/30/03** | Status** | Exped |
|                           | 0/1/02** |           |          | ition |
|                           |          |           |          | Le    |
|                           |          |           |          | ader: |
|                           |          |           |          | Ian   |
|                           |          |           |          | F     |
|                           |          |           |          | oster |
|                           |          |           |          | (Argo |
|                           |          |           |          | nne), |
|                           |          |           |          | Exped |
|                           |          |           |          | ition |
|                           |          |           |          | Co-Le |
|                           |          |           |          | ader: |
|                           |          |           |          | Br    |
|                           |          |           |          | unner |
|                           |          |           |          | (NC   |
|                           |          |           |          | SA)** |
+===========================+==========+===========+==========+=======+
| > Refine plan and hold    | Tue      | Tue       | The      |       |
| > initial all hands       | 10/1/02  | 10/1-2/0  | first    |       |
| > meeting; identify       |          |           | organi   |       |
| > specific liaisons and   |          |           | zational |       |
| > responsible parties.    |          |           | meeting  |       |
|                           |          |           | was held |       |
|                           |          |           | on       |       |
|                           |          |           | 10       |       |
|                           |          |           | /1-10/2, |       |
|                           |          |           | 2002.The |       |
|                           |          |           | second   |       |
|                           |          |           | meeting  |       |
|                           |          |           | was on   |       |
|                           |          |           | 1        |       |
|                           |          |           | 2/10/02. |       |
|                           |          |           | The      |       |
|                           |          |           | third    |       |
|                           |          |           | meeting  |       |
|                           |          |           | is       |       |
|                           |          |           | planned  |       |
|                           |          |           | for      |       |
|                           |          |           | 3/19/03  |       |
+---------------------------+----------+-----------+----------+-------+
| > Train developers on use | Tue      | Tue       | Ongoing  |       |
| > of data grid            | 10/1/02  | 12/31/02  | consu    |       |
| > technologies in the     |          |           | ltations |       |
| > target application      |          |           | with all |       |
| > projects and educate    |          |           | the      |       |
| > administrators of       |          |           | involved |       |
| > Alliance resources      |          |           | parties  |       |
|                           |          |           | (Globus, |       |
|                           |          |           | GridFTP, |       |
|                           |          |           | Condor). |       |
|                           |          |           | Chimera  |       |
|                           |          |           | and VDT  |       |
|                           |          |           | tutorial |       |
|                           |          |           | was held |       |
|                           |          |           | on       |       |
|                           |          |           | 1        |       |
|                           |          |           | 2/10/02. |       |
+---------------------------+----------+-----------+----------+-------+
| **Radio Astronomy         | **Tue    | **Tue     | ** **    | **P   |
| Imaging**                 | 1        | 9/30/03** |          | lante |
|                           | 0/1/02** |           |          | (NC   |
|                           |          |           |          | SA)** |
+---------------------------+----------+-----------+----------+-------+
| > Incorporate GridFTP     | Tue      | Tue       | Initial  |       |
| > into BIMA pipeline      | 1        | 12/31/02  | GridFTP  |       |
| > transfer data between   | 0/1/02.  |           | testing  |       |
| > the archive and compute |          |           | is       |       |
| > platforms; use GridFTP  |          |           | u        |       |
| > to deliver VLA data     |          |           | nderway. |       |
| > from  NRAO to NCSA.     |          |           | GridFTP  |       |
|                           |          |           | servers  |       |
|                           |          |           | inst     |       |
|                           |          |           | allation |       |
|                           |          |           | at NCSA  |       |
|                           |          |           | and      |       |
|                           |          |           | e        |       |
|                           |          |           | xtensive |       |
|                           |          |           | work on  |       |
|                           |          |           | Unitree  |       |
|                           |          |           | and      |       |
|                           |          |           | GridFTP  |       |
|                           |          |           | incorp   |       |
|                           |          |           | oration. |       |
|                           |          |           | E        |       |
|                           |          |           | xtensive |       |
|                           |          |           | research |       |
|                           |          |           | of Data  |       |
|                           |          |           | m        |       |
|                           |          |           | irroring |       |
|                           |          |           | tools.   |       |
+---------------------------+----------+-----------+----------+-------+
| > Develop long-term Data  | Mon      | Mon       | Both     |       |
| > Management plan for     | 1/1/03   | 3/30/03   | l        |       |
| > BIMA and VLA archives   |          |           | ong-term |       |
| > and pipelines based on  |          |           | and      |       |
| > Grid data management    |          |           | sh       |       |
| > tools; identify any     |          |           | ort-term |       |
| > missing capabilities.   |          |           | plans    |       |
|                           |          |           | have     |       |
|                           |          |           | been     |       |
|                           |          |           | de       |       |
|                           |          |           | veloped. |       |
|                           |          |           | The      |       |
|                           |          |           | c        |       |
|                           |          |           | andidate |       |
|                           |          |           | tech     |       |
|                           |          |           | nologies |       |
|                           |          |           | are      |       |
|                           |          |           | gsi      |       |
|                           |          |           | -enabled |       |
|                           |          |           | rsync    |       |
|                           |          |           | (for     |       |
|                           |          |           | sho      |       |
|                           |          |           | rt-term, |       |
|                           |          |           | work is  |       |
|                           |          |           | u        |       |
|                           |          |           | nderway) |       |
|                           |          |           | and      |       |
|                           |          |           | Reliable |       |
|                           |          |           | Rep      |       |
|                           |          |           | lication |       |
|                           |          |           | Service  |       |
|                           |          |           | (c       |       |
|                           |          |           | urrently |       |
|                           |          |           | under    |       |
|                           |          |           | dev      |       |
|                           |          |           | elopment |       |
|                           |          |           | in       |       |
|                           |          |           | Globus   |       |
|                           |          |           | group)   |       |
|                           |          |           | -- for   |       |
|                           |          |           | the      |       |
|                           |          |           | l        |       |
|                           |          |           | ong-term |       |
|                           |          |           | s        |       |
|                           |          |           | olution. |       |
+---------------------------+----------+-----------+----------+-------+
| > Install BIMA pipeline   | Wed      | Mon       |          |       |
| > on NCSA platforms.      | 4/1/03   | 6/30/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Deploy data management  | Tue      | Tue       |          |       |
| > plan, extending use of  | 7/1/03   | 9/30/03   |          |       |
| > Grid data management    |          |           |          |       |
| > tools to make more      |          |           |          |       |
| > efficient use of disk   |          |           |          |       |
| > caching.                |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Establish initial       | Tue      | Tue       |          |       |
| > mirror of VLA archive   | 9/30/03  | 9/30/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| **Bioinformatics: High    | **Tue    | **Mon     | ** **    | Ma    |
| Throughput Genome         | 1        | 9/29/03** |          | ltsev |
| Analysis**                | 0/1/02** |           |          | (Argo |
|                           |          |           |          | nne), |
|                           |          |           |          | Sam   |
|                           |          |           |          | atova |
|                           |          |           |          | (     |
|                           |          |           |          | ORNL) |
+---------------------------+----------+-----------+----------+-------+
| > Develop detailed        | Tue      | Tue       | E        |       |
| > designs and identify    | 10/1/02  | 12/31/02  | xtensive |       |
| > any missing Data Grid   |          |           | consu    |       |
| > technology elements     |          |           | ltations |       |
|                           |          |           | and      |       |
|                           |          |           | meetings |       |
|                           |          |           | on Data  |       |
|                           |          |           | Grid     |       |
|                           |          |           | tech     |       |
|                           |          |           | nologies |       |
|                           |          |           | (        |       |
|                           |          |           | GridFTP, |       |
|                           |          |           | Condor,  |       |
|                           |          |           | C        |       |
|                           |          |           | himera). |       |
|                           |          |           | BLAST is |       |
|                           |          |           | working  |       |
|                           |          |           | with     |       |
|                           |          |           | CondorG. |       |
|                           |          |           | The      |       |
|                           |          |           | e        |       |
|                           |          |           | xtensive |       |
|                           |          |           | work on  |       |
|                           |          |           | adapting |       |
|                           |          |           | of       |       |
|                           |          |           | Bioinf   |       |
|                           |          |           | ormatics |       |
|                           |          |           | tools to |       |
|                           |          |           | the Grid |       |
|                           |          |           | env      |       |
|                           |          |           | ironment |       |
|                           |          |           | is now   |       |
|                           |          |           | underway |       |
|                           |          |           | (work    |       |
|                           |          |           | with     |       |
|                           |          |           | Chimera  |       |
|                           |          |           | and      |       |
|                           |          |           | Virtual  |       |
|                           |          |           | Data     |       |
|                           |          |           | Toolkit) |       |
+---------------------------+----------+-----------+----------+-------+
| > Algorithms Development  | Tue      | Mon       |          |       |
|                           | 10/1/02  | 9/29/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Complete initial        | Tue      | Mon       |          |       |
| > integration of generic  | 4/1/03   | 6/30/03   |          |       |
| > grid data management    |          |           |          |       |
| > tools into applications |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Use standard            | Tue      | Mon       |          |       |
| > bioinformatics tools to | 4/1/03   | 6/30/03   |          |       |
| > deploy a virtual data   |          |           |          |       |
| > infrastructure on NCSA  |          |           |          |       |
| > and Alliance resources  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Execute simple genome   | Tue      | Mon       |          |       |
| > sequence transfer,      | 7/1/03   | 9/29/03   |          |       |
| > transformation, and     |          |           |          |       |
| > import into WIT2/EMP    |          |           |          |       |
| > database                |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| **GriPhyN**               | **Tue    | **Tue     | ** **    | **Fo  |
|                           | 1        | 9/30/03** |          | ster, |
|                           | 0/1/02** |           |          | Wilde |
|                           |          |           |          | (A    |
|                           |          |           |          | NL)** |
+---------------------------+----------+-----------+----------+-------+
| > Develop detailed        | Tue      | Tue       | VDT and  |       |
| > designs and identify    | 10/1/02  | 12/31/02  | Chimera  |       |
| > any missing Data Grid   |          |           | design   |       |
| > technology elements     |          |           | o        |       |
|                           |          |           | verview. |       |
|                           |          |           | Chimera  |       |
|                           |          |           | team is  |       |
|                           |          |           | working  |       |
|                           |          |           | ext      |       |
|                           |          |           | ensively |       |
|                           |          |           | on       |       |
|                           |          |           | helping  |       |
|                           |          |           | the      |       |
|                           |          |           | app      |       |
|                           |          |           | lication |       |
|                           |          |           | groups   |       |
|                           |          |           | with     |       |
|                           |          |           | int      |       |
|                           |          |           | egration |       |
|                           |          |           | of their |       |
|                           |          |           | tools    |       |
|                           |          |           | and VDT. |       |
+---------------------------+----------+-----------+----------+-------+
| > Execute SDSS data       | Wed      | Tue       |          |       |
| > analysis application on | 1/1/03   | 4/1/03    |          |       |
| > PACI resources          |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Conduct large-scale     | Mon      | Mon       |          |       |
| > cluster search on       | 6/30/03  | 6/30/03   |          |       |
| > Alliance resources      |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Conduct CMS simulation  | Tue      | Tue       |          |       |
| > and data analysis run   | 7/1/03   | 9/30/03   |          |       |
| > on Alliance resources   |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Deploy Data Grid        | Tue      | Tue       |          |       |
| > technology on TeraGrid  | 9/30/03  | 9/30/03   |          |       |
| > resources               |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| **Virtual Astronomy**     | **Tue    | **Tue     | ** **    | **Br  |
|                           | 1        | 9/30/03** |          | unner |
|                           | 0/1/02** |           |          | (NC   |
|                           |          |           |          | SA)** |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | Tue      | Tue       | Data     |       |
| | > CIT-\> ANL         |  | 10/1/02  | 12/31/02  | transfer |       |
| | > transfer first     |  |          |           | using    |       |
| | > steps, first dposs |  |          |           | Condor   |       |
| | > analysis, Quest    |  |          |           | and SRB  |       |
| | > document. GridFTP  |  |          |           | from the |       |
| | > testing and        |  |          |           | data     |       |
| | > installation at    |  |          |           | archive. |       |
| | > NCSA and the       |  |          |           |          |       |
| | > archive sites      |  |          |           |          |       |
| +----------------------+  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | Wed      | Tue       |          |       |
| | > CIT-\> ANL         |  | 1/1/03   | 4/1/03    |          |       |
| | > reliable, repeated |  |          |           |          |       |
| | > dposs analysis     |  |          |           |          |       |
| +----------------------+  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > I                       | > Tue    | Tue       |          |       |
|                           | > 7/1/03 | 9/30/03   |          |       |
| +----------------------+  |          |           |          |       |
| | > Prototype Quest2   |  |          |           |          |       |
| | > pipeline           |  |          |           |          |       |
| +----------------------+  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | Tue      | Tue       |          |       |
| | > Automated Q2 pipe  |  | 9/30/03  | 9/30/03   |          |       |
| +----------------------+  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| **Integration of Optical  | **Tue    | **Tue     | ** **    | **De  |
| Networks with Data Grid   | 1        | 9/30/03** |          | Fanti |
| Tools**                   | 0/1/02** |           |          | (U    |
|                           |          |           |          | IC)** |
+---------------------------+----------+-----------+----------+-------+
| Conduct high-performance  | Tue      | Tue       | At this  |       |
| data movement experiments | 10/1/02  | 12/31/02  | point we |       |
| over optical networks.    |          |           | are      |       |
| Position data storage at  |          |           | l        |       |
| StarLight.                |          |           | earning, |       |
|                           |          |           | m        |       |
|                           |          |           | easuring |       |
|                           |          |           | and      |       |
|                           |          |           | waiting  |       |
|                           |          |           | for      |       |
|                           |          |           | Condor   |       |
|                           |          |           | group to |       |
|                           |          |           | specify  |       |
|                           |          |           | disk     |       |
|                           |          |           | cache    |       |
|                           |          |           | requ     |       |
|                           |          |           | irements |       |
|                           |          |           | at       |       |
|                           |          |           | St       |       |
|                           |          |           | arlight. |       |
|                           |          |           | S        |       |
|                           |          |           | tarlight |       |
|                           |          |           | clusters |       |
|                           |          |           | are      |       |
|                           |          |           | ready    |       |
|                           |          |           | and      |       |
|                           |          |           | enabled  |       |
|                           |          |           | on the   |       |
|                           |          |           | network. |       |
|                           |          |           | A new    |       |
|                           |          |           | paper    |       |
|                           |          |           | d        |       |
|                           |          |           | ocuments |       |
|                           |          |           | the high |       |
|                           |          |           | data     |       |
|                           |          |           | per      |       |
|                           |          |           | formance |       |
|                           |          |           | movement |       |
|                           |          |           | tests    |       |
|                           |          |           | p        |       |
|                           |          |           | erformed |       |
|                           |          |           | this     |       |
|                           |          |           | fall     |       |
+---------------------------+----------+-----------+----------+-------+
| > Conduct engineering and | Tue      | Mon       | We are   |       |
| > testing work on         | 10/1/02  | 11/16/02  | doing    |       |
| > Chimera, Terascope, and |          |           | e        |       |
| > Nest technologies       |          |           | xtensive |       |
|                           |          |           | work on  |       |
|                           |          |           | T        |       |
|                           |          |           | erascope |       |
|                           |          |           | (to be   |       |
|                           |          |           | shown at |       |
|                           |          |           | SC02),   |       |
|                           |          |           | and have |       |
|                           |          |           | recently |       |
|                           |          |           | written  |       |
|                           |          |           | a        |       |
|                           |          |           | paper..  |       |
|                           |          |           | Ready to |       |
|                           |          |           | supply   |       |
|                           |          |           | login    |       |
|                           |          |           | accounts |       |
|                           |          |           | at       |       |
|                           |          |           | S        |       |
|                           |          |           | tarLight |       |
|                           |          |           | anytime  |       |
|                           |          |           | now for  |       |
|                           |          |           | Chimera  |       |
|                           |          |           | and Nest |       |
|                           |          |           | testing. |       |
+---------------------------+----------+-----------+----------+-------+
| > Deploy first production | Mon      | Mon       |          |       |
| > data Grid applications  | 6/30/03  | 6/30/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Continue engineering    | Tue      | Tue       |          |       |
| > and testing work on     | 4/1/03   | 9/30/03   |          |       |
| > Chimera, Terascope, and |          |           |          |       |
| > Nest technologies       |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Extensively evaluate    | Tue      | Tue       |          |       |
| > Data Grid tools for     | 4/1/03   | 9/30/03   |          |       |
| > target applications.    |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Integrate the virtual   | Tue      | Tue       |          |       |
| > data browser produced   | 7/1/03   | 9/30/03   |          |       |
| > by the Portals          |          |           |          |       |
| > expedition              |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Deploy Data Grid        | Tue      | Tue       |          |       |
| > technology on TeraGrid  | 9/30/03  | 9/30/03   |          |       |
| > resources               |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| **Network Storage         | **Tue    | **Tue     | ** **    | **    |
| Appliance, DAGMan Job     | 1        | 9/30/03** |          | Livny |
| Flow**                    | 0/1/02** |           |          | (Wi   |
|                           |          |           |          | scons |
|                           |          |           |          | in)** |
+---------------------------+----------+-----------+----------+-------+
| > Manage project and      | Tue      | Tue       |          |       |
| > provide PACS leadership | 10/1/02  | 9/30/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Provide support for the | Tue      | Tue       | BLAST is |       |
| > Bioinformatics          | 10/1/02  | 9/30/03   | Grid-    |       |
| > applications            |          |           | enabled! |       |
|                           |          |           | Su       |       |
|                           |          |           | ccessful |       |
|                           |          |           | multiple |       |
|                           |          |           | BLAST    |       |
|                           |          |           | runs     |       |
|                           |          |           | using    |       |
|                           |          |           | Condor   |       |
|                           |          |           | at UW.   |       |
+---------------------------+----------+-----------+----------+-------+
| > Provide support for the | Tue      | Tue       |          |       |
| > mesoscale weather and   | 10/1/02  | 9/30/03   |          |       |
| > radio astronomy         |          |           |          |       |
| > applications            |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Integrate Network       | Tue      | Mon       |          |       |
| > Storage (NeST) and      | 10/1/02  | 6/30/03   |          |       |
| > DAGMan                  |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Conduct CMS simulation  | Tue      | Tue       |          |       |
| > and data analysis run   | 9/30/03  | 9/30/03   |          |       |
| > on Alliance resources   |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| > Deploy NeST data        | Tue      | Tue       |          |       |
| > caching nodes           | 7/1/03   | 9/30/03   |          |       |
+---------------------------+----------+-----------+----------+-------+
| **MEAD**                  | **Tue    | **Tue     | ** **    | **    |
|                           | 1        | 9/30/03** |          | Wilhe |
|                           | 0/1/02** |           |          | lmson |
|                           |          |           |          | (NC   |
|                           |          |           |          | SA)** |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | Tue      | Tue       | E        |       |
| | > Test reliability   |  | 10/1/02  | 31/12/02  | xtensive |       |
| | > of data movement   |  |          |           | GridFTP  |       |
| | > from x to y (from  |  |          |           | testing, |       |
| | > out there to       |  |          |           | GridFTP  |       |
| | > NCSA), Use ESG\'s  |  |          |           | and      |       |
| | > technologies.      |  |          |           | Unitree  |       |
| | > netcdf, dods++ and |  |          |           | incorp   |       |
| | > design             |  |          |           | oration. |       |
| +----------------------+  |          |           | Java CoG |       |
|                           |          |           | as a     |       |
|                           |          |           | base for |       |
|                           |          |           | GridFTP  |       |
|                           |          |           | impleme  |       |
|                           |          |           | ntation. |       |
|                           |          |           | The      |       |
|                           |          |           | GridFTP  |       |
|                           |          |           | requ     |       |
|                           |          |           | irements |       |
|                           |          |           | meeting  |       |
|                           |          |           | was held |       |
|                           |          |           | on       |       |
|                           |          |           | 1/10/03. |       |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | 1/1/03   | 4/1/03    | ESG      |       |
| | > Joint MDC design   |  |          |           | d        |       |
| | > with ESG, Chimera  |  |          |           | ocuments |       |
| | > design meeting and |  |          |           | are      |       |
| | > experiments,       |  |          |           | ext      |       |
| | > explore multiscale |  |          |           | ensively |       |
| | > runs, establish    |  |          |           | studied. |       |
| | > MDC                |  |          |           | Initial  |       |
| +----------------------+  |          |           | Chimera  |       |
|                           |          |           | Design   |       |
|                           |          |           | meeting  |       |
|                           |          |           | was held |       |
|                           |          |           | on       |       |
|                           |          |           | 1        |       |
|                           |          |           | 2/10/02. |       |
|                           |          |           | Joint    |       |
|                           |          |           | (with    |       |
|                           |          |           | ESG)     |       |
|                           |          |           | meeting  |       |
|                           |          |           | on       |       |
|                           |          |           | Metadata |       |
|                           |          |           | issues   |       |
|                           |          |           | on       |       |
|                           |          |           | 1/22/03. |       |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | 4/1/03   | 7/1/03    |          |       |
| | > Extend MDC to      |  |          |           |          |       |
| | > useful user        |  |          |           |          |       |
| | > scenarios, use of  |  |          |           |          |       |
| | > ESG\'s DODS++      |  |          |           |          |       |
| | > server (OpenDAPg)  |  |          |           |          |       |
| +----------------------+  |          |           |          |       |
|                           |          |           |          |       |
|                           |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+
| +----------------------+  | 71/1/03  | 9/30/03   |          |       |
| | > Pipeline for       |  |          |           |          |       |
| | > publishing and     |  |          |           |          |       |
| | > running at         |  |          |           |          |       |
| | > multiple sites     |  |          |           |          |       |
| +----------------------+  |          |           |          |       |
|                           |          |           |          |       |
|                           |          |           |          |       |
+---------------------------+----------+-----------+----------+-------+

**Accomplishments**

1.  The group has produced a number of internal documents that specify
    the basic requirements for data movement and replication.

2.  The development of new requested Data Grid capabilities has started
    at ANL.

3.  The group holds regular meetings to discuss the progress as a whole
    as well as in the specific technical areas.

4.  Starlight clusters are ready and enabled on the network.

> **Bioinformatics (ANL, ORNL)**

-   Further development of GADU (Genome Analysis and Database Update
    pipeline) (A.Rodrigez, D. Sulakhe)

    -   Using GADU we have analyzed the sequence data from 106 publicly
        available genomes

    -   Have developed the following enhancements to GADU:

        -   Interface to allow usage of GADU by the users outside of ANL

        -   Interface and tools to allow users to customize GADU usage
            (choice of the databases, tools, etc)

    -   Provided access to chalant with Virtual Data Toolkit

    -   Performed test run of Blast on Datagrid cluster.

    -   Installed Chimera and ran examples on ANL systems

    -   Developing approaches for running Blast under Chimera (still
        on-going) on ANL Data Grid Cluster.

    -   We are in a process of installing Blast on DOE Science Grid
        (PNNL) machines

    -   One of the main Bioinformatic's tools (BLAST) has been
        'Grid-enabled': it runs successfully on CondorG pool at UW.

-   Tools development

    -   PhyloBlocks (Luke Ulrich). We have developed a analytical
        pipeline for analysis of evolution of protein families and
        prediction of functions to the genes. The following publicly
        available tools have been integrated into the PhyloBlocks
        framework in order to increase sensitivity of the sequence
        analysis: Blast, ANL Knowledge Base, Blocks, LAMA, ClustalW,
        HMMER, TreeView. We have developed a flexible user interface
        that allows interactive analysis of the protein families by an
        expert and a number of tools for visualization to assist such
        analyses.

    -   SVMMER (Praveen Chandramohan, G.X. Yu) \-- in collaboration with
        ORNL (Nagiza Samatova) we have developed a Support Vector
        Machines algorithm-based tool for classification of the protein
        sequences. SVMMER's performance was optimized with the flexible
        training set selection interface and tools for the development
        of the models libraries.

    -   Demonstrated the SVMMER at Supercomputing 2003, at Baltimore,
        November 16-23, 2003. Posters, tile display and laptop based
        demos have been presented at both ANL and ORNL booth areas

> **Radio Astronomy**

-   We did an analysis of our data mirroring requirements and did a
    comparative assessment of the features of the Globus Reliable File
    Transfer package and the open-source rsync.  We analyzed the
    bandwidth with iperf, and did some comparative tests transferring
    data from the VLA to NCSA using GridFTP and rsync.  We found that
    since we were saturating the VLA\'s 5Mb/s pipe to Abilene, we
    concluded that there would not be much sense in investing much
    effort into grid-izing rsync.  We identified a simple way to
    GSI-enable rsync.

-   We have set up the VLA mirror hardware and are ready to begin
    transferring data.  Analysis of the network suggests it is more
    efficient to ship a box of disks with the current contents of the
    archive (\~1 TB).  Subsequent data can be transferred via the
    network. 

-   Integration of GridFTP into the BIMA pipeline has been postponed
    slightly as we finish the hardware upgrade and installation of
    pipeline on NCSA platforms.

-   Long term Data Management plan for BIMA and VLA archives is under
    way. 

> **Virtual Astronomy**

-   CondorG transfer jobs using Quest2

-   Working on data mining with D2K.

-   Working wit Java Cog so we can publish web services on the Grid.

-   Looking at how to replicate SDSS data, hardware on order.

> **MEAD**

-   MEAD collaboration with PDQ falls into several areas: GridFTP
    invocation and testing, management of model simulation data, and
    exploration of Chimera for use in data analysis. During the last
    period the invocation and movement of data using GridFTP has been
    explored by Alameda, Hampton, and Rossi. Primarily, the jftp client
    (version 0.9.14 plus patches) was tested against a set of gridftp
    (globus version 2.0) servers, to control normal ftp (client-server)
    transfers and third party transfers. A sufficiently large number of
    server and client issues were identified that we held special PDQ
    meeting in January 10, 2003 on GridFTP. In this meeting, we learned
    that a key problem in the protocol implementation was uncovered in
    version of the globus toolkit prior to 2.2.3, and that in order to
    properly transfer files via gridftp we need to bring everything
    (gridftp server, gridftp server for unitree, c and java client
    libraries) to implement the protocol as implemented in 2.2.3. We
    also discussed preferred functionality and preferred mechanisms for
    interacting with gridftp servers, as well as possible enhancements
    for gridftp implementations. Finally we considered issues associated
    with measuring performance at the gridftp server, and possibly
    collecting this information as a whole for the PDQ Expedition.
    Alameda will take up the issue of turning on the instrumentation and
    possibly gathering the data with Alliance grid administrators.

-   The GridFTP analysis and enhancements are a foundation piece to the
    next substantive collaboration between MEAD and PDQ, namely,
    reliable file transfer services. As we iron out the issues with
    GridFTP we will transition to consideration of both reliable file
    transfer services and replica managment services, which in their
    fullest implementation present OGSA grid services for composition in
    complex workflows and for use by grid service client software in the
    creation of complex workflows.

-   Data management and access are key concerns in MEAD as they are in
    the Earth System Grid Project
    ([http://www.earthsystemgrid](http://www.earthsystemgrid/)) funded
    by DOE. Middleton, an unfunded MEAD partner, is one of the leads in
    this effort along with Foster. Wilhelmson is monitoring internal ESG
    communications and recently participated in a day long AG meeting
    (also attended by the NCSA portal group) with ESG (including
    representation from the e-Science effort) on the issue of metadata.
    The metadata and data management working group in MEAD, along with
    PDQ, will pursue further collaboration with ESG. This will include
    incorporation of the gird based Metadata Catalogue Service (MCS)
    into the MEAD framework along with links to the THREDDS Discovery
    System being utilized in the Digital Library for Earth System
    Education (DLESE). An alpha version of the MCS should be available
    in the June time frame with a beta version in the fall. The MCS
    prototype will have capability for storing attributes and querying
    holdings, it will be extensible and consistent, it will support
    authentication and authorization, logical collections, and logical
    views, handle creation information, annotations, audit records,
    transformation history, master copy support, and support for
    containers, and provide good performance and scalability.

> **Publications:**

1.  TeraScope: Distributed Visual Data Mining of Terascale Data Sets
    over High Speed Photonic Networks by Chong (Charles) Zhang, Jason
    Leigh, Thomas A. DeFanti. Marco Mazzucco, and Robert Grossman

2.  QUANTA: A Toolkit for High Performance Data Delivery over Photonic
    Networks, by Eric He, Javid Alimohideen, Naveen K. Krishnaprasad,
    Jason Leigh, Oliver Yu, and Thomas A. DeFanti

3.  Hierarchical feature extraction based approach to functional
    differentiation of highly homologous protein functional groups,
    co-authored by Nagiza F. Samatova, Gong-Xin Yu, Praveen
    Chandramohan, Hoony Park, George Ostrouchov, Al Geist, and Natalia
    Maltsev to be submitted to the Bioinformatics journal.