PACI DATA QUEST Expedition

**Quarterly report (April 1,2003 -- July 1, 2003)**

### Expedition Leads: 

Ian Foster, ANL <foster@mcs.anl.gov>

Robert Brunner, NCSA <rb@ncsa.uiuc.edu>

### Coordinator

Veronika Nefedova, ANL <nefedova@mcs.anl.gov>

### Site/Subproject leads

Tom DeFanti, UIC

Miron Livny, **U.Wisconsin-Madison**

Ray Plante, NCSA

Natalia Maltsev, ANL

Nagiza Samatova, ORNL

### Partners

Bob Wilhelmson, MEAD

Dennis Gannon, Portals

Rob Gardner, U. Chicago

Paul Avery, U. Florida

Rick Cavanaugh, U. Florida

Mike Wilde, ANL

Jens Voeckler, U. Chicago

OVERVIEW

**Radio astronomy (Ray Plante, NCSA)**

The target community here is the users of the VLA/EVLA and BIMA/CARMA
radio telescopes. The BIMA Image Pipeline, now being deployed at NCSA,
automatically calibrates and images radio interferometer data as it
arrives from the BIMA telescope in northern California. The initial
version was implemented to access data from the BIMA Data Archive at
NCSA and distribute the computing between a single serial platform and
the NCSA SGI Origin system. As part of this expedition, we plan to
generalize the pipeline to use general Grid services to manage data and
access compute resources. By incorporating these generic services, we
can use the Grid with other data resources and pipelines. In particular,
we plan to establish an instance of the NRAO End-to-end system at NCSA
for processing VLA data. The distribution of data and computing between
NCSA and NRAO will make a good testbed for extension to the TeraGrid. We
can make real practical use of the distributed resources and users when
the BIMA Array combines with the Caltech OVRO Array to create the CARMA
telescope in 2005.

**Bioinformatics (Natalia Maltsev, ANL; Nagiza Samatova, ORNL)**

The first and most crucial step in genome analysis is the accurate
assignment of function to the genes. This requires the capability to
perform high throughput genetic sequence analysis based on comparative
analysis of multiple genomes, and is essential for the development of
applications in functional genomics and systems biology. However, such
factors as exponential growth of the information in the public sequence
databases, and the development of sophisticated and computationally
intensive tools and algorithms for analysis of genomes dictate an urgent
need for the development of the new approaches for high-throughput
computations to support the bioinformatics effort.

In support of this work we propose to apply existing virtual data
techniques to the labor-intensive process of gathering genomic data from
multiple sources, reformatting that data, and transforming it for
insertion into the ANL sequence database, and newly developed virtual
data techniques that extend virtual data tracking into the realm of
relation data provenance, to enable the recording and tracking of
significant portions of the processes of comparative analysis of genomes
and metabolic reconstructions from sequence data.

Algorithms Development. We will develop the following tools and
algorithms that would allow increasing precision and reliability of
high-throughput analysis of the genomes:

a.  Tools and computational infrastructure for adapting advanced
    TeraGrid based technologies for the needs of high-throughput genome
    analysis

b.  Clustering algorithms for partitioning protein space into protein
    functional families

c.  Algorithms for automated assignment of functions to the

Developed algorithms will enhance analysis of the genomes in a framework
of the public genome analysis server and will be available for the use
by the wide scientific community.

**Virtual astronomy (Robert Brunner, NCSA)**

The aim of this work is to utilize Grid-based data management for
various astrophysical data archives. This effort would first work to
integrate several public astronomical datasets, including SDSS (EDR
only), DPOSS, 2MASS, NVSSS, FIRST, and ROSAT within the NCSA domain.
Once the catalog datasets have been integrated into the Grid, we will
develop and deploy several archive services, including complex spatial
queries and data federation functionality. Different commercial and open
source data-mining toolkits will be tested on the federated datasets. We
will also integrate image datasets within the NCSA domain, requiring
several tens of Terabyte of near- and on-line storage. Finally, we will
work to extend this work to the currently planned QUEST2 survey, which,
once operational, will generate a Terabyte of new data every few months.
QUEST2 requires high-speed data transfer from SDSC (where the data
arrives from the mountain) and complex, on-demand processing for rapid
variable identification. Data management, transport, and access are
critical to the Virtual Astronomy effort, which is developing and
applying data mining technologies over the infrastructure being
developed by the NSF funded NVO project. The major tasks being tackled
by virtual astronomy can be categorized into three areas: data
transport, data management, and data mining. In addition to our
contributions to the PDQ Expedition presence at the upcoming Alliance
All Hands Meeting, three posters will be displayed that detail how
Virtual Astronomy group is beginning to deal with large astronomical
datasets.

**Mesoscale weather (Bob Wilhelmson, NCSA)**

We will work with the [MEAD
expedition](http://www.ncsa.uiuc.edu/expeditions/MEAD/) to apply PDQ
technologies to MEAD problems. In a first stage, we will apply basic
data movement and replica management technologies. In a second, we will
apply the Chimera virtual data system. Details can be found in the MEAD
expedition proposal.

**Integration of Optical Networks with Data Grid Tools (Tom DeFanti,
UIC)**

UIC will support the execution of Alliance data-intensive applications
over high-speed optical networks, including I-WIRE (within Illinois) and
through StarLight (for international connections). Work to be done will
include: enhancements to, and integration with Data Grid tools, of UIC
Quality of Service (QoS) Adaptive Networking Toolkit ("Quanta"), which
enables application developers to utilize the extremely high-bandwidth
afforded by optical networks via support for Generalized Multi Protocol
Label/Lambda Switching and Optical Border Gateway Protocol; support for
application projects running over optical networks, via direct
partnerships with application groups. Work with Reliable Blast UDP.

UIC will also support the deployment of storage capabilities that
support overlay network services for bulk data transfer at StarLight.
These capabilities will be based on Network Storage (NeST) appliances
and store-and-forward middleware developed by the Condor Project at the
University of Wisconsin-Madison.

**Milestones, Deliverables and Status**

Here is the short (table) version of the report:
[**http://www-fp.mcs.anl.gov/pdq/schedule.htm**](http://www-fp.mcs.anl.gov/pdq/schedule.htm).
The long version is below.

**Accomplishments**

1.  The group has produced a number of internal documents that specify
    the basic requirements for data movement and replication.

2.  The development of new requested Data Grid capabilities has been
    finished at ANL: Virtual Data Toolkit, Reliable File Transfer, some
    GridFTP features.

3.  The important Bioninformatics computations were performed using new
    tools with an average speedup of x20.

4.  Starlight clusters are ready and enabled on the network. Preliminary
    testing of Quanta\'s RBUDP (Reliable Blast UDP)

> **Bioinformatics (ANL, ORNL)**

-   Further development of GADU (Genome Analysis and Database Update
    pipeline) (A.Rodrigez, D. Sulakhe)

    -   Using GADU technology we have performed Blast analysis of
        sequence data from 450 publicly available genomes with more than
        20 times improvement in speed of data analysis in comparison to
        previously used technologies and resources

    -   Installed Chimera and ran examples on ANL systems Jazz and Data
        Grid Cluster

-   Tools development

    -   SVMMER -- Continued working on a Support Vector Machines
        algorithm-based tool for automated functional classification of
        protein sequences. Primarily focused on the extraction of "key"
        amino acid positions contributing to functional specificity of
        proteins. A paper is in preparation.

    -   We have developed a novel algorithm, called KeyGeneMiner, for
        identification and functional characterization of "key" genes
        responsible for a particular biochemical process of interest.
        One paper has been accepted for the IEEE Bioinformatics
        Conference. The other paper was submitted to the Bioinformatics
        journal.

    -   PhyloBlocks. We have developed an analytical pipeline for
        analysis of evolution of protein families and prediction of
        functions to the genes. The following publicly available tools
        have been integrated into the PhyloBlocks framework in order to
        increase sensitivity of the sequence analysis: Blast, ANL
        Knowledge Base, Blocks, LAMA, ClustalW, HMMER, TreeView. We have
        developed a flexible user interface that allows interactive
        analysis of the protein families by an expert and a number of
        tools for visualization to assist such analyses.

    -   Had a successful demonstration of a uniform pipeline of protein
        sequence analysis tools (PhyloBlocks, BloBlah, SVMMER and PAT)
        at the NCSA All Hands Meeting, at Utah, April 30 -- May 2, 2003.

> **Radio Astronomy**

-   A demonstration of the BIMA Image Pipeline, featuring data transfer
    between the BIMA Data archive, the serial computing platform, and
    NCSA\'s platinum Linux Cluster, was prepared for the Alliance
    All-Hands meeting.

-   Incremental improvements were made on the NVO Grid demonstration for
    display at the Alliance All-Hands meeting and in preparation for
    SC03.  The Chimera and Pegasus systems for managing virtual data are
    a central component of this demonstration.  A description of the
    demonstration was accepted as an SC03 technical paper.

-   The reorganization of software development at NRAO (and their
    subsequent departure as a funded NCSA Alliance member) has delayed
    some of our planned joint work.  However, we plan to continue
    working on the VLA data mirror in this next quarter.

> **Virtual Astronomy**

-   This quarter was primarily spent preparing for AHM and hiring a new
    database programmer and two REU students. The two undergraduate REU
    students are working on applying data mining tools to SDSS data,
    which was archived at NCSA as part of the PDQ expedition. This work
    may also lead to additional tests of DAGMan, Stork, and Condor in
    collaboration with the University of Wisconsin group.

-   The database programmer will assist in PDQ activities, including
    improving the Astronomy data archives, and in developing a database
    for the QUEST2 project, which is now taking data. Chimera is being
    evaluated for database processing tasks, and additional data sets
    are being targeted for archiving at NCSA.

-   Had a successful demonstration of Reliable File Transfer technology
    with archived astronomy data.

> **MEAD**

-   ESG documents studied

-   Decision not to use Chimera

-   Further testing of GridFTP.

-   Data Transport meeting: identified modifications necessary for Java
    gridFTP client to properly work with mass store

-   Regular consultations with ESG on Metadata Services.

-   Formation of a team to begin populating and testing MCS in the MEAD
    workflow context

> **Condor**

-   We have designed and setup 4 different data-pipeline configurations
    to transfer data between SRB @ SDSC and UniTree \@NCSA. Stork and
    Condor scheduling systems are used in the data-pipeline to perform
    the transfers, and DAGMan manages the whole process.

-   Around 10TB data is transferred so far using these pipelines.

-   We have deployed DiskRouter Tool at Starlight, and incorporated it
    into SRB-UniTree data-pipeline for faster transfers.

-   We have developed several end-to-end network statistics tools. These
    tools create a dynamic visualization of what is happening in the
    data-pipeline. It is very useful to easily identify problems.

-   Dynamic protocol selection feature is added to Stork. Now Stork can
    use alternative protocols in case of failures. The user can give a
    set of alternative protocols. When the primary protocol fails, Stork
    starts trying the alternative protocols. And when the primary
    protocol becomes available again, Stork switches back to it. This
    new feature is incorporated into the SRB-UniTree data-pipeline.

-   Made BLAST Grid-enabled.

-   We are currently working on:

    -   Dynamic DiskRouter deployment: Stork will decide automatically
        which nodes to use for the transfers, and then it will deploy
        DiskRouter servers on those nodes. This (close to) optimum
        DiskRouter chain will be used to perform the transfers.

    -   Dynamically deciding the concurrency level for the transfers:
        Stork will decide what concurrency level to use for each
        transfer, taking into consideration the protocol being used, the
        network bandwidth available, the buffer sizes.. etc.

    -   Making BLAST more portable and easily deployable in Grid.

> **Integration of Optical Networks with Data Grid Tools**

-   Met with Miron Livny to detail a set of photonic experiments to
    stream data from Wisconsin\'s DiskRouters through EVL\'s Calient and
    Glimmerglass photonic switches using Quanta\'s photonic control
    capabilities.

-   Preliminary test of Quanta\'s RBUDP has been conducted by Sarah
    Anderson in Minnesota to stream 8.5Gb/s of traffic over the TeraGrid
    testbed from NCSA to SDSC. Work is underway to debug 64 bit
    implementation of Quanta.

-   Alan Verlo is spec-ing disk clusters to house at StarLight for Miron
    Livny\'s PDQ data caching experiments.

> **Publications:**

-   GADU technology was featured in GenomeWeb journal on May, 5 2003.
    Articles describing this project will appear in NCSA Access journal
    and DOE Computing Week bulletin.

-   Leigh et al. An Experimental OptIPuter Architecture for
    Data-Intensive Collaborative Visualization, the 3rd Workshop on
    Advanced Collaborative Environments (in conjunction with the High
    Performance Distributed Computing Conference), Seattle, Washington,
    June 22, 2003.

> \[<http://www-unix.mcs.anl.gov/fl/events/wace2003/index.html>\]
> \[<http://www.evl.uic.edu/cavern/papers/LeighWACEOptiputer2003.pdf>\]

-   "An SVM-based Algorithm for Identification of
    Photosynthesis-specific Genome Features" by Gong-Xin Yu, Al Geist,
    George Ostrouchov, and Nagiza F. Samatova. Accepted for publication
    in the Proceedings of the IEEE Bioinformatics Conference, Stanford,
    CA, August 11-14, 2003.

-   "KeyGeneMiner for Identification of Biochemical Process-specific
    Genes" by Gong-Xin Yu, Al Geist, George Ostrouchov, and Nagiza F.
    Samatova. Submitted to the Bioinformatics journal.