![](media/image1.png){width="1.375in" height="0.74375in"}SDM Center
Report

July-September 2007

http://sdmcenter.lbl.gov

**[Highlights in this quarter]{.underline}**

-   Presented a tutorial on parallel I/O techniques to the attendees of
    the Center for Scalable Application Development Software (CScADS)
    Workshop series in July and worked in hands-on sessions to help
    application developers improve the I/O performance of their codes.

-   Implemented an optimized MPI-IO library for use with Lustre on Cray
    XT systems. This library has been deployed at ORNL for use in
    production on the Jaguar system.

-   The first use FastBit technology (without involving any of the
    developers) has been reported in a publication and a software,
    called
    [TrixX-BMI](http://www.biosolveit.de/conferences/abstracts/2007-08-21_ACS2_JS.html),
    for enabling screening libraries of ligands [12 times
    faster](http://www.biosolveit.de/conferences/talks/2007-08-21_ACS2_JS.pdf)
    than the state of art screening tools

-   Two PhD theses (UIUC and UC Berkeley) used FastBit as their
    underlying technology

-   The prototype of web-based interface to R, called *WebR*, has been
    completed. The beta version is being released for "friendly" users.

```{=html}
<!-- -->
```
-   Produced Provenance Architecture document based on use-cases.

```{=html}
<!-- -->
```
-   Implemented pilot version of provenance recorder in Kepler that
    writes to MySQL database, saving workflow specification and
    execution information. Completed an integrated provenance system for
    the fusion use-case based on the generated architecture, API and
    implementation.

    ***[Publications this quarter]{.underline}***

\[NAC+07\] Meiyappan Nagappan, Ilkay Altintas, George Chin, Daniel
Crawl, Terence Critchlow, David Koop, Jeff Ligon, Bertram Ludaescher,
Pierre Mouallem, Norbert Podhorszki, Claudio Silva,  Mladen Vouk,
"Provenance in Kepler-based Scientific Workflows Systems," accepted as
poster for the MS e-Science Workshop, UNC-CH, 21-23 October, 2007.

\[OR07\] Ekow Otoo and Doron Rotem, Parallel Access of Out-Of-Core Dense
Extendible Arrays, Cluster Computing, Austin, Texas, 2007.

\[PLK07\] N. Podhorszki, B. Ludäscher, S. Klasky. "Archive Migration
through Workflow Automation", Intl. Conf. on Parallel and Distributed
Computing and Systems (PDCS), November 19--21, 2007, Cambridge,
Massachusetts.

\[VKB+07\] Mladen Vouk, Scott Klasky, Roselyne Barreto, Terence
Critchlow, Ayla Khan,  Jeff Ligon, Pierre Mouallem, Mei Nagappan,
Norbert Podhorszki, Leena Kora, "Monitoring and Managing Scientific
Workflows Through Dashboards," accepted as poster for the MS e-Science
Workshop, UNC-CH, 21-23 October, 2007.

\[W07\] Kesheng Wu. FastBit Reference Guide. LBNL Tech Report
LBNL/PUB-3192. 2007. http://crd.lbl.gov/\~kewu/ps/PUB-3192.pdf

\[YVC07\] Weikuan Yu, Jeffrey S. Vetter, R. Shane Canon, *OPAL: An
Open-Source MPI-IO Library over Cray XT*. International Workshop on
Storage Network Architecture and Parallel I/O (SNAPI\'07). September
2007. San Diego, CA.

Details and additional progress are reported next

**[Introduction]{.underline}**

Managing scientific data has been identified as one of the most
important emerging needs by the scientific community because of the
sheer volume and increasing complexity of data being collected.
Effectively generating, managing, and analyzing this information
requires a comprehensive, end-to-end approach to data management that
encompasses all of the stages from the initial data acquisition to the
final analysis of the data. Fortunately, the data management problems
encountered by most scientific domains are common enough to be addressed
through shared technology solutions. Based on the community input, we
have identified three significant requirements. First, more efficient
access to storage systems is needed. In particular, parallel file system
improvements are needed to write and read large volumes of data without
slowing a simulation, analysis, or visualization engine. These processes
are complicated by the fact that scientific data are structured
differently for specific application domains, and are stored in
specialized file formats. Second, scientists require technologies to
facilitate better understanding of their data, in particular the ability
to effectively perform complex data analysis and searches over large
data sets. Specialized feature discovery and statistical analysis
techniques are needed before the data can be understood or visualized.
To facilitate efficient access it is necessary to keep track of the
location of the datasets, effectively manage storage resources, and
efficiently select subsets of the data. Finally, generating the data,
collecting and storing the results, data post-processing, and analysis
of results is a tedious, fragmented process. Tools for automation of
this process in a robust, tractable, and recoverable fashion are
required to enhance scientific exploration.

Our approach is to employ an evolutionary development and deployment
process: from research through prototypes to deployment and
infrastructure. Accordingly, we have organized our activities in three
layers that abstract the end-to-end data flow described above. We
labeled the layers (from bottom to top):

-   Storage Efficient Access (SEA)

-   Data Mining and Analysis (DMA)

-   Scientific Process Automation (SPA)

The SEA layer is immediately on top of hardware, operating systems, file
systems, and mass storage systems, and provides parallel data access
technology, and transparent access to archival storage. The DMA layer,
which builds on the functionality of the SEA layer, consists of
indexing, feature identification, and parallel statistical analysis
technology. The SPA layer, which is on top of the DMA layer, provides
the ability to compose scientific workflows from the components in the
DMA layer as well as application specific modules.

# This report consists of the following sections, organized according to the three layers, as follows:

-   Storage Efficient Access (SEA) techniques

    -   Task 1.1: Low-Level Parallel I/O Infrastructure

    -   Task 1.2: Collaborative File Caching

    -   Task 1.3: File System Benchmarking and Application I/O Behavior

    -   Task 1.4: Application Interfaces to I/O

    -   Task 1.5: Disk Resident Extendible Array Libraries

    -   Task 1.6: Active Storage in the Parallel Filesystem

    -   Task 1.7: Cray XT I/O Stack Optimization

    -   Task 1.8: Performance Analysis of Jaguar's Hierarchical Storage
        System

```{=html}
<!-- -->
```
-   Data Mining and Analysis (DMA) components

    -   Task 2.1 High-performance statistical computing for scientific
        applications

    -   Task 2.2: Feature Selection in Scientific Applications

    -   Task 2.3: High-dimensional indexing techniques

-   Scientific Process Automation (SPA) tools

    -   Task 3.1: Dashboard Development

    -   Task 3.2: Provenance Tracking

    -   Task 3.3: Outreach

The reports by each of the three areas, SEA, DMA, and SPA, follow.

**[1. Storage Efficient Access (SEA)]{.underline}**

***Participants:*** Rob Ross, Rajeev Thakur, Sam Lang, and Rob Latham
(ANL), Alok Choudhary, Wei-keng Liao, Kenin Coloma, and Avery Ching
(NWU), Arie Shoshani and Ekow Otoo (LBNL), Jeffrey Vetter and Weikuan Yu
(ORNL), Jarek Nieplocha and Juan Piernas Canovas (PNL)

The goal of this project is to provide significant improvements in the
parallel I/O subsystems used on today\'s machines while ensuring that
the capabilities available now will continue to be available as systems
increase in scale and technologies improve. A three-fold approach of
immediate payoff improvements, medium-term infrastructure development,
and targeted longer-term R&D is employed.

Two of our keystone components are the PVFS parallel file system and the
ROMIO MPI-IO implementation. These tools together address the
scalability requirements of upcoming parallel machines and are designed
to leverage the technology improvements in areas such as
high-performance networking. These are both widely deployed and freely
available, making them ideal tools for use in today's systems. Our work
in application I/O interfaces, embodied by our Parallel NetCDF
interface, also promises to provide short-term benefits to a number of
climate and fusion applications.

In addition to significant effort on PVFS, we recognize the importance
of other file systems in the HEC community. For this reason our efforts
include improvements to the Lustre file system, and we routinely discuss
both Lustre and GPFS during tutorials. Our efforts in performance
analysis and tuning for parallel file systems, as well as our work on
MPI-IO, routinely involve these file systems.

At the same time we continue to push for support of common,
high-performance interfaces to additional storage technologies. Our work
on disk resident extendible arrays allows for the growth of the
extendible array without reorganization and no significant performance
degradation of applications accessing elements in any desired order. Our
work in Active Storage will provide common infrastructure for moving
computation closer to storage devices, an important step in tackling the
challenges of petascale datasets.

**[Task 1.1: Low-Level Parallel I/O Infrastructure]{.underline}**

The objective of this work is to improve the state of parallel I/O
support for high-end computing (HEC) and enable the use of high
performance parallel I/O systems by application scientists. The Parallel
Virtual File System (PVFS) and ROMIO MPI-IO implementations, with
development lead by ANL, are in wide use and provide key parallel I/O
functionality. This work builds on these two components by enhancing
them in order to ensure these capabilities continue to be available as
systems continue to scale.

***Progress Report***

PVFS received a number of improvements over the last quarter. The MX
driver, contributed by Myricom, was testing and integrated into the
source tree. Our visiting student Kyle Schochenmaier implemented a 2D
file distribution mechanism for avoiding incast behavior when many
servers are sending data to a single client, or vice versa. Our second
visiting student, David Buettner, along with Aroon Nataraj of U. of
Oregon helped us design a new interface for efficient and
high-performance event tracing within PVFS. During this process we
collected several large traces for studying visualization techniques for
system-level components.

We organized and held a symposium at ANL for PVFS researchers and
developers, helping everyone catch up with other's work and coordinate
future efforts.

Our collaboration with Garth Gibson's group at CMU continues. We have
begun working with his student to devise efficient mechanisms for very
large directories.

Our collaboration with the Argonne Leadership Computing Facility (ALCF)
continues to go well. We have helped them improve the plan for failover
on the upcoming BlueGene/P system that will arrive starting in October,
and we continue to perform testing leading up to the deployment at ANL

***Plans for next quarter***

We plan to release a new version of PVFS in the coming quarter. Our
discussions with Dr. Gibson indicate that he will use PVFS in an
upcoming class, so we will be planning how to best get his students up
to speed on PVFS development. As the ALCF BlueGene/P will begin arriving
during the coming quarter as well, we expect to spend a great deal of
time helping understand and tune PVFS performance on this new system.

**[Task 1.2: Collaborative File Caching]{.underline}**

The objective of this work is to develop a layer of user-level
client-side file caching system for MPI-IO library. The design uses an
I/O thread in each MPI process and collaborates all threads to perform a
coherent file caching

***Progress Report***

To enforce the atomicity of all file system read write calls, we adopted
a two-phase locking policy in the distributed lock manager of the
caching layer. Locks are separated into two types, sharable for read
operations and exclusive for writes. To relax the atomicity requirement,
we changed the exclusive locks to sharable for some write operations,
expecting a reduction of lock contention. However, exclusive locks are
still enforced when a file block is being brought to local memory,
evicted, or migrated, so that cache metadata integrity is maintained.

An alternative approach that uses MPI dynamic process management
functionality is being developed. Instead of using I/O thread, this
approach spawns a group of new MPI processes as cache servers which
carry out the file-caching task. Similar to the I/O thread approach,
servers collaborate with ach other to perform a coherent file caching.

We continue to exercise our design on several parallel machines:
Tungsten running Lustre file system and the IBM TeraGrid machine running
GPFS file system at NCSA, Jazz running PVFS file system at ANL, and Ewok
running Lustre at ORNL. Performance evaluation uses the NASA BTIO
benchmark, the FLASH application I/O kernel, and S3D application I/O
kernel.

***Plans for next quarter***

We will test the implementation of sharable lock for atomicity
relaxation. In addition, effort will be on the development of the
alternative caching design that spawns the cache server.

**[Task 1.3: File System Benchmarking and Application I/O
Behavior]{.underline}**

The objective of this work is to improve the observed I/O throughput for
applications using parallel I/O by enhancements to or replacements for
popular application interfaces to parallel I/O resources. This task was
added in response to a perceived need for improved performance at this
layer, in part due to our previous work with the FLASH I/O benchmark.
Because of their popularity in the scientific community we have focused
on the NetCDF and HDF5 interfaces, and in particular on a parallel
interface to NetCDF files

***Progress Report***

Work has been on evaluating the scalable distributed lock management
method that provides true byte-range locking granularity. We used S3D
I/O and S3aSim benchmarks to evaluate several lock strategies, including
list lock, datatype lock, two-phase lock, hybrid and one-try lock. The
performance results on PVFS2 file system were obtained and used to
compare with the block-based locking on Lustre. We observed the
improvement of locking throughput up to between one to two orders of
magnitude in performance and maintain a low overhead in achieving
atomicity for noncontiguous I/O operations.

We continue the collaboration work on the S3D I/O kernel with Jacqueline
Chen at Sandia National Laboratories, Ramanan Sankaran and Scott Klasky
at ORNL. We have developed the Fortran subroutines using MPI-IO,
parallel netCDF, and HDF5 and tested on the Cray XT at ORNL. A software
bug in MPI library has been identified and its fix provided to Cray. The
fix allows running more than 2000 processes on the Cray XT.

***Plans for next quarter***

We plan to evaluate the S3D I/O kernel on a few parallel file systems,
including Lustre, GPFS, and PVFS. The collaboration with Jacqueline
Chen, Ramanan Sankaran, and Scott Klasky will focus on the metadata to
be stored along with the netCDF and HDF files.

**[Task 1.4: Application Interfaces to I/O]{.underline}**

The objective of this work is to improve the observed I/O throughput for
applications using parallel I/O by enhancements to or replacements for
popular application interfaces to parallel I/O resources. This task was
added in response to a perceived need for improved performance at this
layer, in part due to our previous work with the FLASH I/O benchmark.
Because of their popularity in the scientific community we have focused
on the NetCDF and HDF5 interfaces, and in particular on a parallel
interface to NetCDF files.

***Progress Report***

The IOR benchmark reported by John Shalf and Hongzhang Shan of LBNL
showed a 4GB array size limit for the parallel netCDF implementation.
The NWU team has modified the IOR's parallel netCDF subroutines to use
record variables with one unlimited dimension in order to bypass the 4GB
limitation. The revised codes have been provided to the IOR team at LLNL
and since incorporated in the most recent release of IOR, version
2.10.1. The performance evaluation for this revision has shown a
significant improvement on GPFS. The write bandwidth is very close to
POSIX, MPI-IO, and HDF5 methods. The performance results were also
provided to John Shalf and Hongzhang Shan.

Hongzhang Shan reported a noticeable IOR performance degradation on GPFS
when the parallel netCDF independent I/O mode is used. This issue is
being investigated.

We have also worked with Annette Koontz of PNNL in her initial port to
PnetCDF, tracking down a 13-character file limit bug in the HP MPI
implementation in the process. We've likewise helped in the installation
of PnetCDF on the Jaguar system at ORNL. Rob Latham presented an
overview of PnetCDF at the NetCDF for Data Providers and Developers
Workshop at UCAR.

As part of the Center for Scalable Application Development Software
(CScADS) Workshop series in July, we presented a tutorial on parallel
I/O tailored to the CScADS attendees and worked with attendees on their
applications in hands-on sessions.

***Plans for next quarter***

Detailed performance profiling will be carried out on the IOR in order
to identify the performance bottleneck of the parallel netCDF
independent I/O mode. We have a meeting with HDF5 and netCDF4
representatives scheduled at ANL in October, and we will be presenting
our Parallel I/O in Practice tutorial at SC07.

**[Task 1.5: Disk Resident Extendible Array Libraries]{.underline}**

Datasets used in scientific and engineering applications are often
modeled as dense multi-dimensional arrays. For very large datasets, the
corresponding array models are typically stored out-of-core as array
files. The array elements are mapped onto linear consecutive locations
that correspond to the linear ordering of the multi-dimensional indices.
Two conventional mappings used are the row-major order and the
column-major order of multi-dimensional arrays. Such conventional
mappings of dense array files highly limit the performance of
applications and the extendibility of the dataset. Expansions of such
out-of-core conventional arrays along arbitrary dimensions require
storage reorganization that can be very expensive. We developed a
solution for storing out-of-core dense extendible arrays that uses a
mapping function together with information maintained in axial vectors,
to compute the linear address of an extendible array element when passed
its k-dimensional index.

***Progress Report***

We have shown in a recent paper \[1\] how the mapping function, in
combination with MPI-IO and a parallel file system, allows for the
growth of the extendible array without reorganization and no significant
performance degradation of applications accessing elements in any
desired order.

Our recent work has focused on implementing a library for Disk Resident
Extendible Array suitable for out-of-core array manipulation. This
library is called DRXTA. The parallel version that is based on the
Shared-Memory model for distributed memory systems is referred as
DRXTA-PP. The DRXTA libraries currently are only partially completed.
DRXTA-PP is being implemented to work with PVFS2 and the MPI-2 RMA
methods. Work is underway to provide equivalent APIs that are callable
from the Global Array library.

We have completed implementations of DRXTA functions in C to create,
read, write and manipulate out-of-core arrays stored in Unix file
systems. Array chunks are cached in and out of memory using the
BerkeleyDB cache pool. We are in the process of implementing parallel
extendible array files using PVFS2 and MPICH2.

***Plans for next quarter***

We will continue work on implementation and testing of extendible array
files using chunking and block compaction, and we will begin
implementation of out-core array manipulation functions for extendible
arrays.

***References***

\[1\] Ekow Otoo and Doron Rotem, Parallel Access of Out-Of-Core Dense
Extendible Arrays, Cluster Computing, Austin, Texas, 2007.

**[Task 1.6: Active Storage in the Parallel Filesystem]{.underline}**

The purpose of this work is to extend the active storage/disk concept to
parallel file systems, in particular Lustre and PVFS, and make it
practical for DOE applications. The PNL team leads this work. We are
developing Active Storage technology to reduce network bandwidth
requirements in I/O intensive scientific applications. For the Active
Storage approach to become widely adopted, several additional features
lacking in virtually all existing implementations of this technology,
must be provided.

***Progress Report***

During last quarter, we focused on developing a user-friendly
programming and run-time environment to specify the data processing
tasks to carry out by the storage nodes. The second direction has been
addressing striped files. All previous implementations of Active Storage
have been unable to deal with striped files. Striped files are typically
used to improve aggregate I/O bandwidth and they are quite often used in
scientific applications. We developed a prototype implementation able to
handle striped files under Lustre and PVFS2 and tested using simple
application I/O kernels. This implementation relies on the ability to
run client code on the storage server and exploits striping information
to optimize performance. In addition, we have been working on providing
the STDIO interfaces in context of Active Storage on striped files.

***Plans for next quarter***

In the next quarter, we will continue to develop and improve our user
framework for AS in preparation for the upcoming release.

**[Task 1.7: Cray XT I/O Stack Optimization]{.underline}**

SciDAC applications make use of a variety of different IO interfaces,
including MPI-IO, Fortran I/O, and Parallel netCDF. We have carried out
a series of studies to examine the parallel I/O subsystem at ORNL's
Leadership Computing Facility (LCF), which uses the Lustre file system.

***Progress Report***

The default MPI-IO package over Lustre is supplied from Cray. This
package contains a proprietary ADIO implementation built on top of
*libsysio*, which itself is an IO library for Catamount clients to
access Lustre file system. In order to instrument the internal MPI-IO
implementation, we built an alternative MPI-IO package for Cray XT
\[1\], using the default MPI-IO implementation from Argonne National
Laboratory. We validated this resulting open-source MPI-IO library
against the original package from Cray in terms of performance. Thus, it
provides a valid starting point for analysis and optimization of
Collective IO over XT. We have deployed our open-source MPI-IO library
on Jaguar as a contributed alternative MPI-IO package at Oak Ridge
National Laboratory.

***Plans for next quarter***

In the coming months, we will continue to extend OPAL library with more
features to improve collective I/O performance on Cray XT. We will also
demonstrate the performance benefits of OPAL to other applications
running over Jaguar.

***Publications***

\[1\] Weikuan Yu, Jeffrey S. Vetter, R. Shane Canon, *OPAL: An
Open-Source MPI-IO Library over Cray XT*. International Workshop on
Storage Network Architecture and Parallel I/O (SNAPI\'07). September
2007. San Diego, CA.

**[Task 1.8: Performance Analysis of Jaguar's Hierarchical Storage
System]{.underline}**

To prepare for future exascale computing, it is important to gain a good
understanding on what impacts a hierarchical storage system would have
on the performance of data-intensive high performance computing (HPC)
applications, and accordingly, how to leverage its strengths and how to
mitigate possible risks. To this aim, we have performed a parallel I/O
performance analysis of the storage system on Jaguar. An
application-oriented perspective is adopted to reveal the implications
of storage organization to common I/O patterns in scientific
applications running over Jaguar.

***Progress Report***

We have evaluated the performance of individual storage components, and
examined the scalability of metadata- and data-intensive benchmarks over
Jaguar. We have discovered that the file distribution pattern in a
large-scale application can impact its aggregated I/O bandwidth. Based
on our analysis, we have demonstrated that it is possible to improve the
scalability of a representative application S3D by optimizing its I/O
access pattern. The aggregated I/O bandwidth of S3D can be sustained to
very large scale. For example, we demonstrated that there is a 15%
bandwidth improvement by controlling the file distribution pattern in
the original S3D program. In addition, replacing the original file per
process implementation with a shared file can avoid a 49% bandwidth drop
across 8192 processes because of the reduction of time spent in parallel
file creation.

***Plans for next quarter***

We plan to parameterize and model the I/O performance over Jaguar and
project the expected performance for representative applications.

**[2. Data Mining and Analysis (DMA)]{.underline}**

### 

### [Task 2.1: High-performance statistical computing for scientific applications]{.underline}

***Contact:*** Nagiza Samatova, ORNL

The following people have contributed to the project over this quarter:

Paul Breimier, Guru Kora, Jeff Ligon, Mladen Vouk, and Nagiza Samatova

Scientific workflows often involve extremely large datasets and require
complex analysis. Workflows that leverage the R statistical and graphing
platform are limited by the specifications of the local machine because
R only runs locally. We provide an interface to a remote R server,
tentatively called *WebR*, which allows an organization to support a
single, powerful R server that can meet the needs of the workflow
scientists. *WebR* will also support pR as its back-end, thereby
providing access to robust parallel computing algorithms to further
ameliorate the resource burden of complex analyses.

***Progress Report***

Most of the worked this last quarter focused on designing an
architecture and developing a prototype for both web-based interface and
Kepler interface to R (pR in the future):

-   *Web-browser-based Matlab-like GUI interface to R*: The *WebR* GUI
    is a web browser application built using Java Servlet technology,
    JavaScript, AJAX, CSS, and HTML. The user experience is modeled
    after *Matlab* and provides a native terminal interface that exactly
    mimics the standard *RGui* application. *WebR* also provides a help
    pane and a scratchpad area where users can develop and execute
    scripts (see figure below).

-   *WebR architecture*: The architecture has 3 major components, a
    front-end web-page, back-end request processor (Servlet) and an
    Rengine (headless R library). At the client's webpage, just like a
    normal R session, a user enter his/her commands. Every time the user
    enters a command, it is sent to a server (Servlet). The Servlet then
    pushes this command to the Rengine running in the background.
    Rengine processes the command and sends the results to the Servlet,
    which forwards the results back to the
    browser.![](media/image2.png){width="6.2555555555555555in"
    height="3.7805555555555554in"}

-   *Multi-user support and sharing the environment across multiple
    users*: The *WebR* backend provides a unique R environment instance
    for each user; R is not thread-safe, so multiple users cannot run
    against the same R process. *WebR* allows users to access the same R
    environment from multiple computers, provides offline access to the
    environment if the user has a local R installation, and allows the
    user to upload a local R environment to the server. Users can also
    share their environment with other users by emailing a link to their
    R environment.

-   *Comparative analysis of competing technologies*: We performed an
    in-depth analysis of existing R browser technologies. A 20-page
    report has been written comparing the features of all the
    approaches. For future performance evaluations we decided to focus
    on the three most active projects: Rpad, Rweb and R-php. Rpad is the
    most advanced of the three, and the only one to provide persistent R
    environments; both Rweb and R-php only operate in batch-mode. None
    of the projects behave like the *RGui* interface. The user must
    enter their R commands, hit a *Submit* button, and wait for the
    results. In order to submit a new set of commands, the user either
    has to navigate using the browser *Back* button, or click in the
    text-area with the commands, depending on the project. The *WebR*
    interface is much more advanced than all three of the existing
    projects and is the only one to facilitate parallel computing and
    data sharing.

-   *Performance testing and benchmarks*. We created a thorough test
    environment to compare WebR, Rpad, Rweb, and R-php. We identified
    the following test-cases. A written report on the results is being
    produced:

> *Null Testcase Single User*: execute a single R command (a=3) to test
> the overhead incurred due to R invocation and communication with the
> browser and not the computation. Specifically, we measure how long it
> takes for the software to start R, execute the command, close R, and
> return the information to the browser.
>
> *Image Testcase Single User*: Processing access to the device to
> produce images is an expensive operation in all the existing R browser
> technologies. The testcase executes a single R command (plot(3)) that
> produces an image output file to test how long it takes the project to
> start R, execute the command, close R, handle the image output, return
> the information to the browser.
>
> *Null Testcase Multi-User*: perform the same as the *Null Testcase*,
> except that each user executes the same command multiple times in
> order to calculate the effect of R environment persistence because
> some projects (WebR, Rpad) only instantiate a single R environment per
> user.
>
> *Image Testcase Multi-User*: perform the same as the *Image Testcase*,
> except that each user executes the same command multiple times in
> order to calculate the effect of R environment persistence because
> some projects (WebR, Rpad) only instantiate a single R environment per
> user.

-   *Web Service to WebR and hooks to Kepler*: We provide an open Java
    API and a set of Web Services to *WebR* to facilitate integration
    with other software projects, including Kepler. We are working
    closely with the SPA team on the *WebR* and Kepler integration.
    Kepler includes an *RExpression* actor which interacts with a local
    R installation. We extended the *RExpression* actor to support
    *WebR* by adding two options to the *RExpression* parameter window:
    a Boolean *Use Remote Server* and a String *Remote Server URL*. The
    first parameter specifies whether the user wants to use *WebR* and
    defaults to *False*, and the second allows the user to enter the
    location of the *WebR* server (see figure on next page). The figure
    shows the *RExpression* parameter window and the new options are
    outlined by the red border. If the workflow designer selects WebR in
    the actor, the backend calls the WebR web services to submit the R
    commands, and retrieves the results. This process is seamlessly
    hidden from the user.

**Summary of WebR Features:**

-   *WebR* is a cleaver combination of a scalable statistical package
    and an easily accessible web framework that provides a light weight
    available anywhere open-source alternative to desktop Matlab.

-   WebR makes an exact copy of R terminal available online instead of a
    stand-alone desktop application.

-   Along with the usual R feature set, *WebR* additionally provides
    transparent session management for individual users, so that users
    can access the same copy of R environment from multiple computers.

-   *WebR*'s "Share my R" feature enables *WebR* user to mail a link and
    provide direct access to sender's R environment. Thus the same R can
    be shared between two different users who are located geographically
    apart.

-   *WebR*'s end user interface is built with a combination of
    JavaScript, Ajax, CSS and HTML web front-end technologies.

-   *WebR*'s business logic layer is built using JSP and Servlets.
    Tomcat 6 is used as a Java container for servicing the web requests.

-   Save/Load R environments: At any time during the session users can
    save his/her R environment (all user created variables / functions
    along with the data). Users have an option of saving the environment
    on the server or the environment can be downloaded to the user's
    desktop. Subsequently, users can load previously saved R
    environments, thus taking the session to the precise state they want
    to have.

-   Another way to run R scripts directly is by uploading R scripts to
    the server. Users are given an option to upload the accompanied data
    as well. *WebR*'s simple script/data upload mechanism enables uses
    to schedule data and script uploads and execute them like a batch
    job.

-   *WebR* also supports full suite of R plotting functions.
    Additionally *WebR* gives an extremely easy way to save the
    generated plots/graphs as images.

-   *WebR* also has a dedicated text editor that can be used to write
    scratch R code. The editor is hot pluggable, i.e. an R code that has
    been entered in the editor can be directly executed with a click of
    a button.

## Plans for Next Quarter

By centralizing the R server, we are facilitating information sharing
between users because all data and results reside on the same server.
Numerous security and privacy issues arise and we intend to extend
*WebR* to support user sharing safely and reliably.

The existing *RExpression* implementation only allows batch-mode
processing of R commands. *WebR* supports persistent R environments and
we will investigate incorporating persistence between multiple
*RExpression* actors in Kepler.

We are also working with the SPA team to incorporate *WebR* into the
*Dashboard* project.

### [Task 2.2: Feature Selection in Scientific Applications]{.underline}

# *Contact:* Chandrika Kamath, LLNL

***Progress Report***

## The work on Poincare plot analysis is on hold and will be resumed in early FY08.

## We (my post-doc Nicole Love and I) presented a paper at the SPIE Annual Meeting in August. This summarized our early experiences with different segmentation techniques to identify the blobs in four sample sequences of NSTX images. Each sequence is 300 frames long and the four sample sequences were chosen to include a diverse set of conditions so we could test and evaluate our techniques. Some of the techniques worked well while others failed for certain images. We tried different ways of modifying the techniques so they would work for most of the images. An expanded version of the SPIE conference paper, summarizing these recent results, is in the works.

## Plans for Next Quarter

### As a result of increased costs and subsequent budget cuts due to the contract change at LLNL, SciDAC funds will be used to cover existing people instead of hiring a developer or post-doc. We will resume the work on classification and characterization of orbits in Poincare plots. 

### We will also apply our segmentation techniques to additional short sequences of the NSTX data as well as the longer sequences.

### [Task 2.3: High-dimensional indexing techniques (LBNL)]{.underline}

# Contact: John Wu, LBNL

***Progress Report***

-   Released FastBit to the public at <http://sdm.lbl.gov/fastbit/src/>
    in early August after some internal testing in July. We recorded
    about 140 downloads for August and September. We are aware of two
    groups expressing serious interest in the software, one at
    International Computer Science Institute (ICSI) and one at Yahoo
    Research. The group at ICSI intends to use FastBit in a projection
    dubbed Time Machine II, which combines the analysis capability of
    BRO Intrusion Detection System with some historical records of
    network flow records. FastBit will be used to handle the searching
    of the historical records. The group at Yahoo Research is seeking a
    replacement of their custom database engine used to match the
    advertisements with content of the web pages and interests of the
    visitors. Preliminary performance testing results is favorable for
    FastBit.

-   Added a new indexing class to make the two-level interval-equality
    encoding (one of the best encoding schemes according to our
    analysis) more efficient for bitmap indexes building with binning.
    We also fixed a number of bugs found by users.

-   Prepared a reference document about the FastBit software (published
    as LBNL Tech Report LBNL/PUB-3192). This document is generated from
    source code with doxygen and will be updated as the software is
    updated.

-   Prepared a set of tutorials for various operations using FastBit
    software.

-   First publication involving the use of FastBit software but not
    involving any developers of the software was presented at the ACS
    Fall 2007 meeting in August. [Jochen
    Schlosser](http://www.zbh.uni-hamburg.de/staff.php?mode=_details&id=schlosser)
    and [Matthias
    Rarey](http://www.biosolveit.de/people/vips/rarey.html) presented
    this new virtual screening tool named
    [TrixX-BMI](http://www.biosolveit.de/conferences/abstracts/2007-08-21_ACS2_JS.html),
    and showed that it can screen libraries of ligands [12 times
    faster](http://www.biosolveit.de/conferences/talks/2007-08-21_ACS2_JS.pdf)
    than the state of art screening tools. \[[Presentation as ACS Fall
    2007
    meeting](http://www.biosolveit.de/conferences/talks/2007-08-21_ACS2_JS.pdf)\]

-   Two PhD theses involving the use of FastBit were recently completed.
    Frederick Ralph Reiss of UC Berkeley used FastBit for some work in
    his thesis titled "[Data
    Triage](http://www.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-79.html)."
    His thesis was officially signed off in June. Rishi Sinha of
    University of Illinois at Urbana-Champaign used FastBit in his
    thesis work as well. He successfully defended his thesis titled
    "[Indexing Scientific
    Data](http://usha.cs.uiuc.edu/~rsinha/thesis_skt.pdf)" in August. We
    are also aware of one PhD thesis work based on the idea of "beating
    FastBit."

## Plans for Next Quarter

-   Continue to support FastBit users. We have received a number of
    requests for new features, such as the ability to exclude records
    from queries permanently and a new logging structure to enable users
    to redirect error message to arbitrary files.

-   Explore the potential of applying the FastBit technology to a
    protein sequence analysis.

-   Continuing the work on feature identification in fusion simulation
    data.

-   Revisit the binning strategies in FastBit. Theoretical analysis of
    the new data structure is needed for "real-application data" not
    just the worst-case scenario.

#  [3. Scientific Process Automation]{.underline}

PNNL (Terence Critchlow, George Chin),

ORNL (Scott Klasky, Roselyne Barreto)

NCSU (Mladen Vouk, Jeff Ligon, Pierre Mouallem, Mei Naggapan)

SDSC (Ilkay Altintas, Daniel Crawl)

UC Davis (Bertram Ludaescher, Norbert Podhorszki, Timothy McPhillips)

University of Utah (Steven Parker, Claudio Silva, Ayla Khan, David Koop)

The Internet is becoming the preferred method for disseminating
scientific data from a variety of disciplines. This has resulted in
information overload on the part of the scientists, who are unable to
query all of the relevant sources, even if they knew where to find them,
what they contained, how to interact with them, and how to interpret the
results. Thus instead of benefiting from this information rich
environment, scientists become experts on a small number of sources and
use those sources almost exclusively. Enabling information based
scientific advances, in domains such as functional genomics, requires
fully utilizing all available information. We are developing an
end-to-end solution using leading-edge automatic wrapper generation,
mediated query, and agent technology that will allow scientists to
interact with more information sources than currently possible.
Furthermore, by taking a workflow-based approach to this problem, we
allow them to easily adjust the dataflow between the various sources to
address their specific research needs.

***Major Accomplishments***

-   Produced Provenance Architecture document based on use-cases.

-   Implemented pilot version of provenance recorder in Kepler that
    writes to MySQL database, saving workflow specification and
    execution information. Completed an integrated provenance system for
    the fusion use-case based on the generated architecture, API and
    implementation.

-   Implemented (and continuing to implement) a number of elements from
    the provenances framework (time table for that is on-line at.

-   Extended the version of the Dashboard operating with ORNL fusion
    codes. Ported it to NCSU for evaluation of portability, packaging,
    and for use at the SC07 Kepler Tutorial.

-   Held Dashboard AHM meeting in Raleigh Sept 20-21, 2007.
    Implementation schedule
    ([[DashboardFeatures_v2.xls]{.underline}](http://spa-dev.googlegroups.com/web/DashboardFeatures_v2.xls?gda=QivIhUkAAAB1nSAS-cEBGTaRSO6eqbRTOE1iGHdGCRvB3o0OGVK-iGG1qiJ7UbTIup-M2XPURDQtrXTacQf1bEvx02YZIB3Qz1BUaCaWOesWrCDq1_rPcA))
    and design details are on-line at
    ([[http://groups.google.com/group/spa-dev]{.underline}](http://groups.google.com/group/spa-dev)).

-   Implemented flash viewer for displaying graphs through dashboard

    **[Task 3.1: Dashboard Development]{.underline}**

    ***Progress Report***

```{=html}
<!-- -->
```
-   Generated the following reports (available on-line):

    -   Dashboard Architecture Document, which has the description of
        the dashboard architecture and some notes of the discussion we
        had about it in the dashboard meeting at NCSU on 20th and 21st
        September

    -   Security Model Document, with the security model and sequence
        diagrams that illustrate its use.

    -   Dashboard Features Spreadsheet, with the various features we
        want in the dashboard. It also has a prioritization of the
        features, their required effort and deadlines.

    -   Schema Document

-   The dashboard continues to be hardened, and ORNL is working with
    both Utah and NCSU to incorporate better flash routines for
    monitoring simulations, more integrated data base provenance and
    data tracking, and annotations..

-   Regular (bi-weekly) dashboard teleconferences are in place

-   We have a fully operational (K)LAMP environment with pilot dashboard
    and provenance implementations.

-   Created prototype CCA/Kepler bridge

    ***Plans for next quarter***

```{=html}
<!-- -->
```
-   Start working on open release version of the dashboard framework
    with packaging and documentation

-   Continue work with end-users to identify improvements to the
    dashboard usability and functionality, and implement the
    improvements.

-   Complete CCA/Kepler bridge

-   Expand Dashboard graphing tools based on user feedback

-   Improve dashboard performance and scalability through flash video

    **[Task 3.2: Provenance Tracking]{.underline}**

    ***Progress Report***

```{=html}
<!-- -->
```
-   Continued updates to provenance database schema based on use-cases.

-   Working with PNNL groundwater group to design and build Kepler
    provenance architecture that supports both RDF and SQL storage.

-   Organized and held Provenance/Dashboard design meeting in June at
    SDSC.

-   Produced a series of documents relating to SPA provenance framework
    requirements and design. Full information is on-line at
    [[http://groups.google.com/group/spa-dev]{.underline}](http://groups.google.com/group/spa-dev).

-   Implemented a large fraction of the provenance requirements and
    design. Testing and integration is in progress.

    ***Plans for next quarter***

```{=html}
<!-- -->
```
-   Implement redesigned provenance recording API based on work with
    groundwater group.

-   Design and implement provenance querying API, query actors and
    workflows for Kepler.

-   Complete implementation of the provenance framework.

-   Start working on open release version of the provenance framework
    with packaging and documentation

-   Continue work with end-users to identify improvements to the
    provenance usability and functionality framework, and implement the
    improvements.

-   **[Task 3.3: Outreach]{.underline}**

    ***Progress Report***

```{=html}
<!-- -->
```
-   The SC07 Kepler tutorial presentation and hands-on material
    submitted

-   Daniel Crawl visited PNNL:

    -   Met with bioinformatics group to design "Biopilot" workflow that
        runs Scalablast.

    -   Met with groundwater group to discuss "Stomp" workflow and
        provenance implementation.

-   Established meeting with ITER European scientists to discuss
    possible collaborations

    ***Plans for next quarter***

```{=html}
<!-- -->
```
-   Participate in NSF-Mellon funded Workshop on Interoperability of
    Scientific and Scholarly Workflows

-   Complete Biopilot workflow and help integrate into web page.

-   Present Kepler tutorial at Australian Partnership for Advanced
    Computing (APAC) Conference and Exhibition

-   Present Kepler tutorial at SC07

-   Host a group of visitors from the ITER organization. Organize a
    2-day meeting at SDSC with participants from ITER, SPA and CPES.

-   Work on new "Blondin" workflows effort.

-   Attend CPES AHM, December 3-4, 2007, at Oak Ridge National
    Laboratory

    ***New Publications and Related Presentations***

> ***Publications***

-   N. Podhorszki, B. Ludäscher, S. Klasky. "Archive Migration through
    Workflow Automation", Intl. Conf. on Parallel and Distributed
    Computing and Systems (PDCS), November 19--21, 2007, Cambridge,
    Massachusetts.

-   Mladen Vouk, Scott Klasky, Roselyne Barreto, Terence Critchlow, Ayla
    Khan,  Jeff Ligon, Pierre Mouallem, Mei Nagappan, Norbert
    Podhorszki, Leena Kora, "Monitoring and Managing Scientific
    Workflows Through Dashboards," accepted as poster for the MS
    e-Science Workshop, UNC-CH, 21-23 October, 2007.

-   Meiyappan Nagappan, Ilkay Altintas, George Chin, Daniel Crawl,
    Terence Critchlow, David Koop, Jeff Ligon, Bertram Ludaescher,
    Pierre Mouallem, Norbert Podhorszki, Claudio Silva,  Mladen Vouk,
    "Provenance in Kepler-based Scientific Workflows Systems," accepted
    as poster for the MS e-Science Workshop, UNC-CH, 21-23 October,
    2007.

> ***Presentations and other activities***

-   Submitted slides for the SC07 tutorial, working on the details of
    the SC07 Kepler Tutorial infrastructure and content. Full
    information is on-line at
    [[http://groups.google.com/group/spa-tutorial]{.underline}](http://groups.google.com/group/spa-tutorial).
    (a slightly simplified version will be presented at APAC\'07)

-   Several presentations were made at the NCSU Dashboard meeting