# Presentation: 494846

## Scientific Data Management Center

- Nagiza F. Samatova
**Network and Cluster Computing**
**Computer Sciences and Mathematics Division**

## Scientific Data Management Center

**Lead Institution:**** LBNL **
**PI:**** Arie Shoshani**
**Laboratories:**
**ANL, ORNL, LBNL, LLNL, PNNL**
**Universities:**
**NCSU, NWU, SDSC****, UCD, ****U. Utah**
**Established 5 years ago (SciDAC-1)**
**Successfully re-competed for next  5 years (SciDAC-2)**
**Featured in Fall 2006 issue of **_**SciDAC Review**_** magazine**

## SDM InfrastructureUses three-layer organization of technologies

- SDM InfrastructureUses three-layer organization of technologies

- Integrated approach:
**To provide a scientific workflow capability**
**To support data mining and analysis tools**
**To accelerate storage and access to data**
- Benefits scientists by
**Hiding underlying parallel and indexing technology**
**Permitting assembly of modules using workflow description tool**

- Goal: Reduce data management overhead

## Automating scientific workflow in SPA enables mining of diverse biology databases

- Automating scientific workflow in SPA enables mining of diverse biology databases

**Scientific discovery is a multi-step process. SPA-Kepler workflow system automates and manages this process.**

- Contact: Terence Critchlow, LLNL (critchlow1@llnl.gov)

## Data analysis for fusion plasma

**Plot of orbits in cross-section of a fusion experiment shows different types of orbits, including circle-like “quasi-periodic orbits” and “island orbits.” Characterizing the topology of orbits is challenging, as experimental and simulation data are in the form of points rather than a continuous curve. We are successfully applying data mining techniques to this problem.**

**Feature selection techniques used to identify key parameters relevant to the presence of edge harmonic oscillations in the          DIII-D tokomak.**

- Contact: Chandrika Kamath, LLNL (kamath2@llnl.gov)

## Searching and indexing with FastBit Gleaning insights about combustion simulation

- Searching and indexing with FastBit Gleaning insights about combustion simulation

- About FastBit:
**Extremely fast search of large databases**
**Outperforms commercial software**
**Used by various applications: combustion, STAR, astrophysics visualization**
- Collaborators:
**SNL: J. Chen, W. Doyle **
**NCSU: T. Echekki**

- Contact: John Wu, LBNL (kwu@lbl.gov)

**Searching for regions that satisfy particular criteria is a challenge. FastBit efficiently finds regions of interest.**

## Searches in STAR experiment Finding one event out of a million

- Searches in STAR experiment Finding one event out of a million

- Illustration: A. Tovey

- STAR scientists extract data of interest within 15 minutes. Accomplishing the same task before consumed weeks.

**FastBit and Storage Resource Manager enable efficient mining of hundreds of terabytes of data generated by STAR detector**

- Contact: John Wu, LBNL (kwu@lbl.gov)

## Parallel statistical computing with pR

- Parallel statistical computing with *pR*

- Goal: Provide scalable high-performance statistical data analysis framework to help scientists perform interactive analyses of produced data to extract knowledge

** ****Able to use existing high-level (i.e., *****R*****) code**
** ****Requires minimal effort for parallelizing**
** ****Offers identical interface**
** ****Provides efficient and scalable performance**

- Contact: Nagiza Samatova, ORNL (samatovan@ornl.gov)

## Parallel input/outputScaling computational science

- Parallel input/outputScaling computational science

- Multi-layer parallel I/O design:
**Supports Parallel-netCDF library built **
**on top of MPI-IO implementation called**
**ROMIO, built in turn on top of Abstract **
**Device Interface for I/O system, used **
**to access parallel storage system.**
- Benefits to scientists:
**Brings performance, productivity, and portability**
**Improves performance by order        of magnitude**

**Orchestration of data transfers and speedy analyses depends on efficient systems for storage, access, and movement of data among modules.**

- Contact: Rob Ross, ANL (rross@mcs.anl.gov)

## Speeding data transfer with PnetCDF

**Rate of data transfer using HDF5 decreases when a particular problem is divided among more processors. In contrast, parallel version of netCDF improves because of low-overhead nature of PnetCDF and its tight coupling to MPI-IO.**

**Enables high performance parallel I/O to netCDF**
**data sets**
**Achieves up to 10-fold performance improvement over HDF5**

- Contact: Rob Ross, ANL (rross@mcs.anl.gov)

**Inter-process communication**

## Contact

- Nagiza Samatova
- Network and Cluster Computing
- Computer Science and Mathematics Division
- (865) 241-4351
- samatovan@ornl.gov

-    Samatova_SDM_0611