# Presentation: 590705

## Dr. Francine Berman
Director, San Diego Supercomputer Center
Professor and High Performance Computing Endowed Chair, UC San Diego

**Dr. Francine Berman**
*Director, San Diego Supercomputer Center*
*Professor and High Performance Computing Endowed Chair, **UC San Diego*

***Developing Cyberinfrastructure ******for Data-Oriented ******Science and Engineering***

## The Digital World

***The Digital World***

*Commerce*

*Entertainment*

*Information*

*Education*

## Research, Education, and Data

***Research, Education, and Data***

**Astronomy**

**Physics**

**Geosciences**

**Life ****Sciences**

**Arts, and Humanities**

***NVO – 100+ TB***

***SCEC – 153 TB***

***Japanese Art Images – 70.6 GB***

**Engineering**

***TeraBridge – 800 GB***

***JCSG/SLAC – ******15.7 TB***

***Projected LHC ******Data – 10 PB/year***

## Today’s Research and Education Applications Cover the Spectrum  

**Compute **(more FLOPS)

**Home, Lab, Campus, Desktop**
**Applications**

**Medium, Large, and Leadership HPC**
**Applications**

**Data-oriented ****Applications**

***World of******Warcraft***

***Quicken***

***PDB ******applications***

***TeraShake***

***NVO***

***Molecular******Modeling***

***Large-scale data required as input, intermediate, output for many modern HPC applications***
***Applications vary with respect to how well they can perform in distributed mode (grid computing)***

***Researchers increasingly dependent on ******both****** High Performance Computing  (HPC)******and Highly Reliable Data***
***(HRD)***

## SDSC Mission

*The mission of the San Diego Supercomputer Center (SDSC) is to empower communities in ****data-oriented research, education, and practice**** through the innovation and provision of Cyberinfrastructure** *

**    *****Cyberinfrastructure**** **=** ****resources****** ****(computers, data storage, networks, scientific instruments, experts, etc.) ****+ “glue”****** ****(integrating software, systems, and organizations).*

## SDSC Cyberinfrastructure

**SDSC HIGH PERFORMANCE COMPUTING SYSTEMS**
**DataStar**
- 15.6 TFLOPS Power 4+ system
- 7.125 TB total memory
- Up to 4 GBps I/O to disk
- 115 TB GPFS filesystem
**Blue Gene Data**
- First academic IBM Blue Gene system
- 17.1 TF
- 1.5 TB total memory
- 3 racks, each with 2,048 PowerPC processors and 128 I/O nodes
**TeraGrid Cluster**
- 524 Itanium2 IA-64 processors
- 2 TB total memory
- Also 16 2-way data I/O nodes
_**[http://www.sdsc.edu/](http://www.sdsc.edu/)**_
**user_services/**

**SDSC SCIENCE and TECHNOLOGY STAFF, SOFTWARE, SERVICES**
- User Services
- Application/Community Collaborations
- Education and Training
- SDSC Synthesis Center
- Data-oriented Community SW, toolkits, portals, codes
**http://www.sdsc.edu/**

**SDSC DATA COLLECTIONS, ARCHIVAL AND STORAGE SYSTEMS**
- 2.4 PB Storage-area Network (SAN)
- 25 PB StorageTek/IBM tape library
- HPSS and SAM-QFS archival systems
- DB2, Oracle, MySQL
- Storage Resource Broker
- Supporting servers: IBM 32-way p690s,
- 		 72-CPU SunFire 15K, etc.
**http://datacentral.sdsc.edu/**

***Support for  community data collections and databases***
***Data management, mining, analysis, and preservation***

## Who are SDSC’s “customers”?What role does digital data play in their research?

## Major Earthquakes on the San Andreas Fault, 1680-present

***Major Earthquakes on the San Andreas Fault, 1680-present***

**1906**
**M 7.8**

**1857**
**M 7.8**

**1680**
**M 7.7**

***How dangerous is ******the San Andreas Fault?***

**Researchers use geological, historical, and environmental data ****to simulate massive earthquakes****.**
**These simulations are critical  ****to understand seismic movement,**** and assess potential impact.**

**Simulation results provide new scientific information enabling better**
- Estimation of seismic risk
- Emergency preparation, response and planning
- Design of next generation of earthquake-resistant structures
**Results provide information  which can help in saving many lives and billions in economic losses**

***?***

***Data and Simulations ******-- TeraShake***

## TeraShake Visualization

**Simulation of Southern of 7.7 earthquake on lower San Andreas Fault **
- Physics-based dynamic source model – simulation of mesh of 1.8 billion cubes with spatial resolution of 200 m
- Simulated first 3 minutes of a magnitude 7.7 earthquake, 22,728 time steps of 0.011 second each
- Simulation generates 45+ TB data

**Project leadership:**
- Tom Jordan (SCEC), Bernard Minster (SIO) Reagan Moore (SDSC), Carl Kesselman (ISI)

***TeraShake1***

## TeraShake Data Choreography

***Finer resolution simulations require even more resources.  TeraShake being scaled to run on petascale architectures***

***TeraShake ******Data Choreography***

- Parallelfile system

- Dataparking

*   ****“******I have desired to see a large earthquake simulation for over a decade. This dream has been accomplished.”  ***
*Bernard Minster, Scripps Institute of Oceanography*

## TeraShake at Petascale –  A qualitative difference in prediction accuracy creates even greater data infrastructure demands 

| Estimated figures for simulated 240 second period, 100 hour run-time | TeraShake domain (600x300x80 km^3) | PetaShake domain (800x400x100 km^3) |
| --- | --- | --- |
| Fault system interaction | NO | YES |
| Inner Scale | 200m | 25m |
| Resolution of terrain grid | 1.8 billion mesh points | 2.0 trillion mesh points |
| Magnitude of Earthquake | 7.7 | 8.1 |
| Time steps | 20,000 (.012 sec/step) | 160,000 (.0015 sec/step) |
| Surface data | 1.1 TB | 1.2 PB |
| Volume data | 43 TB | 4.9 PB |

***Estimates courtesy of the Southern California Earthquake Center  ***

- Petascale platform will allow much higher resolution
- for very accurate prediction
- of ground motion

## Data as a driver – the Protein Data Bank:  A resource for the global Biology community

**The Protein Data Bank**
- Largest repository on the planet for structural information about proteins
- Provides free worldwide public access 24/7 to accurate protein data
- PDB maintained by the Worldwide PDB administered by the Research Collaboratory for Structural Bioinformatics (RCSB), directed by Helen Berman

**January 2007 Molecule of the Month:  ****Importins**
- Complex of 3 proteins which aids in protein synthesis by ferrying molecules back and forth between the inside and the outside of the nucleus through tube-shaped nuclear pores

**Growth of ****Yearly****/****Total**** Structures in PDB**

- 1976-1990, roughly 500 structures or less per year

- 2006:  > 5000 structures in one year, >36,000 total  structures

***Each structure costs roughly 200K to generate.  2006 holdings will have cost roughly $80B in research investment***

## How Does the PDB Work?

## Using the PDB

- PDB provides a critical building block for research, education, and practice in the Biosciences
- PDB tools include
- Data Extraction and Preparation
- Data Format Conversion
- Data Validation
- Dictionary and Data management
- Tools supporting the OMB Corba Standard for Macromolecular Structure  Data, etc.

## Cyberinfrastructure to Support  Data-oriented Research, Education, and Practice at SDSC

## Storage of research data in SDSC’s archives show consistent increase in the need for capacity

***Storage of research data in SDSC’s archives show consistent increase in the need for capacity***

*Most of the data is supercomputer simulation output, but digital library collections and experimental data are contributing to growth rates*
*Consistent exponential growth with ~15 month doubling drives planning and cost projections*
*Technology advancements help, but media costs/byte are not decreasing as storage is increasing** *

**Information courtesy of Richard Moore**

## National Data Repository: SDSC DataCentral

**First broad program of its kind ****to support national research ****and community data collections ****and databases **
- “Data allocations” provided on SDSC resources
- Data collection and database hosting
- Batch oriented access, collection management services
**Comprehensive data resources:**  disk, tape, databases, SRB, web services, tools, 24/7 operations, collection specialists, etc.

***Web-based portal access***

## DataCentral Allocated Collections include

| Seismology | 3D Ground Motion Collection for the LA Basin |
| --- | --- |
| Atmospheric | Sciences50 year Downscaling of Global Analysis over California Region |
| Earth Sciences | NEXRAD Data in Hydrometerology and Hydrology |
| Elementary Particle Physics | AMANDA data |
| Biology | AfCS Molecule Pages |
| Biomedical Neuroscience | BIRN |
| Networking | Backbone Header Traces |
| Networking | Backscatter Data |
| Biology | Bee Behavior |
| Biology | Biocyc (SRI) |
| Art | C5 landscape Database |
| Geology | Chronos |
| Biology | CKAAPS |
| Biology | DigEmbryo |
| Earth Science Education | ERESE |
| Earth Sciences | UCI ESMF |
| Earth Sciences | EarthRef.org |
| Earth Sciences | ERDA |
| Earth Sciences | ERR |
| Biology | Encyclopedia of Life |

| Life Sciences | Protein Data Bank |
| --- | --- |
| Geosciences | GEON |
| Geosciences | GEON-LIDAR |
| Geochemistry | Kd |
| Biology | Gene Ontology |
| Geochemistry | GERM |
| Networking | HPWREN |
| Ecology | HyperLter |
| Networking | IMDC |
| Biology | Interpro Mirror |
| Biology | JCSG Data |
| Government | Library of Congress Data |
| Geophysics | Magnetics Information Consortium data |
| Education | UC Merced Japanese Art Collections |
| Geochemistry | NAVDAT |
| Earthquake Engineering | NEESIT data |
| Education | NSDL |
| Astronomy | NVO |
| Government | NARA |
| Anthropology | GAPP |

| Neurobiology | Salk data |
| --- | --- |
| Seismology | SCEC TeraShake |
| Seismology | SCEC CyberShake |
| Oceanography | SIO Explorer |
| Networking | Skitter |
| Astronomy | Sloan Digital Sky Survey |
| Geology | Sensitive Species Map Server |
| Geology | SD and Tijuana Watershed data |
| Oceanography | Seamount Catalogue |
| Oceanography | Seamounts Online |
| Biodiversity | WhyWhere |
| Ocean Sciences | Southeastern Coastal Ocean Observing and Prediction Data |
| Structural Engineering | TeraBridge |
| Various | TeraGrid data collections |
| Biology | Transporter Classification Database |
| Biology | TreeBase |
| Art | Tsunami Data |
| Education | ArtStor |
| Biology | Yeast regulatory network |
| Biology | Apoptosis Database |
| Cosmology | LUSciD |

## Data Systems
SAM/QFS 
HPSS
GPFS
SRB

- SAM/QFS
- HPSS
- GPFS
- SRB

***Services, Tools, and Technologies Key for Data-related Capability***

**Data Services**
- Data migration/upload, usage and support (SRB)
- Database selection and Schema design (Oracle, DB2, MySQL)
- Database application tuning and optimization
- Portal creation and collection publication
- Data analysis (e.g. Matlab) and mining (e.g. WEKA)
- DataCentral
**Data-oriented Toolkits and Tools**
- Biology Workbench
- Montage (astronomy mosaicking)
- Kepler (Workflow management)
- Vista Volume renderer (visualization), etc.

**Notes:**
Data services, comprehensive, maybe different pictures.  Combine with following slide
- Why two archival capabilities?  Explain

## Increasing Need to Sustain Digital Data for the Foreseeable Future

***UCSD Libraries***

***The Entertainment Industry***

***The Private Sector***

***The Public Sector***

***Researchers and Educators***

## What data is the most valuable?

**Key criteria**
- Irreplaceable
- Longitudinal
- Used by many
- Expensive
- Needed in the future
- Culturally or scientifically meaningful
- ...

*Federal records*

*Data needing **rescue*

*Irreplaceable **data*

*Time-series*

*Reference collections*

## Key Challenges for Digital Preservation

**What should we preserve?**
- What materials must be “rescued”?
- How to plan for preservation of materials by design?
**How should we preserve it?**
- Formats
- Storage media
- Stewardship – who is responsible, and for how long?
**Who should pay for preservation?**
- The content generators?
- The government?
- The users?
**Who should have access?**

- Print media provides easy access for long periods of time but is hard to data-mine

- Digital media is easier to data-mine but requires management of evolution of media and resource planning over time

## Preservation and Risk 

| Entity at risk | Size | What can go wrong | Frequency | Minimum number of replicas needed to mitigate risk | Administrative support FTEs |
| --- | --- | --- | --- | --- | --- |
| File | ~2 MB | Corrupted media, disk failure | 1 year | 2 copies in single system | System Admins |
| Tape | ~200 GB | + Simultaneous failure of 2 copies | 5 years | 3 homogeneous systems | + Storage Admin |
| System | ~10 TB | + Systemic errors in vendor SW, or Malicious user, or Operator error that deletes multiple copies  | 15 years | 3 independent, heterogeneous systems | + Database Admin
+ Security Admin |
| Archive | ~1 PB | + Natural disaster, obsolescence of standards | 50 - 100 years | 3 distributed, heterogeneous systems | + Network Admin
+ Data Grid Admin |

***Less risk means more replicants, more resources, more people***

## ChronopolisTM provides a comprehensive approach to infrastructure for long-term preservation integrating
Collection ingestion
Access and Services
Research and development for new functionality and adaptation to evolving technologies
Business model, data policies, and management issues critical to success of the infrastructure

**Collection ingestion**
**Access and Services**
**Research and development **for new functionality and adaptation to evolving technologies
**Business model, data policies, and management **issues critical to success of the infrastructure

***ChronopolisTM:******  An Integrated Approach to Long-term Digital Preservation***

*SDSC , the UCSD Libraries, NCAR, UMd, NARA, Library of congress, NSF working together on long-term preservation of digital collections*

## ChronopolisTM – Replication and Distribution

**3 replicas of valuable collections considered reasonable mitigation for risk of data loss**
**ChronopolisTM Consortium will store 3 copies of preservation collections:**
**“****Bright copy”** – Chronopolis TM site supports ingestion, collection management, user access
**“****Dim copy”** – Chronopolis TM site supports remote replica of bright copy and supports user access
**“****Dark copy”** – Chronopolis TM site supports reference copy that may be used for disaster recovery but no user access
**Each site may play different roles for different collections**

***Bright copy C1***

***Dim copy C1***

***Bright copy C2***

***Dark copy C1***

***Dim copy C2***

***Dark copy C2***

## SDSC Playing a Leadership Role in Development of a National Digital Data Framework

***SDSC storing genetic research data for the City of Hope***

***SDSC Developing Data Visualizations for UCSD Moores Cancer Center***

***SDSC Storing National Collections for National Archives and Records Administration***

***SDSC working with the Library of Congress on Distributed Data Stewardship***

***value ******of******content***

## Community Cyberinfrastructure at SDSC

**Allocated HPC resources **(via TeraGrid)
- http://www.sdsc.edu/resources/CompStorage.html

**SDSC Summer Institutes, Training, Outreach**
- www.sdsc.edu/us/training
- http://education.sdsc.edu

**SW, visualization and other services**
- http://www.sdsc.edu/resources/Resources.html

**Community CI-oriented R&D projects**
- www.sdsc.edu/research/

***Thank You***

***www.sdsc.edu***

**DataCentral **data repository
- datacentral.sdsc.edu