# Presentation: 526697

## DMSS Project

HPSS User Forum
Oakland, California
June 7-9, 2005

- HPSS User Forum
- Oakland, California
- June 7-9, 2005

- NASA Langley Research CenterDistributed Mass Storage System

## DMSS Team

- Frank Thames
- Roger DuBois
- Dave Elliott
- John Kaiser
- Tim Starrin

## Introduction

- Site Overview & Mission
- HPSS’s Role
- Hardware and Software Configuration
- Statistics and Projections
- Unique to Our Site
- Activities
- HPSS Development Wish List

## NASA Langley Research Center

- Established in 1917...

## Langley leads NASA initiatives in...

- Aviation safety
- Quiet aircraft technology
- Small aircraft transportation
- Aerospace vehicles system technology
-    It supports NASA space programs with atmospheric research and technology testing and development.

*www.larc.nasa.gov*

## Langley’s Research Capabilities

- Systems analysis, integration, and assessment
- Aerodynamics and Aerothermodynamics
- Airframe systems and Airborne systems
- Acoustics
- Materials and structures
- Hypersonic air-breathing propulsion
- Atmospheric, radiation, chemistry and dynamics remote sensing

## HPSS’s Role

- User Archive
- Virtual Storage
- System Backups
- Repository for Data Management Systems
- Backend Web Services
- Disaster Recovery Solution

## DVAL provides a centralized capability for enhancing, visualizing, interpreting, and exploring experimental and computational datasets from a variety of disciplines in support of Langley programs and projects.  Highly skilled specialists working with state-of-the-art hardware and 

- DVAL provides a centralized capability for enhancing, visualizing, interpreting, and exploring experimental and computational datasets from a variety of disciplines in support of Langley programs and projects.  Highly skilled specialists working with state-of-the-art hardware and
- software can produce effective
-  data visualizations and
- custom solutions.  Products
- can be developed for
- distributed platforms,
- ranging from laptops and
- desktops to fully immersive
- environments.

## Slide 9

## Slide 10

- Expert assistance in the  construction and analysis of computer-based geometry for numerical engineering analysis

## Mach 6 air,  = 40

- Mach 6 air,  = 40

**Re-entry Vehicle Design & Development**

## Return to Flight

## STK 9490B

- STK 9940A

- FC

- SAN

- Internal Network

## Configuration - Software 

- + NASA LaRC Developed Tools & Interfaces

## Statistics & Performance - 	Storage & Traffic

- Storage
- 4.6 million files & directories: 174 TB’s of data
- Daily Traffic
- 2,887 transfers, peak: 31,291 transfers
- 1.2 TB’s data, peak: 2.7 TB’s
- Weekly Traffic
- 16,724 transfers, peak: 54,234 transfers
- 8.2 TB’s data, peak: 9.7 TB’s data
- Monthly - 414 hosts

## Statistics & Performance -	Hardware

- Network
- Remote transfers up to gigabit ethernet speeds
- Disk & Tape
- FC attached RAID: 250 MB/sec
- FC 9940A tape: 10 MB/sec
- FC 9940B tape: 30 MB/sec
- Migration
- 9940A: .7 MB/sec (1 stream)
- 9940B: 25 MB/sec (2 streams)

## Projections

- If current trends continue...
- Storage
- 1.6x annual growth rate
- 278 TB’s in 1 year, 1.8 PB’s in 5 years
- Capacity of silos 3.4 PB’s with 9940 media
- Daily Traffic
- 2.2x annual growth rate
- 2.6 TB’s in 1 year, 61.8 TB’s in 5 years (.7 GB/sec)

## Unique to our Site

- User Interface
- EARS command suite
- SRB prototype
- Tools
- Monitoring
- Recovery
- Reporting

## The BIG Problem

- Amount of data generated & stored by applications presents serious challenges
- Becomes unmanageable if user has no way of knowing what data is or where to find it
- More information about data content is needed
- Ability of scientists to use depends on ability to access and manage intelligently and efficiently

*Whitepaper on Data Management, R. Sumpter*

## Langley’s Problem

- Current “management systems” of aeronautics data at LaRC are ad hoc, file-based, non-searchable collections of terabytes of numbers...
- Given the growth of technical data sets to tera- and petabyte sizes, it will not be possible to execute modern, interoperable, collaborative efforts on difficult mission tasks if current storage/management methods continue to be utilized.

*J. Z. Pao – NASA LaRC*

## SRB Proposal

- Investigate its feasibility as a platform to manage the vast amount of aeronautical research data at NASA
- Demonstrate ingestion of a select set of current aeronautical research data into an information system in a brief time period w/ minimal impact on Center research staff

## SRB Features

- Transparent access and management of files and metadata in over 95 formats
- Storage resources from HPSS to Unix disk, the web, even your own server/system
- Web-accessible and searchable hierachical database and filesystem
- Both command-line-driven and GUI

## SRB Prototype

- Early Configs
- Sun Blade 100, Solaris 8, SRB 3.0.1, Sun Java 1.4.2, and PostgreSQL 7.3.5
- IBM F50, AIX 5.1, SRB 3.0.1, Sun Java 1.3, and PostgreSQL 7.3.5
- Prototype
- MCAT Server - Sun Fire V440, Solaris 9, SRB 3.2, Sun Java 1.4, Oracle 9.2.0.5, and OpenSSL-0.9.7c
- Multiple Resource Servers, including HPSS 4.5.0.1

## SRB Testing Configuration

- LAN

- Resources

## SRB Performance Matrix

- (Megabytes per second)

## SRB Data/Metadata Ingestion

- Ingested
- - 15,000 objects
- - 250,000 metadata
- - 21 windtunnel tests
- - 3 wind tunnel facilities

## SRB Success

- The SRB Prototype project is considered a success in the validation of its...
- Ease of use
- Low cost to maintain
- Viability and speed to bring all historical wind tunnel data online and searchable by metadata

*J. Z. Pao – NASA LaRC*

## Tools

- Monitoring
- check-hpss, auto-check-hpss
- timeout
- whpss, xwhpss
- xhpssdf, hpssdf
- xhpssmig, auto-check-mig
- Recovery
- hpsstape, lstape

## Auto-check-hpss

- If paging is disabled, issue warning message and exit
- If current time is during backup window, exit
- If “check-hpss” returns error, send message to pager
- Execute “hpss-df”, if warning threshold is reached, send message to pager
- Check user access, if disabled, send message to pager
- Execute HPSS ls (via “masls” and “timeout”), if it takes more than 5 minutes (configurable) to execute, send message to pager
- If non-prime shift, check tape migration via “auto-check-mig” if any storage class has a migration tape which hasn’t been written to in 3 hours, send message to pager

## Activities - Recent

- SRB Prototype
- SAM QFS Evaluation
- StorNext Evaluation
- Centerwide Backup Study

## Activities - Current

- Timberline to 9940A Transition                (3.8 million small files on 4,302 tapes)
- End-to-End Disaster Recovery Test
- Cost Analysis of HPSS and Alternate Solutions

## Activities - Upcoming

- Moving HPSS Metadata to Faster FC Disk
- Core Server Hardware Upgrade
- HPSS 6.2 Upgrade

## Problems & Concerns

- Small File Performance
- Share-able HPSS Filesystem
- Disaster Recovery
- End-to-end DR test
- Import/export of metadata and data
- Converting 2nd copy to primary copy

## Development Wish List

- (1) Fast Locate support for tapes
- (2) SAN Shared FS support??
- (3) Retry logic in HPSS FTP interface and client API, HSI
- (4) Tape VSNs supported as arguments or input to reclaim command

## Questions?