# Presentation: 504659

## Tsengdar Lee

NASA Science Mission Directorate

**NASA Science Mission Directorate**

**Project Columbia &**
**Applications **

## System of Systems Framework

## Turning Observations into Knowledge Products

**Turning Observations into Knowledge Products**

## Orbit Transfer

- _Pareto-Optimal Trajectories_

- _Trade-off between flight time and propellant_

## High-Fidelity Unsteady Simulation of Shuttle Flowliner

**DOWNSTREAM LINER**

**UPSTEREAM LINER**

**Strong backflow causing high-frequency pressure oscillations**** **

**U=44.8 ft/sec**

**High-Fidelity Unsteady Simulation of Shuttle Flowliner**

**Pump Speed**
**=15,761 rpm**

**Damaging frequency on flowliner **
**due to LH2 pump back flow has **
**been quantified in developing flight**
**rationale for the flowliner.**

**Back Flow**
**In/Out of Cavity**

## Fastest Production Supercomputer

**Fastest Production Supercomputer**
** ****( and everybody knows it)**

## Wall Street Journal 10/29/2004 A5

- “NASA is now home to Columbia – the fastest supercomputer in the known universe. And we salute NASA and SGI for this bold achievement. Built with 10,240 Intel Itanium® 2 processors, Columbia performs at the mind-boggling rate of 42 trillion floating-point calculations per second. We can only imagine what NASA will accomplish with computing power that is truly astronomical.  - Intel”

## Components

**Front End**
- 128p Altix 3700 (RTF)
**Networking**
- - 10GigE Switch 32-port
-  10GigE Cards (1 Per 512p)
- - InfiniBand Switch (288port)
-  InfiniBand Cards  (6 per 512p)
-  Altix 3900 2048 Numalink Kits
**Compute Node**
- -  Altix 3700  12x512p
- -  Altix 3900    8x512p
**Storage Area Network**
- Brocade Switch 2x128port
**Storage (440 TB)**
- FC RAID 8x20 TB (8 Racks)
- SATARAID 8x35TB (8 Racks)
**Mass Storage (5 PB)**

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- A512p

- T512p

- T512p

- T512p

- T512p

- T512p

- T512p

- T512p

- T512p

- RTF 128

- SATA
- 35TB

- Fibre
- Channel
- 20TB

- SATA
- 35TB

- SATA
- 35TB

- SATA
- 35TB

- SATA
- 35TB

- SATA
- 35TB

- SATA
- 35TB

- SATA
- 35TB

- FC Switch 128p

- FC Switch 128p

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- Fibre
- Channel
- 20TB

- InfiniBand

- 10GigE

## Major Characteristics

- 10,240 processors-Intel Itanium 1.5GHz 6M cache
- 20 SGI Altix 512p nodes SSI for each node using Numalink 4
- 1TB RAM per 512-20TB total
- 1-2048p supernode
- Linux O/S
- PBS scheduler
- 440TB of Fiber channel  RAID
- Three levels of LAN Fabric-gigE, 10GigE and Infiniband
- 5PBytes of Tape storage
- 10Gig WAN connectivity at the backplane

## Project Columbia dedicated 10/26/04
World’s fourth fastest computer with 51.8 Teraflops throughput
10240 processors
52% allocated to SMD
Earth-Sun modeling has been the prime usage of the systems
Demand exceeds supply

- World’s fourth fastest computer with 51.8 Teraflops throughput
- 10240 processors
- 52% allocated to SMD
- Earth-Sun modeling has been the prime usage of the systems
- Demand exceeds supply

**Project Columbia Supports Science Mission Modeling and Analysis Research**

## Mission Support Backbone

- Bay

- CIEF

- Midwest

- MSFC

- DC

- CIEF

- GSFC

- CIEF

- South

- East

- KSC

- DFRC

- JPL

- HQ

- LRC

- ARC

- GRC

- SSC

- MAF

*OC*

*48*

*OC*

*48*

**Lambda Services**

- CIEF

- South

- Central

- JSC

- CIEF

- Midwest

- MSFC

- DC

- CIEF

- GSFC

- CIEF

- South

- East

- KSC

- DFRC

- JPL

- HQ

- LRC

- ARC

- GRC

- SSC

- MAF

*OC*

*48*

*OC*

*48*

**Core **

**Lambda Services**

- CIEF

- South

- Central

- JSC

- WSC

- WSC

- WSTF

- CIEF

- Bay

**Mission Support Backbone**

## Interactive Visual Supercomputing

**Interactive Visual Supercomputing**

## Ideal Architecture VisionData Centric, Multi-Tiered

**Compute Environment**
**Multi-tiered Platforms**

**Common Front End**

**GB/s**

**Ideal Architecture Vision****Data Centric, Multi-Tiered**

**Shared High Speed Disk**

**Hierarchical Storage Management**

**High Speed Research Network**

**High Speed Access to Other Sites**

**Next**
**Generation**
**Platforms**

**Visualization Environment**

**Capability Systems**

**Capacity Systems**

**Capacity Systems**

## Establishing a Modeling Environment

**Establishing a Modeling Environment**

## MAP06 Project Structure

- PI:  Bill Lapenta
- Project Manager:  Mike Seablom

**GMAO**
- GEOS-5 DAS
- global analysis and forecasts

**SPoRT**
- WRF:  regional forecasts

**SIVO**
- Compute resources
- Visualization
- Real-time satellite images
- Project Web Page
- Product Distribution

**NAMMA**
- SOP3 Field campaign: aerosol impacts on cyclogenesis / W African Monsoon

**Hurricane Forecast Center(s)**
- Best effort - research investigation of multi-model ensemble track and intensity forecasts

**MAP06 Project Structure**

## MAP06 – GMAO Participation

** **Evaluate GEOS-5 DAS in terms of hurricane prediction skill
-  Evaluate impact of NASA satellite data, particularly AIRS and MODIS derived winds and TRMM rainfall, possibly MODIS hi-res SST, on hurricane prediction skill
-  Evaluate of merit of 1/4o vs 1/2o GEOS-5 analysis [forecasts, instrument team products, other science goals] to guide evolution of GEOS products and justify resource requirements
-  Evaluate role of aerosols in tropical cyclogenesis of West Africa
-  Undertake preliminary investigation of the impact of air-sea feedbacks on hurricane evolution/prediction (GEOS-5 CGCM)** **
** ****Operational Goals**
** **Support NAMMA Field Campaign in SOP3 (Sept 06) with real-time forecasts of weather and aerosol distributions
-  Conduct 1/4o GEOS-5 forecasts in NRT to compare with 1/2o production system
-  Contribute NRT meteorology as input (IC and BC) to WRF forecasts at SPoRT
-  Contribute NRT hurricane forecasts for consideration in multi-model ensembles (best effort, low priority)
** **

**MAP06 – GMAO Participation**

## MAP06 – GMAO Participation (cont.)

** ****System**
** ****Initial: **1/2o GEOS-5 DAS;  1/4o forecasts
-  **Sept06: **1/4o GEOS-5 DAS;  1/4o forecasts
-  DAS on NAS/Explore;  GEOS-5 AGCM forecasts on N-G platform
-  1 x 5-day forecast per day (12Z);  2x during NAMMA SOP-3
-  NCEP-standard AIRS + enhanced AIRS (channel and pixel selection);  TRMM rain retrievals
** ****Requirements**
-  Transition model and scripts to MAP06 N-G platform
-  Include relocator in parallel ops  (different from MERRA)
-  Historical runs:  2004 (GEOS-5 validation) and 2005 (reprocessing + selected events for forecasts)
-  Transparent interface to products (across lambda rail network)
-  1/2 degree DAS - regular parallel ops - on Explore
-  1/4 degree forecasts and historical runs on N-G machine
-  1/2 degree DAS sensitivity runs - on Explore and Columbia
-  1/4 degree DAS on Columbia
** ****Timetable**
-  1 May 06 - first tag ready - 1/2 degree DAS
-  May-June:  Validation; historical runs; connection with WRF, aerosol coupling
-  1 July 06 - operations begins
-  1-30 Sept - NAMMA field campaign - goal: 1/4 degree DAS on Columbia

**MAP06 – GMAO Participation (cont.)**

## MAP06 GEOS-5 DAS Timeline Summary

- July

- September

- August

- October

- GEOS-4 operations (to mid-Nov)

- GEOS-5 MERRA Runs
- 0.5o resolution
- Three streams (1979 - present)

- NAMMA
- 0.25o res.
- Sep. 2006

- GEOS-5 development and validation

- GEOS-5 Validation & Aura Spin-Up
- 0.5o resolution
- Period: Jan 1, 2004 - Sep 30, 2006

- GEOS-5 ops
- 0.5o res.
- Oct 1, 2006

- MAP06
- 0.25o res.
- continuation

- GEOS-5 parallel ops (not frozen)
- 0.5o resolution
- MAP06 starting 1 July 2006

## MAP06 DATA FLOWS

**MAP06 DATA FLOWS**

- NCCS
- GMAO Data Processing and Long Term Storage

- DISC
- GMAO Data Archive & Distribution

- SIVO
- Modeling Environment  (ME) interface

- Upon Production:
- Public GMAO
- Data (GEOS-4 CERES, GEOS-5 forward, MERRA)

- Derived Model
- Products

- Instrument Teams;
- Scientists and other users

- Validation; Scientific investigations;
- QA; Modelers; Researchers;
- System Developers

- GMAO Data

- Derived Model
- Products;  web services

## Vision: A Virtual Integrated Data System

**Storage location is transparent to the user**** **

-  Data management has advanced beyond simply acquiring a desired data set, and then searching for science
-  Data management is now about ‘surfing’ datasets, and acquiring specific information-filled data granules, then going back to ‘surf’ some more.

## MERRA as a prototype 

***The MERRA Complete Archive***
- Leverages NCCS, NAS other compute clusters
- Tape and disk based
- Long-term archive
- GOLDS provides prototype for future NCCS data services

**An Integrated Approach to Data Serving **
**MERRA as a prototype**

***GES DISC:***
***GOLDS - External Data Access Services Provider***
***Active Archive***
***Users: Instrument Teams, Public***
***Leverages:***
- new evolved DISC capabilities: All on-line archive served by S4PA; Mirador interface
- existing data services:  Giovanni: data and model intercomparisons; S4PM-DME; On-the-fly subsetting
- Other DISC data servers: OGC; OpenDAP

***SIVO:***
***Portal for MAP ME Services***
***Application of GOLDS to Education & Outreach***
*** ******Leverages:***
- existing web mapping services:
- existing NEO outreach web services
- Interface to other data servers: OGC (provides access to NOMADS); OpenDAP

## Project FastPath

- The Scientific Computing Portfolio is committed to support Applied Science Program
- Many options...

**RTG**