# Presentation: 024030

## CSAR Legacy

**Michael T. Heath**
**February 21, 2007**

- ©2007 Board of Trustees of the University of Illinois

## Center for Simulation of Advanced Rockets

**Detailed, whole-system simulation of solid propellant rockets under normal and abnormal operating conditions**
**Accurate models of physical components**
**Subscale simulations of materials and accident scenarios **
**Software framework to facilitate component integration **
**Computational infrastructure to support large-scale simulations **
**Research collaborations with government laboratories and rocket industry**** **

**Overarching Goal: Simulation of solid propellant rockets from first principles**

## Solid Rocket Booster

- Igniter

- Case

- Core flow

- Propellant

- Nozzle

- Joint & slot

- Ammonium perchlorate (oxidizer)

- Polymeric binder and aluminum particles (fuel)

**200 ********m**

***Micrograph of solid propellant***

## Challenges in Rocket Simulation

**Full 3-D modeling essential to capture physics **
**Strong, nonlinear coupling among components **
**Complex, dynamically changing geometry**
**Extremely diverse spatial and temporal scales**
**Complex material properties and physical processes**
**Enormous computational capacity required for high-resolution simulation of full burn**
**Scalability to 1000s of processors essential**

## CSAR Legacy

**CSAR Legacy**

***Rocstar***** code**
**Major simulations**
**Fundamental research**
**Impact**
**Campus**
**State and local**
**National and international**
**Human resources**
**Lessons learned**

**Aft Joint Slot**

**Forward**
**Joint Slot**

**Titan IV Turbulence**

## Rocstar Code Suite

**Nation’s only 3-D integrated simulation capability for solid rocket motors**
**How did we accomplish this?**

**Adequate financial resources**
**Long time horizon**
**Critical mass of talent across disciplines**
**Availability of massively parallel platforms**
**Now in position to solve problems of national interest, and already doing so**
**Five-segment RSRM for lunar program**

## Rocstar Code Suite

***Rocstar***** Code Suite**

**Scalable to thousands of processors**
**Applicable to many other problems**
**Fluid-structure interaction**
**Reactive, multiphase flows**
**Examples**
**Pyrovalve**
**Helicopter blades**
**Volcanoes**
**Tall buildings**

**NASA**
**Booster Separation Motor**

## Major Simulations“Science with the Code”

**Major Simulations****“Science with the Code”**

**Full RSRM**
**Titan propellant slumping**
**Turbulence around flexible inhibitor**
**Aluminum impingement on nozzle**

**Al particles impacting nozzle**

**Flow field near slumping propellant**

## Verification, Validation, and Uncertainty Quantification

**Verification**
**Convergence studies**
**Comparison with analytical solutions**
**Method of manufactured solutions**
**Comparison between codes**
**e.g., do *****Rocflo***** and *****Rocflu***** give similar results for same problem?**
**Code coverage**
**Coupled codes — verify component codes first**
**Validation **
**RSRM (ATK/Thiokol & NASA) **
**Titan IV (Lockheed Martin)**
**15 and 70 lb BATES (AFRL)**
**Lab scale rocket (NAWC)**
**Attitude Control Motor (Aerojet) **
**Shock tube with thin steel panel**
**New sampling techniques for UQ**

## Navy “StarAft” Motor

**Full burnout**
**Tactical motor**

## Titan IV Motor

**Heavy-lift booster**
**Accident scenario**
**Detailed vorticity studies**

## NASA RSRM

**Late stage in full burn (~100 s)**
**Flexible inhibitor**
**3-D vorticity study**

## NASA RSRM

**Burnout simulation**
**Red is burning propellant; blue is at insulated surface**
**Viewing ****	**
**Propellant/fluid interface**
**Inhibited surfaces**
**Unique capabilities**
**High rate “time zooming”**
**Significant burnout — never seen in industry**
**Case constraints**
**Propellant walkback**
**Inhibitor regression**
**Dynamically changing topology**

## Fundamental Research inFluids and Combustion

**Multiphase flow**
**Equilibrium Eulerian method for fine particles**
**Lagrangian superparticles**
**Turbulence modeling**
**Optimal LES**
**Time zooming**
**Propellant morphology**
**Parallel packing code**
**Propellant modeling**
**3-D simulations with *****Rocfire***

## Fundamental Research inStructures and Materials

**Constitutive and damage modeling**
**Heterogeneous propellants and HE**
**Metallic components**
**Crack propagation**
**Burning, pressure driven**
**Multiscale materials modeling**
**Molecular-level modeling of material interfaces**
**Space-time discontinuous Galerkin methods**

## Fundamental Research inComputer Science & Applied Math

**Fundamental Research in****Computer Science & Applied Math**

**Parallel programming environments**
**Charm++, AMPI, ParFUM**
**Parallel I/O**
**Panda**
**Parallel performance monitoring/modeling**
***Rocprof***
**Meshing**
**Mesh smoothing, repair, and remeshing**
**Mesh quality metrics**
**Hybrid geometric/topological partitioner**
**Visualization**
***Rocketeer***

## Fundamental Research inIntegration Methodology

**Object-oriented integration framework**
**Flexible parallel orchestration**
**Stable component coupling and time-stepping**

**Accurate and conservative data transfer based on common refinement**
**Stable and efficient explicit surface propagation**

## Computational Scienceand Engineering

**Education**
**Program**

- 13 departments
- 130 faculty associates
- 130 grad students
- 10 graduate fellows

**Research**
**Program**

**Computational Science &**
**Engineering Option**

**Center for Simulation of Advanced Rockets**

**Center for**
**Process Simulation and Design**

- DOE funded
- $4-5 million per year
-      10 year program
- 20 faculty
- 35 graduate students
- 5 undergrads
- 20 professional staff

- NSF funded
- $6 million over 6 years
- 12 faculty
- 13 students & postdocs

**Midwest Structural **
**Sciences**
**Center**

- AFRL funded
- $5 million over 5 years
- 10 faculty
- 15 students & postdocs

## Campus Impact

**CSAR put CSE “on the map”**
**CSAR followed by two additional centers hosted by CSE**
**Center for Process Simulation and Design**
**Midwest Structural Sciences Center**
**CSE seen as “center of centerness” on campus, advising other groups seeking or establishing centers**
**CSE taken as model in developing other new interdisciplinary programs (e.g., bioinformatics)**
**CSE’s Turing cluster **
**Established value of large-scale computing facilities for local users**
**Financial commitment from campus administration**

## Turing G5 Xserve Cluster

**1536 compute processors **
**Dual 2GHz G5 processors per node**
**4GB RAM per node**
**Myrinet interconnection network**
**6 Xserve head nodes**
**49TB Xserve RAID**
**512 and 256 node partitions**
**23 racks (15 and 8 racks, respectively)**
**Mac OS 10.4 Server**

## State and Local Impact

**Economic development through industrial collaborations**
**Boeing, Caterpillar**
**Visibility of simulation promoted through various venues**
**Recurring appearances in media (TV, newspapers)**

**RSRM**
**inhibitor vorticity**

**Flexible inhibitor**

## National and International Impact

**Publications (~900 journal articles and proceedings)**
**Conference sessions**
**Industrial collaborations**
**ATK Thiokol**
**Aerojet**
**Boeing**
**Caterpillar**
**Other government agency funded research**
**USAF**
**NASA**
**Technology transfer to and from NNSA labs**
**Participation in Integrated Product Team for USAF solid propulsion program**

## Pedagogical Impact

**1st edition, 1997**

**2nd edition, 2002**

## Human Resources

**Supported 160 graduate students, about 40 of whom have gone on to DOE labs and many others to positions in government or industry**
**Recruited and developed 50 research staff members, many of whom have gone on to positions at DOE labs, industry, and academia**
**Contributed to faculty career development (promotions, awards, etc.), counter to conventional wisdom about collaboration**

## CSAR Students and Staffin DOE/NNSA Labs

**Boyana Norris, ANL **
**Michael Parks, SNL**
**Jason Petti, SNL**
**Ali Pinar, LBNL **
**Jason Quenneville, LANL**
**Christopher Siefert, SNL**
**Donald Siegel, SNL **
**Christopher Tomkins, LANL **
**Michael Tonks, LANL**
**Jason Weber, BBL**
**Bradley Wescott, LANL**
**Amanda White, LANL**
**Steven Wojtkiewicz, SNL**
**Jin Yao, LLNL**
**Jack Yoh, LLNL**
**Former CSAR Staff Now at DOE Labs (5)**
**Michael Ham, ****ORNL (LANL)**
**Dennis Parsons, LLNL **
**James Quirk, LANL **
**Mark Short, LANL**
**Jeff Vetter, LLNL ****(ORNL)**** **

**Former CSAR Students Now at DOE Labs (34)**
**Tyler Alumbaugh, LLNL**
**Balakumar Balasubramanium, LANL**
**Michael Bange, LANL**
**Benjamin Chorpening, SNL**
**Nathan Crane, SNL**
**Zhiqun Deng, PNNL**
**Erik Draeger, LLNL**
**Michelle Duesterhaus, SNL**
**Jeff Grossman, LBNL ****(LLNL****)**
**Arne Gullerud, SNL**
**Thomas Hafenrichter, SNL **
**Rebecca Hartman-Baker, ORNL**
**Jason Hales, SNL **
**Jonghyun Lee, ANL**
**Vanessa Lopez, LBNL**
**Xiaosong Ma, ORNL**
**Greg Mackey, SNL**
**Burkhard Militzer, LLNL**** **
**Jeffrey Murphy, SNL-L**

**Gray now in other employment.**

## What Worked Well?

**Single, over-arching application**
**Maintain focus, avoid mission creep**
**Pre-existing, interdisciplinary framework independent of individual academic departments**
**Tight integration of CS with application disciplines**
**Professional staff in addition to faculty and students**
**Wealth of thesis-sized research issues for graduate students**
**Pipeline of students from Illinois to lab employment**

## What Could Have Worked Better?

**Collaborations with NNSA labs were numerous, but could have been tighter and more comprehensive**
**Suggestions for improvement**
**Common application focus between labs and university**
**Deeper commitment to joint activities, such as long-term visits and joint hiring (e.g., postdocs)**
**Two-way software exchanges**

## http://www.csar.uiuc.edu

- ©2007 Board of Trustees of the University of Illinois

**http://www.csar.uiuc.edu**

- ©

## Prof. Michael T. Heath, Director

- Center for Simulation of Advanced Rockets
- University of Illinois at Urbana-Champaign
- 2270 Digital Computer Laboratory
- 1304 West Springfield Avenue
- Urbana, IL 61801 USA
- heath@uiuc.edu
- http://www.csar.uiuc.edu
- telephone: 217-333-6268
- fax: 217-333-1910