# Presentation: 372560

## Probabilistic Risk Assessment: NASA Strategy for Capability Enhancement

**Dr. Michael G. Stamatelatos**
**Manager, Risk Assessment**
**NASA Headquarters**
**Office of Safety and Mission Assurance**
**mstamate@hq.nasa.gov**
**(202) 358-1668**

## Outline

**Introduction**
**PRA History at NASA**
**NASA PRA Capability Improvement Strategy**
**On-Going Efforts**

## Introduction

## PRA Simply Described

## Types of Risk and Related Consequences

**Deaths, injuries, illness**

**Contamination, loss of use**

**Money lost**

**Mission, schedule,**
**                         ****etc.**

## Risk Sources in Safety Risk Assessment

- Hardware
- Failures

- Organizational
- Factors

- External
- Events,
- Acts of Nature

- Human Error

- Increasing degree
- of complexity in
- modeling

## Risk Assessment & Management

**Qualitative Risk**
_**Assessment**_
- FMEA
- FTA

**Quantitative Risk**
_**Assessment**_
- Initiating Events
- Scenario Modeling
-   (MLD, ESD, ETA, FTA)
- Risk Quantification
- Uncertainty Evaluation

_**Inputs**_
- Mission
-   Success
-   Criteria
- Technical
-   Data
- Cost
- Schedule
- Management
-   Procedures
- Other

- Improved
- Safety &
- Performance

##                Benefits of PRA 

**	****Probabilistic Risk Assessment (PRA) has proven to be a systematic, logical, and comprehensive tool to assess risk (likelihood of unwanted consequences) in modern technological applications (e.g., nuclear power, electric power generation, chemical processing industry), for:**
**Improving system performance**** & ****mission success**
**Increasing safety**** in design, operation & upgrade**
**Saving money**** in design, manufacturing, assembly and operation**

## When Should PRA Be Performed?

**When important decisions must be made about complex systems under uncertainty**
**When information is not sufficient to comprehensively assess strengths and weaknesses of complex systems by other means**
**When important complex jobs must be performed successfully for the first time**
**In all life cycle phases of a complex system**

## Brief History of PRA at NASA

## Beginning with the Challenge of Apollo...

***“******I believe that this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to the earth.”***** -- ****President John F. Kennedy, May 25, 1961**
- Early Apollo program estimate of mission success probability was approximately 0.20--not what people wanted to hear.
- On July 20, 1969, the human race accomplished its single greatest technological achievement of all time when a human first set foot on another celestial body.
- 5 additional successful Moon missions (out of 6 attempts) occurred between 1969 and 1972 = 6 out of 7 = 0.86 demonstrated mission success.
- So much for PRA in NASA for a long time to come!
- Instead NASA would rely on FMEAs

## ...Then Challenger

- January 28, 1986, after 25 successful flights, the Space Shuttle Challenger explodes.
- October 29, 1986, “Investigation of the Challenger Accident,” by the Committee on Science and Technology, House of Representatives:
- 	*“**Without some means of estimating the probability of failure of the various [Shuttle] elements it is not clear how NASA can focus its attention and resources as effectively as possible on the most critical systems.”*
- January 1988, “Post-Challenger Evaluation of Space Shuttle Risk Assessment and Management,” by the Slay Committee:
- 	*“**The Committee recommends that probabilistic risk assessment approaches be applied to the Shuttle risk management program at the earliest possible date.  Data bases derived from STS failures, anomalies, and flight and test results, and the associated analysis techniques, should be systematically expanded to support probabilistic risk assessment, trend analysis, and other quantitative analyses relating to reliability and safety.”*

## The Return to PRA

- Dec. ‘87 -- Space Shuttle Proof-of-Concept Study
- Feb. ‘88 -- Space Shuttle Main Propulsion Pressurization System PRA
- Nov. ‘88 -- Enhanced Hazard Analysis for Space Systems
- Apr. ‘89 -- Independent Assessment of Shuttle Accident Scenario Probabilities for the Galileo Mission
- Jul. ‘90 -- Space Station Freedom -- External Maintenance Task Team -- Final Report -- the “Fisher-Price Study”
- Dec. ‘90 -- Safety of the Thermal Protection System of the Space Shuttle Orbiter -- Quantitative Analysis and Organizational Factors
- Jun. ‘91 -- 8’ High Temperature Tunnel PRA(@ LaRC)
- ‘92 -- Advanced Solid Rocket Motor Field Joint Case Sealing System Leak Check vs. No Leak Check

## The Return to PRA (cont’d)

- Jul. ‘92 -- Risk Assessment of the ASRM Propellant Manufacturing Facility
- Jul. ‘93 -- Reliability and Requirements Analysis for Space Exploration Initiative Vehicles: Comparative Risk Assessment -- the Space Shuttle and an All Rocket Single Stage to Orbit Vehicle
- Sep. ‘93 -- An Analysis of Selected Risk Factors and Uncertainties for Space Station Assembly Up to Human Tended Condition for Space Station Transition Options A1 and A2
- Jan. ‘94 -- An Investigation of the Risk Implications of Space Shuttle Solid Rocket Booster Chamber Pressure Excursions
- Feb. ‘95 -- PRA of the Space Shuttle -- A Study of the Potential of Losing the Vehicle During Normal Operations
- And others including PRA work done in support of Ulysses and Cassini missions.

## The NASA Administrator Steps In...

- July 29, 1996, the NASA Administrator, Dan Goldin:
- 	***“******Since I came to NASA [1992], we’ve spent billions of dollars on Shuttle upgrades without knowing how much they improve safety.  I want a tool to help base upgrade decisions on risk.”***
- Earlier “paper PRAs” prepared by NASA contractors would not serve the purpose.
- October 1997, an early version of the NASA Quantitative Risk Assessment System (QRAS) is demonstrated to the Administrator.
- February 1998, Version 1.0 of QRAS is baselined.
- Two other intermediate version have been tested
- March 2001, Version 1.6 of QRAS will be delivered. It will have full PRA capabilities.

##                 Space Shuttle

- Johnson Space Center and Marshal Space Flight Center have been modeling their Shuttle elements.
- Space Shuttle Program has begun to factor risk into their Upgrades Program.

## International Space Station

- 1999 -- The NASA Advisory Council recommended, the NASA Administrator concurred, and the ISS Program has begun a PRA.
- First portion of PRA (through Flight 7A) delivered in Dec. ‘99; 2nd portion (through Flight 10A) expected in Dec. ‘00.
- Using the SAPHIRE software application for conducting PRA.
- Objectives of ISS PRA:
- Provide a quantitative look at ISS operations risk
- Provide a model for future ISS safety decision-support activities
- Provide a model for safety related operations planning
- Provide a model for trading marginal safety enhancements versus cost

## Mars Sample Return Mission

- Mission must meet a Planetary Protection Program criterion of <10-6 probability of Earth contamination upon return of sample
- Use of PRA is being seriously considered as a means to evaluate mission compliance with the PPP criterion

## The Risk Management Picture at NASA

- NASA Procedures and Guidelines 7120.5A, “NASA Program and Project Management Processes and Requirements,” April 3, 1998
- Requires NASA Program & Project Managers to manage risk formally
- We are seeing evidence of real risk management in numerous NASA projects
- Risk management is a factor in high-level program/project decision-making
- “Continuous Risk Management” training course developed and pilot-tested on numerous NASA project teams
- To be picked up by NASA’s APPL in FY 01
- Risk-Based Acquisition Management (R-BAM)
- Interim rule entitled “Risk Management,” -- published in Federal Register June 14, 2000; effective July 14, 2000
- Changes NASA Supplement to the Federal Acquisition Regulations (FAR) to emphasize considerations of risk management in the acquisition process

## Strategy for PRA Capability Enhancement

##      Current PRA Status at NASA

**Good News:**
**Strong**** ****management interest and support**** for PRA**
**In-house experience with ****traditional FMEA**** and ****some FTA**
**Bad News:**
**Scarce and scattered PRA resources**** (people, tools, data) **
**No corporate memory on PRA**** past work and data**
**Inadequate communication and cooperation**** on PRA among Centers and with HQ**

## NASA Objective For PRA

**	**
**	****Develop a ****world-class *****in-house***** capability**** to perform, manage, and use Probabilistic Risk Assessment (PRA) methods at NASA**

## Ingredients for Success from Experience

**In-house expertise**** to perform, manage and use PRAs to make sound decisions**
**In-house ownership and corporate memory**** of PRA methods, tools, databases and results**
**Lowest dependence**** on outside help to manage, perform, understand, and use PRA methods and results to make management decisions**

## NASA PRA Capability Growth Model

**Total Dependence**
**on External**
**Consultants**

**NASA is Only a**
**Customer of Its**
**PRA Projects**

**NASA on a**
**PRA Learning**
**Curve; Takes**
**Leadership in PRA**

**Consultants**
**Provide Transfer**
**of Technology and**
**Support in PRA**

**NASA With**
**PRA Experts;**
**Manages/Performs**
**PRA Projects**

**Outside PRA**
**Consultants Just**
**Provide Support**

**Today**

**Near-Term**

**Long-Term**

-   1-2 years	         2-5 years

- Time frame

**R**
**o**
**l**
**e**
**S**
**h**
**i**
- f
- t

**PRA** **Renaissance**

**Age of PRA** **Enlightenment**

## Essential Elements for PRA Strategy

**Develop NASA ****personnel skilled in PRA ****to transfer the state-of-the-art PRA technology to**
**Develop/adopt NASA-wide ****PRA policy, procedures, computer tools**
**Develop, coordinate & maintain ****PRA databases**** (reliability, maintenance, initiating events, etc.)**
**Foster communication and cooperation**** of efforts throughout NASA**
**Integrate and coordinate PRA efforts**** throughout NASA**

##        Strategy Implementation Roles

**NASA HQ**
**Organize, coordinate and conduct awareness and practitioner PRA training**
**Develop PRA policy and procedures**
**Mentor and provide specialized PRA assistance**
**Coordinate use of PRA computer tools and consultants**
**Manage the PRA process and ****oversee PRA quality**
**NASA Centers**
**Work with HQ to implement PRA strategy**
**Manage and perform PRA projects**
**Provide local PRA expertise and support in training, consulting and mentoring PRA projects**
**Become local custodians of PRA data and software**

##            We are Up and Running ...

**Initiated work on PRA policy and PRA awareness and practitioner training**
**Started effort on PRA Procedures Guide for aerospace applications**
**Began cooperation with NRC, USG agency most experience in PRA**
**Acquired SAPHIRE PRA computer program and conducted training at HQ and Centers; Trained 50 people to date.**
**Organized PRA information exchange workshop (****this workshop****) and the development of NASA-wide PRA working group**
**Started cooperation with ESA on PRA policy and procedures**
**Planning additional workshops on PRA and on risk-based decision making for management applications**

##                      Conclusion

**NASA needs**
**your help, support and cooperation**
** ****to reach our PRA objective**
**for the benefit of**
**NASA and its Centers**