# Presentation: 263064

## Integrating Software into PRA

Presented by C. Smidts

Center for Reliability Engineering
University of Maryland

- Presented by C. Smidts
- Center for Reliability Engineering
- University of Maryland

- Center for Reliability Engineering

- June 10 2003

## Integrating Software into PRA

- 	Probabilistic Risk Assessment (PRA) is a technique to assess the probability of failure or success of a mission. Current PRA neglects the contributions of software to the risk of the mission. The objective of our research is to extend current PRA methodology to integrate software in the risk assessment process.

- Center for Reliability Engineering

## What We Have Done  to Date

**Built a Software Failure Mode Taxonomy **
**Failure Modes’ Quantification: generic high-level data**
**Public Literature**
**Expert Opinion**
**Collaborated with JSC through Ms. Alice Lee**
**Validate Our Methodology**
**Collect Data**
**Developed a Test-Based Methodology for Integrating Software Into PRA**

- Center for Reliability Engineering

## What We Are Planning to Do in the Future

**Investigate Scalability Issues of the Test-based Approach**
**Continue the validation of our methodology with JSC**
**Apply the approach to JSC system**
**Revise the methodology based on NASA system**
**Develop an Analytical Approach**
**Apply the Analytical Approach to JSC system**
**Revise the Analytical Approach based on NASA system**

- Center for Reliability Engineering

## Integrating Software into PRA:
A Test-based Approach

Presented by C. Smidts

C. Smidts, B. Li, M. Li 
Center for Reliability Engineering
University of Maryland

- A Test-based Approach
- Presented by C. Smidts
- C. Smidts, B. Li, M. Li
- Center for Reliability Engineering
- University of Maryland

- Center for Reliability Engineering

- June 10 2003

## Integrating Software into PRA - Approach

**We are working on an approach to integrate software into PRA.**
**Step 1: Identify events/components controlled/supported by software in MLD, accident scenarios, fault trees.**
**Step 2: Specify the functions involved**
**Step 3: Model software functions in ESDs/ETs and Fault Trees **
**Step 4: Construct the input tree**
**Step 5: Quantify the input tree**
**Step 6: Develop and perform software safety tests**

- Center for Reliability Engineering

## Example System

**	****An exit system in a building is used as the example in this case study.**
**	****The exit system includes an emergency exit system and the PACS system. **
** **
**	****The Emergency exit system includes an emergency exit door and a marked egress router. It provides an escape route for personnel located inside the building during emergency situations.**
**	****The PACS system is a simplified version of an automated personal entry/exit access system used to provide privileged physical access to rooms /buildings, etc. Personal ID and PIN are needed to access this system.**** **

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 1: Identify events/components
- Identify events/components controlled/supported by software in MLD, accident scenarios, fault trees.
- For all such events,  create/expand contributors to account for software.
- Verify that no neglected “events” may now have become possible due to software.

- Center for Reliability Engineering

## MLD

- Center for Reliability Engineering

## MLD

- Center for Reliability Engineering

## Accident Description

- Fire is the initiating event
- Response systems: Emergency system and PACS system
- End State: Loss of life

- Center for Reliability Engineering

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 2: Specify the functions involved
- Not all software functions are involved in accident scenarios, i.e, not all software functions are involved in particular scenarios/fault trees or even in the entire realm of possible scenarios/fault trees.
- To identify the specific functions involved in a scenario, determine the specific input to/output from the software – this will describe one function.
- A list of possible functions can be found in the requirements.
- Match the input/output combinations of these functions to the risk model

- Center for Reliability Engineering

## Integrating software into PRA – Approach

**PACS Functional Decomposition**

- Center for Reliability Engineering

## Integrating software into PRA - Approach

**Actions and their inputs and outputs**

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 3: Modeling software function in ESDs/ETs and Fault trees
- In the ESDs/ETs, the function of interest is modeled as

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 3: Modeling software function in ESDs/ETs and Fault trees
- In the fault tree, the function of interest is modeled as

- Center for Reliability Engineering

## Integrating software into PRA - ESD

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 4: Input Tree
- Build the input tree for the particular function involved
- The input tree is a decomposition of the space of possibilities
- The input tree is mostly generic for a function. But may VARY due to context.(i.e. probabilities of basic events may vary, certain events may conflict with the rest of the scenario conditions.)

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 4: Input Tree

- Center for Reliability Engineering

## Input Fault Tree

**Input Fault Tree for SW1**

- Center for Reliability Engineering

## Input Fault Tree

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 5: Quantify the input tree

- Center for Reliability Engineering

## Integrating software into PRA - Approach

- Step 6: Develop and perform software safety tests
- These tests’ unique objective is to answer the questions contained in the model, i.e. in the MLD, accident scenarios and fault tree.
- The test is completely automated using Test Generation/test execution tools (TestMaster/WinRunner).
- The process is as follows:
- Build a finite State Machine model of the software by following the software functional decomposition derived from the risk model and the software requirements.
- Derive the test profile and output conditions to be quantified from the risk model
- Define and run the test cases according to the following test strategy
- Analysis consists in computing the probabilities of the different outcomes based on the test data.

- Center for Reliability Engineering

## TestMaster Model

- Center for Reliability Engineering

## Test Script Example

- win_activate ("mmount-76.umd.edu - CRT");
- start_time1= get_time();
- type ("1<kReturn>");
- Check_Message(Message_b,1);
- type ("0<kReturn>");
- Check_Message(Message_c,1);
- type ("155721495<kReturn>");
- Check_Message(Message_b,1);
- type ("0<kReturn>");
- Check_Message(Message_c,1);
- type("GayyardLupieN<kReturn>");
- Check_Message(Message_b,1);
- type("1<kReturn>");
- end_time1=get_time();
- report_msg("Cardtime is "&(end_time1-start_time1)"Seconds");
- start_time2= get_time();
- Check_Message(Message_d,1);
- wait( 9);
- type("4");
- Check_Message(Message_e, 1);
- wait(3);
- type("5");
- Check_Message(Message_f, 1);
- wait(1);
- type("1");
- Check_Message(Message_g, 1);
- wait(3);
- type("9");
- end_time2=get_time();
- report_msg("PINtime is "&(end_time2-start_time2)"Seconds");
- Case_Judge(Message_a,1);

- Center for Reliability Engineering

## Test Profile

**Test Profile for PACS**

- Center for Reliability Engineering

## Failure modes application

**Test Case Selection**
**Sample from the profile/input tree to see whether we have a “Normal” or an “Abnormal Input”.**
**If it is a normal input, select randomly from the “Normal Input” domain.**
**If it is an abnormal input, randomly select the failure mode according to the profile/input tree.**
**Then randomly select the “base”value from the “Normal Input” domain and mutate this “base” value using the rules given below:**

- Center for Reliability Engineering

## Testing Results

- 	200 cases have been tested for SW1 and SW2. 19 cases failed.

- 	SW1 fails in only one case (58).  Therefore, the point-estimate probability of Card failure is 1/200=0.005.
- 	18 cases failed for SW2. Therefore, the unsafe probability (gate closed) is 18/199 =0.09.

- 	**Failed cases classification**

- Center for Reliability Engineering

## Testing Results

- Card time and Probability

- Center for Reliability Engineering

## Testing Results

**PIN time and Probability**

- Center for Reliability Engineering

## Test Cases Coverage

**Input Failure Modes Coverage (SW1)**

**Input Failure Modes Coverage (SW2)**

- Center for Reliability Engineering

## ESD

- Center for Reliability Engineering

## Future Work

- Represent hardware-related input failure modes in test model
- Quantification of input fault tree based on field data
- Output failure modes/Support failure modes
- Sensitivity Analysis
- Scalability
- Test case generation
- Test case execution
- Number of test cases for each software component

- Center for Reliability Engineering