# Presentation: 160423

## New Software Quality Assurance Guidance forConsequence Assessment Models

**New Software Quality Assurance Guidan****ce for****Consequence Assessment Models**

- Cliff Glantz
- Pacific Northwest National Laboratory
- _[cliff.glantz@pnl.gov](mailto:cliff.glantz@pnl.gov)_    509-375-2166
- May 8, 2008

## Roadmap 

- Intro to DOE software quality assurance (SQA) requirements for safety software
- The difference between safety and safety-related software
- Problems applying SQA for safety software to safety-related software
- An alternate method of SQA for safety-related software
- A path forward...

## Background Info

- In 2000 the Defense Nuclear Facility Safety Board (DNFSB) issued a report critical of the SQA of safety software at DOE nuclear facilities
- After much back & forth, in 2005 DOE issued an Order and Guide establishing SQA requirements for safety software:
**DOE Order 414.1c**
**DOE Guide 414.1-4**

## Safety Software is Defined to Include:  

- Software that’s “performs a safety function” as part of a structure, system, or component (SSC) at a nuclear facility.
**Software that is used to classify, design, or analyze nuclear facilities**.  This software helps to ensure the proper accident or hazards analysis of nuclear facilities or an SSC with a safety function
**Software that performs a hazard control function** or **a control function**** **necessary to provide adequate protection from hazards.  This software supports** “**eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment...”

## Application of the SQA Order 

**Consequence assessment** models
- are safety software if they are used
- for:
**hazards assessment/safety planning** purposes.
**emergency response purposes** and provide a direct hazard control function.   These are models used to make protection action recommendations (PARs), such as HOTSPOT, EPI, ALOHA,...

## Application of the SQA Order...

**Consequence assessment **models are _**not**_
- safety software if they are used for:
- estimates of exposures at downwind distances where the radiological impacts would **not** **be hazardous **
- estimates plume position or deposition locations for purpose of **deploying field measurement teams **(their radiological monitoring will be used to make  safety decisions for themselves and others)
**sophisticated modeling** of plume movement or impacts that do not serve as the basis for protective action decisions.

## Why Not Apply DOE O 414.1C to Safety-Related Software?

- The average cost to bring widely used simple codes (e.g., GENII, EPICODE) into full compliance with DOE O 414.1C is >$300K/model.
- For more sophisticated models, these costs would be greater.
- Need to find a balance that allows the effective use of resources (i.e., balance technical development  needs against SQA needs).
- Remember – SQA helps to ensure that software does what it is _**designed**_ to do, not that it is doing the right thing!

## Finding the Right Balance

* **Safety** *software* *must comply with DOE O 414.1C
- For safety-related software, SCAPA wants to adopt  an appropriate approach to SQA.
- Be **based on** the DOE SQA Order and Guide – though it should be less burdensome while emphasizing key elements
- Be **compatible** with site- or contractor-specific requirements
- We want to _encourage_ an improvement in SQA and the continued development of better models
- We want to be assertive and **define guidance **for safety-related applications before this guidance is prescribed by an external group.

## What are the SQA Requirements for Safety Software?

- Requirements have been established for custom developed software, COTS, utility calcs (e.g., spreadsheets)
- For each type of software, three different levels (A, B, or C) can be assigned based on applications.
**Ten work activities** are specified in the SQA requirements.  Depending on the SQA level, a full or graded approach to each the work activity is required.
- SQA work must be based on **ASME NQA-1-2000**, *Quality Assurance Requirements for Nuclear Facility Applications*, or other national or international consensus standards that provide an equivalent level of quality assurance.

## Slide 10

## Safety Software SQA Level of Effort 

## Safety-Related Software SQA Level of Effort

## Draft SQA Guidance for Safety-Related Software

- ...

## Proposed SQA Guidance... 

- Review each of the ten SQA work activities
- Determine which activity elements are pertinent for the software (i.e., which do and do not need to be  addressed in detail)
- Develop **SQA *****Project Management and Quality Planning ***documentation.
- Describe needed SQA activities
- Establish SQA milestones
- Assign SQA responsibilities.

## Proposed SQA Guidance...

- Prepare **Design and Implementation documentation** to clearly detail how the software works.  Including:
- sufficient information to support **continuity** in software development and maintenance if there were an abrupt change in personnel
**provide users with sufficient technical documentation** to efficiently use the software and understand what it is doing.
- Develop a **Configuration Management Plan **to:
- Ensure configuration control during software development
- Ensure secure storage of the source code, executables, software documentation, V&V test procedures & results, and SQA documentation.

## Proposed SQA Guidance...

- Design, conduct, and document **V&V Testing**.  Focus:
- on the portions of the code that were modified
- include baseline testing
- include independent testing by someone not on the software development team.
- Develop and implement **Problem Reporting/Tracking **
**Train** the software development team and prepare **user’s guides**

## Applying Safety-Related Software SQA

- SQA must be applied during the **development **of new software and the **modification** of existing software.
- For **legacy software** that is not undergoing modification, there is no requirement to apply this SQA guidance retroactively.
- If resources are available, **priorities** are to ensure that for legacy software:
- code documentation and verification testing data sets are produced
- V&V testing is conducted
- materials are placed into configuration management

## Goals

- We want DOE software model developers to meet a **reasonable **set of SQA requirements for safety-related software.  In many cases this will improve upon current SQA practices.
- Want **consistent standards** to be applied across the complex for developing and maintaining safety-related software.
- Want to encourage software “maintainers” to  **upgrade documentation and V&V testing** materials for** legacy software**
- Want to **encourage technical innovation **and avoid having to apply “stifling” SQA requirements

## Path Forward 

**May**:
- Check on EFCOG progress
- Submit draft to NA-41 for review
**June**:
- Submit draft to SCAPA membership for review
**August: **
- Submit to DOE Health, Safety & Security’s (HSS) Quality Assurance group for review
**Sept**:
- Post SQA guidance on SCAPA website
- Model owners evaluate compliance with guidance/identify gaps