# Presentation: 009870

## Engineering Safety-Related Software

- Bob Quirk
- Defense Nuclear Facilities Safety Board
- The views expressed are those of the author.  No official support or endorsement of this work by the Defense Nuclear Facilities Safety Board is intended or should be inferred.

## Topics

**Topics**

** ****Background**
** ****Board Actions**
** ****Observations**
** ****Standards & Guidance**
** ****Key Processes**
** ****Future Evolutions**
** ****Conclusions**

## Background

**Software is used by DOE and its contractors to analyze hazards, to design effective controls, and for automatic control of safety systems**
**As a result, the safety posture of many facilities is strongly dependent on the quality of this underlying body of analysis, design, and control software**

**Notes:**
Software is used for many safety-related activities at the Department including the analysis of hazards to determine the safety significance of structures, systems, components.  Software is also used in the detailed design process to ensure that structures, systems, and components which have been designated as safety-related are sufficiently robust to reliably perform their safety functions during credible accident conditions.  In addition, software is used to perform automatic control functions for such safety systems.  And finally, software is used to provide supporting functions to the overall safety program.  One example is the linking database at the Pantex Plant which is used as a tool for the change control process.  This database provides a linkage between safety basis documents and the controls which are relied upon for safety, so that if a change is made to the safety basis, the software identifies the controls which are affected by the change. 

Clearly, the safety posture of many DOE nuclear facilities depends on the correct operation of the software used for their design, analysis, and operation. 

## What is Different about Software

**Other engineering projects are governed by laws of physics and material properties, but software engineers work with abstract materials with no natural limits**
**Software errors can be hard to find implying a need for a rigorous, well documented process, particularly for development of safety-related software**

**Notes:**
why bridges are typically built on-time and on-budget and do not typically fall down, and why that is not the case with software development. (This is not to say that bridge building has not had its share of failures. What deserves attention is the massively large rate of software failures when compared to bridge failures).  

Engineers involved in design of bridges are guided and constrained by the natural laws of the materials with which their designs must be implemented. Software engineers, on the other hand, work with abstract materials that have no natural limits imposed by the external world.  Because software is infinitely flexible and malleable, the limits are purely determined by human mental capabilities. 

## MAJOR BOARD ACTIONS

**In January 2000, DNFSB/TECH-25 identified numerous deficiencies in safety-related software at DOE; the Board asked for a corrective action plan (CAP)**
**On October 3, 2000, the Board received the CAP but found it was not responsive to the Board’s concerns**
**On September 23, 2002, the Board issued Recommendation 2002-1, Quality Assurance for Safety-Related Software **

**Notes:**
During 1999, the Board’s staff had noted numerous problems with software at DOE.  Several codes used for safety analysis were reported to have deficiencies.  Subsequent follow-up revealed that those codes did not have adequate verification and validation testing performed during development.  In addition control software at the Los Alamos Critical Experiments Facility had caused an uncontrolled reactivity insertion.  And a search of the Occurrence Reporting and Processing System yielded more than 150 reportable occurrences involving programmable logic devices.  These issues and others were communicated to DOE in DNFSB/TECH-25, Quality Assurance for Safety-Related Software at the Department of Energy Defense Nuclear Facilities, in January 2000.  In the transmittal letter to the Deputy Secretary, the Board asked DOE for a plan of action to address the software quality concerns raised in the report.

The Board received the Department’s corrective action plan on October 3, 2000, over six months past the due date.  Unfortunately, the report was not sufficiently responsive to the Board’s concerns.

## Ongoing Efforts by the Board

**The Board held three public meetings on QA with SQA as a special interest item**
**The staff has reviewed directives and guidance from other agencies as well as industry consensus standards and visited the NASA Independent Verification and Validation Center**
**The staff completed several on-site reviews of SQA activities at Y-12, SNL, Hanford, Pantex, LANL, and SRS**
**Followup process implementation reviews are in progress**

**Notes:**
The Board decided to independently pursue a course of investigation into the software quality assurance practices by other industries and to perform detailed on-site reviews of software practices at DOE sites.  These actions were taken in order to better understand the issues.

The Board held two public meetings in which software quality assurance was a special interest item. In addition, the Board’s staff visited the NASA independent verification and validation facility in West Virginia, and has visited five DOE sites with defense nuclear facilities to assess their programs in detail.  These sites included Y-12 at Oak Ridge, Tennessee; Sandia National Laboratories, in Albuquerque, New Mexico; the Hanford Reservation in Washington state; the Pantex Plant near Amarillo, Texas, and Savanah River Site in Aiken, SC.   

## Key Observations

**Expectations for SQA in DOE directives and guidance are not clearly set **
**Responsibility and authority for SQA functions within DOE are not adequately defined, nor is there an effective champion**

**Notes:**
The following charts summarize the key observations made by the staff as a result of these activities.  

While there is some good guidance in the DOE directives on software quality assurance, the method used to promulgate these requirements does not clearly set expectations for software quality assurance, particularly for safety-related software.  The approach is essentially that of “throwing the hay out where the goats can get it.”  As a result, contractors are forced to develop their own programs.  On-site reviews have shown that contractor implementing procedures do not have sufficient detail to define a robust process or ensure high quality software products.

Responsibilities and authorities for the safety-related aspects of software quality are not clearly defined, nor is there an effective champion for software quality assurance within the Department.  This is a complicated problem since the Clinger-Cohen Act assigns certain policy and oversight responsibilities to the Chief Information Officer, but safety policy and oversight responsibilities lie elsewhere.  And the problem is further complicated by the recent reorganization in the Department.

## Key Observations  (cont.)

**There is no formal DOE training program for SQA**
**Contractor implementing procedures do not have sufficient detail to define a process which will ensure quality software products**

**Notes:**

It is perhaps no surprise that there is no consensus set of training requirements for software quality assurance, because as we have just seen there are no clearly defined set of software quality assurance requirements. As a result, there is no formal DOE training program for software quality assurance, and most contractors do not have formal software quality assurance training.  This was also a finding in the Department’s Training Focus Area Team report, Summary Report on Training to Department of Energy Lead Principal Secretarial Officers.  

## BUT ....

**Accidents are typically the result of the complex interaction of Product/Process/People/Resources**
**All must be balanced to ensure safety**
**Process alone is not the answer**

## Non-Nuclear Software Safety Standards

**IEEE: IEEE STD 1228-1994, Standard for Software Safety Plans**
**NASA: *****NASA-STD-8719.13A,***** *****Software Safety***** and *****NASA NASA-GB-8719.13,***** *****Guidebook for safety-critical systems***
**DoD: *****MIL-STD-882D, *****Standard Practice for System Safety**
**U.K. Ministry of Defense: *****DEF STAN 00-55, Requirements for safety-related software***
**Joint Services Software Safety Committee, G-48 Committee, *****Software System Safety Handbook***** **
**NATO Standardization Agreement,***** STANAG 4404*****: *****Safety design requirements and guidelines for computing systems***
**IEC:IEC 601-1-4, Medical Electric Equipment – Part 1: General Requirements for Safety – 4. Collateral Standard: Programmable Electrical Medical Systems  **

## Nuclear Industry Software Safety Standards

**IEEE STD***** *****7-4.3.2-1993**
**NQA-1 & NQA-2 Subpart 2.7**
**NRC Reg Guides, BTPs, & NUREGs**
**ANSI/ANS 10.4 & 8.1**
**IEC 60880:1986-09**
**CE-1001-STD Rev.1**
**IAEA-TECDOC-1066**
**IAEA Safety Guide NS-G-1.1**

## General Software Engineering Standards

| Process  | Standards |
| --- | --- |
| Glossary | IEEE 610.12 |
| Life-Cycle | IEEE 1074 |
| Project Mgmt | IEEE 1490 & 1058 |
| Safety Review Plans | IEEE 1028 |
| SQA Plans | IEEE 730 |
| Requirements Spec | IEEE 830 |
| Design & Reliable SW | IEEE 1016 & 982.1 |
| Safety Plan | IEEE 1228 |
| V&V | IEEE 1012 |
| Config Mgmt | IEEE 828 & 1042 |
| Documentation | IEEE 829 & 1063 |
| Maintenance | IEEE 1219 |

## Software Safety Process

**Concept of Ops**
**Software Requirements Specification**
**Hi-Level Design**
**Low-Level Design**

** ****Acceptance Test**
** ****System Test**
** ****Integration Test**
** ****Unit Test**

**Safety Assessment Occurs in All Phases**

**Notes:**
Section 4 provides guidance on the management of the Software Systems Safety Program. It also describes the process of using the Preliminary Hazards Analysis to develop safety design requirements for inclusion in the high level software requirements specifications and the tailoring of generic guidelines to the system under development. It also contains guidance on identifying functionality in the software that is safety critical or safety related. It continues through the software development process providing how-to guidance on analyzing the implementation of the safety design and safety critical requirements through design to code and testing. Finally, it give guidance on assessing the residual risk associated with the software in the system context. The Safety Assessment portion is another that requires additional work.

## Tailoring the Software Safety Effort

**Identify safety critical software **
**Determine how critical the software is (classification)**
**Determine the extent of the development effort and oversight required**** **

## Software Requirements Phase

| Milestone Reviews | Software Safety Tasks |
| --- | --- |
| Software Requirements Review 

Preliminary Safety Review
 | -Development of Safety Requirements
-Requirements Analysis
-Specification Analysis
-Software Fault Tree Analysis
-Timing, Throughput, and Sizing Analysis
 |

## Architectural or Preliminary Design

| Milestone Reviews | Software Safety Tasks |
| --- | --- |
| Software Preliminary Design Review (PDR)

 | -Update Requirements Analysis
-Conduct Hazard Assessment
-Analyze Design
-Interdependence & Independence Analysis
-Formal Methods |

## Detailed Design 

| Milestone Reviews | Software Safety Tasks |
| --- | --- |
| Software Critical Design Review 

 | -Design Data, Logic, & Interface Analysis
-Dynamic Flowgraph Analysis
-Software Failure Modes and Effects Analysis  |

## Software Implementation 

| Milestone Reviews | Software Safety Tasks |
| --- | --- |
| Formal Inspections

 | -Code Data, Logic, & Interface Analysis
-Code Inspections
-Unused Code Analysis
-Interrupt Analysis  |

## Software Integration and Test

| Milestone Reviews | Software Safety Tasks |
| --- | --- |
| Test Readiness Review

 | -Test Coverage Analysis (every node and path)
-Software Safety Testing
-Test Results Analysis
 |

## Software Maintenance 

**Same Activities as in Development**

## Future Evolutions

**ANSI/ANS 10.4 Revisions**
**NQA-1 Revisions**
**Federal Working Group Meeting on SQA for safety-related software**
**Oversee execution of the Implementation Plan for DNFSB Recommendation  2002-1**

## Questions