# Presentation: 442681

## Believe it or not, Software Assurance Affects You  Too

**Martha S. Wetherholt**
**NASA Office of Safety & Mission Assurance**
**mwetherh@hq.nasa.gov**

## What is Software Assurance

**Software Assurance (SA) includes:**
**Software Quality Engineering**
**Software Assurance of Product and Processes**
_**Software Safety**_
**Software Independent Verification & Validation (IV&V)**
**Software Reliability**
**It is,**
**Software Risk Management**

## What Makes Software, Safety Critical?

**What Makes Software, Safety Critical?**

**Software that directly or indirectly contributes to the occurrence of a hazardous system state**

**Notes:**
So how does one contribute to the occurrence of a hazardous state anyway?
First the software must be connected somehow to a critical function.  That is, the software must first be capable of affecting a safety critical function, a function that “must work” or  “must not work”.  To prove this is or is not the case may take some analysis (we’ll get to that later).
Only the ability to affect a function in a manner that causes a hazardous system state is necessary to classify the software as safety critical.

## What Makes Software Hazardous?

**Software is hazardous if it:**
**Controls**** hazardous or safety critical hardware**
**Monitors ****safety critical hardware as part of a hazard control**
**Provides information upon which a safety-related decision is made**
**Performs analysis that impacts automatic or manual hazardous operations**
**Verifies ****hardware hazard controls**
**Is used to verify safety critical hardware and/or software**
**Is used to model or simulate safety critical applications**

**Notes:**
How does software contribute to the hazardous world we know and are trying to build

## Software Safety - Why care?

**One or more people are injured or worse**
**Regulatory requirements (e.g. OSHA, UL, etc.)**
**NASA requirements**
**Liability if software fails**
**Reputation (business or personal)**
**“****Good practice” for mission critical or business critical software**

**Notes:**
Now days we need to worry about more than the obvious “safety” issue of injury and potentially death.  In this make it  or break it business world we must be aware of any regulatory requirements, perhaps your company’s good reputation can make your system “critical”.  There is always the chance of getting sued these days even if the product doesn’t injure some one – failure to perform as expected is enough cause in some arenas.  So what we will be talking about today, though we stress safety critical, can be used as good practice and good approaches for mission/business critical systems as well. 

## Why Should We Care, What does it have to do with me?

**Software control of facilities**
**Wind tunnels**
**Simulators**
**Centrifuges**
**Shake & Bake **
**EMI testing**
**Engine Checkout**
**Etc.**
**Software control and monitoring of safety critical projects which run in changeable labs**
**Labs/Tools**
**Vibe tables ........**

## What should be done

**Train Area Safety Managers/Health and Safety Personnel **
**Invite Software Assurance along on Facility set up and changes**
**Ask the right Questions**
**How is this experiment/facility controlled**
**How is it monitored**
**What is the human interface**
**Does software detect and react to safety critical situations**
**How – what is it expected to do**
**What testing was performed on the Consumer Off the Shelf (COTS) Software Purchased to operate the Facility/Experiment**
**What software development processes are to be used to develop the software – including Application SW**
**How are the COTS and Applications written Configuration Managed**
**Does the Software perform a logging function to track faults, failures, errors, etc.   How often is it viewed? By who?**

## Creating Safer Software & Safer Systems

**Creating Safer Software & Safer Systems**

**Good SW Development Process**
**Development Tools**
**Appropriate Reviews**
**Diverse Review Teams**
**Formal Inspections**
**Communication**
**Appropriate Analysis, both Safety & Development**
**Caveat Emptor**

**Notes:**
Once the level of effort is defined via a method like in the last few slides, for software these items are where we determine our level of effort.
Thus far the discussion has been centered on what can be done to create safe systems from the standpoint of identifying, then eliminating or mitigating hazards.  Methods of developing safer software, and not introducing hazards, includes establishing a good software development process.  There are many procedures and methodologies for developing software.
Good SW Development Process - Help Prevent Introduction of New Hazards
Development Tools - To Uncover Errors.
Reviews - To Uncover Errors.
Software Inspections - To Uncover Errors.
Communication - To better understand and create the right, safe system.
Appropriate Analysis - To discover potential hazards and design approaches.
Buyer beware - for COTS and also contracted software.

## Safety Verification Testing

**Safety tests designated for each hazard control.**
**Verify partitions, firewalls, or other software constructs that isolate safety critical code.**
**“****Fail” the hazard controls in a multi-tolerant system.  For example, in a two-fault-tolerant system (three controls), try all combinations of two failures.**
**Verify hazardous commands.**
**Verify software correctly handles out of sequence commands, hazardous commands issued in an incorrect state, and other possible errors.**
**Software Safety (usually SQA) should witness all software safety testing.**

**Notes:**
All the great safety features you designed and coded now need to be tested.  
Formal testing will verify each hazard control or mitigation that software is a part of, as well as hazardous commands. This testing should have written test plan that is reviewed/signed off on, be witnessed by Software Safety or SQA, and be carried out according to a written procedure.
Slightly less formal safety testing would check the software FDIR/error handling system, look for ways that the protection systems might be accidentally circumvented, and other types of problems.  

## When  COTS Software Is UsedSOUP (SW of Uncertain Parentage)

**Is this a case of Reuse?**
**Previous environmental criteria may or may not be valid *****in the new system.***
**Must test COTS for ways it can fail. STAND-ALONE**
**Must test for how system faults/failures affect the COTS and the applications they run on.**
**How does your application software respond to those failures – how does it effect the system, humans, etc.?**
**What of unused portions/features of COTS software? Can they influence the safety critical operations & monitoring.****  **
**Stand-alone testing of all functionality prior to integration in lab or facility. **
**How much Glue-ware and/or wrappers.**

**Notes:**
Ariane 5 - reused Ariane 4 software but in new system/environment.  It worked as designed, but not as desired!
Software fault injection can be used to find out how the COTS can fail, so you can protect your system from those failures.  Examples of COTS failures: 
overwriting a memory location where a safety critical variable is stored
getting into a failure mode where the OTS software uses all the available resources and prevents the safety critical code from executing
clogging the message queue so that safety critical messages do not get through in a timely manner.
Be aware that glueware and wrappers (code to integrate the COTS software and to “buffer” the COTS) may be necessary.  The earlier you find out how the COTS software works, the earlier you can add these types of software to your design.  If you wait until the testing phase to really understand the COTS software, you may find you need to add wrappers to protect your code from malfunctioning COTS!

## What are we asking you to do?

**Be *****AWARE***
**Know what to look for **
**What Questions to ask**
**What are you buying**
**Be Proactive**
**Put Software into Assessment process & plan **
**Train your people**
**Document safety requirements then **_**Test**_** for them**
**Work with SW Assurance**

## Background/Extras

## What Software is included

**Software**
**Firmware**
**Programmable Logic Devices**
**ASICs - Application Specific Integrated Circuits**
**FPGAs  - Field Programmable Gate Arrays**
**COTS Software**
**Program Logic Control**
**Databases**
**Operating Systems**
**Ad infinitum **

**Notes:**
Software is all these things though some try to define some of them as hardware.  In deed at times the lines do seem to blend in the case of ASICs and FPGAs
Still when you look at  what goes into “programming” these devices, it can be seen that while they end up as basic hardware devices, their configuration is determined by the same means as if creating a program to run on a CPU.
Programmable Logic has come a long way to and while in some ways still resembles the hardware switches and relays it was created to replace, it is still programmable and thus must be analyzed, developed and tested as software. And when operating large equipment or safety critical ones, they need to be assessed for safety criticality.

## Categories of S/W Safety Functions

**Caution And  Warning Functions **
**Failure Detection, Isolation, and R---------**
**Recovery**
**Restart**
**Reduced operation**
**Automatic safing software**
**Hot, Warm, Cold Backup**
**Autonomous Decision Making & Operation**

**Notes:**
More will be added to these categories and they can be expanded upon.  In the next few slides we will give some examples to further your thinking along these lines.  But due to time, much will need to be left to the paper or better yet, to your own discovery and decisions.

## Safety critical software is:

_**Safety critical software is:**_

**Software which controls or monitors safety critical functions including mitigation of hazards**
**Software which runs on the same system as safety critical software or impacts systems which run safety critical software**
**Software which handles safety critical data**
**Software used to verify and validate safety critical hardware and software**

**Notes:**
Therefore, determining if the software is safety critical won’t take a rocket scientist.
These four qualifiers and definition mentioned earlier are usually enough to make a determination of criticality.  But the second qualifier can be confusing.  
What it means is if there is software resident on the system that also contains previously defined “safety critical software”, it can be thought of as safety critical as well.  This is because, being on the same system, it may be possible that a fault of non-critical software is passed on to a portion of safety critical software.  One approach to prevent the propagation of software faults is to establish Fault Containment Regions (FCRs).  Techniques known as “firewalling” or partitioning should be used to provide sufficient isolation of FCRs.   There still will be debate, of course. 
There are many who will say that “there is no safety critical software in this vehicle”, “payload”, “experiment”, or “product”.  Some will go to great lengths to remove the title of “safety critical software” from their systems.  For some systems it may be necessary to remove the software from the system, or have a minimum of three mechanical inhibits to make the software superfluous.  So then, what is the purpose of the software?
In the past, hardware was always the first and last line of defense.  Software just hasn’t been trustworthy.  But if properly developed and analyzed, software can perform up to its new obligations in the world of faster, better, cheaper and still be SAFE.

## Categories of S/W Safety Functions

**Must work/must not work functions**
**Mode and State Dependent**
**Must never work**
**Must never fail to work**
**Fault tolerance **
**Redundancy**
**How many levels and where are they best put**
**Multiple Commanding (Ready, Arm, Fire)**

**Notes:**
Thinking about software in these categories often helps define the problem so as to better work it.  While these may not be the only categories of safety critical software, they do cover a wide range of the operational software we need to consider.  When we think of software as performing these functions or when they are thus described within their specification or requirements document, it is much easier to address their consequent analyses and operations.

## Stability/Reliability Testing

**How well does the system operate for extended periods of time?**
**System is run in normal mode of operation - occasional peak performance allowed, but not stress testing the system.**
**Can the system handle intermittent bad data?**
**Is there a sensitivity to event sequences?**
**Does memory leakage cause problems after a period of time?**

**Notes:**
Unlike stress testing, stability or reliability tests are run with the system operating normally.  Getting up to occasional peak loads is part of running “normally”. 
You do want to input “expected” errors and anomalies, such as a user mistyping a command, two sensors wanting servicing at the same time, or other situations that may occur.  Keep away from the really unlikely errors for this type of testing.  

## Summary

**Determine software control and complexity**
**Determine each software portions’ contribution to safety**
**Establish categories with a cross index to hazard level**
**Determine a level of effort needed to assure safer software**
**Further tailor the effort to your particular needs and situation**

**Notes:**
There is just too little time to go into this and we must only give you a briefest look into what is possible. Of course, you should carefully consider what is needed for your organization and how best to categorize and tailor your efforts.  We are just giving you an example here.