# Presentation: 069426

## Software Fault Injection

- Kalynnda Berens
- Science Applications International Corporation NASA Glenn Research Center

## What is Software Fault Injection?

- A testing technique that aids in understanding how software behaves when stressed in unusual ways.
- A *product*-based assurance technique.
- Variations in the technique allow it to be applied to many types of software and for different purposes.

- Software Fault Injection        NASA Glenn Research Center

## How does SFI work?

*Legal *permutations or faults are input at interfaces (external and/or internal).
- Outputs show whether the injected fault propagates through the software.
- Requires *instrumentation* (software code) to observe the propagation process.

- Software Fault Injection        NASA Glenn Research Center

## Uses for Software Fault Injection

- Finding defects in software
- Robustness Testing
- COTS Validation/Determining failure modes
- Safety Verification
- Security Assessment
- Software Testability Analysis

- Software Fault Injection        NASA Glenn Research Center

## SFI Examples

- Operating System Validation
- Ballista (CM) – Linux and VxWorks robustness
- WindowsNT
- Network Security
- NCSA httpd server
- Safety
- Advanced Automatic Train Control system
- Magneto Stereoaxis System

- SFI can be used with or without source code

- Software Fault Injection        NASA Glenn Research Center

## SFI without Source Code

- Create software wrapper for COTS functions and other interfaces
- “Trick” OS to call wrapper functions first
- Software under test usually run in debug mode
- Wrapper can be used
- Pass through for baselining response
- Call alternative function
- Call original function but change result

- Software Fault Injection        NASA Glenn Research Center

## SFI wrapper operations

- Application

- Wrapper

- OS or
- Hardware

**Pass through wrapper:**

**Call alternative function:**

**Call original function but change result:**

- Application

- Application

- Wrapper

- Wrapper

- OS or
- Hardware

- New
- Function

- Software Fault Injection        NASA Glenn Research Center

## Center Initiative on SFI

- Can SFI be used by an **Independent***** ***V&V engineer?
- Is SFI a useful and cost-effective technique for NASA?
- Are the errors and problems found of sufficient severity or abundance?
- Are the costs of applying the technique reasonable for the number/severity of errors found?
- Is SFI a good tool for safer software?

- Software Fault Injection        NASA Glenn Research Center

## Methodology

- Determine scope
- Select projects
- Metrics
- Perform SFI on projects
- Create Test Plan (prototype due 1st quarter, FY02)
- Lessons Learned

- Software Fault Injection        NASA Glenn Research Center

## Determine Scope

- Why narrow the scope?
- SFI is a collection of related techniques
- Comparison across projects requires using one technique for all
- Why no source/interfaces technique chosen
- IV&V perspective (cost effective)
- “Outside” events or system limitations trigger many errors
- Interfaces selected
- COTS software
- Hardware
- User input
- Communications medium

- Software Fault Injection        NASA Glenn Research Center

## Project Selection

- Potential Projects
- CM-2
- Tempest Web Server (VxWorks and Java)
- MDCA, FPP, SAMS, others
- Selection Criteria
- Selection difficulties
- Project support not free
- Contracted software not accessible
- Final Choice

- Software Fault Injection        NASA Glenn Research Center

## Metrics

- Time spent per task
- Familiarization, researching errors, instrumenting software, testing
- Subjective “effort” scale per task
- Software project metrics
- SLOC, #classes/modules, complexity, interface information
- Fault Injection metrics
- #faults, #failures, #faults no effect/correctly handled

- Software Fault Injection        NASA Glenn Research Center

## SFI Process

- Obtain Tempest software (completed)
- Obtain access to VxWorks (completed)
- ***Overcome compatibility problems
- Determine all interfaces to test
- Select errors to inject
- Create necessary wrappers for SFI
- Record test procedure and results

- Software Fault Injection        NASA Glenn Research Center

## Tempest Interfaces

- VxWorks OS
- Task creation and control functions
- C/C++ language functions
- File system functions
- Networking functions
- Outside world
- Requests from external sources
- Standard HTML, built-in functions
- Tempest (VxWorks version) can execute OS functions

- Software Fault Injection        NASA Glenn Research Center

## Example Injection Errors

- OS errors
- Memory allocation failures
- File errors (corrupted, not found)
- Single task abort, hang
- External World errors
- Invalid request
- Too many requests
- Requests too frequent

- Software Fault Injection        NASA Glenn Research Center

## Test Plan

- How to perform software fault injection on “generic” software
- Steps prior to actual testing
- Method of determining errors to inject
- Procedure for performing the test
- Appendices of lessons learned, example faults, other guidance

- Software Fault Injection        NASA Glenn Research Center

## Difficulties Encountered

- Tempest documentation limited
- VxWorks simulator does not support networking
- Cost of hardware and full VxWorks not within the budget
- Attempt to “fake” networking unsuccessful

- Software Fault Injection        NASA Glenn Research Center

## Status and Future Work

- VxWorks incompatibilities not easily overcome
- Shift to Java version of Tempest for now
- Test VxWorks version of Tempest on actual hardware (if possible) or alternate operating system (Linux, uClinux, eCos)
- If funding continues, test on actual flight experiment (CM-2).

- Software Fault Injection        NASA Glenn Research Center