# Presentation: 233009

## NASA OSMA SAS ‘03

**Software Requirements Analysis**
** ****As Fault Predictor**
**Dolores R. Wallace**
- SRS Technologies
- Software Assurance Technology Center
- http://satc.gsfc.nasa.gov
- dwallac@pop300.nasa.gov
**William M. Wilson**
- SRS Technologies/SATC
- wilsonw@pop300.nasa.gov

- SAS03/FaultPrediction

## Software Requirements Analysis As Fault Predictor 

- Presentation Outline
- Research Rationale & Objective
- Requested Project Data
- Planned Approach
- Data Acquisition & Analysis Obstacles
- Results & Findings
- Conclusions
- Recommendations

- SAS03/FaultPrediction

## Research Rationale & Objective

**Rationale**
- The earlier in the lifecycle that software faults can be found the less expensive it is to correct them.
- The earliest opportunity to preclude software faults is during the development of the system's requirements.
**Objective**
- To determine if software requirements analysis results can be used as a predictor of software faults.

- SAS03/FaultPrediction

## Requested Project Data 

**Requirements Specification**
- Accessible in electronic media in MS Word format for use with the ARM tool
- Conforming to NASA-STD-2100
- Specification statements identified by hierarchical numbers

- SAS03/FaultPrediction

## Requested Project Data – Cont.

**Verification Test Matrix**
- Requirements to Test or vice versa
-       **- OR -**
**Traceability Matrix**
- Requirements to Design Elements
- Design Element To Software Component
- 	**- AND -**
**Test Plans**
- Tests vs. Software Components.

- SAS03/FaultPrediction

## Requested Project Data – Cont.

**Failure Reports/Data**
- 	 -   Preferably in DDTS format
- Including: test id, module id, CSCI id, release/build id, suspected cause of failure and resolution (e.g., actual cause of failure - what was fixed)
**Configuration Control Reports**
- Baseline item changed  -e.g., Requirement, Design, Code
- Reason/Source of  change – e.g., RID, Failure Report, PR, etc.

- SAS03/FaultPrediction

## Planned Approach

- Use ARM tool to analyze projects’ software requirements.
- Collected projects’ failure data in a DDTS database.
- Organize the data to represent same components for each project.
- Conduct correlation analysis.
- Study results of step 4 to prove or disprove hypothesis.

- SAS03/FaultPrediction

## ARM TOOL Reports

- Identifies and Counts:
**Size of Requirements Document**
- Lines of Text
- Numbered Paragraphs
- Number of Specification Statements
- Number of Unique Specification Subjects
**Number of Specifications Containing**
- Weak, Optional and/or Incomplete Phrases.
- Compound and/or Complex Statements.
**Requirements Document Structure**
- Depth and Distribution of Specifications

- SAS03/FaultPrediction

## Data Acquisition & Analysis Obstacles

**Common Problems**
- No commonality or standardization of data or formats across project.
- Current projects are reluctant to provide data to outside organizations.
- Data from completed projects is incomplete, not in a usable form or is not accessible.

- SAS03/FaultPrediction

## Data Acquisition & Analysis Obstacles Cont.

**Specific Problems**
- No project staff available to guide through the mass of documents in the collection
- Documents locked on a WEB site
- Documents in “.pdf” format
- Variation of format and media within project
- Complete set of data for a specific Build/Release not available
- Data held by support contractor and released only on a “Project Need-To-Know” basis

- SAS03/FaultPrediction

## Results & Findings

- Data provided by four projects
**Projects A and B**
-  Major subsystems of operational information processing system
**Projects C and D**
- Flight instrumentation packages

- SAS03/FaultPrediction

## Results & Findings Cont

- SAS03/FaultPrediction

## Conclusions

- Data from _many_ more projects is required to improve the possibility of finding a number of data sets with sufficient commonality to support analysis.
- Analysis approach needs to be expanded to include determining if failures attributed to documentation and design flaws are in reality related to deficiencies in requirements.

- SAS03/FaultPrediction

## Recommendations

- Advocate and support:
- The use of NASA-STD-2100 documentation standard
- The use of a format to report problems and failures that is compatible with DDTS
- The creation and use of centralized Center repositories  for data from completed projects.

- SAS03/FaultPrediction

## Summary

- Lessons
- Getting data requires strong, interested Civil servant advocate
- Analyzing data requires tedious effort, even with automated tools
- Future
- Analyze latest set of data
- Attempt correlations for the 4 data sets

- SAS03/FaultPrediction