# Presentation: 273854

## Fault-Based Analysis: Improving IV&V Through Requirements Risk Reduction '02

- Jane Hayes
- Rama Bireddy
- D.N. American
- SAIC

- Department of Computer Science
- University of Kentucky

## Outline

- Research Objective
- Research Approach
- Progress to Date
- Current Plans
- Future Work

## Research Objective

- To improve how we focus resources for IV&V of Critical/Catastrophic High-Risk (CCHR) software functions,
- we use a fault-based analysis method comprised of:
-  a requirements fault taxonomy (Phase I)
-  a method for extending taxonomies (Phase I)
-  a taxonomy of IV&V techniques (Phase I and II)
-  what faults the techniques can detect (Phase II+)
-  a fault-based risk assessment per Class and project (Phase I+)
-  a cost-benefit analysis of technique effectiveness (validated) (Phase II+)

## Research Approach (Phase I)

- Task 1 – Select a Known Fault Taxonomy
- Task 2, 3 – PMR 2,3 (Presentation and Milestone Meeting)
- Task 4 – Examine NASA-specific requirements faults
- Task 5 – Build a list of IV&V techniques
- Task 6 – Adopt or build a method for extending the taxonomy
- Task 7 –Implement the method to extend the fault taxonomy
- Task 8,9 - Year-end report and presentation (also PMR 4)

## Progress to Date

- NUREG/CR-6316 basis of general taxonomy
- Literature survey (55 references) resulted in three** **additions to taxonomy
- Review of defect data for 3 NASA projects resulted in two additions and re-organization of taxonomy

## General Taxonomy

- _[Taxonomy figure](C:\Documents and Settings\t_carroll\Application Data\Qualcomm\Eudora\Attach\sas_taxonomy.ppt)_

## NASA Requirement Faults

- Have proven difficult to obtain
- Level of detail varies greatly
- Have thus far received and examined
- IV&V “comments” on requirement problems for 3 projects
- Project fault reports related to requirements for 1 project
- Data very useful, resulted in several changes to fault taxonomy and taxonomy extension/tailoring processes

## Task 6

- Process for extending fault taxonomy split into two parts: Process A and Process B
- Process A - activities to develop a Class-specific taxonomy
- Outputs of Process A are inputs to Process B
- Process B – activities to develop a project-specific taxonomy

## Processes for Extending Fault Taxonomies

- _[High level process](C:\Documents and Settings\t_carroll\Application Data\Qualcomm\Eudora\Attach\sas_process.ppt)_

## Class-Specific Taxonomy Process

- _[Process](C:\Documents and Settings\t_carroll\Application Data\Qualcomm\Eudora\Attach\sas_processa.ppt)_

## Project-Specific Taxonomy Process

- _[Process](C:\Documents and Settings\t_carroll\Application Data\Qualcomm\Eudora\Attach\sas_processb.ppt)_

## Current Plans

- Obtain requirement-related fault reports for additional NASA projects
- Perform Process A (Class-specific) for classes for which we have data
- Complete list of IV&V techniques
- Continue sharing information and soliciting feedback from ST-5 project
- Prepare final report

## Future Work (Phase II)

- Build taxonomy of IV&V techniques
- Survey literature and determine what techniques have been shown to detect certain types of requirement faults
- Gather expert opinion to fill in gaps of the technique-to-fault matrix
- Design experiments to validate some of the technique-to-fault mappings
- Provide resulting information in an Advanced Risk Reduction Tool (ARRT) friendly format

## Backup

## How does this differ from ODC?

- ODC uses a fixed set of trigger and defect types
- Our emphasis is on building tailored taxonomies
- We use historical information about Classes of related projects
- ODC classification strives to be independent of the specifics of a product or organization

## How does this differ from ODC? (cont’d)

- ODC defect types don’t map well to requirements (function, I/f, checking, assignment, algorithm, etc.)
- We integrate risk analysis in our taxonomy building process
- Our long-term goals include validated cost-benefit information for fault-technique pairs