# Presentation: 334296

## Making Use of a Decade of Widely Varying Historical Data

- Ms. Sally Godfrey (NASA POC)
- Mr. Andre Bechtel
- Mr. Raimund L. Feldmann
- Ms. Myrna Regardie
- Dr. Carolyn Seaman

**Making Use of a Decade of Widely Varying Historical Data**

**SARP project**
**“****Full Life-cycle Defect Management”**

## We are in the second year of our initiative and studying
Parameters that affect the results of inspection
The relation between V&V effectiveness in early lifecycle (e.g. inspection) and late (testing)

We are using this information to provide feedback and decision support to NASA projects, on questions such as:

Can I get guidance on how to plan my inspections based on resultsfrom projects like my own?

Based on my inspection results,what are the implications for the effortrequired to be spent on other non-optionalactivities, like system testing?

- Parameters that affect the results of inspection
- The relation between V&V effectiveness in early lifecycle (e.g. inspection) and late (testing)
- We are using this information to provide feedback and decision support to NASA projects, on questions such as:
*Can I get guidance on how to plan **my inspections based on results**from projects like my own?*
*Based on my inspection results,**what are the implications for the effort**required to be spent on other non-optional**activities, like system testing?*

**Problem we are addressing**

## Unifying different defect classifications

***Motivation*****:** Valuable defect data has been collected over the years across many Centers and projects
***Issue***: Different defect classifications used in historic and contemporary data sets, as well as across Centers;
***Action***: Definition of a unified defect classification schema and mapping of existing data sets into this unified schema
***Benefits*****:**
- Leverages data required by NPR 7150.2 for analysis and feedback to teams
- Enables monitoring and validation of existing guidelines
- Unified classification schema is applicable to inspections and testing

**existing data sets**
**(historic and contemporary)**

**unified data set for **
**contemporary project feedback **

**actions**

**A**

**B**

**Y**

**Z**

**A**

**B’**

**W**

**Y’**

**Z**

**A**

**C**

**W’**

**Y**

**Z**

- Map historical data to new categorization if possible; partition remaining historical categories and refine new schema if needed

- ODC-based new schema

**cat.**
**X’**

**cat.**
**Y**

**cat.**
**A**

**X’**

**cat.**

**Z’**

## Updating existing inspection standards

**Updating existing inspection standards**

- NASA guidelines for effective inspections formulated in early 1990’s
- Number of participants
- Page rate
- Meeting length
***Validating*** these standards basedon a wider set of more recent data
***Refining*** the standards – adding more variables, tailoring to different domains
- New standards built into
- inspection support ***tool***
- inspection ***training***
- Refined standards will increase ***effectiveness*** of inspections in terms of effort expended and defects found

## Comparing test and inspection data

***Research Questions*****:**
- What defects types are typically removed by **inspections** vs. **testing**?
- What project characteristics (size, language, software domain, new development/enhancements) influence the types of defects found?
- What percent of logic
-      errors can be expected
-      to be removed by
-      **inspections**?
- Can **test results** beused for post-mortemanalysis of**inspection** **performance**?

## Providing an inspection experience base

**http://fc-md.umd.edu/EB/**

## Improving tool support for inspections

- built-in experience base access

- reporting and evaluation capabilities

- file import capability

**Notes:**
Mention that as soon as a value is entered the traffic light reports (since there are now effort hours entered yet the traffic light is not reporting)
Note: We could do a face demo (animated ppt sequence)  – e.g., have a popup come up for entering the effort or better a single value such as the number of participants or LoC – fill in a number and have the traffic light change – need to know what you want here!

## Accomplishments so far

- 2529 unified inspection records in database
- more data sets are welcome
- Initial tool that supports inspection planning and reporting
- (test) users are welcome to try the tool
- Tool can accept data from JPL forms as well as various databases
- possibility of later integration into NASA’s eRoom
- Central inspection experience base available
- http://fc-md.umd.edu/EB
- Accepted papers at ESEM and IEEE Software
- additional publications are currently under review or planned

**More details are provided as part of ****our technical presentation**

## Contact information

**Forrest Shull (PI)**
- fshull@fc-md.umd.edu
- 301-403-8970

**Contact information**

**Sally Godfrey (NASA POC)**
- sara.godfrey@nasa.gov
- 301-286-5706

**Myrna Regardie**
- mregardie@fc-md.umd.edu
- 301-403-2050