# Presentation: 018154

## Forrest Shull, 	Fraunhofer Center for Experimental Software
Patricia Larsen,	Engineering – Maryland
Rose Pajerski,
Ioana Rus,

Mike Stark,	GSFC

Allen Nikora,	JPL

- Patricia Larsen,	Engineering – Maryland
- Rose Pajerski,
- Ioana Rus,
- Mike Stark,	GSFC
- Allen Nikora,	JPL

**State-of-the-Art**
**Software Inspections at NASA**

## Project Information

- This project
- A collaboration between JPL, GSFC, CSC, and FC-MD
- Funded by the Office of Safety and Mission Assurance (OSMA) Software Assurance Research Program
- Currently in the third of three years
- Major objectives:
**Improve **software inspection practice and **support** projects in their use
- To **reduce cost** of achieving high-quality software.

## Project Plan

**Interview projects doing inspections.**
**Analyze problems being faced.**

**Introduce advanced techniques to address project issues.**
**Measure benefits in comparison to past inspections.**

**If beneficial, investigate *****where***** beneficial and measure ROI.**
**Further disseminate and measure.**

**Yr1**

**Yr2**

**Yr3**

## Project Reality

**Interview projects and create LLR.**
** ****Found that not many were doing them.**
** ****Most were keeping some of the practices, but not the whole process.**
**Selected a candidate technique (perspective-based reviews).**
**Did initial training, got feedback.**

**Yr1**

**Yr2**

**17 projects from 5 Centers**

**IV&V personnel**

**Product: **Lessons Learned Report

** **

- Planning

- Preparation

- Defect
- Report
- Form

- Meeting

- Follow-
- through

- Software
- Artifact

- PlanningForm

- Defect
- Correction
-  Form

- 1

- 2

- 3

- 4

- organizer

- inspector

- moderator
- inspectors
- author

- author

- Corrected
- Software
- Artifact

- Software
- Inspection

- Defect
- Collection
- Form

- Inspection process overview

**Notes:**
This slide shows all the ingredients that help structure and analyze existing inspection implementations:
 The activities that are performed
 The roles that people in the context of the activities 
 The products 
 The reading techniques. 
[It depends on the available time how detailed the next slides will be discussed. From here one can go directly to the part of reading or step through the various dimension. In the first case, we can say something like:
While we have included some slide material on the first three dimension for reasons of completeness, we want to focus on the reading technique dimension. This derives from the fact that this one is the add-on and of primary concern in this tutorial. We also have a paper available that explains the first three dimensions in more detail and that also provides pointers to more references.]

## Project Reality

**New problem: How do we help projects with regard to their inspection practices, and show benefit of new techniques?**
** ****Modularized, restructured, tailored training**
** ****Incorporated perspectives into training**
** ****Created feasible metrics collection plan**
**(Collected statistics from across industry)**
**(Collected data from SE univ. classes)**
**Delivered training, collected feedback.**

**Yr2**

**Yr3**

**ST5****(GSFC FSB)**

**LaRC personnel**

**NSF CeBASE**

**UMCP, ****USC**

**Products: ****	**Modularized and updated training		Defect reduction metrics collection plan

## Project Reality

**Using perspective-based approach to address project problems and re-introduce/improve inspections.**
** ****Use perspectives & project analysis to edit checklists in existing processes**
** ****Perform tailoring, provide training to projects. Compare efficiency against baselines.**
**Hand off training and support materials.**
** ****Create “what if” model for decision support.**

**Yr3**

**Swift BAT (GSFC FSB)**

**KI****(JPL)**

**JWST (GSFC FSB)**

**GSFC EPG (and others)**

**Planned ****	**Metrics & training refinements** ****products:**	“What if” tool		Final report

## Example of project perspective definition

**Keck Interferometer** (JPL) goals:
- Inspect a reusable class library to make sure that existing and future stakeholders will be satisfied with the code and documentation
- Get inspections reintroduced on the project
- Get individuals to take ownership of different parts of the process
- Achieving these through use of perspective-based reviews:
- Provides a framework for identifying stakeholders and their quality interests
- Tie inspection activities to project activities

## Stakeholder analysis identifies perspectives:

Example of guidance (Dev/Author):
From your point of view as author, consider which classes/methods were most complex or contained the most functionality... Use these questions to check whether the documentation matches the functionality implemented...
We are measuring # of defects, amount of effort, etc., to show that efficient perspectives and roles can be constructed

- Example of guidance (Dev/Author):
*From your point of view as author, consider which classes/methods were most complex or contained the most functionality... Use these questions to check whether the documentation matches the functionality implemented...*
- We are measuring # of defects, amount of effort, etc., to show that efficient perspectives and roles can be constructed

- Example of project **perspective** definition

***Handoff to next project***

**KI**

***Specification, design, and implementation***

***Test-ing***

- Reusable class library (code and documentation)

***Integration***

## Example of project defect definition

**Swift BAT**** **(GSFC FSB) in late-lifecycle code inspections and testing.
- Goals: Examine discrepancy report database to:
- Find out what defects took the longest to close so that we can add these 'high priority' items to the inspection checklists.
- Analyze which defects were caused by problems in the requirements and understand how long they stayed in the system, so that we can feed requirements recommendations forward.
- Expected result:
- Improved defect taxonomy used to focus inspections within FSB
- Improved requirements inspection process in FSB

## Example of using model for decision support

**Inspections Process Model and Simulator **goal:
- To estimate the effect of adding inspections in some/all early lifecycle phases on test effort and total rework.
- To help project managers understand the cost/benefits of different ways of planning inspections.

## Example of using model for decision support

- Example of using model for **decision support**

- Examine effects of possible V&V activity effort as you move through the development phases:

## Example of using model for decision support

- Production Activity Data
- V&V Activity Data
- Rework Data
- Test Data

- Work Product (Type, Size)
- EffortPerItem
- StaffAvailable
- Duration
- DefectsInjected (Type, Severity, Number)
- Work Product (Type, Size)
- EffortPerItem
- StaffAvailable
- Duration
- Effectiveness (Type, Severity)
- DefectsDetected (Type, Severity, Number)
- Work Product (Type, Size)
- EffortPerDefect (Type, Severity)
- StaffAvailable
- Duration
- TestCases (Number)
- EffortPerTestCase
- Duration
- Effectiveness (Type, Severity)
- DefectsRemaining (Type, Severity, Number)
- StaffAvailable

## Conclusions

**About this initiative:**
- We have other potential projects - all results will be included in final report.
- Research direction and results have been driven by project needs
**About the software development technology:**
- Work with projects is showing feasibility of the perspective-based approach.
- Tailorability is helping to address project needs in different contexts.
- Data collection (ongoing) will demonstrate whether similar levels of efficiency are being achieved compared to historical baselines.