# Presentation: 591922

## Independent Verification and ValidationCurrent Status, Challenges, and Research Opportunities

**Dan McCaugherty**
**IV&V Program Manager**
**Titan Systems Corporation**
**5 September, 2001**

## Agenda

- Where we are today
- Developer challenges observed by IV&V
- IV&V challenges
- Opportunities for R&D

## Where we are today

## IV&V Gaining Acceptance

- IV&V is gaining acceptance as a mechanism for mitigating software risk
- IV&V is naturally evolving to also include programmatic assessments such as cost/schedule performance analysis
- Recent NASA IV&V policy presents an opportunity for wider application and understanding of IV&V

## IT Project Success Factors

- Gartner Institute study identified the following essential areas as critical to IT project success**1**
- _Risk Management_
- Communications
- Resource Management
- Lessons Learned
- Additional areas based on professional experience
- Project planning
- Requirements!!!
- Projected that implementation of sound project management approach emphasizing these key areas can reduce number of failed projects by 30%**1**

**1****GartnerGroup**

## IT Project Success Rates

- Odds for Success**1&2**
- > 80% of IT projects are late, over budget, lack essential functionality, or are never delivered
- 28% of IT projects just flat fail representing more than $81 Billion in losses
- Of 15,000 data migration projects started in 1999, as many as 88 percent either overran or failed
- Leading causes for project failure**1**
- poor project planning
- failure to manage scope and change
- _failure to manage risk_

**1****GartnerGroup**
**2****The Standish Group**

## Developer Challenges Observed by IV&V

## Incremental Delivery Definition

- “Build a Little, Test a Little” approach can avoid major development failures, but creates other challenges, such as:
- Maintaining end state specifications concurrent with increment specifications
- Implementing the right requirements in the right builds to meet testable increment specific capabilities
- Understanding the cost/schedule effects of pushing requirements into later builds

## Early Definition of Interfaces and Key Architectural Concepts

- Late interface or architecture changes or problems create significant re-work or are unmanageable
- Distributed computing stresses system engineering efforts for early definition of
- API’s, remote procedures, and reused objects
- Redundancy Management,
- Fault Detection, Isolation, and Recovery
- Exception handling

## Use of New Technologies

- A great way to attract and keep highly skilled professionals
- Majority of focus on what the technology can do, not enough focus on what it can not do or was never designed to do
- Limited skill base for new technologies
- If COTS, may not have future support

## Measuring Progress or Quality

- Measurable definitions for “Done” are needed.
- Metrics efforts and collection mechanisms need to be part of the process from day one
- Optimism (a good thing) must be tempered with facts

## IV&V Challenges

## IV&V Technical Challenges

- Adapting IV&V methods to developer environments
- Understanding system dynamic behavior
- Convincing a project that the process is too risky
- Validating Non-Development-Items
- Reverse engineering incompletely specified systems

## IV&V Managerial and Business Challenges

- Demonstrating that selected IV&V tasks and scope are appropriate for the Project’s acceptable levels of risk
- Appropriate IV&V integration into project
- Demonstrating IV&V ROI
- Accounting for IV&V data needs within developer contracts.

## Opportunities for R&D

## Opportunities for R&D

- Research IV&V ROI
- Past case study**3 **shows 1.25 _<_ IV&V ROI _<_ 1.82
- Recommend a study of many projects with varying amounts and types of IV&V
- Correlate overall project success to IV&V effort types
- Continue development of code reverse engineering tools and interface V&V tools
- Software and Interface Analysis Tool (SIAT)
- Interface Management System (IMS)
- Develop Reconfigurable engineering simulation for Space Craft
- Improve early “validation” analyses

**3****Rogers**

## Opportunities for R&D

- Develop a guidebook identifying sets of IV&V tasks for certain types of systems and acceptable levels of mission risk.
- Research history of IV&V method application and results
- Develop a guidebook for developer managers on how to realize benefits of IV&V.
- Contracts and agreements
- Integration of IV&V into different types of projects
- Reporting and issue resolution

## Summary

## Summary

- There are many software challenges/risks out there
- Many can be managed through use of accepted engineering practices
- IV&V can help mitigate risk if used/applied correctly
- We need to better quantify the effects of IV&V and the ROI
- Recommended areas of R&D
- Research IV&V ROI, effects of IV&V use
- Develop tools that are needed today
- Develop guidance for future IV&V'ers and IV&V users.