# Presentation: 801467

## “ROLE  OF  IV&V”

**Distinguished Speakers:**Mr. Joe Bodmer (OCSE)Mr. Tom Brewer (SC)Mr. Brendon DeLong (Bearing Point / CA)

## INDEPENDENT VERIFICATIONANDVALIDATION

## OBJECTIVES

- To Summarize IV&V Objectives and Benefits
- To Understand the Federal Requirements for IV&V
- To Provide the Federal Perspective on IV&V
- To Provide IV&V Status/Statistics

## Verification/Validation - Take 1(IEEE STD 1012-1998)

**Validation:  **Confirmation by examination and provisions of** **objective evidence that the particular requirements for a specific intended use are fulfilled
**Verification:**  Confirmation by examination and provisions of** **objective evidence that specified requirements have been fulfilled

## Verification/Validation - Take 2(Simplified)

**Validation:**  Ensuring the right product is built
**Verification:**  Ensuring the product is built right

## Potential Barriers to IV&V

- Lack of resources
- Schedule constraints
- Lack of access to project
- Evolving products
- Reporting – “too much” vs. “too little”

## Benefits of IV&V

- Identifies high-risk areas early
- Provides State and Federal governments an objective analysis to deal with system development issues
- Provides management with improved visibility into the progress and quality of the development effort
- Reduces errors in delivered products

## Federal IV&V Requirements

**Independent Verification and Validation** (IV&V) provides an independent appraisal of a system’s development project
**Independent** - The IV&V Provider and associated contract must reside outside the State’s Title IV-D Agency and it’s Umbrella Agency

## Federal Variations to“Traditional” IV&V

- Identifies areas of deficiencies and risk and recommends solutions, but does not actively participate in remedial activities
- Focus is on both management and work products
- Reports are submitted to OCSE at the same time they are provided to State management

## Federal IV&V Regulations

- Federal Regulation **45 CFR 307.15(b)(10)** requires an entity independent of the State Title IV-D Agency and of the CSES Project Management Structure to review all technical and managerial aspects of the project

## IV&V “Triggers”

- Lack of a certified FSA-88 system
- Failure to timely submit APD’s
- Submission of APD for system redesign
- Failure to meet a critical milestone
- Development under a waiver pursuant to 452(d)(3) of Social Security Act
- OCSE determination of risk of project failure, serious delay or cost overrun

## FEDERAL PERSPECTIVEONIV&V

## IV&V vs. Quality Assurance

- Under PRWORA, OCSE requires States to provide evidence of adequate QA
- QA is an on-going process, while IV&V typically provides a “snapshot” of the project at a point in time
- QA Provider -- Works directly under the direction of the State’s project staff
- IV&V Provider -- Is not directly associated with or managed by the project staff

## IV&V Process

- For States meeting one of the IV&V triggers, OCSE conducts a “Scope of IV&V Assessment Review”
- OCSE selects the appropriate activities to investigate in a particular project
- Output of the review is a report identifying specific IV&V requirements for the State
- Report is written to aid the State in developing the Statement of Work for their IV&V RFP

## Contractor vs. State Agency

- Criteria for an IV&V Service Provider are:
- Independence from Title IV- D Agency and it’s Umbrella Agency
- Capability and capacity to handle management, organization and performance assessment and analysis, *and possibly*,
- Capability to accommodate highly technical analyses in areas such as: Capacity Analysis, Software Metrics and Performance, etc.

## THE  IV&V  PROCESS

## IV&V Process - Part 1OCSE

- For States meeting one of the IV&V triggers, OCSE conducts a “Scope of IV&V Assessment Review”
- OCSE selects the appropriate activities to investigate in a particular project
- Output of the review is a report identifying specific IV&V requirements for the State
- Report is written to aid the State in developing the Statement of Work for their IV&V RFP

## IV&V Assessment ActivitiesProject Management

**Project Initiation****	**
**Business Process Re-engineering****	**
**Project Planning and Reporting****	**
**Project Estimating and Scheduling****	**
**Project Personnel****	**
**Project Organization**
**Subcontractors and External Staff**
- Subcontractor commitment
- State oversight
- Acceptance and turnover

## Training and Documentation	
User Training and Documentation	
Developer Training and Documentation	
Process Definition and Standards	
Quality Assurance	
Risk Management
Configuration Management	
Requirements Management	
System Security	
System Capacity

- User Training and Documentation
- Developer Training and Documentation
**Process Definition and Standards****	**
**Quality Assurance****	**
**Risk Management**
**Configuration Management****	**
**Requirements Management****	**
**System Security****	**
**System Capacity**

**IV&V Assessment Activities*****Project-Wide Processes***

## IV&V Assessment ActivitiesEnvironments, Processes and Products

**Systems Engineering**
- Requirements Analysis
- Interface Requirements
- Requirements Allocation and Specification
- Reverse Engineering
**Operating Environment**
- System Hardware
- System Software
**Data Management**
- Database Software
- Data Conversion
- Database Design

## Development Environment
Hardware	
Software	

Software Architecture
High-Level Design
Detailed Design	
Job Control	

- Hardware
- Software
**Software Architecture**
- High-Level Design
- Detailed Design
- Job Control

**IV&V Assessment Activities*****Environments, Processes and Products***

**(Continued)**

**Code and Test**
- Code
- Unit Test
- Integration Test
- System/Acceptance Test
- Pilot Test

## IV&V Process - Part 2The State

- The RFP and IV&V Contract must be submitted to OCSE for Prior Approval
- If another State agency is used as the IV&V Provider, a copy of the Interagency Agreement is provided to OCSE for Prior Approval
- Federal Funding for IV&V Services is available at the 66% FFP Rate

## IV&V Process - Part 3The IV&V Provider

- Works with the State to develop an IV&V Management Plan that identifies the scope, depth, schedules, and resource requirements of the IV&V effort
- All Work Plans and Reports generated must be submitted to OCSE at the same time they are submitted to the State

## IV&V Provider Requirements

- Develop an IV&V Management Plan
- Review and make recommendations on Project Organization, Management, Resources, and Methodologies
- Review and make recommendations on the technical aspects of the CSES
- Assess Stakeholder and User Involvement and Buy-in

## IV&V Provider Requirements

- Analyze past project performance to develop “lessons learned”
- Provide Risk Management Assessment
- Provide Capacity Planning, as required
- Develop performance measures to track project completion
- Address issues from OCSE’s “Scope of IV&V Assessment” Review

**(Continued)**

## IV&V Statistics (1998 – 2004)

- 19   Projects initially required IV&V
- 22   OCSE IV&V Assessment Reviews performed
- 		to-date
- 17   Due to missed certification deadline or major project
- 		    milestone
-   5   Due to a new or replacement system
- 4   Projects currently requiring IV&V (CA, FL, SC, TX)
- Number of projects requiring IV&V (at year end + 2004)
- 1998 (2), 1999 (14), 2000 (13), 2001 (12), 2002 (11),
- 	2003 (11), 2004 (4)

## California IV&V

**February 2000 –** OCSE IV&V Assessment Review.  Required CA to acquire Full Time IV&V Support
**January 2001 –** Bearing Point, Inc. IV&V Team began work as the State’s IV&V Provider
**May 2003 –** OCSE conducted a “Feasibility Study”
- California IV&V work products used as examples for South Carolina IV&V effort

## South Carolina IV&V

**August 1999 – **First OCSE IV&V Assessment Review conducted. Specified periodic IV&V support (Every 6 Months)
**October 2000 – **OCSE conducted a “Feasibility Study and Scope of IV&V Assessment Review”. Resulted in the requirement for the State to acquire a Full Time IV&V Provider
**May 2001 – **USC IV&V Team began work as the State’s IV&V Provider
- State leveraged off California IV&V effort and has incorporated many of CA IV&V work products for SC (e.g., Quarterly IV&V Report, Interim Status Update Report, etc.)

## QUESTIONS  AND  ANSWERS

## “THANKS TO OUR SPEAKERS”

- Mr. Joe Bodmer (OCSE)Mr. Tom Brewer (SC)Mr. Brendon DeLong (Bearing Point / CA)