Electronic records Archives

independent Verification and Validation Plan

(IVVP v1.1)

(WBS \# 1.4.1.1.4)

for the

national archives and

records administration

electronic records archives

program management office

(nara era pmo)

Final

January 7, 2004

Prepared by:\
\
*Integrated Computer Engineering (ICE)*

*A Subsidiary of*

*American Systems Corporation (ASC)*

Contract Number: GS-35F-0673K

Delivery Order Number: NAMA-01-F-0031/03-097

Independent Verification And Validation Plan (IVVP)

[]{#__RefHeading___Toc61249556 .anchor}Signature Page

Program Director,

I recommend approval of the Independent Verification and Validation Plan
(IVVP).

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_

Carmen Colon, Date

ERA Program

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_

Kenneth Thibodeau, Date

ERA Program Director

[]{#__RefHeading___Toc61249557 .anchor}Document Change Control Sheet

**Document Title:** Independent Verification and Validation Plan (IVVP)

  ---------- ---------------------- ------------- ----------------------------
  **Date**   Filename/version \#    **Author**    **Revision Description**

  09/16/03   ERA.DC.IVVP.0.01.doc   Eva           Draft
                                    Freund/Lou    
                                    Pinto         

  12/05/03   ERA.DC.IVVP.1.0.doc    Lou Pinto     Revised for Government
                                                  comments document dated 6
                                                  October, 2003

  01/07/04   ERA.DC.IVVP.1.1.doc    Lou Pinto     Revised for Government
                                                  comments document dated 6
                                                  January, 2004

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  
  ---------- ---------------------- ------------- ----------------------------

> **TABLE OF CONTENTS**

[Signature Page i](#__RefHeading___Toc61249556)

[Document Change Control Sheet ii](#__RefHeading___Toc61249557)

[1.0 PURPOSE 1](#purpose)

[1.1 Introduction 1](#introduction)

[1.2 Goals 2](#goals)

[1.3 Scope 2](#scope)

[1.4 Project Overview 2](#project-overview)

[1.5 Specific Processes and Procedures Covered
2](#specific-processes-and-procedures-covered)

[1.6 Organization of this document 2](#organization-of-this-document)

[1.7 Assumptions 3](#assumptions)

[2.0 REFERENCED DOCUMENTS 3](#referenced-documents)

[2.1 Government documents 3](#government-documents)

[2.1.1 Contract Documents 3](#contract-documents)

[2.1.2 Federal Laws and Regulations 3](#federal-laws-and-regulations)

[2.1.3 NARA Policies 4](#nara-policies)

[2.2 Non Government documents 4](#non-government-documents)

[3.0 DEFINITIONS AND ACRONYMS 5](#definitions-and-acronyms)

[3.1 Definitions 5](#definitions)

[3.2 Acronyms 7](#acronyms)

[4.0 IV&V OVERVIEW 8](#ivv-overview)

[4.1 Organization 8](#organization)

[4.2 Master Schedule 10](#master-schedule)

[4.3 System Integrity Level Scheme 11](#system-integrity-level-scheme)

[4.4 Resource Summary 11](#resource-summary)

[4.4.1 Staffing 11](#staffing)

[4.4.2 Facilities 12](#facilities)

[4.4.3 Tools 12](#tools)

[4.4.4 Finances 12](#finances)

[4.4.5 Special Procedural Requirements (security, access rights,
documentation control)
12](#special-procedural-requirements-security-access-rights-documentation-control)

[4.5 Responsibilities 12](#__RefHeading___Toc61249593)

[4.6 Tools, Techniques, and Methods 14](#tools-techniques-and-methods)

[4.6.1 Tools 14](#tools-1)

[4.6.2 Techniques 14](#techniques)

[4.6.3 Methods 14](#methods)

[5.0 IV&V Life Cycle Verification and Validation
15](#ivv-life-cycle-verification-and-validation)

[5.1 Management Process 16](#management-process)

[5.1.1 Management Activity 16](#management-activity)

[5.2 Acquisition Process 21](#acquisition-process)

[5.2.1 Acquisition Support Activity 21](#acquisition-support-activity)

[5.3 Supply Process 31](#supply-process)

[5.3.1 Planning IV&V 31](#planning-ivv)

[5.4 Development Process 33](#development-process)

[5.4.1 Concept Activity 33](#concept-activity)

[5.4.2 Requirements Activity 38](#requirements-activity)

[5.4.3 Design Activity 45](#design-activity)

[5.4.4 Implementation Activity 52](#implementation-activity)

[5.4.5 Test Activity 58](#test-activity)

[5.4.6 Installation and Checkout IV&V Activity
65](#installation-and-checkout-ivv-activity)

[5.5 Operation Process 69](#operation-process)

[5.5.1 Operation IV&V Activity 69](#operation-ivv-activity)

[5.6 Maintenance Process 71](#maintenance-process)

[5.6.1 Maintenance IV&V Activity 72](#maintenance-ivv-activity)

[6.0 IV&V Reporting Requirements 76](#ivv-reporting-requirements)

[6.1 Required Reports 76](#required-reports)

[6.1.1 Progress and Status Reports 76](#progress-and-status-reports)

[6.1.2 Task Reports 77](#task-reports)

[6.1.3 End of Activity Summary Report
77](#end-of-activity-summary-report)

[6.1.4 IV&V Final Report 77](#ivv-final-report)

[6.2 Optional Reports 77](#optional-reports)

[6.2.1 Engineering White Papers 78](#engineering-white-papers)

[6.2.2 Meeting Summary Reports 78](#meeting-summary-reports)

[6.2.3 Red Flag Reports 78](#red-flag-reports)

[6.3 Report Management Procedures 78](#report-management-procedures)

[7.0 ADMINISTRATIVE REQUIREMENTS 78](#administrative-requirements)

[7.1 Anomaly Resolution And Reporting
79](#anomaly-resolution-and-reporting)

[7.1.1 Methodology 79](#methodology)

[7.1.2 Distribution 79](#distribution)

[7.1.3 Authority (this section needs to include the timeline for
resolution)
79](#authority-this-section-needs-to-include-the-timeline-for-resolution)

[7.2 Task Iteration Policy 79](#task-iteration-policy)

[7.3 Deviation Policy 80](#deviation-policy)

[7.4 Control Procedures 80](#control-procedures)

[7.5 Standards, Practices, And Conventions
80](#standards-practices-and-conventions)

[8.0 IV&V DOCUMENTATION REQUIREMENTS
80](#ivv-documentation-requirements)

[8.1 Test Plan 80](#test-plan)

[8.2 Test Design 81](#test-design)

[8.3 Test Cases 81](#test-cases)

[8.4 Test Procedures 81](#test-procedures)

[8.5 Test Results 81](#test-results)

[8.6 Test Log 81](#test-log)

[Appendix A: IV&V Work Breakdown Structure (WBS) and Schedule
A-1](#__RefHeading___Toc61249675)

[Appendix B: IV&V Templates B-1](#__RefHeading___Toc61249676)

[IV&V TASK REPORT B-1](#__RefHeading___Toc61249677)

[IV&V QUICK LOOK REPORT B-2](#__RefHeading___Toc61249678)

[IV&V SPECIAL REPORT B-3](#__RefHeading___Toc61249679)

[IV&V MEETING ATTENDANCE MEMORANDUM B-9](#__RefHeading___Toc61249680)

[IV&V WEEKLY PROGRESS REPORT B-10](#__RefHeading___Toc61249681)

[IV&V MONTHLY STATUS REPORT B-11](#__RefHeading___Toc61249683)

[Appendix C: Document Assessment Checklist (sample)
C-1](#__RefHeading___Toc61249684)

List of Figures

[Figure 4-1: Organization Hierarchy 9](#__RefHeading___Toc61244749)

[Figure 4-2: Organization Communication 10](#__RefHeading___Toc61244750)

List of Tables

[Table 2-1: Applicable IEEE Standards 5](#__RefHeading___Toc61244751)

[Table 3-1: Definitions List 6](#__RefHeading___Toc61244752)

[Table 3-2: Acronyms List 8](#__RefHeading___Toc61244753)

[Table 4-1: Staffing Details to Support IV&V Activities
12](#__RefHeading___Toc61244754)

[Table 4-2: Primary Participant responsibilities
13](#__RefHeading___Toc61244755)

# PURPOSE

This section describes the purpose and scope of the Electronic Records
Archives (ERA) *Independent Verification and Validation Plan* (IVVP)
prepared for the National Archives and Records Administration (NARA).
IEEE Std. 1012-1998, *IEEE Standard for Software Verification and
Validation* was utilized to provide guidance for production of this
plan. The IVVP describes the purpose, goals, and scope of the
Independent Verification and Validation (IV&V) effort.

## Introduction

The NARA ERA Program encompasses the policies, procedures, practices,
and the necessary technology that will enable NARA to build ERA. The
system will receive, preserve, and provide access to electronic records
maintained by NARA. Details of the ERA system are found in **Section
1.4, System Description**.

IV&V provides input on deliverables, procedures, standards, resources,
effort, and duration to support the planning and deployment of ERA. The
Verification process provides objective evidence that all life cycle
processes have been properly and adequately performed. The Validation
process examines and provides objective evidence of product compliance
with the system's functional requirements and the users' needs.

The content of IV&V tasking is determined by the integrity level,
described in **Section 4.3, System Integrity Level,** which itself was
the result of the first ERA System Criticality Analysis. The resulting
task layout is identified in this plan. The IV&V process provides
independent assessments of the ERA system documents, plans, processes,
and configuration components. Additionally, IV&V provides independent
assessment of the test methodology applied at the integration, system,
and acceptance test levels as well as providing an assessment of the
deployment plan and process.

The focus of the IVVP is a description of the mechanisms, requirements,
and management activities that will allow successful execution of IV&V.
These activities will assist in ensuring procurement, integration,
implementation, and deployment of a high quality ERA system. IV&V
provides a disciplined approach to assessing ERA throughout the
acquisition life cycle. Detailed descriptions of the tasks performed by
IV&V are contained in **Section 5.0, IV&V Life Cycle Verification and
Validation**.

This plan is based on the *IEEE Standard for Software Verification and
Validation,* IEEE Std. 1012 - 1998. IEEE Std. 1012-1998 has been
tailored to describe system-level activities rather than the software
activities and tasks focused on by the standard. The life cycle
acquisition processes used in this plan conform to those described in
IEEE/EIA 12207.0-1996, *IEEE/EIA Guide -- Industry Implementation of
ISO/IEC 12207:1995, IEEE/EIS Standard for Information Technology --
Software life cycle processes*, and in IEEE Std. 1012-1998. They are
also in accordance with the ERA development life cycle as described in
the *ERA Life Cycle (ELC)* document and the *ERA Program Management Plan
(PMP)*.

## Goals

The IV&V goal is to ensure that the deployed system is capable of
performing its intended functions and is testable, reliable,
maintainable, usable, and easily enhanced. This IV&V program is designed
to provide NARA with increased visibility into the acquisition process
and can be used as a supplementary tool to help NARA reduce overall
project risk. IV&V provides an independent assessment of ERA throughout
its life cycle, continuously verifying and validating that the ERA
system and documentation conforms to contract requirements, and meets
user needs.

## Scope

This plan describes the activities, methods, criteria, organizational
structure, management, schedule, and resources planned for conducting
IV&V in the ERA Program. This plan applies to the design, integration,
test, acceptance, and deployment of all acquired configuration
components of ERA.

## Project Overview

ERA will provide a comprehensive, systematic, and dynamic means for
preserving virtually any kind of electronic record, free from dependence
on any specific hardware or software. ERA, when operational, will make
it easy for NARA customers to find records they want and easy for NARA
to deliver those records in formats suited to customers' needs.

## Specific Processes and Procedures Covered

This IVVP describes the tasks that will be undertaken to verify and
validate the acquisition process (**Section 5.2**), supply process
(**Section 5.3**), development process (**Section 5.4**), operation
process (**Section 5.5**), and maintenance process (**Section 5.6**).
This IVVP also includes tasking required for the IV&V management process
(**Section 5.1**).

The particular set of tasks chosen for ERA and described in this plan
accommodates a modified integrity level three undertaking. At the time
of production of this IVVP, verification of developer component test
documentation and results, production of IV&V documentation for
component and integration testing, and IV&V performance of component and
integration testing, were considered outside of the scope for the ERA
effort.

## Organization of this document

This document is organized in a manner compliant with the content and
format specified for a Software Verification and Validation Plan, as
specified in Section 7 of IEEE Std. 1012-1998. **Section 1.0** of this
document is concerned with describing the purpose, goals, and scope of
the IV&V effort for ERA. **Section 2.0** lists the references used to
generate this document. **Section 3.0** provides a definition of terms.
**Section 4.0** provides an overview of the IV&V infrastructure.
**Section 5.0** describes each primary life cycle process and identifies
associated IV&V activities and tasks. Any changes to the list of
deliverables or IV&V tasking will be reflected in **Section 5.0** of
this document. **Sections 6.0**, **7.0**, and **8.0** describe,
respectively, the IV&V reporting, administrative, and documentation
requirements for ERA.

## Assumptions

To achieve the objectives defined in this plan, it is assumed that:

-   The IV&V team will have:

```{=html}
<!-- -->
```
-   Continuous access to ERA system plans, specifications, Configuration
    Management (CM) data, test data, tools, Commercial Off-The-Shelf
    (COTS) products, ERA Contract Data Requirements List (CDRL)
    products, test results, and identified issues (e.g. meeting minutes,
    contract exceptions, action items, etc.); and

-   Preliminary draft documents from the ERA contractor(s) available in
    sufficient time to allow feedback prior to formal delivery to the
    ERA Contract Office Representative (COR).

```{=html}
<!-- -->
```
-   Documents (such as the System Requirements Definition, Concept of
    Operations, Detailed Design Document, and Security Plan) are
    complete, consistent, and current; and

```{=html}
<!-- -->
```
-   The ERA Program Management Office (PMO) is empowered to maintain
    configuration item change control, and change control for
    engineering and user documentation.

# REFERENCED DOCUMENTS

The following documents, of the exact issue shown, form a part of this
plan to the extent specified herein.

## Government documents

The following sections list contract documents, laws, regulations, and
guidelines that were either used in the development of this plan or are
referenced in this plan.

### Contract Documents

The contract specific documents are listed below.

-   TBD

-   Other legislative and executive directives as they may apply to the
    management and deployment of this system.

### Federal Laws and Regulations

The Federal laws and regulations are listed below.

-   Office of Management and Budget (OMB) Circular No. A-11,
    *Preparation, Submission and Execution of the Budget*, (Revised
    07/25/2003)

-   Office of Management and Budget (OMB) Circular No. A-119,
    *Transmittal Memorandum, Federal Participation in the Development
    and Use of Voluntary Standards* (02/10/1998)

-   Federal Acquisition Regulation, issued September 2001 by GSA, DoD,
    NASA

-   General Accounting Office Information Technology: *An Audit Guide
    for Assessing Acquisition Risks*, December 1992 (Chapter 5)

-   Defense Finance and Accounting Service, *Analysis of Alternatives
    Report*, DFAS 8000.1-R, Part C, Chapter 1

-   E-Government Act of 2002, Public Law 107-347

-   Federal Information Security Management Act of 2002, U.S.C. Vol. 44,
    Sections 3541-9

### NARA Policies

TBD

## Non Government documents

The following documents form a part of this plan to the extent as
specified herein. Unless otherwise indicated, the contract version of
the document shall apply.

  -----------------------------------------------------------------------
  **Standard            **Standard Title**
  Identifier**          
  --------------------- -------------------------------------------------
  IEEE Std. 610.12      *IEEE Standard Glossary of Software Engineering
  -1990                 Terminology*

  IEEE Std. 730-1998    *IEEE Standard for Software Quality Assurance
                        Plans*

  IEEE Std. 828-1998    *IEEE Standard for Software Configuration
                        Management Plans*

  IEEE Std. 829-1998    *IEEE Standard for Software Test Documentation*

  IEEE Std. 830-1998    *IEEE Recommended Practice for Software
                        Requirements Specifications*

  IEEE Std. 1012-1998   *IEEE Standard for Software Verification and
                        Validation*

  IEEE Std. 1016-1998   *IEEE Recommended Practice for Software Design
                        Descriptions*

  IEEE Std. 1028 -1997  *IEEE Standard for Software Reviews*

  IEEE Std. 1058-1998   *IEEE Standard for Software Project Management
                        Plans*

  IEEE Std. 1061-1998   *IEEE Standard for a Software Quality Metrics
                        Methodology*

  IEEE Std. 1062-1998   *IEEE Recommended Practice for Software
                        Acquisition*

  IEEE Std. 1063-1987   *IEEE Standard for Software User Documentation*

  IEEE Std. 1220-1998   *IEEE Standard for Application and Management of
                        the Systems Engineering Process*

  IEEE Std. 1233 -1998  *IEEE Guide for Developing System Requirements
                        Specifications*

  IEEE Std. 1362-1998   *IEEE Guide for Information Technology-System
                        Definition - Concept of Operations - Document*

  IEEE Std. 2001-2002   *IEEE Recommended Practice for the Internet --
                        Web Site Engineering, Web Site Management, and
                        Web Site Life Cycle*

  IEEE/EIA 12207.0      *IEEE/EIA Standard - Software life cycle
  -1998                 processes*

  IEEE/EIA 12207.1      *IEEE/EIA Standard - Software life cycle
  -1997                 processes--Life cycle data*

  IEEE/EIA 12207.2      *IEEE/EIA Standard - Software life cycle
  -1997                 processes-- Implementation considerations*

  J-STD-016-1995        *IEEE/EIA Standard - Software life cycle
                        processes-- Software Development
                        Acquirer-Supplier Agreement*

  Department of Defense *Mandatory Procedures for Major Defense
  5000.2-R, April 5,    Acquisition Programs (MDAPs) and Major Automated
  2002                  Information System (MAIS) Acquisition Programs*

  SEI-CMM               Technical Report CMU/SEI-93-TR-024 *Capability
                        Maturity Model for Software*, version 1.1
  -----------------------------------------------------------------------

[]{#__RefHeading___Toc61244751 .anchor}Table 2-1: Applicable IEEE
Standards

# DEFINITIONS AND ACRONYMS

**Section 3.1** provides a glossary of commonly used terms and their
definitions to aid in the understanding of this document. **Section
3.2** provides a list of acronyms and respective definitions that are
used throughout.

## Definitions

The following technical terms, when used in this document, are defined
as stated below. Definitions obtained from a standardized source are so
annotated.

  -----------------------------------------------------------------------
  **Term**           **Definition**
  ------------------ ----------------------------------------------------
  Acceptance testing Formal testing conducted to determine whether a
                     system satisfies its acceptance criteria and enables
                     the customer to determine whether or not to accept
                     the system \[IEEE Std. 610.12-1990, IEEE Standard
                     Glossary of Software Engineering Terminology\].

  Anomaly            Anything observed in the documentation or operation
                     of a system that deviates from expectations based on
                     previously verified system products or reference
                     documents. The criticality of an anomaly is assessed
                     in accordance with the standards described in the
                     template for the NARA IV&V Anomaly Report, found in
                     **Appendix B** of this document. Three areas of
                     classification are considered. They are: impact on
                     the success of the system (red, yellow, green),
                     urgency of the corrective action at the project
                     level, and urgency of corrective action in terms of
                     the impact on the enterprise.

  Audit              An audit is defined, in the context of an
                     acquisition effort, as a detailed accounting of a
                     delivered product in the context of its defining
                     baseline.

  Concept phase      The initial set of activities and products of a
                     \[development\] project, in which user needs are
                     described and evaluated through documentation.

  Functional         In configuration management, the initial approved
  baseline           technical documentation for a configuration item
                     \[IEEE Std. 610.12-1990\]. The Functional Baseline
                     is usually defined by the System Requirements
                     Specification (SyRS).

  Hardware           Physical equipment used to process, store, or
                     transmit computer programs or data \[IEEE Std.
                     610.12-1990\].

  Increment          A useful and supportable operational capability that
                     can be effectively developed, produced, or acquired,
                     deployed or sustained. Each increment of capability
                     will have its own set of thresholds and objectives
                     set by the user.

  Independent        Verification and validation tasks performed by an
  verification and   organization that is technically, managerially, and
  validation         financially independent of the development
                     organization.

  Installation and   The period of time in the system life cycle during
  Checkout           which system components are integrated into the
                     operational environment and tested in this
                     environment to ensure that they perform as required.

  Integration        Testing in which system components are combined and
  testing            tested to evaluate the interaction between them.

  Interface testing  Testing conducted to evaluate whether systems or
                     components pass data and control correctly to one
                     another \[IEEE Std. 610.12-1990\].

  Operation and      The period of time in the system life cycle during
  maintenance phase  which the system is employed in the operational
                     environment, monitored for satisfactory performance,
                     and modified as necessary to correct problems or to
                     respond to changing requirements.

  Release            In the context of the ERA program, each release
                     presents a subset of the functionality prescribed
                     for its parent increment.

  System life cycle  The period of time that begins when a system is
                     conceived and ends when the system is no longer
                     available for use \[IEEE Std. 610.12-1990\].

  Technical Review   A Technical Review is a review conducted to assess
                     the technical progress of an effort as compared to
                     the requirements of an associated major milestone
                     for the purpose of determining readiness to progress
                     to the next milestone or event.

  Test Phase         The period of time in the system life cycle during
                     which its components are integrated, and the product
                     is evaluated to determine whether or not
                     requirements have been satisfied.

  Test readiness     A review conducted to evaluate preliminary test
  review             results for one or more configuration items; to
                     verify that the test procedures for each
                     configuration item are complete, comply with test
                     plans and descriptions, and satisfy test
                     requirements; and to verify that a project is
                     prepared to proceed to formal testing of the
                     configuration item \[IEEE Std. 610.12-1990\].

  Validation         The process of evaluating a system or component
                     during or at the end of the development process to
                     determine whether or not it satisfies specified
                     requirements \[IEEE Std. 610.12-1990\].

  Verification       The process of evaluating a system or component to
                     determine whether or not the products of a given
                     development phase satisfy the conditions imposed at
                     the start of the phase \[IEEE Std. 610.12-1990\].
  -----------------------------------------------------------------------

[]{#__RefHeading___Toc61244752 .anchor}Table 3-1: Definitions List

## Acronyms

The following abbreviations appear in this document.

  -----------------------------------------------------------------------
  **Acronym**        **Definition**
  ------------------ ----------------------------------------------------
  CCB                Configuration Control Board

  CDR                Critical Design Review

  CDRL               Contract Data Requirements List

  CM                 Configuration Management

  ConOps             Concept of Operations

  COR                Contracts Office Representative

  COTS               Commercial Off-The-Shelf

  DoD                Department of Defense

  ELC                ERA Life Cycle

  ERA                Electronic Records Archives

  FCA                Functional Configuration Audit

  FOC                Full Operational Capability

  IDD                Interface Design Description

  IEEE               Institute of Electrical and Electronics Engineers,
                     Inc.

  IOC                Initial Operational Capability

  IRD                Interface Requirements Document

  IRS                Interface Requirements Specification

  IV&V               Independent Verification and Validation

  IVVP               Independent Verification and Validation Plan

  NARA               National Archives and Records Administration

  OMB                Office of Management and Budget

  PCA                Physical Configuration Audit

  PD                 Program Director

  PMO                Program Management Office

  PMP                Program Management Plan

  POST               Program Office Support Team

  QA                 Quality Assurance

  RD                 Requirements Document

  RFP                Request for Proposal

  RM                 Risk Management

  SARAD              System Architecture and Requirements Allocation
                     Description

  SE                 Senior Systems Engineer

  SEI-CMM            Software Engineering Institute -- Capability
                     Maturity Model

  SME                Subject Matter Expert

  SOO                Statement of Objectives

  SRS                Software Requirements Specification

  SyRS               System Requirements Specification

  TBD                To Be Determined

  TRR                Test Readiness Review

  WBS                Work Breakdown Structure
  -----------------------------------------------------------------------

[]{#__RefHeading___Toc61244753 .anchor}Table 3-2: Acronyms List

# IV&V OVERVIEW

This section describes the organization, schedule, resources,
responsibilities, tools, techniques, and methodologies necessary to
perform the independent verification and validation tasks.

## Organization

The NARA ERA Program Director (PD) has responsibility for the ERA
program. That responsibility includes establishing policies, procedures
and practices, and providing the necessary technology to enable NARA to
build the ERA system to receive, preserve, and provide access to
electronic records. The ERA PMO provides operational support for
acquiring, operating, and maintaining the ERA system.

Through coordination with the NARA ERA PD and the ERA PMO, IV&V receives
products delivered by the ERA contractor(s) and sub-contractors. These
products undergo an assessment of their conformance to the appropriate
standards specifying their format, purpose and content and of the
completeness and accuracy of their technical content. Comments (usually
in the form of an IV&V Task Report) together with a delivery letter are
then provided to the ERA COR who submits them to the ERA PD and/or its
designated representative for review and acceptance or rejection. The
ERA COR will inform the IV&V contractor of the acceptance or rejection
of the product delivery. If the delivery is rejected, it then becomes
subject to comment adjudication. IV&V supports the ERA PMO during
technical interchange meetings, program management reviews, milestone
reviews, test witnessing, and other activities as authorized.

As shown in **Figure 4-1, Organization Hierarchy**, and **Figure 4-2,
Organization Communications Flow**, IV&V has an ongoing relationship
with the ERA PMO and its members. IV&V has a tangential relationship
with the ERA contractor(s) and their representatives.

[]{#__RefHeading___Toc61244749 .anchor}Figure 4-1: Organization
Hierarchy

[]{#__RefHeading___Toc61244750 .anchor}Figure 4-2: Organization
Communication

## Master Schedule

The ERA Master Schedule is maintained by the ERA PMO and is published as
a part of the *ERA PMP*. The ERA Master Schedule identifies program life
cycle milestones including formal reviews and audits. As a living
document, updated as program events are affected, the ERA Master
Schedule is developed and maintained through an iterative process
throughout the development life cycle.

IV&V tasks providing feedback on the life cycle processes that support
ERA management functions (e.g., comments on design review material) are
scheduled prior to each milestone review. The timely "turn-around" of
information provides ERA management with an independent product
assessment prior to the milestone review. The IV&V schedule, addressing
individual IV&V tasks, is included in the ERA Program Master Schedule.

## System Integrity Level Scheme

The granularity of tasking undertaken by IV&V is, in large measure,
determined by the integrity level of the system being developed. ERA has
initially been assessed at a modified level 3 integrity (for a
discussion on software integrity levels, consult IEEE Std. 1012-1998,
Section 4.1). At this level, a function or system capability that fails
may have catastrophic consequences on the NARA mission, however there is
an infrequent likelihood of such a failure. For integrity level 3, IV&V
is required to generate test documentation and execute IV&V tests
(component, integration, system, and acceptance). Typically, for level 3
integrity, IV&V would not review the ERA contractor(s) test
documentation but would prepare its own test documentation and execute
its own tests.

However, for the ERA modified level 3 integrity, IV&V will evaluate the
ERA contractor(s) test documentation, and witness the integration and
system testing. IV&V will evaluate the ERA acceptance test documentation
and witness the ERA acceptance tests. IV&V will execute a randomly
selected number of system and acceptance test cases. IV&V will then,
through statistical means, validate the results of the formal tests
originally conducted. Further information regarding IV&V testing is
contained in **Section 8.0, IV&V Documentation Requirements**. IV&V, at
a minimum, will sample the tests at the completion of each release
within each increment in the ERA development life cycle, and at Initial
Operational Capability (IOC) and Full Operational Capability (FOC).

## Resource Summary

The following subparagraphs identify the resources needed to perform all
of the planned IV&V activities and tasks including staffing, facilities,
tools and special procedural requirements (e.g., licenses, access rights
and points of contact). An overview of the resources needed for the IV&V
effort is contained in **Sections 4.4.1**, **Staffing, 4.5,
Responsibilities**, and **4.6, Tools, Techniques and Methods**, and will
not be repeated for individual tasks in **Section 5.1** through **5.6**
of this plan.

### Staffing

**Table 4-1, Staffing Details to Support IV&V Activities**, provides the
staffing numbers for IV&V activities in the ERA Program.

  -------------- ---------------------- --------------------- ------------
  **Number of    **Percentage of Time   **Title**             **Skill
  Staff**        Assigned**                                   Level**

  1              100%                   IV&V Technical        Manager
                                        Director              

  6              100%                   IV&V SME              Expert

  2              75 %                   Test SME              Expert

  TBD            As required            Consultant            TBD
  -------------- ---------------------- --------------------- ------------

[]{#__RefHeading___Toc61244754 .anchor}Table 4-1: Staffing Details to
Support IV&V Activities

### Facilities

Offices and computing facilities in Archives II, in College Park, MD and
offices co-located in or accessible to the ERA contractor(s) facilities.

### Tools

IV&V will develop tools and templates as required to facilitate the
assessment of the products delivered for evaluation. IV&V anticipates
that these tools may all be developed and managed through use of the
"off-the-shelf" capabilities of Microsoft Office XP Professional. For
the evaluation of Web based products, IV&V may require the use of other
automated tools specifically designed for the purpose of evaluating the
content of Web based products. That need is TBD.

### Finances

This section is not applicable to this plan.

### Special Procedural Requirements (security, access rights, documentation control)

The Government will provide the IV&V contractor with "non-escort
required" clearances to gain access to NARA in the College Park
facility. Personnel assigned to areas requiring special briefings for
need-to-know material will be provided those briefings in a timely
manner. All IV&V contractor staff members will require access to the
NARA ERA PD and other interfacing customer and contractor personnel.
IV&V staff members require the accesses and authority necessary to
attend program reviews and technical interchange meetings in order to
effectively execute assigned tasks.

The ERA PMO PD will ensure that distribution procedures are implemented
to provide documentation, information and policy memoranda, meeting
notices and minutes, and program schedules and activities to IV&V in a
timely manner.

## Responsibilities

**Table 4-2, Primary Participant Responsibilities**, details the
responsibilities for participants in the IV&V process.

+----------------------+------------------------+----------------------+
| **Primary            | **Primary              | **Support            |
| Participants**       | Responsibilities**     | Responsibilities**   |
+======================+========================+======================+
| IV&V Technical       | Overall management of  | Reports to the ERA   |
| Director             | the ERA IV&V effort    | PD                   |
+----------------------+------------------------+----------------------+
| IV&V Subject Matter  | IV&V specialist with a | Participates in      |
| Expert (SME)         | high degree of         | technical            |
|                      | knowledge of the V&V   | interchange and      |
|                      | process and            | other meetings as    |
|                      | program/project life   | required. Reports to |
|                      | cycle process          | the IV&V Technical   |
|                      | requirements. Performs | Director             |
|                      | technical reviews,     |                      |
|                      | assessments and tests  |                      |
|                      | as required            |                      |
+----------------------+------------------------+----------------------+
| Test SME             | Specialist with        | Participates in      |
|                      | specific knowledge of  | technical            |
|                      | Testing and Test       | interchange and      |
|                      | techniques. Performs   | other meetings as    |
|                      | technical reviews,     | required. Reports to |
|                      | assessments and tests  | the IV&V Technical   |
|                      | as required            | Director when        |
|                      |                        | supporting IV&V      |
|                      |                        | activities and tasks |
+----------------------+------------------------+----------------------+
| ERA Program Director | Overall management     | Primary interface    |
| (PD)                 | responsibility for the | for issues,          |
|                      | ERA Program\*          | concerns, risks      |
|                      |                        | and/or anomalies     |
|                      | \***NOTE:** *The PD is | identified by IV&V.  |
|                      | responsible for        |                      |
|                      | management of the IV&V |                      |
|                      | effort for the ERA     |                      |
|                      | program and may        |                      |
|                      | delegate management    |                      |
|                      | tasks to Government    |                      |
|                      | staff as appropriate.* |                      |
+----------------------+------------------------+----------------------+
| ERA Contractor(s)    |                        | Provide support as   |
|                      |                        | needed during IV&V   |
|                      |                        | testing, NARA ERA    |
|                      |                        | PMO acceptance,      |
|                      |                        | installation, and    |
|                      |                        | operational testing. |
|                      |                        | Provide access to    |
|                      |                        | facilities and       |
|                      |                        | supporting work      |
|                      |                        | products related to  |
|                      |                        | ERA development and  |
|                      |                        | testing.             |
+----------------------+------------------------+----------------------+
| Consultant           | TBD                    |                      |
+----------------------+------------------------+----------------------+

[]{#__RefHeading___Toc61244755 .anchor}Table 4-2: Primary Participant
responsibilities

## Tools, Techniques, and Methods

The following paragraphs describe the tools, techniques, and methods to
be used by IV&V.

### Tools

IV&V will verify the accuracy of the requirements traceability matrix
and maintain an independent anomaly tracking system using Microsoft
Access. IV&V will maintain document assessment checklists in Microsoft
Access. This will allow IV&V to record, track, and compare evaluations
of iterative releases of the same document. If needed IV&V, also will
maintain IV&V identified risks in Microsoft Access 2002.

IV&V anticipates that an automated tool for evaluating the components of
a Web interface will be needed. Description and details are TBD.

### Techniques

IV&V will participate in each technical interchange meeting with the ERA
PD, or designated representative, and the ERA contractor(s). IV&V will
provide information to aid in the delivery of products that are
compliant with the technical and contractual requirements of the ERA
program. All products identified herein will be reviewed. Additionally,
IV&V will randomly select and perform a sampling of System and
Acceptance Test Cases. Statistical analysis techniques will be used to
project the results of the IV&V testing on the population of System or
Acceptance Test results previously achieved. Results of IV&V analyses,
evaluations and tests will be provided to the ERA PD, the IV&V COR, the
ERA contractor(s) (if so directed), and appropriate members of the
Program Office Support Team (POST), in a timely manner.

### Methods

The methodology to be used for the ERA IV&V approach has been selected
to be consistent with the overall ERA life cycle model. The focus of the
method is to identify any flaws or errors as early as possible, when
they are easier and more cost effective to correct. This plan provides a
framework of activities that are performed during releases and
increments within phases of the ERA life cycle. IV&V maintains an
overall system perspective and analyzes each activity to ensure that
progress continues toward the completion of the ERA system. This
analysis includes ensuring that an ERA configuration baseline is
established and maintained. This approach will be taken in order to
maintain system integrity.

IV&V will accomplish system verification and validation by examining the
correctness, completeness, consistency, reliability, and maintainability
of the ERA products at each step in the process. Correctness means that
the product being evaluated satisfies the baseline system requirements.
Completeness means that all required functions are implemented and all
necessary products are procured to fully support the program life cycle.
Consistency means that the relationships between all requirements,
documentation, and products are constant and appropriately detailed at
each stage of development. Reliability means that the final (end item)
product can be expected to perform its intended functions without error
or failure as reflected in the individual products. Maintainability is
achieved when each product supports life cycle maintenance,
modifications, and future enhancements.

Throughout the acquisition life cycle, IV&V will conduct verification
and validation activities for the initial configuration as well as for
modifications, enhancements, and approved changes. The results of these
efforts may require certain task reiterations, such as documentation
reviews for each version update.

ERA will be acquired through a down-select process to be followed by
development using an incremental life cycle model. During the period
prior to the down-select, the two ERA contractors will be developing a
System Requirements Specification (SyRS) and a System Architecture and
Requirements Allocation Description (SARAD). These documents usually
define the Functional and Allocated Baselines. At this point the PMO
will select a contractor to continue the development effort. IV&V will
review these, and any other documents or plans, and will generate an
IV&V Task Report for each document or plan formally reviewed. After
down-select, the ERA contractor will be responsible for delivering ERA.
A minimal level of operational capability will exist at the completion
of Increment #1. This will be known as the IOC. Even as the ERA
contractor completes work on the system IOC, the contractor will be
initiating work on the capabilities that will be part of Increment #2.
There will be no less than two releases for increment #2 and for
successive increments until FOC is attained.

For each increment, and its respective releases, the ERA contractor will
provide a full set of product documentation or change pages. IV&V will
review each document and/or artifact and provide an IV&V Task Report on
each. Additionally, IV&V, for each release within each increment, will
participate in design reviews, technical interchange meetings, Test
Readiness Reviews (TRRs), management reviews, and audits. IV&V will
prepare an IV&V Meeting Summary Report for each meeting attended.

As a result of the incremental model, IV&V may be performing Operations
IV&V Activity tasks for the capabilities delivered as IOC while
performing Installation IV&V Activity tasks for the first release of
Increment #2. IV&V may be performing Development IV&V Activity tasks for
the second release of Increment #2 and concurrently performing
Requirements IV&V Activity tasks for the first release of Increment #3.

In addition to the IV&V Task Reports generated at the completion of each
IV&V task, IV&V also will produce an End of Activity Summary Report at
the completion of each iteration of each IV&V activity. IV&V will
generate a Final IV&V Summary Report after IOC and FOC.

# IV&V Life Cycle Verification and Validation

This section provides detailed information about the IV&V activities
performed during the ERA life cycle as described in the *ERA Life Cycle
(ELC)* document. IV&V activities, and the tasks to implement them, are
described as specified in IEEE Std. 1012--1998. For each identified
task, this plan includes the methods and criteria, inputs and outputs,
resources, schedules, risks, assumptions, and roles and
responsibilities. Items that have been discussed elsewhere in this plan
are not repeated for each identified task. Specifically, information
relating to assumptions is provided in **Section 1.7**, resources in
**Section 4.4**, and roles and responsibilities in **Section 4.5**. For
those tasks where the risks have been identified, a discussion of the
risk is provided with the task description. The methodology used for the
IV&V approach is derived from the IEEE/EIA Std. 12207.0 life cycle
methodology. This methodology, in conjunction with IEEE Std. 1028-1997
governing reviews, provides the root evaluation criteria for
configuration items and the entry/exit criteria associated with each
milestone.

Each task in this section identifies the inputs required by IV&V to
perform assigned responsibilities in accordance with this plan.

## Management Process

The management process consists of a single activity with five (5) tasks
that are performed by those responsible for managing the IV&V effort
during the ERA acquisition life cycle. The IV&V management
responsibilities are to:

-   Prepare the plan for execution of the IV&V Process,

-   Implement the plan,

-   Monitor the execution of the plan,

-   Analyze problems uncovered during the execution of the plan,

-   Report progress,

-   Ensure that products satisfy requirements,

-   Assess evaluation results,

-   Determine task completion, and

-   Check the results for completeness.

### Management Activity

The management responsibilities of the IV&V Technical Director are
planning, reviewing, and controlling the IV&V process. Management's
responsibility is to ensure positive, successful interaction between
IV&V and other program activities to promote the deployment of a
defect-free system. The planning for IV&V management considers the
iterative nature of the IV&V tasks described in **Section 5.0**. IV&V
management activities are executed throughout the ERA life cycle.

Specific IV&V management activity tasks include:

-   Generate Independent Verification and Validation Plan,

-   Perform Baseline Change Assessment,

-   Perform Management Review of IV&V,

-   Support Management and Technical Reviews,

-   Interface With Organizational and Supporting Processes, and

-   Identify process improvement opportunities in the conduct of IV&V.

#### Generate Independent Verification and Validation Plan 

IV&V planning is most effectively performed in conjunction with the
overall planning for system acquisition. ERA is being acquired and
deployed as a series of releases and increments. This will require that
IV&V tasks and activities be performed iteratively. Revisions will be
made to this plan when changes to the scope of the ERA require
modification (addition/deletion) to tasks specified in the plan, or as
more detailed task management is required for future activities and
tasks.

Revisions to the IVVP will not be made as a way of resolving deviations
(e.g., although the IVVP calls for witnessing the ERA contractor(s)
integration test, integration test may not be performed) from the
program master plan. Significant variances from the baselined IVVP will
be addressed in the appropriate Activity Summary Report and in the IV&V
Final Report.

+-----------------+----------------------------------------------------+
| Task:           | Generate Independent Verification and Validation   |
|                 | Plan                                               |
+-----------------+----------------------------------------------------+
| Methods:        | Material upon which the document is to be based    |
|                 | will be collected and compiled. A proposed         |
|                 | approach will be chosen and the IVVP will be       |
|                 | generated and released for review. Updates will be |
|                 | published semi-annually or as necessary to         |
|                 | accommodate significant program changes or as      |
|                 | directed by the IV&V COR.                          |
+-----------------+----------------------------------------------------+
| Inputs:         | IEEE Std. 1012-1998, CONOPS, PMP, Program Master   |
|                 | Schedule/Work Breakdown Structure (WBS) see        |
|                 | **Appendix A, IV&V Work Breakdown Structure (WBS)  |
|                 | and Schedule**, Program Criticality Analyses,      |
|                 | other program-level and contractor planning        |
|                 | documents as required.                             |
+-----------------+----------------------------------------------------+
| Outputs:        | IV&V Plan                                          |
+-----------------+----------------------------------------------------+
| Schedule:       | Initial Final: Q3 2003                             |
|                 |                                                    |
|                 | Modified Final: Q1 & Q3 2004, Q1 & Q3 2005, etc.,  |
|                 | and, as necessary.                                 |
|                 |                                                    |
|                 | Plan submitted to the government for approval,     |
|                 | upon completion                                    |
+-----------------+----------------------------------------------------+
| Resources:      | Refer to **Section 4.4**                           |
+-----------------+----------------------------------------------------+
| Risks and       | The quality and completeness of the IVVP is highly |
| Assumptions:    | dependant on the quality and completeness of       |
|                 | program-level and contractor planning documents.   |
+-----------------+----------------------------------------------------+
| Roles and       | Refer to **Section 4.5**                           |
| Re              |                                                    |
| sponsibilities: |                                                    |
+-----------------+----------------------------------------------------+

#### Perform Baseline Change Assessment

Baseline change assessment may be the most dynamic task performed during
the IV&V effort. Any proposed change to an existing baseline could
affect an unknown amount of previously completed development and IV&V
work. IV&V evaluates proposed changes for completeness, consistency,
potential benefits, and technical impact; and to determine if the change
will be provided without negatively impacting the system as a whole.

  ------------------- ----------------------------------------------------
  Task:               Perform Baseline Change Assessment

  Methods:            Whenever a change request is generated, IV&V will
                      review the proposed change to identify its impact on
                      the system and on planned IV&V activities. If the
                      proposed change would have an adverse or otherwise
                      undesirable impact, IV&V will identify that impact
                      and the associated risk. The proposed change
                      evaluation will be completed and returned to the
                      IV&V COR and the ERA Configuration Control Board
                      (CCB).

  Inputs:             Change proposals and other applicable documents.

  Outputs:            IV&V Task Reports, input to Activity Summary
                      Reports, Anomaly Reports.

  Schedule:           IV&V will respond to the issuance of each formal
                      change request and provide a preliminary change
                      assessment (in the form of an IV&V Quick Look
                      Report) within three working days to the IV&V COR. A
                      full IV&V technical analysis and IV&V Assessment
                      Task Report will be completed within five (5)
                      working days after issuance of the IV&V Quick Look
                      Report.

  Resources:          Refer to **Section 4.4**.

  Risks and           IV&V's ability to effectively analyze system change
  Assumptions:        requests in a timely manner depends on the quality
                      of the baseline documentation and the effectiveness
                      of baseline control mechanisms.

  Roles and           Refer to **Section 4.5**.
  Responsibilities:   
  ------------------- ----------------------------------------------------

#### Perform Management Review of IV&V

Management review is used by IV&V and comprises all of the general
responsibilities of management for the monitoring, controlling, and
reporting of the IV&V effort.

+-----------------+----------------------------------------------------+
| Task:           | Perform management review of IV&V.                 |
+-----------------+----------------------------------------------------+
| Methods:        | Manage implementation of the IV&V Plan, monitor    |
|                 | effectiveness of IV&V activities, identify IV&V    |
|                 | training needs, perform risk management, and       |
|                 | ensure the quality of the IV&V products. Review    |
|                 | the IV&V effort, effecting changes to IV&V tasks   |
|                 | or redirecting the IV&V effort, when appropriate.  |
|                 | Recommend to the PD whether to proceed to the next |
|                 | acquisition life cycle activity and provide IV&V   |
|                 | Task Reports and End of Activity Summary Reports   |
|                 | to the organization(s) identified in **Section     |
|                 | 6.0**.                                             |
|                 |                                                    |
|                 | The NARA ERA PD ensures that the IVVP is supported |
|                 | by all ERA contractor and Government personnel.    |
|                 | Meetings are held to report progress and plan work |
|                 | for the coming period. IV&V staff, as needed, will |
|                 | obtain authorization to attend program reviews and |
|                 | technical exchange meetings. For each meeting      |
|                 | attended, the IV&V lead prepares a report          |
|                 | summarizing the proceedings and documenting the    |
|                 | results. The IV&V Technical Director prepares a    |
|                 | Monthly Status Report for the ERA PD.              |
|                 |                                                    |
|                 | IV&V participates in risk management               |
|                 | identification and mitigation activities. IV&V     |
|                 | follows general risk management principles         |
|                 | including teamwork, global perspective,            |
|                 | forward-looking-view, open communications, and     |
|                 | continuous process improvement. The IV&V staff     |
|                 | applies these principles and discusses the         |
|                 | identified risks and associated mitigation plans   |
|                 | with the appropriate ERA personnel.                |
+-----------------+----------------------------------------------------+
| Inputs:         | IV&V Plan, ERA deliverable products.               |
+-----------------+----------------------------------------------------+
| Outputs:        | IV&V Program Reviews and Status Reports, program   |
|                 | review briefings, identified risks and risk        |
|                 | mitigation recommendations, Meeting Summary        |
|                 | Reports, IV&V Task Reports, IV&V products and      |
|                 | updates, inputs to End of Activity Summary         |
|                 | Reports, Anomaly Reports.                          |
+-----------------+----------------------------------------------------+
| Schedule:       | Program reviews and deliverable status reports are |
|                 | scheduled activities. All other outputs are driven |
|                 | by meetings attended and products received.        |
+-----------------+----------------------------------------------------+
| Resources:      | Refer to **Section 4.4**.                          |
+-----------------+----------------------------------------------------+
| Risks and       | It is assumed that status, concerns, issues and    |
| Assumptions:    | program risks identified through the management    |
|                 | review activity will be accepted by the ERA PMO as |
|                 | a method of providing added value to the overall   |
|                 | ERA program. The risk associated with this         |
|                 | activity is the ERA PMO failure to recognize       |
|                 | observations and findings; and to act on them in a |
|                 | timely manner to prevent or correct potential      |
|                 | problems.                                          |
+-----------------+----------------------------------------------------+
| Roles and       | Refer to **Section 4.5**.                          |
| Re              |                                                    |
| sponsibilities: |                                                    |
+-----------------+----------------------------------------------------+

#### Support Management and Technical Reviews

Formal review meetings are a common means of evaluating and approving
the products of one ERA life cycle phase before going on to the next.
IV&V contributes to the effectiveness of management and technical
reviews through a variety of functions and tasks. In most cases, these
functions and tasks will be similar from one review to the next. As a
result, this plan will not have specific details for each individual
review to be supported; rather, it will identify the general
preparations to be made for a review, together with specific information
such as the material to be evaluated. IV&V, as required, will support
the following reviews: Preliminary Design Reviews, Critical Design
Reviews (CDRs), TRRs, Technical Interchange Meetings, Program Management
Reviews, Operational Readiness Reviews, and configuration audits,
especially the Functional Configuration Audit (FCA) and the Physical
Configuration Audit (PCA).

+-----------------+----------------------------------------------------+
| Task:           | Provide Management and Technical Review Support.   |
+-----------------+----------------------------------------------------+
| Methods:        | Before attending a review, IV&V will evaluate the  |
|                 | relevant documents. The objectives of the review,  |
|                 | as stated in the ERA Program Management Plan, are  |
|                 | the focus of the evaluation. A list of questions   |
|                 | and/or issues appropriate to the review may be     |
|                 | prepared prior to the review. While attending the  |
|                 | reviews, IV&V will evaluate the material presented |
|                 | based on predefined criteria. At the conclusion of |
|                 | each review, IV&V will support the Government      |
|                 | meetings held to discuss the completeness of the   |
|                 | review and any open technical issues.              |
|                 |                                                    |
|                 | Following the review, IV&V will analyze the review |
|                 | data, issues, resolutions, and discussions. A      |
|                 | report summarizing the events of the review will   |
|                 | be prepared to identify any risks or open issues.  |
+-----------------+----------------------------------------------------+
| Inputs:         | Integration contractor planning documents, product |
|                 | deliverables appropriate to each review,           |
|                 | guidelines for contractor reviews and milestones,  |
|                 | and standards for reviews.                         |
+-----------------+----------------------------------------------------+
| Outputs:        | IV&V Task Reports, inputs to the End of Activity   |
|                 | Summary Report, Anomaly Reports.                   |
+-----------------+----------------------------------------------------+
| Schedule:       | Prior to each meeting associated with a milestone  |
|                 | review.                                            |
+-----------------+----------------------------------------------------+
| Resources:      | Refer to **Section 4.4**.                          |
+-----------------+----------------------------------------------------+
| Risks and       | None.                                              |
| Assumptions:    |                                                    |
+-----------------+----------------------------------------------------+
| Roles and       | Refer to **Section 4.5**.                          |
| Re              |                                                    |
| sponsibilities: |                                                    |
+-----------------+----------------------------------------------------+

#### Interface with Organizational and Supporting Processes

The IV&V Technical Director is responsible for coordinating the
interfaces with the organizational and supporting processes. This task
is designed to help insure the smooth flow of information and data
between the IV&V effort and other ERA acquisition life cycle processes.

  ------------------- -----------------------------------------------------
  Task:               Coordinate the IV&V effort with organizational (e.g.,
                      management, improvement) and supporting processes
                      (e.g., quality assurance, joint review, and problem
                      resolution).

  Methods:            Identify the IV&V data to be exchanged with these
                      processes. Document the data exchange requirements in
                      the IVVP.

  Inputs:             The IVVP and data identified in the IVVP from the
                      organizational and supporting processes.

  Outputs:            Updates to the IVVP

  Schedule:           As required.

  Resources:          The IV&V Technical Director.

  Risks and           None.
  Assumptions:        

  Roles and           Refer to **Section 4.5**.
  Responsibilities:   
  ------------------- -----------------------------------------------------

#### Identify process improvement opportunities in the conduct of IV&V.

Identification of process improvement opportunities offers another
opportunity for IV&V to add value.

+------------+---------------------------------------------------------+
| Task:      | Identify process improvement opportunities              |
+------------+---------------------------------------------------------+
| Methods:   | Opportunities will be identified as IV&V executes the   |
|            | tasks identified in this IVVP. Each identified          |
|            | opportunity for process improvement will be evaluated   |
|            | and a determination made concerning the most efficient  |
|            | and effective means of implementing the improvement.    |
|            |                                                         |
|            | Process improvement may relate to the sequence in which |
|            | tasks are executed, the manner in which tasks are       |
|            | executed, the way in which task results are reported or |
|            | even the way in which staff are assigned to each task.  |
+------------+---------------------------------------------------------+
| Inputs:    | Tasks identified in IVVP                                |
+------------+---------------------------------------------------------+
| Outputs:   | Change in IV&V processes; modification to IVVP; input   |
|            | to End of Activity Summary Reports                      |
+------------+---------------------------------------------------------+
| Schedule:  | This task is ongoing and not schedule driven            |
+------------+---------------------------------------------------------+
| Resources: | Refer to **Section 4.4**                                |
+------------+---------------------------------------------------------+
| Risks and  | None                                                    |
| As         |                                                         |
| sumptions: |                                                         |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

## Acquisition Process

The ERA acquisition process begins with the identification of the need
to acquire ERA. The process continues with the preparation and issuance
of the Request For Proposal (RFP), selection of a supplier, and
management of the acquisition process through to the acceptance and
deployment of the system. The IV&V effort uses the acquisition process
to scope the IV&V effort, plan interfaces with the ERA contractor(s) and
the Government, and review the draft system requirements contained in
the RFP.

### Acquisition Support Activity

Providing acquisition support, IV&V addresses program initiation, RFP,
contract preparation, contractor(s) monitoring, and the acceptance and
deployment of ERA. IV&V shall scope the IV&V effort; plan the interfaces
between the IV&V effort, the Government and the contractor(s); review
the draft requirements, the RFP, the Source Selection Evaluation Board
report, and the draft contract(s).

Specific acquisition support activity tasks include:

-   Scope the IV&V Effort (preliminary);

-   Plan the Interface between the IV&V effort, NARA, and the
    contractor(s);

-   Review the *ERA Source Selection Plan*;

-   Review the *ERA Acquisition Strategy* document;

-   Review the *ERA Analysis of Alternatives* document;

-   Review the *ERA Configuration Management Plan*;

-   Review the *ERA Concept of Operations* document;

-   Review the *ERA Life Cycle* document;

-   Review the *ERA Metrics Plan*;

-   Review the *ERA Program Management Plan*;

-   Review the *ERA Quality Management Plan*;

-   Review the *ERA Requirements Document*;

-   Review the *ERA Requirements Management Plan*;

-   Review the *ERA Risk Management Plan*;

-   Review ERA *System Security Plan*;

-   Review the *ERA Request for Proposal*;

-   Review and Evaluate the Source Selection Process; and

-   Review the ERA Draft Contract.

#### Scope the IV&V Effort (Preliminary)

Identify the ERA program criticality levels. Assign an integrity level
to ERA. Establish the degree of independence required for IV&V. Identify
the IV&V tasks appropriate to the identified integrity level.

  ------------------- -----------------------------------------------------
  Task:               Scope the IV&V effort (Preliminary)

  Methods:            Identify the system integrity level, determine the
                      minimum tasks based on Integrity level, identify
                      optional tasks, establish scope (breadth and depth)
                      of IV&V

  Inputs:             *ELC* document, IEEE Std. 1012-1998, Table 2

  Outputs:            Draft IV&V Plan, input to End of Acquisition Support
                      Activity Summary Report

  Schedule:           Q2 2003

  Resources:          ERA COR, ERA PD, IV&V Technical Director, IV&V SMEs,
                      ERA/POST Team Leads

  Risks and           Delay in identifying integrity level will have a
  Assumptions:        negative impact on the scoping of the IV&V effort.
                      IV&V will support the Government decision making
                      process. The assumption is that the Government will
                      make the criticality decision in a timely manner.

  Roles and           Refer to **Section 4.5**
  Responsibilities:   
  ------------------- -----------------------------------------------------

#### Plan the Interfaces between the IV&V Effort, NARA, and the ERA Contractor(s) 

Plan the schedule for each IV&V task. Identify the preliminary list of
development processes and products to be evaluated by IV&V.

  ------------------- -----------------------------------------------------
  Task:               Identify the IV&V/ERA Contractor(s)/Government
                      Interface.

  Methods:            Based on the identified system integrity level
                      (**5.2.1.1**) develop the preliminary list of
                      development processes and products to be evaluated.

  Inputs:             ERA Request for Proposal, ERA Life Cycle document,
                      ERA Acquisition Strategy, ERA WBS.

  Outputs:            Draft IV&V Plan, input to RFP, input to End of
                      Acquisition Support Activity Summary Report.

  Schedule:           Q2 2003.

  Resources:          Refer to **Section 4.4**.

  Risks and           ERA Request for Proposal, ERA Life Cycle document,
  Assumptions:        ERA Acquisition Strategy and related documents
                      contain adequate detail for task and will be
                      available in a timely manner. IV&V planning will be
                      incorrect if the documentation is not accurate and
                      complete.

  Roles and           Refer to **Section 4.5**.
  Responsibilities:   
  ------------------- -----------------------------------------------------

#### Review the Source Selection Plan

Review the ERA Source Selection Plan for conformance to Government
Regulations.

#  {#section .Style-Header-+-Tahoma}

+----------------+-----------------------------------------------------+
| Task:          | Evaluate *ERA Source Selection Plan (SLP)*          |
|                | document.                                           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to *Federal Acquisition    |
|                | Regulation*                                         |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA SLP* document, *Federal Acquisition            |
|                | Regulation*                                         |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q3/Q4 2003                          |
|                |                                                     |
|                | #  {#section-1 .Style-Header-+-Tahoma}              |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document.               |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Acquisition Strategy document

Review the Acquisition Strategy for conformance to Government
Regulations and applicable standards in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Acquisition Strategy (AS)* document     |
+----------------+-----------------------------------------------------+
| Methods:       | Review for conformance to DoD 5000.2-R, OMB A-11,   |
|                | *Federal Acquisition Regulation* and IEEE Std.      |
|                | 12207.1, section 6.1, IEEE Std. 1062, and other     |
|                | applicable federal laws and regulations.            |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA AS* document, DoD 5000.2-R, OMB A-11*, Federal |
|                | Acquisition Regulation*, IEEE Std. 12207.1, section |
|                | 6.1, IEEE Std. 1062                                 |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q3/Q4 2003                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Analysis of Alternatives document

Review Analysis of Alternatives document for conformance to Government
regulations and applicable standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review ERA *Analysis of Alternatives (AoA)*         |
|                | document                                            |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to Government regulations  |
|                | especially General Accounting Office, Information   |
|                | Technology: An Audit Guide for Assessing            |
|                | Acquisition Risks, December 1992 (Chapter 5),       |
|                | Defense Finance and Accounting Service, Analysis of |
|                | Alternatives Report, DFAS 8000.1-R, Part C, Chapter |
|                | 1                                                   |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA *AoA* document, General Accounting Office,      |
|                | Information Technology: An Audit guide for          |
|                | Assessing Acquisition Risks, December 1992 (Chapter |
|                | 5), Defense Finance and Accounting Service,         |
|                | Analysis of Alternatives Report, DFAS 8000.1-R,     |
|                | Part C, Chapter 1                                   |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Configuration Management Plan

Review the Configuration Management Plan for conformance to the
applicable standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review ERA *Configuration Management Plan (CMP)*    |
|                | document.                                           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 828-1998 in   |
|                | format, purpose, and content                        |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA CMP* document, IEEE Std. 828-1998              |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Concept of Operations document

Review the Concept of Operations document for conformance to the
applicable standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Concept of Operations* (*ConOps)*       |
|                | document                                            |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 1362-1998 for |
|                | formal, purpose, and content                        |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA ConOps* document, IEEE Std. 1362-1998          |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2/Q3 2003                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Life Cycle document

Evaluate the ERA Life Cycle document for conformance to the applicable
standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review the *ERA Life Cycle* (*ELC)* document        |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE 12207, section 5.2 |
|                | and IEEE Std. 1062-1998 in format, purpose, and     |
|                | content                                             |
+----------------+-----------------------------------------------------+
| Inputs:        | *ELC* document, IEEE 12207, section 5.2, IEEE Std.  |
|                | 1062-1998                                           |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: TBD                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Metrics Plan

Evaluate the Metrics Plan for conformance to the applicable standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Metrics Plan (MP)* document.            |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate contents for minimum metrics identified by |
|                | the *SEI-CMM for Software* (Level 2) and IEEE Std.  |
|                | 12207.1-1997, Section 5.2 in format, purpose, and   |
|                | content and general conformance with the principles |
|                | stated in IEEE Std. 1028-1998, IEEE Std. 1045-1992  |
|                | and IEEE Std. 1061-1998.                            |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA MP* document, *SEI-CMM for Software*, version  |
|                | 1.1, IEEE Std. 12207.1-1997, IEEE Std. 1028-1998,   |
|                | IEEE Std. 1045-1992 and IEEE Std. 1061-1998         |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Program Management Plan

Review Program Management Plan for conformance to applicable standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Program Management Plan (PMP)*          |
|                | document.                                           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 1058-1998 in  |
|                | format, purpose, and content.                       |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA PMP* document, IEEE Std. 1058-1998, PMBOK^®^   |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input End of Acquisition Support  |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003.                            |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document.               |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Quality Management Plan

Review Quality Management Plan for conformance to applicable standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Quality Management Plan (QMP)*          |
|                | document.                                           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 730-1998,     |
|                | PMBOK^®^, and SEI-CMM level 2 in format, purpose,   |
|                | and content                                         |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA QMP* document, IEEE Std. 730-1998, PMBOK^®^,   |
|                | SEI-CMM level 2                                     |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2/Q3 2003                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Requirements Document

Review Requirements Document for conformance to applicable standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Requirements Document (RD)* document.   |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 1233-1998 in  |
|                | format, purpose, and content, verify that criteria  |
|                | described in IEEE Std. 1012-1998, Table 1 are       |
|                | satisfied                                           |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA RD* document, IEEE Std. 1233-1998, IEEE Std.   |
|                | 1012-1998, Table 1                                  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2/Q3 2003                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Requirements Management Plan

Review Requirements Management Plan for conformance to applicable
standard in format, purpose, and content.

  ------------------- -----------------------------------------------------
  Task:               Review *ERA Requirements Management Plan (RQM)*
                      document.

  Methods:            Evaluate for conformance to IEEE Std. 12207.1,
                      section 5.2 in format, purpose, and content

  Inputs:             *ERA RQM* document, IEEE Std. 12207.1, section 5.2,
                      IEEE Std. 828-1998, SEI-CMM (Level 2)

  Outputs:            IV&V Task Report, input to End of Acquisition Support
                      Activity Summary Report, Anomaly Reports

  Schedule:           Report submitted to the Government within seven (7)
                      business days of receipt of document

  Resources:          Refer to **Section 4.4**

  Risks and           None
  Assumptions:        

  Roles and           Refer to **Section 4.5**
  Responsibilities:   
  ------------------- -----------------------------------------------------

#### Review Risk Management Plan

Review Risk Management Plan for conformance to applicable standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Risk Management Plan (RKM)* document.   |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 1540-2001 in  |
|                | format, purpose, and content, and the SEI-CMM Risk  |
|                | Management Paradigm                                 |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA RKM* document, IEEE Std. 1540-2001, SEI-CMM    |
|                | (Level 2,)                                          |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review System Security Plan

Review the System Security Plan for conformance to applicable standards
in format, purpose, and content, and applicable Government law and
regulation.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA System Security Plan (SSP)* document.   |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to P.L 107-347, U.S.C.     |
|                | Vol. 44, 3541-9, NIST SP 800-18                     |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA SSP* document, *E-Government Act of 2002*,     |
|                | Public Law 107-347, 17 December 2002, IEEE/EIA      |
|                | 12207.1-1997, *Software Life Cycle Processes --     |
|                | Life cycle data*, *Federal Information Security     |
|                | Management Act of 2002, U.S. Code,* Vol. 44,        |
|                | sections. 3541-9, NIST SP 800-18                    |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q2 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document.               |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Request for Proposal

Review the Request for Proposal for conformance to applicable Government
law and regulation. Review the RFP for inclusion of applicable standards
for integration contractor(s) deliverables and for inclusion of ERA
specific requirements.

+----------------+-----------------------------------------------------+
| Task:          | Review *ERA Request for Proposal (RFP)* document.   |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to Government law and      |
|                | regulation and for inclusion of ERA specific        |
|                | requirements, verify the criteria described in IEEE |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA RFP* document, OMB A-11, *Federal Acquisition  |
|                | Regulation*, IEEE Std. 1012-1998, Table 1           |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Initial Final: Q1 2003                              |
|                |                                                     |
|                | Modified Final: Q3 2003                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review Evaluate the Source Selection Process

Evaluate the ERA source selection process to ensure conformance to
Source Selection Plan.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate ERA Source Selection Process               |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to ERA source selection    |
|                | requirements and guidelines identified in the ERA   |
|                | Source Selection Plan                               |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA Source Selection Evaluation Report*, *ERA      |
|                | Source Selection Plan,* IV&V assessment instrument  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Q1 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of completion of task                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Review the Draft Contract

Review the Draft Contract to ensure the inclusion of ERA specific
requirements and for adequacy, correctness, and completeness.

+----------------+-----------------------------------------------------+
| Task:          | Review ERA Draft Contract                           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate Draft Contract (sec. B, C, E, F, H)        |
|                | against the proposal and any negotiated changes to  |
|                | the proposal and for inclusion of ERA specific      |
|                | requirements such as standards for deliverables and |
|                | active role of Government IV&V, verify the criteria |
|                | described in IEEE Std. 1012-1998, Table 1 are       |
|                | satisfied                                           |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA Draft Contract, Request for Proposal, Proposal, |
|                | negotiated changes, IEEE Std. 1012-1998, Table 1    |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Acquisition       |
|                | Support Activity Summary Report, Anomaly Reports    |
+----------------+-----------------------------------------------------+
| Schedule:      | Q1 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

## Supply Process

The supply process is initiated by signing and entering into a contract
with the supplier to provide ERA. The process continues with the
identification of procedures and resources needed to manage the project.
IV&V uses the supply process products to verify that the RFP
requirements and the contract requirements are consistent with each
other and with NARA's needs.

### Planning IV&V

Planning IV&V addresses the initiation, preparation of response,
contract planning, execution and control, review, and evaluation and
delivery and completion of IV&V tasks and reports.

Specific IV&V planning activity tasks include:

-   Planning the interface between the IV&V effort and the development
    contractor(s), and

-   Verifying the contract.

#### Plan the Interface between the IV&V Effort and the ERA Contractor(s)

Review the ERA contractor(s) plans and schedules to coordinate the IV&V
effort with development activities.

  ------------------- -----------------------------------------------------
  Task:               Plan the IV&V and ERA Contractor(s) Interface

  Methods:            Review ERA contractor(s) plans and schedules to
                      coordinate the IV&V effort with ERA contractor(s)
                      activities

  Inputs:             IV&V Task Report, contract (technical), integration
                      contractor(s) plans and schedules

  Outputs:            Revised IV&V Plan, IV&V Task Report, input to End of
                      Planning IV&V Activity Summary Report

  Schedule:           Q1 2004

  Resources:          Refer to **Section 4.4**

  Risks and           Planning the IV&V interface will be incorrect if the
  Assumptions:        input documentation is not accurate and complete.

  Roles and           Refer to **Section 4.5**
  Responsibilities:   
  ------------------- -----------------------------------------------------

#### Verify the Contract 

Verify the contract for conformance to Government need and consistency
with draft contract and negotiated items.

+----------------+-----------------------------------------------------+
| Task:          | Verify the ERA contract(s)                          |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate the ERA contract(s) (sec. B, C, E, F, H)   |
|                | for consistency with RFP, and negotiated items.     |
|                |                                                     |
|                | Verify that criteria described in IEEE Std.         |
|                | 1012-1998, Table 1 are satisfied.                   |
+----------------+-----------------------------------------------------+
| Inputs:        | RFP, ERA contract(s), draft contract, negotiated    |
|                | items, IEEE-Std. 1012-1998, Table 1                 |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Planning IV&V     |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q1/Q2 2004                                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

## Development Process 

The development process contains the ERA contractor(s) development
activities and tasks. The process contains the activities for
requirements analysis, design, coding, integration, testing, and
installation and acceptance for the ERA software and ERA. The IV&V
activities verify and validate all delivered products. The ERA IV&V
activities, paralleling the development activities, are titled Concept,
Requirements, Design, Implementation, Test, and Installation and
Checkout.

### Concept Activity

The concept activity represents the delineation of a specific
implementation solution to address NARA's needs. During this activity,
the system architecture is selected and system requirements are
allocated to hardware, software, networking, database and interface
components. The objectives of IV&V during this activity are to verify
the allocation of system requirements, validate the selected
architecture and system design solution, and ensure that no false
assumptions have been incorporated into the proposed solution.

IV&V anticipates that there will be two ERA contractors performing tasks
during the Concept phase. To insure that IV&V tasks are performed in an
efficient, thorough and consistent manner, the IV&V team will be split
to allow an independent team to be assigned to each of the two ERA
contractors. The IV&V Technical Director will ensure that both teams are
consistent in their application of evaluation criteria and standards.

The specific IV&V tasks are:

-   Evaluate system requirements specification,

```{=html}
<!-- -->
```
-   Evaluate system architecture description,

-   Evaluate system/subsystem design descriptions,

-   Evaluate allocation of requirements,

-   Evaluate plans,

-   Perform (requirements) traceability analysis,

-   Perform criticality analysis, and

-   Perform risk analysis.

#### Evaluate System Requirements Specification

Evaluate ERA contractor(s) SyRS for conformance to the applicable
standard in format, purpose, and content. The system requirements
specification document is assessed for completeness, correctness,
accuracy, and testability.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate the ERA contractor(s) system requirements  |
|                | specification document                              |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate the system requirements for conformance to |
|                | IEEE Std. 1233-1998. Ensure the requirements are    |
|                | complete, correct, accurate, and testable. Verify   |
|                | that the criteria of IEEE Std. 1012-1998, Table 1   |
|                | are satisfied.                                      |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) SyRS, ERA RD, ERA IRD, ERA        |
|                | ConOps, ERA Use Cases, IEEE Std. 1233-1998, IEEE    |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Concept Activity  |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within ten       |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate System Architecture Description

The system architecture description displays the depth to which the ERA
contractor(s) understands NARA needs. Evaluate the system architecture
description portion of the SARAD for conformance to the applicable
standard in format, purpose, and content. The system architecture
description is assessed for completeness, correctness and accuracy.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate the ERA contractor(s) system architecture  |
|                | description                                         |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate the system architecture for conformance to |
|                | IEEE Std. 1471-2000, IEEE/EIA 12207.1, section 6.25 |
|                | and to insure the proposed approach is consistent   |
|                | with NARA needs. Verify that the criteria described |
|                | in IEEE Std. 1012-1998, Table 1 are satisfied       |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) SyRS, system architecture         |
|                | description, ERA ConOps, ERA RD, ERA IRD, ERA Use   |
|                | Cases, IEEE Std. 1471-2000, IEEE?EIA 12207.1,       |
|                | section 6.25, IEEE Std. 1012-1998, Table 1          |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Concept Activity  |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate System/Subsystem Design Descriptions

Evaluate the system design description for conformance to the applicable
standard in format, purpose, and content. Verify the correctness,
accuracy, and completeness of the system design description.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate the integration contractor(s) system       |
|                | design document                                     |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate the system/subsystem designs for           |
|                | conformance to J-STD-016-1995, section G.2.1.       |
|                | Insure the proposed approach is consistent with     |
|                | NARA needs. Verify the criteria described meet the  |
|                | requirements of IEEE Std. 1012-1998, Table 1.       |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) SyRS, system architecture and     |
|                | system/subsystem design documents, ERA concept      |
|                | documents, ERA RD, J-STD-016-1995, section G.2.1,   |
|                | IEEE Std. 1012-1998, Table 1                        |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Concept Activity  |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate Allocation of Requirements

Verify the correctness, accuracy, and completeness of the requirements
allocation to hardware, software, networking resources, databases, and
user and external interfaces in the context of known user needs.
Requirements allocation to the various subsystems that comprise the
overall system is usually described in a SARAD.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate the ERA contractor(s) Requirements         |
|                | Allocation Description                              |
+----------------+-----------------------------------------------------+
| Methods:       | This analysis is a manual activity to ensure that   |
|                | all system requirements have been allocated and to  |
|                | insure the correctness, accuracy, and completeness  |
|                | of the allocation in the context of NARA needs.     |
|                | Verify that the requirements allocation description |
|                | meets that part of the content requirements         |
|                | specified in IEEE/EIA 12207.1, section 6.25, for    |
|                | the System Architecture and Requirements Allocation |
|                | Description. Verify that the criteria described in  |
|                | IEEE Std. 1012-1998, Table 1 are satisfied.         |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) requirements allocation           |
|                | description, ERA contractor(s) concept documents,   |
|                | ERA contractor(s) system architecture description,  |
|                | ERA contractor(s) system/subsystem design           |
|                | descriptions, IEEE/EIA 12207.1-1997, section 6.25,  |
|                | IEEE Std. 1012-1998, Table 1                        |
+----------------+-----------------------------------------------------+
| Outputs:       | The accuracy of IV&V's assessment depends on the    |
|                | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate Plans

Management plans (e.g., project management plan, configuration
management plan, etc.) may be developed by the ERA contractor(s). Each
plan, to be effective as a complete specification of the tasks required
to achieve the objectives, must be thorough, comprehensive, and
specific. Each plan will be reviewed against the appropriate standard in
format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Review ERA contractor(s) plans when directed by the |
|                | ERA PD or his designated representative. Review     |
|                | plans generated by NARA.                            |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate plans to insure conformance to applicable  |
|                | standards and to insure they are complete,          |
|                | consistent, concise, comprehensive, specific, and   |
|                | unambiguous. Documents will be evaluated to ensure  |
|                | they are internally and externally consistent with  |
|                | the governing documents, applicable standards, and  |
|                | with each other.                                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Project Management Plan, Configuration Management   |
|                | Plan, Cut-over Plan, Quality Assurance Plan,        |
|                | Maintenance Plan, and any others specified in the   |
|                | contractor(s) Statement of Objectives (SOO) or      |
|                | similar attachment, applicable consensus standards. |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report for each document reviewed, input  |
|                | to End of Concept Activity Summary Report, Anomaly  |
|                | Reports                                             |
+----------------+-----------------------------------------------------+
| Schedules:     | Q2/Q3 2004                                          |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document. Subsequent    |
|                | evaluations take place each time a modified         |
|                | document or a request for change becomes available. |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | planning documents.                                 |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform (Requirements) Traceability Analysis

Verify that the allocated requirements documented in the various
subsystem, component, database, and interface requirements sections of
the requirements allocation description are traceable to viable
requirements documented in the SyRS and to program-level requirements.
Verify that all viable requirements from the SyRS have been correctly
and completely allocated to the various subsystem, component, database
and interface requirements components described in the requirements
description.

+----------------+-----------------------------------------------------+
| Task:          | Trace the ERA contractor(s) allocation of           |
|                | requirements to insure all system requirements have |
|                | been allocated and no new requirements have been    |
|                | introduced                                          |
+----------------+-----------------------------------------------------+
| Methods:       | Trace the system requirements to the allocated      |
|                | requirements. Trace the allocated requirements to   |
|                | the system requirements and to the program-level    |
|                | requirements. Verify that the criteria described in |
|                | IEEE Std. 1012, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) SyRS and requirements allocation  |
|                | description, program-level requirements documents.  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Concept Activity  |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Criticality Analysis

Determine whether or not to modify integrity levels as established for
requirements, detailed functions, modules, subsystems, or other system
partitions.

+----------------+-----------------------------------------------------+
| Task:          | Re-assess ERA Integrity Level.                      |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that the assigned integrity level(s) are     |
|                | correct. Verify that the most critical level        |
|                | assigned to an individual element is assigned to    |
|                | the entire system.                                  |
+----------------+-----------------------------------------------------+
| Inputs:        | Prior IV&V Task Report(s)---Criticality Analysis.   |
+----------------+-----------------------------------------------------+
| Outputs:       | Identification of revised ERA Integrity Level, IV&V |
|                | Task Report input to End of Concept Activity        |
|                | Summary Report, Updated IV&V Plan.                  |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of activity.                                        |
+----------------+-----------------------------------------------------+
| Resources:     | ERA PD, ERA COR, ERA POST Team Leads, IV&V          |
|                | Technical Director, IV&V SMEs.                      |
+----------------+-----------------------------------------------------+
| Risks and      | Delay in identifying integrity level has negative   |
| Assumptions:   | impact on the scoping of the IV&V effort and the    |
|                | overall IV&V schedule. IV&V will support Government |
|                | decision making process. The assumption is that the |
|                | Government will make the criticality decision in a  |
|                | timely manner.                                      |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**.                           |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget and schedule risks identified by IV&V
are correctly reviewed and tracked.

+----------------+-----------------------------------------------------+
| Task:          | Identify technical and management risks.            |
+----------------+-----------------------------------------------------+
| Methods:       | Documents being reviewed for conformance to user    |
|                | needs and applicable standards also will be         |
|                | reviewed for risk to program budget and schedule.   |
|                | Other risks will be noted during assessments of     |
|                | processes (CM Assessment, RM Assessment) or during  |
|                | technical interchange meetings, reviews, and        |
|                | audits. All identified risks will be entered in the |
|                | ERA Risk Management data base through the process   |
|                | described in the ERA Risk Management Plan.          |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA PMO concept documentation, integration          |
|                | contractor(s) concept documentation, architectural  |
|                | design, allocation of requirements, IV&V Task       |
|                | Reports.                                            |
+----------------+-----------------------------------------------------+
| Outputs:       | Input to ERA Risk Management Program, IV&V Task     |
|                | Report, input to End of Concept Activity Summary    |
|                | Report, Anomaly Reports.                            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Identified risks submitted to ERA Risk Management   |
|                | Program for reporting and tracking.                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Risks identified by IV&V may not be accepted or     |
| Assumptions:   | managed expeditiously by the ERA Risk Management    |
|                | Team. Should this occur IV&V will maintain its own  |
|                | risk database and will issue its own risk reports.  |
|                | The assumption is that the ERA Risk Management Team |
|                | will accept and manage IV&V identified risks if     |
|                | there is a perceived failure in the ERA Risk        |
|                | Management System.                                  |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**.                           |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

### Requirements Activity

The requirements activity defines and identifies the requirements
including functional and performance requirements, external interfaces,
security requirements, data definitions, installation and acceptance
requirements, user operation and execution requirements, and user
maintenance requirements. The IV&V activity addresses requirements
analysis. The objectives of IV&V are to ensure the correctness,
completeness, accuracy, testability, and consistency of the ERA
requirements.

The specific IV&V requirements activity tasks include:

-   Perform traceability analysis;

-   Perform software/hardware requirements evaluation;

-   Perform interface requirements evaluation;

-   Perform criticality analysis;

-   Verify ERA contractor(s) system test plan;

-   Verify ERA PMO acceptance test plan;

-   Generate IV&V system test plan;

-   Generate IV&V acceptance test plan;

-   Perform configuration management assessment; and

-   Perform risk analysis.

#### Perform Traceability (Requirements) Analysis

Traceability analysis insures that the system requirements have been
allocated correctly to software, hardware, networking resources,
databases and interfaces. It also ensures that no new requirements have
been introduced.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate developer(s) software requirements and     |
|                | interface requirements to the system requirements   |
|                | to insure that all requirements have been allocated |
|                | and no new requirements have been introduced.       |
|                | Perform Traceability (Requirements) Analysis        |
+----------------+-----------------------------------------------------+
| Methods:       | Assess traceability from software, hardware,        |
|                | networking, and interface requirements to the SyRS  |
|                | and from the SyRS to software, hardware,            |
|                | networking, and interface requirements. Verify that |
|                | the criteria described in IEEE Std. 1012-1998,      |
|                | Table 1 are satisfied                               |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) SyRS, Software Requirements       |
|                | Specifications (SRSs), Interface Requirements       |
|                | Specifications (IRSs), database requirements        |
|                | specifications, network component requirements      |
|                | specifications, information security requirements   |
|                | specifications, any other subsystem or component    |
|                | requirements specifications, IEEE Std. 1012-1998,   |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Software/Hardware Requirements Evaluation

Evaluate the software/hardware requirements for conformance to the
applicable standard in format, purpose, and content. Requirements
evaluation insures that the hardware and software requirements are
correct, consistent, complete, accurate, readable, and testable.

+----------------+-----------------------------------------------------+
| Task:          | Perform Software/Hardware Requirements Evaluation   |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate allocated requirements for conformance to  |
|                | IEEE Std. 830-1998, IEEE/EIA 12207.1,               |
|                | J-STD-016-1995 and to insure they are correct,      |
|                | consistent, complete, accurate, readable, verify    |
|                | the criteria described in IEEE Std. 1012-1998,      |
|                | Table 1 are satisfied                               |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) subsystem and component           |
|                | requirements specifications, IEEE Std. 830-1998,    |
|                | IEEE/EIA 12207.1-1997, J_STD-016-1995, IEEE Std.    |
|                | 1012-1998, Table 1                                  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Interface Requirements Evaluation

Evaluate the IRSs for conformance to J-STD-016-1995, Section F.2.3, for
purpose, format, and content. Interface requirements evaluation insures
that the interface requirements are correct, consistent, complete,
accurate, readable, and testable.

+----------------+-----------------------------------------------------+
| Task:          | Perform Interface Requirements Evaluation           |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate ERA contractor(s) IRSs for conformance to  |
|                | J-STD-016-1995, section F.2.3, and to insure they   |
|                | are correct, consistent, complete, accurate and     |
|                | testable, verify the criteria described in IEEE     |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | Integration contractor(s) software interface        |
|                | requirements, J-STD-016-1995, section F.2.3, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Criticality Analysis

Determine whether or not correct integrity levels are established for
requirements, modules, subsystems, or other system partitions.

+---+---------------+---+-----------------------------------------------+---+
| T |               | R |                                               |   |
| a |               | e |                                               |   |
| s |               | - |                                               |   |
| k |               | a |                                               |   |
| : |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | I |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | L |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| M |               | V |                                               |   |
| e |               | e |                                               |   |
| t |               | r |                                               |   |
| h |               | i |                                               |   |
| o |               | f |                                               |   |
| d |               | y |                                               |   |
| s |               | t |                                               |   |
| : |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | g |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | ( |                                               |   |
|   |               | s |                                               |   |
|   |               | ) |                                               |   |
|   |               | a |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | . |                                               |   |
|   |               | V |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | y |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | o |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | c |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | g |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | d |                                               |   |
|   |               | u |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | g |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | y |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| I |               | P |                                               |   |
| n |               | r |                                               |   |
| p |               | i |                                               |   |
| u |               | o |                                               |   |
| t |               | r |                                               |   |
| s |               | I |                                               |   |
| : |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | T |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | ( |                                               |   |
|   |               | s |                                               |   |
|   |               | ) |                                               |   |
|   |               | - |                                               |   |
|   |               | - |                                               |   |
|   |               | - |                                               |   |
|   |               | C |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | A |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | y |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| O |               | I |                                               |   |
| u |               | d |                                               |   |
| t |               | e |                                               |   |
| p |               | n |                                               |   |
| u |               | t |                                               |   |
| t |               | i |                                               |   |
| s |               | f |                                               |   |
| : |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | I |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | L |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | , |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | T |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | p |                                               |   |
|   |               | u |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | E |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | A |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | S |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | r |                                               |   |
|   |               | y |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | , |                                               |   |
|   |               | U |                                               |   |
|   |               | p |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | P |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| S |               | Q |                                               |   |
| c |               | 2 |                                               |   |
| h |               | 2 |                                               |   |
| e |               | 0 |                                               |   |
| d |               | 0 |                                               |   |
| u |               | 4 |                                               |   |
| l |               |   |                                               |   |
| e |               | R |                                               |   |
| : |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | b |                                               |   |
|   |               | m |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | G |                                               |   |
|   |               | o |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | n |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | u |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | m |                                               |   |
|   |               | p |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | E |                                               |   |
| e |               | R |                                               |   |
| s |               | A |                                               |   |
| o |               | P |                                               |   |
| u |               | D |                                               |   |
| r |               | , |                                               |   |
| c |               | E |                                               |   |
| e |               | R |                                               |   |
| s |               | A |                                               |   |
| : |               | C |                                               |   |
|   |               | O |                                               |   |
|   |               | R |                                               |   |
|   |               | , |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | P |                                               |   |
|   |               | O |                                               |   |
|   |               | S |                                               |   |
|   |               | T |                                               |   |
|   |               | T |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | L |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | d |                                               |   |
|   |               | s |                                               |   |
|   |               | , |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | T |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | D |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | , |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | S |                                               |   |
|   |               | M |                                               |   |
|   |               | E |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | D |                                               |   |
| i |               | e |                                               |   |
| s |               | l |                                               |   |
| k |               | a |                                               |   |
| s |               | y |                                               |   |
| a |               | i |                                               |   |
| n |               | n |                                               |   |
| d |               | i |                                               |   |
| A |               | d |                                               |   |
| s |               | e |                                               |   |
| s |               | n |                                               |   |
| u |               | t |                                               |   |
| m |               | i |                                               |   |
| p |               | f |                                               |   |
| t |               | y |                                               |   |
| i |               | i |                                               |   |
| o |               | n |                                               |   |
| n |               | g |                                               |   |
| s |               | i |                                               |   |
| : |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | g |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | m |                                               |   |
|   |               | p |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | p |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | e |                                               |   |
|   |               | f |                                               |   |
|   |               | f |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | o |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | s |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | u |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | . |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | p |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | G |                                               |   |
|   |               | o |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | n |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | k |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | p |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
|   |               | T |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | p |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | G |                                               |   |
|   |               | o |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | n |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | k |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | y |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | R |                                               |   |
| o |               | e |                                               |   |
| l |               | f |                                               |   |
| e |               | e |                                               |   |
| s |               | r |                                               |   |
| a |               | t |                                               |   |
| n |               | o |                                               |   |
| d |               | * |                                               |   |
| R |               | * |                                               |   |
| e |               | S |                                               |   |
| s |               | e |                                               |   |
| p |               | c |                                               |   |
| o |               | t |                                               |   |
| n |               | i |                                               |   |
| s |               | o |                                               |   |
| i |               | n |                                               |   |
| b |               | 4 |                                               |   |
| i |               | . |                                               |   |
| l |               | 5 |                                               |   |
| i |               | * |                                               |   |
| t |               | * |                                               |   |
| i |               | . |                                               |   |
| e |               |   |                                               |   |
| s |               |   |                                               |   |
| : |               |   |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Task:         |   | Re-assess ERA Integrity Level.                |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Methods:      |   | Verify that the assigned integrity level(s)   |   |
|   |               |   | are correct. Verify that the most critical    |   |
|   |               |   | level assigned to an individual element is    |   |
|   |               |   | assigned to the entire system.                |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Inputs:       |   | Prior IV&V Task Report(s)---Criticality       |   |
|   |               |   | Analysis.                                     |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Outputs:      |   | Identification of revised ERA Integrity       |   |
|   |               |   | Level, IV&V Task Report input to End of       |   |
|   |               |   | Concept Activity Summary Report, Updated IV&V |   |
|   |               |   | Plan.                                         |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Schedule:     |   | Q2 2004                                       |   |
|   |               |   |                                               |   |
|   |               |   | Report submitted to the Government upon       |   |
|   |               |   | completion of activity.                       |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Resources:    |   | ERA PD, ERA COR, ERA POST Team Leads, IV&V    |   |
|   |               |   | Technical Director, IV&V SMEs.                |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Risks and     |   | Delay in identifying integrity level has      |   |
|   | Assumptions:  |   | negative impact on the scoping of the IV&V    |   |
|   |               |   | effort and the overall IV&V schedule. IV&V    |   |
|   |               |   | will support Government decision making       |   |
|   |               |   | process. The assumption is that the           |   |
|   |               |   | Government will make the criticality decision |   |
|   |               |   | in a timely manner.                           |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Roles and     |   | Refer to section 4.5.                         |   |
|   | Resp          |   |                                               |   |
|   | onsibilities: |   |                                               |   |
+---+---------------+---+-----------------------------------------------+---+

#### Verify ERA Contractor(s) System Test Plan 

Evaluate the system test plan for conformance to the applicable standard
for purpose, format, and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify the integration contractor(s) System Test    |
|                | Plan to insure that all requirements will be tested |
+----------------+-----------------------------------------------------+
| Methods:       | Verify System Test Plan conformance to IEEE Std.    |
|                | 829-1998 in format, purpose, and content. Verify    |
|                | that it includes a test traceability matrix         |
|                | (traceability to specific requirements in the SyRS) |
|                | is included, verify the criteria described in IEEE  |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs;        | ERA contractor(s) System Test Plan document, SyRS,  |
|                | IEEE Std. 829-1998, IEEE Std. 1012-1998, Table 1    |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q3/Q4 2004                                          |
|                |                                                     |
|                | Report submitted to the Government within 7         |
|                | business days of completion of task                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA PMO Acceptance Test Plan 

Evaluate the Acceptance Test Plan for conformance to the applicable
standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify the ERA PMO Acceptance Test Plan             |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that ERA PMO Acceptance Test Plan conforms   |
|                | to IEEE Std. 829-1998 in format, purpose, and       |
|                | content. Verify that tests are traceable to         |
|                | specific use cases and program-level requirements.  |
|                | Verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied.                   |
+----------------+-----------------------------------------------------+
| Inputs;        | ERA PMO Acceptance Test Plan, program-level         |
|                | requirements documents, IEEE Std. 829-1998, IEEE    |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q3 2004                                             |
|                |                                                     |
|                | Report submitted to the Government within 7         |
|                | business days of completion of task.                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Generate IV&V System Test Plan 

Generate the IV&V System Test Plan as described in **Section 8.0**.
Ensure conformance to the applicable standard in format, purpose, and
content. **NOTE:** *The scope of IV&V testing will be determined by the
ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V System Test Plan                      |
+----------------+-----------------------------------------------------+
| Methods:       | Produce an abbreviated IV&V System Test Plan that   |
|                | conforms to IEEE Std. 829-1998 and insures test     |
|                | coverage of system requirements, appropriateness of |
|                | test methods and standards used, conformance to     |
|                | expected results, feasibility of system             |
|                | qualification testing, and feasibility and          |
|                | testability of operation and maintenance            |
|                | requirements. Use statistical sampling for          |
|                | determining which SyRS requirements to test.        |
|                | Analyze the results and project to the population.  |
|                | Verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | SyRS, deficiencies noted in integration             |
|                | contractor(s) system test plan, IEEE Std. 829-1998, |
|                | IEEE Std. 1012-1998, Table 1, Program-level         |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V System Test Plan, input to End of Requirements |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q3/Q4 2004                                          |
|                |                                                     |
|                | Plan submitted to the Government upon completion of |
|                | task                                                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's test plan depends on the     |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Generate IV&V Acceptance Test Plan 

Generate the IV&V Acceptance Test Plan as described in **Section 8.0**.
Ensure conformance to the applicable standard in format, purpose, and
content. **NOTE:** *The scope of IV&V testing will be determined by the
ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V Acceptance Test Plan                  |
+----------------+-----------------------------------------------------+
| Methods:       | Produce an abbreviated IV&V Acceptance Test Plan    |
|                | that conforms to IEEE Std. 829-1998 and will        |
|                | demonstrate that ERA will be operated and           |
|                | maintained in accordance with user needs. Using     |
|                | statistical sampling and analysis techniques,       |
|                | verify all operations, including system             |
|                | administration and maintenance, are performed in    |
|                | accordance with the respective documentation to     |
|                | insure the completeness, correctness, and           |
|                | consistency of the user documentation. Verify the   |
|                | criteria described in IEEE Std. 1012-1998, Table 1  |
|                | are satisfied                                       |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA Use Cases, ERA User documentation, ERA CONOPS,  |
|                | program-level requirements documentation, IEEE Std. |
|                | 829-1998, IEEE Std. 1012-1998, Table 1              |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Acceptance Test Plan, input to End of          |
|                | Requirements Activity Summary Report, Anomaly       |
|                | Reports                                             |
+----------------+-----------------------------------------------------+
| Schedule:      | Q3 2004                                             |
|                |                                                     |
|                | Plan submitted to the Government within seven (7)   |
|                | business days of upon completion of task            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The accuracy of IV&V's assessment depends on the    |
| Assumptions:   | accuracy and completeness of the ERA program-level  |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Configuration Management Assessment

Verify that there is a process for describing product functionality,
tracking system program versions and managing changes. Verify that the
configuration management process is adequate for the development size
and complexity, integrity level, plans and user needs.

+----------------+-----------------------------------------------------+
| Task:          | Perform Configuration Management Assessment         |
+----------------+-----------------------------------------------------+
| Methods:       | Verify ERA contractor and ERA PMO configuration     |
|                | management processes are consistent (internally and |
|                | with each other), complete and adequate. Use        |
|                | criteria from SA/SW-CMM Level 3                     |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) and ERA PMO configuration         |
|                | management plans, configuration management          |
|                | processes and procedures, other configuration       |
|                | management artifacts, SEI CMM for SA and SW         |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Requirements      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | Q2/Q3 2004                                          |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task                                             |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget, and schedule risks identified by IV&V
are correctly reviewed and tracked.

+---+---------------+---+-----------------------------------------------+---+
| T |               | I |                                               |   |
| a |               | d |                                               |   |
| s |               | e |                                               |   |
| k |               | n |                                               |   |
| : |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | y |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| M |               | D |                                               |   |
| e |               | o |                                               |   |
| t |               | c |                                               |   |
| h |               | u |                                               |   |
| o |               | m |                                               |   |
| d |               | e |                                               |   |
| s |               | n |                                               |   |
| : |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | w |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | f |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | f |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | u |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | p |                                               |   |
|   |               | p |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | b |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | r |                                               |   |
|   |               | d |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | s |                                               |   |
|   |               | o |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | w |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | f |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | p |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | b |                                               |   |
|   |               | u |                                               |   |
|   |               | d |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | s |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | u |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | . |                                               |   |
|   |               | O |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | s |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | d |                                               |   |
|   |               | u |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | p |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | ( |                                               |   |
|   |               | C |                                               |   |
|   |               | M |                                               |   |
|   |               | A |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | , |                                               |   |
|   |               | R |                                               |   |
|   |               | M |                                               |   |
|   |               | A |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | ) |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | d |                                               |   |
|   |               | u |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | s |                                               |   |
|   |               | , |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | w |                                               |   |
|   |               | s |                                               |   |
|   |               | , |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | u |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
|   |               | A |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | s |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | b |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | u |                                               |   |
|   |               | g |                                               |   |
|   |               | h |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | c |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | P |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| I |               | E |                                               |   |
| n |               | R |                                               |   |
| p |               | A |                                               |   |
| u |               | P |                                               |   |
| t |               | M |                                               |   |
| s |               | O |                                               |   |
| : |               | p |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | - |                                               |   |
|   |               | l |                                               |   |
|   |               | e |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | l |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | d |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | , |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | ( |                                               |   |
|   |               | s |                                               |   |
|   |               | ) |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | s |                                               |   |
|   |               | y |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | d |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | , |                                               |   |
|   |               | a |                                               |   |
|   |               | r |                                               |   |
|   |               | c |                                               |   |
|   |               | h |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | u |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | g |                                               |   |
|   |               | n |                                               |   |
|   |               | , |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | , |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | b |                                               |   |
|   |               | s |                                               |   |
|   |               | y |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | c |                                               |   |
|   |               | o |                                               |   |
|   |               | m |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | p |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | i |                                               |   |
|   |               | c |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | s |                                               |   |
|   |               | , |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | T |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | p |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| O |               | I |                                               |   |
| u |               | n |                                               |   |
| t |               | p |                                               |   |
| p |               | u |                                               |   |
| u |               | t |                                               |   |
| t |               | t |                                               |   |
| s |               | o |                                               |   |
| : |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | P |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | , |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | T |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | , |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | p |                                               |   |
|   |               | u |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | E |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | o |                                               |   |
|   |               | f |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | q |                                               |   |
|   |               | u |                                               |   |
|   |               | i |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | A |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | y |                                               |   |
|   |               | S |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | r |                                               |   |
|   |               | y |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | , |                                               |   |
|   |               | A |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | l |                                               |   |
|   |               | y |                                               |   |
|   |               | R |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| S |               | Q |                                               |   |
| c |               | 4 |                                               |   |
| h |               | 2 |                                               |   |
| e |               | 0 |                                               |   |
| d |               | 0 |                                               |   |
| u |               | 4 |                                               |   |
| l |               |   |                                               |   |
| e |               | I |                                               |   |
| : |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | b |                                               |   |
|   |               | m |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | o |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | P |                                               |   |
|   |               | r |                                               |   |
|   |               | o |                                               |   |
|   |               | g |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | f |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | r |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | k |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | g |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | R |                                               |   |
| e |               | e |                                               |   |
| s |               | f |                                               |   |
| o |               | e |                                               |   |
| u |               | r |                                               |   |
| r |               | t |                                               |   |
| c |               | o |                                               |   |
| e |               | * |                                               |   |
| s |               | * |                                               |   |
| : |               | S |                                               |   |
|   |               | e |                                               |   |
|   |               | c |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | 4 |                                               |   |
|   |               | . |                                               |   |
|   |               | 4 |                                               |   |
|   |               | * |                                               |   |
|   |               | * |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | R |                                               |   |
| i |               | i |                                               |   |
| s |               | s |                                               |   |
| k |               | k |                                               |   |
| s |               | s |                                               |   |
| a |               | i |                                               |   |
| n |               | d |                                               |   |
| d |               | e |                                               |   |
| A |               | n |                                               |   |
| s |               | t |                                               |   |
| s |               | i |                                               |   |
| u |               | f |                                               |   |
| m |               | i |                                               |   |
| p |               | e |                                               |   |
| t |               | d |                                               |   |
| i |               | b |                                               |   |
| o |               | y |                                               |   |
| n |               | I |                                               |   |
| s |               | V |                                               |   |
| : |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | y |                                               |   |
|   |               | n |                                               |   |
|   |               | o |                                               |   |
|   |               | t |                                               |   |
|   |               | b |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | x |                                               |   |
|   |               | p |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | u |                                               |   |
|   |               | s |                                               |   |
|   |               | l |                                               |   |
|   |               | y |                                               |   |
|   |               | b |                                               |   |
|   |               | y |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | T |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | . |                                               |   |
|   |               | S |                                               |   |
|   |               | h |                                               |   |
|   |               | o |                                               |   |
|   |               | u |                                               |   |
|   |               | l |                                               |   |
|   |               | d |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | o |                                               |   |
|   |               | c |                                               |   |
|   |               | c |                                               |   |
|   |               | u |                                               |   |
|   |               | r |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | o |                                               |   |
|   |               | w |                                               |   |
|   |               | n |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | d |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | b |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | o |                                               |   |
|   |               | w |                                               |   |
|   |               | n |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | o |                                               |   |
|   |               | r |                                               |   |
|   |               | t |                                               |   |
|   |               | s |                                               |   |
|   |               | . |                                               |   |
|   |               | T |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | s |                                               |   |
|   |               | s |                                               |   |
|   |               | u |                                               |   |
|   |               | m |                                               |   |
|   |               | p |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | o |                                               |   |
|   |               | n |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | a |                                               |   |
|   |               | t |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | T |                                               |   |
|   |               | e |                                               |   |
|   |               | a |                                               |   |
|   |               | m |                                               |   |
|   |               | w |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | l |                                               |   |
|   |               | a |                                               |   |
|   |               | c |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | p |                                               |   |
|   |               | t |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | d |                                               |   |
|   |               | m |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | I |                                               |   |
|   |               | V |                                               |   |
|   |               | & |                                               |   |
|   |               | V |                                               |   |
|   |               | i |                                               |   |
|   |               | d |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | i |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | r |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | s |                                               |   |
|   |               | i |                                               |   |
|   |               | f |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | a |                                               |   |
|   |               | p |                                               |   |
|   |               | e |                                               |   |
|   |               | r |                                               |   |
|   |               | c |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | v |                                               |   |
|   |               | e |                                               |   |
|   |               | d |                                               |   |
|   |               | f |                                               |   |
|   |               | a |                                               |   |
|   |               | i |                                               |   |
|   |               | l |                                               |   |
|   |               | u |                                               |   |
|   |               | r |                                               |   |
|   |               | e |                                               |   |
|   |               | i |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | h |                                               |   |
|   |               | e |                                               |   |
|   |               | E |                                               |   |
|   |               | R |                                               |   |
|   |               | A |                                               |   |
|   |               | R |                                               |   |
|   |               | i |                                               |   |
|   |               | s |                                               |   |
|   |               | k |                                               |   |
|   |               | M |                                               |   |
|   |               | a |                                               |   |
|   |               | n |                                               |   |
|   |               | a |                                               |   |
|   |               | g |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | e |                                               |   |
|   |               | n |                                               |   |
|   |               | t |                                               |   |
|   |               | S |                                               |   |
|   |               | y |                                               |   |
|   |               | s |                                               |   |
|   |               | t |                                               |   |
|   |               | e |                                               |   |
|   |               | m |                                               |   |
|   |               | . |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
| R |               | R |                                               |   |
| o |               | e |                                               |   |
| l |               | f |                                               |   |
| e |               | e |                                               |   |
| s |               | r |                                               |   |
| a |               | t |                                               |   |
| n |               | o |                                               |   |
| d |               | * |                                               |   |
| R |               | * |                                               |   |
| e |               | S |                                               |   |
| s |               | e |                                               |   |
| p |               | c |                                               |   |
| o |               | t |                                               |   |
| n |               | i |                                               |   |
| s |               | o |                                               |   |
| i |               | n |                                               |   |
| b |               | 4 |                                               |   |
| i |               | . |                                               |   |
| l |               | 5 |                                               |   |
| i |               | * |                                               |   |
| t |               | * |                                               |   |
| i |               | . |                                               |   |
| e |               |   |                                               |   |
| s |               |   |                                               |   |
| : |               |   |                                               |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Task:         |   | Identify technical and management risks.      |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Methods:      |   | Documents being reviewed for conformance to   |   |
|   |               |   | user needs and applicable standards also will |   |
|   |               |   | be reviewed for risk to program budget and    |   |
|   |               |   | schedule. Other risks will be noted during    |   |
|   |               |   | assessments of processes (CM Assessment, RM   |   |
|   |               |   | Assessment) or during technical interchange   |   |
|   |               |   | meetings, reviews, and audits. All identified |   |
|   |               |   | risks will be entered in the ERA Risk         |   |
|   |               |   | Management data base through the process      |   |
|   |               |   | described in the ERA Risk Management Plan.    |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Inputs:       |   | ERA PMO concept documentation, integration    |   |
|   |               |   | contractor(s) concept documentation,          |   |
|   |               |   | architectural design, allocation of           |   |
|   |               |   | requirements, IV&V Task Reports.              |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Outputs:      |   | Input to ERA Risk Management Program, IV&V    |   |
|   |               |   | Task Report, input to End of Concept Activity |   |
|   |               |   | Summary Report, Anomaly Reports.              |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Schedule:     |   | Q2 2004                                       |   |
|   |               |   |                                               |   |
|   |               |   | Identified risks submitted to ERA Risk        |   |
|   |               |   | Management Program for reporting and          |   |
|   |               |   | tracking.                                     |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Resources:    |   | Refer to section 4.4                          |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Risks and     |   | Risks identified by IV&V may not be accepted  |   |
|   | Assumptions:  |   | or managed expeditiously by the ERA Risk      |   |
|   |               |   | Management Team. Should this occur IV&V will  |   |
|   |               |   | maintain its own risk database and will issue |   |
|   |               |   | its own risk reports. The assumption is that  |   |
|   |               |   | the ERA Risk Management Team will accept and  |   |
|   |               |   | manage IV&V identified risks if there is a    |   |
|   |               |   | perceived failure in the ERA Risk Management  |   |
|   |               |   | System.                                       |   |
+---+---------------+---+-----------------------------------------------+---+
|   | Roles and     |   | Refer to section 4.5.                         |   |
|   | Resp          |   |                                               |   |
|   | onsibilities: |   |                                               |   |
+---+---------------+---+-----------------------------------------------+---+

### Design Activity

In the Design activity, system requirements are transformed into an
architecture and detailed design for each component. The design activity
includes hardware, networking components, software, databases, Web
components, and interfaces (internal and external). The objectives of
IV&V are to demonstrate that the design is a correct, accurate, and
complete transformation of the requirements and that no unintended
features are introduced. These tasks are performed iteratively to
accommodate the incremental development and delivery process that
defines the ERA life cycle. During this activity IV&V typically verifies
the component and component integration test documentation. However,
IV&V will neither be verifying the development contractor(s) component
test plans, nor developing IV&V integration test documents.

Specific IV&V design activity tasks include:

-   Perform Preliminary Design Description evaluation and traceability
    analysis,

-   Perform Detailed Design Description traceability analysis,

-   Perform Detailed Design Description evaluation,

-   Perform Interface Design Description Analyses,

-   Verify ERA contractor(s) Integration Test Plan,

-   Verify ERA contractor(s) Integration Test Design Specifications,

-   Verify ERA contractor (s) System Test Design Specifications,

-   Verify ERA PMO Acceptance Test Design Specifications,

-   Generate IV&V Test Design Specifications (system, acceptance),

-   Perform Risk Analysis, and

```{=html}
<!-- -->
```
-   Perform Criticality Analysis.

#### Perform Preliminary Design Description Evaluation and Traceability Analysis

Evaluate the ERA contractor(s) preliminary designs for correctness,
consistency, and completeness. Analyze design implementation coverage of
the subsystem and component requirements.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate developer(s) Detailed Design Traceability. |
|                | Perform Preliminary Design Description Evaluation   |
|                | and Traceability Analysis                           |
+----------------+-----------------------------------------------------+
| Methods:       | Assess traceability from allocated requirements to  |
|                | preliminary design and from preliminary design to   |
|                | allocated requirements. Analyze requirements        |
|                | to/from preliminary design relationships for        |
|                | correctness, consistency, and completeness. Verify  |
|                | the criteria described in IEEE Std. 1012-1998,      |
|                | Table 1 are satisfied. Use statistical sampling and |
|                | analysis techniques.                                |
+----------------+-----------------------------------------------------+
| Inputs:        | Subsystem and component requirements                |
|                | specifications, subsystem and component preliminary |
|                | design descriptions, IEEE Std. 1012-1998, Table 1   |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Quick Look Report, input to End of Design      |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within five      |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Detailed Design Description Traceability Analysis

Evaluate requirements to design relationships for correctness,
consistency, and completeness according to the applicable standard.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate developer(s) Detailed Design Traceability. |
|                | Perform Detailed Design Description Traceability    |
|                | Analysis                                            |
+----------------+-----------------------------------------------------+
| Methods:       | Assess traceability from allocated requirements     |
|                | named in the various subsystem and component        |
|                | requirements specifications to detailed design      |
|                | descriptions and from detailed design descriptions  |
|                | to allocated requirements specifications. Analyze   |
|                | requirements to/from detailed design relationships  |
|                | for correctness, consistency, and completeness.     |
|                | Verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Requirements allocation description, subsystem and  |
|                | component requirements specifications, detailed     |
|                | design document, IEEE Std. 1012-1998, Table 1       |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Design Activity   |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Detailed Design Description Evaluation

Evaluate the detailed design elements for conformance to the applicable
standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Perform Detailed Design Evaluation                  |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate for conformance to IEEE Std. 1016-1998 and |
|                | J-STD-016-1995 and to ensure that the design is     |
|                | correct, consistent, complete, accurate, readable   |
|                | and testable. Verify the criteria described in IEEE |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) Detailed Design Descriptions,     |
|                | subsystem and component requirements                |
|                | specifications, J-STD-016-1995, IEEE Std.           |
|                | 1016-1998, IEEE Std. 1012-1998, Table 1             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Design Activity   |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Interface Design Description Analyses

Evaluate the design interfaces according to the requirements of the
applicable standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Perform Interface Design Description (IDD) Analysis |
+----------------+-----------------------------------------------------+
| Methods:       | Evaluate the IDDs for conformance to                |
|                | J-STD-016-1995, section G.2.2, and to insure they   |
|                | are correct, consistent, complete, accurate and     |
|                | testable, verify the criteria described in IEEE     |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | Integration contractor(s) software interface        |
|                | requirements, J-STD-016-1995, section G.2.2, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Design Activity   |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contactor(s) Integration Test Plan 

Evaluate the ERA contractor(s) integration test plan for conformance to
the applicable standard in format, purpose, and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify Integration Test Plan                        |
+----------------+-----------------------------------------------------+
| Methods:       | Verify for conformance to IEEE Std. 829-1998 in     |
|                | format, purpose, and content, validate that the     |
|                | requirements and design are correctly implemented   |
|                | as the system is incrementally integrated, verify   |
|                | the criteria described in IEEE Std. 1012-1998,      |
|                | Table 1 are satisfied                               |
+----------------+-----------------------------------------------------+
| Inputs:        | Integration contractor(s) integration test plan,    |
|                | subsystem and component requirements specifications |
|                | and design descriptions, IEEE Std. 829-1998, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Design Activity   |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document.               |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contractor(s) Integration Test Design Specifications

Verify that the ERA contractor(s) test design specifications for
conformance to the applicable standard in format, purpose, and content.
The test design specifications may not be stand-alone documents and may
be part of the test case specifications.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA contractor(s) integration test design    |
|                | specifications.                                     |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that the test designs correctly emanate from |
|                | the test plans and ensure conformance to IEEE Std.  |
|                | 829-1998 in format, content and purpose, verify the |
|                | criteria described in IEEE Std. 1012-1998, Table 1  |
|                | are satisfied                                       |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) integration test design           |
|                | specifications, ERA contractor(s) integration test  |
|                | plans, IEEE Std. 829-1998, IEEE Std. 1012-1998,     |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Design Activity   |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contractor(s) System Test Design Specifications

Verify that the test design conforms to the applicable standard in
purpose, format, and content. Verify that the test design traces to the
test plan. The system test design may be a stand-alone document or it
may be part of the test case document.

+------------+---------------------------------------------------------+
| Task:      | Verify ERA System Test Design Specification             |
+------------+---------------------------------------------------------+
| Methods:   | Verify that the test design correctly emanates from the |
|            | test plan and conforms to IEEE Std 829-1998 in format,  |
|            | content, and purpose; verify the criteria described in  |
|            | IEEE Std 1012-1998 are satisfied                        |
+------------+---------------------------------------------------------+
| Inputs:    | ERA system test design specification, ERA system test   |
|            | plan, IEEE Std 829-1998; IEEE Std 1012-1998, Table 1    |
+------------+---------------------------------------------------------+
| Outputs:   | IV&V Task Report; input to End of Design Activity       |
|            | Summary Report                                          |
+------------+---------------------------------------------------------+
| Schedule:  | TBD                                                     |
|            |                                                         |
|            | Report submitted to the Government within seven (7)     |
|            | business days of receipt of document                    |
+------------+---------------------------------------------------------+
| Resources: | Refer to **Section 4.4**                                |
+------------+---------------------------------------------------------+
| Risks and  | None                                                    |
| As         |                                                         |
| sumptions: |                                                         |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

#### Verify ERA PMO Acceptance Test Design Specifications

Verify that the test design conforms to the applicable standard in
purpose, format, and content. Verify that the test design traces to the
test plan. The acceptance test design may be a stand-alone document or
it may be part of the test case document.

+------------+---------------------------------------------------------+
| Task:      | Verify ERA PMO Acceptance Test Design Specification     |
+------------+---------------------------------------------------------+
| Methods:   | Verify that the test design correctly emanates from the |
|            | test plan and conforms to IEEE Std 829-1998 in format,  |
|            | content, and purpose; verify the criteria described in  |
|            | IEEE Std 1012-1998 are satisfied                        |
+------------+---------------------------------------------------------+
| Inputs:    | ERA acceptance test design specification, ERA           |
|            | acceptance test plan, IEEE Std 829-1998; IEEE Std       |
|            | 1012-1998, Table 1                                      |
+------------+---------------------------------------------------------+
| Outputs:   | IV&V Task Report; input to End of Design Activity       |
|            | Summary Report                                          |
+------------+---------------------------------------------------------+
| Schedule:  | TBD                                                     |
|            |                                                         |
|            | Report submitted to the Government within seven (7)     |
|            | business days of receipt of document                    |
+------------+---------------------------------------------------------+
| Resources: | Refer to **Section 4.4**                                |
+------------+---------------------------------------------------------+
| Risks and  | None                                                    |
| As         |                                                         |
| sumptions: |                                                         |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

#### Generate IV&V System and Acceptance Test Design Specifications 

Generate IV&V test designs and insure conformance to the applicable
standard in purpose, format, and content. Verify traceability to IV&V
test plan (system, acceptance). The test design documents will not be
stand-alone documents. They will, instead, be the first part of each
respective test case document. **NOTE:** *The scope of IV&V testing will
be determined by the ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V System and Acceptance Test Design     |
|                | Specifications                                      |
+----------------+-----------------------------------------------------+
| Methods:       | Ensure that the test designs correctly emanate from |
|                | the test plans and ensure conformance to IEEE Std.  |
|                | 829-1998 regarding purpose, format and content,     |
|                | verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Test Plans, IEEE Std. 829-1998, IEEE Std.           |
|                | 1012-1998, Table 1                                  |
+----------------+-----------------------------------------------------+
| Outputs:       | Input to End of Design Activity Report              |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Document submitted to the Government upon           |
|                | completion of task                                  |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The correctness of the IV&V documents depends on    |
| Assumptions:   | the accuracy and completeness of program-level      |
|                | requirements documents.                             |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget, and schedule risks identified by IV&V
are correctly reviewed and tracked.

+----------------+-----------------------------------------------------+
| Task:          | Identify technical and management risks.            |
+----------------+-----------------------------------------------------+
| Methods:       | Documents being reviewed for conformance to user    |
|                | needs and applicable standards also will be         |
|                | reviewed for risk to program budget and schedule.   |
|                | Other risks will be noted during assessments of     |
|                | processes (CM Assessment, RM Assessment) or during  |
|                | technical interchange meetings, reviews, and        |
|                | audits. All identified risks will be entered in the |
|                | ERA Risk Management data base through the process   |
|                | described in the ERA Risk Management Plan.          |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA PMO program-level requirements and acceptance   |
|                | test documentation, ERA contractor(s) requirements, |
|                | design, and integration and system test             |
|                | documentation, architectural design, allocation of  |
|                | requirements, subsystem and component requirements  |
|                | specifications, IV&V Task Reports and test plans.   |
+----------------+-----------------------------------------------------+
| Outputs:       | Input to ERA Risk Management Program, IV&V Task     |
|                | Report, input to End of Design Activity Summary     |
|                | Report, Anomaly Reports.                            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Identified risks submitted to ERA Risk Management   |
|                | Program for reporting and tracking.                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Risks identified by IV&V may not be accepted or     |
| Assumptions:   | managed expeditiously by the ERA Risk Management    |
|                | Team. Should this occur IV&V will maintain its own  |
|                | risk database and will issue its own risk reports.  |
|                | The assumption is that the ERA Risk Management Team |
|                | will accept and manage IV&V identified risks if     |
|                | there is a perceived failure in the ERA Risk        |
|                | Management System.                                  |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**.                           |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Criticality Analysis

Determine whether or not correct integrity levels are established for
requirements, modules, subsystems, or other system partitions.

+----------------+-----------------------------------------------------+
| Task:          | Re-assess ERA Integrity Level                       |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that the assigned integrity level(s) are     |
|                | correct. Verify that the most critical level        |
|                | assigned to an individual element is assigned to    |
|                | the entire system.                                  |
+----------------+-----------------------------------------------------+
| Inputs:        | Prior IV&V Task Report(s)---Criticality Analysis    |
+----------------+-----------------------------------------------------+
| Outputs:       | Identification of revised ERA Integrity Level,      |
|                | Updated IV&V Plan, IV&V Task Report, input to End   |
|                | of End of Design Activity Summary Report            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | ERA PD, ERA COR, IV&V Director, ERA POST Team       |
|                | Leads, IV&V SMEs                                    |
+----------------+-----------------------------------------------------+
| Risks and      | Delay in identifying integrity level has negative   |
| Assumptions:   | impact on the scoping of the IV&V effort and the    |
|                | overall IV&V schedule. IV&V will support the        |
|                | Government decision making process. The assumption  |
|                | is that the Government will make the criticality    |
|                | decision in a timely manner.                        |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

### Implementation Activity

The implementation activity transforms the design into code, database
structures, and related machine executable representations. The
implementation IV&V activity addresses software coding and testing. The
objectives of IV&V are to verify and validate that these transformations
are correct, accurate, and complete. During this activity, IV&V
typically verifies the component test cases and test procedures and
witnesses component test execution. However, IV&V will be neither
verifying the integration contractor(s) component test documentation,
witnessing integration contractor(s) component testing nor performing
IV&V component tests.

The specific IV&V implementation activity tasks include:

-   Perform traceability analysis,

-   Evaluate as-built documentation,

-   Perform interface analyses,

-   Verify ERA System Test Case Specifications,

-   Verify ERA System Test Procedure Specifications,

-   Verify ERA PMO Acceptance Test Case Specifications

-   Generate IV&V System and Acceptance Test Case Specifications,

-   Generate IV&V System Test Procedure Specifications,

-   Perform criticality analysis, and

-   Perform risk analysis.

#### Perform Traceability Analysis

Evaluate the identified relationships for correctness, consistency, and
completeness.

+----------------+-----------------------------------------------------+
| Task:          | Trace the ERA contractor(s) design execution        |
+----------------+-----------------------------------------------------+
| Methods        | Assess traceability from the detailed design        |
|                | descriptions to the executed product and from the   |
|                | executed product to the detailed design             |
|                | descriptions. Verify the criteria described in IEEE |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | System/Subsystem and Component Detailed Design      |
|                | Descriptions, product "as-built" documentation,     |
|                | IEEE Std. 1012-1998, Table 1                        |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Implementation    |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of artifacts.              |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Timeliness of the delivery of the "as-built"        |
| Assumptions:   | documentation is the primary concern. The IV&V      |
|                | evaluation cannot be produced in a timely manner if |
|                | the ERA contractor delays delivery of these         |
|                | documents. IV&V recommends that their analysis be   |
|                | considered a mandatory input for the System Test    |
|                | TRR as a way of mitigating any technical risk       |
|                | associated with a delay in the delivery of as-built |
|                | documentation.                                      |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate "As-Built" Documentation

Evaluate "as-built" documentation for correctness, consistency,
completeness, accuracy, readability, and testability.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate ERA contractor(s) "as-built" documentation |
+----------------+-----------------------------------------------------+
| Methods        | Evaluate the completed components and documentation |
|                | for correctness, consistency, completeness,         |
|                | accuracy, readability, and testability, verify the  |
|                | criteria described in IEEE Std. 1012-1998, Table 1  |
|                | are satisfied                                       |
+----------------+-----------------------------------------------------+
| Inputs:        | Completed components, "as-built" documentation,     |
|                | IEEE Std. 1012-1998                                 |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Implementation    |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt or availability of         |
|                | artifacts                                           |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Timeliness of the delivery of the "as-built"        |
| Assumptions:   | documentation is the primary concern. The IV&V      |
|                | evaluation cannot be produced in a timely manner if |
|                | the ERA contractor delays delivery of these         |
|                | documents. IV&V recommends that their analysis be   |
|                | considered a mandatory input for the System Test    |
|                | TRR as a way of mitigating any technical risk       |
|                | associated with a delay in the delivery of as-built |
|                | documentation.                                      |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Interface Analysis

Verify and validate that the interfaces with hardware, user, operator,
software, and other systems, subsystems and components have been
correctly identified and described.

+----------------+-----------------------------------------------------+
| Task:          | Perform Interface Analysis                          |
+----------------+-----------------------------------------------------+
| Methods        | Assess the interfaces for correctness, consistency, |
|                | completeness, accuracy, and testability, verify the |
|                | criteria described in IEEE Std. 1012-1998, Table 1  |
|                | are satisfied                                       |
+----------------+-----------------------------------------------------+
| Inputs:        | Concept documentation (e.g., mission needs, concept |
|                | of operation), system requirements,                 |
|                | system/subsystem architecture and detailed design   |
|                | descriptions, interface design descriptions,        |
|                | "as-built" documentation, IEEE Std. 1012-1998,      |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Implementation    |
|                | Activity Summary Report, Anomaly Reports            |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within ten       |
|                | business days of receipt or availability of         |
|                | artifacts                                           |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Timeliness of the delivery of the "as-built"        |
| Assumptions:   | documentation is the primary concern. The IV&V      |
|                | evaluation cannot be produced in a timely manner if |
|                | the ERA contractor delays delivery of these         |
|                | documents. IV&V recommends that their analysis be   |
|                | considered a mandatory input for the System Test    |
|                | TRR as a way of mitigating any technical risk       |
|                | associated with a delay in the delivery of as-built |
|                | documentation.                                      |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA System Test Case Specifications

Verify that the test cases conform to the applicable standard in
purpose, format, and content. Verify that the test cases trace to the
test plan.

+------------+---------------------------------------------------------+
| Task:      | Verify ERA System Test Case Specifications              |
+------------+---------------------------------------------------------+
| Methods:   | Verify that the test cases correctly emanate from the   |
|            | test plan and conform to IEEE Std 829-1998 in format,   |
|            | content, and purpose; verify the criteria described in  |
|            | IEEE Std 1012-1998 are satisfied                        |
+------------+---------------------------------------------------------+
| Inputs:    | ERA system test case specifications, IEEE Std 829-1998; |
|            | IEEE Std 1012-1998, Table 1                             |
+------------+---------------------------------------------------------+
| Outputs:   | IV&V Task Report; input to End of Design Activity       |
|            | Summary Report                                          |
+------------+---------------------------------------------------------+
| Schedule:  | TBD                                                     |
|            |                                                         |
|            | Report submitted to the Government within seven (7)     |
|            | business days of receipt of document                    |
+------------+---------------------------------------------------------+
| Resources: | Refer to **Section 4.4**                                |
+------------+---------------------------------------------------------+
| Risks and  | Timeliness of the delivery of the test case             |
| As         | specifications is the primary concern. The IV&V         |
| sumptions: | evaluation cannot be produced in a timely manner if the |
|            | ERA contractor delays delivery of these documents. IV&V |
|            | recommends that their analysis be considered a          |
|            | mandatory input for the System Test TRR as a way of     |
|            | mitigating any technical risk associated with a delay   |
|            | in the delivery of test documentation.                  |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

#### Verify ERA System Test Procedure Specifications

Verify that the test procedure specifications conform to the applicable
standard in purpose, format, and content. The system test procedures may
be a stand-alone document or it may be part of the test case document.

+------------+---------------------------------------------------------+
| Task:      | Verify ERA System Test Procedure Specifications         |
+------------+---------------------------------------------------------+
| Methods:   | Verify that the test procedures correctly implement the |
|            | test cases and conform to IEEE Std 829-1998 in format,  |
|            | content, and purpose; verify the criteria described in  |
|            | IEEE Std 1012-1998 are satisfied                        |
+------------+---------------------------------------------------------+
| Inputs:    | ERA system test procedure specifications, IEEE Std      |
|            | 829-1998; IEEE Std 1012-1998, Table 1                   |
+------------+---------------------------------------------------------+
| Outputs:   | IV&V Task Report; input to End of Design Activity       |
|            | Summary Report                                          |
+------------+---------------------------------------------------------+
| Schedule:  | TBD                                                     |
|            |                                                         |
|            | Report submitted to the Government within seven (7)     |
|            | business days of receipt of document                    |
+------------+---------------------------------------------------------+
| Resources: | Refer to **Section 4.4**                                |
+------------+---------------------------------------------------------+
| Risks and  | Timeliness of the delivery of the test documentation is |
| As         | the primary concern. The IV&V evaluation cannot be      |
| sumptions: | produced in a timely manner if the ERA contractor       |
|            | delays delivery of these documents. IV&V recommends     |
|            | that their analysis be considered a mandatory input for |
|            | the System Test TRR as a way of mitigating any          |
|            | technical risk associated with a delay in the delivery  |
|            | of test documentation.                                  |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

#### Verify ERA PMO Acceptance Test Case Specifications

Verify that the test case specifications conform to the applicable
standard in purpose, format, and content.

+------------+---------------------------------------------------------+
| Task:      | Verify ERA PMO Acceptance Test Case Specifications      |
+------------+---------------------------------------------------------+
| Methods:   | Verify that the test cases correctly emanate from the   |
|            | test plan and conform to IEEE Std 829-1998 in format,   |
|            | content, and purpose; verify the criteria described in  |
|            | IEEE Std 1012-1998 are satisfied                        |
+------------+---------------------------------------------------------+
| Inputs:    | ERA acceptance test case specifications, IEEE Std       |
|            | 829-1998; IEEE Std 1012-1998, Table 1                   |
+------------+---------------------------------------------------------+
| Outputs:   | IV&V Task Report; input to End of Design Activity       |
|            | Summary Report                                          |
+------------+---------------------------------------------------------+
| Schedule:  | TBD                                                     |
|            |                                                         |
|            | Report submitted to the Government within seven (7)     |
|            | business days of receipt of document                    |
+------------+---------------------------------------------------------+
| Resources: | Refer to section 4.4                                    |
+------------+---------------------------------------------------------+
| Risks and  | Timeliness of the delivery of the test case             |
| As         | specifications is the primary concern. The IV&V         |
| sumptions: | evaluation cannot be produced in a timely manner if the |
|            | ERA PMO delays delivery of these documents.             |
+------------+---------------------------------------------------------+
| Roles and  | Refer to **Section 4.5**                                |
| Respons    |                                                         |
| ibilities: |                                                         |
+------------+---------------------------------------------------------+

#### Generate IV&V System and Acceptance Test Case Specifications

Generate test cases and verify that the test cases comply with IEEE Std.
829-1998 for document purpose, format, and content. IV&V will be
performing statistics-based testing for both System and Acceptance
Testing. The IV&V test cases will be based on contractor and Government
produced test cases. For each, IV&V will randomly select a sample of
test cases and adapt them to IV&V use. The adaptation will usually
consist of additional preparation steps to establish suitable initial
conditions since the IV&V tests will be performed outside of the context
that would be built in a sequential series of tests. **NOTE:** *The
scope of IV&V testing will be determined by the ERA PD or his designated
representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V Test Case Specifications              |
+----------------+-----------------------------------------------------+
| Methods        | Ensure test case specifications (system,            |
|                | acceptance) conform to IEEE Std. 829-1998; ensure   |
|                | adequacy based on IV&V Test Designs (system,        |
|                | acceptance), verify criteria described in IEEE Std. |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | System requirements, use cases, ERA contractor(s)   |
|                | test cases, ERA program-level acceptance test       |
|                | cases, Std. 829-1998, Std. 1012-1998, Table 1       |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V (system, acceptance) test case specifications, |
|                | input to End of Implementation Activity Summary     |
|                | Report                                              |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Test cases will be delivered to the Government upon |
|                | completion                                          |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | The timeliness of IV&V performance for this task    |
| Assumptions:   | depends on the timeliness of the delivery of ERA    |
|                | contractor(s) system test case specifications and   |
|                | ERA PMO acceptance test case specifications.        |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Generate IV&V System Test Procedure Specifications

Generate test procedures and verify that the test procedures comply with
IEEE Std. 829-1998 for document purpose, format, and content. Test
Procedures are not usually published as stand-alone documents rather,
they are included with their respective test case specifications. IV&V
will be performing statistics-based testing for both System and
Acceptance Testing. The IV&V test procedures will be based on contractor
and Government produced test procedures. For each, IV&V will randomly
select a sample of test cases together with their procedures and adapt
them to IV&V use. The adaptation will usually consist of additional
preparation steps to establish suitable initial conditions since the
IV&V tests will be performed outside of the context that would be built
in a sequential series of tests. **NOTE:** *The scope of IV&V testing
will be determined by the ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V System Test Procedure Specifications  |
+----------------+-----------------------------------------------------+
| Methods        | Ensure system test procedures conform to IEEE Std.  |
|                | 829-1998; ensure adequacy based on IV&V test cases, |
|                | verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V Test Cases, Std. 829-1998, Std. 1012-1998,     |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V system test procedure specifications, input to |
|                | End of Implementation Activity Summary Report       |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Test Procedures will be delivered to the Government |
|                | upon completion                                     |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Timeliness of the delivery of IV&V System Test      |
| Assumptions:   | Procedure Specifications depends on the timeliness  |
|                | of the delivery of ERA contractor developed system  |
|                | test procedures.                                    |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform criticality Analysis

Determine whether or not correct integrity levels are established for
requirements, modules, subsystems, or other system partitions.

+----------------+-----------------------------------------------------+
| Task:          | Re-assess ERA Integrity Level.                      |
+----------------+-----------------------------------------------------+
| Methods        | Verify that the assigned integrity level(s) are     |
|                | correct. Verify that the most critical level        |
|                | assigned to an individual element is assigned to    |
|                | the entire system.                                  |
+----------------+-----------------------------------------------------+
| Inputs:        | Prior IV&V Task Reports, Anomaly Reports,           |
|                | Criticality Analysis.                               |
+----------------+-----------------------------------------------------+
| Outputs:       | Identification of revised ERA Integrity Level, IV&V |
|                | Task Report, Updated IV&V Plan, input to End of     |
|                | Implementation Activity Summary Report.             |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of the task.                                        |
+----------------+-----------------------------------------------------+
| Resources:     | ERA PD, ERA COR, ERA POST Team Leads, IV&V          |
|                | Director, IV&V SMEs.                                |
+----------------+-----------------------------------------------------+
| Risks and      | Delay in identifying integrity level has negative   |
| Assumptions:   | impact on the scoping of the IV&V effort and the    |
|                | overall IV&V schedule. IV&V will support the        |
|                | Government decision making process. The assumption  |
|                | is that the Government will make a criticality      |
|                | decision in a timely manner.                        |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**.                           |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget, and schedule risks identified by IV&V
are correctly reviewed and tracked.

  ------------------- -----------------------------------------------------
  Task:               Identify technical and management risk.

  Methods             Documents being reviewed for conformance to user
                      needs and applicable standards also will be reviewed
                      for risk to program, budget and schedule. Other risks
                      will be noted during assessments of processes (CM
                      Assessment, RM Assessment) or during technical
                      interchange meetings, reviews, and audits. All
                      identified risks will be entered in the ERA Risk
                      Management data base through the process described in
                      the ERA Risk Management Plan.

  Inputs:             IV&V task reports on ERA contractor(s) deliverables,
                      ERA PMO process documents, and other artifacts.

  Outputs:            Input to ERA Risk Management Program, IV&V Task
                      Report, input to End of Implementation Activity
                      Summary Report, Anomaly Reports.

  Schedule:           TBD

  Resources:          Refer to **Section 4.4**.

  Risks and           Risks identified by IV&V may not be accepted or
  Assumptions:        managed expeditiously by the ERA Risk Management
                      Team. Should this occur IV&V will maintain its own
                      risk database and will issue its own risk reports.
                      The assumption is that risks identified by IV&V will
                      be accepted and managed by the ERA Risk Management
                      Team.

  Roles and           Refer to **Section 4.5**.
  Responsibilities:   
  ------------------- -----------------------------------------------------

### Test Activity 

The Test activity covers component and subsystem testing, component and
subsystem integration testing, system integration testing, system
testing, and acceptance testing. The objectives of IV&V are to ensure
that the component, subsystem, and system requirements are satisfied by
execution of integration and system test; and that the system satisfies
the needs of the users as demonstrated by the execution of acceptance
test. IV&V will not be executing IV&V integration test. IV&V will
execute statistics-based IV&V System and Acceptance tests. For each, a
random sample of ERA contractor or PMO generated test cases will be
adapted for IV&V use. The results of the IV&V testing will be analyzed
and projected on the population for comparison.

The specific IV&V test activity tasks include:

-   Perform traceability (test documentation) analysis,

-   Verify ERA contractor(s) integration test execution,

-   Verify ERA contractor(s) system test execution,

-   Verify ERA contractor(s) integration and system test reports,

-   Execute IV&V system test,

-   Verify ERA PMO Acceptance Test Procedure Specifications,

-   Generate IV&V Acceptance Test Procedure Specifications,

-   Verify ERA PMO acceptance test execution,

-   Verify ERA PMO acceptance test report,

-   Execute IV&V acceptance test, and

-   Perform risk analysis.

#### Perform Traceability Analysis

Analyze the from/to relationships in the ERA contractor(s) integration
and system test plans, test designs, test cases and procedures for
correctness, consistency, and completeness.

+----------------+-----------------------------------------------------+
| Task:          | Perform Traceability (Test Documentation) Analysis  |
+----------------+-----------------------------------------------------+
| Methods        | Assess traceability from the test plans and test    |
|                | designs to the test cases and procedures and back   |
|                | again to verify the correctness, consistency, and   |
|                | completeness of the test documentation, verify that |
|                | the criteria described in IEEE Std. 1012-1998,      |
|                | Table 1 are satisfied                               |
+----------------+-----------------------------------------------------+
| Inputs:        | Test plan(s), test designs, test cases, test        |
|                | procedures, IEEE Std. 1012-1998, Table 1            |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within 10        |
|                | business days of receipt of documentation           |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contractor(s) Integration Test Execution

Witness the ERA contractor(s) integration test execution to verify that
the software and/or hardware integration satisfies the test acceptance
criteria.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA contractor(s) integration test execution |
+----------------+-----------------------------------------------------+
| Methods        | Witness the integration test execution and use the  |
|                | ERA contractor(s) test results to verify that the   |
|                | subsystem or component satisfies the test           |
|                | acceptance criteria, verify the criteria described  |
|                | in IEEE Std. 1012-1998, Table 1 are satisfied       |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) integration test plan, test       |
|                | design, test cases, test procedures, anomaly        |
|                | reports, test logs, IEEE Std. 1012-1998, Table 1    |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task                                             |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contractor(s) System Test Execution 

Witness the ERA contractor(s) system test to verify that the system
satisfies the test acceptance criteria.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA contractor(s) system test execution      |
+----------------+-----------------------------------------------------+
| Methods        | Witness the system test execution and use the ERA   |
|                | contractor(s) test results to verify that the       |
|                | system satisfies the test acceptance criteria,      |
|                | verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) system test plan, test cases,     |
|                | test procedures, anomaly reports, test logs, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task                                             |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Contractor(s) Integration and System Test Reports

Verify that the test report complies with IEEE Std. 829-1998 in purpose,
format, and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify the ERA contractor(s) integration and system |
|                | test reports                                        |
+----------------+-----------------------------------------------------+
| Methods        | Verify conformance to IEEE Std. 829-1998 and that   |
|                | testing (integration, system) is accurately         |
|                | described, verify the criteria described in IEEE    |
|                | Std. 1012-1998, Table 1 are satisfied               |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA contractor(s) test reports, plans, test cases,  |
|                | test procedures, anomaly reports, test logs, IV&V   |
|                | Task Reports---Test Witness, IEEE Std. 1012, 1998,  |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days after receipt of documentation        |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Execute IV&V System Tests

Execute IV&V system test to verify the system satisfies the IV&V
acceptance criteria. IV&V will execute statistics-based IV&V System
tests. For each test, a random sample of ERA contractor generated test
cases will be adapted for IV&V use. The results of the IV&V testing will
be analyzed and projected on the population for comparison. Statistical
analysis techniques will be used to validate the accuracy of ERA Test
results. **NOTE:** *The scope of IV&V testing will be determined by the
ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Execute IV&V system testing                         |
+----------------+-----------------------------------------------------+
| Methods        | Validate that the IV&V acceptance criteria are met, |
|                | verify criteria described in IEEE Std. 1012-1998,   |
|                | Table 1 are satisfied.                              |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V test cases, test procedures, test data, ERA    |
|                | System Test results, IEEE Std. 1012-1998, Table 1   |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V test log, anomaly reports, IV&V test report,   |
|                | V&V Task Report, input to End of Test Activity      |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of activity                                         |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to Section 4.4                                |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to Section 4.5                                |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA PMO Acceptance Test Procedure Specifications

Verify that the test procedures conform to the applicable standard in
purpose, format and content. The acceptance test procedures may be a
stand-alone document or may be part of the test case document.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA PMO Acceptance Test Procedure            |
|                | Specifications                                      |
+----------------+-----------------------------------------------------+
| Methods        | Verify that the test procedures correctly implement |
|                | the test cases and conform to IEEE Std. 829-1998 in |
|                | format, content, and purpose; verify the criteria   |
|                | described in IEEE Std. 1012-1998 are satisfied      |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA acceptance test procedures, IEEE Std. 829-1998, |
|                | IEEE Std. 1012-1998, Table 1                        |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report; Input to End of Test Activity     |
|                | Summary Report                                      |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of document                |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Generate IV&V Acceptance Test Procedure Specifications

Generate test procedures and verify that the test procedures comply with
IEEE Std. 829-1998 for document purpose, format, and content. Test
procedures are not usually published as stand-alone documents rather;
they are included with their respective test case specifications. IV&V
will be performing statistics-based testing for both System and
Acceptance Testing. The IV&V test procedures will be based on contractor
and Government produced test procedures. For each test, IV&V will
randomly select a sample of test cases together with their procedures
and adapt them to IV&V use. The adaptation will usually consist of
additional preparation steps to establish suitable initial conditions
since the IV&V tests will be performed outside of the context that would
be built in a sequential series of tests. **NOTE:** *The scope of IV&V
testing will be determined by the ERA PD or his designated
representative.*

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V Acceptance Test Procedure             |
|                | Specifications                                      |
+----------------+-----------------------------------------------------+
| Methods        | Ensure acceptance test procedures conform to IEEE   |
|                | Std. 829-1998; ensure adequacy based on IV&V test   |
|                | cases, verify the criteria described in IEEE Std.   |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V Test Cases, Std. 829-1998, Std. 1012-1998,     |
|                | Table 1                                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V acceptance test procedure specifications,      |
|                | input to End of Implementation Activity Summary     |
|                | Report                                              |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Test Procedures will be delivered to the Government |
|                | upon completion                                     |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Timeliness of the delivery of IV&V Acceptance Test  |
| Assumptions:   | Procedure Specifications depends on the timeliness  |
|                | of the delivery of ERA PMO developed acceptance     |
|                | test procedures.                                    |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA PMO Acceptance Test Execution

Witness the acceptance test execution to verify that the system
satisfies the acceptance criteria.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA PMO acceptance test execution            |
+----------------+-----------------------------------------------------+
| Methods        | Witness ERA acceptance test execution and use the   |
|                | ERA acceptance test results to verify that the      |
|                | system satisfies the test acceptance criteria,      |
|                | verify the criteria described in IEEE Std.          |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA Acceptance Test Plan* document, test cases,    |
|                | test procedures, anomaly reports, test logs, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of activity                                         |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA PMO Acceptance Test Report

Verify that the Acceptance Test Report complies with the applicable
standard in purpose, format, and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify *ERA Acceptance Test Report*                 |
+----------------+-----------------------------------------------------+
| Methods        | Ensure conformance to IEEE Std. 829-1998 in format, |
|                | content, and purpose and verify correct             |
|                | representation of ERA acceptance test execution     |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA Acceptance Test Report*, IEEE Std. 829-1998,   |
|                | IV&V Task Report---ERA acceptance testing, ERA      |
|                | anomaly reports, and test logs                      |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to End of Test Activity     |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days after receipt of documentation        |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Execute IV&V Acceptance Testing

Execute IV&V acceptance test to verify the system satisfies the IV&V
acceptance criteria. IV&V will execute statistics-based IV&V Acceptance
tests. For each, a random sample of ERA PMO generated test cases will be
adapted for IV&V use. The results of the IV&V testing will be analyzed
and projected on the population for comparison. Statistical analysis
techniques will be used to validate the accuracy of the ERA Acceptance
Test results. **NOTE:** *The scope of IV&V testing will be determined by
the ERA PD or his designated representative.*

+----------------+-----------------------------------------------------+
| Task:          | Execute IV&V acceptance testing                     |
+----------------+-----------------------------------------------------+
| Methods        | Validate that the IV&V acceptance criteria are met, |
|                | verify criteria described in IEEE Std. 1012-1998,   |
|                | Table 1 are satisfied.                              |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V test cases, test procedures, test data, IEEE   |
|                | Std. 1012-1998, Table 1                             |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V test log, anomaly reports, IV&V test report,   |
|                | V&V Task Report, input to End of Test Activity      |
|                | Summary Report, Anomaly Reports                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of activity                                         |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget, and schedule risks identified by IV&V
are correctly reviewed and tracked.

  ------------------- -----------------------------------------------------
  Task:               Identify technical and management risk.

  Methods             Documents being reviewed for conformance to user
                      needs and applicable standards also are being
                      reviewed for risk to program, budget and schedule.
                      Other risks will be noted during assessments of
                      processes (CM Assessment, RM Assessment) or during
                      technical interchange meetings, reviews, and audits.
                      All identified risks will be entered in the ERA Risk
                      Management data base through the process described in
                      the ERA Risk Management Plan.

  Inputs:             IV&V task reports on ERA contractor(s) and ERA
                      development process documents and other artifacts.

  Outputs:            Input to ERA Risk Management Program, IV&V Task
                      Report, input to End of Test Activity Summary Report.

  Schedule:           TBD

  Resources:          Refer to **Section 4.4**.

  Risks and           Risks identified by IV&V may not be accepted or
  Assumptions:        managed expeditiously by the ERA Management Team.
                      Should this occur IV&V will maintain its own risk
                      database and will issue its own risk reports. The
                      assumption is that risks identified by IV&V will be
                      accepted and managed by the ERA Risk Management Team.

  Roles and           Refer to **Section 4.5**.
  Responsibilities:   
  ------------------- -----------------------------------------------------

###  Installation and Checkout IV&V Activity

The ERA installation and checkout activity addresses the installation of
the system in the operational environment. The installation and checkout
activity addresses installation and acceptance support. During this
activity, the objective of IV&V is to verify and validate the
correctness of the installation in the operational environment. The
first instance of the ERA installation and checkout is at the end of
Increment #1 (IOC) and is repeated at the end of each increment. The
last instance of the ERA installation and checkout is at FOC. The
installation and checkout IV&V activity also occurs for each release
within an increment. If a particular release is identified as
non-operational, IV&V will not perform the installation and checkout
tasks.

The specific installation and checkout IV&V activities include:

-   Perform Installation Configuration Audit,

-   Verify ERA Installation Checkout (Test),

-   Perform Installation Checkout (Test),

-   Verify ERA Installation Test Report,

-   Perform Risk Analysis, and

-   Generate IV&V Final Report.

#### Perform Installation Configuration Audit

Verify that all products required to correctly install and operate ERA
are present in the installation package. This task will be performed for
each installation of each release and/or iteration.

+----------------+-----------------------------------------------------+
| Task:          | Perform Installation Configuration Audit            |
+----------------+-----------------------------------------------------+
| Methods        | Verify that required products are present in the    |
|                | installation package, verify supplied values for    |
|                | all components are correct, validate that all       |
|                | site-dependent parameters or conditions are         |
|                | correct, verify the criteria described in IEEE Std. |
|                | 1012-1998, Table 1 are satisfied                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Installation package, listing of required products, |
|                | listing of supplied values for individual           |
|                | components, site-dependent parameters or            |
|                | conditions, IEEE Std. 1012-1998, Table 1            |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to the IV&V Final Report,   |
|                | Anomaly Reports                                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Installation Checkout (Test)

Witness the ERA Installation Checkout (Test) to verify the system
satisfies the installation acceptance criteria. This task will be
performed for each installation of each release and/or increment.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA Installation Test                        |
+----------------+-----------------------------------------------------+
| Methods        | Use the ERA installation test results to verify     |
|                | that the system satisfies the installation          |
|                | acceptance criteria, verify the criteria described  |
|                | in IEEE Std. 1012-1998, Table 1 are satisfied       |
+----------------+-----------------------------------------------------+
| Inputs:        | Installation package, user documentation, ERA       |
|                | installation test plan, test design, test cases,    |
|                | test procedures, IEEE Std. 1012-1998, Table 1       |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to the IV&V Final Report,   |
|                | Anomaly Reports                                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of the of task                                      |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Checkout will be incorrect if the assumption that   |
| Assumptions:   | input documents are accurate and complete is not    |
|                | met                                                 |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Installation Checkout (Test)

Verify that the installed system corresponds to the system subjected to
IV&V Acceptance Test. This task will be performed for each installation
of each release and/or increment.

+----------------+-----------------------------------------------------+
| Task:          | Perform IV&V Installation Test                      |
+----------------+-----------------------------------------------------+
| Methods        | Verify that all components operate as specified,    |
|                | for transition from one version to another,         |
|                | validate that the new or updated components can be  |
|                | removed from the system without affecting the       |
|                | functionality of the remaining system components,   |
|                | for transition from one version to another, verify  |
|                | requirements for continuous operation and service   |
|                | including user notification, verify the criteria    |
|                | described in IEEE Std. 1012-1998, Table 1 are       |
|                | satisfied                                           |
+----------------+-----------------------------------------------------+
| Inputs:        | Installation Package, user documentation,           |
|                | installation test documentation, IEEE Std.          |
|                | 1012-1998, Table 1                                  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V test log, IV&V test report, input to the IV&V  |
|                | Final Report, Anomaly Reports                       |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task                                             |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Checkout will be incorrect if the assumption that   |
| Assumptions:   | input documentation and artifacts are accurate and  |
|                | complete is not met.                                |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Verify ERA Installation Test Report

Verify that the test report complies with the applicable standard in
purpose, format and content.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA Installation Test Report                 |
+----------------+-----------------------------------------------------+
| Methods        | Verify the test report complies to the requirements |
|                | of IEEE Std. 829-1998 and correct reflection of the |
|                | ERA installation test execution                     |
+----------------+-----------------------------------------------------+
| Inputs:        | ERA Installation Test Report, IEEE 829-1998         |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, input to IV&V Final Report,       |
|                | Anomaly Reports                                     |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of the receipt of the documentation.  |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Review will be incorrect if the assumption that     |
| Assumptions:   | input documents are accurate and complete is not    |
|                | met                                                 |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Determine that technical, budget, and schedule risks identified by IV&V
are correctly reviewed and tracked.

+----------------+-----------------------------------------------------+
| Task:          | Identify technical and management risk.             |
+----------------+-----------------------------------------------------+
| Methods        | Documents being reviewed for conformance to user    |
|                | needs and applicable standards also are reviewed    |
|                | for risk to program, budget and schedule. Other     |
|                | risks are noted during assessments of processes (CM |
|                | Assessment, RM Assessment) or during technical      |
|                | interchange meetings, reviews, and audits. All      |
|                | identified risks will be entered in the ERA Risk    |
|                | Management data base through the process described  |
|                | in the ERA Risk Management Plan.                    |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V task reports on contractor and ERA development |
|                | process documents and other artifacts.              |
+----------------+-----------------------------------------------------+
| Outputs:       | Input to ERA Risk Management Program, IV&V Task     |
|                | Report, input to IV&V Final Report.                 |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Identified risks submitted to ERA Risk Management   |
|                | Program for reporting and tracking.                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**.                           |
+----------------+-----------------------------------------------------+
| Risks and      | Risks identified by IV&V may not be accepted or     |
| Assumptions:   | managed expeditiously by the ERA Risk Management    |
|                | Team. Should this occur IV&V will maintain its own  |
|                | database and will issue its own risk report. The    |
|                | assumption is that the ERA Risk Management Team     |
|                | will accept and manage IV&V identified risks.       |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**.                           |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Generate IV&V Final Report

The IV&V Final Report summarizes the IV&V activities, tasks, and
results, including status and disposition of anomalies. It provides an
assessment of the overall system quality and provides recommendations
regarding product and process.

+----------------+-----------------------------------------------------+
| Task:          | Generate IV&V Final Report                          |
+----------------+-----------------------------------------------------+
| Methods        | Ensure that the IV&V Final Report complies with     |
|                | IEEE Std. 1012-1998 in format, content, and purpose |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V End of Activity Summary Reports, IV&V Task     |
|                | Reports, Anomaly Reports, all other IV&V            |
|                | deliverables, IEEE Std. 1012-1998                   |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Final Report                                   |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of the task.                                        |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | No risks are expected                               |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

## Operation Process

The operation process covers the operation of the system and operational
support to users. Operation IV&V activities evaluate the impact of any
proposed changes to the operating environment, evaluate operating
procedures for compliance with the intended use, analyze risks, and
assess the effect on the system of any proposed changes.

### Operation IV&V Activity

The operation IV&V activity addresses operational testing, system
operation, and user support. During this activity the objectives of IV&V
are to evaluate new constraints in the system, assess proposed changes
and their impact, and evaluate operating procedures for correctness and
usability.

The specific operation IV&V activities include:

-   Verify ERA Operational Acceptance Test,

```{=html}
<!-- -->
```
-   Evaluate new constraints,

-   Assess proposed changes,

-   Evaluate operating procedures, and

-   Perform risk analysis.

#### Verify ERA Operational Acceptance Test

Witness ERA operating on an ongoing basis to verify that the system
satisfies the acceptance criteria.

+----------------+-----------------------------------------------------+
| Task:          | Verify ERA Operational Acceptance Test              |
+----------------+-----------------------------------------------------+
| Methods:       | TBD                                                 |
+----------------+-----------------------------------------------------+
| Inputs:        | TBD                                                 |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report will be submitted to the Government upon     |
|                | completion of task                                  |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Review will be incorrect if the assumption that     |
| Assumptions:   | input documentation is accurate and complete is not |
|                | met.                                                |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate New Constraints

Evaluate new constraints (e.g., operational requirements, platform
characteristics, operating environment) on the system or the
requirements to verify the potential impact on the IV&V Plan.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate new constraints                            |
+----------------+-----------------------------------------------------+
| Methods:       | New constraints will be evaluated for impact on the |
|                | IV&V Plan and a determination will be made          |
|                | concerning the need to update the IV&V Plan         |
+----------------+-----------------------------------------------------+
| Inputs:        | IV&V Plan, new constraints                          |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task                                             |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Assess Proposed Changes

Assess proposed changes (e.g., modifications, enhancements, or
additions) to determine the effect of the proposed changes to ERA.

+----------------+-----------------------------------------------------+
| Task:          | Assess Proposed changes                             |
+----------------+-----------------------------------------------------+
| Methods:       | Proposed changes will be evaluated for impact to    |
|                | ERA. A determination will be made as to the extent  |
|                | IV&V tasks need to be iterated.                     |
+----------------+-----------------------------------------------------+
| Inputs:        | Proposed changes                                    |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of documentation           |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate Operating Procedures

Verify that the operating procedures are consistent with the user
documentation and conform to the system requirements.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate operating procedures                       |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that the operating procedures are consistent |
|                | with the user documentation and conform to the      |
|                | system requirements                                 |
+----------------+-----------------------------------------------------+
| Inputs:        | Operating procedures, user documentation, system    |
|                | requirements                                        |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within ten       |
|                | business days of receipt of documentation           |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Review and update risk analysis using prior task reports. Provide
recommendations to eliminate, reduce, or mitigate the risk.

+----------------+-----------------------------------------------------+
| Task:          | Identify technical and management risks             |
+----------------+-----------------------------------------------------+
| Methods:       | Documents being reviewed for conformance to user    |
|                | needs and applicable standards also will be         |
|                | reviewed for risk to program, budget, and schedule. |
|                | Other risks will be noted during assessments of     |
|                | processes (CM Assessment, RM Assessment) or during  |
|                | technical interchange meetings, reviews, and audits |
|                | and proposed changes. All identified risks will be  |
|                | entered in the ERA Risk Management database through |
|                | the process described in the ERA Risk Management    |
|                | Plan.                                               |
+----------------+-----------------------------------------------------+
| Inputs:        | Proposed changes, development contractor(s) program |
|                | management plan, IV&V Task Reports                  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Identified risks submitted to ERA Risk Management   |
|                | Program for reporting and tracking.                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Risks identified by IV&V may not be accepted or     |
| Assumptions:   | managed expeditiously by the ERA Risk Management    |
|                | Team. Should this occur IV&V will maintain its own  |
|                | risk database and will issue is own risk reports.   |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

## Maintenance Process

The maintenance process is activated when the system undergoes
modifications caused by a problem or a need for improvement. The
Maintenance IV&V activity addresses those IV&V tasks required for
modifications, migration, or retirement of the system during the
operation process.

Modifications of the system shall be treated as development processes
and shall be verified and validated as described in **Section 5.1,
Management Process**, and **Section 5.4, Development Process**. The
assigned integrity level shall be revised as appropriate to reflect the
requirements of the maintenance process.

### Maintenance IV&V Activity

The maintenance activity covers modifications (e.g., corrective,
adaptive, and perfective), migration, and retirement of the system. The
maintenance IV&V activity addresses problem and modification analysis,
modification implementation, maintenance review/acceptance, migration,
and system retirement. The objectives of IV&V are to assess proposed
changes and their impacts, evaluate anomalies discovered during
operation, assess migration requirements, assess retirement
requirements, and re-perform IV&V tasks. The incremental and iterative
development life cycle of ERA may result in early deliverables being in
maintenance at the same time that later deliverables are undergoing
integration test.

The specific IV&V activities include:

-   Revise IV&V Plan,

-   Assess proposed changes,

-   Evaluate anomalies,

-   Perform task iteration,

-   Perform migration assessment,

-   Perform retirement assessment,

-   Perform risk analysis, and

-   Perform criticality analysis.

#### Revise IV&V Plan

Revise the IV&V Plan to include approved changes and ensure that the
revised IV&V Plan conforms to IEEE Std. 1012-1998.

+----------------+-----------------------------------------------------+
| Task:          | Revise the *ERA Independent Verification and        |
|                | Validation Plan (IVVP)*.                            |
+----------------+-----------------------------------------------------+
| Methods:       | Material upon which the revision is to be based     |
|                | will be collected and relevant information          |
|                | compiled, proposed approach changes agreed to, and  |
|                | the document generated and released for review.     |
+----------------+-----------------------------------------------------+
| Inputs:        | *ERA IVVP* document, approved changes, IEEE Std.    |
|                | 1012-1998,                                          |
+----------------+-----------------------------------------------------+
| Outputs:       | Revised IV&V Plan                                   |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | The revised IV&V Plan will be submitted to the      |
|                | Government upon completion of task.                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Assess Proposed Changes

Assess proposed changes (e.g., modifications, enhancements, or
additions) to determine the effect of the proposed changes to ERA.

+----------------+-----------------------------------------------------+
| Task:          | Assess proposed changes                             |
+----------------+-----------------------------------------------------+
| Methods:       | Proposed changes will be evaluated for impact to    |
|                | ERA and to validate that changes do not cause       |
|                | unacceptable or unintended system behavior. A       |
|                | determination will be made as to the extent IV&V    |
|                | tasks need to be iterated                           |
+----------------+-----------------------------------------------------+
| Inputs:        | Proposed changes, IV&V Plan                         |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report, modified IV&V Plan                |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government within seven (7) |
|                | business days of receipt of documents and modified  |
|                | IV&V Plan submitted to the Government upon          |
|                | completion of task.                                 |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Evaluate Anomalies

Evaluate the effect of system operation anomalies on ERA.

+----------------+-----------------------------------------------------+
| Task:          | Evaluate anomalies                                  |
+----------------+-----------------------------------------------------+
| Methods:       | Assess the effect on ERA of anomalies detected      |
|                | during operation                                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Anomaly reports                                     |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Task Iteration

Perform IV&V tasks, as needed, to ensure that planned changes are
implemented correctly, documentation is complete and current, and
changes do not cause unacceptable or unintended system behavior.

+----------------+-----------------------------------------------------+
| Task:          | Perform task iteration                              |
+----------------+-----------------------------------------------------+
| Methods:       | Validate that planned changes are implemented       |
|                | correctly, documentation is complete and current,   |
|                | and changes do not cause unacceptable or unintended |
|                | system behavior.                                    |
+----------------+-----------------------------------------------------+
| Inputs:        | Approved changes, installation package              |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Migration Assessment (this may not be required for ERA)

Assess whether system requirements and implementation address specific
migration requirements, migration tools, conversion of products and
data, archiving, support for the prior environment, and user
notification.

+----------------+-----------------------------------------------------+
| Task:          | Perform migration assessment                        |
+----------------+-----------------------------------------------------+
| Methods:       | Assess whether system requirements and              |
|                | implementation address specific migration           |
|                | requirements, migration tools, conversion of        |
|                | products and data, archiving, support for the prior |
|                | environment, and user notification.                 |
+----------------+-----------------------------------------------------+
| Inputs:        | System requirements                                 |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Retirement Assessment (this may not be required for ERA)

Assess whether the installation package addresses support, impact on
existing systems and databases, archiving, transition to new products,
and user notification.

+----------------+-----------------------------------------------------+
| Task:          | Perform retirement assessment                       |
+----------------+-----------------------------------------------------+
| Methods:       | Assess whether the installation package addresses   |
|                | support, impact on existing systems and databases,  |
|                | archiving, transition to new products, and user     |
|                | notification                                        |
+----------------+-----------------------------------------------------+
| Inputs:        | Installation package                                |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government upon completion  |
|                | of task.                                            |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | None                                                |
| Assumptions:   |                                                     |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Risk Analysis

Review and update risk analysis using prior task reports. Provide
recommendations to eliminate, reduce, or mitigate the risk.

+----------------+-----------------------------------------------------+
| Task:          | Identify technical and management risks             |
+----------------+-----------------------------------------------------+
| Methods:       | Documents being reviewed for conformance to user    |
|                | needs and applicable standards will also be         |
|                | reviewed for risk to program, budget and schedule.  |
|                | Other risks noted during assessments of processes   |
|                | (CM Assessment, RM Assessment) or during technical  |
|                | interchange meetings, reviews and audits and        |
|                | proposed changes. All identified risks will be      |
|                | entered in the ERA Risk Management database through |
|                | the process described in the ERA Risk Management    |
|                | Plan                                                |
+----------------+-----------------------------------------------------+
| Inputs:        | Proposed changes, development contractor(s) program |
|                | management plan, IV&V Task Reports                  |
+----------------+-----------------------------------------------------+
| Outputs:       | IV&V Task Report                                    |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Identified risks submitted to ERA Risk Management   |
|                | Program for reporting and tracking                  |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Risks identified by IV&V may not be accepted or     |
| Assumptions:   | managed expeditiously by the ERA Risk Management    |
|                | Team. Should this occur IV&V will maintain its own  |
|                | risk database and will issue its own risk reports.  |
|                | The assumption is that the ERA Risk Management Team |
|                | will accept and manage IV&V identified risks.       |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

#### Perform Criticality Analysis

Determine whether or not correct integrity levels are established for
requirements, modules, subsystems, or other system partitions.

+----------------+-----------------------------------------------------+
| Task:          | Re-assess ERA Integrity Level                       |
+----------------+-----------------------------------------------------+
| Methods:       | Verify that the assigned integrity level(s) are     |
|                | correct. Verify that the most critical level        |
|                | assigned to an individual element is assigned to    |
|                | the entire system                                   |
+----------------+-----------------------------------------------------+
| Inputs:        | Prior IV&V Task Reports---Criticality Analysis      |
+----------------+-----------------------------------------------------+
| Outputs:       | Identification of revised ERA Integrity Level, IV&V |
|                | Task Report, Updated IV&V Plan                      |
+----------------+-----------------------------------------------------+
| Schedule:      | TBD                                                 |
|                |                                                     |
|                | Report submitted to the Government completion of    |
|                | task.                                               |
+----------------+-----------------------------------------------------+
| Resources:     | Refer to **Section 4.4**                            |
+----------------+-----------------------------------------------------+
| Risks and      | Delay in identifying integrity level has negative   |
| Assumptions:   | impact on the scoping of the IV&V effort. IV&V will |
|                | support Government decision making process. The     |
|                | assumption is that the Government will make a       |
|                | criticality decision in a timely manner.            |
+----------------+-----------------------------------------------------+
| Roles and      | Refer to **Section 4.5**                            |
| Res            |                                                     |
| ponsibilities: |                                                     |
+----------------+-----------------------------------------------------+

# IV&V Reporting Requirements

The paragraphs in this section describe how the results of performing
the IV&V activities and tasks described in this IVVP will be documented.
IV&V reporting will occur throughout the program life cycle. Reporting
will communicate the status of the IV&V effort and its findings. The
reports provide visibility into the progress of development and an
assessment of the products, and risks associated with the identified
deficiencies. All IV&V tasks are reported upon the completion of the
task.

## Required Reports

All IV&V reports are delivered to the ERA PD for delivery to the ERA
COR. Reports will convey the results of IV&V activities as well as
provide a summary of the programmatic status of the overall IV&V
activities. The purpose and a brief description of the contents of each
type of report are provided in the following subparagraphs.

### Progress and Status Reports

Progress and Status Reports detail the IV&V activities conducted during
the current reporting period, the planned activities to be conducted
during the next reporting period and the current contract financial
status (if appropriate). IV&V will deliver Weekly Progress Reports and
Monthly Status Reports. **Appendix B, IV&V Templates** contains
templates for these reports.

### Task Reports

Task Reports are generated as a result of completing an evaluation of an
ERA-related product, process, or activity. The purpose of these reports
is to document any deficiencies in the product and to assist in
resolving the deficiencies as quickly as possible. **Appendix B**
contains a template that shall be used for generating Task Reports. All
evaluations will be undertaken with an evaluation form appropriate for
that product or activity. The evaluation form may be integrated into the
report.

Whenever the ERA-related product is a document, it shall be evaluated
using the appropriate standard(s) and the document review checklist
template contained in **Appendix C**. Early identification of problems
provides management with the maximum opportunity for remedial action.
The timeliness of these reports is paramount to maintaining both the
cost and schedule of implementing ERA. IV&V will provide these reports
to the ERA PD for delivery to the ERA COR within seven (7) business days
of the receipt of the document or within ten business days for more
complex tasks such as requirements traceability. IV&V reports for
non-document evaluations (e.g., criticality analysis or risk analysis)
will be provided to the ERA PD for delivery to the ERA COR upon
completion of the task.

### End of Activity Summary Report

The End of Activity Summary Report (e.g., End of Acquisition Support
Summary Report) will provide an independent assessment of the overall
technical state of the ERA implementation at that milestone. This report
will provide a recap of tasks conducted during the Activity,
discrepancies identified and their disposition, unresolved issues and
the risks associated with them, and recommendations. The report will
carry a cumulative effect of outstanding issues to provide NARA with a
view of the technical and program risks. **Appendix B** contains a
template that shall be used for generating each End of Activity Summary
Report.

### IV&V Final Report

The IV&V Final Report provides the independent assessment of the overall
ERA suitability to support the system's mission as specified in the
Mission Needs Statement and the ConOps document. This report will
include the following information: a summary of life cycle IV&V
activities, a summary of IV&V analyses results, a summary of anomalies
with proposed resolutions and dispositions, outstanding issues and
concerns, an overall assessment of the system, and final conclusions and
recommendations. **Appendix B** contains a template that shall be used
for generating the IV&V Final Report.

## Optional Reports

Additional reports, as identified, may be provided to inform the ERA PD
of an activity's results. The delivery of these reports will vary with
the activities performed. Reports will convey the results of IV&V
activities and issues of significant value. Optional reports may consist
of engineering white papers, meeting summary reports, and red flag
reports. The purpose and a brief description of the contents of these
reports are provided in **Sections 6.2.1** to **6.2.3**.

### Engineering White Papers

Engineering white papers are optional reports that may be initiated to:
document results of trade-off analyses; identify and recommend solutions
for system issues, or ERA-related special studies. These white papers
include a statement of the issue, analysis methodology, results,
conclusions, and recommendations. White papers also may be requested to
provide the results of an independent analysis or trade study.

### Meeting Summary Reports

After each authorized meeting, the IV&V representative will prepare a
Meeting Summary Report indicating the meeting purpose, accomplishments,
issues, and concerns. This report is provided to the ERA PMO PD for
delivery to the ERA COR.

### Red Flag Reports

Red Flag Reports identifying a detected anomaly that has the potential
of significant cost, schedule, or mission impact will be provided as
necessary. These reports will provide a description of the issue,
program impact, cause (if known), criticality, and recommendation. Red
Flag Reports are provided to the ERA PD for delivery to the ERA COR.

## Report Management Procedures

All reports will be routed through ERA Program Configuration Management
(CM) by the ERA PD for official delivery to the ERA COR as follows:

-   IV&V Peer Review Draft - The first draft will be named appropriately
    and will be designated as Version 0.01 for IV&V Peer review. At the
    completion of Peer Review, comments will be resolved by the IV&V
    Technical Director. After revision, the document will be incremented
    to Version 0.02 and will be forwarded by the IV&V Technical Director
    through the POST Technical Editor and QM Specialist to the ERA PD
    for the Director's review

-   ERA Program Director's Review and Final Disposition -- The ERA PD
    will forward comments to the IV&V Technical Director. The IV&V
    Technical Director will resolve any comments or issues, revise the
    document as appropriate, increment the version number to 1.0 and
    release the document to ERA Configuration Management, the ERA COR
    and the ERA PD through the POST Technical Editor and QM Specialist.

# ADMINISTRATIVE REQUIREMENTS

This section describes the IV&V administrative procedures for anomaly
reporting and resolution; task iteration policy; deviation policy;
control procedures; and standards, practices, and conventions. These
procedures are established to ensure a smooth flow of information from
IV&V to, the ERA POST Program Manager, the ERA COR, the ERA PD, and the
ERA contractor(s).

## Anomaly Resolution And Reporting

An anomaly is anything observed in the documentation or operation of the
system that deviates from expectations based on requirements,
specifications, design, user documents, standards, or from someone's
perceptions or experiences. Anomalies are reported as they are
discovered. **Appendix B** contains a template that shall be used for
generating the Anomaly Report. Anomalies shall be documented and managed
by IV&V in a tracking tool in a format to be described in an Appendix to
this document.

This section describes the method of reporting and resolving anomalies
including the methodology for reporting an anomaly, the anomaly report
distribution list, and the authority and time lines for resolving
anomalies. The classification schema for anomalies will follow the
schema described in IEEE Std 1044-1993\[B9\].

### Methodology

The specific methodology to be employed will be contained in the next
release of the IVVP.

### Distribution

The specific methodology to be employed will be contained in the next
release of the IVVP.

### Authority (this section needs to include the timeline for resolution)

The specific methodology to be employed will be contained in the next
release of the IVVP.

## Task Iteration Policy

IV&V task iteration depends on changes, correction of identified
defects, and the life cycle phases. Both the ERA life cycle phase and
IV&V are iterative processes. An iteration of an activity may be driven
by the correction of a defect, a change to a requirement specification
or the incremental development of a document and/or product that is
phase or contractually driven. These will be referred to generically as
a "change."

When a change is identified, an analysis will be performed to determine
which aspects of each completed life cycle phase will need to be
repeated to maintain the integrity of the product. The results of this
analysis will be used to generate an impact statement for the change.

The ERA life cycle will require IV&V to execute multiple life cycle
activities concurrently. For example, IV&V may be performing system test
support activities on Increment #1 capability while at the same time
performing design analysis support activities on a release within
Increment #2. Thus, the IV&V activities will be repeated for each
release.

## Deviation Policy

Project changes or external factors may make it necessary to deviate
from this plan. Any such deviations will be documented for approval
before they occur. A memorandum will be prepared that identifies the
task, deviation rationale, and the effect on the overall IV&V effort.
This memorandum will first be delivered to the ERA PD who will then
deliver it to the ERA COR prior to IV&V initiating the deviated process.
If the deviation is approved, this re-direction will be documented in
memo form. Deviations in the execution of this plan will be documented
in the Progress and Status Report, as well as in the End of Activity
Summary Report.

## Control Procedures

This paragraph identifies IV&V products and tasking results, how they
are controlled, and where they are stored. IV&V deliverable products
consist of Task Reports, End of Activity Summary Reports, Weekly
Progress Reports, Monthly Status Reports, memoranda, white papers, and
the IV&V Final Report. The official copies of these products are
maintained by the ERA PMO CM. IV&V maintains file and backup copies of
all deliverables.

## Standards, Practices, And Conventions

ERA follows Government laws, rules, and regulations, and NARA internal
standards for system acquisition. ERA IV&V will follow those Standards
identified in **Section 2.2.1** of this plan.

The standards, along with internal procedures and checklists, are used
in the review of documents. IV&V maintains and updates internal
procedures and checklists to incorporate new ideas or to reflect changes
in direction required by NARA.

# IV&V DOCUMENTATION REQUIREMENTS

In this section of the IVVP the purpose, format and content of the IV&V
test documentation is defined. The purpose, format, and content of the
IV&V test documentation will be in accordance with IEEE Std-829-1998,
except as noted below. IV&V expects to plan, execute, and report on IV&V
testing for system and acceptance testing for each release within the
various increments, and then again at IOC and FOC. **NOTE:** IV&V tests
will be conducted, or not, at the direction of the ERA PD.

## Test Plan

Test plans will be generated for each release within each increment. The
test plan(s) will describe the system-level test and the
acceptance-level effort for that specific release. System-level test
will test the requirements associated with that release while
acceptance-level test will validate the user functionality of the
release. Test plans will reflect the IV&V test strategy of statistical
sampling. This requires that a statistically determined sub-set of test
cases be identified and executed. Additionally, test plans will be
generated, if appropriate, for IOC and for FOC. These also will reflect
the strategy of statistical sampling.

## Test Design

The test design documents will not be separate documents as identified
in IEEE Std-829-1998 but will be the first section of the respective
test case document.

## Test Cases

For each test plan there will be a test case document. The test cases
will be traceable to the test plan and will implement the test design.

## Test Procedures

Test procedures will provide the information required to execute test
cases. These will not be separate documents, as identified in IEEE Std.
829-1998 but rather will be specified within the respective test case.

## Test Results

All test results will be documented in a test results report and in a
test results summary report for the respective test. Test discrepancies
will be entered in the IV&V database maintained in Microsoft Access.

## Test Log

A test log will be maintained during test execution. The test log will
contain such information as test executed, time started, time stopped,
problems encountered, and anomaly reports entered.

[]{#__RefHeading___Toc61249675 .anchor}Appendix A: IV&V Work Breakdown
Structure (WBS) and Schedule

The IV&V WBS will be integrated into the master program WBS.

IV&V Milestones will be integrated into the program master schedule.

The IV&V schedule will be integrated into the program master schedule.

Cost and Basis of Estimate data is supplied under separate cover.

[]{#__RefHeading___Toc61249676 .anchor}Appendix B: IV&V Templates

**IV&V Task Report Template:**

  -------------------------------- -----------------------------------------------------------
  []{#__RefHeading___Toc61249677   
  .anchor}IV&V TASK REPORT         

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- -----------------------------------------------------------

1.  **Description**

Describe, here, the IV&V task that is the subject of this report.
Include a listing of standards or other documents consulted.

2.  **Summary of Results**

Provide an executive-level summary of the task results.

3.  **Summary of Anomalies and Resolutions**

Provide a listing of anomaly reports produced as a result of this task
and include their current status.

4.  **Assessment of Quality**

Provide a detailed description of the task and methods employed followed
by a detailed description of the task results and conclusions. Detailed
results may be provided as an attachment.

5.  **Identification and Assessment of Technical and Management Risks**

Provide a full description of specific technical and management risks
detected and reported as a result of this task. Include disposition.

6.  **Recommendations**

Describe, here, any IV&V recommendations related to the task described
in this report.

**IV&V Quick Look Report Template:**

  -------------------------------- -----------------------------------------------------------
  []{#__RefHeading___Toc61249678   
  .anchor}IV&V QUICK LOOK REPORT   

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- -----------------------------------------------------------

1.  **Description of Task**

Describe, here, the IV&V task that is the subject of this report.
Include a listing of standards or other documents consulted.

2.  **Summary of Task Results**

Provide an executive-level summary of the task results.

3.  **Summary of Anomalies and Resolutions**

Provide a listing of anomaly reports produced as a result of this task
and include their current status.

4.  **Assessment of Quality**

Provide a summary description of the task and methods employed followed
by a summary description of the task results and conclusions. Detailed
results may be provided as an attachment.

5.  **Identification and Assessment of Technical and Management Risks**

Provide a summary description of technical and management risks detected
and reported as a result of this task. Include disposition.

6.  **Recommendations**

Describe, here, any IV&V recommendations related to the task described
in this report.

**IV&V Special Report Template:**

  -------------------------------- -----------------------------------------------------------
  []{#__RefHeading___Toc61249679   
  .anchor}IV&V SPECIAL REPORT      

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- -----------------------------------------------------------

1.  **Description**

Describe, here, the subject of this report (special task or study).
Include a listing of standards or other reference materials consulted.

2.  **Summary of Results**

Provide an executive-level summary of the task or study results.

3.  **Summary of Anomalies and Resolutions**

Provide a listing of anomaly reports produced as a result of this task
or study and include their current status.

4.  **Assessment of Quality**

Provide a detailed description of the task or study that is the subject
of this report and the methods employed followed by a detailed
description of the results and conclusions. Supporting information may
be provided as an attachment.

5.  **Identification and Assessment of Technical and Management Risks**

Provide a full description of specific technical and management risks
detected and reported as a result of this task or study. Include
disposition.

6.  **Recommendations**

Describe, here, any IV&V recommendations related to the task or study
described in this report.

**IV&V End of Activity Summary Report Template:**

  -------------- -----------------------------------------------------------
  **IV&V         
  ACTIVITY       
  SUMMARY        
  REPORT**       

  **TO:**             

  **THRU:**           

  **FROM:**           

  **IV&V OF:**        

  **SUBJECT:**        

  **DATE:**           

  **CC:**             
  -------------- -----------------------------------------------------------

1.  **Description of IV&V Tasks Performed**

Describe, at a high level, the IV&V tasks that were performed during
this activity. Provide references to delivered IV&V Task Reports.

2.  **Summary of Task Results**

Provide an executive-level summary of the task results, including an
overall assessment of the results of the activity as a whole.

3.  **Summary of Anomalies and Resolutions**

Provide a listing of anomaly reports produced as a result of all tasks
performed in support of this activity and include their current status.

4.  **Assessment of Quality**

Provide a description of the objectives of the activity followed by an
assessment of how well those objectives were achieved. Detailed results
and analyses may be provided as an attachment.

5.  **Identification and Assessment of Technical and Management Risks**

Provide a description of specific technical and management risks
detected and reported by IV&V while performing tasks in support of this
activity. Include disposition.

6.  **Recommendations**

IV&V will recommend that the PMO proceed to the next activity, proceed
conditionally, or not proceed. Include full justification.

**IV&V Final Report Template:**

  -------------- -----------------------------------------------------------
  **IV&V FINAL   
  REPORT**       

  **TO:**             

  **THRU:**           

  **FROM:**           

  **IV&V OF:**        

  **SUBJECT:**        

  **DATE:**           

  **CC:**             
  -------------- -----------------------------------------------------------

1.  **Summary of IV&V Life Cycle Activities**

Provide an executive-level summary of all IV&V activities performed in
support of this release or increment.

2.  **Summary of Task Results**

Provide an executive-level summary of all tasks performed in support of
this release or increment.

3.  **Summary of Anomalies and Resolutions**

Provide a categorical summary of anomalies discovered during tasking
performed in support of this release or increment. Include an
enumeration of any anomalies that remain outstanding or unresolved.

4.  **Assessment of Release/Increment Quality**

Provide an overall assessment of the quality of the products delivered
as a part of this increment or release. Include justification for the
assessment.

5.  **Lessons Learned/Best Practices**

Describe any lessons learned and resulting changes in practice that were
discovered during the conduct of IV&V for this release or increment.

6.  **Recommendations**

Describe IV&V's recommendations concerning Government acceptance of the
product delivered as a result of this release or increment. Provide full
justification for these recommendations.

**Product Evaluation Criteria:**

Specific product evaluation criteria will be supplied after ERA contract
negotiations yield a final list of contract deliverables.

**IV&V Anomaly Report Template:**

  -------------- ----------------------------- ------------------------------
  **IV&V ANOMALY                               
  REPORT**                                     

  **TO:**                                      

  **THRU:**                                    

  **FROM:**                                    

  **SUBJECT:**                                 

  **IV&V OF:**                                 

  **DATE:**                                    Control Number:  - 000

  **CC:**                                      
  -------------- ----------------------------- ------------------------------

1.  **Project and Phase**

State the long name of the project being evaluated. Include a brief
description of the project and a reference to the life cycle process
effected by this anomaly (ref. IEEE/EIA 12207.0-1996, §5, §6 & §7).
Indicate the equivalent IV&V activity in progress when this anomaly was
discovered (ref. IEEE Std 1012-1998, §5).

2.  **Description and Location in Document or Code**

Enter a detailed description of the anomaly.

3.  **Impact on Project Success**

Red - Critical, if this anomaly is not corrected, the project will
certainly fail.

Yellow - Moderate, though the anomaly may be moderate to serious in
nature, the impact to the successful completion of the project can be
mitigated by work-arounds or relatively minor adjustments to the
process.

Green - The anomaly is trivial to moderate in nature and the impact to
overall project success is negligible.

4.  **Cause of the Anomaly and Description of the Error Scenario**

Describe the conditions under which the anomalous event occurred and
identify the root cause.

5.  **Urgency of Corrective Action (Project)**

Enter an assessment of the urgency of the project level corrective
action required.

High - Corrective action must be taken immediately to avoid certain
project failure.

Medium - Corrective action is required, however, overall project success
is moderately affected and the corrective action must be taken prior to
customer acceptance.

Low - Corrective action has little or no immediate impact on the success
of the project and need not be implemented prior to customer acceptance.

6.  **Urgency of Corrective Action (Enterprise)**

Enter an assessment of the urgency of the enterprise level corrective
action which must be taken to correct the anomalous condition in this
report.

High - Corrective action must be taken immediately to avoid certain,
critical failure at the enterprise level.

Medium - Corrective action is required, however, moderate to high risk
to enterprise operations exists.

Low - Corrective action is required, however, the risk to enterprise
level operations is low to medium.

7.  **Recommendations**

State all recommendations in term of management actions.

**IV&V Meeting Summary Report Template:**

  -------------------------------- ------------------------------------------------------------
  []{#__RefHeading___Toc61249680   
  .anchor}IV&V MEETING ATTENDANCE  
  MEMORANDUM                       

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- ------------------------------------------------------------

1.  **IV&V Personnel in Attendance**

2.  **Summary of Meeting Objectives**

3.  **Summary of Meeting Results**

4.  **Summary of IV&V Participation**

5.  **IV&V Action Items**

6.  **General Observations**

**IV&V Weekly Progress Report Template:**

  -------------------------------- ------------------------------------------------------------
  []{#__RefHeading___Toc61249681   
  .anchor}IV&V WEEKLY PROGRESS     
  REPORT                           

  **TO:**                          

  **THRU:**                        

  **FROM:**                        

  **IV&V OF:**                     

  **SUBJECT:**                     

  **DATE:**                        

  **CC:**                          

  IV&V MEETING ATTENDANCE          
  MEMORANDUM                       

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- ------------------------------------------------------------

**As of:**

  ---- ------------------------------------------------ ------------- -----------
       **Products Due Within 60 days of Report**        **Baselined   **Planned
                                                        Delivery      Delivery
                                                        Date**        Date**

  Pi                                                                  

                                                                      
  ---- ------------------------------------------------ ------------- -----------

**Key Activities In-Process / Activities Completed This Week**

1.  

  ---- ------------------------------------------ ----------------- ---------- --------------
       **Issue to be Resolved**                   **Criticality**   **Issue to **Latest Date
                                                                    be         for
                                                                    Resolved   Resolution**
                                                                    By**       

  R1                                                                           

                                                                               
  ---- ------------------------------------------ ----------------- ---------- --------------

  ----- ------------------------------------------- ----------------- ---------- ----------
        **Contracting Action Required**             **Criticality**   **Action   **Latest
                                                                      to be      Date for
                                                                      Resolved   Action**
                                                                      By**       

  C1.                                                                            
  ----- ------------------------------------------- ----------------- ---------- ----------

**Other Comments**

1.  

  ----------- ----------------------- ------------ ------------------------
  **Report    Lou Pinto               **Position   Technical Director, IV&V
  Author:**                           Title:**     

  ----------- ----------------------- ------------ ------------------------

**IV&V Monthly Status Report Template:**

  -------------------------------- ------------------------------------------------------------
  []{#__RefHeading___Toc61249683   
  .anchor}IV&V MONTHLY STATUS      
  REPORT                           

  **TO:**                               

  **THRU:**                             

  **FROM:**                             

  **IV&V OF:**                          

  **SUBJECT:**                          

  **DATE:**                             

  **CC:**                               
  -------------------------------- ------------------------------------------------------------

**As of:**

  -- ----------------------------------------- ------------- ------------ ------------
     **IV&V Milestones in this Reporting       **Baselined   **Actual     **Planned
     Period**                                  Completion    Completion   Completion
                                               Date**        Date**       Date**

                                                                          

                                                                          
  -- ----------------------------------------- ------------- ------------ ------------

  -- ----------------------------------------- ------------- ------------ ------------
     **IV&V Product Deliveries this Reporting  **Baselined   **Actual     **Planned
     Period**                                  Completion    Completion   Completion
                                               Date**        Date**       Date**

                                                                          

                                                                          
  -- ----------------------------------------- ------------- ------------ ------------

**Exception Analysis**

1.  

  ---- ------------------------------------------ ----------------- ---------- --------------
       **Issue to be Resolved**                   **Criticality**   **Issue to **Latest Date
                                                                    be         for
                                                                    Resolved   Resolution**
                                                                    By**       

  R1                                                                           

                                                                               
  ---- ------------------------------------------ ----------------- ---------- --------------

  ----- ------------------------------------------- ----------------- ---------- ----------
        **Contracting Action Required**             **Criticality**   **Action   **Latest
                                                                      to be      Date for
                                                                      Resolved   Action**
                                                                      By**       

  C1.                                                                            
  ----- ------------------------------------------- ----------------- ---------- ----------

**Other Comments**

1.  

  ----------- ----------------------- ------------ ------------------------
  **Report    Lou Pinto               **Position   Technical Director, IV&V
  Author:**                           Title:**     

  ----------- ----------------------- ------------ ------------------------

**IV&V Test Plan Template:**

![](media/image1.wmf)

**IV&V Test Case Specification Template:**

![](media/image2.wmf)

**IV&V Test Results Report Template:**

![](media/image3.wmf)

**IV&V Test Summary Report Template:**

![](media/image4.wmf)

[]{#__RefHeading___Toc61249684 .anchor}Appendix C: Document Assessment
Checklist (sample)

Actual content of TOPIC AREA, CONTENT ITEM, and CONTENT REQUIREMENT is
based on the requirements of the applicable standard.

[Data Items]{.underline}: [Content]{.underline}:

ID Section Number as identified in the appropriate Standard

TOPIC AREA Primary area (e.g. Introduction)

CONTENT AREA Secondary area (e.g. Summary of System)

CONTENT REQUIREMENT Textual description extracted from the appropriate
Standard

LOCATION IN DOCUMENT Section number of the document under review that
contains the required information; if not then "Not included"

ASSESSMENT "Satisfied"; "Partially Satisfied"; "Not Satisfied";
indicates the degree to which the document under review satisfies the
required information

COMMENTS Textual comment by the reviewer