Electronic records Archives

TECHNICAL REVIEW PROCESS

(TEP)

(TOMP Version 2.0, Task 4.3.9)

**for the**

National archives and

Records administration

electronic records archives

program MANAGEMENT office

(NARA ERA PMO)

Final

July 18, 2003

Prepared by:

***Integrated Computer Engineering (ICE)***

***A Subsidiary of***

***American Systems Corporation (ASC)***

Contract Number: GS-35F-0673K

Delivery Order Number: NAMA-01-F-0031/03-010

TECHNICAL REVIEW PROCESS (TEP)

[]{#__RefHeading___Toc46220996 .anchor}Signature Page

Program Director,

I recommend approval of the Technical Review Process (TEP).

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Dyung Le, Date

ERA Program

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Virginia White, Date

ERA Program

Approved,

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Kenneth Thibodeau, Date

ERA Program Director

[]{#__RefHeading___Toc46220997 .anchor}Document Change Control Sheet

**Document Title:** Technical Review Process (TEP)

  ---------- -------------------- --------------- ---------------------------
  **Date**   Filename/version \#  **Author**      **Revision Description**

  7/18/03    ERA.DC.TEP.1.0.DOC   Ursula Parker   Baseline Technical Review
                                                  Process

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  

                                                  
  ---------- -------------------- --------------- ---------------------------

Signature Page [i](#__RefHeading___Toc46220996)

Document Change Control Sheet [ii](#__RefHeading___Toc46220997)

1.0 Introduction [1](#introduction)

1.1 Purpose [1](#purpose)

1.2 ERA Program Overview [2](#era-program-overview)

1.3 Definitions and Acronyms [2](#definitions-and-acronyms)

1.4 References [3](#references)

1.4.1 Standards [3](#standards)

1.4.2 ERA PMO Documentation [3](#era-pmo-documentation)

2.0 Technical Review Panel [3](#technical-review-panel)

2.1 Review Agenda [5](#review-agenda)

2.2 Review Preparation [5](#review-preparation)

2.3 Review Goals and Conduct [6](#review-goals-and-conduct)

2.4 Review Closeout [6](#review-closeout)

3.0 Technical Reviews [7](#technical-reviews)

3.1 Requirements Review [7](#requirements-review)

3.1.1 Purpose [7](#purpose-1)

3.1.2 Input [7](#input)

3.1.3 Procedures [8](#procedures)

3.1.4 Exit Criteria [8](#exit-criteria)

3.1.5 Output [8](#output)

3.2 System Requirements Review [8](#system-requirements-review)

3.2.1 Purpose [8](#purpose-2)

3.2.2 Input [9](#input-1)

3.2.3 Procedures [9](#procedures-1)

3.2.4 Exit Criteria [9](#exit-criteria-1)

3.2.5 Output [9](#output-1)

3.3 System Design Review [10](#system-design-review)

3.3.1 Purpose [10](#purpose-3)

3.3.2 Input [10](#input-2)

3.3.3 Procedures [10](#procedures-2)

3.3.4 Exit Criteria [11](#exit-criteria-2)

3.3.5 Output [11](#output-2)

3.4 Increment System Requirements Review
[11](#increment-system-requirements-review)

3.4.1 Purpose [11](#purpose-4)

3.4.2 Input [12](#input-3)

3.4.3 Procedures [12](#procedures-3)

3.4.4 Exit Criteria [12](#exit-criteria-3)

3.4.5 Output [12](#output-3)

3.5 Release System Requirements Review
[12](#release-system-requirements-review)

3.5.1 Purpose [13](#purpose-5)

3.5.2 Input [13](#input-4)

3.5.3 Procedures [13](#procedures-4)

3.5.4 Exit Criteria [13](#exit-criteria-4)

3.5.5 Output [13](#output-4)

3.6 Preliminary Design Review [14](#preliminary-design-review)

3.6.1 Purpose [14](#purpose-6)

3.6.2 Input [14](#input-5)

3.6.3 Procedures [15](#procedures-5)

3.6.4 Exit Criteria [15](#exit-criteria-5)

3.6.5 Output [15](#output-5)

3.7 Critical Design Review [15](#critical-design-review)

3.7.1 Purpose [16](#purpose-7)

3.7.2 Input [16](#input-6)

3.7.3 Procedures [16](#procedures-6)

3.7.4 Exit Criteria [16](#exit-criteria-6)

3.7.5 Output [17](#output-6)

3.8 Test Readiness Review [17](#test-readiness-review)

3.8.1 Purpose [17](#purpose-8)

3.8.2 Input [17](#input-7)

3.8.3 Procedures [18](#procedures-7)

3.8.4 Exit Criteria [18](#exit-criteria-7)

3.8.5 Output [18](#output-7)

3.9 Operational Readiness Review [19](#operational-readiness-review)

3.9.1 Purpose [19](#purpose-9)

3.9.2 Inputs [19](#inputs)

3.9.3 Procedures [19](#procedures-8)

3.9.4 Exit Criteria [20](#exit-criteria-8)

3.9.5 Output [20](#output-8)

4.0 Metrics and Measures [20](#metrics-and-measures)

5.0 Reviewers Roles and Responsibilities by Technical Review Type
[20](#reviewers-roles-and-responsibilities-by-technical-review-type)

6.0 Plan Maintenance [28](#plan-maintenance)

APPENDIX A: ERA Technical Review Action Item Log
A-[1](#__RefHeading___Toc46221252)

APPENDIX B: Technical Review Summary Report
B-[1](#__RefHeading___Toc46221253)

APPENDIX C: Technical Review Checklist
C-[1](#__RefHeading___Toc46221254)

List of Tables

Table 1‑1: Acronyms List [3](#__RefHeading___Toc46220948)

Table 5-1: Reviewers Roles and Responsibilities by Technical Review Type
[28](#__RefHeading___Toc46220949)

Technical Review Process (TEP)

# Introduction

The technical review process is a systematic evaluation of a product by
a team of qualified personnel that examines the acceptance of the
product for its intended use and identifies discrepancies from
specifications and standards. Technical reviews, as defined in Institute
of Electrical and Electronics Engineers (IEEE) Std 1220-1998, *IEEE
Standard for Application and Management of the Systems Engineering
Process*, assess the maturity of the development effort, determine
readiness to conduct acceptance testing, and determine whether the
investment should be made to continue into production (i.e., become
operational). Technical reviews may also provide recommendations of
alternatives and examination of various alternatives, as defined in IEEE
Std 1028-1997, *IEEE Standard for Software Reviews*. This allows the
Electronic Records Archive (ERA) Program Management Office (PMO) and
Program Director (PD) to review the evolving system and to approve the
transition to the next activity in the life cycle. The objectives of the
reviews, except for the Requirements Review (RR), are to assure the ERA
PMO that the design will satisfy all aspects of the requirements. The
objective of the RR, on the other hand, is to determine the adequacy of
the requirements prior to their inclusion in the Request For Proposal
(RFP).

## Purpose

The purpose of this document is to define the individual technical
review processes for the ERA PMO and the Development Contractor. This
*Technical Review Process (TEP)* document presents the technical review
process that supports the ERA technical deliverables throughout the
development and maintenance life cycle. The review products discussed in
this document will focus on such things as the adequacy of system
requirements for the RFP, the completeness of the system requirements in
terms of identification, whether the design of the system and its
comprising hardware and software satisfies all aspects of the
requirements, and the assurance of product completeness at each release
and increment. Detailed information on the release and incremental
approaches is found in the *ERA Life Cycle* *(ELC)* document. The
following reviews will be described in greater detail in **Section 2.0,
Technical Reviews**.

-   Requirements Review (RR)

-   System Requirements Review (SRR)

-   System Design Review (SDR)

-   Increment System Requirements Review (ISRR)

-   Release System Requirements Review (RSRR)

-   Preliminary Design Review (PDR)

-   Critical Design Review (CDR)

-   Test Readiness Review (TRR)

-   Operational Readiness Review (ORR)

## ERA Program Overview

ERA will be a comprehensive, systematic, and dynamic means for
preserving virtually any kind of electronic record, free from dependence
on any specific hardware or software. The ERA, when operational, will
make it easy for NARA customers to find records they want and easy for
NARA to deliver those records in formats suited to customers' needs.

## Definitions and Acronyms

The technical terms used in this plan are defined in IEEE Std
610.12-1990*, IEEE Standard Glossary of Software Engineering
Terminology.* **Table 1-1, Acronyms List,** contains a list of acronyms
used herein.

  -----------------------------------------------------------------------
  **ACRONYM**     **DEFINITION**
  --------------- -------------------------------------------------------
  AI              Action Item

  CCB             Configuration Control Board

  CDR             Critical Design Review

  CI              Configuration Item

  CM              Configuration Management

  CMP             Configuration Management Plan

  CP              Change Proposal

  CR              Change Request

  ELC             ERA Life Cycle

  ERA             Electronic Records Archive

  HWCI            Hardware Configuration Item

  ICD             Interface Control Document

  IEEE            Institute of Electrical and Electronics Engineers

  IRD             Interface Requirements Document

  ISRR            Increment System Requirements Review

  MP              Metrics Plan

  NARA            National Archives and Records Administration

  ORR             Operational Readiness Review

  PD              Program Director

  PDR             Preliminary Design Review

  PMO             Program Management Office

  QM              Quality Management

  QMP             Quality Management Plan

  RD              Requirements Document

  RFP             Request for Proposal

  RQM             Requirements Management Plan

  RR              Requirements Review

  RSRR            Release System Requirements Review

  SDR             System Design Review

  SME             Subject Matter Expert

  SRR             System Requirements Review

  Std             Standard

  TEP             Technical Review Process

  TOMP            Task Order Management Plan

  TR              Technical Review

  TRR             Test Readiness Review

  TSP             Testing Management Plan
  -----------------------------------------------------------------------

[]{#__RefHeading___Toc46220948 .anchor}Table 1‑1: Acronyms List

## References

The standards and ERA documents used to develop the TEP are described in
the sections that follow.

### Standards

The standards used in preparation of this document are listed below.

-   IEEE Std 610.12-1990, IEEE Standard Glossary of Software Engineering
    Terminology

-   IEEE Std 1028-1997, IEEE Standard for Software Reviews

-   IEEE Std 1220-1998, IEEE Standard for Application and Management of
    the Systems Engineering Process

### ERA PMO Documentation

The following ERA PMO documentation was used to support the generation
of this document.

-   ERA Life Cycle (ELC), version 1.1

-   Testing Management Plan (TSP), version 2.1

-   Metric Plan (MP), version 1.2

-   Configuration Management Plan (CMP), version 1.5

-   Quality Management Plan (QMP), version 2.0

# Technical Review Panel 

The Technical Review (TR) Panel is made up of the ERA PMO staff. The TR
Panel is a team of appropriate experts appointed by the PD to guarantee
the integrity and consistency of the technical review process. The TR
Panel is responsible for evaluating the review products, overseeing each
review, preparing minutes and a summary report, and making
recommendations to the PD for decision-making and approval.

The producer of a product begins the technical review process by
informing the TR Panel's Review Leader that the product is complete and
ready for review. The Review Leader then evaluates the product for
readiness before sending out a review agenda and distributing the
product for review. With the exception of the Requirements Review (RR)
and the Operational Readiness Review (ORR), the Development Contractor
is responsible for conducting the technical reviews while the
appropriate ERA PMO staff (e.g., system engineers, requirements
engineers, testing engineers, Quality Management (QM) specialists, and
Configuration Management (CM) specialists) and the Independent
Verification & Validation (IV&V) team will attend.

The TR Panel consists of key member roles that are present for all
reviews and advisory members that may be invited by the Review Leader.
The members filling these roles may vary, as necessary, based on the
type of review being conducted. The following key member roles are
established for technical reviews:

-   **Chair** -- The PD or a designee serves as the Chair. The Chair is
    the decision maker and determines if the review objectives have been
    met, reviews minutes and summary reports, and approves products;

-   **Review leader** -- The Review Leader serves as the facilitator for
    the review, performs administrative tasks (e.g., schedules and
    announces the review, prepares and distributes review agenda), and
    ensures planned reviews are conducted in an orderly manner;

-   **Scribe** -- The Scribe captures minutes, action items, decisions,
    and recommendations made during the review; and

-   **Technical staff** -- The designated technical representatives
    (e.g., systems engineers, requirements engineers, testing engineers,
    QM specialists, CM specialists, operations and support) review and
    evaluate the products.

The following advisory member roles can be established for technical
reviews:

-   **Management staff** -- Management (e.g., Program Manager, Executive
    Officers, and Contracting Officer's Representative) can participate
    in the technical review for the purpose of identifying issues that
    require management resolution;

-   **Risk management staff** -- Risk Officers can participate in the
    technical review for the purpose of identifying and capturing risks
    that require mitigation;

-   **IV&V Team** -- IV&V can participate in the technical review
    process for the purpose of evaluating the products, observing the
    proceedings, and raising questions as necessary; and

-   **Customer or user representative** -- Customers or user
    representatives can serve as Subject Matter Experts (SMEs). The
    Review Leader determines their participation prior to the review.

## Review Agenda

A review agenda provides the TR Panel and product producer an
understanding of what is to be accomplished during the review. Technical
reviews will be consistent with the stated input listed for each review.
Technical review announcements will require a published agenda. The
Review Leader will distribute the agenda, product, and any supporting
documentation ten (10) working days in advance to allow for adequate
review time. Minimally, the agenda will include:

-   A list of attendees,

-   A statement of objectives for the technical review,

-   The product to be examined, and

-   A current anomalies or issues list for the product.

When the TR Panel convenes, the review leader will present an overview
of the review procedures and the product. The review procedures and
product description can also be included with the technical review
announcement. Minimally, input to the technical reviews will include:

-   An agenda,

-   Documented review procedures,

-   Product undergoing review,

-   Relevant review reports, and

-   Any regulations, standards, guidelines, plans, and procedures
    against which the product is to be examined.

## Review Preparation

Each TR Panel member will examine the product and other review inputs
prior to the review.

The PMO will ensure that the review is performed as required by
contract. To this end, the PMO will:

-   Plan time and resources required for reviews, including support
    functions;

-   Provide funding and facilities required to plan, define, execute,
    and manage the reviews;

-   Provide training and orientation on review procedures; and

-   Ensure that the TR Panel possesses appropriate level of expertise
    and knowledge sufficient to comprehend the product under review.

## Review Goals and Conduct

The technical reviews will accomplish the following goals:

-   Evaluate the product and determine if:

```{=html}
<!-- -->
```
-   The product is complete;

-   The product conforms to the applicable regulations, standards,
    guidelines, plans, and procedures;

-   Changes to the product are properly completed and affect only the
    specified areas;

-   The product is acceptable for its intended use; and

-   The product is ready for the next activity in the life cycle;

```{=html}
<!-- -->
```
-   Identify system (i.e., hardware and software) anomalies;

-   Generate a list of action items;

-   Generate Change Requests (CRs) when necessary;

-   Document the review (e.g., minutes, action items, summary report);
    and

-   Provide recommendation to the PD for moving to the next phase.

After the technical review is held, the producer will update the
product(s) based on the CR process that is described in the *ERA
Configuration Management Plan (CMP)*. The producer of the product is
then responsible for turning over the marked up copies of everything
distributed at the review, the Technical Review Action Item (AI) Log,
and attendee signatures to the QM team. The QM team will track the
action items to closure. The action item log appears in **Appendix A,
Technical Review Action Item (AI) Log**. Additionally, minutes and a
Technical Review Summary Report will be generated to document the review
and submit recommendations (e.g., accept product without further
modification, reject the product due to severe errors, accept the
product provisionally and minor errors must be corrected) to the PD.
Furthermore, through consensual means the TR Panel also will recommend
proceeding to the next activity in the life cycle or not proceeding to
the next activity. The summary report format appears in **Appendix B,
Technical Review Summary Report**.

## Review Closeout

A technical review will be considered complete when the review goals
have been accomplished and the review output exists. Minimally, review
output will consist of documented evidence that identifies:

-   The TR Panel members;

-   The product reviewed;

-   Specific inputs to the review;

-   Review objectives and whether they were met;

-   A list of resolved and unresolved review product anomalies;

-   A list of management issues, technical issues, and issues that
    impact the review;

-   Action item's status (open, closed), ownership and target date (if
    open), or completion date (if closed);

-   CR status (approve, defer, or disapprove) which is controlled by the
    Configuration Control Board (CCB);

-   Any recommendations made by the TR Panel; and

-   Whether the product meets the applicable regulations, standards,
    guidelines, plans, and procedures.

# Technical Reviews

This section describes in detail the reviews listed in **Section 1.1,
Purpose**. Each technical review discussion includes the following
sections: Purpose, Input, Procedures, Exit Criteria, and Output. Refer
to the *ERA* *ELC* document for a diagram of where technical reviews fit
in the ERA life cycle processes and activities.

Technical reviews are conducted to assess the degree of completion of
technical efforts related to major milestones before proceeding with
further technical effort.

## Requirements Review

The Requirements Review (RR), which is part of the acquisition process,
is an ERA PMO activity conducted to systematically evaluate requirements
before they are affixed to the RFP.

### Purpose

The purpose of the RR is to ascertain the adequacy of the requirements
in defining the proposed characteristics and functionality of the ERA
system for inclusion in the RFP. The RR will assess each requirement for
the following reasons.

-   Understandability -- Is each individual requirement and the set of
    requirements understandable?

-   Completeness -- Do the requirements describe a complete product? Do
    they cover quality and performance characteristics?

-   Verifiability -- Can each of the requirements be verified?

-   Consistency -- Are the requirements consistent?

-   Traceability -- Will the requirements be traceable throughout the
    development life cycle?

-   Testability - Can each of the requirements be tested?

### Input

Input to the RR will include the following:

-   Requirements Document (RD), and

-   Published Agenda

### Procedures

The RR reviewers are responsible for ensuring the following*:*

-   That the refined requirements are necessary and sufficient (from
    both a functional and technical point of view) for the RFP, and

-   That any assumptions and/or issues are documented and a plan is
    established to follow-up on these items.

### Exit Criteria

The RR will be considered complete when:

-   Minutes and the review summary report are published,

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status, and

-   The PD accepts the RR Technical Review Summary Report and approves
    the RD.

### Output

Output from the RR will include the following:

-   Minutes,

-   RR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative,

```{=html}
<!-- -->
```
-   Completed Technical Review Checklist (See **Appendix C, Technical
    Review Checklist**), and

-   Approved RD.

## System Requirements Review

A System Requirements Review (SRR) resolves, finalizes, and formalizes
the requirements of systems and subsystems. In the technical review
process, the SRR follows the RR. The Development Contractor is
responsible for conducting the SRR.

### Purpose

The purpose of the SRR is to ascertain the adequacy of the Development
Contractor's efforts in focusing on the completeness of system
requirements in terms of their identification, definition, and
determination of the initial direction and progress of the Development
Contractor's system engineering management effort.

The SRR is conducted when the system functional requirements have been
decomposed and allocated to the system level design.

### Input

Input to the SRR will include the following:

-   System Requirements Document (e.g., system requirements
    specification),

-   Interface Requirements Document (IRD) (e.g., interface requirements
    specifications), and

-   Published Agenda.

### Procedures

The SRR reviewers are responsible for ensuring the following:

-   That a significant portion of the system functional requirements
    have been established,

-   That any assumptions and/or issues are documented and a plan is
    established to follow-up on these items, and

```{=html}
<!-- -->
```
-   That the approach integrates well with existing functionality.

### Exit Criteria

The SRR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status;

-   Requirements to hardware, software, and operations are allocated;
    and

-   The PD approves the System Requirements document, IRD, and accepts
    the SRR Technical Review Summary Report.

### Output

Output from the SRR will include the following:

-   Minutes,

-   SRR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative,

-   Completed Technical Review Checklist, and

-   Requirements, decomposed and allocated to the system level design.

## System Design Review

The System Design Review (SDR) is conducted to evaluate the
optimization, traceability, correlation, completeness, and risks
associated with the allocated program/design requirements, including the
corresponding test requirements in fulfilling the performance
requirements specified in the system/subsystem design description (i.e.,
functional configuration identification). In the technical review
process, the SDR follows the SRR. The Development Contractor is
responsible for conducting the SDR.

### Purpose

The purpose of the SDR is to ensure that the ERA PMO and the Development
Contractor concur that the proposed system design meets baseline
functionality and performance requirements. The key factors to evaluate
are:

-   The Development Contractor's readiness to move into the further
    development phases;

-   If risks, impacts, and mitigation plans are identified;

-   Whether design meets baseline requirements and scalability; and

-   Operational/support requirements.

The SDR is conducted when the system definition effort has proceeded to
the point where system characteristics are defined and the configuration
items are identified.

### Input

Input to the SDR will include the following:

-   System Design documents (e.g., interface design document);

```{=html}
<!-- -->
```
-   System Requirements documents;

```{=html}
<!-- -->
```
-   Published Agenda; and

```{=html}
<!-- -->
```
-   Successful completion of all CRs with "Approved for current
    release," "Immediate/ emergency change," "Deferred," or "Disapproved
    with explanation" resolution status related to the previous review.

### Procedures

The SDR reviewers are responsible for ensuring the following:

-   That system characteristics are defined and the configuration items
    are identified,

-   That any assumptions and/or issues are documented and a plan is
    established to follow-up on these items,

-   That alternative approaches have been identified and an explanation
    as to why a certain choice was made is documented,

```{=html}
<!-- -->
```
-   That the approach describes all affected system requirements and
    high level modifications, and

-   That the approach integrates well with existing functionality.

### Exit Criteria

The SDR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status; and

-   The PD approves the System Design documents, System Requirements,
    and accepts the SDR Technical Review Summary Report.

### Output

Output from the SDR will include the following:

-   Minutes,

-   SDR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Increment System Requirements Review

An Increment System Requirements Review (ISRR) resolves, finalizes, and
formalizes the requirements of systems and subsystems for the defined
increment. The ISRR is held near the beginning of each increment. The
Development Contractor is responsible for conducting the ISRR.

### Purpose

The purpose of the ISRR is to ascertain the adequacy of the Development
Contractor's efforts in focusing on the completeness of system
requirements in terms of their identification, definition, and
determination of the initial direction and progress of the Development
Contractor's system engineering management effort for the defined
increment. The ISRR is conducted when the system functional requirements
have been allocated to that increment.

### Input

Input to the ISRR will include the following:

-   System Requirements documents (e.g., system requirements
    specification),

-   IRD (e.g., interface requirements specifications), and

-   Published Agenda.

### Procedures

The ISRR reviewers are responsible for ensuring the following:

-   That the system functional requirements have been allocated to that
    increment,

-   That any assumptions and/or issues are documented and a plan is
    established to follow-up on these items, and

```{=html}
<!-- -->
```
-   That the approach integrates well with existing functionality.

### Exit Criteria

The ISRR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status;

-   Requirements to hardware, software, and operations are allocated;
    and

-   The PD approves the System Requirements documents, IRD, and accepts
    the ISRR Technical Review Summary Report.

### Output

Output from the ISRR will include the following:

-   Minutes,

-   ISRR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Release System Requirements Review

A Release System Requirements Review (RSRR) resolves, finalizes, and
formalizes the requirements of systems and subsystems for the defined
release. In the technical review process, the RSRR follows the ISRR. The
Development Contractor is responsible for conducting the RSRR.

### Purpose

The purpose of the RSRR is to ascertain the adequacy of the Development
Contractor's efforts in focusing on the completeness of system
requirements in terms of their identification, definition, and
determination of the initial direction and progress of the Development
Contractor's system engineering management effort for the defined
release. The RSRR is conducted when the system functional requirements
have been allocated to a release.

### Input

Input to the RSRR will include the following:

-   System Requirements documents (e.g., system requirements
    specification),

-   IRD (e.g., interface requirements specifications), and

-   Published Agenda.

### Procedures

The RSRR reviewers are responsible for ensuring the following:

-   That the system functional requirements have been allocated to a
    release,

-   That any assumptions and/or issues are documented and a plan is
    established to follow-up on these items, and

```{=html}
<!-- -->
```
-   That the approach integrates well with existing functionality.

### Exit Criteria

The RSRR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status;

-   Requirements to hardware, software, and operations are allocated;
    and

-   The PD approves the System Requirements documents, IRD, and accepts
    the RSRR Technical Review Summary Report.

### Output

Output from the RSRR will include the following:

-   Minutes,

-   RSRR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Preliminary Design Review

A Preliminary Design Review (PDR) is a technical review or a series of
reviews of the basic design approach for each Configuration Item (CI),
or aggregate of CIs, or for a functionally related group of CIs and will
be held prior to the start of detailed design. The PDR ensures that a
sufficient level of detail for each release has been provided to allow
detailed design to begin, and that the design meets all the functional
requirements allocated to that release. In the technical review process,
the PDR follows the RSRR. The Development Contractor is responsible for
conducting the reviews.

### Purpose

The purpose of a PDR is to give the Government its first opportunity to
closely observe and approve the Development Contractor\'s hardware and
software design. The Development Contractor is expected to describe all
design changes made with respect to the original design disclosed in the
system and to provide rationale for the changes. The Development
Contractor may also provide a hardware or hands-on demonstration of some
of the preliminary designs to better illustrate important aspects. The
PDR is conducted after preliminary design efforts, but before start of
critical design.

The PDR review will focus on:

-   Evaluating the progress, consistency, and technical adequacy of the
    selected top-level design and test approach;

-   The compatibility between software requirements and preliminary
    design;

-   The preliminary version of the operation and support documents; and

-   Determining its compatibility with performance and engineering
    specialty requirements of the Hardware Configuration Item (HWCI)
    development specification.

### Input

Input to the PDR will include the following:

-   Software Requirements documents;

-   Preliminary Design Documents;

```{=html}
<!-- -->
```
-   System Design documents, and

-   IRDs;

```{=html}
<!-- -->
```
-   Published Agenda;

-   Successful completion of all CRs with "Approved for current
    release," "Immediate/ emergency change," "Deferred," or "Disapproved
    with explanation" resolution status related to the previous review;
    and

-   Acceptance of all applicable requirements.

### Procedures

The PDR reviewers are responsible for ensuring that:

-   The description of processing is not too detailed so as to make it
    impossible to map back to the functional description presented in
    the allocated requirements. The preliminary design reflects and
    depicts the main processing flows;

-   The major relationships between modules are clearly defined;

-   All design assumptions, constraints, and issues are documented;

-   The design follows documentation standards; and

-   All related modifications have been presented and they are
    consistent with other preliminary designs and requirements, if
    applicable.

### Exit Criteria

The PDR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status;

-   The compatibility of the physical and functional interfaces is
    established; and

-   The PD approves the System Requirements documents, System Design
    documents, IRD, and accepts the PDR Technical Review Summary Report.

### Output

Output from the PDR will include the following:

-   Minutes,

-   PDR Technical Review Summary Report completed and signed by all
    attendees,

-   Final versions of all preliminary design documents,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Critical Design Review

A Critical Design Review (CDR) is a technical review or a series of
reviews of the basic design approach for each CI or aggregate of CIs or
for a functionally related group of CIs, and will be held prior to the
start of detailed design. The CDR ensures that a sufficient level of
detail for each release has been provided to allow detailed design to
begin and that the design meets all the functional requirements
allocated to that release. In the technical review process, the CDR
follows the PDR. The Development Contractor is responsible for
conducting the CDR.

### Purpose

The purpose of the CDR is to determine the acceptability of the detailed
design, performance, test characteristics (of the design solution), and
the adequacy of the operation and support documents. Further, the CDR
determines whether the critical design (of the CIs under review)
satisfies cost, schedule, and performance requirements, and establishes
detail design compatibility among the CI and other items of equipment,
facilities, computer software, and personnel.

### Input

Input to the CDR will include the following:

-   Critical Design Documents -- Final Versions,

```{=html}
<!-- -->
```
-   Preliminary Design Documents, and

-   ICDs,

```{=html}
<!-- -->
```
-   Published Agenda, and

```{=html}
<!-- -->
```
-   Successful completion of all CRs related to the previous review with
    an "Approved for current release," "Immediate/emergency change,"
    "Deferred," or "Disapproved with explanation" resolution status.

### Procedures

The CDR reviewers are responsible for ensuring that:

-   The description of processing is detailed enough to make it possible
    to map back to the functional description presented in the allocated
    requirements. The critical design must still reflect and depict the
    main processing flows;

-   The detailed relationships between modules are clearly defined;

-   All design assumptions and issues are documented;

-   The design follows documentation standards; and

```{=html}
<!-- -->
```
-   All related modifications have been presented and they are
    consistent with other critical designs and requirements, if
    applicable.

### Exit Criteria

The CDR will be considered complete when:

-   Formal identification of specific software documentation that will
    be released for coding and testing is established;

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change" resolution status, "Deferred," or
    "Disapproved with explanation"; and

-   The PD approves Critical Design Documents, Preliminary Design
    Documents, Interface Control Document, and accepts the CDR Technical
    Review Summary Report.

### Output

Output from the CDR will include the following:

-   Minutes,

-   CDR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Test Readiness Review

The Test Readiness Review (TRR) provides an independent evaluation and
assessment of the system's readiness for testing to program managers and
project engineers. TRRs will be held for each release and increment at
the completion of system testing. In the technical review process, the
TRR follows the CDR. The Development Contractor is responsible for
conducting the TRR.

### Purpose

The purpose of the TRR is to provide management with the assurance that
the product under development has reached the degree of completeness and
validity to ensure that the ERA PMO is ready to begin acceptance testing
(formal and monitored). The scope of the TRR is to inspect the test
products and test results from the completed test phase for completeness
and accuracy, and to verify that test procedures, test cases, test
scenarios, test scripts, environment, and test data have been prepared
for the next test phase.

The TRR is the Government's decision milestone in determining the
completion of unit, integration, and system tests. The Development
Contractor will demonstrate that all deficiencies were corrected or
provide satisfactory explanation to the contrary. The TRR will be
conducted on a release and incremental basis as established in the *ERA*
*Testing Management Plan (TSP)*.

### Input

Input to the TRR will include the following:

-   *ERA TSP* and an acceptance test plan (when created);

-   Published Agenda;

-   Development Contractor's TRR Package Documentation;

```{=html}
<!-- -->
```
-   Software/Interface requirements changes;

-   Design changes;

-   Test plans, test cases, procedures, and results; and

-   Problem/Change reports;

```{=html}
<!-- -->
```
-   Notification by the Development Contractor that they are ready for
    the Government to conduct the TRR; and

-   Test data has been obtained or prepared for testing.

### Procedures

The TRR reviewers are responsible for ensuring that:

-   All functions which need be tested are being tested;

-   Test cases, inputs, and actual results are documented completely;

-   Meaningful error message text and error actions are relevant; and

-   Expected results agree.

### Exit Criteria

The TRR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status;

```{=html}
<!-- -->
```
-   The PD approves the acceptance test plan, TRR Package Documentation,
    and accepts the TRR Technical Review Summary Report; and

-   All discrepancies determined by the ERA PMO to be within the scope
    of the contract have been corrected. On some occasions the ERA PMO
    may want to proceed with a conditional acceptance of the TRR.

### Output

Output from the TRR will include the following:

-   Minutes,

-   TRR Technical Review Summary Report completed and signed by all
    attendees,

-   Changes/Comments to the *ERA TSP* or acceptance test plan (when
    created),

-   Test results,

-   Corrective action plans,

-   Technical Review AI Log signed by the QM representative, and

-   Completed Technical Review Checklist.

## Operational Readiness Review

An Operational Readiness Review (ORR) is intended to determine the
status of completion of the specific actions that will be satisfactory
and accomplished prior to the PD executing an operational go-ahead
decision. In the technical review process, the ORR follows the TRR.

The ORR conducted by the ERA PMO and the Development Contractor will
support the ORR, as needed.

### Purpose

The purpose of the ORR is to accomplish, in an incremental fashion
during the development phase, initial reviews to assess the risk in
exercising the operational go-ahead decision. Timing of the incremental
ORRs is a function of program posture and is not specifically locked
into other reviews. The ORR is performed to decide if the system is in a
suitable condition to become an operational release.

The ORR verifies that necessary approved requirements documentation is
in place and that procedures, personnel, equipment, and systems support
the approved requirements. It provides the verification process that
management needs to be assured that the system is ready to operate.

### Inputs

Input to the ORR will include the following:

-   Preliminary Site Plan,

-   Facilities Requirements Assessment,

-   Training Plan,

-   Maintenance Document,

-   Repair/Replace Procedures,

-   Operations and user manuals,

-   Operational Readiness Review Document,

-   Published Agenda,

-   Transition Plan,

-   Deployment Plan,

-   Approved Operations and Support Plan,

-   Estimated level of effort and schedule requirements,

-   Established objectives and milestones for the review, and

-   Background and reference information on third-party software and
    hardware.

### Procedures

The ORR reviewers are responsible for ensuring the following:

-   That the facility conforms with applicable standards and regulatory
    requirements,

-   That the facility operates safely and efficiently, and

-   That all necessary background and reference information for the
    facility and equipment are documented.

### Exit Criteria

The ORR will be considered complete when:

-   Minutes and the review summary report are published;

-   All CRs are dispositioned with "Approved for current release,"
    "Immediate/ emergency change," "Deferred," or "Disapproved with
    explanation" resolution status; and

-   The PD approves ORR inputs and accepts the ORR Technical Review
    Summary Report.

### Output

Output from the ORR will include the following:

-   Minutes,

-   ORR Technical Review Summary Report completed and signed by all
    attendees,

-   Technical Review AI Log signed by the QM representative, and

```{=html}
<!-- -->
```
-   Completed Technical Review Checklist.

# Metrics and Measures

The ERA PMO has selected performance metrics to provide insight into the
development and operation of the ERA system with metrics collection
beginning during the development process and continuing through the
remainder of the ERA system development life cycle. Specific activities
associated with the TEP are subject to metrics collection by the QM
specialist. Metrics collection and reporting process are described in
detail in the **Metrics Collection and Use** section of the *ERA Metrics
Plan (MP)* document.

# Reviewers Roles and Responsibilities by Technical Review Type

**Table 5-1, Reviewers Roles and Responsibilities by Type of Technical
Review,** lists the primary roles and responsibilities of each reviewer
by type of technical review.

+-------------+-----------------+-------------------------------------+
| **Type of   | **Reviewers**   | **Role/Responsibilities**           |
| Review**    |                 |                                     |
+=============+=================+=====================================+
| R           | SME             | -   Review Requirements             |
| equirements |                 |                                     |
| Review      |                 | -   Ensure requirements are         |
|             |                 |     adequately defined              |
|             |                 |                                     |
|             |                 | -   Attend requirements             |
|             |                 |     review/Provide feedback         |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review Requirements             |
|             | team            |                                     |
|             | representative, | -   Attend review/Provide Feedback  |
|             | Technical       |                                     |
|             | staff, CM       | -   Ensure the requirements cover   |
|             | representative, |     major functionality             |
|             | Test team       |                                     |
|             | representative, | -   Sign Technical Review AI Log    |
|             | and IV&V team   |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review Requirements             |
|             | Representative  |                                     |
|             |                 | -   Ensure document conforms to     |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Ensure Paper package is         |
|             |                 |     complete                        |
|             |                 |                                     |
|             |                 | -   Attend review/Provide feedback  |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Ensure the review forms are     |
|             |                 |     completed and filed with the    |
|             |                 |     hard copy mark-up documents     |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
| System      | SME             | -   Review System Requirements      |
| R           |                 |                                     |
| equirements |                 | -   Ensure Development Contractor   |
| Review      |                 |     adequately defines system       |
|             |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review System Requirements      |
|             | team            |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | Technical       |     Feedback                        |
|             | staff, CM       |                                     |
|             | representative, | -   Ensure the requirements cover   |
|             | Test team       |     major functionality             |
|             | representative, |                                     |
|             | and IV&V team   | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review System Requirements      |
|             | Representative  |                                     |
|             |                 | -   Ensure document conforms to     |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Ensure document package is      |
|             |                 |     complete                        |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical review     |
|             |                 |     forms are completed and filed   |
|             |                 |     with the hard copy mark-up      |
|             |                 |     documents                       |
+-------------+-----------------+-------------------------------------+
| System      | SME             | -   Review System Design            |
| Design      |                 |                                     |
| Review      |                 | -   Ensure system design meets      |
|             |                 |     functional requirements         |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review System Design            |
|             | team            |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | Technical       |     feedback                        |
|             | staff, CM       |                                     |
|             | representative, | -   Ensure system design meets      |
|             | Test team       |     major functionality             |
|             | representative, |                                     |
|             | and IV&V team   | -   Ensure that a technical         |
|             |                 |     understanding of requirements   |
|             |                 |     has been reached and technical  |
|             |                 |     direction is provided           |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Tech Support    | -   Review System Design            |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Ensure that the allocated       |
|             |                 |     requirements represent a        |
|             |                 |     complete and optimal synthesis  |
|             |                 |     of the system requirements      |
|             |                 |                                     |
|             |                 | -   Ensure that the technical       |
|             |                 |     program risks are identified,   |
|             |                 |     ranked, avoided, and reduced    |
|             |                 |                                     |
|             |                 | -   Gain understanding of program   |
|             |                 |     functionality                   |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review System design            |
|             | Representative  |                                     |
|             |                 | -   Ensure system design conforms   |
|             |                 |     to Project Standards            |
|             |                 |                                     |
|             |                 | -   Attend Technical review/        |
|             |                 |     Provide feedback                |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Ensure the Technical Review AI  |
|             |                 |     Log is complete and filed with  |
|             |                 |     the hard copy mark-up documents |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
| Increment   | SME             | -   Review System Requirements      |
| System      |                 |                                     |
| R           |                 | -   Ensure Development Contractor   |
| equirements |                 |     adequately defines system       |
| Review      |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review System Requirements      |
|             | team            |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | Technical       |     Feedback                        |
|             | staff, CM       |                                     |
|             | representative, | -   Ensure the requirements cover   |
|             | Test team       |     major functionality             |
|             | representative, |                                     |
|             | and IV&V team   | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review System Requirements      |
|             | Representative  |                                     |
|             |                 | -   Ensure document conforms to     |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Ensure document package is      |
|             |                 |     complete                        |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical review     |
|             |                 |     forms are completed and filed   |
|             |                 |     with the hard copy mark-up      |
|             |                 |     documents                       |
+-------------+-----------------+-------------------------------------+
| Release     | SME             | -   Review System Requirements      |
| System      |                 |                                     |
| R           |                 | -   Ensure Development Contractor   |
| equirements |                 |     adequately defines system       |
| Review      |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review System Requirements      |
|             | team            |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | Technical       |     Feedback                        |
|             | staff, CM       |                                     |
|             | representative, | -   Ensure the requirements cover   |
|             | Test team       |     major functionality             |
|             | representative, |                                     |
|             | and IV&V team   | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review System Requirements      |
|             | Representative  |                                     |
|             |                 | -   Ensure document conforms to     |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Ensure document package is      |
|             |                 |     complete                        |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical review     |
|             |                 |     forms are completed and filed   |
|             |                 |     with the hard copy mark-up      |
|             |                 |     documents                       |
+-------------+-----------------+-------------------------------------+
| Preliminary | SME             | -   Review Preliminary Design       |
| Design      |                 |                                     |
| Review      |                 | -   Ensure preliminary design meets |
|             |                 |     functional requirements         |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review Preliminary Design       |
|             | team            |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | Technical       |     feedback                        |
|             | staff, CM       |                                     |
|             | representative, | -   Ensure design meets major       |
|             | Test team       |     functionality                   |
|             | representative, |                                     |
|             | and IV&V team   | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Tech Support    | -   Review Preliminary Design       |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Ensure common routines are      |
|             |                 |     being used accurately           |
|             |                 |                                     |
|             |                 | -   Gain understanding of program   |
|             |                 |     functionality                   |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review Preliminary design       |
|             | Representative  |                                     |
|             |                 | -   Ensure design conforms to       |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical Review AI  |
|             |                 |     Log is complete and filed with  |
|             |                 |     the hard copy mark-up documents |
+-------------+-----------------+-------------------------------------+
| Critical    | SME             | -   Review Critical design          |
| Design      |                 |                                     |
| Review      |                 | -   Ensure critical design          |
|             |                 |     adequately addresses functional |
|             |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Verify the adequacy and         |
|             |                 |     completeness of the critical    |
|             |                 |     design                          |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review Critical design          |
|             | team            |                                     |
|             | representative, | -   Ensure Critical design          |
|             | Technical       |     adequately addresses functional |
|             | staff, CM       |     requirements                    |
|             | representative, |                                     |
|             | Test team       | -   Review test results from items  |
|             | representative, |     fabricated during this phase    |
|             | and IV&V team   |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Tech Support    | -   Review Critical design          |
|             |                 |                                     |
|             |                 | -   Ensure Critical design          |
|             |                 |     accurately addresses processing |
|             |                 |                                     |
|             |                 | -   Verify that interfaces between  |
|             |                 |     system elements are properly    |
|             |                 |     designed                        |
|             |                 |                                     |
|             |                 | -   Verify that all documentation   |
|             |                 |     has been completed              |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Gain understanding of program   |
|             |                 |     functionality                   |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review Critical Design          |
|             | Representative  |                                     |
|             |                 | -   Ensure Critical design          |
|             |                 |     adequately addresses functional |
|             |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Ensure Critical design package  |
|             |                 |     is complete and conforms to     |
|             |                 |     Project Standards               |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical Review AI  |
|             |                 |     Log is complete and filed with  |
|             |                 |     the hard copy mark-up documents |
+-------------+-----------------+-------------------------------------+
| Test        | SME             | -   Review prior test results       |
| Readiness   |                 |                                     |
| Review      |                 | -   Review/identify known problems  |
|             |                 |                                     |
|             |                 | -   Review test procedures          |
|             |                 |                                     |
|             |                 | -   Verify software build           |
|             |                 |                                     |
|             |                 | -   Ensure tool configuration and   |
|             |                 |     calibration status              |
|             |                 |                                     |
|             |                 | -   Ensure that there is adequate   |
|             |                 |     test coverage of the functional |
|             |                 |     requirements                    |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review Test Readiness materials |
|             | team            |     and Results                     |
|             | representative, |                                     |
|             | Technical       | -   Ensure adequate test converge   |
|             | staff, CM       |     of the functional requirements  |
|             | representative, |                                     |
|             | Test team       | -   Attend Technical review/Provide |
|             | representative, |     feedback                        |
|             | and IV&V team   |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Tech Support    | -   Review Test Readiness review    |
|             |                 |     materials and Results           |
|             |                 |                                     |
|             |                 | -   Ensure Test Readiness review    |
|             |                 |     accurately reflects processing  |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Gain understanding of program   |
|             |                 |     functionality                   |
|             |                 |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review Test Readiness review    |
|             | Representative  |     materials and Results           |
|             |                 |                                     |
|             |                 | -   Ensure adequate test converge   |
|             |                 |     of the functional requirements  |
|             |                 |                                     |
|             |                 | -   Ensure Test Readiness review    |
|             |                 |     package is complete and         |
|             |                 |     conforms to Project Standards   |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical Review AI  |
|             |                 |     Log is complete and filed with  |
|             |                 |     the hard copy mark-up documents |
+-------------+-----------------+-------------------------------------+
| Operational | SME             | -   Review Operational Readiness    |
| Readiness   |                 |     review materials and Results    |
| Review      |                 |                                     |
|             |                 | -   Ensure Operational readiness    |
|             |                 |     review adequately estimates     |
|             |                 |     level of effort                 |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Development     | -   Review Operational Readiness    |
|             | team            |     materials and Results           |
|             | representative, |                                     |
|             | Technical       | -   Ensure Operational Readiness    |
|             | staff, CM       |     review has an approved          |
|             | representative, |     Implementation Plan             |
|             | Test team       |                                     |
|             | representative, | -   Attend Technical review/Provide |
|             | and IV&V team   |     feedback                        |
|             |                 |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | Tech Support    | -   Review Operational Readiness    |
|             |                 |     review materials and Results    |
|             |                 |                                     |
|             |                 | -   Ensure Operational Readiness    |
|             |                 |     review accurately reflects      |
|             |                 |     processing                      |
|             |                 |                                     |
|             |                 | -   Ensure Operational Readiness    |
|             |                 |     review meets objectives and     |
|             |                 |     milestones                      |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Gain understanding of program   |
|             |                 |     functionality                   |
|             |                 |                                     |
|             |                 | ```{=html}                          |
|             |                 | <!-- -->                            |
|             |                 | ```                                 |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
+-------------+-----------------+-------------------------------------+
|             | QM              | -   Review Operational Readiness    |
|             | Representative  |     review materials and Results    |
|             |                 |                                     |
|             |                 | -   Ensure Operational Readiness    |
|             |                 |     review defines team membership  |
|             |                 |                                     |
|             |                 | -   Ensure Operational Readiness    |
|             |                 |     review package is complete and  |
|             |                 |     conforms to Project Standards   |
|             |                 |                                     |
|             |                 | -   Attend Technical review/Provide |
|             |                 |     feedback                        |
|             |                 |                                     |
|             |                 | -   Sign Technical Review AI Log    |
|             |                 |                                     |
|             |                 | -   Track AIs to closure            |
|             |                 |                                     |
|             |                 | -   Sign Technical Review Summary   |
|             |                 |     Report                          |
|             |                 |                                     |
|             |                 | -   Ensure the Technical Review AI  |
|             |                 |     Log is complete and filed with  |
|             |                 |     the hard copy mark-up documents |
+-------------+-----------------+-------------------------------------+

[]{#__RefHeading___Toc46220949 .anchor}Table 5-1: Reviewers Roles and
Responsibilities by Technical Review Type

# Plan Maintenance

The ERA PD is responsible for this plan. As a part of process
improvement (e.g., IV&V assessments, lessons learned, QM assessments),
the TEP and the overall technical review approach will continue to
evolve. The plan will be updated as needed to maintain current and
sufficient CM activities. The plan will be placed under CM control
following its initial approval by the ERA PMO. Updates to the TEP will
be controlled by the CCB.

[]{#__RefHeading___Toc46221252 .anchor}APPENDIX A: ERA Technical Review
Action Item Log

<table>
<colgroup>
<col style="width: 10%" />
<col style="width: 11%" />
<col style="width: 9%" />
<col style="width: 10%" />
<col style="width: 0%" />
<col style="width: 10%" />
<col style="width: 0%" />
<col style="width: 10%" />
<col style="width: 0%" />
<col style="width: 10%" />
<col style="width: 0%" />
<col style="width: 10%" />
<col style="width: 15%" />
</colgroup>
<tbody>
<tr class="odd">
<td><strong>Start Time</strong></td>
<td></td>
<td>a.m./p.m.</td>
<td></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td><h3 id="section" class="unnumbered"> </h3>
<p><strong>Date</strong></p></td>
<td></td>
</tr>
<tr class="even">
<td><strong>End Time</strong></td>
<td></td>
<td>a.m./p.m.</td>
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"><strong>Project</strong></td>
<td></td>
</tr>
</tbody>
</table>

Section 1: Work Product Information

  ------------- ----------------------- ----------- ----------------------
  **Product                                         
  Reviewed:**                                       

  **Author:**                                       

  **Review                                          
  Scribed By:**                                     
  ------------- ----------------------- ----------- ----------------------

-   Product Type (Check one): \_\_\_\_ Requirements, \_\_\_\_ Design,
    \_\_\_\_Code, \_\_\_\_ Test, \_\_\_\_Operations

```{=html}
<!-- -->
```
-   Documents only: \# of pages \_\_\_\_\_\_\_\_\_

Other size measurement (optional):
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

We concur that this Technical Review has been completed.

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\
(Team Leader) (Author) (QM Specialist)

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\
(Reviewer) (Reviewer) (Reviewer)

*Please submit this completed form with the review subject document or
artifact to the QM Specialist for this project.*

Section 2: Defects, Issues, and Action Items

+----+---+---------------------------+----------+--------+-------+---+
| ** | * | **Problem                 | **Se     | **Revi | **Ass | * |
| It | * | Description/Resolution**  | verity** | ewer** | igned | * |
| em | P |                           |          |        | To**  | C |
| N  | a |                           | (check   |        |       | o |
| o. | g |                           | one)     |        |       | m |
| ** | e |                           |          |        |       | p |
|    | \ |                           |          |        |       | * |
|    | N |                           |          |        |       | * |
|    | o |                           |          |        |       |   |
|    | . |                           |          |        |       |   |
|    | * |                           |          |        |       |   |
|    | * |                           |          |        |       |   |
+====+===+===========================+==========+========+=======+===+
| *  |   |                           | -        |        |       |   |
| *1 |   |                           | Critical |        |       |   |
| ** |   |                           |          |        |       |   |
|    |   |                           | -   High |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Inte |        |       |   |
|    |   |                           | rmediate |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Low  |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Action |        |       |   |
|    |   |                           |     Item |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Defect |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |    Issue |        |       |   |
+----+---+---------------------------+----------+--------+-------+---+
| *  |   |                           | -        |        |       |   |
| *2 |   |                           | Critical |        |       |   |
| ** |   |                           |          |        |       |   |
|    |   |                           | -   High |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Inte |        |       |   |
|    |   |                           | rmediate |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Low  |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Action |        |       |   |
|    |   |                           |     Item |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Defect |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | ``       |        |       |   |
|    |   |                           | `{=html} |        |       |   |
|    |   |                           | <!-- --> |        |       |   |
|    |   |                           | ```      |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |    Issue |        |       |   |
+----+---+---------------------------+----------+--------+-------+---+
| *  |   |                           | -        |        |       |   |
| *3 |   |                           | Critical |        |       |   |
| ** |   |                           |          |        |       |   |
|    |   |                           | -   High |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Inte |        |       |   |
|    |   |                           | rmediate |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -   Low  |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Action |        |       |   |
|    |   |                           |     Item |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |   Defect |        |       |   |
|    |   |                           |          |        |       |   |
|    |   |                           | ``       |        |       |   |
|    |   |                           | `{=html} |        |       |   |
|    |   |                           | <!-- --> |        |       |   |
|    |   |                           | ```      |        |       |   |
|    |   |                           | -        |        |       |   |
|    |   |                           |    Issue |        |       |   |
+----+---+---------------------------+----------+--------+-------+---+

> Columns: **Item#** = sequential number of action item. **Page** = Page
> number(s) of work product where the problem was found.
>
> **Sev**=De Defect Severity: (Critical), (High), (Intermediate), (Low).
> **Comp**(lete) = enter "Yes" or "100%" when complete.

[]{#__RefHeading___Toc46221253 .anchor}APPENDIX B: Technical Review
Summary Report

[Section 1: Review Identification]{.underline}

Start Time \_\_\_\_\_\_\_\_\_\_\_\_\_ a.m./p.m.
Project\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

End Time\_\_\_\_\_\_\_\_\_\_\_\_\_\_ a.m./p.m.
Date\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Location\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Review Scribed By:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

[Section 2: Product Identification]{.underline}

Material Reviewed
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Producer: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Product Type (check one): \_\_\_\_Requirements, \_\_\_\_\_ Design,
\_\_\_\_\_ Code, \_\_\_\_\_Test, \_\_\_\_\_ Operations

[Section 3: Product Appraisal]{.underline}

Accepted: as is ( ) with minor modification ( ) Recommendation: proceed
to next life cycle activity ( )

Not Accepted: major revision ( ) minor revision ( ) do not proceed to
next life cycle activity ( )

[Section 4: Supplementary Material Attached]{.underline}

Action Item Log ( ) Minutes ( )

Other
(describe)\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Technical Review Panel:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

(Chair) (Producer) (Scribe)

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

(QM Specialist) (Reviewer) (Reviewer)

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

(Reviewer) (Reviewer) (Reviewer)

[]{#__RefHeading___Toc46221254 .anchor}APPENDIX C: Technical Review
Checklist

**[Disclaimer]{.underline}: This checklist is not intended to be
comprehensive, but to convey a representative set of checklist
questions. Checklist refinement will likely occur when the technical
review process begins.**

  ----------------------------------------------------------------------------------------------
  TECHNICAL                                 
  REVIEW                                    
  PROCESS                                   
  CHECKLIST                                 
  ----------- -------- ---------- --------- ----------------------------------------------------
  **Yes**     **No**   **Part**   **N/A**   GENERAL CONTENT

                                            Have all of the pre-conditions been met?

                                            Has the review been scheduled?

                                            Has an agenda been prepared and distributed?

                                            Is the product to be reviewed ready for review?

                                            Have all action items been satisfied and corrections
                                            made?

                                            Has an in-house technical review been conducted?

                                            Were action items captured and due dates assigned?

                                            Has the reviewer had adequate time to review item in
                                            question?

  **Yes**     **No**   **Part**   **N/A**   REQUIREMENTS REVIEW

                                            Are requirements at a low enough level of detail to
                                            sufficiently develop preliminary designs?

                                            Are the requirements consistent with each other?

                                            Are the requirements modular? Can they accommodate
                                            change?

                                            Are there duplicate requirements?

                                            Are the requirements traceable?

                                            Are the requirements needed by the customer?

                                            Are the requirements testable?

                                            Are the requirements realistic?

  **Yes**     **No**   **Part**   **N/A**   SYSTEM REQUIREMENTS REVIEW

                                            Are system requirements at a low enough level of
                                            detail to sufficiently develop preliminary designs?

                                            Are all assumptions and/or issues documented?

                                            Are all constraints identified?

                                            Have alternative approaches been identified and an
                                            explanation as to why a certain choice was made?

                                            Do system requirements satisfy ERA's expectations?

                                            Are all system requirements traceable to a function
                                            or functions defined in the System Architecture?

                                            Have all system requirements been validated in
                                            accordance with the PMO system guidance?

                                            Is each interface traceable to an associated set of
                                            requirements?

                                            Is the functional analysis complete to a level
                                            equivalent to system requirements development?

  **Yes**     **No**   **Part**   **N/A**   SYSTEM DESIGN REVIEW

                                            Has the RD been updated per SRR results and
                                            baselined by the project for SDR?

                                            Have external interface requirements been described
                                            in the IRD?

                                            Has ERA architecture been documented in the System
                                            Design documents?

  **Yes**     **No**   **Part**   **N/A**   INCREMENT SYSTEM REQUIREMENTS REVIEW

                                            Are system requirements at a low enough level of
                                            detail to sufficiently develop preliminary designs
                                            for this increment?

                                            Are all assumptions and/or issues documented?

                                            Have alternative approaches been identified and an
                                            explanation as to why a certain choice was made?

                                            Do system requirements satisfy ERA's expectations
                                            for this increment?

                                            Are all system requirements traceable to a function
                                            or functions defined in the System Architecture?

                                            Have all system requirements been validated in
                                            accordance with the PMO system guidance?

                                            Is each interface traceable to an associated set of
                                            requirements?

                                            Is the functional analysis complete to a level
                                            equivalent to system requirements development?

  **Yes**     **No**   **Part**   **N/A**   RELEASE SYSTEM REQUIREMENTS REVIEW

                                            Are system requirements at a low enough level of
                                            detail to sufficiently develop preliminary designs
                                            for this release?

                                            Are all assumptions and/or issues documented?

                                            Have alternative approaches been identified and an
                                            explanation as to why a certain choice was made?

                                            Do system requirements satisfy ERA's expectations
                                            for this release?

                                            Are all system requirements traceable to a function
                                            or functions defined in the System Architecture?

                                            Have all system requirements been validated in
                                            accordance with the PMO system guidance?

                                            Is each interface traceable to an associated set of
                                            requirements?

                                            Is the functional analysis complete to a level
                                            equivalent to system requirements development?

  **Yes**     **No**   **Part**   **N/A**   PRELIMINARY DESIGN REVIEW

                                            Is the processing description detailed enough to
                                            make it possible to map back to the functional
                                            description in the requirements?

                                            Does the draft unit test plan cover appropriate
                                            functionality?

                                            Are the major relationships between modules clearly
                                            defined?

                                            Are all design assumptions and issues documented?

                                            Does the design follow documentation standards?

                                            Have all related modifications been presented and
                                            consistent with other preliminary designs and
                                            requirements?

  **Yes**     **No**   **Part**   **N/A**   CRITICAL DESIGN REVIEW

                                            Is the processing description detailed enough to
                                            make it possible to map back to the functional
                                            description in the requirements?

                                            Does the integration test plan cover major
                                            functionality?

                                            Are the detailed relationships between modules
                                            clearly defined?

                                            Are all design assumptions and issues documented?

                                            Does the design follow documentation standards?

                                            Have all related modifications been presented and
                                            consistent with other critical designs and
                                            requirements?

                                            Have all system and subsystem test procedures been
                                            completed?

  **Yes**     **No**   **Part**   **N/A**   TEST READINESS REVIEW

                                            Are all functions (that need be tested) being
                                            tested?

                                            Has the test environment been configured?

                                            Have all test tools been demonstrated prior to use?

                                            Has all test equipment been calibrated?

                                            Have all test procedure changes been documented and
                                            agreed to?

                                            Have all software versions been identified and
                                            verified?

                                            Are test cases, test scenarios, inputs, expected
                                            results, and actual results being documented
                                            completely?

                                            Are error message text and error actions relevant?

                                            Does the program, as written, properly utilize
                                            inputs (outputs)?

                                            Has all test data been obtained/built for testing?

                                            Are the software components ready for turnover to
                                            the test group?

                                            Are resources available for testing?

  **Yes**     **No**   **Part**   **N/A**   OPERATIONAL READINESS REVIEW

                                            Is the facility constructed in accordance with the
                                            approved design?

                                            Can the facility be operated safely and efficiently?

                                            Will the facility be operated, maintained, and
                                            supported by trained and competent personnel?

                                            Will the facility be designed and operated in
                                            conformance with applicable standards and regulatory
                                            requirements?

                                            Are all activities formally and adequately
                                            documented?

                                            Is the Staffing plan adequate for the scope of work?

                                            Is the operational/implementation schedule
                                            reasonable?

                                            Is the existing hardware, with planned upgrades,
                                            sufficient for the task?
  ----------------------------------------------------------------------------------------------