# *DEVELOPMENT PHASE*

# 5.1 Introduction

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

## 5.1.1 Purpose

During the Development Phase the Project Manager and the System
Development Manager work closely together to:

a.  Develop, integrate, and test the AIS,

b.  Update and finalize plans to deploy the AIS, and

c.  Complete business transition planning and initiate business
    transition activities.

## 5.1.2 Overview

The Development Phase is divided into two stages: Construction and
Production Readiness. In the Construction Stage, the System Development
Manager ensures that all AIS subsystems, modules, and components are
fully documented, have been coded and tested, and that identified
discrepancies have been corrected. The Construction Stage is completed
when the Technical Review Board provides approval at Test Readiness
Review. Earlier test results are reviewed and operational support
arrangements are finalized during the Production Readiness Stage. The
Production Readiness Stage ends with successful completion of the
Production Readiness Review. This review affirms that Acceptance Testing
has been successfully completed and that the AIS is suitable for
production use.

## 5.1.3 Tasks

The tasks to be completed during the Development Phase are:

a.  Allocate resources to support the current project plan.

b.  Complete business transition planning, and begin business transition
    activities.

c.  Perform component provisioning of reusable components.

d.  Code software that is not commercial off-the-shelf (COTS) and is not
    generated by the CASE tools.

e.  Perform module construction and application assembly of reusable
    components, and complete integration.

f.  Complete acquiring equipment and approved COTS software for the AIS.

g.  Complete AIS support documentation, including training.

h.  Perform system testing activities to include unit and integration
    testing, performance, and stress testing.

i.  Update the Configuration Management Plan.

j.  Perform a Configuration Management Build.

k.  Update the Economic Analysis.

l.  Update the Project Management, AIS Development, and Quality
    Assurance Plans.

m.  Update support plans.

n.  Develop the Production Installation and Data Conversion Plans.

o.  Update the technical architecture with details of the physical
    implementation.

p.  Perform Formal Qualification Testing[^1] (FQT) and Beta Testing[^2].

q.  Obtain approval to install the AIS into work areas.

r.  Software development environment planning activities should be
    defined and submitted to OSDM/SDI.

## 5.1.4 Activities and Documentation

Development Phase activities and documentation requirements as
summarized in the following table must conform to the indicated
Technical Standard and Guidelines or other standards as noted. Work
products included in the Product Baselines are identified under the
"Baseline" heading in Table 5.1.4. Published standards and guidelines
may be augmented by tailoring and documented in the Quality Assurance
Plan.

+---------------------+-------+-----------+-----+----+----+---+-----+
| Work Product        | TSG   | Who's     | M   | Sh | Mu | B | M   |
|                     |       |           | ust | ou | st | a | ust |
|                     | /     | Re        |     | ld |    | s |     |
|                     | Sta   | sponsible | Cre |    | Up | e | Co  |
|                     | ndard |           | ate | Up | da | - | mpl |
|                     |       |           |     | da | te | l | ete |
|                     |       |           |     | te |    | i |     |
|                     |       |           |     |    |    | n |     |
|                     |       |           |     |    |    | e |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Business Case       |       | ##        |     |    |    | X |     |
|                     |       | # Program |     |    |    |   |     |
|                     |       |           |     |    |    |   |     |
|                     |       | Sponsor,  |     |    |    |   |     |
|                     |       |           |     |    |    |   |     |
|                     |       | Project   |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Project Management  | I     | Project   |     |    | X  |   |     |
| Plan and supporting | T-212 |           |     |    |    |   |     |
| baseline project    | .2-01 | Manager,  |     |    |    |   |     |
| schedules           |       |           |     |    |    |   |     |
|                     |       | System    |     |    |    |   |     |
|                     |       | De        |     |    |    |   |     |
|                     |       | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| AIS Project Quality | I     | Office of |     | X  |    |   |     |
| Assurance Plan      | T-212 | System    |     |    |    |   |     |
|                     | .2-04 | Product   |     |    |    |   |     |
|                     |       | Assurance |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Data Management     | I     | Office of |     | X  |    |   | X   |
| Plan                | T-212 | Data      |     |    |    |   |     |
|                     | .2-05 | M         |     |    |    |   |     |
|                     |       | anagement |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Data Conversion     | I     | System    | X   |    |    |   | X   |
| Plan                | T-212 | De        |     |    |    |   |     |
|                     | .2-05 | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| AIS Configuration   | I     | Office of |     | X  |    |   | X   |
| Management Plan     | T-212 | System    |     |    |    |   |     |
|                     | .2-06 | Product   |     |    |    |   |     |
|                     |       | Assurance |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Concept of          | I     | Program   |     |    | X  |   | X   |
| Operations          | T-212 | Sponsor,  |     |    |    |   |     |
|                     | .2-11 | Project   |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Detailed Design     | I     | System    |     |    | X  |   | X   |
|                     | T-212 | De        |     |    |    |   |     |
|                     | .2-12 | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Economic Analysis   | I     | Project   |     |    | X  |   | X   |
|                     | T-212 | Manager,  |     |    |    |   |     |
|                     | .2-13 | System    |     |    |    |   |     |
|                     |       | De        |     |    |    |   |     |
|                     |       | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Business Transition | I     | Program   |     | X  |    |   | X   |
| Plan                | T-212 | Sponsor,  |     |    |    |   |     |
|                     | .3-05 | Project   |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Requirements        | I     | System    |     | X  |    | X | X   |
| Specification       | T-212 | De        |     |    |    |   |     |
|                     | .3-10 | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Test Plan and       | I     | Office of |     |    | X  | X | X   |
| Materials           | T-212 | System    |     |    |    |   |     |
|                     | .3-01 | Product   |     |    |    |   |     |
|                     |       | Assurance |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Test Specifications | I     | Office of | X   |    |    |   | X   |
| and Procedures      | T-212 | System    |     |    |    |   |     |
|                     | .3-01 | Product   |     |    |    |   |     |
|                     |       | Assurance |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Training Plan and   | I     | System    |     |    | X  | X |     |
| Materials           | T-212 | De        |     |    |    |   |     |
|                     | .3-02 | velopment |     |    |    |   |     |
|                     |       | Manager   |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+
| Requirements        | I     | Office of |     | X  |    |   |     |
| Traceability Matrix | T-212 | System    |     |    |    |   |     |
|                     | .3-11 | Product   |     |    |    |   |     |
|                     |       | Assurance |     |    |    |   |     |
+---------------------+-------+-----------+-----+----+----+---+-----+

**Table 5.1.4 Development Phase Activities and Documentation
Requirements**

+----------------------+-------+----------+----+----+----+---+-----+
| Work Product         | TSG   | Who's    | Mu | Sh | Mu | B | M   |
|                      |       |          | st | ou | st | a | ust |
|                      | /     | Res      |    | ld |    | s |     |
|                      | Sta   | ponsible | Cr |    | Up | e | Co  |
|                      | ndard |          | ea | Up | da | - | mpl |
|                      |       |          | te | da | te | l | ete |
|                      |       |          |    | te |    | i |     |
|                      |       |          |    |    |    | n |     |
|                      |       |          |    |    |    | e |     |
+----------------------+-------+----------+----+----+----+---+-----+
| Users Manual and/or  | I     | System   | X  |    |    |   | X   |
| Guide                | T-212 | Dev      |    |    |    |   |     |
|                      | .4-13 | elopment |    |    |    |   |     |
|                      |       | Manager  |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+
| Infrastructure       | I     | Office   | X  |    |    |   | X   |
| Disaster Recovery    | T-212 | of       |    |    |    |   |     |
| Plan                 | .2-08 | System   |    |    |    |   |     |
|                      |       | Network  |    |    |    |   |     |
|                      |       | Ma       |    |    |    |   |     |
|                      |       | nagement |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+
| Programmer           | I     | System   | X  |    |    |   | X   |
| Maintenance Manual   | T-212 | Dev      |    |    |    |   |     |
|                      | .4-15 | elopment |    |    |    |   |     |
|                      |       | Manager  |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+
| CM Build             | I     | System   | X  |    |    |   | X   |
| Instructions with    | T-212 | Dev      |    |    |    |   |     |
| Version Description  | .2-06 | elopment |    |    |    |   |     |
| Document             |       | Manager  |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+
| Production           | I     | System   | X  |    |    |   | X   |
| Installation Plan    | T-212 | Dev      |    |    |    |   |     |
|                      | .5-01 | elopment |    |    |    |   |     |
|                      |       | Manager  |    |    |    |   |     |
|                      | 01    |          |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+
| Operational Support  | I     | System   |    |    |    |   | X   |
| Plan                 | T-212 | Dev      |    |    |    |   |     |
|                      | .5-01 | elopment |    |    |    |   |     |
|                      |       | Manager  |    |    |    |   |     |
+----------------------+-------+----------+----+----+----+---+-----+

**Table 5.1.4 Development Phase Activities and Documentation
Requirements (Concluded)**

# 5.2 System Development

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

## 5.2.1 Software Development and Construction

Application code, databases, database conversion software, interfaces,
and code to integrate COTS products into the AIS are constructed during
the Development Phase. The following points apply to software
development.

a.  To improve software documentation and maintainability, whenever
    possible, changes to AIS application code will be implemented using
    USPTO standard CASE tool procedures. For later enhancement or
    maintenance, source code that had been automatically generated will
    be regenerated and not manually modified.

b.  All code will be designed, documented, and developed using modern
    software development concepts, tools, and techniques. The supporting
    documentation will be reviewed by the Office of System Product
    Assurance for accuracy, efficiency, completeness, and ease of use.

c.  Developers will thoroughly document and test all software that
    performs automated conversion of data from existing to new
    databases. The supporting documentation will be reviewed by the
    Office of System Product Assurance for accuracy, efficiency,
    completeness, and ease of use.

d.  Using the component specification, the component provisioning task
    delivers an executing and tested implementation of the component.
    Each component implementation is designed, coded, and tested. For
    each reusable CASE component:

-   The component is specified in the CASE encyclopedia.

-   The component implementation model is constructed to meet the
    component specification model.

-   The component executable is generated from the component
    implementation model.

e.  The component publishing task places the component specification in
    a USPTO Enterprise Repository component catalog that should be
    browsed for review in future projects. Publishing produces an
    installation package to deploy the component into an AIS development
    environment. The entire model specification, including the data and
    behavior of the component, should be published for browsing.

f.  TRM designated programming languages will be used in all USPTO
    multi-user applications. Nonstandard, high-order programming
    languages, as are commonly associated with COTS products, may be
    used for single user applications or where it can be shown to be
    more cost-effective for multi-user applications.

g.  To help ensure code portability and scalability, developers will not
    use vendor-unique extensions unless it can be shown to be
    cost-effective over the life of the application. Waiver requests to
    use vendor-unique extensions must be submitted to the Technical
    Review Board for approval. Waivers are not required to use:

-   COTS software packages and advanced software technology that is not
    modified or maintained by the USPTO and can operate on the USPTO
    information technology infrastructure.

-   Programming languages that require installation of vendor-provided
    updates to COTS software. Use of such languages shall be restricted
    to implementing the vendor updates.

## 5.2.2 Integration

Hardware, software (including COTS software), and documentation
components which have been individually tested are assembled into
functional subsystems and systems in accordance with the project\'s
quality assurance and configuration management plans. Application
components must be integrated. Either part or all of an entire
application may be built by assembling components. The component
verification activity includes installation of components into the
environment where they will reside when assembled into an application to
verify that installation procedures work. The System Development Manager
will complete and update all system design and support documentation in
accordance with the applicable Technical Standards and Guidelines.
Integration may be performed iteratively with increasingly larger and
more complex combinations of components. The steps in which integration
will occur shall be documented in the project\'s AIS project management
and test plans. Integration is completed when the entire AIS has been
assembled, integration level testing has been performed, and integration
test results have been approved by the Technical Review Board at Test
Readiness Review.

# 5.3 Testing and Reviews

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

Unit, integration, and independent acceptance testing activities are
performed during the Development Phase. Unit and integration testing are
performed under the direction of the Project Manager and System
Development Manager. Independent acceptance testing is performed
independently from the developing organization and is coordinated by the
Office of System Product Assurance. The independent acceptance testing
activities performed during the Development Phase are Formal
Qualification Testing and Beta Testing. Defects uncovered by acceptance
testing may lead to changes in the work products created in the Concept,
Detailed Analysis and Design, and Development Phases. The technical
standards and guidelines for those phases apply, as needed, during the
Development Phase. As much as possible, acceptance testing will be
performed in a test environment that duplicates the production
environment. The Technical Review Board reviews the results of each
test. Additional information on testing is provided in the Testing TSG,
IT-212.3-01.

## 5.3.1 Design Characteristics

Each program module will be separately tested and verified to ensure
that all applicable business requirements and module interface
specifications are addressed, and that all unit products conform to
design specifications. Unit testing should verify that every logical
path through the pseudo-code (or detailed module description) is
implemented and functions as designed, and that no unit level code is
accepted that is not designed and documented). All errors and
discrepancies noted will be corrected.

## 5.3.2 Independent Security Testing

The Office of System Product Assurance and the Office of Information
Systems Security will conduct independent system security testing during
the Development Phase. System security testing includes both the testing
of the particular AIS components, including both purchased and "in-house
developed" components, and the testing of the entire AIS. Security
management, physical facilities, personnel, procedures, the use of
commercial or COTS products, the use of OCIO services (such as PTOnet
services), and contingency plans are examples of security areas that
affect the entire AIS. Because Project Managers or System Development
Managers usually specify or determine requirements relating to these
security areas before or after the Development Phase, additional
security testing is usually necessary throughout the life of the system.
The Operational Support TSG, IT-212.5-01, and the Testing TSG,
IT-212.3-01, provide additional information about testing AIS security
requirements.

## 5.3.3 Peer Reviews

Peer reviews will be conducted on designated critical AIS projects to
identify and remove defects from the products early and efficiently.
Peer reviews can be particularly effective when used to review the work
products and documentation developed during the Detailed Analysis and
Design and Development phases of the life cycle. AIS or infrastructure
project development documentation will be reviewed in accordance with
the Quality Assurance TSG, IT-212.2-04. For designated AIS projects,
reviews, inspections and walk-throughs will be scheduled, conducted,
tracked, and the results reported in accordance with the project\'s
Quality Assurance Plan.

## 5.3.4 Test Readiness Review

This review is conducted at the end of the Development Phase to
determine whether unit and integration testing has been conducted in
accordance with the test procedures and that the tests are complete. The
Project Manager reviews and approves test procedures, specifications,
test procedure modifications (i.e., redlines), and the results of all
integration tests before requesting Technical Review Board review and
approval of integration testing at Test Readiness Review. A version of
the Developmental Configuration is placed under formal configuration
management after Test Readiness Review and is used for acceptance
testing.

## 5.3.5 COTS Acceptance Testing

Acceptance testing of COTS products will be performed and all problems
and defects will be corrected in accordance with the terms of the
contract or other purchase agreement. Unit level testing of COTS
products may not be required in circumstances where acceptance testing
of the COTS product has been performed and documented. COTS acceptance
testing procedures and specifications, and all results of COTS
acceptance testing will be reviewed, and if accepted by the Project
Manager, will forwarded to the Technical Review Board for review and
approval at Test Readiness Review. The Office of System Product
Assurance may perform independent testing of COTS products.

## 5.3.6 Integration Testing

Integration testing verifies the interoperability of AIS modules,
components, and subsystems, up to the complete AIS, to ensure that all
functional and technical objectives are achieved. Typically, test data
must be specifically designed and developed to ensure full coverage of
all requirements, and to ensure that all procedural and control options
in the AIS code are tested. When possible, actual data should also be
used in integration testing to ensure consistency between AIS
specifications and the desired business process. The results of all
integration tests must be recorded. Test Procedures are corrected by
redlining the test procedures document to show changes.

## 5.3.7 Formal Qualification Testing (FQT) 

Formal Qualification Testing is an independent acceptance testing
activity performed on an approved Configuration Management Build by the
Office of System Product Assurance. During this testing activity all AIS
functional capabilities, including security measures, are independently
exercised to verify that the AIS meets all functional and security
requirements. Formal Qualification Testing is performed against the
requirements identified in the project\'s Requirements Traceability
Matrix in accordance with the redlined test specifications and test
procedures developed during integration testing.

## 5.3.8 Beta Readiness Review

When a Beta Readiness Review is conducted, it verifies that formal
testing is complete and that the system is ready to be tested by
selected users, including employee union representatives. The Office of
System Product Assurance will forward the results of Formal
Qualification Testing to the Technical Review Board for review at the
Beta Readiness Review. This review will be conducted in accordance with
the project\'s Quality Assurance Plan and test plans. The AIS will not
be installed in production work areas or operational facilities, or
connected to production systems without the concurrence of the Project
Manager.

## 5.3.9 Beta Testing

During Beta Testing selected users test routine features of the AIS to
verify that the AIS performs as expected. Beta testing exercises the
entire system as it is commonly used, rather than testing it extensively
against formal statements of requirements. In preparation, the Beta
testers will receive training on the AIS using the training material
delivered with the AIS. The Project Manager selects test participants
and establishes the schedule for Beta Testing.

# 5.4 Project Risk Management

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

The Project Manager will continue to evaluate risks to AIS project
requirements, cost, and schedule. The most common risks encountered
during the Development Phase relate to events that may prevent full
development or implementation of AIS project requirements. Security
requirements and controls, if not previously addressed, will impose
significant project risk during the Development Phase. This is due to
the potential cost and complexity necessary to retrofit these
requirements and designs into the AIS late in the life cycle.
Significant risks will also result if review and approval of designs and
supporting plans, development tools, development skills, appropriate
unit and integration testing, or specification, installation and testing
of security controls are inadequate. AIS project managers must also
evaluate the risks associated with recent changes and upgrades to the
hardware configuration, operating system and system utilities, software
development tools, and AIS COTS components. The Project Manager will
advise the Program Sponsor and the CIO on matters of significant risk.
The Project Manager will also coordinate risk management activities with
key managers and analysts based upon their ability to control risk. The
Project Management TSG, IT-212.2-01, provides additional guidance on
managing risk.

## 5.4.1 Project Risk Assessment

The Project Manager and the System Development Manager will continue to
analyze the existing project situation to identify, analyze, and
mitigate significant AIS project risks. These managers will focus on
ensuring the full implementation of AIS project requirements. AIS
project management will also focus on providing appropriate review and
approval of designs and supporting plans, and completing appropriate
unit and integration testing that includes testing of security
requirements. The Project Manager must also provide appropriate review
and approval for changes in hardware, software, development tools, and
COTS components. The Project Manager and the System Development Manager
will ensure the complete documentation of information and activities
that identify, estimate, and evaluate significant risks.

## 5.4.2 Project Risk Review and Mitigation

Risk assessment findings will be presented to the Program Sponsor,
Technical Review Board, or the CIO, as necessary. These presentations
will reaffirm the significance of risks and will solicit management
support for addressing significant risks. Based upon the recommendations
of the Project Manager, the Program Sponsor or the CIO may assign
additional staff or allocate additional resources to control significant
risks.

# 5.5 AIS and COTS Security

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

Security activities for an AIS may include:

a.  developing AIS security aspects,

b.  monitoring the development process itself for security problems,

c.  responding to requirements changes from a security perspective,

d.  monitoring security risks.

Security risks that may arise during the Development Phase include risks
associated with code intentionally designed to defeat security measures
and disguised as useful code, incorrect code, poorly functioning
development tools, manipulation of code, and malicious insiders.

Security activities for COTS software may include monitoring to ensure
that security is part of market surveys, contract solicitation
documents, and evaluation of proposed systems. Many systems use a
combination of "in-house" development and COTS acquisition. In this case
all the security activities discussed above are applicable.

As the AIS is built, choices are made about the system, which can impact
security including: selection of specific COTS products, AIS
architecture, and, AIS site or hardware selection. In addition, security
activities such as contingency planning, security awareness training,
and security documentation must be conducted. The Operational Support
TSG, IT-212.5-01 provides additional information regarding the
operational aspects of AIS security. Security, like all requirements,
must be anticipated from the beginning of the Initiation Phase.

# 5.6 Business Transition Planning

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

Business Transition Planning, which began during the Detailed Analysis
and Design Phase, continues through the Development Phase following the
guidance contained in the Business Transition Planning TSG, IT-212.3-05.
All significant changes to the business process made during the
Development Phase must be reviewed by the System Development Manager,
and reviewed and approved by the Project Manager. The Program Sponsor
and Project Manager should coordinate significant business process
changes with the employee union representatives.

## 5.6.1 Transition Activities

Transition activities should be reviewed and evaluated during the
Development Phase to ensure that all personnel, business process, and
organizational changes can be implemented with minimum disruption to
ongoing production. Where feasible, interviews and surveys should be
conducted to identify issues and determine the level of acceptance to
changes in policies, procedures, and job descriptions. Additional
changes in the Business Transition Plan may be necessary based upon
feedback collected though this process. Moreover, specific presentations
and specialized training sessions may be needed to further emphasize the
intended goals and benefits of the process improvement activity,
including AIS support. All transition and change management activities
will be documented and tracked in the Project Management Plan according
to IT-212.02-01. The Project Manager may brief the Technical Review
Board on the expected impact the deployed AIS will have on their
organization at Test Readiness Review.

## 5.6.2 Preparing the Organization for Change

End user involvement in the specification, design, development, and
testing of transition activities is absolutely essential to promote
broad-based process ownership and acceptance. During the Development
Phase, end users and the employee union representatives should be
heavily involved in transition activities which require pilot testing
(e.g., workflow redesign, team-oriented examination) and should play a
major role in reporting the results of pilot tests. Moreover, end users
should be involved in updating the desired business process and in
planning AIS deployment throughout the organization. Where feasible, end
users should take the lead in validating revised policies and
procedures, new forms and formats, job aids, and performance measures.

# 5.7 Production Readiness Review

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

The products of the Development Phase, and updates to existing plans and
specifications must be reviewed by the Technical Review Board. The
Production Readiness Review will confirm that the AIS is complete,
correct, fully tested, and ready for the Deployment Phase to begin. This
review establishes the product and operational baselines and confirms
that appropriate operational and maintenance support is available. The
Product Baseline consists of the documentation describing all the
necessary functional and physical characteristics of the elements that
compose the system and the actual equipment and software. The Technical
Review Board will confirm that the installation of the AIS into the
production environment will have no adverse impact on other AISs or the
PTO information technology infrastructure.

Deficiencies identified in unit, integration, Formal Qualification, and
Beta Testing must be resolved, and changes generated by the review must
be incorporated into the appropriate work products as directed by the
Technical Review Board. Based upon a finding that the AIS is suitable
for production use, the Technical Review Board will recommend to the
Program Sponsor and CIO to the deploy the AIS in accordance with the
Production Installation Plan.

[^1]: *Formal Qualification Testing (FQT)* exercises the entire AIS to
    independently confirm and extend integration testing results in an
    environment that more closely duplicates the production environment.
    OSPA will test all AIS requirements that can reasonably be tested.
    These requirements are identified in the current AIS requirements
    traceability matrix. See 5.4.6 below.

[^2]: Beta testing may be tailored in or out of the life cycle. End
    users perform *Beta Testing* in an ad hoc manner to confirm that the
    system will provide the expected functionality before placing the
    system into full service. This testing will simulate the production
    environment as closely as is reasonably possible. See 5.4.7 and
    5.4.8 below.