**NASA**

**Integrated Baseline Review (IBR)**

**toolkit**

# 

> ![](media/image1.png){width="4.75in" height="3.95625in"}

**[TABLE OF CONTENTS]{.underline}**

[1 -- INTRODUCTION 5](#__RefHeading___Toc157574696)

[1.1 Background 5](#background)

[1.2 Purpose 5](#__RefHeading___Toc157574698)

[1.3 Authority 5](#__RefHeading___Toc157574699)

[1.4 Applicability 5](#__RefHeading___Toc157574700)

[1.5 Document Structure 5](#__RefHeading___Toc157574701)

[2 -- IBR PREPARATION 5](#__RefHeading___Toc157574702)

[2.1 General 5](#__RefHeading___Toc157574703)

[2.2 Determination of Need for an IBR 5](#__RefHeading___Toc157574704)

[2.3 IBR Planning 5](#__RefHeading___Toc157574705)

[2.4 Team Roles and Responsibilities 5](#__RefHeading___Toc157574706)

[2.5 Determine Supplier Readiness for the IBR
5](#__RefHeading___Toc157574707)

[2.6 IBR Notification Letter/Documentation
5](#__RefHeading___Toc157574708)

[2.7 IBR Team Handbook 5](#__RefHeading___Toc157574712)

[2.8 Training 5](#__RefHeading___Toc157574713)

[2. 9 Travel Arrangements/IBR Logistics 5](#__RefHeading___Toc157574714)

[IBR ON-SITE REVIEW 5](#__RefHeading___Toc157574715)

[3.1 General 5](#__RefHeading___Toc157574716)

[3.2 On-Site Discussions 5](#__RefHeading___Toc157574717)

[**3.2.1** **In-Briefs** 5](#__RefHeading___Toc157574718)

[**3.2.1.1 Joint IBR Team and Supplier**
5](#__RefHeading___Toc157574719)

[**3.2.1.2** **IBR Team-Only** 5](#__RefHeading___Toc157574720)

[**3.2.2** **CAM Discussions** 5](#__RefHeading___Toc157574721)

[**3.2.2.1** **CAM Discussion Guidelines**
5](#__RefHeading___Toc157574722)

[**3.2.2.2** **Discussion Topics** 5](#__RefHeading___Toc157574723)

[**3.2.2.3** **IBR Documentation and Forms**
5](#__RefHeading___Toc157574724)

[**3.2.3** **Out-briefs** 5](#__RefHeading___Toc157574725)

[**3.2.3.1** **Daily** 5](#__RefHeading___Toc157574726)

[**3.2.3.2** **Exit** 5](#__RefHeading___Toc157574727)

[4- IBR CLOSEOUT 5](#ibr-closeout)

[4.1 General 5](#__RefHeading___Toc157574729)

[4.2 Reporting Results of the IBR 5](#__RefHeading___Toc157574730)

[4.3 Tracking Concerns 5](#__RefHeading___Toc157574731)

[4.4 IBR Closeout Letter 5](#__RefHeading___Toc157574732)

+-------+---------------------------------------------+----------------+
| **R   |                                             |                |
| ECORD |                                             |                |
| OF    |                                             |                |
| R     |                                             |                |
| EVISI |                                             |                |
| ONS** |                                             |                |
+=======+=============================================+================+
| **    | **DESCRIPTION**                             | **DATE**       |
| REV** |                                             |                |
|       |                                             |                |
| **    |                                             |                |
| LTR** |                                             |                |
+-------+---------------------------------------------+----------------+
|       | Basic Issue                                 | June 30, 2005  |
+-------+---------------------------------------------+----------------+
|       | First Revision                              | July 7, 2006   |
+-------+---------------------------------------------+----------------+

[]{#__RefHeading___Toc157574696 .anchor}**[1 --
INTRODUCTION]{.underline}**

## 1.1 Background

An Integrated Baseline Review (IBR) is a formal review that includes the
total project (contracted as well as in-house NASA) efforts. The IBR is
led by the Program/Project Manager or technical designee and is
conducted jointly with the customer and supplier. It is important to
note that an IBR cannot be failed.

The objective of the IBR is to confirm compliance with the following
business rules:

-   The technical scope of work is fully included and consistent with
    authorizing documents

-   Key schedule milestones are identified

-   Supporting schedules reflect a logical flow to accomplish the
    technical work scope

-   Resources (budgets, facilities, personnel, skills, etc.) are
    adequate and available for the assigned tasks

-   Tasks are planned and can be measured objectively, relative to
    technical progress

-   Underlying Performance Measurement Baseline (PMB) rationales are
    reasonable

-   Managers have appropriately implemented required management
    processes.

The IBR team should assess the management reserve with respect to
project risk not accounted for in the Performance Measurement Baseline
(PMB). For reference in this document, the IBR Team is the team
conducting the IBR.

Anything that does not support the intent of the IBR should be moved
outside the review. Technical, Schedule, Cost, Resource, and Management
Processes risks identified during the IBR should be reviewed; action
risks should be incorporated into the project risk management planning.

2.  []{#__RefHeading___Toc157574698 .anchor}**Purpose**

This toolkit provides guidance on conducting IBR's for both the
contractual and in-house portions of projects. The toolkit also offers
tips for tailoring the IBR based on risk, cost, and need for management
insight. Appendices to this toolkit contain samples of other documents
used in connection with the IBR. Please note that these appendices are
samples only, and should be tailored to meet the needs of individual
projects . These attachments are available on the NASA EVMS Web Site at
<http://evm.nasa.gov/evmsdrd.html>.

The Program Manager holds the overall responsibility for the total
project IBR. The Project Manager is responsible for conducting IBR's on
the contractual and in-house elements, components, or sub-projects when
applicable. While the same principles and objectives of the IBR apply
for projects, in-house, and contracts, the steps in conducting an IBR
for these efforts differ in some areas. When applicable, a section will
contain a distinction of the steps when the process differs. Unless
noted, the language applies to IBR's for projects, contracts, and
in-house. For simplicity, the term "supplier" is used to indicate the
organization performing the work.

Appendix A contains an IBR Checklist that can be used as a guide in
conducting the IBR and ensuring that all steps are considered.

[]{#__RefHeading___Toc157574699 .anchor}**1.3 Authority**

This document is intended to be a how-to-guide to conduct and complete
an IBR. It is not a requirement to use this document, however following
this guide will help to ensure that the IBR is successful. NPR 7120.5
contains IBR requirements for NASA projects and contracts.

[]{#__RefHeading___Toc157574700 .anchor}**1.4 Applicability**

The objectives of an IBR are applicable and beneficial to projects and
contracts of all sizes and types, however the level of detail and
formality of the review vary based on dollar value and risk. This
document lists the steps that should be considered for each IBR,
tailored based on the dollar amount, risk, and requirements for
management insight. The IBR Lead should determine if the Decision
Authority or Technical Authority have prepared a Terms of Reference for
the project. The IBR should be consistent with the Terms of Reference
and assure an adequate level of detailed information and analysis is
provided to the Decision Authority and Technical Authority. The Mission
Directorate may impose unique guidelines for the IBR.

Appendix B contains a sample format for an IBR for a very small,
low-risk project. For a project of this size, it is not unusual for the
project manager to be the Control Account Manager (CAM) for all control
accounts. In this case, the IBR usually occurs in one day and consists
of one team conducting all CAM discussions. Often, a presentation format
is used to present relevant data to the IBR team, and this information
should be furnished to the IBR team at least two weeks before the
review.

5.  []{#__RefHeading___Toc157574701 .anchor}**Document Structure**

This guide is divided into three major sections: IBR Preparation, IBR
On-Site Review, and IBR Closeout.

[]{#__RefHeading___Toc157574702 .anchor}**[2 -- IBR
PREPARATION]{.underline}**

[]{#__RefHeading___Toc157574703 .anchor}**2.1 General**

Preparation is the foundation for a successful IBR. The preparation
portion of the IBR is the most important part of the entire process. It
is all the work that happens in this area that will pay off when the IBR
Team goes on location to the supplier's facility. Preparation includes
the planning that identifies key responsibilities, required technical
expertise, IBR and/or EVM training, review dates, review scope, risk
evaluation criteria, documentation needs, disposition of findings, and
procedures for risk identification, documentation, and incorporation
into project risk management planning.

2.  []{#__RefHeading___Toc157574704 .anchor}**Determination of Need for
    an IBR**

The need for the IBR must be determined prior to the start of any
project. NASA Program and Project Management Processes and Requirements
in 7120.5 provides requirements on the applicability of the IBR. An IBR
may also be planned for efforts that do not meet the dollar thresholds
in 7120.5, but that have significant risk or require more management
attention.

IBR's occur at different periods in the life cycle of the project, and
it is important to note that the IBR goals and objectives are the same
regardless of when the IBR is conducted.

In some cases, the Mission Directorate may establish unique IBR
requirements and thresholds that must be followed.

The Program Plan will identify the projects that require an IBR. The
Project Plan will identify the contract and in-house portions of the
project that will require an IBR. For contracts, the need for the IBR
must be determined prior to contract award. NFS 182-242-74, Notice of
Earned Value Management System, must be included in the solicitation of
a contract to notify potential bidders of the EVMS and IBR requirements.
NFS 1852-242-75, Earned Value Management System, must be included on
contracts that require EVM. Paragraph (c) of this clause contains the
requirement for an IBR. NFS clauses are included in on the NASA EVMS
Website. If required, the contract Statement of Work (SOW) should be
used to tailor project requirements for the IBR. Sample SOW input can be
found in Appendix C.

[]{#__RefHeading___Toc157574705 .anchor}**2.3 IBR Planning**

Preparation for the IBR should begin as soon as practical after
determining the need for an IBR. The program or project manager should
appoint an IBR Coordinator and a Review Facilitator early-on to help
facilitate the IBR. The IBR Coordinator helps to coordinate various
activities to ensure a successful IBR. The Review Facilitator provides
the team members with earned value management expertise. The roles of
the IBR Coordinator and Review Facilitator are further described below.

The first step in planning the IBR is to determine which control
accounts will be reviewed at the IBR. A Responsibility Assignment Matrix
(RAM) that shows the budget for each control account will help in this
assessment. See Appendix D for a sample RAM and Section 2.4 for a more
detailed description of the RAM. A RAM or equivalent document must be
requested from the supplier. Typically 85% of the dollar value and all
of the high risk areas should be examined. Usually, at least one
Level-Of Effort (LOE) and one material account will be included to
provide insight into various aspects of the project. The Program or
Project Manager, with advice from the Review Facilitator, should choose
the control accounts for the IBR and inform the supplier, usually
informally, of the ones selected.

Team composition and assignments are based on which control accounts are
selected for review. Participants should be identified based on their
expertise as required for the review. These disciplines include program
and project management, business management, and technical management
(e.g., system engineering, software engineering, manufacturing,
integration and test engineering, and integrated logistics support).
When appropriate, the team should include subcontractor, DCMA, or DCAA
personnel. There may be several sub-teams with discussions scheduled
concurrently, or one team can interview every Control Account Manager
(CAM). Each of these teams should be assigned a sub-team leader to lead
the discussion. To be effective, the discussion group should remain
small and focused, usually 3-4 people per team. Typically, the technical
experts for NASA will lead the discussion. A sample team may consist of
an EVMS analyst, a Project Manager, and one or two technical experts in
the area of the Control Account (CA) to be discussed. A good rule of
thumb is to allow at least two hours to conduct the CAM discussions and
one hour for completing documentation.

[]{#__RefHeading___Toc157574706 .anchor}**2.4 Team Roles and
Responsibilities**

**Program or Project Manager (PM):** The PM either acts as or assigns
the Team Leader for the IBR. The overall responsibility of conducting
the IBR lies with the Team Leader. The Team Leader is responsible for
coordinating the activities of all individuals assigned to perform the
review. Some individual tasks to be performed by the Team Leader are
described below.

a.  Coordinate with the supplier to discuss the IBR strategy, areas to
    be reviewed, and tentative dates for the IBR.

b.  Choose appropriate team members and assign responsibilities.

c.  Ensure team members are adequately trained and prepared. Provide IBR
    training and workshops as necessary to prepare the team before the
    review begins.

d.  Provide technical direction and leadership emphasizing the
    importance of thorough cost, schedule and technical integration of
    work.

e.  Provide an in-brief and exit-brief at the IBR.

f.  Provide an overall evaluation of the IBR and assess whether or not a
    follow-on review will be required.

g.  Document review findings. The format of this documentation is at the
    discretion of the Team Leader. Sample formats are included as part
    of the IBR Team Handbook in Appendix F.

h.  Ensure that all issues identified during the review are resolved in
    an appropriate and timely manner. Assign action items and completion
    dates to appropriate government and contractor personnel. Schedule
    action item review as necessary.

i.  Define the risk evaluation criteria to generally categorize the risk
    that may be identified at an IBR: cost, schedule, technical,
    resource, and management processes. Appendix F, the IBR Handbook,
    contains sample criteria for each type of risk.

**IBR Coordinator**: The Team Leader may appoint an individual as IBR
Coordinator whose function entails coordination of various activities to
ensure a successful Integrated Baseline Review. This includes but is not
limited to coordinating review dates, obtaining information necessary
for the review, organizing this material, and coordinating the various
aspects among the government and contractor review participants. The
following is a recommended guide to assist the coordinator in performing
this role.

a.  Coordinate the review dates that are amenable to all parties.
    Consideration should be given to a time period that is the least
    disruptive to ongoing activities. Normally scheduling the review in
    the early part of the month after the close of the accounting period
    and statusing activities is the most appropriate time.

b.  Provide written notification of the IBR to the supplier. Send the
    letter to all of the team members for informational purposes.

c.  Obtain a list of the IBR Team members from the Team Leader.

d.  Coordinate with individual team members to ensure travel and
    security arrangements have been accomplished.

e.  Prepare the IBR Team Handbook.

f.  Collect information and data requested in the IBR notification
    letter, and compile and sort the information by sub-team
    responsibilities and make it available to the respective sub-team
    leads along with the IBR Handbook.

g.  Schedule a pre-IBR meeting to allow team members to familiarize
    themselves with the documentation and develop questions to be
    addressed during the review.

h.  Collect the IBR Concern Area Report Forms or questions that have
    been completed during the pre-IBR meeting for the Team Leader's
    review. Forward those deemed appropriate by the Team Leader to the
    supplier for resolution. The IBR Coordinator will perform this
    function during the on-site review also.

i.  Coordinate with the Team Leader to ensure all team members receive
    necessary IBR training prior to the review.

j.  Distribute IBR forms, either hard-copy or electronic format, to team
    members prior to CAM discussions.

k.  Collect documentation at the IBR.

l.  Establish and maintain an IBR log.

> **Review Facilitator**: Duties and responsibilities of the IBR Review
> Facilitator include providing the team members with earned value
> management expertise. This is accomplished through the review of
> documentation, assessing earned value status, and interpreting issues
> that relate to overall earned value management. The Review Facilitator
> should be an EVM Working Group member from the Center if possible. The
> items listed below are provided as a recommended guide to assist the
> facilitator in the accomplishment of this role.

a.  Determine status of the EVM system. If applicable, request
    documentation showing that the EVM system has been previously
    accepted by the government. For contracts, contact the local DCMA
    representative to determine if there are any outstanding earned
    value management system problems that would affect the quality of
    the performance measurement data.

b.  If the EVM system requires previous government acceptance and has
    not met that requirement, coordinate with appropriate personnel the
    means by which this acceptance can be accomplished.

c.  Review management processes that will be used in the management of
    the work.

d.  Provide earned value management expertise to the IBR Team by
    assisting team members in the understanding any EVMS data obtained
    prior to the review.

e.  Coordinate with the Team Leader and IBR Coordinator in determining
    the interview schedule.

f.  Review the experience and background of the team participants and
    provide a recommendation to the team leader of the type and amount
    of EV training deemed necessary.

g.  Provide assistance to IBR Team members in determining whether
    individual tasks have been assigned appropriate earned value
    techniques used for measuring progress.

h.  Assist the team leader in the development of the risk evaluation
    criteria.

i.  The IBR Facilitator reviews all action items generated to ensure
    that earned value issues requiring follow-up are appropriate and
    necessary.

j.  Provide assistance to the Team Leader with the preparation of both
    the in-briefing and exit-briefing at the IBR.

**Sub-team Technical Lead:** The Sub-team Technical Lead is normally a
technical expert who specializes in the area that is being addressed
during the CAM discussion. The responsibilities of the Sub-team
Technical Lead include:

a.  Attend the pre-IBR meeting, including training and documentation
    review prior to the start of the IBR.

b.  Review documentation prior to baseline discussions with the CAM.

c.  Lead CAM and senior manager discussions.

d.  Establish the strategy for conducting the CAM discussions, e.g.,
    designate a single person to lead the discussion, someone to take
    notes, etc.

e.  Ensure all applicable documentation is properly completed (i.e.
    Discussion Assessment Form, Concern Area Report, Documentation
    Request Form, and Risk Assessment Form).

f.  Provide daily out-briefs to keep the IBR Team informed.

g.  Provide an assessment of risk based on the prescribed risk
    evaluation criteria.

h.  Assist in the preparation of the IBR exit-brief.

**Technical SME Team Members:** Team members may include technical
experts, EVM analysts, contracts representatives, DCMA, as well as other
personnel who may be of benefit during the review. Duties of team
members include:

a.  Attend the pre-IBR meeting, including training and documentation
    review prior to the start of the IBR.

b.  Review documentation prior to baseline discussions with the CAM.

c.  Conduct CAM and senior manager discussions.

d.  Assist in completing all applicable documentation.

e.  Provide an assessment of risk based on the prescribed risk
    evaluation criteria.

f.  Assist in completing IBR documentation.

g.  Assist in the preparation of the IBR exit-brief.

**Schedule Analyst**: The duties of the Schedule Analyst include:

a.  Attend the pre-IBR meeting, including training and documentation
    review prior to the start of the IBR.

b.  Review documentation prior to baseline discussions with the CAM.

c.  Conduct CAM and senior manager discussions.

d.  Review the EVM Systems Description or management processes to
    understand the processes and procedures by which the Integrated
    Master Schedule is created and maintained.

e.  Provide an assessment of all project risk based on the defined risk
    evaluation criteria.

f.  Assist in completing all IBR documentation.

g.  Assist in the preparation of the IBR exit-brief.

**Cost Analyst**: The responsibilities of the Cost Analyst include:

a.  Attend the pre-IBR meeting, including training and documentation
    review prior to the start of the IBR.

b.  Review documentation prior to baseline discussions with the CAM.

c.  Provide insight into the reliability of the Basis Of Estimate.

d.  Inform the team of any areas where the independent or non-advocate
    estimates differ from the advocate estimate.

e.  Conduct CAM and senior manager discussions.

f.  Provide an assessment of all project risk based on the defined risk
    evaluation criteria.

g.  Assist in completing all IBR documentation.

h.  Assist in the preparation of the IBR exit-brief.

Having an IPAO and/or SMO Representative on the IBR team is recommended
to ensure an independent member of the review panel. The team member
will provide a common thread for all reviews at a particular Center to
ensure consistency across all the IBR reviews and provide an avenue for
lessons learned to be communicated across the Center. This will also
familiarize the SMO or IPAO representative with the project to make
independent evaluations that occur later in the project life cycle more
thoroughly and efficiently conducted and completed.

[]{#__RefHeading___Toc157574707 .anchor}**2.5 Determine Supplier
Readiness for the IBR**

Prior to the IBR On-Site Review, the IBR Team must assess the readiness
of the contractor or in-house activity to conduct an IBR. Below are some
rules of thumb to help determine IBR readiness.

-   The PMB should reflect the entire scope of work on the project or
    contract.

-   The schedules reflect the entire scope of work on the project or
    contract.

-   It is good to have at least two reporting cycles worth of data
    before the IBR. This will provide the IBR Team with some performance
    measurement data and give them a feel for the accuracy of the data.

-   Look for detail planning to the work package level for six months to
    one year to ensure an adequate assessment of the baseline.

The IBR Team must measure the ability of the contractor or in-house
activity to meet the intended objectives prior conducting the IBR
On-Site Review. The IBR is not just a check in the box! If after the
IBR, the Program or Project Manager does not feel that the IBR met the
objectives, a second IBR must be performed.

[]{#__RefHeading___Toc157574708 .anchor}**2.6 IBR Notification
Letter/Documentation**

An IBR notification letter should be prepared and sent to the Project
Team within 3 months of the scheduled On-Site IBR. This notification
letter can take any form but should include the following information:
the purpose of the review, the time frame of the review, the agenda,
documentation requested, team requirements (ex: fax and phone access,
meeting rooms, etc), IBR Team members and roles, and points of contact
for follow-up. The formality of the IBR Notification Letter will depend
on the size of the IBR. In some cases, an IBR notification letter can be
informal such as by email.

Project IBR's

For Project IBR's, the Program Office will prepare a Terms Of Reference
(TOR) document in place of the IBR Notification Letter. The TOR contains
unique guidance from the Program Office. A generic TOR is included as
Appendix E.

Contracts:

For contracts, the IBR Notification Letter should be sent from the
government Contract Technical Representative to the contractor
organization. The local DCMA and DCAA organization, if applicable,
should be informed of the review.

In-House:

For in-house efforts, the letter should be sent from the NASA Project
Manager to the project lead at the supplier.

One of the main functions of the notification letter is to request
documentation for use on site and prior to the review so that the team
can get prepared. This documentation will be used by the team to
identify risk areas and to develop preliminary questions for the on-site
review, and should be received 2 -- 4 weeks prior to the IBR. The
following is a listing of some of the documentation that may be
requested.

a.  Organizational Breakdown Structure (OBS)

b.  Control Account Plans (CAP): Basis of Estimates (BOE), Assumptions,
    and Risk

c.  Work Authorization Documents (all levels)

d.  Performance Measurement Reports (internal cost/schedule reports)

e.  Estimate at Completion (EAC)

f.  Budget Change Requests (BCR):

g.  CA/Work Package (WP) summary containing:

    -   Number of work packages by type of EVM method

    -   Longest CA, shortest CA, mean and median duration, total value
        of account

    -   Largest CA, smallest CA, mean and median values

h.  Baseline Control Logs (Management Reserve, Undistributed Budget,
    Contract Budget Base)

i.  SOW with Work Breakdown Structure (WBS) Matrix

j.  Schedules (Master, Intermediate, and Detailed)

k.  EVM System Description

l.  Description of Management Process such as baseline maintenance, risk
    management, and other business processes

m.  Financial Management Reports such as 533M/Q or equivalent

n.  All Cost Performance Reports (CPR) submitted to date

o.  Responsibility Assignment Matrix (RAM)

p.  WBS and Contract Work Breakdown Structure (CWBS) and Dictionary

q.  Subcontractor listing and value of subcontracts

r.  All contract changes

s.  Contractor/Subcontractor or In-House EVMS Flow-Down Requirements

t.  Giver/Receiver Schedule (Rec/Del List)

Only request data that has not already been received. Also, this list
should be tailored based on information required for preparation and
insight. The supplier may have the data described above in different
formats, and these documents are acceptable and even preferred as
replacements when the information is the same.

**Responsibility Assignment Matrix (RAM):**

The RAM correlates the work required by a WBS or CWBS element to the
functional organization responsible for accomplishing the assigned
tasks. The RAM results from the intersection of the WBS or CWBS with the
project organizational structure. This intersection identifies the CA.
The RAM includes the functional organization and the individual
responsible for the CA, known as the CAM as well as the budget for each
CA.

## Work Authorization Documents:

All work, regardless of origin, is described and authorized through the
work authorization system. Work authorization assures that performing
organizations are specifically informed regarding their work scope,
schedules for performance, and budget for that work.

Work authorization is a formal process related to various levels of the
organization. Agreement is reached to each level of authorization so
that there is no question as to what is required. The CA authorization
represents an agreement between the CAM and the authorizing party. The
documents involved in work authorization, at each level, are maintained
in a current status throughout the life of the project as revisions take
place.

## Cost/Control Account Plans (CAP):

A CAP is a plan of all effort to be performed in a cost/control account.
The CA is the focus for planning, monitoring, and controlling because it
represents work within a single WBS or CWBS element, and it is the
responsibility of a single organizational unit or work team. Virtually
all aspects of a project come together at the CA level, including
budgets, schedules, work assignments, cost collection, progress
assessment, problem identification, corrective actions, and EAC
development.

A Work Package (WP) is a detailed job that is established by the CAM for
accomplishing work within a CA. A WP can be characterized by the
following:

a.  The WP represents units of work at the levels where the work is
    performed.

b.  A WP is clearly distinct from all other WP's, and is the
    responsibility of a single organization element.

c.  A WP has scheduled start and completion dates (with interim
    milestones, if applicable), which are representative of physical
    accomplishment.

d.  A WP has a budget or assigned value expressed in terms of dollars,
    labor hours, or other measurable units.

e.  The duration of a WP is relatively short, unless it is subdivided
    into discrete value milestones that permit objective measurement of
    work performed.

f.  The WP schedule is integrated with all other schedules.

g.  The WP has a unique earned value technique for determining the value
    of completed in-process work.

If a CA cannot be subdivided into detailed WP's, far-term effort may be
identified in larger Planning Packages (PP's) for baseline control
purposes. The budget for a PP is identified specifically for the work
that is intended, is time-phased to the extent possible, and has
controls that prevent its use in performance of other work. Eventually,
all work in PP's will be planned to the appropriate level of detail in
WP's.

## Project Schedules:

**Integrated Master Schedule (IMS)**

Various schedules are essential to review before IBR discussions begin.

-   **Integrated Master Schedule**

> The Integrated Master Schedule is the controlling schedule for the
> entire project. It contains contractual as well as project
> milestones/activities. An integrated schedule developed by logically
> networking all detailed program/project activities. The highest level
> schedule is the Master Schedule supported by Intermediate Level
> Schedules and by lowest level detail schedules.

-   **Intermediate Schedule(s)**

> Intermediate Schedules are the integrating schedules for a project.
> There may be single or multiple levels of intermediate schedules. They
> may be oriented to either WBS/CWBS elements or to functional
> organizations. In either case, the purpose is to form a bridge between
> the Project Master Schedule and CA schedules. The term "integrating"
> is used since the purpose is to coordinate the effort of a number of
> CA's that are related in one way or another to the scope of the
> intermediate schedule. Intermediate schedules are often the primary
> tool for identifying key project interfaces through network logic.

-   **Cost/Control Account (Detail) Schedules**

> See Cost/Control Account Plans.

Electronic versions are necessary for providing insight into the various
schedules.

**Cost Reports:**

As part of the documentation review process, it is important to review
the supplier's latest cost report deliverables, i.e. 533M or 533Q, etc.

**WBS/CWBS and Dictionary:**

The Work Breakdown Structure or Contract WBS is the basis for further
extension to lower levels that represent how the contractor or in-house
activity plans to accomplish the entire work scope and is consistent
with internal organizations and processes. The term CWBS is only
applicable to contracted effort. This extended CWBS or WBS serves as the
framework for reporting of cost and schedule status. The WBS or CWBS
Dictionary is used to describe the effort and tasks associated with
corresponding elements.

Once the documentation is received the next step is for the IBR Team to
evaluate this documentation and determine risk areas. This evaluation of
documentation, as well as other activities, can be conducted at a
pre-IBR meeting. Questions may be held for discussion at the IBR or can
be provided to the supplier prior to the IBR, allowing time to respond
prior to the IBR team arriving at the facility. The questions can be
informal or documented in a Concern Area Report (CAR). See Appendix F
for a sample CAR. If the documentation review uncovers new risk areas,
those control accounts should be added to the review schedule.

[]{#__RefHeading___Toc157574712 .anchor}**2.7 IBR Team Handbook**

An IBR Team Handbook should be prepared that serves as a guide for the
IBR Team. The IBR Team Handbook will assist the team members in
conducting the review. The handbook provides the project background,
sample discussion questions, documentation guidelines, sample
documentation, risk evaluation criteria, and a glossary of EVMS terms.
The review agenda and team assignments should also be included in the
handbook. See Appendix F for a sample IBR Team Handbook

[]{#__RefHeading___Toc157574713 .anchor}**2.8 Training**

Training should also be conducted at the Pre-IBR meeting. The three
components of the training are basic EVM training, IBR training, and an
overview by the supplier. The supplier can be a great help during the
Pre-IBR meeting, by explaining documentation and answering any
preliminary team questions. The Project Team personnel can also give the
team an understanding of management processes, such as baseline
maintenance, risk management, and other business processes including
EVM, that will be used to manage the project. The team will also review
the IBR Handbook during the training, and learn the use of the various
forms and assessments used to document interview findings.

Training may be obtained from various sources and should be tailored
based upon the need and experience of the individual members of the
review team. Training sources may be found by contacting the local NASA
Training Coordinator or the NASA EVM Working Group. Names of members and
contact information are listed on the NASA EVM Website.

If at all possible the training and documentation review needs to occur
prior to the IBR. This will help to ensure a smoother review on-site. It
will also provide the team an opportunity to review the data for
completeness and ensure readiness of the supplier. If the team is not
able to have ample training and preparation time prior to the on-site
review, sufficient on-site time needs to be allocated to accomplish
these efforts. Conducting the training and documentation review on-site
immediately followed by the CAM discussions may also be done. If this is
done, make sure to allow adequate time for the team members to complete
a review of the documentation prior to the start of the IBR discussions.
See Appendix J for sample IBR and EVM Training.

[]{#__RefHeading___Toc157574714 .anchor}**2. 9 Travel Arrangements/IBR
Logistics**

Team members need to be informed as early as possible of the travel
plans, of the review schedule and agenda, and of their assignments.
Additionally, team members should know the name and number of a POC for
the IBR, and security arrangements need to be made for each team member
visiting the facility (i.e. Visit Requests need to be sent to the
Security Office.) IBR Team members should also be provided area maps,
directions to facilities, directions to visitor check-in, and specific
directions to the building and room where the initial meeting or
in-briefing will be held.

[]{#__RefHeading___Toc157574715 .anchor}**[IBR ON-SITE
REVIEW]{.underline}**

[]{#__RefHeading___Toc157574716 .anchor}**3.1 General**

The On-Site IBR is where all the preparation and planning pay off. The
primary purpose is for the Program and Project Managers to gain a mutual
understanding of the risks inherent in the PMB and management processes.
Anything that does not support this purpose should be discussed outside
of the IBR. Any issues that arise during the IBR should be recorded, but
resolved after the review.

The intent of this section is to provide sample tools and documentation
for conducting the IBR On-Site Review. While each IBR may be tailored,
these tools will provide a starting point for conducting the review.

[]{#__RefHeading___Toc157574717 .anchor}**3.2 On-Site Discussions**

Once the IBR Team is on-site at the supplier's facility, several
different activities must take place. This section describes of each of
these activities and their role in making the IBR successful.

[]{#__RefHeading___Toc157574718 .anchor}**3.2.1 In-Briefs**

[]{#__RefHeading___Toc157574719 .anchor}**3.2.1.1 Joint IBR Team and
Supplier**

> In some cases the IBR team may want to provide an in-brief to the
> supplier at the start of the IBR. The main theme of this discussion is
> to reemphasize that the purpose of the IBR is to evaluate the adequacy
> of the baseline and identify concerns, not to try to solve problems.

2.  []{#__RefHeading___Toc157574720 .anchor}**IBR Team-Only**

> The team may also want to conduct an IBR team in-brief before the CAM
> discussions begin. This meeting is the final opportunity to focus the
> IBR team on the objectives.

[]{#__RefHeading___Toc157574721 .anchor}**3.2.2 CAM Discussions**

> The CAM discussions are the key events of the IBR. These discussions
> focus on key risk areas and management processes. During this period
> the team members should cover the key aspects of the planning of the
> work scope. Overall success of the IBR results, in part, through
> productive CAM discussions. Following the techniques and formats below
> will help to ensure that adequate information is obtained in a timely
> manner to accomplish the objectives of the IBR.
>
> To evaluate project PMB risks, the IBR team should assess the degree
> to which the following statements are true:

-   The technical scope of work is fully included and consistent with
    authorizing documents.

-   Key schedule milestones are identified.

-   Supporting schedules reflect a logical flow to accomplish the
    technical work scope.

-   Resources (budgets, facilities, personnel, skills, etc.) are
    adequate and available for the assigned tasks.

-   Tasks are planned and can be measured objectively, relative to
    technical progress.

-   Underlying PMB rationales are reasonable.

-   Management processes support successful execution of the work scope.

> Additionally, the IBR team should assess the Management Reserve with
> respect to project risk not accounted for in the PMB.

[]{#__RefHeading___Toc157574722 .anchor}**3.2.2.1 CAM Discussion
Guidelines**

> Below is a list of suggested techniques for all team members to
> consider before and during the CAM discussions.

a.  Prior to the discussions, the IBR Team members should be familiar
    with areas previously identified for discussion. Review the
    documentation thoroughly.

b.  Introduce yourself and identify the organization you represent. You
    may also wish to indicate your team affiliation in the review.

c.  Be well prepared and maintain a tempo that keeps the discussion
    moving along toward satisfying your objective. Be friendly, but
    avoid long conversations extraneous to the discussion.

d.  Take notes. The discussion leader (normally the Sub-team Technical
    Lead) or an accompanying team member should take notes during the
    discussion.

e.  Request copies of documents only if necessary to accomplish the
    objective of the discussion. If documentation is not readily
    available, complete a Documentation Request Form and submit it to
    the IBR Coordinator. See Appendix F for a sample Documentation
    Request Form.

f.  Watch the time. Discussions are normally scheduled for two hours in
    length. Should additional time be required to complete the
    discussion, coordinate with the Team Leader

g.  If disagreements arise which cannot be resolved, the team member
    should write a description of the disagreement in a Concern Area
    Report and submit it to the Team Leader for disposition. The Team
    Leader will handle any continuing discussion.

The supplier must ensure that each CAM is well prepared (documentation
available, understands documentation content, can support answers)

[]{#__RefHeading___Toc157574723 .anchor}**3.2.2.2 Discussion Topics**

> Below are some examples of information that should be obtained from
> the CAM discussions. This list of questions is not all-inclusive and
> should be tailored for each project. Additional sample questions can
> be found in the IBR Handbook in Appendix G.

a.  What is the Manager's scope of work? Does the Manager know what
    scope of work is his/her responsibility?

> The Manager should be able to show the team a Statement of Work (SOW)
> paragraph, a WBS, or CWBS narrative, and a Work Authorization Document
> that correlates to the task.
>
> b\. How did the Manager plan the scope of work into control accounts?
>
> The SOW defines the effort. The WBS or CWBS provides specifics, such
> as work definition. The work authorization and change documentation
> should show information such as the dollars/hours, period of
> performance, and description of the scope of work and any changes.

c.  How did the Manager ensure that all elements of the scope of work
    were planned?

    The Manager should be able to show the team the work packages
    breaking down the scope of work, and the budget associated with each
    work package. The sum of the work packages and planning packages
    should be the control account budget. Each control account should
    contain a specific scope of work.

d.  How did the Manager obtain the resources for assigned work?

    This information should be obtained through the Work Authorization
    Document. The work authorization should provide specific resources
    used to manage the scope of work.

e.  Ask the Manager to describe the resource development process.

f.  How did the Manager come to an agreement of the scope, hours,
    dollars and schedule?

g.  Ask the Manager to show the team the schedule milestones used in the
    planning of the control accounts. The Manager should discuss:

-   Schedule impacts.

-   Schedule impacts from other organizations.

-   Resource levels which support schedule milestones.

-   Schedule constraints.

-   Relationships to other tasks.

-   Logical relationships of work package to milestones.

-   Level of Effort (LOE) tasks that support the schedule.

-   The Manager should be able to show the team the activity supported
    on the appropriate schedule.

h.  Ask the Manager how the work was time phased to achieve the
    schedule.

    All work should be logically planned in accordance with the scope of
    work and the schedule.

i.  Ask the Manager to identify any known risks that may affect the
    plan.

j.  Are the Manager\'s resources adequately planned and correctly time
    phased to meet the plan? Progress of each plan should relate to
    earned value.

k.  Ask the Manager why he/she chose a particular type of earned value
    method.

> Does the earned value method used make sense for the type of work
> being performed? Can you measure performance? Is the Manager using
> milestones to measure accomplishments? Does the Manager use the LOE
> method; and is the LOE in support of measurable scope? Is LOE the
> appropriate earned value method for the task?

l.  How does the Manager manage his/her plan?

> Does the manager use monthly reports, latest schedules, technical
> assessment of impacts, daily meetings, and continuous assessment of
> the plan?
>
> m\. How is the baseline maintained?
>
> How does the CAM have changes incorporated into the baseline? What are
> the controls (retroactive, freeze period, etc.) to ensure the
> integrity of the baseline?
>
> n\. What is the project's risk management process? How is risk
> incorporated into the EAC?

o.  What is the process for reviewing and updating EAC's?

p.  How does the CAM update the schedule? How often are schedules
    updated?

    []{#__RefHeading___Toc157574724 .anchor}**3.2.2.3 IBR Documentation
    and Forms**

> After each CAM discussion, each Team/Sub-team should sit down and
> review what was discussed. This is an opportune time to collect
> documentation to better prepare for the out-brief presentation. Each
> Team/Sub-team Leader is responsible for documenting the team's
> assessments and findings. The Team Leader uses the documentation to
> support overall team assessments and required corrective actions.
>
> The following forms have been developed to facilitate review
> documentation. Sample IBR forms are provided in the IBR Team Handbook
> in Appendix F.

-   **Discussion Assessment Form:**

> The Team/Sub-team should complete one of these forms after each
> discussion. The Team/Sub-team Technical Leader is responsible for
> reviewing this form and submitting it to the Team Leader or his/her
> Review Facilitator. Concerns noted should be documented in the Concern
> Area Report (CAR). Keep in mind that the Discussion Assessment Form
> can be tailored based on risk.

-   **Concern Area Report (CAR):**

> A CAR should be completed for each concern noted. Generally, concerns
> noted are items that require follow-up action. Information on this
> form should be clear and as specific as possible because it will be
> provided to the supplier to obtain a response and resolution. Spell
> out abbreviations. These forms should be submitted immediately to the
> Team Leader, and as soon as practical to the supplier. Responses
> should be provided to the IBR Team as soon as possible. Again, the
> Team/Sub-team Leader is responsible for reviewing this form, and
> submitting it to the Team Leader or his/her Review Facilitator. All
> forms will be compiled and logged.

-   **Documentation Request Form:**

> During your discussions, you may find that you require additional
> documentation to gain a better understanding of the issue in question.
> Use this form to obtain required documentation. Submit the completed
> form to the Team Leader and/or Review Facilitator. All forms will be
> tracked in order to reduce redundancies and ensure receipt of all the
> requested material.

-   **Risk Assessment Form:**

> A risk assessment form should be completed after each discussion. Each
> risk and opportunity identified during the CAM Discussion is assigned
> a probability of occurrence. A potential cost and schedule impact for
> each risk and opportunity should be estimated, and a determination
> made as to whether this risk has been accounted for in the baseline.
> Each risk is classified as cost, schedule, technical, resource, or
> management process and assigned a rating based on risk evaluation
> criteria established by the NASA Program or Project Manager prior to
> the IBR. The ultimate goal of the Risk Assessment area of the IBR is
> an updated Estimate At Complete (EAC) or Life Cycle Cost Estimate
> (LCCE) which incorporates quantified risks. The Team/Sub-team Leader
> is responsible for reviewing this form and submitting it to the Team
> Leader. Risks identified at the IBR should be incorporated into the
> project's existing risk management plan.
>
> All forms should be compiled and logged in an IBR Log. The IBR log
> should consist of both potential and actual issues revealed prior to
> and during the review, and should contain the appropriate actions
> required along with anticipated completion dates. This IBR Log will
> support the Letter of Findings during the IBR Close-Out Phase
> discussed in Section 4. See Appendix G for a sample IBR Log.

[]{#__RefHeading___Toc157574725 .anchor}**3.2.3 Out-briefs**

1.  []{#__RefHeading___Toc157574726 .anchor}**Daily**

> At the end of each day, the NASA IBR Team should conduct out-briefs
> where each team/sub-team reports findings from the CAM discussions.
> This will ensure that the entire team is aware of issues that have
> been raised by other teams, allowing further investigation during
> other CAM discussions.

2.  []{#__RefHeading___Toc157574727 .anchor}**Exit**

> An IBR team meeting, without the supplier, should be conducted prior
> to the final out-brief where the IBR Team can discuss the brief and
> concerns. This will ensure that the IBR Team members have a clear
> understanding of any concerns and action items. The agenda for the IBR
> must ensure that ample time is made available to prepare the exit
> brief presentation.
>
> The Team Leader should provide an exit brief at the end of the IBR.
> The exit brief should be tailored for each project; however several
> areas should be included. First, the brief should include assessments
> of each area's PMB discussed (pros and cons) as well as all Concerns
> and Action Items. Next, a summary of the overall project risk should
> be included as part of the exit brief as well. Finally, the Team
> Leader should provide an overall evaluation of the IBR and assess
> whether or not a follow-on review will be required. This decision is
> based on the Team Leader's assessment of whether the objectives of the
> IBR have been achieved.

A sample IBR Exit Brief can be found in Appendix H.

> The Team Leader may also want to have the Sub-team Technical Leaders
> brief their area(s) of expertise. In addition, the Review Facilitator
> should note to the team any concerns and issues related to the
> management processes.

# **[4- IBR CLOSEOUT]{.underline}**

[]{#__RefHeading___Toc157574729 .anchor}**4.1 General**

Once the IBR exit-brief is completed, the closeout portion of the IBR
process entails the review of all corrective actions identified during
the IBR and the establishment of a plan to formally close out the IBR.
The Program and Project Managers should agree on the closure plan of
action and identify the individual(s) responsible for all identified
risks.

[]{#__RefHeading___Toc157574730 .anchor}**4.2 Reporting Results of the
IBR**

[General:]{.underline}

The results of the IBR should be reported to the supplier. An IBR Report
or a Letter of Findings can be used, depending on the formality of the
IBR. The purpose of the letter or report is to summarize all of the
concerns that were found during the IBR discussions and to request a
corrective action plan from the supplier. The corrective action plan
should identify proposed corrective/preventative actions, responsibility
assignments, and projected completion dates. Issues that could impact
the performance measurement data should also be identified and a copy
provided to the appropriate EVM Working Group member. Appendix I
contains an IBR Report format. The IBR Concern Area reports or a Concern
Area summary spreadsheet would typically be acceptable attachments to
the letter. Also, if an IBR Report is prepared, it should be included as
part of the letter to the supplier.

[Contracts:]{.underline}

The IBR is an event required by the contract, therefore a letter to the
contractor issued by the Contract Technical Representative (COTR) is
customarily sent. The Review Facilitator should work with the Team
Leader to generate the letter to the contractor. A letter will help to
prevent the contractor from claiming that resolving actions resulting
from the IBR are out of scope.

As an alternative, the customer and the supplier could agree on a
corrective action plan immediately after the IBR exit-brief to ensure
more timely resolution of issues. This will make certain that the
concerns are understood and that the IBR team agrees with the
corresponding corrective action plan. This can speed resolution of the
Concern Area Reports and closeout of the IBR.

[]{#__RefHeading___Toc157574731 .anchor}**4.3 Tracking Concerns**

Tracking the progress in resolving each concern rests with the Team
Leader, including the Defense Contract Management Agency (DCMA) team
representatives located at the contractor's facility when applicable.
The Review Facilitator should work closely with the IBR Team to ensure
that all actions are closed. Regular project reviews are also another
good place to have the follow-up on any actions that resulted from the
review. Approval of corrective actions rests with the Team Leader, and
most likely would be an iterative process.

[]{#__RefHeading___Toc157574732 .anchor}**4.4 IBR Closeout Letter**

When all Concern Area Reports have been closed, the Team Leader will
approve close out of the IBR. An IBR closeout letter is sent to the
supplier to indicating that all CAR's are either closed or being tracked
and that the IBR is complete. Reporting the results of the IBR may take
the form of an informal letter or memo for record, depending on the size
and formality of the IBR.