![](media/image1.wmf){width="2.1222222222222222in"
height="0.7326388888888888in"}

Closeout Presentations

Director's/DOE FSO's

Earned Value Management System (EVMS) Review

of

the DECam Project

December 18-19, 2007

This page intentionally left blank

Table of Contents

[Executive Summary 5](#__RefHeading___Toc185829172)

[1.0 Introduction 6](#__RefHeading___Toc185829173)

[2.0 Management 7](#__RefHeading___Toc185829174)

[3.0 Schedule 9](#__RefHeading___Toc185829175)

[4.0 Earned Value 10](#__RefHeading___Toc185829176)

[5.0 Charge Questions 11](#__RefHeading___Toc185829177)

This page intentionally left blank

[]{#__RefHeading___Toc185829172 .anchor}Executive Summary

The DECam project has satisfactorily implemented the bulk of the
elements required for a "self-certified" EVMS. In general, the elements
of the system that are lacking can be made compliant via straightforward
administrative actions to formally capture existing processes in the
system description and clarify others to more clearly describe
noteworthy practices. Additionally, existing system capabilities, such
as producing EAC's, need to be applied to project activities in order to
have robust, fully compliant system. Lastly, the project would greatly
benefit by freezing their baseline in the near future and begin
operating under formal change control prior to receiving CD-2 as a means
of exercising the system and providing the project team with a clear
understanding of those constraints.

The performance management system consists of the software tools, Earned
Value Management System (EVMS) descriptions, project management
documents, and many processes which satisfy DOE Order 413.3A, DOE Manual
413.3-1 requirements, and the ANSI/EIA 748-A-1998 objectives. The
financial aspects of the system are currently being fully and
successfully exercised, and applied to project activities. The committee
interviewed all seven DECam Control Account Managers (CAMs) and found
that while their level of knowledge varied on earned value and the DECam
EVMS Implementation Plan, the level of knowledge was generally
appropriate for the DECam project. The CAMs demonstrated ownership of
their work breakdown structure (WBS) scope, schedule and budget, and had
a good understanding of their roles and responsibilities. Some training
on change control and associated processes will enhance the CAMs ability
to fully understand and benefit from this system.

The DECam project team has successfully addressed performance
measurement issues associated with project effort performed by non-FNAL
organizations, whether funded via FNAL, or effort performed "in-kind".
Such effort funded by FNAL includes direction to those organizations via
MOU's on reporting requirements necessary to support earned value
reporting. "In-kind" effort, while not reflected in the project TPC, is
identified in the schedule, with appropriate logic ties, and a process
for identifying the status of those activities is in place.

On a broader note, because the FRA "site-wide" EVMS description, with
supporting documentation is not yet in place, smaller projects such as
DECam are forced to develop their own plans. While it is intended that
the DECam EVMS be consistent with the future "site-wide" system, there
will undoubtedly be discrepancies when the site-wide system is finally
in place. Additionally, expending the effort to complete a site-wide
EVMS with supporting documentation would seem much more efficient than
having individual project prepare such documentation. Lastly, developing
and implementing a site-wide EVMS would better emphasize and support the
project management culture necessary for successful execution of
projects at FNAL.

Finally, to verify complete and successful EVMS implementation, a follow
up EVMS review similar to this review, should be conducted approximately
three months following CD-2 approval to validate implementation of
recommendations identified.

[]{#__RefHeading___Toc185829173 .anchor}1.0 Introduction

A Fermilab Director's and Department of Energy Fermi Site Office's
Earned Value Management System Review of the DECam Project was held on
December 18-19, 2007. The charge included a list of topics and questions
to be addressed as part of the review. The assessment of the Review
Committee is documented in the body of this closeout presentation.

Each section in this closeout presentation is generally organized by
Findings, Comments and Recommendations. Findings are statements of fact
that summarize noteworthy information presented during the review. The
Comments are judgment statements about the facts presented during the
review and are based on reviewers' experience and expertise. The
comments are to be evaluated by the project team and actions taken as
deemed appropriate. Recommendations are statements of actions that
should be addressed by the project team. A response to the
recommendations is expected prior to the DOE/NSF Review of DES scheduled
for January 29-31, 2008 and actions taken will be reported on during
future Working Group Meetings and reviews.

[]{#__RefHeading___Toc185829174 .anchor}2.0 Management

Primary Writer: Elaine McCluskey

Contributors: Tim Barr, Ed Temple

Findings

-   The DECam project has a comprehensive work breakdown structure that
    includes all project work scope.

-   The project scheduling (MS Project) and project controls (COBRA)
    tools are capable of providing earned value and actual cost data.

-   The project is producing monthly status reports, including a
    narrative and cost performance reporting.

-   CAMs are aware that a change control process exists and that it will
    be implemented when the project is baselined.

-   The project is not calculating estimates at completion and reporting
    these as part of their cost performance reporting.

-   The project plans to analyze variances between planned and actual
    schedule and cost and report on them on a monthly basis, but does
    not have variance thresholds documented at this time.

-   EVMS implementation is described in the DECam EVMS Implementation
    Plan. The CAMs were familiar with how this is to be implemented.

Comments

-   The scheduling tool is being used for in-kind activities including
    resource loading and planning, even though these are at zero cost, a
    good management practice.

-   The project financial officer is commended for producing very useful
    monthly cost performance reports separating M&S and labor for more
    effective analysis.

-   The project CAMs should include producing up-to-date estimates to
    complete as part of the statusing process; or at least make
    management aware of impending changes and that these are documented
    in the monthly progress report.

-   CAMS do not have a clear understanding of the change control process
    and thresholds, which will be utilized after freezing the schedule.

-   The project intends to use DocDB signoffs for controlled project
    documents as part of its configuration management process, but has
    yet to implement it.

Recommendations

1.  "DECam Performance EVMS Implementation Plan\" document needs to be
    identified as a controlled document, approved and put under
    configuration control before the DOE/NSF Review of DES at the end of
    January.

2.  Make Estimates at Completion (EAC) a part of the regular monthly
    cost performance reporting. In addition, a comprehensive bottoms-up
    EAC should be periodically performed, at least annually.

3.  The CAM's should undergo brief training on change management to
    ensure proper procedures will be followed after baselining.

4.  The project should freeze the schedule several weeks before the
    DOE/NSF Review of DES in order to implement change control and fully
    exercise the EVMS process.

5.  The EVMS Implementation plan should address

-   Variance analysis, including thresholds for monthly reporting of
    performance indicator variances and how related corrective actions
    will be addressed.  These thresholds need to be negotiated with the
    Federal Project Director.

-   Ability to manage project contingency in the event of cost savings.

-   The process to perform an annual EAC.

[]{#__RefHeading___Toc185829175 .anchor}3.0 Schedule

Primary Writer: Fran Clark

Contributors: Dean Hoffer, Jeff Sims

Findings

-   The Level 2 managers all have a good working knowledge of the
    scheduling tool.

-   Schedule is linked sufficiently between L2 WBS elements.

-   Task durations, dictionaries, and BOE/resource loading are well
    documented in the MSP schedule notes field.

Comments

-   Schedule is in good shape; freezing the schedule should be done as
    soon as possible.

-   Formal Work Authorization documents have not been completed.

Recommendations

6.  Completion of Work Authorization documents is necessary prior to the
    DOE review in order to be in compliance. These documents are
    produced at the cost account level and include a signed agreement
    between the project manager and the cost account manager stating the
    scope of work, budget for each cost account, and the start date and
    completion date for each cost account. Recommended supplemental
    information includes a copy of the schedule after freezing formatted
    with appropriate columns and a total-cost table from COBRA can be
    attached to the signature page.

[]{#__RefHeading___Toc185829176 .anchor}4.0 Earned Value

Primary Writer: Steve Neus

Contributors: Fran Clark, Elaine McCluskey

Findings

-   The DECam project is capable of providing earned value data via the
    project schedule and the financial information system through COBRA.

-   The DECam project uses appropriate performance measuring techniques
    (PMT). The L2 managers use 0-100%, 50%-50%, % complete, and Level of
    Effort. Primarily they use % complete to determine performance for
    tasks longer than four weeks.

-   Some of the L2 managers have better defined and documented criteria
    for determining the % complete than others. The WBS 1.5 control
    account manager has a good description in DocDB of how % complete is
    applied to his tasks. The L2 managers are very good at reporting the
    PMT data, which is fed into the schedule.

-   The PMT data and schedule are linked to the Laboratory accounting
    system in COBRA. Cost accounts, work packages, budgeted resources,
    and PMTs are maintained in COBRA at the work package level to
    produce a monthly cost performance report.

Comments

-   The DECam good practice of statusing the schedule of "in kind"
    effort should be described in the EVMS Implementation Plan.

Recommendations

7.  \% complete PMTs needs to document specific deliverables at set
    points or steps. For example, the PMT definition for WBS 1.4.4.2
    Design of Prototype Cell, could identify a deliverable for a 30%
    design (x number of drawings), 60% design (x number of drawings),
    90% design (x number of drawings), and delivered prototype.

[]{#__RefHeading___Toc185829177 .anchor}5.0 Charge Questions

1.  Is the system effectively documented?

Yes, if the recommendations from this review are implemented

2.  Will the system produce timely and accurate reports in a readable
    and meaningful format?

A process is in place that will produce both standard and
manager-defined performance reports. Examples of these reports were
shown to the committee. For compliance purposes, the CPR should include
Estimate at Completion/Variance at Completion columns. Monthly reports
were provided to the committee, but it was noted that they did not
include a variance analysis section.

3.  Will the system satisfy Fermilab's and DOE's information needs?

Yes the team believes the EVMS Implementation Plan will satisfy
Fermilab's and DOE's information needs if they follow the above
recommendations.

4.  Are the relevant personnel adequately knowledgeable about operation
    of the system and do they make use of the information?

Some of the CAMs were not yet familiar with how the EVMS will operate
once the project is baselined and MIE activities are authorized. DECam
has been "trying out" this process on the R&D funded activities for
about 3 months, but has not yet implemented their change control process
as documented in the DECam Configuration Management Plan, the Project
Management Plan, and the Project Execution Plan.

5.  Does the DECam Project meet the objectives of the
    ANSI/EIA-748-A-1998 EVMS Guidelines under the:

**a. Organization category?**

I-1 The review committee believes the project complies with the
criteria.

I-2 The review committee believes the project complies with the
criteria.

I-3 The review committee believes the project will be in compliance with
the criteria once a formal work authorization process has been put in
place.

I-4 The review committee believes the project will be in compliance with
the criteria, when the issue of how to control indirect costs from
collaborating institutions is addressed.

I-5 The review committee believes the project is in compliance with the
criteria.

**b. Planning, Scheduling and Budgeting category?**

2-1 The review committee believes the project complies with this
criteria.

2-2 The review committee believes the project complies with this
criteria.

2-3 It is the committee's understanding that FNAL does not use planning
packages. The description should contain a statement to this effect.

2-4 The review committee believes the project complies with this
criteria. However, the review committee believes the response to the
criteria was inappropriate because budgets are established in the
schedule and cost modules. This criteria does not relate to the opening
of cost accounts in the financial system.

2-5 The review committee believes the project complies with this
criteria.

2-6 The review committee believes the project complies with this
criteria. However, the review committee believes the response to the
criteria was inappropriate because the schedule does not burden or
escalate values. COBRA should be used to sum budgets.

2-7 The review committee believes the project complies with this
criteria.

2-8 The review committee believes the project complies with this
criteria.

2-9 The review committee believes the project complies with this
criteria. The description should indicate that FNAL does not maintain
management reserves and undistributed budget. Contingency values should
be identified in the CPR.

2-10 The review committee believes the project complies with this
criteria. This should be done in the CPR, where the BAC + contingency =
TPC.

**c. Accounting Considerations category?**

3-1 The Review Committee believes the project complies with this
criteria.

3-2 The Review Committee believes the project complies with this
criteria.

3-3 The Review Committee believes the project complies with this
criteria, but disagrees with the response from the project because FRA
is the contractor.

3-4 The Review Committee believes the project complies with this
criteria.

3-5 The Review Committee believes the project complies with this
criteria.

3-6 The Review Committee believes the project complies with this
criteria.

**d. Analysis and Management Reports category?**

4-1 The review committee believes the project complies with the
criteria.

4-2 The review committee believes the project will comply with the
criteria when variance thresholds are established and variance analysis
is documented.

4-3 The review committee believes the project complies with the
criteria.

4-4 The review committee believes the project complies with the
criteria.

4-5 The review committee believes the project complies with the
criteria.

**e. Revisions and Data Maintenance category?**

5-1 through 5-5. In their DECam document labeled "Earned Value
Management Systems 32 Criteria," that documents how they meet the
criteria there is a consistent description of the Revisions and Data
Maintenance plans for operation once the project is baselined. When
change control and configuration control processes are implemented, the
system will meet the ANSI / EIA 748 criteria for DECam.

6.  Does the proposed performance management system meet current DOE
    Earned Value Management System (EVMS) assessment and reporting
    requirements?

While the DECam project has satisfactorily implemented the bulk of the
elements required for a "certified" EVMS, the system does not currently
meet DOE EVMS assessment and reporting requirements. In general, the
elements of the system that are lacking can likely be made compliant
prior to the current timeframe for achieving CD-2.