This is the accessible text file for GAO report number GAO-05-123 
entitled 'Department of Energy: Further Actions Are Needed to 
Strengthen Contract Management for Major Projects' which was released 
on April 19, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Committee on Government Reform, House of Representatives: 

March 2005: 

Department of Energy: 

Further Actions Are Needed to Strengthen Contract Management for Major 
Projects: 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-123]: 

GAO Highlights: 

Highlights of GAO-05-123, a report to the Committee on Government 
Reform, House of Representatives

Why GAO Did This Study: 

The Department of Energy (DOE) pays its contractors billions of dollars 
each year to implement its major projectsthose costing more than $400 
million each. Many major projects have experienced substantial cost and 
schedule overruns, largely because of contract management problems. GAO 
was asked to assess, for major departmental projects, (1) DOEs use of 
performance incentives to effectively control costs and maintain 
schedules, (2) the reliability of the data DOE uses to monitor and 
assess contractor performance, and (3) the reliability of the Project 
Assessment and Reporting System (PARS) data that senior managers use 
for project oversight. 

What GAO Found: 

DOE could use performance incentives more effectively for controlling 
costs and schedules if it developed performance incentive guidance and 
assigned responsibility for reviewing a contracts project management 
provisions prior to award. DOE has awarded contracts for 15 of 33 major 
projects that use a schedule or other performance incentive without an 
associated cost incentive or constraint; thus a contractor could 
receive full fees by meeting all schedule baselines while substantially 
overrunning costs. 

DOE has relied on unvalidated contractor data to monitor contractors 
progress in executing major projects and to award fees for performance. 
In particular, DOEs self-assessment of contract administration in 2002 
found that field personnel overly relied on contractors accounting 
systems and contractor-collected data in assessing performance, without 
significant validation of those data. No subsequent self-assessment has 
been conducted to determine if this problem continues. Furthermore, DOE 
has not required that its contracting officers receive the training 
needed to assess the adequacy of contractors project management 
systems that generate data used to monitor progress. 

Although development of PARS is a positive step, the reliability of the 
project performance data that PARS provides to senior DOE managers is 
limited by problems with accuracy, completeness, and timeliness. 
Regarding accuracy, DOE has not assessed the reliability of 
contractors project management systems that feed data into PARS for 31 
of 33 major projects, even though DOE believes that some systems are 
deficient. Regarding completeness, GAO identified 3 major projects that 
are not in PARS. As to timeliness, cost and schedule data for 6 major 
projects in the June 2004 PARS report were significantly out of date 
because DOE has not required contractors to submit timely performance 
data. 

These contract management problems limit DOEs ability to effectively 
manage its major projects and avoid further cost and schedule 
slippages. 

Contractors Performance Awards Relied on Unvalidated Data for 30 of 
the 33 Major Projects GAO Reviewed: 

[See PDF for image]

[End of figure]

What GAO Recommends: 

GAO recommends that DOE strengthen its contract management for major 
projects by, among other things, (1) developing a chapter in DOEs 
Acquisition Guide that specifies a systematic contracting approach for 
major projects; (2) reducing DOEs overreliance on unvalidated 
contractor data in awarding contract fees; and (3) developing a 
schedule for assessing the reliability of contractors project 
management systems, giving priority to systems that DOE believes are 
deficient. In commenting on the draft report, DOE generally concurred 
with all of the recommendations. 

www.gao.gov/cgi-bin/getrpt? GAO-05-123. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Robin Nazzaro at (202) 
512-3841 or nazzaror@gao.gov. 

[End of section]

Contents: 

Letter: 

Results in Brief: 

Background: 

DOE Could Use Performance Incentives More Effectively for Controlling 
Costs and Schedules: 

DOE Has Relied on Unvalidated Contractor Data to Monitor and Assess 
Contractors' Performance for Major Projects: 

The Reliability of PARS Data Is Limited: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: The Department of Energy's 33 Major Projects That We 
Reviewed: 

Appendix II: Scope and Methodology: 

Appendix III: Comments from the Department of Energy: 

Related GAO Products: 

Table: 

Table 1: PARS Reports Assess a Project's Cost and Schedule Performance 
against Only the Current DOE-Approved Baselines: 

Abbreviations: 

DOE: Department of Energy: 

FAR: Federal Acquisition Regulation: 

NNSA: National Nuclear Security Administration: 

OMB: Office of Management and Budget: 

PARS: Project Assessment and Reporting System: 

Letter March 18, 2005: 

The Honorable Tom Davis: 
Chairman: 
The Honorable Henry A. Waxman: 
Ranking Minority Member: 
Committee on Government Reform: 
House of Representatives: 

The Department of Energy (DOE) pays its contractors billions of dollars 
each year to implement projects costing more than $400 million each, 
which the department designates as major projects. These major projects 
include environmental cleanup at its current and former nuclear weapons 
production facilities, refurbishment of nuclear weapons, and 
construction of specialized scientific facilities. Our previous reports 
have found that many of DOE's major projects have experienced 
substantial cost overruns and delays. As a result, since 1990, we have 
designated DOE's contract management--broadly defined to include 
contract administration and project management--as a high-risk area for 
fraud, waste, abuse, and mismanagement because of the department's 
history of inadequate management and oversight and failure to hold its 
contractors accountable. In recent years, we have continued to find 
problems with DOE's contract management of its major projects. (See the 
list of related GAO products at the end of this report.) For example, 
our 2002 assessment of DOE's contract reform initiatives found that 5 
of the 16 major projects we examined had more than doubled in cost--for 
billions of dollars in total cost overruns--and experienced more than 5 
years in delays.[Footnote 1]

In response to these problems, DOE has instituted contracting and 
project management reforms. To better align DOE functions with 
challenges to improve the planning, execution, and management of its 
contracts, the department established the Office of Contract Management 
in 2000 with responsibility for assisting the field and other DOE 
offices in the planning, implementation, and oversight of (1) the 
contract award process, which involves those front-end activities 
necessary to ensure that contracts are structured in a way that best 
fulfills DOE's needs, and (2) the contract administration process, 
which encompasses all dealings between DOE and its contractors from the 
time a contract is awarded until the work has been completed and 
accepted or the contract has been terminated, payment made, and any 
disputes resolved. To improve the contract award process, the Office of 
Contract Management has, among other things, developed updates to DOE's 
Acquisition Guide, which amplifies upon the requirements contained in 
the Federal Acquisition Regulation (FAR) that governs governmentwide 
federal procurement activities. Each year, the Office of Contract 
Management reviews a limited number of pending contract actions, 
including any matters affecting a contract's deliverables, schedule, 
and cost. To improve the contract administration process, the Office of 
Contract Management issued, in May 2000, a Reference Book for Contract 
Administrators, a consolidated reference tool for DOE's contract 
administrators. The office also has performed periodic self-assessments 
of contract administration practices. 

As part of its project management reforms, DOE established the Office 
of Engineering and Construction Management in 1999 to oversee the 
project management process by developing project management policy, 
improving project oversight systems, and implementing a career 
development program for DOE project managers. In particular, the Office 
of Engineering and Construction Management in 2001 implemented the 
Project Assessment and Reporting System (PARS), a Web-based system for 
collecting and analyzing current performance data for projects costing 
more than $5 million. Although the department has other ways to 
communicate project status information, including quarterly reports for 
some projects, PARS monthly project status reports for the Deputy 
Secretary of Energy (hereafter referred to as PARS reports) are DOE's 
primary tool for keeping senior managers apprised of a project's 
performance. In August 2003, the office initiated a certification 
program for contractors' project management systems that assesses the 
accuracy of the cost and schedule performance data that contractors 
generate by examining, in particular, whether each contractor's system 
complies with private industry's standard for earned value management-
-a systematic approach for integrating and measuring cost, schedule, 
and technical (scope) accomplishments on a project. 

You asked us to examine DOE's contract management actions designed to 
control cost growth and schedule slippage for its major projects. 
Specifically, you asked us to assess, for major departmental projects, 
(1) DOE's use of performance incentives to effectively control costs 
and maintain schedules, (2) the reliability of the data DOE uses to 
monitor and assess contractor performance, and (3) the reliability of 
the PARS data that senior managers use for project oversight. Our 
review focused primarily on the 33 major projects that had passed, as 
of March 2004, DOE's Critical Decision 2 milestone--the point at which 
the department approves a project's cost, schedule, and scope baselines 
on the basis of an approved conceptual design report and acquisition 
strategy. (See app. I for the 33 major projects that we reviewed.) The 
projects we reviewed include 28 projects that cost more than $400 
million each and 5 projects that our 2002 assessment defined as major 
projects because their total costs exceeded $100 million. DOE's Office 
of Environmental Management (Environmental Management), Office of 
Science, and National Nuclear Security Administration (NNSA) are 
funding the 33 major projects. 

To determine DOE's guidance for using different performance incentives 
for major projects, we examined the contract provisions for the 33 
major projects that have DOE-approved performance baselines, reviewed 
various recent contract-related documents associated with these 33 
projects, and discussed the contents of these documents with key DOE 
contracting officials. We then compared the contract provisions with 
requirements in the FAR; the DOE Acquisition Regulation; and DOE's 
contract-award guidance, including the Performance-Based Contracting 
Guide and the Acquisition Guide. To assess DOE's oversight of contract 
administration activities, we reviewed DOE's Reference Book for 
Contract Administrators, compared the findings made in DOE's various 
contract administration self-assessment reports with the actions the 
department has taken, and interviewed contracting officials at the 
Department of Defense and administrators at the National Aeronautics 
and Space Administration because DOE officials acknowledged their 
leadership in implementing such project and contract management reforms 
as earned value management principles. To evaluate the reliability of 
PARS data, we assessed the data's accuracy, completeness, and 
timeliness by examining the underlying data and the way data are 
reported to senior managers. For example, we compared the information 
in monthly PARS reports from January through September 2004 with 
project-specific data obtained from various DOE program offices. When 
we identified discrepancies, we assessed whether the discrepancies 
limited the reliability of data reported in PARS. Given our review of 
the documentation provided by DOE and our discussions with DOE 
officials, we have reservations about the reliability of PARS data. 
These issues are discussed in this report. We conducted our work 
between January 2004 and January 2005 in accordance with generally 
accepted government auditing standards, which included an assessment of 
data reliability and internal controls. (See app. II for further 
information about the scope and methodology of our review.)

Results in Brief: 

DOE could use performance incentives more effectively for controlling 
costs and schedules if it developed performance incentive guidance and 
assigned responsibility for reviewing a contract's project management 
provisions. DOE has awarded contracts for 15 of 33 major projects that 
use a technical, schedule, or other performance incentive without an 
associated cost incentive or cost constraint, so a contractor could 
receive full fees by meeting all schedule baselines even though the 
contractor substantially overran baseline costs. For 13 additional 
major projects, DOE has used cost-plus-incentive-fee contracts without 
certifying that contractors' project management systems generate 
reliable cost and schedule data for measuring performance and awarding 
incentive fees. In addition, for 11 major projects that are components 
of the environmental cleanup of a DOE facility, no award fee amount is 
directly linked to the successful completion of the major project, 
generally because the project is part of the contractor's larger 
cleanup responsibility. As a result, the contractor may not give 
sufficient attention to the completion of a complex major project and 
focus, instead, on less critical activities that are easier to achieve. 
Furthermore, an Environmental Management review found that the award 
fee incentives in its contracts were not sufficiently focused on site 
cleanup activities. As a result, Environmental Management decided that, 
beginning in fiscal year 2003, award fee determinations would consider 
only contractor activities directly related to cleanup work, while 
excluding such indirect work-related activities as providing timely and 
accurate reports to DOE because they are basic expectations of any 
contractor. In contrast, NNSA has not conducted a review similar to 
Environmental Management's and, for at least 1 of its major projects, 
NNSA has considered a contractor's indirect work-related activities in 
awarding incentive fees. DOE has not issued guidance that specifies the 
indirect work-related activities to be included in determining 
incentive fee awards. 

DOE has relied on unvalidated contractor data to monitor contractors' 
progress in executing major projects and to award fees for performance. 
This reliance on unvalidated data limits the department's ability to 
ensure it gets what it is paying for. Specifically, DOE's self-
assessments of contract administration in 1997 and 2002 found that 
field personnel overly relied on contractor accounting systems and 
contractor-collected project data in awarding fees, without significant 
validation of those data. However, the 2002 self-assessment made no 
recommendation for fixing the problem, and no broad self-assessment of 
contract administration has been conducted since 2002. Without a 
specific recommendation, implementation of that recommendation, and 
periodic self-assessments, DOE lacks a mechanism for ensuring that 
overreliance on contractor data and other contract administration 
problems are corrected. DOE has begun to certify the reliability of 
contractors' project management systems that generate the performance 
data used to monitor contractors' progress; however, the department has 
no time table for the completion of this certification program. 
Furthermore, with the exception of NNSA, DOE has not required its 
contracting officers to receive training in earned value management, 
even though contracting officers are required to determine whether the 
contractor's project management system, after contract award, meets 
private industry's earned value management standard. In contrast, 
administrators at the National Aeronautics and Space Administration 
told us that their contracting officers need training to adequately 
assess contractors' use of earned value management. 

Although the development of PARS is a positive step toward improving 
DOE's project oversight, the reliability of the project performance 
data that PARS provides to senior managers is limited by problems with 
the data's accuracy, completeness, and timeliness. 

* The accuracy of the PARS cost and schedule data is uncertain for 
three reasons. First, even though DOE believes that some of its 
contractors' project management systems are deficient, the department 
has assessed the reliability of systems for only 2 of the 33 major 
projects we reviewed. Additionally, despite the department's goal of 
assessing all systems by the end of fiscal year 2006, it has not 
developed a schedule for reviewing other contractors' project 
management systems. As a result, senior DOE managers cannot rely on 
PARS performance data for many major projects to determine whether the 
projects are on cost and on schedule. For example, DOE officials told 
us that they do not know if the $5.7-billion Tank Waste Treatment and 
Immobilization Plant project at Hanford, Washington, is meeting its 
performance baselines, even though PARS data show that the project is 
on track. Second, the cost and schedule variances reported in PARS for 
many projects are small and do not accurately reflect the considerable 
cost growth and schedule slippage that have occurred in the past 
because DOE updates the PARS baselines when project changes are 
approved. DOE officials told us that departmental project management 
guidance does not require the reporting of life-of-project cost and 
schedule variances; however, PARS cannot reflect prior cost overruns 
and schedule slippages without such reporting. Third, most of DOE's 
project directors have not received earned value management training, 
which is needed to ensure that contractors' project management systems 
are providing accurate performance data to PARS, and DOE does not 
expect to complete this training until May 2006. 

* Regarding the completeness of these data, we identified 3 major 
projects, as well as 2 smaller projects, that are not included in PARS. 
In particular, while DOE stated in July 2003 that it intended to treat 
each of its nuclear weapons refurbishment programs as projects, none of 
these refurbishments have been added to the PARS database. Although DOE 
recently asked its program offices to identify and enter additional 
projects into PARS, implementation has been slow in part because key 
program office officials lack project management training, which is 
necessary for determining what activities are subject to PARS 
reporting, according to Office of Engineering and Construction 
Management officials. For projects that are being reported, PARS lacks 
forward-looking trend data, such as the projects' estimated cost at 
completion that, according to numerous officials, are critical for PARS 
to effectively report possible performance challenges. Although these 
data are available, they are not included in PARS reports because 
Office of Engineering and Construction Management officials want to 
minimize the amount of information that senior managers review. 

* Regarding timeliness, the June 2004 PARS report's performance data 
for 6 major projects were significantly out of date primarily because 
contractors had not submitted more current performance data, according 
to Office of Engineering and Construction Management officials. As a 
result, the PARS monthly reports did not show senior DOE managers the 
need for corrective actions in response to cost and schedule slippages. 
While some minor lag is to be expected in reporting project 
performance, Office of Engineering and Construction Management 
officials noted they are open to exploring options to improve the 
timeliness of PARS data in those cases where data is significantly out 
of date. 

We are making several recommendations to the Secretary of Energy to 
ensure the use of effective performance incentives, strengthen 
oversight of contract administration, and improve the reliability of 
the project performance data provided by PARS. DOE generally concurred 
with all of the recommendations. DOE also provided a number of comments 
to improve the report's accuracy, which we incorporated where 
appropriate. 

Background: 

DOE is the largest civilian contracting agency in the federal 
government; about 90 percent of its annual budget is spent on contracts 
for carrying out its activities and operating its facilities. In 
fulfilling their missions, DOE's program offices are responsible for 
contracting for and overseeing the execution of the department's major 
projects, many of which are first-of-a-kind efforts and thus involve 
substantial risk and may also be separate line items in DOE's budget. 
For example: 

* Environmental Management's mission is to accelerate risk reduction 
and cleanup of the environmental legacy of the nation's nuclear weapons 
program and government-sponsored nuclear energy research. Environmental 
Management has used a single sitewide contract that involves several 
major projects costing billions of dollars for cleaning up some of its 
former facilities. In addition, Environmental Management has undertaken 
many large-scale individual projects. For example, the Hanford Tank 
Waste Treatment and Immobilization Plant project is an important part 
of the cleanup effort at Hanford, Washington. The project, which was 
initiated in December 2000, is intended to treat and prepare for 
disposal 55 million gallons of high-level radioactive waste by July 
2011 at an estimated cost of $5.7 billion. 

* NNSA's mission is to meet national security requirements by, among 
other things, maintaining and enhancing the safety, reliability, and 
performance of the U.S. nuclear weapons stockpile, which includes 
maintaining the capability to design, produce, and test nuclear 
weapons. To fulfill this mission, NNSA undertakes such projects as 
refurbishing W-80 nuclear warheads to extend their operational lives. 
The W-80 refurbishment project was initiated in September 1998 and is 
expected to be completed in fiscal year 2017 at an estimated cost of 
about $2.45 billion. 

* The Office of Science's mission is to deliver the remarkable 
discoveries and scientific tools that transform our understanding of 
energy and matter and advance the national, economic, and energy 
security of the United States. To fulfill this mission, the Office of 
Science has constructed specialized scientific research facilities, 
such as the Spallation Neutron Source at the Oak Ridge National 
Laboratory. This project consists of an accelerator system that 
delivers short (microsecond) pulses to a target/moderator system where 
neutrons are produced by a nuclear reactor process called spallation. 
This project is designed to provide the next-generation spallation 
neutron source for neutron scattering and related research in broad 
areas of the physical, chemical, materials, biological, and medical 
sciences. The Spallation Neutron Source project began in October 1998 
and is expected to be completed in June 2006 at an estimated cost of 
about $1.4 billion. 

DOE's principal official responsible for the execution of a major 
project is the federal project director, who is located at the project 
site and is supported by project managers. The project director is 
responsible for overseeing a project's design, execution, budgeting, 
and performance. For contracts with award fee provisions, senior DOE 
program office managers consult with contracting and project officers 
to assess a contractor's performance and determine the appropriate 
award fees. 

In addition to the contract management problems our prior reports have 
identified, a recent series of reports by the National Research Council 
of the National Academies identified weaknesses in DOE's project 
management. The council's 2004 report cited several factors that have 
contributed to the slow pace of project management improvements and 
resulted in inconsistent project performance.[Footnote 2] These factors 
include the desire of DOE site office personnel and contractors to be 
independent of oversight from DOE headquarters, insufficient support 
for training, inadequate numbers of DOE project managers to oversee 
contractors' performance, and the absence of a champion for project 
managers and process improvement who has the authority to ensure both 
adherence to policies and procedures and the availability of necessary 
funding and personnel resources. 

During the past year, DOE has continued to implement contracting and 
project management reforms. In particular, in December 2003, the 
Secretary of Energy appointed an Associate Deputy Secretary with 
responsibility, among other things, for both contract and project 
management, addressing a key National Research Council concern. DOE 
also entered into an agreement with the Defense Contract Management 
Agency, within the Department of Defense, to support the certification 
of contractors' project management systems. More recently, DOE is 
developing an action plan in response to the Civil Engineering Research 
Foundation's assessment of departmental project management that 
recommended that DOE, among other things, develop a core group of 
highly qualified project directors, require peer reviews for first-of-
a-kind and technically complex projects when the projects' preliminary 
baselines are approved, and enhance PARS by making the data more 
timely.[Footnote 3] Furthermore, to improve its contract award process, 
DOE revised its Acquisition Guide by adding chapter 16, which lists the 
various contract types available and discusses their respective 
advantages and constraints. To address future skill gaps in its 
procurement organization, DOE established an acquisition career 
development program and has certified 90 percent of its procurement 
professionals as attaining mandatory training and experience standards 
under this program. Within the Office of Environmental Management, a 
series of contract and project management improvements have occurred 
consisting of, but not limited to, providing additional training and 
managing more of the cleanup work as projects. Within the Office of 
Contract Management, a series of contract award and administration 
initiatives have been completed. These initiatives include, among other 
things, strengthening contract competition policies and practices, 
improving acquisition workforce effectiveness, increasing small 
business utilization throughout DOE, and strengthening DOE management 
and fiscal effectiveness. For fiscal year 2005, the Office of Contract 
Management has multiple initiatives planned, including identifying and 
implementing follow-on actions related to the DOE management challenge 
pertaining to contract competition. 

DOE Could Use Performance Incentives More Effectively for Controlling 
Costs and Schedules: 

Because many of DOE's major projects are first-of-a-kind and thus 
involve substantial risk, DOE's contracting decisions can be critical 
to the successful completion of its major projects. However, DOE could 
use performance incentives more effectively for controlling costs and 
schedules for its major projects if the department developed criteria 
for using different performance incentives and assigned responsibility 
for reviewing a contract's project management provisions prior to 
award. For example, DOE has used contracts that have a technical, 
schedule, or other performance incentive without an associated cost 
incentive or cost constraint (other than the annual funding level for 
the contract). DOE also has used cost-plus-incentive-fee contracts 
without certifying that contractors' project management systems 
generate reliable cost and schedule data for measuring performance and 
awarding fees. In addition, we found that the contract incentives for 
most of the 25 major environmental restoration projects substantially 
differ from the "Gold Chart" performance metrics that Environmental 
Management uses to assess its performance and report its progress to 
the Congress. Furthermore, for 11 major projects that are components of 
the environmental cleanup of a DOE facility, Environmental Management 
has not directly linked incentive fees to the successful completion of 
the project, generally because the project is part of the contractor's 
larger cleanup responsibility. Finally, while Environmental Management 
has decided that incentive fee determinations would consider only 
contractor activities directly related to cleanup work, NNSA has, for 
at least 1 of its major projects, considered a contractor's indirect 
work-related activities in awarding incentive fees. 

DOE Has Not Fully Developed Performance Incentive Guidance to 
Effectively Control Costs and Maintain Schedules: 

Despite efforts in recent years to improve contract and project 
management, DOE has not fully developed performance incentive guidance 
to effectively control costs and maintain schedules. DOE has issued the 
following guidance, order, and manual that are applicable to the 
contract award process for major projects and that supplement the FAR 
and the DOE Acquisition Regulation: 

* In the late 1990s, DOE issued its Acquisition Guide to, among other 
things, supplement the FAR and the DOE Acquisition Regulation and be a 
repository of best practices found throughout the department. Chapter 
16 of the guide discusses contract types; however, the chapter notes 
that it was not intended to provide a template for matching a contract 
type to given contracting situations. While the guide's index shows 
that chapter 34 is reserved for guidance to contracting officials 
related to major projects, DOE has never drafted the chapter, according 
to the DOE official responsible for maintaining revisions to the 
Acquisition Guide. 

* In October 2000, DOE issued Order 413.3, "Program and Project 
Management for the Acquisition of Capital Assets," to ensure that 
capital assets, including major projects, would be delivered on 
schedule, within budget, and fully capable of meeting mission needs. To 
accomplish these goals, the order states, in part, that DOE officials 
are to develop an acquisition plan during the acquisition process that 
includes such elements as contracting options and a contractor 
incentive process. The order, however, does not elaborate on the 
possible contracting and performance incentive options whatsoever. 

* In March 2003, DOE issued manual 413.3-1, "Project Management for the 
Acquisition of Capital Assets," to improve the implementation of DOE 
Order 413.3. The manual addresses various activities, including a 
chapter on contracting that contains no direct reference to major 
projects. The chapter states that the type of contract and incentives 
proposed should be based on an overall view of the principal risks to 
the project and provides a limited discussion of the types of contracts 
available. For example, it states that fixed-price contracts are not 
appropriate for research and development efforts or other complex 
projects where there is a high degree of uncertainty in the execution 
or DOE requirements. While the chapter mentions that DOE generally uses 
a cost-plus-award-fee contract for contractors managing and operating 
DOE sites, it does not address the other available types of contract. 

Furthermore, DOE has not used its Acquisition Guide to identify best 
practices, or lessons learned, based on its major project contracting 
experiences. In our view, given DOE's long history with major projects, 
considerable information could be added to this guide detailing those 
major project contracting approaches that worked and those that did 
not. Improved guidance could help DOE better control costs and maintain 
schedules for its major projects. 

DOE Has Not Always Reviewed the Project Management Provisions of Its 
Major Contracts Prior to Award: 

Neither the Office of Contract Management nor the Office of Engineering 
and Construction Management always reviews the project management 
provisions of major project contracts prior to award to ensure that the 
performance incentives are appropriately used. At the heart of this 
problem is confusion over responsibility. The Director of the Office of 
Contract Management and the Director of the Office of Engineering and 
Construction Management each believe that the other office has 
headquarters responsibility for reviewing the project management 
provisions of contracts prior to approval. 

The confusion exists because the chapter in DOE's Acquisition Guide on 
the headquarters review of contract and financial assistance actions is 
silent on the role of the Office of Engineering and Construction 
Management in the review process. This chapter indicates that packages 
pertaining to contract actions will be sent to nine different DOE 
offices for review, none of which is the Office of Engineering and 
Construction Management. As a consequence, if this office has a role in 
the contract review process, it has not been clearly defined. 

According to the Director, Office of Contract Management, the Office of 
Engineering and Construction Management should be responsible for 
reviewing the project management provisions in major project contracts 
because of its responsibility for project management matters. The 
director told us that his office typically reviews from 60 to 70 
pending contract actions each year, and these reviews follow a general 
approach looking at any matters that might affect timing, delivery, and 
cost--but no specific, formalized list is followed. 

According to the Director of the Office of Engineering and Construction 
Management, his office reviews certain documentation that could affect 
which company is selected for a contract, but his office has no role in 
reviewing the actual provisions of the contract. While the Office of 
Contract Management sends contract proposals to the Office of 
Engineering and Construction Management for review, the director noted 
that his office has only one staff person with contracting experience. 
The director believes the solution to improving the review of major 
project contracts is for contracting officials within the Office of 
Contract Management to become more familiar with earned value 
management, a DOE contracting requirement for integrating and measuring 
a contractor's performance. 

Problems Have Developed Because DOE Has Not Effectively Used 
Performance Incentives: 

For many of the 33 major projects we reviewed, DOE has used performance 
incentives that limit its ability to effectively control cost and 
schedule performance. For example, almost all of DOE's cost-plus-award-
fee contracts for major projects have included a performance incentive 
without also using an associated cost incentive or cost constraint 
(other than the annual funding level for the contract). Also, DOE has 
used cost-plus-incentive-fee contracts without certifying that 
contractors' project management systems generate reliable cost and 
schedule data for measuring performance and awarding fees. We also 
found that (1) Environmental Management's contracts included 
environmental cleanup performance incentives that differed 
substantially from its new Gold Chart performance metrics; (2) DOE did 
not always link its fee awards to contractors' performance on major 
projects; and (3) DOE's program offices have treated indirect work-
related activities, such as providing timely and accurate reports to 
DOE, differently in determining the contractors' incentive award fees. 

DOE's Implementation of Performance Incentive Provisions Does Not 
Enable DOE to Effectively Control Costs: 

For 15 of the 17 major projects that use a cost-plus-award-fee 
contract, the contract contained a technical, schedule, or other 
performance incentive without including an associated cost incentive or 
cost constraint (other than the annual funding level for the contract). 
Under such circumstances, the potential exists that a contractor could 
meet all incentives and overrun baseline costs but still receive full 
fees. The other 2 major projects used a cost-plus-award-fee contract 
that included an associated cost incentive or cost constraint for each 
technical, schedule, or other performance incentive. 

The FAR, the DOE Acquisition Regulation, and DOE guidance preclude the 
inclusion of a schedule or other performance incentive without also 
including a cost incentive or cost constraint. FAR  16.402-1 states 
that no incentive contract may provide for other incentives without 
also providing for a cost incentive or cost constraint. Similarly, DOE 
Acquisition Regulation  970.5215-3 provides that requirements 
incentivized by other than cost incentives must be performed within 
their specified cost constraint. DOE's Performance-Based Contracting 
Guide, dated October 2003, states that (1) cost incentives should be 
included if other incentives are included because a schedule or other 
performance incentive may result in the contractor paying little 
attention to the cost of achieving those incentives unless cost is also 
a consideration and (2) DOE contracts, in developing incentives and 
incentive programs, must comply with the incentive contract provisions 
of the FAR and the DOE Acquisition Regulation. 

The Director of the Office of Contract Management told us that to 
implement the FAR requirement to include a cost incentive or cost 
constraint whenever a noncost incentive is in the contract, each 
noncost incentive does not necessarily need an associated cost 
constraint dedicated to that noncost incentive. According to the 
director, a single cost constraint, which could be equivalent to the 
project's annual funding level, would fulfill the FAR 
requirement.[Footnote 4]

However, DOE contracting officials at Oak Ridge, West Valley, and 
Savannah River believe that to implement the FAR and DOE Acquisition 
Regulation requirements in a way that effectively controls costs, a 
contract with a technical, schedule, or other noncost incentive should 
also have an associated cost incentive to function as a constraint on 
the expenditure of funds. One of these officials added that as the 
noncost incentives become more objective and measurable, the cost 
constraint should be more clearly defined in relation to each noncost 
incentive. Similarly, another one of these officials told us that using 
the annual funding level or the project's cost baseline as the 
constraint is too vague and unworkable, and that some funding levels 
and cost baselines do not track down to the performance incentive 
level. As a result, neither the funding level nor the cost baseline 
would indicate whether the performance incentive was accomplished 
within the cost constraint. 

These views are consistent with the findings from DOE's 1997 assessment 
of performance-based incentives, which found that DOE's and 
contractors' financial systems generally are budget-based and do not 
segregate and track costs at the performance incentive level. The 
assessment added that this limits DOE's ability to establish meaningful 
cost baselines and to monitor the cost of performance under specific 
incentivized work efforts in relation to the total cost of the 
contract. 

DOE Has Used Cost-Plus-Incentive-Fee Contracts without Certifying the 
Reliability of Contractors' Performance Data: 

For 13 of the 33 major projects we reviewed, DOE used a cost-plus-
incentive-fee contract that provides the contractor with an initially 
negotiated fee that is subsequently adjusted by a formula based on the 
relationship of total allowable costs to total target costs. The 
formula provides, within limits, for fee increases when total allowable 
costs are less than target costs. In recent years, DOE has made a major 
effort to move toward the use of cost-plus-incentive-fee contracts. 

Because a cost-plus-incentive-fee contract provides higher fee awards 
to the extent that actual costs are lower than anticipated, it depends 
upon reliable cost estimating at the outset in the form of a target 
cost and reliable cost reporting later. In July 1997, the Office of 
Management and Budget (OMB) issued requirements regarding the 
acceptability of contractors' project management systems. However, DOE 
has not certified the reliability of contractors' project management 
systems that generate the target cost data for the 13 major 
projects.[Footnote 5] As a result, a contractor might receive a high 
fee payment because its project management system generated an 
unreliable high initial cost estimate and subsequently reported lower 
actual costs. A U.S. Army Corps of Engineers' report, issued in May 
2004, concluded that it was not appropriate to use a cost-plus-
incentive-fee contract for the Hanford Tank Waste Treatment and 
Immobilization Plant project, in part because reliable cost data could 
not be generated in advance.[Footnote 6]

Furthermore, DOE site personnel may not provide adequate surveillance 
of the contractors' cost records for these 13 projects. According to 
DOE's Performance-Based Contracting Guide, it is inappropriate to use a 
cost-plus-incentive-fee contract if there is an overreliance on 
contractor accounting systems and contractor-collected data without 
significant validation of those data.[Footnote 7] In such situations, 
the guide states, any potential cost savings reported might be the 
result of a poor estimate of the amount of labor or material required, 
the approach planned, or the associated costs. The Office of Contract 
Management's self-assessment of contract administration in 2002 found 
that most of the DOE field locations visited relied almost exclusively 
on the contractors' data because they did not have the staff resources 
capable of validating cost or technical baselines. The report, however, 
did not identify the DOE field locations visited, and, according to an 
Office of Contract Management official, no individual field location 
reports were prepared. 

Contracts' Performance Incentives Differed from the Gold Chart 
Performance Metrics: 

For 16 of the 25 major environmental restoration projects that we 
reviewed, the contracts' performance incentives differed substantially 
from the Gold Chart performance metrics that Environmental Management 
uses to assess its performance and report its progress to the 
Congress.[Footnote 8] Environmental Management developed the Gold Chart 
performance metrics in October 2002 as a basis for clearly and 
objectively showing the progress being made in the environmental 
cleanup program. We found, however, that these Gold Chart metrics were 
not being used to measure contractors' performance or award fees. 
Instead, DOE measures performance and awards fees on the basis of 
information from the contractors' project management systems, which DOE 
has not yet certified as capable of producing reliable information. 

For 4 projects at the Fernald Closure Site in Ohio, a lower performance 
fee might have been appropriate if the Gold Chart metric had been used. 
For fiscal year 2003, DOE awarded the contractor about $7.7 million of 
the $8 million in available fee, or 97 percent, on the basis of 
acceptable cost and schedule performance toward closure of the entire 
site during fiscal year 2003. However, according to the fiscal year 
2003 Gold Chart metrics, the goal for the Fernald Closure Project was 
to accomplish four radioactive facility completions and dispose of 
2,568 cubic meters of radioactive waste.[Footnote 9] According to 
Environmental Management information, the contractor did not fully 
complete one of these tasks. Because the contractor accomplished only 
three of the four radioactive facility completions, Environmental 
Management might have given a different fee amount if the two Gold 
Chart metrics had been used to determine award fee. 

Conversely, a different fee amount might have been warranted for the 
Solid Waste Stabilization and Disposition project at Hanford, 
Washington. For fiscal year 2003, DOE awarded the contractor about $2.2 
million of about $3 million in available fee, or 73 percent, on the 
basis of the contractor's disposal of radioactive waste in accordance 
with an approved schedule that DOE determined the contractor had met. 
In contrast, Environmental Management data for fiscal year 2003, using 
Gold Chart metrics, show that the contractor actually disposed of 3,634 
cubic meters of waste as compared with a goal of disposing 2,320 cubic 
meters of waste, or about 157 percent of the work intended. If the Gold 
Chart metrics had been used to determine the award fee, the contractor 
might have received a different fee amount. 

For the Spent Nuclear Fuels project, at Hanford, Washington, the Gold 
Chart metric and the contract's performance incentive were so 
dissimilar that it was difficult to determine how to gauge the 
contractor's performance. For fiscal year 2003, DOE awarded the 
contractor about $2.8 million of about $3.3 million in available fee, 
or 85 percent, on the basis of the contractor's removing 777 metric 
tons, or 87 percent, of the 890 metric tons that had been planned. 
However, Environmental Management data for fiscal year 2003, using the 
Gold Chart metrics, show the contractor removed 805 units, or 94 
percent, of the goal's 855 units. Because the Gold Chart metric and the 
contract's performance incentive were so dissimilar, we could not 
reconcile the information. 

Environmental Management officials told us that the performance 
incentives contained in environmental cleanup contracts and the Gold 
Chart metrics should be aligned. In commenting on the draft report, 
Environmental Management officials stated that the new Savannah River 
cleanup contract incorporates Gold Chart metrics. They added, however, 
that the contract renewals for the Oak Ridge, Fernald, and Rocky Flats 
facilities do not contain the Gold Chart metrics because each is a cost-
plus-incentive-fee contract that awards fee based on the final closure 
costs and date for the site. It is unclear whether these cost-plus-
incentive-fee contracts will more effectively track contractors' 
performance because they rely on contractors' project management 
systems that DOE has yet to certify. In contrast, the Gold Chart 
metrics assess the accomplishment of discrete amounts of work that is 
verifiable. 

Incentive Fees Paid to Contractors Were Not Directly Tied to 
Performance for Some Major Projects: 

In 1996, we reported that a key factor inhibiting the successful 
completion of DOE's major projects was the lack of effective 
incentives.[Footnote 10] To the extent that incentives are properly 
applied, they can help achieve agency goals. On the other hand, if 
incentives are nonexistent or not effectively applied, a project may 
not be successfully completed. 

Sixteen of the 33 major projects we reviewed had no incentive fees 
directly associated with the successful completion of work. Nine of 
these 16 projects involve closure work at the Fernald and Rocky Flats 
sites, where the payment of incentive fees is based on an overall 
average of the cost and schedule status for all site closure 
activities, including major projects and other site activities. 
Environmental Management officials told us that rather than awarding 
incentive fees specifically for completing any of the 9 major projects, 
or for other key interim milestones, the Fernald and Rocky Flats 
contracts award provisional incentive fees for meeting or exceeding 
overall targets for a fiscal year, provided the contractors 
successfully achieve site closure on schedule. 

However, it remains to be seen whether this approach will be effective 
in completing major projects on time and within cost. For example, 
although a major project at the Fernald site that we reviewed was 
experiencing cost growth to the point where it was expected to exceed 
its cost baseline--the total cost estimate to accomplish the project--
DOE considered the overall average of the cost and schedule status for 
all site activities at Fernald to be acceptable and paid the contractor 
provisional incentive fees for fiscal year 2003. Similarly, a major 
project at the Rocky Flats site had overrun its estimated cost by about 
$42 million through fiscal year 2003. However, this overrun was offset 
by an underrun of about $46 million in activities such as general 
counsel work and planning and integration that, according to DOE 
information, had historically been understaffed. The net effect was 
that DOE paid the contractor provisional incentive fees because the 
contractor's overall cost and schedule status for fiscal year 2003 was 
considered to be acceptable. 

DOE Program Offices Have Treated Indirect Work-Related Activities 
Differently in Awarding Incentive Fees: 

In addition to these other contracting problems, we found that DOE 
program offices treated indirect work-related activities differently in 
awarding incentive fees. In late 2002, Environmental Management decided 
that award fee determinations will consider only contractor activities 
directly related to cleanup work, while excluding such indirect work-
related activities as providing timely and accurate reports to DOE, 
providing support services to the government, and complying with the 
contract because these activities are basic expectations of any 
contractor. Environmental Management made this determination after its 
review of contractors' authorized fee incentives identified numerous 
examples of incentive fee payments for indirect work-related 
activities. The review also found that Environmental Management was 
paying some contractors additional fees for performing work safely that 
the review concluded was a basic expectation, and not exceptional 
performance worthy of additional fee. 

NNSA has not conducted a review similar to Environmental Management's 
assessing what, if any, indirect work-related activities are worthy of 
incentive payments. The contractor for one NNSA major project received 
incentive fee payments for providing timely and accurate reports to DOE 
and other indirect work-related activities during fiscal year 2003. 
Discrepancies in the treatment of various indirect work-related 
activities have occurred because DOE's guidance does not address the 
appropriateness of including a contractor's performance of indirect 
work-related activities in determining incentive fee awards. 

In commenting on the draft report, Environmental Management expressed 
concern that it would be virtually impossible to develop meaningful 
guidance that could be applied universally to DOE's diverse programs. 
We disagree. We believe that all DOE programs should use incentive fees 
to reward contractors for achieving work-related activities, as opposed 
to such indirect activities as providing the DOE programs with timely 
reports. 

DOE Has Relied on Unvalidated Contractor Data to Monitor and Assess 
Contractors' Performance for Major Projects: 

Because most of DOE's operations are carried out through contracts, 
contract administration is a significant part of DOE's work. DOE has 
relied on unvalidated contractor data to monitor contractors' progress 
in executing major projects and awarding fees for performance.[Footnote 
11] This reliance on unvalidated data limits the department's ability 
to ensure it gets what it is paying for. Specifically, DOE's self-
assessments of its contract administration in 1997 and 2002 both found 
that field personnel overly relied on contractor accounting systems and 
contractor-collected project data without significant validation of 
these data. However, unlike the 1997 self-assessment, the one in 2002 
made no recommendation to fix this problem, and no subsequent self-
assessment has been initiated to determine if the problem has 
continued. DOE has begun to certify the reliability of contractors' 
project management systems that generate the performance data used to 
monitor contractors' progress; however, the department has no time 
table for the completion of this certification program. In addition, 
DOE has not required its contracting officers and contracting officer 
representatives to receive training in earned value management--a 
systematic approach for integrating and measuring cost, schedule, and 
technical (scope) accomplishments on a project or task--even though 
these officials are required to determine whether contractors' project 
management systems meet the private industry's earned value management 
standard. 

DOE Has Not Used Self-assessments to Correct an Overreliance on 
Contractor Data: 

Self-assessment is an important tool for evaluating organizational 
effectiveness. By taking a comprehensive look at itself, an 
organization can identify weaknesses and plot a course of corrective 
action. DOE performed comprehensive self-assessments of its contract 
administration practices in 1997, 1999, and 2002. 

In 1997, DOE assessed 20 contracts to ensure that financial incentives 
contained in those contracts were rational, linked to well-defined 
performance objectives and measures, and properly administered. The 
self-assessment reported both positive and negative findings. For 
example, it found that the use of performance-based objectives 
generally had been effective in directing contractors' management 
attention to desired performance outcomes. However, it also found that 
field personnel overly relied on contractor accounting systems and 
contractor-collected data without significant validation of these data, 
and that DOE's approval of fees earned by the contractors relied upon 
contractor-generated documents. To correct this deficiency, the self-
assessment recommended (1) that the cognizant DOE heads of contracting 
at each field location, as part of their overall contract 
administration plan, identify the mechanisms, responsibilities, and 
authorities for ensuring that contractor performance against 
established objectives is appropriately monitored and (2) that 
performance achievements are verified. 

In 1999, DOE's follow-up assessment of the effectiveness of the actions 
taken in response to the 1997 self-assessment found that the 
recommendation that contractor performance be monitored and 
achievements verified had been implemented. Specifically, field offices 
reported that their plans for administering contracts had been 
appropriately modified and instituted. In addition, the follow-up 
assessment stated that (1) early results indicated a substantial 
improvement in the way incentives were being managed from DOE 
headquarters and administered at DOE field contracting offices and (2) 
anecdotal evidence suggested that contractor performance had improved. 

In 2002, the Contract Administration Division again performed a self-
assessment that examined, in part, how contract administration planning 
and execution was conducted at various DOE field locations. The 
findings and conclusions of this review were somewhat inconsistent with 
those of the 1999 follow-up assessment. The 2002 review, like the 1997 
assessment, determined that few sites had the resources capable of 
validating contractor cost or technical information and most sites must 
rely almost exclusively on the contractor's data. The review noted, in 
one instance, that financial data provided by the contractor were 
generally accepted by DOE, not on the basis of reasonableness and 
allowability, but on the basis of the contractor's "acceptable" self-
assessment of the procedures used to collect those data. However, 
unlike the 1997 assessment, the 2002 review contained no specific 
recommendation to correct this overreliance on contractor data. 

According to the Director of DOE's Contract Administration Division, 
because of funding constraints and other factors, no broad self-
assessment of contractor administration has been done since 2002. The 
director added that DOE now conducts individual site assessments as 
necessary rather than conducting more comprehensive assessments. 
According to information provided to us in April 2004, the last 
individual site assessment was made in August 2003 and documented in 
December 2003. This site assessment identified problems similar to 
those reported in the 2002 self-assessment. Specifically, the site 
assessment noted that, with respect to one contract reviewed, there was 
no evidence of effective cost controls and/or contract management. The 
site assessment contained no formal recommendation to fix this problem. 
On the other hand, the site assessment contained a recommendation to 
address the high rate of expenditure on this contract over the 
remaining 2-year-option period. The assessment recommended that the DOE 
site office review the scope and cost of its current task orders for 
prioritization and inclusion in the remaining option term. 

In August 2003, DOE began to certify the reliability of contractors' 
project management systems that generate the performance data used to 
monitor contractors' progress. However, as of December 2004, the 
department has assessed and certified project management systems for 
only 2 of the 33 major projects we reviewed and does not have a time 
table for completing this certification program. 

In commenting on the draft report, DOE noted that both Environmental 
Management and the Office of Engineering and Construction Management 
have been validating contractors' cost and schedule performance 
baselines for several years. In our view, DOE validation of contractor 
baselines will not fully address the problems that have been 
identified. Validating baselines is just the first step in performing 
adequate contractor oversight. After baselines have been validated, DOE 
must not overly rely on contractor accounting systems in reporting 
costs and on contractor-collected project data in awarding fees. That 
is the message from two DOE self-assessments of performance-based 
contracting. With respect to DOE's experience in baseline validation, 
the Civil Engineering Research Foundation's July 2004 report for the 
Office of Engineering and Construction Management found that some 
improvements in baseline validation were needed. This report noted that 
many of the DOE projects it reviewed were formulated with inadequate 
baseline estimates. In addition, the report stated that periodic 
baseline changes were occurring that masked the true status of certain 
projects. The report recommended that DOE develop guidelines that 
appropriately control the rebaselining of projects. 

DOE further stated that the promulgation of contract management 
planning guidance and the requirement for a contract management plan 
addressed many of the issues that the 2002 self-assessment identified. 
However, in our view, until a subsequent assessment is done, it remains 
unclear whether this DOE action has adequately resolved the issues 
identified in the 2002 self-assessment. For fiscal year 2005, DOE is 
planning to examine the contract management plans and contractors' 
purchasing systems. 

DOE Has Not Required Its Contracting Officers and Contracting Officer 
Representatives to Receive Earned Value Management Training: 

During the early 1990s, OMB issued several reports on civilian 
agencies' contract administration practices that found that agencies 
frequently experienced cost overruns and delays in receiving goods and 
services because their contracting officials allocated more time to 
awarding contracts than to administering existing ones. In response, 
OMB revised its Circular A-11 to require that federal agencies assess 
and certify contractors' project management systems for proper use of 
earned value management principles. OMB also identified several other 
deficiencies, including a lack of proper training for agency officials 
performing contract oversight. According to administrators at the 
National Aeronautics and Space Administration, earned value management 
training is essential for their contracting officers to adequately 
assess whether a contractor's project management system complies with 
the private industry's standard. 

We found that, with the exception of NNSA, DOE has not required its 
contracting officers or contracting officer representatives to receive 
earned value management training, even though they are responsible for 
determining whether the contractor's project management system complies 
with the private industry's earned value management standard after the 
contract is awarded. The following three DOE documents contain the 
contracting officer's responsibilities, the standards against which 
those responsibilities are to be discharged, and the training 
requirements for contracting officers: 

* Chapter 1 of DOE's Reference Book for Contract Administrators, issued 
in 2000 and in effect through October 2004, outlines the contracting 
officers' many responsibilities, including a review of the adequacy of 
the contractor's project management system.[Footnote 12] The reference 
book states that the system's adequacy must be confirmed by the 
contracting officer with the support of other DOE headquarters and 
field office personnel, as appropriate. The reference book also 
indicates that corrective action plans resulting from DOE reviews of 
contractor project management systems are to be tracked until the DOE 
contracting officer confirms that all open issues are closed. 

* DOE Order 413.3, "Program and Project Management for the Acquisition 
of Capital Assets," also issued in 2000, specifies that contractors' 
project management systems must comply with the American National 
Standards Institute's standard on earned value management. The order 
states that this requirement applies only to systems involved in 
controlling the performance of projects costing more than $20 million 
in total. The order also requires that contractors' systems provide 
cost and schedule performance, milestone status, and financial status 
to DOE on a monthly basis. 

* DOE Order 361.1A, "Acquisition Career Development Program," issued in 
April 2004, outlines the training and certification requirements for 
DOE contracting officers and contracting officer representatives. The 
order identifies a training curriculum for these officers by functional 
area--including, among others, procurement contracts; interagency 
agreements and sales contracts; grants and cooperative agreements; 
loans and loan guarantees; and the government purchase card. The order, 
however, does not require either the contracting officer or the 
contracting officer representative to receive earned value management 
training. 

The Director of the Contract Administration Division corroborated our 
assessment of DOE's order for acquisition career development. The 
director noted that the only reference to earned value management 
training in DOE Order 361.1A requires that DOE project directors, not 
contracting officers, complete a course on earned value management 
systems. Without this training, however, it is unclear how DOE 
contracting officers and contracting officer representatives can 
fulfill their responsibilities and properly assess the adequacy of the 
project management systems of departmental contractors. In providing us 
with exit conference comments, DOE Office of Contract Management 
officials acknowledged that contracting officers do have a 
responsibility in the area of earned value management and will be 
receiving training on that subject in the future. Subsequently, in 
December 2004, DOE provided contracting professionals at DOE 
headquarters with a 1-hour course on earned value management. DOE said 
that this training session, which was video recorded, is being required 
nationwide for all DOE contracting officials. As opposed to this 1-hour 
course, we noted that NNSA requires its contracting officials to 
participate in a 48-hour course on the fundamentals of earned value 
management.[Footnote 13]

The Reliability of PARS Data Is Limited: 

The reliability of the project cost and schedule data that PARS 
provides to senior DOE managers is limited by problems with the data's 
accuracy, completeness, and timeliness. In general, the accuracy of 
PARS report data is uncertain because DOE (1) has assessed the 
reliability of contractors' project management systems for only 2 of 
the 33 major projects we reviewed, (2) generally measures projects' 
cost and schedule performance in PARS against the current DOE-approved 
cost and schedule baselines without also tracking performance against 
the original targets, and (3) has not provided most of its major 
project directors with the training needed to ensure contractors are 
generating accurate performance data. PARS report data are not complete 
because DOE program offices have not submitted performance data to PARS 
for 3 major projects, as well as at least 2 smaller projects, and PARS 
reports do not provide each project's estimated cost at completion or 
other helpful, forward-looking data. In addition, the Office of 
Engineering and Construction Management stated that the June 2004 PARS 
report's performance data for 6 major projects and 5 smaller projects 
were significantly out of date, primarily because contractors did not 
provide updated project performance information. Senior managers have 
used PARS data to take actions that averted cost increases for certain 
projects that were experiencing cost or schedule challenges. Without 
reliable data, however, PARS has not provided senior managers with 
information about cost increases and schedule slippages for many 
projects, and the status of many other projects is uncertain. 

The Accuracy of Most PARS Data Is Uncertain: 

Three factors impair the accuracy of cost and schedule data reported in 
PARS. First, DOE officials told us they have little assurance that the 
cost and schedule data for most projects in PARS are accurate because 
DOE has not assessed the reliability of contractors' project management 
systems that generate such data for data reliability, particularly 
those systems believed to be using incorrect methods. Second, for 
almost all projects, PARS reports compare cost and schedule performance 
against DOE's current baselines, without identifying the extent of cost 
or schedule slippages that previously occurred. Third, most DOE project 
directors lack the necessary training to evaluate and verify the 
accuracy of the performance data that contractors generate, according 
to DOE officials. 

DOE Has Assessed Few Contractors' Systems That Generate Project Data: 

OMB Circular A-11 and DOE Order 413.3 require that DOE assess and 
certify contractors' project management systems for proper use of 
earned value management principles in generating cost and schedule 
performance data before the department approves a project's cost and 
schedule baseline at its Critical Decision 2 milestone. Earned value 
management, when used correctly, produces data that reflect a 
contractor's progress toward completing a project within cost and 
schedule targets. In essence, earned value management measures the 
value of work completed against the cost and schedule of work planned, 
as opposed to comparing actual with planned expenditures. To 
illustrate, assume a contract calls for 4 miles of railroad track to be 
laid in 4 weeks at a cost of $4 million. After 3 weeks of work, assume 
$2 million has been spent. By analyzing planned versus actual 
expenditures, it appears the project is underrunning the estimated 
costs. However, an earned value analysis reveals that the project is in 
trouble because even though only $2 million has been spent, only 1 mile 
of track has been laid; thus, the contract is only 25 percent complete. 
On the basis of the value of work done, the project will cost $8 
million ($2 million to complete each mile of track), and the 4 miles of 
track will take a total of 12 weeks (3 weeks for each mile of track) to 
complete instead of the originally estimated 4 weeks. 

To ensure correct application of earned value management principles, 
contractors must develop budgets and schedules based on measurable 
components of a project, which include defined start points, end 
points, and scopes of work. In addition, contractors must calculate the 
value of work performed against the budgets and schedules for the 
measurable project components. Experts in earned value management told 
us that without defined start and end points and other measurable 
project components, project performance data give little insight as to 
whether cost and schedule performance are on track, and the data might 
mask more serious problems. 

DOE's Office of Engineering and Construction Management and the Defense 
Contract Management Agency assess whether a given contractor's project 
management system properly uses earned value management principles by 
examining whether the contractor's system complies with the industry 
standards and verifying that the contractor is using the system to 
manage the project.[Footnote 14] Once a contractor has fully addressed 
the concerns identified by the assessment, DOE is to certify the 
project management system, attesting that project performance data--
data that convey progress toward the approved cost and schedule 
targets--are generated reliably. Assessment and certification of 
contractors' earned value management systems are critical components of 
DOE's management of its performance-based contracting, according to DOE 
earned value management training documents. While only three systems 
have been assessed since August 2003, Office of Engineering and 
Construction Management officials told us that they and the Defense 
Contract Management Agency, working together, could assess the project 
management systems for about 10 contractors in a given year now that 
they are becoming more familiar with the process. 

In August 2003, the Office of Engineering and Construction Management 
and the Defense Contract Management Agency began the process of 
assessing contractors' project management systems as a basis for 
certifying that they properly use earned value management principles. 
In September 2004, DOE certified Sandia National Laboratories' project 
management system for 1 major project, the Microsystems and Engineering 
Sciences Applications project, and 6 smaller projects. DOE also plans 
to certify Oak Ridge National Laboratory's project management system 
for another major project, the Spallation Neutron Source, once minor 
deficiencies are corrected.[Footnote 15] Overall, however, DOE has 
assessed project management systems for only 2 of the 33 major projects 
we reviewed--and 8 of the 73 projects in PARS--that have passed 
Critical Decision 2 with DOE-approved cost and schedule baselines. (The 
remaining 65 projects in PARS whose systems have not been assessed have 
baseline costs of nearly $75 billion.) According to an Office of 
Engineering and Construction Management official, the first three 
contractors' systems were selected for assessment on the basis of 
visibility, significance, and criticality to the Department's success, 
but also because cognizant DOE officials were confident that the 
contractors' project management systems would meet certification 
criteria. 

The National Research Council's 2004 report on DOE's project management 
found that the quality of earned value management across the 
department's projects was inconsistent and stated that senior DOE 
managers do not know whether the reported data on cost and schedule 
performance are accurate unless contractors' systems are assessed and 
certified. Because DOE has only recently begun to assess contractors' 
project management systems that feed data into PARS, DOE officials 
acknowledged to us that they lack assurance regarding the accuracy of 
PARS performance data, adding that they believe some of the project 
management systems not yet assessed have important deficiencies. For 
example, a DOE expert in earned value management noted that contractors 
for most Environmental Management projects--about half of the projects 
in PARS that have passed Critical Decision 2--have not properly 
implemented earned value management principles because, among other 
things, many of the projects' components lack defined start and end 
points. For example, the earned value management expert believes, on 
the basis of his assessment of work breakdown structures and other 
project components, that the contractor's project management system for 
the $10-billion Yucca Mountain Nuclear Waste Repository project does 
not properly use earned value management principles and generates 
performance data that cannot be regarded as accurate. Consequently, 
senior DOE managers have no assurance that cost and schedule targets 
will be met, even if the data suggest they will. 

Similarly, for several major projects we examined, the contractors' 
project management systems do not seem to properly implement earned 
value management principles to measure cost and schedule performance. 
For example, the $2-billion East Tennessee Technology Park project at 
Oak Ridge lacks measurable project components. In some instances, work 
is categorized into activities such as "general operations" and 
"contractor operations" that have no apparent defined start and end 
points. According to the expert in earned value management, the 
categories of work for this project make it difficult to accurately 
measure project performance because there is no clear activity or time 
frame against which to measure costs incurred or time spent. Instead, 
PARS data for this project seem to measure only the project's 
expenditures, which can conceal information on the project's cost and 
schedule status and progress toward completion. In addition, the $5.7-
billion Tank Waste Treatment and Immobilization Plant at Hanford, 
Washington, lacks discrete, measurable project components because work 
is categorized into activities such as "providing technology" and 
"providing infrastructure" that lack defined start and end points. 
While we recognize that it is appropriate, according to industry 
standards, to categorize a small amount of work in this fashion, DOE 
project management officials said the particular categories of work in 
these instances reflected a poor comprehension of earned value 
management and limited their confidence in the assessment of project 
performance. 

Two Office of Engineering and Construction Management officials 
acknowledged that the accuracy of data for these projects is uncertain 
because DOE has not assessed whether the contractors' project 
management systems properly applied earned value management principles. 
One of these officials suggested that the contractors' project 
management systems for such projects should be assessed as soon as 
possible to correct deficiencies and improve the reliability of project 
performance data provided to senior managers to oversee progress toward 
cost and schedule targets. The Director of the Office of Engineering 
and Construction Management agreed that DOE should develop a schedule 
that would give priority to assessing these and other high-risk and 
high-cost systems. As of January 2005, a schedule had not been 
developed, but the director told us that he was in the process of doing 
so. 

PARS Reports Generally Do Not Show Total Cost Overruns and Schedule 
Slippages: 

The accuracy of the PARS report data is further impaired because PARS 
reports generally do not show total cost overruns and schedule 
slippages, even though DOE requires each project team to estimate life-
cycle costs and assess project performance against established cost and 
schedule baselines. Instead, a project's DOE project director updates 
the cost and schedule baselines in PARS when DOE approves a contract 
modification. As a result, PARS reports show relatively small variances 
between a project's actual performance and its approved baselines, so 
that many of the projects we reviewed appear not to have experienced 
problems when, in fact, they did. For almost all projects, PARS reports 
do not provide data that would enable senior DOE managers to assess (1) 
a contractor's performance against the project's original DOE-approved 
baselines to identify total cost overruns and schedule slippages or (2) 
the effect of any DOE initiatives to control a project's costs. The 
Civil Engineering Research Foundation's July 2004 report similarly 
found that PARS cost and schedule data often do not convey the actual 
status of projects since their inception because of periodic revisions 
of cost or schedule baselines. Furthermore, for most Environmental 
Management projects, PARS measures project performance from arbitrary 
dates, such as the beginning of the fiscal year, which do not 
necessarily correspond to progress toward DOE-approved targets. The 
following examples illustrate how PARS has masked problems with 
projects by giving an incomplete picture of project costs or project 
performance: 

* The January 2004 PARS report showed that the $1.6-billion Spent 
Nuclear Fuels Stabilization and Disposition project at Hanford, 
Washington, was on track to meet cost and schedule performance 
targets.[Footnote 16] However, by April, total costs for the project 
increased by nearly $150 million. DOE officials acknowledged that 
because the January 2004 PARS report to senior DOE managers measured 
only project performance from the beginning of the fiscal year, instead 
of against the DOE-approved baselines, the PARS report concealed longer 
term problems that threatened the project's completion within costs. 

* In October 2002, the Tritium Extraction Facility at Savannah River, 
South Carolina, had an approved total cost of about $400 million. Costs 
for the project increased more than $100 million by September 2003, and 
subsequent PARS reports showed that costs were on track to meet cost 
targets, despite the 25 percent increase in the project's costs. 

* In June 2004, Environmental Management restructured the PARS 
reporting for 4 projects at Oak Ridge, Tennessee, by combining their 
respective costs and schedules with those of other Oak Ridge 
projects.[Footnote 17] As a result, Environmental Management stopped 
reporting project performance data for each project, masking the fact 
that 2 of them, totaling about $300 million, were significantly behind 
schedule. Two Office of Engineering and Construction Management 
officials believe the projects should be reported separately because 
combining projects' respective cost and schedule data can inhibit the 
correct use of earned value management. 

* The April 2004 PARS report showed that the total cost of the Soil and 
Water Remediation project at Ashtabula, Ohio, would be $45 million, 
although the performance data indicated the project would not likely 
meet its baselines. However, this amount does not include about $109 
million in expenditures on this project by October 1, 2003. 
Environmental Management reports this project's total costs to be about 
$157 million--more than three times the amount reported in PARS. 

* PARS reports that total project costs for the Nuclear Facility 
Deactivation and Decommissioning project at Columbus, Ohio, will be 
about $31.5 million. However, this amount does not include about $106 
million in expenditures prior to 2004. Environmental Management 
estimates that this project's total cost will exceed $163 million--more 
than five times the amount reported in PARS. 

The June 2004 PARS report showed that 90 percent of the 63 projects 
with approved baselines were expected to meet their cost and schedule 
baselines.[Footnote 18] However, this percentage may reflect project 
managers' efforts to keep the projects' baselines up to date rather 
than improvements in project management performance because PARS 
generally measures projects' performance against the most current DOE-
approved baselines. For example, as shown in table 1, the October 2002 
PARS report's assessment of 2 major projects was red because both 
projects were expected to breach their cost/schedule performance 
baselines. However, the September 2003 PARS report's assessment of 
these major projects was green because total project costs were within 
the revised baseline that DOE had subsequently approved. The September 
2003 PARS report did not indicate the extent to which each project's 
total costs had exceeded the costs that DOE approved at Critical 
Decision 2 on the basis of an approved conceptual design report and 
acquisition strategy. 

Table 1: PARS Reports Assess a Project's Cost and Schedule Performance 
against Only the Current DOE-Approved Baselines: 

Dollars in millions: 

Project name: Waste Treatment and Immobilization Plant Project 
(Hanford): 

October 2002 PARS report; 
Total cost: $4,746.9; 
Overall assessment: Red[A]; 
PARS comments: Proposed baseline approval is scheduled. Unable to 
validate baseline; 
cannot recommend approval. 

September 2003 PARS report; 
Total cost: $5,781.0; 
Overall assessment: Green[B]; 
PARS comments: No comment. 

Project name: Tritium Extraction Facility Project (Savannah River). 

October 2002 PARS report; 
Total cost: $401.0; 
Overall assessment: Red[A]; 
PARS comments: This project has a baseline breach. Baseline Change 
Proposal is being prepared for Secretarial approval. 

September 2003 PARS report; 
Total cost: $506.4; 
Overall assessment: Green[B]; 
PARS comments: No comment. 

Source: GAO compilation of data for selected projects from fiscal year 
2003 PARS reports. 

[A] Red indicates that a project is expected to breach its 
cost/schedule performance baseline. 

[B] Green indicates that a project is expected to meet its 
cost/schedule performance baseline. 

[End of table]

In addition to these projects, the 90 percent figure includes many 
Environmental Management projects, whose performance is measured over 
time frames that do not necessarily reflect performance against DOE-
approved baselines. Further, the 90 percent figure does not reflect the 
4 Oak Ridge projects whose performance data showed imminent performance 
problems before being combined with the performance data of other 
projects at the site. 

DOE officials told us that the monthly PARS reports are the primary 
tool for communicating project performance information to senior 
management. However, for many projects--particularly those overseen by 
Environmental Management--PARS does not report projects' life-cycle 
costs or performance against original baselines, even though DOE 
requires each project team to estimate life-cycle costs and assess 
project performance against established cost baselines and schedule 
milestones. Office of Engineering and Construction Management officials 
acknowledged that reporting life-cycle costs and project performance 
against original cost and schedule baselines in PARS would make cost or 
schedule challenges easier to identify, and Environmental Management 
officials told us they plan to report life-cycle costs and project 
performance against original baselines in PARS reports beginning by 
December 2004. 

In addition to Environmental Management's plans for PARS reporting, the 
Office of Engineering and Construction Management intends to make 
several upgrades to the PARS database, such as making the process for 
entering monthly data more efficient and easier for users to understand 
and ensuring that the correct data are being entered. Office of 
Engineering and Construction Management officials reported that they 
are in the process of implementing these improvements. However, these 
upgrades do not address the limitations to reporting accurate data that 
we identified. Furthermore, these improvements do not address 
limitations in the reliability of data stemming from contractor's 
project management systems that have not been assessed or data that 
have not been reviewed. 

Most DOE Project Directors Lack Certification in Earned Value 
Management: 

Project directors are DOE's focal point for assessing the contractors' 
cost and schedule performance data that feed into PARS. However, most 
of DOE's project directors have not been certified in earned value 
management, further reducing assurances that PARS data are accurate. 
Because DOE believes that it is critical for project directors to 
understand earned value management, the department informally 
designates its project directors as "acting directors" if they have not 
completed the project manager career development program, which 
includes training in earned value management. Office of Engineering and 
Construction Management officials told us that while some acting 
project directors are proficient in earned value management and capable 
of evaluating the reliability of contractor-generated data, other 
acting project directors are not. 

DOE recently implemented the project management career development 
program through which project directors are being trained in, among 
other things, earned value management. However, DOE had trained only 
about 25 percent of them through this program as of July 2004, with 
plans to train the remaining 75 percent by May 2006. A DOE official 
told us that the appropriate level of earned value management training 
for acting project directors depends on their experience in using 
earned value management. While DOE aims to assess project directors' 
capabilities in earned value management to ensure that they are 
competent, validating the adequacy of prior earned value management 
experience for acting project directors has been time consuming. The 
lack of trained projects directors reviewing the accuracy of a 
project's performance data may, in some cases, adversely affect the 
ability of senior DOE managers to properly assess the status of major 
projects. 

PARS Lacks Complete Information: 

In addition to reporting data of questionable accuracy, PARS provides 
incomplete data, therefore senior DOE managers may not be aware of the 
need to implement corrective actions to prevent cost overruns or 
schedule slippages. We identified the following 5 projects--3 major 
projects to refurbish nuclear weapons and 2 projects costing more than 
$100 million each--that are not in the PARS database, despite DOE's 
requirement that projects costing more than $5 million provide monthly 
reporting:[Footnote 19]

* W80 Life Extension Program. NNSA recently increased the total cost of 
this program, designed to extend the service life of the W80 nuclear 
warhead by replacing components, from $1.3 billion to about $2.45 
billion. 

* W76 Trident Missile Life Extension Program. NNSA expects this 
project, designed to extend the service life of the W76 nuclear warhead 
by replacing components, to cost about $680 million over the next 4 
years. 

* B61 Alteration 357 Life Extension Program. NNSA expects this project, 
designed to extend the service life of the B61 bomb, to cost nearly 
$500 million. Our July 2003 report recommended that DOE improve its 
oversight of the life extension program's cost and schedule status. 

* Purple and BlueGene/L Supercomputers under the Advanced Simulation 
and Computing Program. NNSA expects this project, to cost about $290 
million and be completed in 2005. 

* Enterprise Project. NNSA increased the total cost of this project, 
which will replace the accounting and management systems at Los Alamos 
National Laboratory, from about $70 million when it was initiated in 
2001 to nearly $160 million. 

The National Research Council's 2004 report found that DOE has not 
acted in a timely fashion to include all projects costing more than $5 
million in PARS. Office of Engineering and Construction Management 
officials told us DOE is still in the process of applying project 
management principles to many of the department's operational 
activities. While DOE's program offices are responsible for converting 
these activities to projects, many of the program office personnel 
responsible for applying project management principles do not have the 
necessary training, according to an Office of Engineering and 
Construction Management official. While project management training is 
available, DOE has required only project directors and other senior-
level employees to take this training. An Office of Engineering and 
Construction Management official told us this training would help 
expedite the application of project management principles to DOE's 
operational activities. 

In addition, for many projects included in the PARS database, PARS 
reports do not provide important performance information that senior 
DOE managers need to assess the projects' status. In some cases, 
project performance data are not reported because the project is 
incorrectly listed as being in the design phase when, in fact, it has 
passed Critical Decision 2. For example, contractors have spent almost 
half of the approved funds for 2 projects at the Idaho National 
Engineering and Environmental Laboratory projected to cost $4.3 billion 
without reporting performance data in PARS.[Footnote 20] The PARS 
reports show that these projects are still in the design phase and, 
therefore, are not subject to reporting performance data, but a DOE 
official acknowledged that both projects have, in fact, passed Critical 
Decision 3 and other subsequent milestones. As a result, senior DOE 
managers cannot rely on PARS for accurate and current performance 
information for these projects, nor can they rely on PARS to determine 
whether these projects require corrective actions. 

For these and other projects, PARS also lacks forward-looking data, 
such as scheduled work to be performed, the projects' upcoming 
milestones, and the projects' estimated cost at completion. Without 
such data, PARS cannot provide information on projects' cost or 
schedule challenges and DOE management does not have a basis for 
projecting progress or identifying trends. While not in PARS, this 
information is available from acting project directors. For example, 
although early cost savings for the Microsystems and Engineering 
Sciences Applications project at Sandia National Laboratories led to 
favorable performance data, DOE's project director identified supply 
imbalances in the steel market that would increase the estimated 
construction costs. Using this information, the project director 
revised the project's estimated total cost. Currently, PARS reports to 
senior DOE managers lack such forward-looking data that could alert 
them to future cost or schedule challenges. The National Research 
Council's 2004 report stated that PARS reports should display forward-
looking data to notify senior managers of upcoming milestones. In 
addition, several acting project directors told us that forward-looking 
data, such as data on estimated costs at completion, should be included 
in PARS to identify project performance challenges for senior DOE 
managers. 

To further illustrate this need, the total costs of some DOE projects 
are projected to increase dramatically in the future, despite PARS 
reports showing that they are expected to be completed on time and 
within budget. For example, PARS report data show that the Hanford's 
Tank Waste Treatment and Immobilization Plant is projected to meet the 
DOE-approved baseline of $5.78 billion. However, PARS does not show 
that DOE approved a $1.4-billion increase above the project's original 
contract estimate of $4.35 billion in April 2003, nor does it show that 
the U.S. Army Corps of Engineers, in a May 2004 report, stated that 
project costs would probably exceed the $5.78-billion cost baseline by 
$720 million.[Footnote 21]

Even though the DOE project management teams knew of cost and schedule 
performance problems for the Tank Waste Treatment and Immobilization 
Plant project, PARS reports have shown that this project was on track 
for meeting cost and schedule targets. An Office of Engineering and 
Construction Management official told us that PARS monthly reports do 
not include forward-looking data and trend data to minimize the amount 
of time necessary for senior managers' review. As a result, PARS did 
not provide senior DOE managers for this and other projects with 
important information to analyze potential future challenges. Forward-
looking performance information, such as scheduled work to be performed 
and estimated cost at completion, would better enable senior managers 
to address project management challenges and minimize cost overruns or 
schedule slippages. 

PARS Lacks Timely Information: 

Further compounding reliability concerns, we identified problems with 
the timeliness of PARS data that may limit the ability of senior DOE 
managers to effectively identify and apply corrective actions. 
Specifically, we found that cost and schedule performance data were 
significantly out of date at some time during our review for 8 of the 
33 major projects we reviewed and 20 smaller projects in PARS that had 
passed Critical Decision 2. In these instances, data were out of date 
because DOE has not effectively enforced requirements that contractors 
produce updated monthly cost and schedule performance data, and that 
project directors ensure current performance data are reported into 
PARS. For some projects, the lack of up-to-date data masked problems 
that resulted in cost overruns and schedule slippages. For instance: 

* The September 2003 PARS report showed that the Spent Nuclear Fuels 
project at Hanford, Washington, was on track to meet its DOE-approved 
total project cost of about $1.6 billion and its schedule completion 
date of 2007; however, these data were 3 months out of date. 
Subsequently, the April 2004 PARS report (1) showed that total project 
costs had exceeded the project's cost baseline by nearly $150 million 
and (2) indicated that the project would exceed this revised total cost 
and the scheduled completion date would slip. In June 2004, the 
contractor requested additional funding from DOE because both cost and 
schedule performance continued to worsen. 

* The September 2003 PARS report showed that the K25/27 Buildings 
Deactivation and Decommissioning Removal project at Oak Ridge, 
Tennessee, was on track to meet its DOE-approved total project cost of 
about $265 million and its schedule completion date of 2008. However, 
the contractor did not update the project's performance data until 
April 2004, when the PARS report showed the project would still meet 
its cost baseline. Environmental Management officials told us that 
although they knew for several months that the K25/27 project's total 
cost would exceed its baseline, the PARS cost data were not updated 
because the project was being combined with 5 other Oak Ridge projects. 
The total cost of the K25/27 project could exceed $400 million--more 
than 50 percent above the DOE-approved total project cost. 

* In June 2004, the Soil and Water Remediation project at Pantex, 
Texas, had a DOE-approved total project cost of about $175 million, but 
the Office of Engineering and Construction Management could not assess 
the project's performance because data were not provided. Subsequently, 
the September 2004 PARS report showed that the project was at risk of 
exceeding its DOE-approved schedule target. 

In addition to these timeliness problems, the monthly data in PARS 
reports typically lag a project's actual performance by 2 to 3 months 
because of the time contractors need to generate the data and the time 
DOE project managers need to review and incorporate the summary data 
into the PARS database. The 2004 National Research Council report 
stated that the lack of timely data prevents senior managers from using 
PARS to assess the performance of projects in real time. Similarly, 
Department of Defense officials familiar with project management have 
said that using such data to assess project performance is like 
"overseeing by looking through a rear view mirror" because performance 
problems have usually gotten worse by the time departmental managers 
become aware of them. 

We found that the Department of Defense requires all of its newer 
contracts to use electronic data interchange to provide more timely 
information to department program managers.[Footnote 22] In addition, 
some acting project directors told us that electronically linking PARS 
to contractors' project management systems would improve timeliness 
because manually entering cost and schedule data into the PARS database 
had often resulted in delays of 2 to 3 months to complete the process. 
In some instances, data were entered incorrectly, although in each 
instance the data were corrected before being reported to senior 
managers. While the DOE project directors we contacted uniformly agree 
that manually entered data are correctly entered by the time PARS 
monthly reports are delivered to senior managers, electronically 
linking PARS to contractor systems could eliminate the potential for 
such errors and enhance senior managers' ability to address potential 
cost or schedule challenges in real time. Alternatively, DOE might 
include a provision requiring timely monthly reporting in all 
applicable contracts. 

When data can be relied upon, DOE senior managers have taken corrective 
actions to address cost or schedule challenges while minimizing costs 
to the government. For example, NNSA terminated the Sandia Underground 
Reactor Facility project, which was intended to reduce the future 
operational costs associated with securing a reactor, when management 
learned that cost estimates had increased by more than 150 percent 
between project conception and the final design phase. The project was 
terminated before costs were incurred. In another instance, 
Environmental Management approved a contractor's recovery plan to 
complete the Melton Valley Closure project at Oak Ridge, Tennessee, 
whose schedule performance had slipped dramatically and required 
corrective actions. The contractor lengthened work hours and modified 
its approach for constructing a subproject. As a result, the recovery 
plan showed that the scope of work could be accomplished without 
increasing project schedule. 

Conclusions: 

Since 1990, we have designated DOE's contract management, which we have 
broadly defined to include contract administration and project 
management, as a high-risk area for fraud, waste, abuse, and 
mismanagement. Although DOE has implemented important contract 
administration and project management reforms, problems persist and 
many major projects continue to experience millions of dollars in cost 
overruns and years of delays. Two deficiencies--the lack of contracting 
criteria for major projects and the lack of reviews of the project 
management terms in major project contracts--have resulted in 
questionable DOE contracting decisions that limit its ability to 
effectively control cost and schedule performance. For example, many of 
DOE's contracts for major projects have used performance incentives 
that have used a technical, schedule, or performance incentive without 
an associated cost incentive or cost constraint, thereby giving 
contractors an incentive to pay limited attention to costs when working 
toward meeting technical or performance levels in order to earn a 
higher award fee. 

Furthermore, for major projects, DOE has given insufficient emphasis to 
the oversight of contract administration, which begins after contracts 
are awarded and helps ensure that the department gets what it pays for. 
DOE needs to give increased emphasis to reviewing how it administers 
contracts; correcting previously identified weaknesses, such as 
overreliance on contractor data; and providing training to its 
contracting officers. Without such actions, the department is totally 
dependent on its contractors' self-reports on their performance. 

Because of problems with the accuracy, completeness, and timeliness of 
the PARS data, senior DOE managers lack key project performance 
information for assessing the progress of many major projects and 
making decisions about corrective actions. In particular, because DOE 
has assessed the reliability of only three contractors' project 
management systems that feed data into PARS, senior managers cannot be 
certain that the contractor systems are producing reliable data. Such 
data are critical to good project management and affect DOE's 
assessment of contractor performance. Absent reliable data from the 
contractor systems, DOE lacks assurance that the fees it awards for a 
contractor's project management actions are well deserved. 

Recommendations for Executive Action: 

To ensure the use of effective performance incentives for major 
projects, we recommend that the Secretary of Energy direct the 
Associate Deputy Secretary with responsibility for contract and project 
management to take the following two actions: 

* develop a major projects chapter in the DOE Acquisition Guide that 
specifies a systematic contracting approach, including, for example, 
criteria for (1) ensuring that incentive fee awards are based on 
reliable performance data, (2) using appropriate cost and schedule 
incentives, (3) better linking fee awards to performance for major 
projects that are part of larger site cleanups, and (4) determining 
which indirect work-related activities should and should not be 
considered in awarding contractors' fees, and: 

* clarify roles and responsibilities for reviewing contracts prior to 
award to ensure project management consistency. 

To strengthen departmental oversight of contract administration for 
major projects, we recommend that the Secretary of Energy direct the 
Associate Deputy Secretary with responsibility for contract and project 
management to take the following three actions: 

* conduct comprehensive self-assessments of contract administration at 
least every 3 years,

* identify corrective actions to reduce the overreliance on unvalidated 
contractor data in awarding contract fees that was identified in 
previous self-assessments, and: 

* train contracting officials in earned value management. 

To improve the reliability and usefulness of project performance data 
in PARS, we recommend that the Secretary of Energy direct the 
appropriate managers to take the following seven actions: 

* develop a schedule for assessing the reliability of the contractors' 
project management systems, giving priority to major projects and those 
projects with systems believed to be using incorrect methods to 
generate PARS data;

* revise DOE manual 413.3-1 to provide guidance that enhances the 
accurate reporting of total cost and project performance data into 
PARS, such as the reporting of life-of-project cost and schedule 
variances;

* expedite training for major project directors in earned value 
management concepts;

* ensure that program office officials receive currently available 
project management training so that they can better identify the 
elements of a project, and apply the project management concepts 
necessary for them to report performance data in PARS;

* incorporate forward-looking trend data into PARS reports so that 
senior managers can better identify negative trends and potentially 
take corrective action;

* explore options for ensuring that contractors provide cost and 
schedule performance data to PARS on a monthly basis, such as making 
monthly submissions a requirement in all applicable contracts; and: 

* explore options for providing senior DOE managers with more timely 
project performance data by, for example, electronically linking 
contractors' project management systems to PARS. 

Agency Comments and Our Evaluation: 

We provided DOE with a draft of this report for its review and comment. 
In written comments, DOE generally concurred with our recommendations 
but provided clarifying comments on four of the recommendations. (See 
app. III.) First, concerning our recommendation that DOE develop a 
major projects chapter in its Acquisition Guide, DOE stated that the 
department has already developed an extensive body of material that 
constitutes a "systematic contracting approach" for the acquisition and 
management of departmental major projects, but added that the 
department will develop an overview and summary of this information in 
a major projects chapter in its Acquisition Guide. We believe this 
chapter will further enhance DOE's guidance, particularly if the 
department provides criteria that address each of the four issues 
identified in our first recommendation. Second, concerning our 
recommendation on DOE's comprehensive assessment of contract 
administration, DOE stated that the department did not stop conducting 
comprehensive assessments. In response, we have revised our 
recommendation to state that DOE should conduct these assessments at 
least every 3 years. Third, concerning our recommendation that DOE 
identify corrective actions for reducing overreliance on unvalidated 
contractor data, DOE stated that the department had already taken 
positive steps to reduce its overreliance on contractor data by, for 
example, reviewing and validating such data and project baselines. DOE 
added that the department would continue to identify any corrective 
actions necessary to reduce overreliance on contractors' data in 
awarding fees. While we agree that validating project baselines is an 
important first step, we believe that DOE's efforts to ensure that 
contractor performance data are reliable by certifying contractors' 
project management systems is vital. Fourth, concerning our 
recommendation that DOE link PARS and contractors' project management 
systems, DOE stated that our recommendation is too narrowly focused, 
particularly in light of DOE's efforts to implement a departmentwide 
enterprise architecture solution. We agree, and we have revised our 
recommendation accordingly. In addition, DOE stated that it believes 
the draft report contained a number of inaccuracies and provided 
detailed comments. We have revised the report, where appropriate, in 
response to these comments. 

As arranged with your offices, unless you publicly announce its 
contents earlier, we plan no further distribution of this report until 
30 days after the date of this letter. At that time, we will send 
copies to the Secretary of Energy and other interested parties. We will 
also make copies available to others upon request. In addition, the 
report will be available at no charge on the GAO Web site at 
[Hyperlink, http://www.gao.gov]. 

If you or your staff have any questions on this report, please contact 
me at (202) 512-3841. Key contributors to this report were Richard 
Cheston, Robert Baney, Nathan Anderson, Bernice Dawson, Cynthia Norris, 
Judy Pagano, and Doreen Feldman. 

Signed by: 

Robin M. Nazzaro: 
Director, Natural Resources and Environment: 

[End of section]

Appendixes: 

[End of section]

Appendix I: The Department of Energy's 33 Major Projects That We 
Reviewed: 

Fernald, Ohio: 

Project name and construction line number: Non-Nuclear Facility 
Decontamination and Decommissioning (OH-FN-0050); 
Contract type: Fernald, Ohio: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes;
Fee available for individual project? No[A]. 

Project name and construction line number: Silos (OH-FN-07)[B]; 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Soil & Water Remediation (OH-
FN-0030); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Solid Waste Stabilization & 
Disposition (OH-FN-0013); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Hanford Reservation: River Protection: 

Project name and construction line number: Hanford Tank Waste Treatment 
and Immobilization Plant (01-D-416); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Radioactive Liquid Tank 
Waste Stabilization & Disposition (ORP-0014); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Interim Tank Retrieval 
System (94-D-407)[C]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Tank Farm Restoration and 
Safe Operations (97-D-402)[B]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Hanford Reservation at Richland, Washington: 

Project name and construction line number: Nuclear Facility 
Decontamination and Decommissioning--Fast Flux Test Facility Project 
(RL-0042); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Nuclear Facility 
Decontamination and Decommissioning--Remainder of Hanford (RL-0040); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Nuclear Facility 
Decontamination and Decommissioning--River Corridor Closure Project (RL-
0041); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Nuclear Material 
Stabilization & Disposition-PFP (RL-0011); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Soil and Water Remediation-
Vadose Zone (RL-0030); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? No; 
Fee available for individual project? Yes. 

Project name and construction line number: Solid Waste Stabilization & 
Disposition--200 Area (RL-0013); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Spent Nuclear Fuels (RL-
0012); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Idaho National Engineering and Environmental Laboratory: 

Project name and construction line number: Advanced Mixed Waste 
Treatment Facility (97-PVT-2); 
Contract type: Fixed price; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? No; 
Fee available for individual project? No. 

Project name and construction line number: Spent Nuclear Fuel Dry 
Storage (98-PVT-2)[B]; 
Contract type: Fixed price; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? No; 
Fee available for individual project? No. 

Lawrence Livermore National Laboratory: 

Project name and construction line number: National Ignition Facility 
(96-D-111); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[D]. 

Oak Ridge, Tennessee: 

Project name and construction line number: East Tennessee Technology 
Park Three-Building D&D and Recycle Project (OR-0040); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Facilities Capability 
Assurance Program (88-D-122-27 & 88-D-122-42)[C]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Spallation Neutron Source 
(99-E-334); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Rocky Flats Facility at Denver, Colorado: 

Project name and construction line number: Nuclear Facility D&D/North 
Side Facility Closures (RF-0040); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Nuclear Facility D&D/South 
Site Facility Closures (RF-0041); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Nuclear Material 
Stabilization & Disposition (RF-0011); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Soil & Water Remediation (RF-
0030); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Project name and construction line number: Solid Waste Stabilization & 
Disposition (RF-0013); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[A]. 

Sandia National Laboratories: 

Project name and construction line number: Microsystems and Engineering 
Science Application (01-D-108); 
Contract type: Fixed price; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[D]. 

Savannah River Site: 

Project name and construction line number: High-Level Waste Removal 
from Filled Waste Tanks (SR-0014C); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Project name and construction line number: Tritium Extraction Facility 
(98-D-125); 
Contract type: Cost plus incentive fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

West Valley, New York: 

Project name and construction line number: Nuclear Facility 
Decontamination and Decommissioning (OH-WV-0040); 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
Yes; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? Yes. 

Nuclear Weapons Refurbishment: 

Project name and construction line number: Life Extension Program--
B61[E]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[D]. 

Project name and construction line number: Life Extension Program--
W76[E]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[D]. 

Project name and construction line number: Life Extension Program--
W80[E]; 
Contract type: Cost plus award fee; 
Cost incentive or cost constraint for individual project in contract? 
No; 
Schedule performance incentive for individual project in contract? Yes; 
Fee available for individual project? No[D]. 

Source: GAO compilation of DOE data. 

[A] Fee is based on reaching site closure, rather than on completing 
individual projects. 

[B] Although the September 2004 PARS report showed that this project 
would cost less than $400 million, we included it in our review because 
it was included in our 2002 review of DOE's major projects. (GAO, 
Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure 
Initiatives Have Improved Results, [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-02-798] (Washington, D.C.: Sept. 13, 2002).

[C] This project was designated as a major project when DOE's threshold 
was $100 million. 

[D] NNSA factors performance on these projects into each responsible 
management and operating contractor's annual performance evaluation and 
decisions on the amount of the fee awards. 

[E] NNSA stated that each of these life extension projects involved 
multiple management and operating contractors (not 1 contract) in 
multiple locations, which is different from every other project that is 
listed in this appendix. 

[End of table]

[End of section]

Appendix II: Scope and Methodology: 

Our review focused primarily on 33 major projects that had passed, as 
of March 2004, the Department of Energy's (DOE) Critical Decision 2 
milestone--the point at which the department approves a project's cost, 
schedule, and scope baselines on the basis of an approved conceptual 
design report and acquisition strategy. The projects we reviewed 
include 28 projects that cost more than $400 million each and 5 
projects that our 2002 assessment defined as major projects because 
their total costs exceeded $100 million each. Our review did not 
include 46 major projects that, as of March 2004, had not passed the 
Critical Decision 2 milestone. Since March 2004, at least 6 major 
projects have passed the Critical Decision 2 milestone and now have 
approved baselines. The remaining major projects do not have approved 
baselines for measuring performance. 

To assess DOE's use of performance incentives in contracts to 
effectively control cost and maintain schedules, we reviewed relevant 
requirements in the Federal Acquisition Regulation (FAR) and the DOE 
Acquisition Regulation, as well as DOE Order 413.3, DOE manual 413.3-1, 
and DOE's Acquisition Guide, to obtain information on the factors that 
should be used in determining a contractor's fee. Through this effort, 
we identified whether the department provided guidance on the 
appropriate circumstances for using each contract type and the 
appropriate factors for determining a contractor's fee. In particular, 
we examined requirements regarding contract provisions for award fees; 
cost, schedule, and performance incentives; and fee determination 
plans. 

We then compared government and departmental requirements with project-
specific elements found in the contracts for each of the 33 major 
projects that have DOE-approved cost, schedule, and scope baselines to 
determine whether DOE has used appropriate (1) types of performance 
incentives, such as cost or schedule incentives, and (2) fee 
determination plans and fee payments. For instance, to assess whether 
DOE's contracts used the appropriate incentives for each of three types 
of contracts, we compared the types of incentives that DOE's contracts 
and relevant modifications used for each of the 33 major projects with 
the types of incentives that the FAR and the DOE Acquisition 
Regulation, as well as departmental orders and guidance, require. We 
then interviewed cognizant DOE officials to discuss reasons for the 
inconsistencies we found. In addition, we examined various contract-
related documents associated with the 33 major projects we reviewed, 
such as the "Gold Chart" metrics that Environmental Management uses to 
measure its progress in DOE's annual budget submission to the Congress. 
Specifically, we compared the Gold Chart's performance metrics for each 
of Environmental Management's 25 major projects with the performance 
measures in each projects' contract. Where differences were identified, 
we discussed the contents of the Gold Chart and the associated 
projects' contract with appropriate DOE contracting officials. 
Furthermore, we interviewed officials in DOE's Office of Contract 
Management and officials in the Office of Engineering and Construction 
Management to determine the extent to which DOE had reviewed, prior to 
award, the contracts for the 33 major projects to ensure that they 
included appropriate project management provisions. 

To assess the reliability of the data DOE uses to monitor and assess 
contractor performance, we reviewed the Office of Management and 
Budget's (OMB) directives, DOE's Reference Book for Contract 
Administrators, and other DOE documents and studies to identify 
relevant requirements and departmental guidance. We identified the 
roles and responsibilities of contract administration officials and 
examined the extent to which these officials adhered to their 
responsibilities. More specifically, we reviewed the department's 
recent contract administration self-assessments and the frequency with 
which they were conducted. In so doing, we examined the department's 
recommendations for improving contract administration and determined 
whether the recommendations were followed. If they were not followed, 
we discussed the reasons with cognizant officials in the Contract 
Administration Division. We also examined DOE's order for acquisition 
career development, and other related DOE directives, to assess 
training requirements for DOE's contracting officers and contracting 
officer representatives, particularly regarding training in earned 
value management principles. 

To determine the reliability of Project Analysis and Reporting System 
(PARS) data used by senior managers for project oversight, we assessed 
the accuracy, completeness, and timeliness of PARS data. To assess the 
accuracy of the project performance data in PARS, we did the following: 

* Reviewed DOE Order 413.3, "Program and Project Management for the 
Acquisition of Capital Assets," and its implementing guidance; OMB 
Circular A-11, part 7, "Planning, Budgeting, Acquisition, and 
Management of Capital Assets"; and various documents outlining the 
requirements in American National Standards Institute/Electronic 
Industries Association-748-1998, which defines the requirements for 
earned value management--the component of contractors' project 
management systems critical for producing reliable project performance 
data. 

* Interviewed cognizant DOE officials in the Office of Engineering and 
Construction Management, the Office of Environmental Management, the 
Office of Science, and the National Nuclear Security Administration on 
the extent to which the performance data that DOE contractors' project 
management systems produced for PARS met earned value management 
requirements. These officials included a DOE expert in earned value 
management, who is responsible for assessing the accuracy of the data 
that various projects' systems produce. Where specific deficiencies in 
a contractor's project management system were identified, we obtained 
relevant documents from the appropriate acting DOE project director and 
analyzed whether the contractor generated project performance data in 
accordance with the industry standard. We also interviewed officials in 
two other major contracting agencies--the Department of Defense and the 
National Aeronautics and Space Administration--about their experience 
in implementing earned value management requirements. 

* Compared data in monthly PARS reports provided to senior DOE managers 
from January through September 2004 with project-specific cost and 
schedule data obtained from earlier PARS reports, cognizant program 
offices, project status reports, Inspector General reports, and 
external reviews. When we identified total cost or project performance 
data discrepancies between PARS and these other sources, we contacted 
relevant project officials to determine their cause. 

* Identified the extent to which contractor-generated data in PARS were 
sufficiently reviewed and verified by DOE by (1) identifying 
requirements in DOE Order 413.3 and its implementing guidance for the 
departmental review and verification of contractor project performance 
data and (2) interviewing DOE project management officials to determine 
whether the current breadth of review was adequate and what plans, if 
any, DOE had for increasing the rigor of its review and verification of 
contractor data. 

To assess the completeness of PARS data, we determined whether the PARS 
database included major DOE activities--those costing more than $400 
million or that DOE management had designated--identified in our prior 
reports, Inspector General reports, DOE press releases, and printouts 
from DOE's Management Accounting and Reporting System. For projects 
that were not included in PARS, we contacted headquarters project 
management officials to determine if the projects met the criteria for 
PARS reporting. For projects that were included in PARS, we examined 
the completeness of reported data in various data fields by reviewing 
printouts from the PARS database and by reviewing the reports of the 
National Academies' National Research Council and the Civil Engineering 
Research Foundation, which also examined the completeness of PARS data. 
In addition, we reviewed a 2004 report by the U.S. Army Corps of 
Engineers on a major project at Hanford, Washington. Moreover, we 
discussed options with DOE officials for reporting additional data that 
would improve PARS' ability to enable senior DOE managers to identify 
potential cost or schedule challenges. 

To assess the timeliness of PARS data, we reviewed PARS monthly reports 
to senior DOE managers and identified projects whose performance data 
were out-of-date. For many of these projects, we talked to headquarters 
and project officials to determine the reasons for delay and explored 
options with them on how timeliness could be improved. We also 
interviewed numerous acting DOE project directors to learn how data 
from their project management systems were summarized and incorporated 
into the PARS database. In addition, we explored options with DOE 
headquarters and project officials for improving the timeliness of all 
data reported in PARS. 

Given our review of the documentation provided by DOE and our 
discussions with DOE officials, we have reservations about the 
reliability of PARS data. These issues are discussed in this report. 

We conducted our work from January 2004 through January 2005 in 
accordance with generally accepted government auditing standards, which 
included an assessment of data reliability and internal controls. 

[End of section]

Appendix III: Comments from the Department of Energy: 

Department of Energy: 
Washington, DC 20585:

FEB 02 2005:

Ms. Robin M. Nazzaro:
Director, Natural Resources and Environment: 
U.S. Government Accountability Office:
441 G Street, NW Room 2964: 
Washington, DC 20548:

Dear Ms. Nazzaro:

We have reviewed the draft Government Accountability Office (GAO) 
Report entitled "Department of Energy: Further Actions Are Needed to 
Strengthen Contract Management for Major Projects" (GAO-05-123). We 
appreciated the opportunity to review prior informal drafts of the 
report to eliminate any inaccuracies and questionable conclusions. We 
also appreciate that the report recognizes the Department of Energy's 
(DOE) many improvements in a number of areas related to project and 
contract management.

The GAO report specifically recognizes a number of improvements in 
DOE's management of major projects. For example, it notes: the 
establishment of the Office of Engineering and Construction Management 
(OECM) to oversee project management; the issuance of a comprehensive 
project management policy; implementation of a career development 
program for DOE project managers; implementation of the Project 
Assessment and Reporting System (PARS); required use of earned value 
management techniques; a certification program for contractors' project 
management systems to assure compliance with private industry's 
standard for earned value management; improved contract processes 
through the issuance of updates to DOE's Acquisition Guide and the 
Reference Book for Contract Administrators; and periodic assessments of 
its contract administration practices.

Because of the importance of the subject matter, the beneficial nature 
of its suggestions, and the Department's commitment to continuous 
improvement in this area, DOE concurs generally in all of the draft 
report recommendations. Nonetheless, despite its prior revisions, the 
draft report still contains a number of factual inaccuracies and 
inappropriate conclusions which affect the report's credibility. 
Although these inaccuracies were addressed in the Department's comments 
on the draft "statement of facts," they remain an issue in the report. 
The DOE's responses to the technical and factual inaccuracies, 
including the report's misrepresentations of certain conversations, are 
provided to you as an attachment to this letter. We request that the 
GAO revise its draft report to correct these inaccuracies. If this 
course of action is not acceptable, DOE requests that the full text of 
its response (including the attachment to this letter) be published 
with the final GAO report.

The report provides three sets of recommendations (a total of 13 
recommendations in all) to the Secretary in order to improve: (1) the 
use of effective performance incentives for major projects, (2) 
departmental oversight of contract administration for major projects, 
and (3) the reliability and usefulness of project performance data in 
the PARS. Notwithstanding that the Department disagrees with a number 
of the report's findings, we do agree with the general thrust of the 
recommendations.

With respect to areas of disagreement, the Department offers the 
following:

1. GAO Recommendation: Develop a major projects chapter in the DOE 
Acquisition Guide that specifies a systematic contracting approach, 
including, for example, criteria for (1) ensuring that incentive fee 
awards are based on reliable performance data, (2) using appropriate 
cost and schedule incentives, (3) better linking fee awards to 
performance for major projects that are part of larger site cleanups, 
and (4) determining which indirect work-related activities should and 
should not be considered in awarding contractors' fees.

DOE Comment: The Department has already developed an extensive body of 
material that constitutes a "systematic contracting approach" for the 
acquisition and management of the Department's major projects. The 
principles for determining the contract type, the nature of the 
incentive structure, etc. for contracts are already well established in 
the Federal Acquisition Regulation and the Department of Energy 
Acquisition Regulation. Unlike other Federal agencies, DOE has expanded 
on this information in the form of more detailed internal guidance on 
contract types and the effective use of performance incentives. The 
regulatory coverage when taken with the additional requirements of the 
DOE Acquisition Regulation, the DOE Acquisition Guide, the DOE 
Performance-Based Contracting Guide, DOE Order 413.3, DOE Manual 413.3-
1, the various workshops that the Department has conducted and the 
various professional training courses made available (both project 
management and acquisition courses) provides a solid basis for 
procurement and project management professionals to make informed 
decisions regarding appropriate types of contracts, incentives, and 
other pertinent components of properly executed major projects and 
contracts. Both the Office of Contract Management (OCM) and the OECM 
work together on acquisition strategies for the Department's major 
contracts. DOE Manual 413.3-1 requires, prior to approval of CD-1, that 
the Integrated Project Team develop an acquisition strategy for each 
project.

Based on the Department's previous review of materials made available 
by other agencies for the management of major projects, we know of no 
other agency which has provided as extensive and detailed a body of 
guidance to its contracting professionals in order to improve the state 
of contract and project management. Nonetheless, DOE will develop an 
overview and summary of the various information already extant in this 
major projects chapter in the DOE Acquisition Guide in accordance with 
the GAO's recommendation. Additionally, we will update our benchmarking 
of other agencies to identify other areas of information guidance, and 
direction that may be useful to the DOE project and contract management 
practitioners.

2. GAO Recommendation: Reinstitute comprehensive self-assessments of 
contract administration.

DOE Comment: This recommendation implies that the Department has ceased 
conducting comprehensive assessments of contract administration. 
Nothing could be further from the truth. As stated in the attached 
comments, the Department conducted assessments in 1997, 1999, 2002 and 
2003. That OCM did not conduct an assessment in 2004 does not indicate 
that the Department no longer values such assessments or that they will 
not be performed in the future. Assessments are, and will be, conducted 
on a periodic basis. For example, during fiscal year 2005, two separate 
sets of assessments are planned. One set of assessments will examine 
the effective use by DOE sites of contract management plans (CMP) to 
manage contracts. A second set of assessments will be conducted on 
contractor purchasing systems. The Department concurs in the GAO's 
recommendation "to continue such assessments."

3. GAO Recommendation: Identify corrective actions to reduce the over-
reliance on contractors' data in awarding contract fees that was 
identified in previous self-assessments.

DOE Comment: Every procurement agency in the Federal Government relies 
to some extent on contractor data, a necessity which the Federal 
Acquisition Regulation recognizes. The Department fully acknowledges 
the importance of ensuring that such data has been validated to the 
extent necessary for the effective management of the Department's 
contracts and projects.

Consequently, the Department has already taken positive steps to reduce 
any over-reliance on contractor submitted data and the associated 
baselines. For example, the Office of Environmental Management (EM) 
regularly reviews and validates such data and baselines. Specifically, 
EM review teams have performed independent reviews of the Integrated 
Life Cycle Baselines and Contract Project Baselines of all EM sites 
over the past two years. To date, all but six EM sites (West Valley, 
Nevada, Stanford Linear Accelerator Center, Waste Isolation Pilot 
Plant, Los Alamos National Laboratory, and Energy Technology 
Engineering Center) have approved baselines which are under 
configuration control. The criteria for the reviews included, but were 
not limited to, cost estimation, scope definition, project schedule, 
earned value management system, performance metrics, regulatory 
requirements, and government furnished services and items. The goal is 
to have the baselines for the remaining six EM sites under 
configuration control by the end of fiscal year 2005.

Additionally, the OECM has been validating project baselines for 
several years. This was previously commented to GAO at its exit 
conference and in the Department's written comments on the draft 
"statement of facts." OECM recently sent a report to Congress 
documenting its efforts in validating EM project baselines. The 
Subcommittee on Energy and Water Development Appropriations stated that 
it supports OECM's efforts and that OECM "continues to provide a strong 
focal point for the improvement of project management capabilities 
throughout the Department." Furthermore, the Department is in the 
process of certifying that its contractors' Earned Value Management 
Systems comply with the industry standard. We are not aware that any 
other civilian agency is doing this.

Clearly, DOE has been proactive in addressing the issue of ensuring 
that data and baselines for the Department's major management contracts 
and projects is validated and that the systems which track this data 
are in compliance with the industry standard. Nonetheless, the 
Department concurs with the general thrust of the GAO's recommendation 
and will continue to identify any corrective actions necessary to 
reduce over-reliance on contractors' data in awarding contract fees.

4. GAO Recommendation: Explore options for electronically linking 
contractors' project management systems to PARS to provide senior DOE 
managers with timely project performance data.

DOE Comment: The Department agrees with the need to ensure timely 
project reporting. However, the GAO's linking of PARS and contractors' 
project management systems is too narrowly focused, particularly in 
light of the Department's efforts to implement a Department-wide 
enterprise architecture solution. The recommendation would be 
beneficial if it were not focused on the existing stand-alone system 
(PARS), but rather addressed a more comprehensive exploration of 
options that would improve the timeliness and accuracy of project 
performance reporting. Consequently, the Department concurs with the 
concept behind the GAO recommendation and will explore options to 
improve the timeliness and accuracy of project performance reporting.

Once again, DOE appreciates GAO's identification of opportunities to 
improve its project and contract management systems. Additional 
comments and corrections on the draft report provided by the cognizant 
DOE organizations are attached for your review. The Department 
regularly reviews its approach to implementing and managing its 
projects and their supporting contracts as evidenced by the numerous 
internal reviews/reports examining the Department's implementation of 
these contracts.

The Department continues to seek ways to expand the implementation of 
what works and correct what does not. We appreciate the GAO's 
recommendations and will incorporate them into our future improvement 
efforts.

Sincerely,

Signed by: 

Susan J. Grant: 
Director, Office of Management, Budget and Evaluation/Chief Financial 
Officer: 

Attachments: 

[End of section]

Related GAO Products: 

[End of section]

Nuclear Waste: Absence of Key Management Reforms on Hanford's Cleanup 
Project Adds to Challenges of Achieving Cost and Schedule Goals. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-611] 
Washington, D.C.: June 9, 2004. 

Nuclear Waste Cleanup: DOE Has Made Some Progress in Cleaning Up the 
Paducah Site, but Challenges Remain. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-457] 
Washington, D.C.: April 1, 2004. 

Nuclear Weapons: Opportunities Exist to Improve the Budgeting, Cost 
Accounting, and Management Associated with the Stockpile Life Extension 
Program. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-583] 
Washington, D.C.: July 28, 2003. 

Nuclear Waste: Challenges and Savings Opportunities in DOE's High-Level 
Waste Cleanup Program. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-930T] 
Washington, D.C.: July 17, 2003. 

Nuclear Waste: Challenges to Achieving Potential Savings in DOE's High-
Level Waste Cleanup Program. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-593] 
Washington, D.C.: June 17, 2003. 

Department of Energy: Status of Contract and Project Management 
Reforms. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-570T] 
Washington, D.C.: March 20, 2003. 

Major Management Challenges and Program Risks: Department of Energy. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-100] 
Washington, D.C.: January 1, 2003. 

High-Risk Series: An Update. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-119] 
Washington, D.C.: January 1, 2003. 

Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure 
Initiatives Have Improved Results. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-798] 
Washington, D.C.: September 13, 2002. 

Nuclear Waste: Technical, Schedule, and Cost Uncertainties of the Yucca 
Mountain Repository Project. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-191] 
Washington, D.C.: December 21, 2001. 

Department of Energy: Follow-Up Review of the National Ignition 
Facility. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-677R] 
Washington, D.C.: June 1, 2001. 

Nuclear Cleanup: Progress Made at Rocky Flats, but Closure by 2006 Is 
Unlikely, and Costs May Increase. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-284] 
Washington, D.C.: February 28, 2001. 

High-Risk Series: An Update. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-263] 
Washington, D.C.: January 1, 2001. 

Major Management Challenges and Program Risks: Department of Energy. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-246] 
Washington, D.C.: January 1, 2001. 

(360423): 

FOOTNOTES

[1] GAO, Contract Reform: DOE Has Made Progress, but Actions Needed to 
Ensure Initiatives Have Improved Results, GAO-02-798 (Washington, D.C.: 
Sept. 13, 2002). 

[2] National Research Council of the National Academies, Progress in 
Improving Project Management at the Department of Energy: 2003 
Assessment (Washington, D.C.: 2004). 

[3] Civil Engineering Research Foundation, Independent Research 
Assessment of Project Management Factors Affecting Department of Energy 
Project Success (July 12, 2004). 

[4] DOE's management and operating contracts include the DOE 
Acquisition Regulation clause providing discretion to the fee 
determination official to reduce the contractor's fee by up to 75 
percent when the contractor fails to meet stipulated cost performance 
levels. (See 48 C.F.R.  970.5215-3.) In DOE's management and operating 
contracts, the cost performance level for the contract, unless 
otherwise stipulated, is typically the annual funding level for the 
contract. 

[5] DOE has assessed the contractor's project management system for the 
Spallation Neutron Source project and plans to certify the system after 
some minor changes are made. 

[6] U.S. Army Corps of Engineers, Independent Cost & Schedule Baseline 
Review Summary Report, Hanford Waste Treatment and Immobilization Plant 
(Walla Walla, Washington: May 28, 2004). 

[7] DOE developed the Performance-Based Contracting Guide as a 
reference document for agency personnel involved in all aspects of 
performance-based management contracting. 

[8] The other 9 Environmental Management projects did not have 
associated Gold Chart metrics. 

[9] Environmental Management defines a radioactive facility completion 
as a decommissioning, deactivation, dismantlement, demolishment, or 
transfer of responsibility for the facility to another program or 
owner. 

[10] GAO, Department of Energy: Opportunity to Improve Management of 
Major System Acquisitions, GAO/RCED-97-17 (Washington, D.C.: Nov. 26, 
1996). 

[11] In addition, DOE's project directors and support field staff 
assess contractor performance by walking the site, evaluating work 
performed, and periodically meeting with the contractor. 

[12] In November 2004, DOE revised this chapter in the Reference Book 
for Contract Administrators. Although the revised chapter does not 
mention this specific responsibility, in at least 2 contracts we 
reviewed, we found that the contracting officer had this 
responsibility. 

[13] In addition, during fiscal year 2004, the Office of Environmental 
Management offered five 5-day classes on earned value management to DOE 
project directors and other individuals involved in project management. 

[14] The American National Standards Institute/Electronic Industries 
Association-748-1998 established 32 criteria for use of earned value 
management. Earned value management is widely employed by the private 
sector as a means of ensuring reliable project performance information. 
Contractor project management systems, which include earned value 
management systems, may have been validated by other external 
validation entities, but DOE's certification review in collaboration 
with the Defense Contract Management Agency verifies not only that 
project management systems have earned value management systems in 
place but that they are used in a correct fashion. 

[15] The Defense Contract Management Agency also has assessed the 
Washington Group International's project management system for the 
Elimination of Weapons Grade Plutonium project in Russia, which has not 
passed the Critical Decision 2 milestone. 

[16] The January 2004 total cost was about $1 billion more than DOE's 
original projected total cost, as noted in our report entitled Nuclear 
Waste: DOE's Hanford Spent Nuclear Fuel Storage Project--Cost, 
Schedule, and Management Issues, GAO/RCED-99-267 (Washington, D.C.: 
Sept. 20, 1999). 

[17] These 4 projects are the East Tennessee Technology Park Three 
Building Deactivation and Decommissioning, K25/27 Buildings 
Deactivation and Decommissioning Removal, Molten Salt Reactor 
Experiment, and Oak Ridge National Laboratory Burial Grounds. 

[18] These 63 projects did not include 25 Environmental Management 
projects. Nineteen, or 76 percent, of the 25 Environmental Management 
projects were expected to meet cost and schedule baselines. 

[19] Our report entitled Nuclear Weapons: Opportunities Exist to 
Improve the Budgeting, Cost Accounting, and Management Associated with 
the Stockpile Life Extension Program, GAO-03-583 (Washington, D.C.: 
July 28, 2003) recommended that DOE manage its weapons refurbishment 
programs as projects. Although DOE agreed with this recommendation, 
NNSA has not implemented it. DOE defines a project as a specific 
undertaking, with defined beginning and end points, that supports a 
program mission. 

[20] These are the Solid Waste Stabilization and Disposition and the 
Radioactive Liquid Tank Waste Stabilization and Disposition projects. 

[21] U.S. Army Corps of Engineers, Independent Cost & Schedule Baseline 
Review Summary Report, Hanford Waste Treatment and Immobilization Plant 
(Walla Walla, Washington: May 28, 2004). Also GAO, Nuclear Waste: 
Absence of Key Management Reforms on Hanford's Cleanup Project Adds to 
Challenges of Achieving Cost and Schedule Goals, GAO-04-611 
(Washington, D.C.: June 9, 2004). 

[22] GAO, Major Acquisitions: Significant Changes Underway in DOD's 
Earned Value Management Process, GAO/NSIAD-97-108 (Washington, D.C.: 
May 5, 1997). 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: