DOE Contracting: Better Performance Measures and Management Needed to Address Delays in Awarding Contracts
Released: 2006-08-01T00:00:00

Letter

Background

DOE has numerous sites and facilities around the country where the department carries out its missions, including developing, maintaining, and securing the nation’s nuclear weapons capability; cleaning up the nuclear and hazardous wastes resulting from more than 50 years of weapons production; and conducting basic energy and scientific research, such as mapping the human genome. DOE relies on contractors to operate its facilities and accomplish its missions. This mission work is carried out under the direction of NNSA and DOE’s program offices, including the Offices of Environmental Management and Science.

The Federal Acquisition Regulation provides the basic guidelines that all federal agencies must follow in planning for and carrying out the process for awarding contracts. DOE, like most federal agencies, has supplemental regulations that it has set forth in the Department of Energy Acquisition Regulation to recognize requirements unique to the department, such as maintaining nuclear safety and security. Furthermore, in the mid-1990s, DOE’s Office of Procurement and Assistance Management developed the DOE Acquisition Guide that provides further information on contract award planning activities, including meeting federal requirements for competition, establishing evaluation criteria, and selecting the appropriate contract type.

In planning and carrying out competition and award of contracts, there are three main phases, and each phase generally includes several steps.

Planning. This phase involves identifying the mission need for the contract, alternatives for obtaining the goods or services, estimated costs, and any program or technical risks. Federal regulations state that the agency should use an integrated team approach—involving all personnel responsible for significant aspects of the contract award process—to developing the plan and associated milestone schedule. The plan should also identify the appropriate contract type for the work, considering the alternatives and risks. In addition, a selection official is designated, and an evaluation board is formed to carry out the contract award process.

Developing the solicitation. Generally, the evaluation board, with input from technical and other advisors, develops a statement of work for the contract and criteria that will be used to evaluate proposals submitted by companies. During this phase, DOE may also do market research to determine if sources capable of meeting the agency’s needs exist. Finally, DOE issues the request for proposals that describes the work to be done, provides instructions to companies on how and when to submit their proposals, and describes the criteria that will be used to evaluate the proposals and select a contractor.

Evaluating proposals and awarding the contract. Once proposals are received, the evaluation board evaluates the proposals against the established criteria. Evaluation criteria may differ for each contract, but generally include factors such as a company’s proposed technical approach; past performance on DOE, other federal agency, and commercial contracts; the proposed cost; and the qualifications of the key personnel identified in the proposal. During this phase, DOE may determine that it is necessary to hold discussions with individual companies about issues raised during the evaluation of their proposals. Based on a written report of the evaluation board that outlines the strengths and weaknesses of each proposal, the selection official should select a proposal that represents the best value for the government. The DOE contracting officer, who has generally provided contracting support to the evaluation team, then awards the contract.

The Competition in Contracting Act of 1984 provides statutory authority for GAO to decide cases involving bid protests. GAO’s bid protest decisions address specific allegations raised by unsuccessful companies claiming that particular contracting actions were contrary to federal laws and regulations. A bid protest may be filed by an “interested party,” defined as an actual or prospective bidder or offeror with a direct economic interest in the contract. Unless the protest is dismissed at the outset because it is procedurally or substantively defective, such as if the protest was not filed in a timely manner, the contracting agency is required to file a report with GAO responding to the protest. GAO will consider the facts and legal issues raised and is required to resolve the protest not later than 100 days from the date the protest was filed, either by sustaining the protest and recommending that the agency take corrective action, or by denying or dismissing the protest.

DOE’s Office of Procurement and Assistance Management and NNSA’s Office of Acquisition and Supply Management establish policies and guidance for awarding contracts according to federal and departmental regulations. Officials from DOE’s programs, such as NNSA and the Offices of Environmental Management and Science, usually manage the contract planning and award process since they are most knowledgeable about the work needed and are ultimately responsible for managing the work and overseeing the contracts. Other organizations within DOE work with the program officials to plan and carry out the contract award process by providing contracting, legal, and technical advice and assistance. DOE’s Offices of General Counsel and Procurement and Assistance Management review, evaluate, and approve key documents during the entire process, such as the initial plan for the contract award process and the request for proposals.

Delays Occurred in Most of the Contract Awards We Reviewed

Most of DOE’s 31 contract awards that we reviewed experienced delays when comparing the planned and actual award dates for the contracts. The 24 contracts awarded with competition faced delays of months to years; of the 7 contracts awarded without competition, 6 were awarded on time or nearly so. Delays occurred throughout the contract award process due to factors such as time spent repeating steps of the process to resolve a company’s formal protest, or because of unanticipated complications that affected DOE’s ability to adhere to the planned schedule.

DOE Experienced Delays of Months to Years in Awarding Its Contracts

For the 31 contracts we reviewed, most were awarded months to years after the planned award date. To determine whether contract awards were delayed, we compared the planned award date with the date DOE actually made the award. None of the 24 competitive contracts we reviewed was awarded according to DOE’s planned schedule, and nearly all were delayed by months or years. Only 1 of these competitive contract awards was made within 1 month of the planned award date. The remaining awards experienced more lengthy delays of from 2 months to over 2 years. In contrast, delays were less prevalent among the noncompetitive contract awards included in our sample. Six of these 7 contracts were awarded on time or within 1 month of the planned award date. The other contract was awarded 2 months late. (See fig. 1.)

While the sample of contract awards reviewed did not allow us to generalize to all of DOE’s contract awards, they represent about 73 percent of the department’s total contracting dollars for awards of $5 million or greater during fiscal years 2002 through 2005. Further information on our scope and methodology is presented in appendix I. eliminate parts of the process in which delays typically occurred on the competitively awarded contracts we reviewed. For example, for a noncompetitively awarded contract, DOE would not need to evaluate multiple proposals because it had already identified a company with the qualifications and experience necessary to perform the work. DOE generally awarded the seven contracts we reviewed without competition because of an urgent need to obtain the goods and services and to ensure that the contractors could start work quickly. This is consistent with one of the exceptions to the requirement for full and open competition allowed by federal regulations. Nevertheless, since federal regulations generally require that agencies use full and open competition, the contracts awarded competitively better reflect the timeliness of DOE’s overall contract award process.

The schedules for the contract awards we reviewed showed that DOE planned to take from about 1 month to up to 2 years to award its contracts. Variation in the length of the planned schedule may depend on several characteristics of the contract award process, such as whether or not the contract was awarded competitively or noncompetitively, the dollar value of the award, or the complexity of the work. DOE officials indicated that facility management contracts tend to be more complex and, therefore, can take longer and could be more prone to delay, but we found some facility management contract awards were less delayed than some smaller dollar value, more straightforward contract awards. For example, the $4.8 billion facility management contract to operate the Idaho National Laboratory took just over 1 year to award and was delayed about 4 months, while a $5 million administrative support services contract at the Savannah River site took almost 2 years to award and was awarded 1-1/2 years later than the planned award date.

Delays Occurred Because DOE Had to Modify Its Approach after Beginning the Contract Award Process

Delays in awarding contracts occurred when DOE modified its approach after the start of the contract award process. DOE frequently had to modify its approach to take corrective action in response to bid protests, or because DOE took more time than planned to evaluate proposals, hold discussions with companies that had submitted proposals, or obtain headquarters approval for key documents.

Taking corrective action due to bid protests. One cause of delay for DOE’s contract awards was the time the department spent reworking portions of the process in response to companies that filed bid protests with GAO. Companies generally filed protests because they believed that DOE had violated federal acquisition regulations in carrying out the contract award process, most typically in evaluating or soliciting proposals. Rework in response to a bid protest was initiated in one of two ways: (1) GAO found fundamental flaws in the contract award process, sustained the company’s protest, and recommended that DOE take corrective action or (2) DOE initiated corrective action after conducting an assessment of the validity of the company’s protest, without waiting for a formal GAO decision. In either case, rework can include canceling or revising the solicitation, reevaluating proposals, or accepting proposals from firms that were previously excluded from the competition.

Rework resulting from corrective action can significantly delay the contract award. For the 31 contracts we reviewed, 10 resulted in bid protests filed with GAO. Protests on 8 of these were denied or dismissed by GAO, or withdrawn by the protestor. DOE agreed to take corrective action on the remaining 2, once as a result of a protest being sustained by GAO, and once before GAO issued a decision. Delays due to such corrective action added up to 2 years to the contract award process. For example, a contract worth $1.6 billion to clean up the River Corridor area at DOE’s Hanford, Washington, site was delayed more than 1-1/2 years while DOE took corrective action in response to a bid protest. GAO sustained a bid protest on the initial award because DOE failed to properly apply its criteria for evaluating the cost proposals. In response, DOE revised and reissued the solicitation, evaluated new proposals, and selected a new contractor about 2-1/2 years after the initial planned award date. In another example, a NNSA contract award for administrative support services was delayed almost 2 years when NNSA took corrective action in response to a bid protest. Corrective action involved revising and reissuing the solicitation. However, the NNSA evaluation board then had difficulties completing cost evaluations of the revised proposals, adding more time to the process, and finally awarded contracts to all three companies almost 2 years later than the planned award date.

Evaluating proposals. DOE generally estimated between 1 and 6 months to review and evaluate proposals for its competitive contract awards. However, for the 24 competitively awarded contracts we reviewed, it took on average 5 months longer than planned for this part of the process, accounting for over half of the overall delay in carrying out the process. For example, DOE was planning to award multiple contracts to provide environmental remediation and deactivation of facilities across the DOE complex. DOE estimated that it would receive about 20 proposals and allowed about 3 months for evaluating the proposals and selecting the contractors. The department actually received more than 100 proposals and was delayed over 9 months, in part, due to the amount of additional time required to review the larger than expected number of proposals. In another example, the Savannah River Operations Office allowed about 8 weeks to evaluate proposals for an administrative support services contract at the site. However, the evaluation board encountered several difficulties during the evaluation process, including concerns about the wide range of the cost estimates in the proposals. These concerns necessitated a more detailed cost evaluation, resulting in the evaluation taking 5 months longer than planned.

Holding discussions with companies submitting proposals. For eight of the contracts we reviewed, DOE had to modify its approach to holding discussions. In some cases, the planned schedule did not include any time for discussions and, in other cases, the time DOE took to discuss proposals with the companies submitting them exceeded the time estimated in the plan. For these contracts, actual time to discuss proposals exceeded the planned time by an average of 3 months. For example, two contracts for conceptual design of a waste processing facility at the Savannah River site were awarded 4 months late because DOE initially intended to make the contract awards without discussions and, therefore, did not plan time in the original schedule for this part of the process. However, after the contracting process was under way, the department decided it was necessary to conduct discussions on the proposals because none of the companies had provided all of the cost and pricing data necessary to fully evaluate the proposals. Similarly, a contract for medical services for the Hanford site was delayed more than 5 months, in part because the department held discussions with companies concerning their cost proposals and technical approach but did not plan for this activity in the schedule.

Obtaining headquarters approval of key documents. Awarding of DOE contracts was also delayed due to obtaining headquarters approval of key documents. In five contract awards we reviewed, delays in obtaining the required review and approval from DOE headquarters officials caused an average 5-month delay in contract award. For example, for a contract to manage and operate DOE’s Berkeley National Laboratory, DOE allowed 1 month for headquarters review and approval of the draft solicitation. However, headquarters review and approval took 5 months, partly due to the complexities of competing a contract that had been in place for decades. As a result, the contract award was 4 months late. In another example, the Oak Ridge Operations Office allowed 1 month for headquarters review and approval of a solicitation for a contract for medical isotope production. However, headquarters review and concurrence took about 1 year longer than anticipated because of additional requests for information from the Congress on DOE’s strategy for going forward with the work under this contract. Overall, the contract award was delayed nearly 2 years due to this lengthy review and the need for extensive discussions during the evaluation phase with companies that had submitted proposals.

It is unclear the extent to which these delays in awarding contracts were avoidable. At times, delays in the process appeared to be outside DOE’s control and, therefore, more difficult to anticipate and manage, such as difficulty obtaining funding or to address concerns raised by the Congress or other stakeholders. For example, a $562 million contract for facilities and services to stabilize the department’s depleted uranium hexafluoride inventory at Portsmouth and Paducah was delayed after the Office of Management and Budget, late in the process, raised concerns about the need to build facilities at both locations. Dealing with this issue contributed to the delay of the contract award. While external factors such as this can impact the planned schedule, some delays are clearly avoidable, such as when DOE had to take corrective action in response to bid protests because the department failed to follow its contract award process. In such cases, better management and oversight of the contract award process would provide opportunities for improvement. According to the Director of DOE’s Office of Procurement and Assistance Management, DOE cannot plan for all possible events when developing a schedule for a contract award process and must respond to changing circumstances and incorporate additional steps into the award process when necessary.

Delays in Awarding Contracts Could Increase Costs to the Government and Industry and Affect Future Competition

Delays in awarding contracts could increase costs both to DOE and the companies that are competing for the potential contracts and could also affect whether companies are willing to compete for DOE contracts in the future. Specifically, awarding a contract months or years after the date anticipated in the contract award schedule could increase the overall costs to the government. DOE staff involved in a contract award may include officials from the program offices, such as the Offices of Environmental Management or Science; contracting and legal specialists; and nuclear safety and regulatory experts. Both salary and travel costs associated with keeping these integrated teams together for the duration of the contract award process can be significant. However, these costs cannot be readily quantified since the department does not accumulate or track the costs associated with individual contract awards. Nevertheless, DOE officials acknowledge that there may be additional costs associated with contract awards that are delayed. The added time and resources spent on delayed contract awards may impact the department’s ability to meet its management and oversight responsibilities, including planning for and carrying out other contract awards.

According to DOE officials, prolonged delays in awarding contracts could also increase costs for companies that are submitting proposals. Companies competing for DOE contracts invest considerable time and resources in developing their proposals. However, since companies generally consider their proposal costs to be proprietary information, we could not quantify those costs. For DOE’s contracts, once companies submit proposals, DOE generally requires the companies to ensure that key personnel identified in the proposals continue to be available until the award decision has been made. If a contract award is delayed, companies may have to keep key personnel available for months or even years, which could increase their costs and/or represent a possible lost opportunity to the companies since those key personnel may not be available for other work. For example, our review of file documents found that several small businesses competing for a contract to provide infrastructure services at DOE’s Paducah, Kentucky, site expressed frustration over the costs of keeping key personnel available while DOE took nearly 1 year after the planned award date to complete the process and award the contract.

The length of time and increased costs to carry out a contract award may also discourage some companies from competing for DOE work, thus potentially affecting the extent of competition in the future. Ensuring adequate competition for contracts can help obtain the best value for the government. According to DOE officials who oversee the contracting process, they have not seen a decline in the number of companies willing to compete for DOE’s contracts, but the officials agree that this is a potential concern, and it has been a concern in the past. For example, a January 2001 report that analyzed the contractor base for DOE’s environmental cleanup work stated that the number of companies that were both willing and able to compete for major remediation contracts had declined over the past several years. The report concluded, among other things, that promoting competition among a large number of qualified companies was crucial to getting the best value for the government and accomplishing the environmental cleanup mission. Therefore, anything that could reduce the pool of potential companies competing for DOE’s work, such as an inefficient contract award process, would not be in the department’s best interest of fostering full and open competition for its work.

Efforts to Address the Causes of Delays Are in Early Stages, but Concerns Remain about the Potential Effectiveness of DOE’s Planned Actions

When we began our review in July 2005, DOE officials generally were not addressing delays in awarding contracts because DOE’s data was showing that its contracts were being awarded in a timely manner. However, since late 2005, DOE has been implementing three main efforts to improve its contracting, partly in response to criticism by GAO and others. These efforts include (1) an initiative to address the DOE contracting weaknesses that led GAO to designate the area as high risk for fraud, waste, abuse, or mismanagement; (2) a plan to restructure DOE’s Office of Environmental Management to focus more attention on the contract award process; and (3) steps by NNSA and DOE’s Office of Procurement and Assistance Management to improve performance measures on the timeliness of contract awards. While these improvement efforts appear to be constructive and may also help to address delays in awarding contracts, we have two remaining concerns related to the timeliness of DOE’s contract awards. First, the efforts do not reflect a comprehensive approach within the department to improve the timeliness of contract awards. Second, even if DOE uses improved performance data to identify the causes of delays in awarding its contracts, the department does not have a systematic method to develop and disseminate best practices and lessons learned.

DOE Was Not Addressing Delays Because Incomplete Performance Data Indicated Contracts Were Awarded in a Timely Manner

When we began our work in July 2005, DOE officials generally were not taking steps to address the causes of delays in awarding contracts. One of the main reasons for this was that DOE’s performance data was generally showing its contracts were being awarded in a timely manner. Although our analysis uncovered delays in most of the 31 contract awards valued at $5 million or greater from fiscal years 2002 through 2005 that we reviewed, DOE’s performance data for the same period showed that most of its contract awards were timely. For example, the data for fiscal years 2003 and 2004 showed that, respectively, 88 and 86 percent of the contract awards were timely. According to DOE’s performance measures, a timely award meant that DOE had awarded the contracts within 150 days after receiving proposals from companies. An official from the Office of Procurement and Assistance Management who manages the performance data said that the department used 150 days as a timeliness standard because it represented a realistic yet challenging time frame for awarding the contracts.

However, DOE’s performance data did not provide a complete picture of the department’s timeliness in awarding its contracts for two reasons. First, the data for those years did not include all of the contract award process. Instead, the performance data DOE used measured only the timeliness in carrying out the final stage of the process (the evaluation phase), from the point at which DOE received the proposals through contract award. Consequently, the performance data excluded the earlier steps of the process, from planning the contract award through the process of soliciting proposals from interested companies. For the 24 competitively awarded contracts valued at $5 million or greater we reviewed, the excluded steps accounted for over half of the total time that DOE typically spent awarding its contracts. As a result, some of DOE’s contract awards may appear timely, when, in fact, DOE awarded the contracts weeks or months after the planned award date. For example, in fiscal year 2003, DOE awarded a contract for laundry services at the Hanford site in Washington about 2 months after its planned award date. However, DOE considered the contract award to have been timely because the department was able to award the contract within 150 days of receiving proposals.

Officials from the Office of Procurement and Assistance Management said DOE’s timeliness data are intended to measure the performance of contracting staff. Therefore, the officials said that the timeliness measure includes only the evaluation phase of the contract award process because that is the aspect of the process that would more likely be under the direct control of the contracting staff. Prior to the evaluation phase, officials said that contracting staff generally rely on program staff to develop key information, such as a description of the work to be carried out under the contract. Furthermore, according to DOE’s guidance, program office staff are generally responsible for planning the contract award process. However, we found that DOE’s contracting staff were generally involved throughout the entire contract award process as part of an integrated team approach. Furthermore, it is unclear to what extent contracting staff have direct control over any phase of the process.

Second, DOE’s performance data were incomplete because data on DOE’s facility management contracts were excluded. For example, in fiscal year 2005, DOE competitively awarded seven facility management contracts valued at a total of about $10.4 billion. However, DOE excluded the data on these facility management contracts from its performance measures for timeliness. DOE officials from the Office of Procurement and Assistance Management said they excluded the facility management contracts from the department’s performance data because those contracts are particularly complicated to award. Unlike the contracts included in the performance measures for timeliness, the department could not determine an appropriate timeliness standard for the facility management contracts. As a result of excluding the facility management contracts, the performance data for fiscal year 2005 did not reflect the department’s timeliness in awarding about 97 percent of the nearly $11 billion in contracts that the department awarded that year.

In contrast to DOE’s approach, we measured the department’s timeliness in awarding its contracts by comparing the planned award dates with the dates DOE actually awarded the contracts. We used the planned award date as the standard for determining whether contracts were awarded in a timely manner, rather than comparing the time it took DOE to award the contract after proposals were received with a fixed standard, such as 150 days. A fixed standard is arbitrary because the actual time needed to carry out a contract award process can differ greatly for legitimate reasons, depending on the circumstances of each individual contract award. Furthermore, in doing so, we included the facility management contracts because those contracts can represent a significant portion of the total value of DOE’s contract awards.

DOE Recently Began Three Main Efforts to Improve Its Process for Awarding Contracts

Since late 2005, DOE has initiated three new efforts to improve its contracting processes. DOE is implementing these actions as part of its continuing efforts to improve its contracting processes and, in part, because of long-standing criticisms by GAO and others. Since 1990, GAO has designated DOE’s contract management, including project management, as an area at high risk for fraud, waste, abuse, and mismanagement. Our reviews have uncovered weaknesses in DOE’s management and oversight of its contracts which, in some cases, have undermined DOE’s ability to carry out its missions or dramatically increased its costs. More recently, criticisms have included the frequency of delays in awarding contracts and the need for reforms. For example, in congressional testimony in November 2005 and March 2006, the Assistant Secretary for Environmental Management cited the need for improvements in the department’s contract award practices. Furthermore, conferences on DOE environmental cleanup issues held in February and October 2005 and attended by DOE and contractor officials included presentations calling for a reevaluation of DOE’s contract award process.

DOE’s recent efforts to improve its contracting include the following: Initiative to address DOE’s high-risk contracting practices. In November 2005, DOE drafted a plan aimed at improving the department’s contract and project management, including management of the contract award process. DOE developed the plan in response to an effort by the Office of Management and Budget that requires agencies to address areas identified by GAO as high risk for fraud, waste, abuse, or mismanagement. Actions included in the plan to improve the contract award process include conducting better oversight of the planning process, maximizing competition for contracts, and ensuring that contracts allow DOE to properly hold contractors accountable for their performance. To date, DOE has developed performance measures and targets for the focus areas in its action plan, as well as target dates for implementing the plan. As part of this initiative, in February 2006, DOE began to increase the frequency of training for new evaluation boards and selection officials and to provide this training at the beginning of the planning process. Although this plan does not specifically address the timeliness of contract awards, the overall effort to improve contracting and project management could help identify and address underlying causes for delays in awarding contracts.

Initiative to restructure DOE’s Office of Environmental Management.

Also in late 2005, the Assistant Secretary for Environmental Management announced plans to restructure the headquarters-based administrative functions within his program office. According to officials in the Office of Environmental Management, the reorganization is aimed, in part, at improving contract and project management by bringing these functions together under a new Deputy Assistant Secretary for Acquisition and Project Management. As the reorganization is implemented, the Office of Environmental Management, at the request of a congressional subcommittee, has also commissioned the National Academy of Public Administration to undertake a general management review of the program, including reviewing the program office’s policies and processes for awarding contracts and providing advice on improving their effectiveness. As the study progresses, DOE will receive interim reports and recommendations on how to implement the reorganization and increase its effectiveness, which may also result in improving the timeliness of contract awards.

Initiative to improve performance data on contract timeliness. NNSA and DOE are implementing separate efforts to help them track contract awards against the planned dates for carrying out the process. NNSA’s effort is focused on a recently implemented data system that will help its contracting staff track ongoing contract awards against a detailed schedule of milestones. The requirement to input a detailed milestone schedule at the beginning of the contract award process applies to all of NNSA’s contracts, regardless of their dollar value. As staff carry out the process, the system will record the dates on which key steps were completed and document any delays from the schedule. The system also allows the contracting officials to generate reports on the status of ongoing contract awards, as well as analyze data on existing contract awards for evidence of systemic delays or other problems. As of April 2006, the data system was operational, and NNSA contracting officials have started using this milestone information to manage ongoing contract awards.

Similarly, in March 2006, DOE’s Office of Procurement and Assistance Management began implementing plans to track the timeliness of DOE’s facility management or other high-visibility contracts against a detailed milestone schedule. According to the Director of the Office of Procurement and Assistance Management, representatives from his office, as well as other department officials responsible for awarding facility management contracts, will work to develop a realistic, but challenging, milestone schedule for carrying out the award process.

According to the Director, revised DOE guidance will recommend that planning for a new facility management contract should begin no later than 2 years prior to the expiration of an existing contract. Once the milestone schedule has been established, his office will then track the contract award process to compare planned and actual key dates and provide monthly updates to the Deputy Secretary. As of April 2006, DOE had established schedules for the current and upcoming facility management contract awards, began monitoring the progress against those schedules, and began submitting monthly status reports to the Deputy Secretary.

DOE’s efforts to improve its contracting are generally in the early stages of being planned or implemented, and not all of the efforts specifically address the timeliness of contract awards. Nevertheless, we believe these initiatives are a constructive step towards addressing long-standing weaknesses in the department’s contracting and improving its contract awards. As the department addresses the weaknesses and takes steps to improve its contracting, improved efficiency and effectiveness could also result in fewer delays in awarding contracts. However, as discussed below, we are concerned that the efforts may fall short in two main respects, which could limit the department’s effectiveness in improving the timeliness of contract awards.

Lack of a Comprehensive Approach to Improvement Efforts and Inconsistent Practices for Communicating Lessons Learned May Limit the Effectiveness of DOE’s Efforts

Standards for a well-functioning contract award process come from federal acquisition regulations and GAO’s framework for assessing the process at federal agencies. The framework integrates the views of federal government and industry experts on the characteristics and practices of a well-functioning contract award and management process, and it draws upon decades of experience within GAO in analyzing contracts and contracting organizations. Among the standards for a well-functioning contract award process set forth by the framework and regulations are that federal agencies should use knowledge from prior contract awards to assess the effectiveness of the contracting process and, in the interest of continuous improvement, have a systematic means for instituting best practices and lessons learned. DOE’s improvement initiatives, as they apply to the timeliness of contract awards, appear to fall short in both respects.

First, DOE has not ensured that the efforts to develop better data on the timeliness of contract awards will include all of the department’s contract awards that require a written plan and schedule. While DOE and NNSA’s efforts to develop better timeliness data will include all of the department’s facility management contracts, as well as NNSA’s other contract awards, the rest of DOE’s contracts may not be included. This could be a considerable portion of DOE’s contracts valued at $5 million or greater. For example, of the contracts that the department awarded in fiscal years 2002 through 2005, nearly one-quarter of the total contract value, or $3.9 billion, may not have been included in DOE’s improvement efforts. These contracts may be excluded because they were neither facility management contracts, other DOE high-visibility contracts, nor contracts awarded by NNSA. The Director of DOE’s Office of Procurement and Assistance Management said that the tracking against milestone schedules includes only contracts that represent significant risk or are critical to carrying out DOE’s programs or missions. For example, the contracts that are currently being tracked include facility management contracts, other large cleanup contracts, and contracts that may also be of particular interest to headquarters, such as those for guard services. The Director added that it is not the intent of this effort to monitor the progress of all ongoing contract awards of $5 million or greater, and that he would expect contracting officers in the field locations to be accountable for contracts within their approval authority. Nevertheless, we believe that efforts to develop better timeliness data should include all contracts valued at $5 million or greater because DOE guidance generally requires written plans and schedules for these contracts.

Second, the department has neither had a systematic approach for instituting best practices or lessons learned in its contract award process, nor is it clear that the department’s efforts will result in a more systematic approach in the future. As DOE develops better data on the timeliness of its contract awards and analyzes the data to identify and address systemic problems or underlying causes of delays, the department will be developing useful information that could help ensure that future contract awards benefit from the best practices or lessons learned. Moreover, federal regulations require agencies to have a process for ensuring that lessons learned are developed and used in planning future contract awards.

Based on federal regulations and GAO’s framework, standards for a well- functioning contract award process would include a systematic approach for identifying and communicating lessons learned and best practices. Such an approach would include activities such as routinely evaluating performance data across a broad range of contracts to identify the extent to which the contract award process went well or encountered problems. In doing so, DOE would then look for trends or systemic problems and investigate the underlying causes. A systematic approach would also include a mechanism to ensure this information is readily available, in order to help officials who plan and carry out contract awards do so more efficiently.

We did find some examples in which best practices and lessons learned were identified and in some cases shared within the department, but we did not find a systematic approach for doing so. For example, in a recent DOE evaluation of the problems that occurred in awarding contracts for environmental cleanup and infrastructure services at its facilities in Portsmouth, Ohio, and Paducah, Kentucky, DOE’s review confirmed that poor planning and management of the process caused delays of 6 months to 1 year in awarding the contracts. Some of the factors mentioned were lack of timely management direction in key areas, a lengthy headquarters review and approval process, and unrealistic schedules for the awards. However, DOE concluded that the factors leading to the delays were not typical for DOE’s contracts and did not represent a systemic problem. Therefore, DOE did not communicate the information as a systemic problem. In contrast, our analysis of 31 contracts valued at $5 million or greater indicated that these types of causes for delays were commonplace.

In addition, best practices or lessons learned have been communicated at training sessions and presentations to staff carrying out contract awards, such as those offered by DOE’s Office of General Counsel. According to the DOE attorney who has led a number of them, the sessions are generally aimed at helping staff carry out the contract award process effectively and in accordance with federal regulations and DOE guidance. This may increase the department’s ability to avoid successful bid protests. Best practices or lessons learned have also been shared at annual procurement directors’ conferences held for officials who oversee the contract award process, with the most recent conference in October 2005 covering topics such as proper documentation of decisions made during the contract award process.

However, several DOE officials that were responsible for contract awards said that these best practices or lessons learned were not consistently available or may have come too late in the planning for the contract award to be very beneficial. The officials added that there was no centralized source within DOE for best practices, but they could obtain lessons learned by contacting others within the department who had recently awarded contracts. For example, an official from the Office of Environmental Management said that, in planning the contract award process for DOE’s Savannah River site near Aiken, South Carolina, she traveled to DOE’s offices in Idaho Falls to meet with officials there who had just awarded contracts for environmental cleanup at the site, and management and operation of the Idaho National Laboratory. While she said the meetings were beneficial and she obtained lessons learned documents, she saw this process as the only way to obtain best practices or lessons learned from those contract awards. As another example, this same DOE official conducted a recent contract award process, and said she would have benefited from early training on how to properly document decisions made while evaluating the proposals that were submitted. She added that the training was not made readily available, and it was not mandatory. After submitting draft documents for headquarters review, she eventually received helpful information on how to improve the documentation, but this would have been more beneficial earlier in the process.

Conclusions

DOE relies on contractors to carry out its environmental cleanup, scientific research, nuclear weapons management, and other missions vital to national health, safety, and security. Our analysis of the contracts we reviewed showed that DOE experienced delays in awarding many of these contracts to carry out its critical missions. It is unclear the extent to which these delays may have been avoidable with better oversight or management, but at least some of the delays were avoidable, such as when DOE had to rework parts of the contract award process to correct errors. While DOE eventually awarded these contracts, a contract award process that takes too long runs the risk of undermining the department’s efforts to obtain needed goods or services at the best value for the government. As DOE goes forward with its efforts to improve its contract award process, it will be important to ensure that performance measures address the timeliness of all of the department’s contract awards that require written plans and schedules, and that lessons learned and best practices are consistently shared with DOE staff that will plan for and carry out future contract awards. Improving the department’s performance in awarding contracts is an important part of ensuring that DOE’s contracts provide the best value to the government.

Recommendations for Executive Action

To help ensure that DOE’s contract award process for contracts that require a written plan and schedule is efficient and effective and that DOE is obtaining the best value for the government, we recommend that the Secretary of Energy take the following two actions: Ensure that performance measures for the timeliness of the department’s contract awards include the entire contract award process from planning to contract award and include all of the department’s contracts that require a written plan and schedule.

Establish a more systematic way of identifying lessons learned from past and current contract awards and sharing those lessons and best practices with the staff involved in planning and managing the department’s efforts to award its contracts.

Agency Comments and Our Evaluation

We provided a draft of the report to DOE for review and comment. In written comments, the Director of the Office of Management generally agreed with our recommendations, but questioned whether we intended the recommendations to apply to all of the department’s contracts. We modified the wording to clarify that our recommendations apply only to those contracts that require a written plan and schedule. In addition, DOE raised concerns about our nonprobability sample, our characterization of contract awards as delayed, and our view that DOE’s existing performance measures were incomplete.

Regarding our sampling methodology, DOE said that the results of our nonprobability sample should not be used to reach overall conclusions about DOE’s contracting practices because the 31 contracts we reviewed represented less than 1 percent of the 5,000 contracts DOE awarded during fiscal years 2002 through 2005 and were not representative of the entire group of contracts. However, we believe DOE has mischaracterized our scope and methodology. First, we did not derive the sample of 31 contracts from the 5,000 total contracts that DOE awarded during the period. Instead, we focused our efforts on the 131 contracts awarded by DOE that were valued at $5 million or greater because such contracts generally required a written plan, including estimated dates. The 31 contracts we reviewed represented about 73 percent of the total dollar value of those 131 contract awards valued at $5 million or greater during the period and about 24 percent of the total number of these larger contract awards. Furthermore, our report clearly states that the results of a nonprobability sample cannot be used to make inferences about a population, and we did not draw conclusions about DOE’s overall contracting processes based on the sample results.

DOE also took exception to our characterization of contract awards as delayed and our use of the department’s originally planned award date as an indication of timeliness. DOE said that the milestone schedule developed in the written plan is an internal planning tool that may change over time, and the department cannot factor in a schedule contingency for every possible event. However, as we reported, federal regulations state that the purpose of planning the contract award process is to ensure that the government obtains the needed goods and services in the most effective, economical, and timely manner. The regulations stress the importance of timeliness in awarding contracts and require that written plans for carrying out the contract award process include milestones for completing the steps in the process. Adherence to the milestones established in the written plan provides a consistent measure of timeliness and is one indicator of how well the contract award process is being managed. In addition, our report clearly states that some of the contracts DOE awarded later than planned were subject to external events and complications beyond DOE’s immediate control and recognizes that DOE’s milestone schedule cannot anticipate all possible events. Even so, we continue to believe that better management and oversight of the contract award process, including tracking actual against planned dates in the milestone schedule, could have prevented lengthy delays in awarding some of the contracts we reviewed.

In an attachment to its letter, DOE also disagreed with our conclusions that the department’s performance measures for timeliness in awarding contracts were incomplete and, as a result, the department was not addressing delays in awarding contracts. As we reported, DOE has stated that its timeliness measures were intended to evaluate the performance of contracting staff and thus included only the parts of the contract award process under their direct control. DOE also said the department had been addressing delays in awarding contracts through its lessons learned process. We continue to believe that DOE’s performance measures for timeliness in awarding contracts could be improved by being more comprehensive. DOE’s performance measures included only the evaluation phase of the contract award process and excluded the facility management contracts, which represent a significant percentage of DOE’s overall contracting dollars. We also believe that DOE was generally not addressing these delays before our review began. The three initiatives DOE started in late 2005—to address GAO’s high-risk designation for DOE’s contract management, to restructure the Office of Environmental Management, and to improve performance data on contract timeliness—were the main efforts the department identified as steps to address delays in awarding contracts.

DOE also provided technical comments in an attachment to the letter, which we have incorporated and summarized as appropriate. DOE’s comments on our draft report are included in appendix II.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Secretary of Energy. We will also make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions on this report, please contact me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Scope and Methodology

In response to a congressional request, we determined (1) the extent to which the Department of Energy (DOE) adhered to its planned dates for awarding contracts and the factors contributing to any delays, (2) the impacts of any delays in awarding contracts, and (3) the extent to which DOE has taken steps to address delays in its contract award process. To conduct our work, we analyzed contracts valued at $5 million or greater from multiple DOE locations and interviewed contracting and program officials at DOE headquarters and field locations. We also reviewed federal laws and regulations on contracting, agency policies and guidance, GAO guidance on bid protests, and standards for assessing DOE’s management and oversight of its contract award process.

To determine the extent to which DOE adhered to its planned dates for awarding contracts, we analyzed 31 contracts from seven DOE locations that (1) were affiliated with the department’s three largest component organizations—the Office of Environmental Management, Office of Science, and the National Nuclear Security Administration—and (2) had awarded a large share of the department’s contracts valued at $5 million or greater in fiscal years 2002 through 2005, including facility management contracts. We focused on the three largest component organizations because they comprised over 80 percent of DOE’s annual budget and because they were affiliated with about two-thirds of the locations departmentwide that had awarded contracts in the four fiscal years we reviewed. We considered the four most recent fiscal years, 2002 through 2005, in the scope of our study to help ensure that the results reflected DOE’s recent activity, while increasing the number of contracts that could potentially be selected for analysis.

In fiscal years 2002 through 2005, the department awarded over 130 contracts valued at $5 million or greater, for a total value of about $16.3 billion. From seven DOE locations, we selected a nonprobability sample of 31 of these contracts, accounting for about $11.9 billion, or 73 percent, of the $16.3 billion (see table 1). Contracts in our sample ranged in value from about $5.5 million to $4.8 billion, and they included 7 of the 8 facility management contracts that the department competitively awarded during the four fiscal years we studied. In selecting our sample, we analyzed DOE contract award data to identify locations that had awarded a large share of the department’s contracts in fiscal years 2002 through 2005. To verify the completeness and accuracy of contract award data, we interviewed knowledgeable DOE officials at the locations we visited (six of the seven field locations from which we selected contracts). On the basis of our checks, we determined that these data were sufficiently reliable for identifying DOE locations with a large share of contracts.

To determine whether DOE adhered to its planned dates for awarding the contracts in our nonprobability sample, we used contract file documents, obtained from the seven locations, to identify and then compare the planned and actual contract award dates. We considered contracts to be delayed if the actual award date was later than the planned award date. In most cases, we identified the planned award date in DOE’s schedule for completing key steps of the contract award process that DOE included in its formal written plan for awarding the contract. Because DOE guidance generally requires a plan and schedule when awarding contracts valued at $5 million or greater, the schedules in the written plan generally provided a consistent source for the planned award date. However, for 9 of the 31 contracts in our sample, we had to obtain the planned award date from another source because the associated plan and/or schedule was missing from the contract file. In these cases, we obtained the planned award date from references in other required file documents, such as the document used to justify noncompetitive awards or presentations by officials involved in awarding the contract. Similarly, for nearly all of the 31 contracts in our sample, we identified the actual award date from the award date written on the signed contract. However, for 2 of the 31 contracts, the signed contract was not available in the contract file, and we instead obtained the award date from the data system DOE uses to track its contract awards.

In addition to determining whether the 31 contracts in our nonprobability sample were awarded by the planned date, we used contract file documents to calculate how long it took DOE to make the awards. We did so by calculating the elapsed time from the date DOE initiated the contract award process to the date it awarded the contract. For 23 contracts, we used the date of the formal written plan for carrying out the contract award process as the date DOE initiated the process. Although some planning steps typically preceded the preparation of the written plan, documentation of the steps was generally not consistently available in the contract files we reviewed. In contrast, the formal written plan was a required and more consistently available source. However, for 8 of the 31 contracts in our sample, we had to obtain the initiation date of the contract award process from a source other than the date of the written plan because the plan was undated or missing from the contract file. In these cases, we used contract file documents prepared during the early steps of the contract award process, such as correspondence from DOE officials approving the start of the process.

Furthermore, we identified some of the reasons for delays DOE experienced in awarding contracts in our sample, in part, by contacting DOE officials involved in awarding the contract. We also reviewed contract file documents, such as meeting records and the descriptions of the contract award process included in key documents generated during the contract award process. When contract file documents were used to determine the causes of delays, two analysts involved in this study independently reviewed the documents to verify that the causes were reasonably identified.

To further identify causes for delays, we analyzed documentation from bid protests filed with GAO in fiscal years 2002 through 2005, in addition to the bid protest guidance issued by GAO’s Office of General Counsel. We analyzed this documentation because it could potentially describe problems encountered by DOE during the contract award process. In correcting any problems, DOE may have repeated steps in the process, eventually delaying the contract award. When bid protest documents were used to determine whether (1) the contract award being protested was also in our sample of contracts that we analyzed for delays or (2) corrective action was agreed to by DOE or recommended by GAO as a result of the protest, two analysts involved in this study independently reviewed the documents to verify that the causes were reasonably identified.

To determine overall impacts of any delays in awarding DOE’s contracts, as well as the extent to which the department has taken steps to address the delays, we interviewed headquarters contracting officials from the Office of Procurement and Assistance Management, as well as contracting and program officials from the department’s three largest DOE component organizations, and officials from the Office of Engineering and Construction Management, who oversee planning and management of capital projects. Also, we interviewed contracting and program officials at six of the seven field locations from which we selected contracts for our nonprobability sample as follows: Albuquerque Service Center (NNSA), New Mexico; Idaho Operations Office (EM and Nuclear Energy), Idaho; Oak Ridge Operations Office (EM, Science, NNSA), Tennessee; Portsmouth and Paducah Project Office (EM), Lexington, Kentucky; Richland Operations Office (EM), Washington; and Savannah River Operations Office (EM and NNSA), South Carolina.

Furthermore, for contracts in our sample, we analyzed file documents for evidence of adverse impacts upon companies competing for the work resulting from any delays in carrying out the contract award process. We examined transcripts from some debriefing sessions that DOE held with companies after awarding the contract, as well as other documents, such as correspondence from companies or their representatives. To further understand the impacts of delays, we also examined documents related to DOE’s contract award for environmental cleanup of its Paducah, Kentucky, site, even though this contract was not part of our sample. DOE’s Office of Procurement and Assistance Management provided us with its analysis of lessons learned and the factors that delayed this contract award. We reviewed the contract file documents to better understand the context of this analysis. However, we excluded the contract from our sample because DOE awarded it in fiscal year 2006 and, therefore, it did not fit our selection criteria. As a result, we did not include the contract in calculations or figures showing the extent of delays in awarding DOE contracts.

In addition, we reviewed federal and DOE regulations that govern contracting, DOE policy and guidance on contracting, and documents on the department’s assessment of its performance in carrying out its contract awards. We also reviewed GAO’s Framework for Assessing the Acquisition Function at Federal Agencies for determining the effectiveness of DOE’s management and oversight of the contract award process.

We conducted our work from July 2005 to June 2006 in accordance with generally accepted government auditing standards.

Comments from the Department of Energy

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Bill Swick, Assistant Director; Carole Blackwell, Nora Boretti, Kevin Jackson, Greg Marchand, Alison O’Neill, Jeff Rueckhaus, and Gretchen Snoey made key contributions to this report.