National Laboratories: DOE Needs to Improve Oversight of Work Performed for Non-DOE Entities
Published: 2013-10-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOE's 17 national laboratories house cutting-edge scientific facilities and equipment, ranging from high-performance computers to ultra-bright X-ray sources for investigating fundamental properties of materials. DOE allows the capabilities of the laboratories to be made available to perform work for other federal agencies and nonfederal entities through its WFO program, provided that the work does not hinder DOE's mission or compete with the private sector, among other things. GAO was asked to review the WFO program. GAO examined (1) the amount and type of work conducted under the program, (2) the extent to which DOE has ensured that WFO program requirements are met, and (3) the extent to which program performance is measured against WFO program objectives. GAO reviewed DOE and laboratory data and documents, internal and external review reports, and interviewed officials from DOE and the laboratories.
section_title: What GAO Found
paragraphs: In fiscal years 2008 through 2012, the Department of Energy (DOE) performed about $2 billion annually of Work for Others (WFO) projects, as measured by the costs incurred. Although the amount of WFO performed has remained relatively constant over the last 5 years overall, WFO as a percentage of the total work performed at the laboratories--measured in total laboratory costs incurred--has declined from 17 percent in fiscal year 2008 to about 13 percent in fiscal year 2012. In fiscal year 2012, the WFO program included more than 6,500 projects. About 88 percent of this work was for other federal agencies, with the majority of it performed for the Department of Defense. For example, one project for the Army applies a laboratory's expertise in laser decontamination of surfaces to develop a system that will remove chemical agent residues from equipment. The remaining WFO work was sponsored by nonfederal entities, including state and local governments, universities, private industry, and foreign entities.
DOE officials have not ensured that WFO program requirements are consistently met. For example, a DOE official is required to determine whether a proposed WFO project has met DOE requirements for accepting work before approving, or certifying, the work and this responsibility may not be delegated to the laboratories. However, DOE officials from site offices at 8 of the 17 laboratories reported that these determinations were made by the laboratories and that the DOE officials did not take steps to independently verify the determinations prior to approving the work. DOE also cannot ensure that the full costs of materials and services for WFO projects are charged to sponsors because 12 of 17 laboratories have limited or no written procedures for developing WFO project budgets or charging costs to ongoing projects, two important steps for recovering the full costs of materials and services. A 2013 DOE Office of Inspector General report found that the costs of administering WFO projects at one laboratory were allocated to DOE projects, resulting in an estimated $400,000 in WFO project costs that were not reimbursed to the laboratory. DOE requires that its program offices annually review the WFO program at each of its laboratories. However, DOE requirements do not specify what the reviews should include, and DOE program offices varied in what they consider to be an annual review. DOE also requires the department's Chief Financial Officer to report annually on the activities conducted under the WFO program, but DOE officials told GAO that they no longer produce the report because the requirement is outdated, choosing instead to fulfill data requests on a case-by-case basis. As a result, DOE does not have data that are comparable across laboratories or over time.
DOE has not measured the extent to which WFO program performance is measured against program objectives and has not established performance measures to do so. Some DOE site offices and laboratories have taken steps to evaluate the performance of the WFO program, but these steps are not consistent across the laboratories, do not incorporate key attributes of successful performance measures, and do not address the WFO program objectives. Recent internal and external reviews of the laboratories have recommended that DOE establish clear measures to evaluate laboratory WFO program performance against the WFO program objectives. DOE officials told GAO that they do not believe it is appropriate to develop one set of measures for all laboratories and that they do not plan to do so.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that DOE take steps to ensure compliance with project approval requirements; require laboratories to establish written procedures for charging costs to projects; specify what the annual program reviews should include; produce annual reports on WFO activities; and establish performance measures for the WFO program. DOE generally agreed with the report and its recommendations.

section_title: Letter
paragraphs: Since the 1940s, the Department of Energy’s (DOE) national laboratories  and specialized research facilities have played a central role in advancing  research in physics and other basic and applied sciences. As one of the  largest research agencies in the federal government, DOE spends billions  of dollars each year on publicly funded research to support its diverse  missions, including advancing scientific research and technology  development, ensuring efficient and secure energy, and ensuring the  safety and reliability of the nation’s nuclear weapons stockpile. In fiscal  year 2012, from its appropriations DOE allocated nearly $11 billion to its  17 national laboratories. These laboratories house cutting-edge scientific  facilities and equipment, ranging from high-performance computers to  ultra-bright X-ray sources for investigating fundamental properties of  materials. The majority of the research and development funded by DOE  is carried out under the department’s direction and overseen by scientists,  engineers, and others employed at the 17 national laboratories, 16 of  which are managed and operated by contractors under management and  operating (M&O) contracts, and 1 of which DOE directly manages and  operates.and nonprofit organizations.
The M&O contractors include universities, private companies,  To allow non-DOE entities to benefit from the significant public investment  in DOE’s laboratories and from the unique and highly specialized facilities  and scientific and technical expertise at these laboratories, DOE operates  the Work for Others (WFO) program. This program allows work to be  performed at the laboratories for both federal agencies and nonfederal  entities, provided that several requirements are met, including that WFO  projects do not hinder DOE’s mission or compete with the private sector.  Several laws provide the foundation for DOE’s WFO program. The  Economy Act of 1932 allows federal agencies to obtain goods and  services from other agencies as long as the work cannot be performed as  conveniently or cheaply by the private sector and is in the best interests  of the federal government. DOE the authority to perform work for other federal agencies, as well as  for state and local governments, private-sector organizations, academic  institutions, and foreign entities, if private-sector facilities or laboratories  are not adequate for the purpose. In addition, the Energy Reorganization  Act of 1974 gives the Nuclear Regulatory Commission special access to  DOE research and development capabilities.
The Atomic Energy Act of 1954 also provides  DOE has established program and financial management policies,  requirements, and procedures for WFO that are contained in several DOE  directives. DOE Order 481.1C establishes WFO management  requirements, program roles and responsibilities, as well as defines the  program’s objectives. laboratories—including National Nuclear Security Agency (NNSA)
31 U.S.C. § 1535.
DOE, Work for Others (Non-Department of Energy Funded Work), DOE Order 481.1C  (Washington, D.C.: Mar. 14, 2011). laboratoriesfederal agencies and nonfederal entities and avoid duplication of effort at  federal facilities, to provide access for non-DOE entities to DOE’s unique  and highly specialized facilities when private-sector facilities are  inadequate, to increase the research and development interaction  between the laboratories and industry, and to maintain core  competencies and enhance the science and technology base at the  laboratories. DOE’s WFO order also establishes requirements for project  selection, program reviews, and annual reporting on WFO activities  performed by DOE. According to the order, WFO projects must meet the  following criteria:  —to accomplish goals that may be otherwise unattainable by be consistent with or complementary to the missions of DOE and the  facility to which the work is to be assigned; avoid adverse impact to programs assigned to the facility; not place the facility in direct competition with the domestic private not create a detrimental future burden on DOE resources.
DOE Order 522.1 requires that the entity requesting the work reimburse  the laboratory for the full cost of the work—including labor, materials,  overhead, and other costs—plus a federal administrative charge, unless  waived under departmental guidelines.
In a time of uncertainty over future federal budgets and calls to reduce  spending, DOE may have difficulty sustaining its current laboratory  structure. Under budget pressure, DOE’s Office of Inspector General  officials and others have observed that the laboratories could become  dependent on reimbursements received under the WFO program to  maintain the laboratories’ current facilities. In this context, you asked us to  review the WFO program at DOE’s laboratories. Our objectives were to  determine (1) the amount and type of work conducted under DOE’s WFO  program, (2) the extent to which DOE has ensured that WFO program  requirements are met, and (3) the extent to which DOE and the  laboratories have measured WFO program performance against WFO  program objectives.
To determine the amount and type, or sponsors, of the work conducted  under DOE’s WFO program, we reviewed DOE data on the amount of  work performed under the WFO program in total and by laboratory for  fiscal years 2008 through 2012, as measured by costs incurred for WFO  projects. We also obtained total costs incurred for each laboratory for  fiscal years 2008 through 2012. To assess the reliability of these data, we  interviewed DOE officials who oversee them for DOE’s Office of the Chief  Financial Officer (CFO) to understand how the data are collected and  reported. We also reviewed recent external audits of DOE’s financial  reporting system and financial data. We determined that the data were  sufficiently reliable to describe the amount of WFO activities performed at  the laboratories. To determine the amount of work conducted by federal  and nonfederal sponsors under the program, we gathered data on the  costs of WFO activities performed for federal versus nonfederal sponsors.  In addition, we interviewed DOE headquarters and site officials and  officials for the 17 laboratories to gather examples of projects performed  for federal and nonfederal sponsors. We visited 3 laboratories and  contacted the remaining 14 laboratories by telephone. To determine the  extent to which DOE has ensured that WFO program requirements are  met, we reviewed federal regulations; DOE requirements; and DOE and  laboratories’ policies and procedures governing the WFO program,  including those for approval of WFO projects; cost recovery for WFO  projects (i.e., development of project budgets and charging of costs); and  reports of reviews of WFO programs. To determine the extent to which  DOE and the laboratories have measured WFO project performance  against program objectives, we reviewed DOE orders; performance  requirements for WFO projects, if any, in DOE contracts with laboratories;  DOE evaluations of laboratories’ performance; and laboratories’ strategic  plans and procedures for establishing and executing WFO projects. We  discussed laboratories’ strategies and goals for the WFO program and  WFO program performance measurement with DOE headquarters and  site officials and officials from each of the laboratories. See appendix I for  a more detailed description of our scope and methodology.
We conducted this performance audit from August 2012 to October 2013,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DOE program offices manage the department’s 17 national laboratories  and support the department’s diverse missions, as follows:
The Office of Science oversees 10 national laboratories and, for fiscal  year 2012, received appropriations of more than $4.3 billion to  operate these laboratories. The Office of Science is the nation’s single  largest funding source for basic research in the physical sciences,  supporting research in energy sciences, advanced scientific  computing, and other fields.
NNSA oversees 3 national laboratories and, for fiscal year 2012,  received appropriations of more than $4.6 billion to operate these  laboratories. NNSA helps support understanding of the physics  associated with the safety, security, and reliability of nuclear weapons  and maintains core competencies in nuclear weapons science,  technology, and engineering.
The Office of Nuclear Energy oversees 1 laboratory and received  appropriations for fiscal year 2012 totaling more than $1 billion to  operate this laboratory. The primary mission of the Office of Nuclear  Energy is to advance nuclear power as a resource capable of meeting  the nation’s energy, environmental, and national security needs by  resolving technical, cost, safety, proliferation resistance, and security  barriers.
The Office of Fossil Energy oversees 1 laboratory and received  appropriations for fiscal year 2012 totaling more than $551 million to  operate this laboratory. The Office of Fossil Energy’s primary mission  is to ensure reliable fossil energy resources for clean, secure, and  affordable energy while enhancing environmental protection.
The Office of Energy Efficiency and Renewable Energy oversees 1  laboratory and received appropriations for fiscal year 2012 totaling  over $271 million to operate this laboratory. The Office of Energy  Efficiency and Renewable Energy’s mission is to develop solutions for  energy-saving homes, buildings, and manufacturing; sustainable  transportation; and renewable electricity generation.
The Office of Environmental Management oversees 1 laboratory and,  for fiscal year 2012, received appropriations of about $7.6 million to  operate this laboratory. The Office of Environmental Management is  responsible for cleaning up hazardous wastes left from decades of  nuclear weapons research and production.
See figure 1 for the locations of the laboratories managed by the various  program offices. DOE also maintains individual site offices that provide  federal oversight at 16 of the 17 laboratories. In addition, DOE’s field  CFOs are responsible for overseeing financial activities at each location.For a complete list of DOE’s national laboratories and more information  on each, see appendix II.
DOE’s WFO program aims to provide benefits to organizations such as  U.S. companies or academic institutions that have work performed at the  laboratories and to the national laboratories as well. For example,  according to a 2011 DOE report to Congress, the WFO program helps  to leverage DOE investment in the laboratories by further developing  technical expertise for accomplishing critical tasks needed to fulfill DOE’s  mission priorities. Furthermore, according to the DOE report, the WFO  program can also help the national laboratories to retain highly trained  scientists and engineers in support of DOE and national priorities by, for  example, providing opportunities for them to stay engaged during times  when mission critical work has slowed.
According to officials from DOE and laboratories and DOE  documentation, a potential WFO project begins when an entity desiring to  have work performed becomes aware of the capabilities available at a  laboratory. Entities can become aware of these capabilities in a number  of different ways, including as a result of networking by laboratory  researchers and scientists, by responding to funding opportunity  announcements of other federal agencies, or through information posted  on the laboratories’ public websites. Once a potential project has been  identified, representatives of the entity (i.e., the sponsor) and the  laboratory begin to negotiate the work to be performed and the estimated  costs. When the laboratory and sponsor have agreed to the terms of  work, DOE site officials review the proposed WFO agreement. Finally, if  the terms are approved by DOE and the sponsor has provided  certification of funding or advance payment for the work, a DOE  contracting officer certifies that the requirements have been met for the  laboratory to conduct the work.
DOE’s WFO order establishes DOE policy and requirements for  accepting, authorizing, and administering such work. This order applies to  work for all non-DOE entities except the Department of Homeland  Security (DHS), which is covered by a separate order, and is not  considered to be WFO work. In addition, under the WFO order, each  site office is responsible for establishing its own procedures and  processes for the review and approval of work performed under WFO  agreements and for conducting periodic reviews of laboratory policies and  procedures for negotiating and administering WFO projects.  Requirements for establishing prices and charges for materials and  services sold or provided to outside entities either directly or through the  department’s M&O contracts are established through DOE’s pricing order,  including for the WFO program.
section_title: DOE Has Performed About $2 Billion of Work under the WFO Program Annually, but the Amount and Sponsors of Work Varied Among Laboratories
paragraphs: The total amount of work performed under the WFO program, as  measured by costs incurred for WFO projects, has remained relatively  constant over the last 5 fiscal years overall, but the amount of WFO work  performed and the sponsors of the work varied widely among the  laboratories.
subsections: section_title: Amount of Work Performed under the WFO Program Has Been About $2 Billion Annually
paragraphs: In fiscal years 2008 through 2012, DOE performed about $2 billion of  work annually under the WFO program, as measured by costs incurred  (see fig. 2). Although the amount of work performed under the WFO  program has remained relatively constant over the last 5 years, it has  declined slightly relative to total work performed at the laboratories during  this period. From fiscal year 2008 through fiscal year 2011, total work  performed at the laboratories increased from $12.0 billion to $17.1 billion  and fell to $16.3 billion in fiscal year 2012. As a result, the proportion of  WFO performed as a percentage of total work performed declined from  17 percent in fiscal year 2008 to about 13 percent in fiscal year 2012. In  fiscal year 2012, more than 6,500 WFO projects were carried out at  DOE’s laboratories.
During the period we reviewed, each of DOE’s 17 national laboratories  performed some work on WFO projects, with some laboratories involved  in significantly more WFO activities than others. In fiscal year 2012, the  amount of work performed by the laboratories on WFO projects ranged  from about $1.5 million at the National Energy Technology Laboratory to  over $803 million at the Sandia National Laboratories. The proportion of  WFO activities relative to all work at the laboratories also varied widely.  Specifically, work performed on WFO projects as a percentage of total  work performed at the laboratories ranged from less than 1 percent at the  National Energy Technology Laboratory to nearly 33 percent at the  Sandia National Laboratories (see table 1).
According to DOE officials, the variation in the amount of WFO performed  is due in large part to differences in the core mission capabilities of each  laboratory. For example, Sandia National Laboratories has extensive  expertise in systems engineering, a capability that is heavily utilized by  other federal agencies. According to the officials, other laboratories’  capabilities are less in demand and, therefore, less WFO work is  performed at these laboratories.
section_title: Type, or Sponsors, of WFO Projects Performed Varied by Laboratory
paragraphs: DOE’s laboratories carried out a variety of WFO projects for many  different sponsors, but the majority of the work was for other federal  agencies—in particular, the Department of Defense (DOD). Of the $2.1  billion in work performed on WFO projects in fiscal year 2012, over $1.8  billion, or about 88 percent, was for other federal agencies, with DOD  sponsoring about $1.5 billion, or 71 percent of the work performed (see  fig. 3). The majority of the work performed under the program for DOD for  fiscal year 2012 was carried out at six national laboratories—Idaho,  Lawrence Livermore, Los Alamos, Oak Ridge, Pacific Northwest, and  Sandia. The type of work sponsored by DOD included a variety of  projects. For example, Pacific Northwest National Laboratory has  developed a software technology for the Air Force that performs, among  other things, automatic topical analysis and organization of large  collections of documents graphically. The laboratory is training the Air  Force on applying the software to a diverse set of Air Force data to help  analysts extract information to better identify potential uses for emerging  technologies. In another example, the Idaho National Laboratory has  applied its expertise in the area of laser decontamination of surfaces to  develop and deploy a laser cleaning system for the Army. The objective  of the project is to develop a system to, among other things, remove  chemical agent residues from contaminated surfaces and equipment so  that equipment can be reused, reduce personnel exposure risks, and  reduce secondary waste streams.
Other federal agencies also sponsored a variety of WFO projects in fiscal  year 2012, such as climate change and energy efficiency research  conducted at the Lawrence Berkeley National Laboratory for the  Environmental Protection Agency, and plutonium production research  conducted at the Idaho National Laboratory for the National Aeronautics  and Space Administration’s (NASA) space exploration program. For more  information on the amount and type of work carried out at each  laboratory, see appendix III.
Nonfederal entities sponsored $251.5 million, or 12 percent, of the WFO  performed at the laboratories in fiscal year 2012. Sponsors and types of  projects included the following:
State and local governments. For example, officials in Sonoma  County, California, entered into an agreement with the Lawrence  Berkeley National Laboratory to conduct research on local rivers and  dams to better understand the natural riverbed processes that occur  as a function of dam and pumping operations, including sedimentation  and evolution of biomass. These processes can lead to clogging of  the riverbed, which in turn can limit the ability to pump water from  beneath the riverbed as is needed for subsequent distribution as  drinking water in the county. Under the current phase of this WFO  project, the laboratory is using its hydrological, geochemical, and  biological tools and expertise to quantify the riverbed clogging  mechanisms and to represent them in a computer model.
Colleges and universities. For example, researchers at the University  of Chicago studied the structures of proteins and, in particular, how  proteins in lung cancer patients are affected by certain drugs. These  researchers set up a WFO agreement with Argonne National  Laboratory, under which the laboratory drew upon its extensive  protein sequence database and its research capabilities to assist the  university in analyzing, modeling, detecting, and characterizing  proteins in lung cells. Under this agreement, Argonne National  Laboratory also updated and maintained a database of protein  structures that will be accessible to the broad biology community.
Private industry. For example, GE Global Research, a private  research laboratory, has entered into a WFO agreement with  Lawrence Livermore National Laboratory to help refine wind prediction  and control capabilities at wind farms. Under this agreement, the  laboratory is evaluating high-resolution wind farm modeling tools in  natural terrain using high-performance computers to run wind  simulation and forecasting models.
Foreign entities. For example, following the events at the Fukushima  Daiichi Nuclear Power Station resulting from the earthquake and  tsunami in March 2011, Tokyo Electric Power Company (TEPCO)  entered into a WFO agreement with the Savannah River National  Laboratory and the Pacific Northwest National Laboratory. The  laboratories were to evaluate and define the scope of work for TEPCO  to accomplish a number of tasks including the prevention of  underground water contamination and the treatment and disposal of  nuclear waste. The laboratories each drew upon their expertise in  these areas to develop a schedule for how the work could proceed  and to identify the necessary equipment, materials, facilities,  personnel, and the costs to perform the work.
section_title: DOE Has Not Ensured That WFO Program Requirements Are Consistently Met
paragraphs: DOE has not ensured that WFO program requirements are consistently  met. Specifically, DOE has not ensured compliance with requirements for  the approval of WFO projects, cost recovery, program reviews, and  annual reporting.
subsections: section_title: DOE Has Relied on Laboratories’ Determinations That WFO Projects Meet Requirements for Approval
paragraphs: According to DOE’s WFO order, WFO projects must meet specific DOE  requirements, including being consistent with or complementary to DOE’s  missions and those of the laboratories, not hindering those missions, and  not placing the laboratories in direct competition with the domestic private  sector. A DOE contracting officer or other authorized DOE designee is  required to determine whether a proposed WFO project has met all of  these requirements before approving or certifying the work. These  determinations may not be delegated to the laboratories. However, DOE  officials from site offices at 8 of the 17 laboratories told us that the  laboratories provide written justification in the WFO package to support,  or determine, that some or all of the requirements are met and that the  DOE officials have often accepted the laboratories’ determinations  without taking steps to independently verify them. DOE officials cited  various reasons for relying on the laboratories to make the  determinations. For example, an official from one site office told us that  he relies on the laboratory’s determination that WFO projects are  consistent with the mission. He explained that he does not believe that  the laboratory would accept work that would be inconsistent with its  mission. Similarly, an official at another DOE site office explained that she  relies on the laboratory’s determination that the work cannot be  performed by the private sector because she believes the laboratory staff  are better informed about the capabilities available in the private sector.  By relying on the laboratories’ determinations without taking steps to  independently verify the information, DOE does not have assurances that  the WFO projects selected meet DOE’s requirements. The DOE Office of  Inspector General has identified the WFO program as a priority area and  is reviewing laboratories with major WFO programs to determine whether  they meet internal control and compliance requirements established by  DOE.
section_title: DOE Has Not Ensured That Cost Recovery Procedures for WFO Projects Are in Place
paragraphs: DOE’s pricing order requires that the department charges for the full cost  of materials and services provided to external organizations, including the  amounts charged for work under the WFO program. DOE has not  ensured, however, that all laboratories have formal, written procedures for  developing WFO project budgets or charging costs to ongoing projects,  two important steps for recovering the full costs of materials and services  provided.procedures for developing WFO project budgets and charging costs to  projects that include, among other things, detailed instructions on the  types of costs to include in a WFO project budget and specific instructions  for calculating the costs and for ensuring that the costs of the work are  charged to the sponsor. However, while the remaining laboratories did  provide a description of their WFO budget development and cost charging  processes, five of these laboratories had limited written procedures or  tools in place for these processes. For example, one laboratory had a  template that could be used to prepare a WFO budget but did not have  detailed, written requirements or procedures for using this tool. The  remaining seven laboratories did not provide any formal, written  procedures to guide the development of WFO project budgets or the  charging of costs. Without detailed, written guidance, DOE may not be  able to ensure that its cost recovery requirements are consistently met.
Specifically, five laboratories have detailed, written  In addition, DOE field CFOs do not always review costs charged to WFO  projects in accordance with DOE’s pricing order, which requires that  DOE’s field CFOs conduct biennial reviews of the pricing of materials and  services and other costs charged to WFO projects at the laboratories.  Under the pricing order, the reviews are required to include steps to  ensure that (1) prices charged conform to the requirements of OMB  Circular A-25 and departmental pricing policy or other legislative  authority, as applicable; (2) adequate documentation exists for prices  established for materials and services; and (3) exceptions to the full cost  recovery requirements were limited to those authorized in the order. The  reviews are intended to provide assurance to the department’s CFO that  the full cost of the work, including all applicable direct and indirect costs,  is charged to the sponsor of each WFO project. We requested DOE’s  most recent biennial review reports of WFO project pricing for each of the  17 laboratories. Reviews for 16 of the 17 laboratories were provided.  These reviews were conducted by the seven field CFOs that oversee the  laboratories and covered fiscal years 2010 and 2011. We found that 6 of  the 16 reviews did not include steps to ensure that prices for WFO  projects conformed to DOE requirements. For example, in order to ensure  that adequate documentation exists for prices established for materials  and services charged to sponsors of WFO projects, reviewers need to  examine pricing documentation for a sample of WFO projects. Reports for  6 reviews indicated that, for a sample of WFO projects, pricing  documentation for overhead and administrative costs was examined, but  the reports did not provide evidence that documentation to support direct  costs—such as labor and materials—was examined. Furthermore, one  biennial pricing review was conducted by the laboratory instead of by the  DOE field CFO, as required, and no details on the review steps were  provided in the report. Reports for 9 of the 16 reviews indicated that a  sample of WFO projects was reviewed. Reviewing a sample of WFO  projects can provide useful information, such as identifying errors in costs  charged to those projects. For example, 2 of the reviews that included an  examination of a sample of WFO project documentation identified errors  in general and administrative costs charged to WFO projects that resulted  in either undercharges or in overcharges to the sponsor. In the case of  undercharges, DOE paid part of the sponsor’s cost of the WFO project. In  the case of overcharges, the sponsor subsidized a portion of DOE’s  mission work. The DOE Office of Inspector General has also identified  errors related to the charging of general and administrative costs to WFO  projects in their audits of WFO programs at the laboratories. For example,  a 2013 review at Lawrence Berkeley National Laboratory found that costs  of administering WFO projects were allocated in part to other DOE  projects, resulting in estimated $400,000 in project costs that the sponsor  did not reimburse to the laboratory.
section_title: DOE Did Not Consistently Conduct Required Annual WFO Program Reviews
paragraphs: DOE’s WFO order requires each DOE program office to annually review  the WFO program at each of its laboratories to ensure compliance with  WFO policies and procedures. The order does not specify what the  reviews should include. As a result, the program offices varied in what  they consider to be their annual review. For example, DOE Office of  Science program officials explained that they believe their annual WFO  planning efforts with the laboratories fulfill the program review  requirements in the WFO order. Specifically, each of the Office of Science  laboratories develops a plan that contains a section that summarizes the  laboratory’s WFO portfolio and discusses the WFO strategy for the future.  DOE Office of Science officials told us that, as part of this process, each  federal site office manager provides a review of the laboratory’s ongoing  WFO program operations and proposed WFO funding level that includes  a statement about the adequacy of the laboratory’s management and  oversight of WFO activities. However, these plans were primarily focused  on planning for WFO project work to be performed in the future and were  not reviews to ensure compliance with WFO policies and procedures.  NNSA officials told us that they had performed annual reviews of their  WFO program since 2008; however, in response to our request for  information, they provided one briefing from 2012 that focused on  improving the WFO agreement processing times.
Program officials also conducted reviews of WFO policies and procedures  at 6 of the 16 site offices. However, officials from the Office of Science  told us that these reviews were not conducted to satisfy the annual review  requirement in the WFO order because they focused on the site offices’  WFO policies and procedures rather than laboratories’ policies and  procedures. Specifically, as of May 2013, Office of Science program  officials conducted a total of six reviews—one at each of 6 site offices— since 2008. These generally consisted of a review of WFO program policy  and procedure documents at the site offices including flow charts, forms,  and correspondence. Each of the six reviews identified areas of concern  within the WFO programs. For example, one program office review  reported that the laboratory was dependent on funds from a single WFO  project to support a major DOE program. The review cautioned that if the  WFO project was discontinued, it could create a detrimental financial  burden on the laboratory in the future, which is not consistent with DOE’s  requirement to avoid doing so. In addition, four of the reviews found  deficiencies related to WFO program documentation or procedures. For  example, one review reported that a DOE site office WFO program  procedural guide was out of date and did not reflect all relevant DOE  policy requirements or current DOE site office operating procedures.  Although the reviews were not conducted to satisfy the annual review  requirement in the WFO order, the reviews appeared to include useful  information about the WFO program that was not included in the annual  laboratory planning processes, which were focused on future WFO  efforts. No program office reviews were conducted at the other 10 site  offices.
section_title: DOE Does Not Produce the Required Annual Summary Report on Project Activities
paragraphs: DOE’s WFO order also requires that the DOE headquarters CFO prepare  an annual summary report of WFO activities performed at its laboratories.  DOE headquarters officials told us they have not produced this report in  the past several years because the requirement to produce it is an  outdated requirement that was put into place to facilitate data collection  before the implementation of DOE’s current financial system. The officials  said that they plan to eliminate this requirement. They also said that they  have made better use of their limited program resources by choosing to  fulfill requests for WFO project data from Congress and others on a case- by-case basis. For example, in the Conference Report to accompany  the Energy and Water Development and Related Agencies Appropriations  Act of 2010, DOE was directed to submit an annual report on the status  of its WFO activities. DOE officials provided the requested information on  WFO activities; however, this information might have already been  available if DOE had prepared the annual summary report as required by  the WFO order. In addition, other DOE programs regularly need and  collect WFO project data that could be provided by the annual summary  report if it had been prepared in accordance with the order. For example,  DOE’s technology transfer coordinator has been collecting data on  nonfederal WFO project activities from each of the laboratories since  2001 for inclusion in the Department of Commerce’s annual report on  technology transfer. Choosing to report data on a case-by-case basis,  rather than in an annual report, may make it difficult for those providing  oversight and for some users of the data. This is because the data are not  readily available, requiring DOE to generate it, which is time-consuming  and, because depending on how the data are generated, they may not be  comparable across laboratories or over time.
section_title: DOE Has Not Measured WFO Program Performance against Established Objectives
paragraphs: DOE has not measured the extent to which WFO program objectives are  being met, even though DOE site offices are required under the WFO  order to measure their laboratories’ WFO program performance. Some  DOE site offices and laboratories have taken steps to evaluate WFO  program processes, but these steps are not consistent across the  laboratories, generally do not address the program objectives in the WFO  order, and do not incorporate key attributes of successful performance  measures.
DOE’s WFO order requires that DOE site offices establish performance  goals and measures to assess field performance of the WFO program at  the laboratories they oversee, including the effectiveness and impact of  WFO program processes and improvements.President directed each agency with a federal laboratory to establish   Moreover, in 2011, the  performance goals, measures, and evaluation methods related to  technology transfer. Although the presidential memo does not  specifically mention the WFO program, an objective of the program is to  transfer technology originating at DOE facilities to industry for further  development or commercialization. We found that, although some DOE  site offices and laboratories have taken steps to evaluate the  performance of the WFO program, these steps do not directly address the  WFO program objectives. Moreover, DOE site offices’ and laboratories’  efforts to evaluate the WFO programs are focused on reviewing  processes; including tracking the number of WFO agreements and  improving timeliness of project selection and approval, rather than  developing goals and measures for assessing performance against WFO  program objectives.
According to our discussions with officials from DOE headquarters and  site offices and laboratory representatives, efforts to evaluate the  performance of the WFO program at the laboratories include the  following:
Assessing customer satisfaction or other qualitative measures.
Officials from 4 of the 17 laboratories told us that they have some  mechanisms in place to collect qualitative information about WFO  projects, such as customer satisfaction. This information is shared  with DOE site officials. For example, officials at one laboratory send  surveys to sponsors to assess customer satisfaction after the  completion of WFO projects. Another laboratory reports to DOE on  success stories about WFO projects once they are complete.
Tracking the number of WFO agreements. Officials from 6 of the 17  laboratories told us that they track the amount of or have set targets  related to the number of WFO agreements in place.
Evaluating WFO agreement processing times. Officials from 6 of the  17 laboratories told us that they have goals to streamline the WFO  agreement process, as measured by the time it takes from initiation to  approval of a WFO agreement. In addition, in 2009 DOE  commissioned a study of the time it takes to process WFO  agreements across the laboratories and identified best practices for  streamlining that process. Moreover, in 2011 NNSA commissioned a  similar study of processing time for WFO and other interagency  agreements at the 3 laboratories that it oversees.
We have previously reported on the nine attributes most often associated  with successful performance measures, which are summarized in table  2.
Our analysis shows, however, that the steps DOE and the laboratories  have taken to evaluate performance did not include some of these key  attributes. For example, while customer satisfaction surveys or other  qualitative measures, such as success stories gathered by the  laboratories, may provide some useful information and indicate areas for  improvement, customer satisfaction is not included in the WFO order as a  program goal. Furthermore, the performance measures do not directly  link with the WFO program goals or objectives such as providing access  to DOE laboratories to accomplish goals that may be otherwise  unattainable by federal agencies and nonfederal entities. Without such  linkage, DOE and decision makers may not have the needed information  to track the program’s progress in meeting its objectives. Additionally,  some WFO qualitative measures such as customer satisfaction may lack  clarity and a measurable target, making it difficult to compare  performance across laboratories. These types of qualitative measures  also may not meet the key attribute of objectivity due to the potential for  bias or other manipulation, depending on how the information is gathered  and assessed. Other efforts to measure the performance of the  program—specifically, the number of WFO agreements in place and  WFO agreement processing time—both provide some helpful information  but do not include all key attributes of successful performance measures.  For example, tracking the number of agreements is clear and measurable  and provides some information about the number of WFO projects at a  laboratory. However, without linkage to the program’s objectives,  measuring the number of agreements in place does not capture the  program’s effectiveness in meeting the program’s objectives laid out in  the WFO order, such as maintaining core competencies and enhancing  the science and technology base at the laboratories.
As we have reported,the ability to track the progress they are making toward their mission and  objectives. We also have found that performance measures can create  powerful incentives to influence organizational and individual behavior.  Our analysis indicates that site office and laboratory performance  measures lack these key attributes. In addition, DOE headquarters  officials told us that there are no performance measures to assess the  WFO program’s performance against WFO program objectives across all  laboratories. The officials said that this is because the WFO program is  decentralized, and laboratories are managed individually.   performance measures provide organizations with  Recent external and internal reviews of the DOE laboratories have  recommended that clear performance measures are needed to measure  laboratory WFO program performance against the WFO program  objectives. In January 2013, the National Academy of Public  Administration (NAPA) reported that while DOE officials at the  laboratories have measures to assess DOE funded work; these measures  do not always apply to non-DOE funded work such as work performed   NAPA also noted that DOE’s decentralized  under the WFO program. approach to managing the WFO program raised questions about DOE’s  ability to oversee the program as a whole. NAPA recommended that DOE  include measures for WFO work performed in its evaluations of laboratory  performance. Similarly, in 2012, a working group set up by DOE  headquarters to oversee efforts at the laboratories to share technology  with non-DOE entities, reported concerns that measures did not exist to  evaluate the impact and success of WFO work activities in achieving  program objectives. DOE officials told us that, because WFO agreements  are unique to each laboratory, they do not believe it is appropriate to  develop one set of measures for all laboratories and that they have no  plans to do so. However, without measures that apply to all laboratories, it  is difficult to compare performance across laboratories in meeting overall  program objectives.
NAPA, U.S. Department of Energy: Positioning DOE’s Labs for the Future: A Review of  DOE’s Management and Oversight of the National Laboratories (Washington, D.C.:  January 2013). NAPA was established in 1967 and chartered by Congress as an  independent, nonpartisan organization to evaluate, analyze, and make recommendations  on the nation’s most critical and complex public management, governance, policy and  operational challenges.
section_title: Conclusions
paragraphs: DOE laboratories’ highly specialized facilities, cutting-edge technologies,  and highly trained scientists, technicians, and other staff represent a  significant investment of public funds. DOE’s WFO program has allowed  the department to share these capabilities with both other federal  agencies and nonfederal entities. DOE has established WFO program  requirements—including for project approval, cost recovery, program  reporting, and program review—that together are intended to help DOE  operate a successful WFO program and avoid adverse impacts on the  laboratories’ missions and facilities and to avoid competition with the  private sector. DOE falls short, however, in ensuring that these  requirements are consistently met. For example, DOE has frequently  relied on the laboratories to determine whether WFO projects selected  meet the requirements of the WFO order, and DOE officials have  accepted the laboratories’ determinations without taking steps to  independently verify these determinations. Ensuring that the WFO  projects selected meet DOE requirements is a governmental  responsibility and, according to the WFO order, a DOE contracting officer  or other authorized DOE designee is required to determine whether a  proposed WFO project has met all of these requirements before  approving the work. By relying instead on the laboratories to make these  determinations, DOE cannot ensure that all WFO projects meet  requirements. DOE may also not be able to ensure that the costs of WFO  projects are recovered according to its pricing order because it has not  required the laboratories to establish written procedures to guide  development of project budgets or charging of costs to projects, important  steps for determining and recovering the full costs of WFO projects’  materials and services. The department’s CFO also does not have  assurance that the full costs of WFO projects are charged to the projects’  sponsors because field CFOs do not always conduct biennial pricing  reviews according to requirements.
Furthermore, the WFO order does not specify what should be included in  the annual WFO program reviews required by the order. Without clear  and specific requirements, it may be difficult for DOE to identify WFO  program deficiencies, if any. In addition, DOE officials told us that the  requirement to prepare an annual summary report of WFO activities  performed at its laboratories is outdated and has not been followed for  several years, and that the department plans to eliminate this  requirement. However, members of Congress have directed DOE to  provide information on the status of WFO activities, and other DOE  programs regularly need and collect WFO project data that would be  available if DOE prepared the annual summary report. Choosing to report  data on a case-by-case basis, rather than in an annual report, may make  it difficult for those providing oversight and some users of the data, and  because the data are not readily available, DOE will need to generate  them, which is time-consuming, and the data may not be comparable  across laboratories or over time. Furthermore, while some site offices  have made efforts to evaluate the performance of the WFO program,  these efforts do not always incorporate key attributes of successful  performance measures, such as being quantifiable or having a numerical  goal. Moreover, these efforts generally do not directly address the  objectives of the WFO program. Without better measures to evaluate  program performance, DOE and decision makers will not have the  needed information to track the program’s progress in meeting its  objectives.
section_title: Recommendations for Executive Action
paragraphs: To improve DOE’s management and oversight of the WFO program, we  recommend that the Secretary of Energy take the following six actions:
Ensure compliance with the requirements in the WFO order for project  approval.
Require laboratories to establish and follow written procedures for  developing WFO project budgets and for charging costs to WFO  projects.
Ensure compliance with the requirements for conducting biennial  pricing reviews.
Specify in the WFO order what the annual WFO program reviews  should include.
Ensure that annual summary reports of WFO activities are prepared  so that data on those activities are readily available for those who  need this information.
Establish performance measures that incorporate key attributes of  successful performance measures and that address the objectives of  the WFO program.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided DOE with a draft of this report for its review and comment. In  written comments, reproduced in appendix IV, DOE stated that it  concurred with the recommendations in the report and provided  information on planned actions to address each recommendation. We  believe that many of the proposed actions, while good first steps, fall  short of our recommendations, however, and may not fully address the  issues we discussed in our report.
For example, in response to the first three recommendations (i.e., ensure  compliance with the requirements in the WFO order for project approval;  require laboratories to establish and follow written procedures for  developing WFO project budgets and for charging costs to WFO projects;  and ensure compliance with the requirements for conducting biennial  pricing reviews), DOE stated that it will issue a policy flash, or notice, on  the requirements of the WFO program. While reminding staff of the  requirements of the WFO order would likely be beneficial, to improve its  management and oversight of the WFO program, DOE also needs to take  steps to ensure that these requirements are consistently being met by  periodically monitoring the processes for project approval, full cost  recovery of projects, and biennial pricing reviews.
DOE also stated that it concurred in principle with the fourth  recommendation to specify in the WFO order what the required annual  program reviews should include. DOE added, however, that the current  WFO order appropriately provides discretion to DOE program offices in  determining the scope and extent of WFO program reviews. Again, DOE  states that it will issue a policy flash reminding the program offices of the  annual review requirement for the WFO program. DOE’s planned action,  however, does not directly address our recommendation to update the  WFO order with specific requirements for these reviews. Likewise, DOE  stated that it concurred in principle with the fifth recommendation that it  ensure that annual summary reports of WFO activities are prepared so  that data on those activities are readily available for those who need this  information. However, rather than preparing annual summary reports,  DOE stated that the WFO order requires revision to reflect its current  practice of providing current WFO program summary information. As we  said in our report, choosing to report data on a case-by-case basis rather  than in an annual report may make it difficult for those providing  oversight, and for some users of the data, because the data are not  readily available and DOE will need to generate them, which is time- consuming, and they may not be comparable across laboratories or over  time. Finally, in response to the sixth recommendation to establish  performance measures that incorporate key attributes of successful  performance measures and that address the objectives of the WFO  program, DOE plans to issue a policy flash on the current requirements of  the WFO program related to program assessments. In our report, we  discussed that the WFO order, however, does not include specific  requirements for program assessments and a policy flash that repeats the  current requirements for program assessments would, therefore, not  address the recommendation.
DOE also provided clarifying comments that we incorporated, as  appropriate. In particular, DOE requested that we consider its laboratory  contractors’ Cost Accounting Standards (CAS) disclosure statements  describing their cost accounting practices and procedures as written  procedures for charging costs to ongoing projects. We have added  information to our report about the CAS disclosure statements. While  these statements describe how the laboratory plans to allocate costs, we  do not agree that these disclosure statements constitute procedural  guidance for developing project budgets and charging costs to projects.  Moreover, as we point out in the report, we found that several laboratories  have developed detailed written procedures for developing project  budgets and charging costs to projects, which would be unnecessary if  the disclosure statements were sufficient as guidance.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Energy,  the appropriate congressional committees, and other interested parties. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made key contributions to  this report are listed in appendix V.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the amount and type of work conducted at the laboratories  under the Department of Energy’s (DOE) Work for Others (WFO)  program, we requested and obtained DOE data on WFO costs from  DOE’s Office of the Chief Financial Officer (CFO) for fiscal years 2008  through 2012, including WFO program costs by sponsor and by  laboratory, as measured in costs incurred. We also requested and  obtained total costs incurred for each laboratory for these same fiscal  years. We selected costs incurred as the measure of the amount of WFO  projects performed for our purposes because WFO agreements can span  multiple years and these costs are similar to the costs presented in  reports by DOE and its Office of Inspector General. We used data for  fiscal years 2008 through 2012 because our last report on DOE WFO  projects reported data through fiscal year 2008. We interviewed DOE  officials who oversee and collect these cost data for DOE’s Office of the  CFO and determined that these data were obtained from DOE’s Standard  Accounting and Reporting System (STARS) financial reporting system.
To assess the reliability of DOE’s WFO project cost data and total  laboratory costs, we reviewed information about DOE’s STARS financial  reporting system, as well as a recent external audit of the STARS system  and DOE’s financial data. We also reviewed recent GAO assessments of  the STARS system. No material weaknesses were reported. For data  verification purposes, we obtained fiscal year 2012 cost data on WFO  projects from each laboratory and compared that data with the fiscal year  2012 cost data on WFO projects obtained from the DOE CFO. Although  we identified some differences between the data, through discussions  with the DOE CFO, we determined that those differences were the result  of variances in the cost elements that were included in the information  provided by each laboratory. Based on our assessment, we concluded  that the data obtained from the DOE CFO were sufficiently reliable to  describe the dollar amount and percentage of WFO projects performed at  the laboratories. To determine the number of WFO projects that were  active during fiscal year 2012, we used information from the fiscal year  2012 WFO project lists provided by each laboratory, as the DOE CFO  does not have the capability to identify individual nonfederal projects. To  determine the type of WFO projects performed at DOE’s laboratories, we  gathered data on the costs of WFO activities performed for federal versus  nonfederal sponsors. We also reviewed the lists of WFO projects  provided by each laboratory, we conducted interviews with DOE  headquarters officials, and we conducted structured interviews with DOE  site officials and officials for the 17 laboratories to gather more  information about examples of WFO projects performed for one federal  and one nonfederal WFO sponsor from each laboratory with activity  during fiscal year 2012. We judgmentally selected three laboratories to  visit; including the National Nuclear Security Agency’s (NNSA) Sandia  National Laboratory in Albuquerque, New Mexico, the DOE laboratory  with the largest dollar amount of WFO project work conducted in fiscal  year 2012. We also visited NNSA’s Los Alamos National Laboratory in  Los Alamos, New Mexico, due to its size and proximity to Sandia National  Laboratory; and Pacific Northwest National Laboratory—a larger Office of  Science of laboratory located in Richland, Washington. We contacted  officials at the remaining 14 laboratories by phone.
To determine the extent to which DOE ensures that WFO program  requirements are met, we reviewed federal regulations, DOE  requirements, and DOE and laboratories’ policies and procedures  governing the WFO program, including for selection and approval of WFO  projects and for cost recovery—establishing project budgets and charging  costs to ongoing projects—for WFO projects. We discussed these  policies and procedures and how they are carried out in practice in our  structured interviews with DOE site office and laboratory officials  responsible for the WFO program at each laboratory. We reviewed and  analyzed biennial pricing reviews conducted by the DOE field CFOs  responsible for oversight of each laboratory covering fiscal years 2010  and 2011, the most recent data available. We discussed procedures for  review and approval of WFO project proposals in our structured  interviews with DOE site office and laboratory officials. We discussed  DOE requirements for WFO project pricing, pricing reviews, and review  results with officials from DOE’s CFO offices at headquarters and the  seven DOE field CFOs that oversee the laboratories. We also reviewed  DOE program office and site office review reports on laboratory WFO  programs that were conducted from calendar years 2008 through 2012 to  identify findings related to the WFO programs. For additional information  applicable to all of our objectives, we reviewed external reports on the  WFO program, including several reports issued from calendar years 2009  through 2013 by the DOE Office of Inspector General, a December 2010  report issued by the Department of Defense Office of Inspector General,  a February 2012 report issued by the National Research Council, and a  January 2013 report issued by the National Academy of Public  Administration. We contacted officials from these external entities to  discuss their reports.
To determine the extent to which DOE and the laboratories have  measured WFO program performance against WFO program objectives,  we reviewed DOE orders, performance requirements in DOE’s contracts  with laboratory management and operating contractors, DOE evaluations  of laboratories’ performance, and laboratories’ strategic plans. We also  reviewed policies and procedures for establishing and for executing WFO  projects at each laboratory. We discussed laboratories’ strategies and  goals for their WFO programs and WFO program performance  measurements with DOE headquarters officials, and in our structured  interviews with DOE site office officials, and with officials from each of the  laboratories.
section_title: Appendix II: List of the 17 Department of Energy (DOE) National Laboratories
subsections: section_title: National Laboratory Ames Laboratory
paragraphs: Research area  Rare earths and other critical  materials, applied energy, fossil  energy, and nonproliferation programs.
Physical, energy, environmental, and  life sciences; energy technologies and  national security.
Experimental and theoretical particle  physics, astrophysics, and accelerator  science.
Sustainable energy and national and  homeland security.
Particle and nuclear physics; physical,  chemical, computational, biological,  and environmental systems.
National defense, nuclear weapons  stockpile stewardship, weapons of  mass destruction, and nuclear  nonproliferation.
Los Alamos, NM  National defense, nuclear weapons  stockpile stewardship, weapons of  mass destruction, and nuclear  nonproliferation.
Environmental stewardship, clean  energy.
Renewable energy and energy  efficiency research.
Neutron scattering, advanced  materials, high-performance  computing, and nuclear science and  engineering.
Electricity management, sustainability,  threat detection and reduction, in situ  chemical imaging and analysis,  simulation and analytics.
Plasma and fusion energy sciences.
National Laboratory  Sandia National Laboratories  Albuquerque, NM  National defense, weapons of mass  destruction, transportation, energy,  telecommunications, and financial  networks, and environmental  stewardship.
Environmental stewardship, national  and homeland security, clean energy.
Menlo Park, CA  Materials, chemical and energy  science, structural biology, and particle  physics and astrophysics.
Fundamental nature of confined states  of quarks, gluons, and nucleons;  superconducting radio-frequency  technology.
section_title: Appendix III: Department of Energy National Laboratory Work for Others Costs and Total Costs, as Measured in Costs Incurred, in Fiscal Year 2012
paragraphs: Appendix III: Department of Energy National  Laboratory Work for Others Costs and Total  Costs, as Measured in Costs Incurred, in Fiscal  Year 2012 Total Work for  Others (WFO)
section_title: Appendix IV: Comments from the Department of Energy
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Dan Feehan and Janet Frisch,  Assistant Directors; Joseph Cook; Elizabeth Curda; Paul Kinney; Jeff  Larson; Cynthia Norris; Josie Ostrander; Kathy Pedalino; Tim Persons;  Cheryl Peterson; Carl Ramirez; and Kiki Theodoropoulos made key  contributions to this report.