Department of Energy: Mission Support Challenges Remain at Los Alamos and Lawrence Livermore National Laboratories
Released: 2004-03-31T00:00:00

Letter

Background

NNSA carries out its nuclear weapons research missions at research laboratories located in two states—California and New Mexico. NNSA and DOE have traditionally relied on contractors to carry out the department’s missions. However, the department’s history of inadequate management and oversight and failure to hold its contractors accountable for results led GAO in 1990 to designate DOE contract management as a high-risk area vulnerable to fraud, waste, abuse, and mismanagement. As of February 2004, this high-risk designation was still in effect.

At the Los Alamos and Lawrence Livermore laboratories, about 200 NNSA personnel at laboratory site offices are responsible for oversight of the work performed under contract by over 16,000 employees of the University of California. The contracts with the University provide for reimbursement of all allowable costs plus a fee that is in addition to the allowable costs. The total fee available to the University includes a base or fixed amount that is guaranteed and an “at-risk” amount that is tied to performance measures in the contract. (See table 1.)

For more than a decade, GAO and others have reported on problems with the mission support activities at the two laboratories, including project management, nuclear safety management, and facilities management. The problems have included cost and schedule overruns on major projects such as the Dual-Axis Radiographic Hydrodynamic Test Facility at Los Alamos; lack of adequate safety documentation for nuclear facilities; and a deferred maintenance backlog of about $318 million at Lawrence Livermore and $564 million at Los Alamos. In addition, in the early 1990s, we reported on poor controls over business operations such as procurement and property management at the Lawrence Livermore laboratory.

For several years, NNSA has chosen to address these performance problems using contract mechanisms. For example, when DOE extended the Los Alamos contract in October 1997, it included a special provision in the contract that would allow DOE to terminate the contract if the University failed to make improvements in several mission support areas— ensuring that workers, the public, and the environment are protected; cleaning up radioactive and hazardous wastes; and maintaining a good relationship with the local community. The department subsequently decided to continue the contract after the Los Alamos laboratory made improvements in these three areas. However, when NNSA extended the contracts for both of the laboratories in January 2001, it included additional requirements to improve project management, facilities management, and nuclear safety management.

Oversight of the laboratories occurs at several different levels. NNSA provides direct oversight of the two laboratories through its site offices. In addition, NNSA headquarters staff offices, such as the Offices of Defense Programs and Nonproliferation, provide funding and program direction to the site offices. DOE’s Offices of Independent Oversight and Performance Assurance and Price-Anderson Enforcement also oversee laboratory activities. Finally, the Defense Nuclear Facilities Safety Board, an independent oversight organization created by the Congress in 1988, provides advice and recommendations to the Secretary of Energy to ensure adequate protection of public health and safety at all of the department’s defense nuclear facilities, including the Los Alamos and Lawrence Livermore laboratories.

NNSA and University’s Actions Taken to Address Mission Support Problems Identified in 2001 Have Not Been Fully Implemented

NNSA and the University have taken a number of steps to address the major mission support problems that were known when NNSA extended the University’s contracts in 2001, but all actions will not be complete until mid-2005. When NNSA decided to extend the contracts for the two laboratories, concerns had emerged in three areas: project management, facilities management, and nuclear safety. For its part, NNSA incorporated into the two contracts new agencywide requirements related to each area, developed performance measures focusing on these activities, and changed its overall structure and approach for overseeing the actions taken at the laboratory level. In response, the University has implemented many of NNSA’s requirements at the two laboratories, particularly in project and facilities management, but progress has been slower for nuclear safety. The University also has taken steps to improve its oversight of the two laboratories and to foster coordination and collaboration between them.

NNSA Set New Requirements and Moved to Strengthen Its Oversight

One step NNSA took to improve laboratory performance in mission support activities was to incorporate into the contracts new agencywide requirements related to all three of the identified problem areas. In general, these new requirements call for the laboratories to be more disciplined and businesslike in carrying out their management responsibilities.

Project Management. In October 2000, DOE approved Order 413.3, which set agency wide standards for managing projects, including NNSA projects. The order is intended to help ensure that projects are delivered on schedule and within budget and are fully capable of meeting mission requirements as well as environmental, safety, and health standards. It requires that all projects costing more than $5 million go through five decision checkpoints, such as approving mission need and approving the start of operations. At each checkpoint, NNSA must make a formal determination to allow the project to proceed.

Facilities Management. Many of the facilities at NNSA sites are decades old and in poor condition. For fiscal year 2003, NNSA required each nuclear weapons site to develop a 10-year comprehensive site plan for facilities management that incorporated NNSA’s strategic goals. The main objective of the plans is to restore, rebuild, and revitalize the physical infrastructure of the nuclear weapons complex. Two of NNSA’s specific goals are to stabilize the deferred maintenance backlog by the end of fiscal year 2005 and to reduce the amount of deferred maintenance to within industry standards by the end of fiscal year 2009.

Nuclear Safety. DOE has always required that the contractors responsible for a nuclear facility analyze the facility, the work to be performed, and the associated hazards. Contractors use this information to identify the conditions, safe boundaries, and hazard controls necessary to protect workers, the public, and the environment from adverse consequences. Finally, the contractors document the safety requirements and operating procedures for each facility, referred to as the safety basis. Although DOE included these steps in an order for contractors to follow, the laboratories had not consistently done a thorough and quality analysis of their nuclear facilities. In January 2001, DOE finalized a revised nuclear safety rule, requiring that contractors responsible for nuclear facilities establish and maintain a safety basis. Contractors may be subject to civil penalties for failing to comply with this requirement. This revised rule strengthened NNSA’s ability to hold contractors accountable for the safety of nuclear facilities In addition to incorporating new agencywide requirements into the two contracts, NNSA established new contract mechanisms and performance measures to help ensure that the laboratories put in place management improvements for mission support activities. These measurements have changed over time. Initially, NNSA added a number of specific improvement initiatives in a new appendix (appendix O) to the contract, established to address known problems areas and provide a framework for improved results. For the first 2 years of the contract, the laboratories had to meet all of the requirements in appendix O on a pass/fail basis in order to qualify for performance incentive fees. NNSA phased out appendix O at the end of fiscal year 2002 after determining that the laboratories had met all of the provisions and requirements.

Starting in fiscal year 2003, NNSA shifted to assessing a “critical few” broadly defined performance objectives. In prior fiscal years, the two laboratory contracts contained dozens of individual performance objectives, and poor performance in one of these individual objectives would have minimal impact on NNSA’s evaluation of overall contractor performance or the amount of fee that could be earned by the contractor. In fiscal year 2003, NNSA narrowed the number of performance objectives in the two contracts to nine objectives that define the mission of the laboratories, including two objectives that cover mission support activities—achieving successful completion of projects and maintaining secure, safe, environmentally sound, effective, and efficient operations and infrastructure in support of mission objectives. In fiscal year 2004, NNSA added a third mission support objective—improving or maintaining effective business systems and practices that safeguard public assets and support mission objectives.

The third main step NNSA took was to revise its approach to overseeing the laboratories. NNSA moved additional oversight staff, which had previously been in operations offices located in distant cities, to the laboratories themselves to bring more oversight staff closer to the laboratory’s day-to- day operations. NNSA also began regular meetings with senior managers at the University and the two laboratories to identify and resolve emerging issues or areas that need improvement. Finally, NNSA designated the senior on-site NNSA representative—the site office manager—as the contracting officer for the laboratory. The contracting officer is the main point of contact and the single point of accountability for the contract. Before this change, the contracting officer was located in the procurement division at the operations office and did not report to the NNSA site office manager. NNSA believed this organizational change would clarify roles and responsibilities, eliminate a layer of management, and provide more effective federal oversight.

University Strengthened Management and Began Implementing New Requirements

The University made several changes designed to strengthen management of mission support activities. The first was to redefine the relationship between the two laboratories, making their efforts at managing mission support more collaborative and interactive. Previously, the laboratories had operated more autonomously and competitively—an approach the University fostered as the best means for achieving world-class science. The recent onset of mission support problems convinced University officials that increased coordination in mission support activities would be beneficial. According to University officials, they wanted to create an environment in which the laboratories would identify problems, share solutions, create best practices, and be more consistent in their approaches to laboratory management.

Specific steps the University took to develop a more collaborative relationship between the two laboratories included creating a position of vice president of laboratory management, loaning staff between the two laboratories and from the University as needed, and creating standardized policies. For example, after the Lawrence Livermore laboratory developed a model for a risk-based approach to facilities management, the Lawrence Livermore staff visited Los Alamos to help that laboratory apply the approach to its facilities management. In addition, when the University determined that the two laboratories had different security policies regarding foreign nationals and different definitions of “sensitive property” for property management, it required the laboratories to work out the differences and devise one best practice.

The second major change by the University was to provide more outside assistance to the laboratories. The University contracted with firms from private industry for expertise and advice on nuclear facility safety at the Los Alamos laboratory and on project management at both laboratories. In addition, since the early 1990s, a University President’s Council supported by five different panels had been in place to make University expertise available to the laboratories in such areas as project management and environment, safety, and health. In 2001, the University increased the interaction between the Council, the panels, and the laboratories, for example, by assigning a mentor from the project management panel to each of the major projects at the laboratories. In November 2003, the University also created two additional oversight groups to strengthen management and oversight of the two laboratories and to make more expertise available from outside the University.

Laboratories’ Actions Are in Various Stages of Completion

Although NNSA had substantially completed its efforts at incorporating new requirements into the contracts, establishing new performance measures, and modifying its oversight approach, the laboratories’ efforts to implement the new requirements are in various stages of completion.

Project management. By June 2002, both laboratories had completed planned actions, including implementing new DOE procedures for managing projects, providing improved training for project managers, and providing improved assistance to project teams. In addition, both laboratories had standardized the formats for monthly reporting on cost and schedule for major projects to make it easier to identify negative performance trends.

Facilities management. By October 2002, both laboratories had completed 10-year strategic plans detailing how the laboratories will restore, rebuild, and revitalize their physical infrastructures. The plans incorporated NNSA’s goals of stabilizing deferred maintenance by fiscal year 2005 and reducing deferred maintenance to industry standards by fiscal year 2009. According to the Los Alamos 10-year plan, industry standards set maintenance costs at about 2 to 4 percent of the estimated replacement value of the entire facility. To achieve this goal, Los Alamos plans to increase maintenance funding and implement the corrective actions necessary to create a more efficient program. Los Alamos and NNSA officials noted that some of the actions would involve long-term efforts spanning several years. In contrast, since 1998, the Lawrence Livermore laboratory has had a risk-based facility maintenance program to prioritize facility maintenance requirements. In addition, the laboratory began to assess a “laboratory facility charge” per square foot to building occupants, both to fund facility maintenance and to encourage users to give up unnecessary space. As a result, the Lawrence Livermore laboratory has already met NNSA’s fiscal year 2005 goal of stabilizing the deferred maintenance backlog.

Nuclear safety. The two laboratories did not submit all required safety analysis documentation by the April 2003 milestone date, and they may not be fully in compliance until mid-2005. Federal regulations required that by April 10, 2003, the laboratories must have provided assurance that they could meet new and enhanced nuclear safety requirements. The Los Alamos laboratory initially reported that it had met the deadline for providing that assurance, but later disclosed to NNSA that some radioactive waste sites were not included in the original analyses submitted to NNSA. In December 2003, NNSA and the laboratory agreed that 11 radioactive waste sites were subject to the nuclear safety requirement. NNSA expects the laboratory to provide the safety basis documentation on these sites by April 2004. In addition, NNSA reported in its fiscal year 2003 assessment for the Los Alamos laboratory that the quality of 7 of the 12 safety analyses submitted on time had been unsatisfactory and required revision. As for the Lawrence Livermore laboratory, it completed 5 of the 9 required safety analyses by the April 10, 2003, deadline and requested and received extensions for submitting the remaining analyses. NNSA has granted an extension until April 2005 for the laboratory to submit the safety analysis for the final facility. Lawrence Livermore laboratory officials said they did not meet the original deadline because of resource constraints and changes in the work activities at some of the facilities.

After 2001, Additional Mission Support Problems Developed at the Laboratories, and Further Corrective Actions Are Underway

Despite the steps taken in 2001 to address mission support problems at the two laboratories, the laboratories have encountered additional problems in mission support activities and are taking further actions to address those problems. New problems in business operations, such as controls over purchase cards and property, emerged at the Los Alamos laboratory in 2002, while developing an emergency management program that complied with NNSA requirements continued to be a problem at the Lawrence Livermore laboratory.

Problems Surfaced with Business Operations at Los Alamos

Beginning in the summer of 2002, a series of problems with business operations at the Los Alamos laboratory surfaced, raising questions about the effectiveness of controls over government purchase cards and property. These events included allegations of fraudulent use of government purchase cards and purchase orders by a few Los Alamos employees, questions about the adequacy of property controls over items such as computers, and disputed rationales for the laboratory’s firing of two investigators who were working on some of these cases. In an April 2003 report on the problems with business operations at Los Alamos, the department identified multiple causes, including (1) the University’s supervision of business processes at the laboratory was ineffective, (2) NNSA’s oversight was narrowly focused on specific performance measures in the contract rather than on overall effectiveness, and (3) both the University and NNSA may have ignored warning signs of problems because other evaluations of contractor performance did not identify significant weaknesses. The report concluded that the actual loss to the federal government could have been far greater than it actually was, and the business practices in place in 2002 would not have been able to prevent or detect such losses. In addition, the report concluded that the firing of the two investigators was inappropriate and demonstrated the degree to which the laboratory’s management was out of touch with activities at the laboratory.

In January 2001, when NNSA extended and modified the contract for the Los Alamos laboratory, NNSA officials were not aware of the problems with business operations at the laboratory. NNSA did not consider business operations at Los Alamos to be an area of concern to include in the ongoing mission support improvement efforts. Furthermore, even after the problems with business operations at Los Alamos emerged in the summer of 2002, NNSA did not include any performance measures for business operations at the start of fiscal year 2003. It was not until February 2003 that NNSA added performance measures to the contract to address the business operations problems. NNSA also modified its method of assessing contractor performance in fiscal year 2003 to provide a more integrated evaluation of performance in business operations.

Once the significance of the problems with business operations at the Los Alamos laboratory was known, the University responded with corrective actions. The University made sweeping changes in the Los Alamos management team, replacing, among others, the laboratory director, principal deputy director, chief financial officer, and laboratory auditor. In addition, the University commissioned a series of internal and external reviews to identify further problems and control weaknesses in procurement and property management. These reviews resulted in more than 300 findings and recommendations to improve controls. The laboratory is in the process of implementing more than 600 corrective actions to respond to these recommendations and implement other laboratory initiatives and expects to complete these corrective actions by June 2004.

As a fuller understanding of the scope of the problem emerged, managers at the Los Alamos laboratory decided to address the problems with business operations in three phases. Initially, the laboratory is taking hundreds of specific actions in response to identified problems and recommendations. These actions included changing procedures and strengthening internal controls. According to the laboratory managers, the second phase involves a more comprehensive change to business systems by designing and implementing an improved, integrated business computer system that will facilitate both more efficient and more effective business operations. Laboratory officials most recently estimated the cost of this system, currently under development, at about $150 million. Laboratory management said the last phase will involve developing and implementing a set of performance goals and measures for business operations so that laboratory management can better track and sustain results in this important area. The laboratory plans on having these additional measures in place by the end of 2005.

In April 2003, primarily because of these ongoing problems with business operations, the Secretary of Energy announced his intention to open the Los Alamos contract to competition for the first time, when the current contract expires in September 2005. Although acknowledging the University’s contribution to high caliber science at Los Alamos, the Secretary stated that he held the University responsible for the systemic management failures at the laboratory. The Secretary encouraged the University to enter the contract competition, but made clear that the laboratory’s performance in business services needed to be as good as its performance in science.

Lawrence Livermore Continued to Have Problems with Emergency Management

The Lawrence Livermore laboratory has had difficulty developing an emergency management program that complies with NNSA requirements. Emergency planning consists of identifying hazards, threats, and ways to mitigate hazards; developing and preparing emergency plans and procedures; and identifying personnel and resources needed to ensure effective emergency response. Effective emergency management has been an issue of increasing significance since the terrorist attacks against the United States in September 2001.

When NNSA extended the contract in January 2001, NNSA recognized that the emergency management program at the Lawrence Livermore laboratory was not in compliance with DOE Order 151.1, which sets out the program requirements. For example, the laboratory had not included in its previous assessment of hazards and risks even the possibility that a release of materials or other incidents on site would potentially travel off site and affect the local community outside the boundaries of the Livermore site. As a first step toward compliance, NNSA included a requirement in the contract for 2001 that the laboratory prepare and submit for approval hazard assessments for each of its facilities and activities that met the specified thresholds for such assessments. These assessments were submitted on time in May 2001 and were reviewed and approved by NNSA.

In the fiscal year 2002 performance measures for the contract, NNSA required the laboratory to develop and implement a plan to achieve substantial compliance with DOE Order 151.1 by September 2003. This plan was to include a schedule and milestones to satisfy all the elements of an emergency management program, such as emergency preparedness training and an emergency public information program. Although the laboratory submitted the first draft of the plan on time early in the fiscal year, the quality of the plan did not meet NNSA expectations, and the laboratory received a marginal rating on this measure for fiscal year 2002. NNSA required the laboratory to revise and resubmit its plan. In its overall evaluation of the laboratory’s performance for that year, NNSA identified implementing an effective emergency management program as one of the three institutional management challenges facing the laboratory.

In July of 2002, DOE’s Office of Independent Oversight and Performance Assurance criticized the laboratory’s lack of progress in resolving its problems with emergency management. The review team identified a number of important procedural and performance weaknesses. For example, the laboratory did not have clearly defined processes for deciding on the appropriate on-site and off-site protective actions to take. The report concluded that the laboratory faced significant challenges in implementing improvements in the program and that the poor quality of the documents provided to NNSA raised serious questions about the laboratory’s ability to meet the implementation milestones in the plan.

Starting in fiscal year 2003, NNSA officials took additional steps to help ensure that the laboratory addressed its emergency management problems. In the fiscal year 2003 and 2004 performance measures in the laboratory contract, emergency management was included as one of the “critical few” measures. According to an NNSA official, this increased the focus of senior management attention on the activity. Prior to fiscal year 2003, performance measures for emergency management were included as part of the overall performance objective of environment, safety, and health and received less emphasis and attention. The laboratory received a satisfactory rating (one step higher than the fiscal year 2002 marginal rating) on the fiscal year 2003 emergency management measure. NNSA now estimates that the laboratory will have an emergency management program that is in substantial compliance with DOE orders by the end of fiscal year 2004.

In contrast to the Los Alamos contract, the Secretary had not signaled his intention about whether to extend or compete the Lawrence Livermore laboratory contract. However, in the fiscal year 2004 appropriations act for energy and water development, the Congress required DOE to compete the Lawrence Livermore laboratory contract when the current contract expires in 2005. The statute provides that no appropriated funds for fiscal year 2004 or any previous fiscal year could be used for specified laboratory contracts that had been awarded more than 50 years ago without competition, unless the Secretary of Energy announces a decision to compete the contracts at the end of their current terms. In late January 2004, the Secretary announced the decision to compete the specified laboratory contracts, including those for the Los Alamos and Lawrence Livermore laboratories.

Challenges Remain for Sustaining Mission Support Improvements at the Laboratories

NNSA and the University face three main challenges to sustaining improvements over the long term in mission support activities at the two laboratories. These challenges include (1) ensuring that the mission support activities are effectively performed, (2) ensuring that NNSA provides effective oversight of laboratories’ activities, and (3) ensuring that management improvement initiatives such as the improvements to business systems at the Los Alamos laboratory fully address the existing problems and are carried out in a systematic manner consistent with best practices.

Ensuring that Mission Support Activities Are Effectively Performed

A major factor leading to the problems with managing mission support activities at the laboratories, including an increased potential for fraud, waste, and abuse, was what one internal DOE assessment referred to as the devaluing of mission support activities by laboratory personnel. Although significant investments in improving mission support activities and controls have subsequently occurred, there are continuing concerns about whether the laboratories will continue to place sufficient emphasis on mission support activities to ensure that these functions are effectively performed.

Ensuring that actions taken to address mission support problems translate into effective performance of mission support requires establishing and maintaining an effective system of management control. Office of Management and Budget Circular No. A-123 defines management controls as the organization, policies, and procedures used to reasonably ensure that (1) programs achieve their intended results; (2) resources are used consistent with agency missions; (3) programs and resources are protected from waste, fraud, and mismanagement; (4) laws and regulations are followed; and (5) reliable and timely information is obtained, maintained, reported, and used for decision making. Effective management controls require leadership and commitment on the part of management. Internal control standards state that the attitude and philosophy of management toward information systems, accounting, and monitoring can have a profound effect on internal control. The standards require management to establish and maintain an organizational environment that sets a positive and supportive attitude toward internal control.

The two laboratories have differed in the degree to which they have been successful in ensuring that mission support activities are effectively performed and in maintaining effective management controls. The Los Alamos laboratory had over time been weakening its management controls over some mission support activities. Responding to pressures to reduce overhead costs and a view that the NNSA laboratories were unnecessarily burdened with administrative and procedural requirements that were not adequately serving mission needs, the laboratory pursued cost efficiencies in mission support activities without sufficient regard for ensuring that the overall management control system was effective. For example, since fiscal year 1995, the Los Alamos laboratory has reduced the relative funding for mission support activities in order to provide more funding to mission activities. The reduced funding contributed to the weakening of business controls as the laboratory scaled back or eliminated steps, such as reviewing small item purchases because fewer staff were available to perform the reviews.

The problems with management controls at the Los Alamos laboratory were to some extent due to the laboratory’s organizational culture. An April 2003 DOE report on the business operations problems at the Los Alamos laboratory cited cultural problems as one of the underlying causes of the systemic management failure of business systems at the laboratory. The report stated that the Los Alamos culture exalted science and devalued business practices, and that changing this attitude would be the most difficult long-term challenge facing the laboratory, regardless of who manages it in the future. NNSA and laboratory officials at Los Alamos have stated that the pressures to reduce mission support costs will probably continue, which increases the challenges associated with improving controls and ensuring that mission support activities are effectively performed.

In contrast to the problems documented at the Los Alamos laboratory, the Lawrence Livermore laboratory has apparently been more successful in emphasizing the importance of mission support activities and ensuring that these support activities are effectively performed. The laboratory encountered similar problems with its business operations in the early- 1990s, including weaknesses in procurement and property management, and took steps at that time to improve its financial and accounting systems. Thus, when faced with the same pressures to reduce overhead costs in recent years, the Lawrence Livermore laboratory was better able to accomplish those reductions without significantly degrading the quality or effectiveness of its internal controls. For example, a June 2003 external assessment of business systems at the Lawrence Livermore laboratory identified no material weaknesses in internal control systems but did contain recommendations to enhance management controls in such areas as procurement and property management. In contrast, external reviews of procurement systems at the Los Alamos laboratory identified significant weaknesses in internal controls, such as insufficient policies and procedures and inadequate management.

NNSA recently began to address these organizational culture issues and the need to understand the importance of effective management controls. The problems with management controls at the Los Alamos laboratory are similar to the organizational problems documented at the National Aeronautics and Space Administration (NASA) after the Columbia space shuttle accident in February 2003. The independent panel tasked with investigating the causes of the accident reported in August 2003 that NASA’s organizational culture was a contributing factor to the breakdown in management controls intended to ensure safety for the shuttle and its crew. Specifically, the report cited as one of the root causes of the accident the organizational culture at NASA, which emphasized mission rather than safety. In addition, under pressure to reduce costs, the agency had transferred responsibilities to the private sector while reducing federal oversight. As part of its efforts to improve operations at the laboratories, the NNSA Administrator has required all of the senior NNSA managers to review the Columbia Accident Investigation Report for findings that could be applied to NNSA. In addition, the Administrator chartered a task force to perform an in-depth review of the cultural and organizational issues described in the Columbia report and to make recommendations on how the department could improve the effectiveness of mission support functions to ensure the safe performance of high-risk mission work. NNSA officials estimate that the task force report will be available in March 2004.

The laboratories have also taken steps to address organizational attitudes about mission support activities. At the Lawrence Livermore laboratory, the director and senior management developed a list of values for the laboratory that includes 10 items deemed to be critical to success. One of the items is “simultaneous excellence in science and technology, operations, and business practices.” At the Los Alamos laboratory, the director established priorities in 2003 to help guide the laboratory’s efforts. The priorities include safety and security, mission, and business operations. The laboratory is also developing a strategic plan that includes both mission and mission support goals and objectives.

The University is also exploring other ways to improve management of the two laboratories. In January 2004, the University Board of Regents took steps to allow the University to form partnerships with outside companies for the upcoming competition for the laboratory contracts. The University’s Vice President for Laboratory Management said that outside partners with strong management and business experience could strengthen its performance in the areas of business operations and other mission support areas.

These actions are positive steps toward increasing the awareness of the importance of mission support activities, but for several reasons concerns remain about whether the laboratories will continue to ensure that mission support activities are effectively performed. First, although current senior management at both laboratories supports the importance of having effective mission support activities, there has been a long history of emphasizing mission over mission support. Second, NNSA has also typically placed much more emphasis on mission than mission support and has often failed to detect problems when they existed. For example, the April 2003 DOE report on business operations at the Los Alamos laboratory stated that the NNSA evaluation system in place at Los Alamos failed to consider relationships between different processes at the laboratory and therefore failed to detect overall systemic problems. One of the report’s recommendations was to ensure that NNSA reviews of contractor performance capture cross-cutting information in both mission and mission support areas to form a more complete picture of performance.

Furthermore, although much work has been done to implement new mission support requirements and improve management of mission support at the two laboratories, considerable time may be needed to determine the extent to which the actions taken will result in improved performance. For example, some of the efforts, such as the longer-term efforts needed to reduce deferred maintenance to industry standards by fiscal year 2009, will take years to complete. In addition, the Los Alamos laboratory continues to have problems with workers failing to comply with nuclear safety procedures. For fiscal years 2001 through 2003, the Los Alamos laboratory had filed 51 reports on nuclear safety incidents, some resulting in exposures of workers to radiation. Efforts are underway to improve performance in this area, but safety officials at the laboratory acknowledge that some of the improvement efforts may take months or years of sustained effort to complete.

The laboratories will also need to ensure that improvement efforts are sustained and effective. NNSA officials, including the senior technical safety advisor at the Los Alamos site office, noted they did not have a high level of confidence in the laboratory’s ability to sustain improvements because the laboratory’s track record in this regard has not been good. For example, a March 2003 report on nuclear safety at the Los Alamos laboratory analyzed 32 corrective actions between fiscal years 1996 and 2003 and concluded that many of the improvement efforts had not been sustained or followed up on, allowing many of the safety violations to recur. NNSA officials, including the assistant manager for business management, also noted that the laboratory was reporting on when corrective actions had been implemented rather than on their effectiveness. For example, in December 2003, Los Alamos reported that it had completed 80 percent of its improvement efforts in procurement but did not report on the effectiveness of those efforts. Based on complaints from vendors and others outside the laboratory, NNSA officials sampled procurement records and found recurring problems. The NNSA officials said that they had been in discussions with Los Alamos laboratory officials since September 2003 in efforts to reach agreement on how to assess improvement efforts for effectiveness, but as of December 2003, no agreement had been reached.

Ensuring that NNSA’s Oversight Is Effective

NNSA’s reliance on contractors to operate its facilities and carry out its missions makes effective oversight of contractor activities crucial to success. In the past, however, oversight of the laboratories’ mission support activities has been inadequate. Both the University and NNSA had failed to ensure that the laboratories’ mission support activities were effective.

The University had in general taken a “hands off” approach to overseeing the laboratories. For example, in its April 2003 report on evaluating problems at the Los Alamos laboratory, NNSA stated that prior to November 2002, the University’s oversight of Los Alamos was ineffective in the area of business processes. The report added that the University was slow to respond to allegations of problems with business practices, initially limiting its involvement to providing assistance as requested by the laboratory director and not ensuring that the laboratory director was taking sufficient steps to address the problem.

NNSA oversight also was not adequate to identify and address the critical shortcomings in management controls. Regarding the Los Alamos laboratory, in May 2003, DOE’s Office of Inspector General reported that improvements were needed in NNSA’s project management oversight and control. This conclusion was based on the Inspector General’s review of a new facility that will be used to evaluate the effects of aging on the nation’s nuclear weapons stockpile. The report stated that the project did not have a viable baseline and that at least $57 million in cost increases had occurred, but NNSA oversight was inadequate to identify the problem. Also, the April 2003 DOE report on problems at the Los Alamos laboratory stated that NNSA’s direct federal oversight had been narrowly focused on specific performance measures in the contract, rather than on overall effectiveness. Weaknesses in NNSA oversight also occurred at the Lawrence Livermore laboratory. For example, in a May 2003 report on a new waste treatment facility at the laboratory, we concluded that a delay in initiating storage and treatment operations at the new facility occurred because NNSA managers took over a year to resolve disagreements with the laboratory on technical issues affecting the safe operation of the new building for temporarily storing wastes. Providing clear requirements and ensuring that the contractor complies with those standards in a timely manner is part of NNSA’s oversight responsibilities.

These past problems with oversight raise concerns about NNSA’s proposed change to its oversight approach. NNSA’s August 2003 draft Line Oversight and Contractors’ Assurance System policy would rely more on contractor oversight and self-assessment and less on NNSA’s direct oversight. The proposal would require a comprehensive contractor assurance system, or system of management controls, to be in place and would primarily rely upon these systems and controls to ensure that its missions and activities are properly executed in an effective, efficient, and safe manner. NNSA would use a risk-based, graded approach to its oversight and tailor the extent of federal oversight to the quality and completeness of the contractor’s assurance systems and to evidence of acceptable contractor performance. NNSA’s oversight functions would include review and analysis of contractor performance data, direct observations of contractor work activities in nuclear and other facilities, annual assessments of overall performance under the contract, and certifications by the contractor or independent reviewers that the major elements of risk associated with the work performed are being adequately controlled. NNSA stated in its draft policy and in public meetings before the Defense Nuclear Facilities Safety Board that the department plans to phase in this new oversight approach over the next few years.

Although we believe that the overall concept of a risk-based approach to federal oversight is reasonable, concerns exist about whether NNSA will be able to effectively carry out this approach while successfully meeting its responsibility for safe and secure operations. For example, considerable work will be needed to successfully implement a risk-based approach to laboratory oversight. According to the Associate Director for Operations at the Los Alamos laboratory, the laboratory’s ability to manage risk is at a beginning level of maturity. Other officials at the Los Alamos laboratory, including officials from the Performance Surety Division and the Quality Improvement Office, said that the laboratory and NNSA have different perceptions of risks at the laboratory and how to manage those risks. In addition, they said that both the laboratory and NNSA have been reacting to problems after they have come to light rather than managing risks to prevent problems from occurring.

In addition to these concerns specific to the two laboratories, DOE and others have raised broader concerns about the adequacy of oversight. For example, in November 2003, DOE’s Office of Inspector General released its annual report on management challenges, including oversight of contracts and project management as two of the three internal control challenges facing the department. The report stated that these challenges represent issues that, if not addressed, may impede the department’s ability to carry out its program responsibilities and ensure the integrity of its operations. The department also included program oversight of contractors as a significant matter of concern in its performance and accountability report for fiscal year 2003. Furthermore, the Defense Nuclear Facilities Safety Board, in recent public meetings, has expressed concerns about nuclear safety under the proposed NNSA contractor assurance policy and said that NNSA should not delegate responsibility for such an inherently high-risk area of operations.

Ensuring that Improvement Initiatives Follow Best Practices

NNSA and the University have not ensured that the laboratories manage major improvement initiatives using a best practices framework that helps ensure successful implementation. For example, one aspect of improving mission support activities at the Los Alamos laboratory has involved a major upgrade of business systems including budgeting, accounting, procurement, and property management. This business systems improvement initiative is planned to take 18 months and be completed at the end of June 2004. Unfortunately, laboratory officials have not followed best practices in managing the improvement initiative, increasing the risk that the initiative may not fully address existing problems or be the most effective approach.

In previous work, we found that best practices by leading organizations to sustain management improvement initiatives involved using a systematic, results-oriented approach that incorporated a rigorous measurement of progress. Such an approach typically included the following elements: (1) define clear goals for the initiative, (2) develop an implementation strategy that sets milestones and establishes responsibility, (3) establish results- oriented outcome measures early in the process to gauge progress toward the goals, and (4) use results-oriented data to evaluate the effectiveness of the initiative and to make additional changes where warranted.

For its business systems improvement initiative, the Los Alamos laboratory established an implementation strategy that set milestones and assigned responsibility for carrying out the strategy. For example, the business process improvement plan included over 600 required actions, each of which had a time frame for completion and a laboratory employee responsible for the action. The laboratory is tracking each of the actions to ensure that they are completed.

While the Los Alamos laboratory had an implementation strategy for its business systems improvement initiative, it implemented those actions largely without clearly defined goals, results-oriented measures, or results- oriented data to evaluate the effectiveness of its actions.

Clear goals not defined. Although the Los Alamos laboratory had a strategy for business improvement that included general goals, it did not define the goals in measurable terms. The laboratory’s primary goal for 2003 was to reduce the risks associated with internal control vulnerabilities in its business systems. This general goal does not provide a measurable end point; it does not indicate how much risk reduction is enough or how changes in risk could be measured. Nor does the even more general objective of restoring trust in the laboratory’s business systems, mentioned by some Los Alamos officials, provide a measurable end point. While addressing internal control problems is important, it does not by itself indicate that improvements are sufficient or effective.

Results-oriented outcome measures not established. The laboratory did not establish results-oriented outcome measures for its business improvement initiative. Instead, the laboratory generally focused on measuring the progress of implementing its improvement actions, such as the percentage of improvements that have been implemented. Such measurements, however, do not provide an indication of progress toward the overall goal of reducing the risk of fraud, waste, and abuse. Laboratory officials told us that instead of developing such measures early in the business improvement initiative, their strategy is to define measurable performance goals after the many actions associated with the business improvement initiative are in place.

Results-oriented data not used to evaluate effectiveness. The laboratory did not have the results-oriented outcome data needed to evaluate the effectiveness of its business improvement initiative. Again, laboratory officials told us that after the improvement actions are in place, they plan to define and generate results-oriented data to correspond with measurable performance goals. In addition to the lack of results- oriented outcome data, the laboratory also lacks the information necessary to determine if these improvement efforts are cost-effective. The laboratory has only partial information on the cost of the improvement initiative. The laboratory’s Associate Director for Administration told us that the laboratory has not focused on the costs of the current business improvements. He added that the laboratory will consider costs once the new systems are in place and decisions must be made about balancing the cost of business activities against the risks of removing some internal control activities.

Because the Los Alamos laboratory lacked several elements of a best practices approach to managing improvement initiatives, the laboratory did not have a sufficient basis from which to objectively review the results of the improvement initiative, assess the reasonableness of costs incurred, or take further corrective actions if necessary to achieve the overall goals of the initiative. Laboratory officials explained that they had given immediate priority to fixing business system problems rather than measuring and sustaining improved business results. Furthermore, the Associate Director for Administration concluded that good performance measures would take considerable time to develop and that implementing corrective actions was a higher priority. However, by waiting to focus on results and costs later, the laboratory increases its risk that the initiative may not fully address existing problems or be the most cost-effective approach to reducing its internal control vulnerabilities to appropriate levels.

Laboratory officials said that they generally follow an organized process for implementing improvement initiatives that includes defining the tasks to be accomplished, creating a schedule with milestones, and assigning responsibility for the actions. However, such a process does not include all the elements that we have identified as necessary for a best practices approach. Neither the University nor NNSA was influential in ensuring that the laboratory followed best practices in managing the business system improvement initiative, even after the department had issued guidance on managing improvement initiatives. In October 2003, DOE issued Notice 125.1, Managing Critical Management Improvement Initiatives, which describes best practices for managing improvement initiatives and requires that those practices be followed by NNSA. However, this notice does not apply to DOE’s contractors, and NNSA has not incorporated similar requirements into NNSA’s contracts with the University to manage the Los Alamos and Lawrence Livermore laboratories.

Conclusions

Effectively accomplishing the mission of conducting world-class scientific work at the Los Alamos and Lawrence Livermore laboratories also requires the laboratories to maintain good business practices; accountability for mission support activities; and safeguards against fraud, waste, abuse, and mismanagement. Sufficient emphasis on mission support activities has been lacking, especially at the Los Alamos laboratory, and achieving and sustaining effective performance in mission support will require strong leadership and commitment. Efforts to improve performance in mission support activities are still underway at the laboratories, and it may take considerable time to determine if the efforts are effective. Managing these efforts using best practices will help ensure that they succeed.

Keeping these improvements in place over the long term also requires an effective process for assessing contractor performance on mission support activities. We continue to have concerns about NNSA’s oversight approach. Under its proposed risk-based approach to federal oversight, NNSA would determine the risks associated with a given operation or function, evaluate how good the contractor assurance system is in that area, and also factor in past contractor performance. NNSA would take these factors into consideration to determine whether it could reduce federal oversight of an operation and rely more on the contractor’s assurance that the risk is being adequately addressed and controlled. In our view, such autonomy for the laboratories is inadvisable this soon into the process of recovery from a string of embarrassing revelations. Regardless of whether the University of California retains the contracts when they are competed in 2005 or another organization is selected to operate one or both of the laboratories, until the laboratories have demonstrated the maturity and effectiveness of contractor assurance systems and the adequacy of the contractor’s oversight has been validated, NNSA needs to maintain sufficient oversight of mission support activities to fulfill its responsibilities independently.

Recommendations for Executive Action

We recommend that the Secretary of Energy direct the Administrator of NNSA to ensure through contract and other management mechanisms that the University of California and any future contractor managing Los Alamos and Lawrence Livermore National Laboratories provide leadership, resources, and oversight to ensure effective mission support activities, including evaluating the impact of improvement actions on performance; ensure that NNSA performance assessments at the laboratories include evaluations of the adequacy of leadership, resources, and internal controls associated with mission support activities; ensure that as NNSA implements its proposed oversight and contractor assurance policy at Los Alamos and Lawrence Livermore National Laboratories, NNSA retains sufficient independent federal oversight of mission support activities to fulfill its responsibilities associated with protecting public resources and safety; and include in its contract with the University of California and any future contractor at Los Alamos and Lawrence Livermore National Laboratories a requirement that major improvement initiatives be managed consistent with the best practices of high-performing organizations, as defined in DOE Notice 125.1.

Agency Comments

We provided a draft of this report to NNSA and the University of California for their review and comment. The University provided its comments through NNSA. In written comments, NNSA’s Associate Administrator for Management and Administration generally agreed with the accuracy of the report and acknowledged that both NNSA and the University face challenges in improving mission support activities at the two laboratories.

NNSA also cited actions taken or planned that it said met the intent of our recommendations.

However, regarding the report’s accuracy, NNSA said our report substantially understates the extent of progress made in correcting the laboratories’ mission support problems. We believe that we have accurately described the progress made in implementing actions aimed at improving mission support. Even though the University has made progress in implementing corrective actions and new requirements, the extent to which these actions have resulted in improvements in mission support performance at the laboratories is still unclear.

In an attachment to the letter, NNSA raised a concern about our discussion of its efforts to oversee the laboratories and our recommendation concerning NNSA’s proposed risk-based approach to laboratory oversight. NNSA disagreed with our reservations about its proposal to rely more on a contractor’s system of management controls and less on NNSA’s own independent oversight. NNSA acknowledged that there have been problems with its oversight in the past but believes that its proposed risk- based approach will be successfully implemented, resulting in improved contractor oversight. Therefore, NNSA said that our recommendation to ensure that it retains sufficient independent oversight of the laboratories’ mission support activities was not necessary. However, as we discussed in our report, a risk-based approach to federal oversight appears reasonable in concept, but the University of California has not demonstrated that its contractor assurance systems can be relied on to prevent or detect fraud, waste, abuse, or mismanagement. And, in the past, NNSA has not been effective at detecting these weaknesses. Until improved performance in these areas has been clearly demonstrated, we continue to be concerned about whether NNSA can effectively implement a risk-based approach to contractor oversight. That is why we are recommending that NNSA retain sufficient independent oversight of mission support activities to ensure that those activities are safe and effective.

The attachment to the letter also discussed or referred to the report’s other recommendations. Regarding our recommendation that NNSA’s performance assessments at the laboratories include an evaluation of the adequacy of leadership, resources, and internal controls associated with mission support activities, NNSA said that the performance assessment process it began using in fiscal year 2003 already includes such an evaluation, so this recommendation is no longer required. For fiscal year 2003, the “critical few” performance measures used as a basis for evaluation of contractor performance did include maintaining secure, safe, environmentally sound, effective, and efficient operations and infrastructure in support of mission objectives. However, we do not believe that NNSA’s assessment of contractor performance on this measure is equivalent to evaluating the adequacy of leadership, resources, and internal controls associated with mission support activities. Therefore, we continue to believe that such an evaluation is an important part of NNSA’s oversight of contractor performance and that the recommendation is warranted.

Regarding our recommendations that NNSA ensure that any contractors operating the laboratories (1) provide the leadership, resources, and oversight to ensure effective mission support activities and (2) manage improvement initiatives consistent with best practices, NNSA was silent on the usefulness of the recommendations, but stated that the University is committed both to providing the leadership, resources, and oversight to ensure that mission support activities are conducted effectively and to ensuring that its improvement efforts continue to achieve the desired results. We believe that the oversight activities inherent in these recommendations are an important part of improving the management of mission support at the laboratories.

NNSA also provided technical comments, which we have incorporated as appropriate. NNSA’s written comments on our draft report are included in appendix II.

We conducted our review from May 2003 through February 2004, in accordance with generally accepted government auditing standards. Appendix I provides details on our scope and methodology.

As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to the Secretary of Energy and to the University of California Office of the President. We will also make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions on this report, please call me at (202) 512-3841. Other staff contributing to this report are listed in appendix III.

Scope and Methodology

We addressed three of the five problem areas NNSA identified when it extended the laboratory contracts with the University of California in January 2001. NNSA had identified problems with management accountability; safeguards and security management; facility safety, including nuclear facility operations; critical skills, knowledge, and technical capabilities; and project management. Based on discussions with committee staff, we reviewed three of these initiatives—management accountability, including steps taken by the University to strengthen its oversight of the two laboratories; facility safety, including nuclear safety and facilities management; and project management. The other two areas NNSA identified—safeguards and security management and critical employee skills—were outside the scope of our review. These topics are the subject of other GAO engagements.

To identify the status of actions taken by NNSA and the University of California to address the mission support problems that were highlighted when NNSA decided to extend the contracts for the Los Alamos and Lawrence Livermore National Laboratories in January 2001, we reviewed and analyzed prior GAO reports and testimonies, reports from the Inspector General, and other reports in order to determine the types of problems identified in the past. In addition, we reviewed and analyzed documentation on the two contracts, including the annual performance measures and NNSA’s annual evaluations of contractor performance. We reviewed DOE orders and other agencywide requirements that had been incorporated into the contracts, as well as the documentation provided by the laboratories that demonstrated compliance with these requirements. We interviewed officials in the University of California Office of the President, Laboratory Administration Office, to understand the contractor’s perspective and management role. In addition, we interviewed officials at the two laboratories and the NNSA site offices to determine actions they had taken or were taking to address identified problem areas. We also reviewed documentation obtained from DOE’s Office of Environment, Safety and Health regarding nuclear safety violations and similar problems. In addition, we viewed public meetings of the Defense Nuclear Facilities Safety Board (DNFSB), an independent oversight board charged with providing safety oversight of the nuclear weapons complex, and interviewed a DNFSB official at the Los Alamos National Laboratory to obtain DNFSB views on the progress made by Los Alamos in the area of nuclear safety. We also interviewed officials from DOE’s Office of Engineering and Construction Management regarding project management, the Office of Price-Anderson Enforcement regarding nuclear safety, and the Office of Environmental Management regarding environmental cleanup.

Lastly, we interviewed officials with the Department of Energy Inspector General at the Los Alamos and Lawrence Livermore National Laboratories and in Washington, D.C., to obtain additional information on actions taken to address problems and the progress that has been made.

To identify the status of the actions taken to address additional mission support problems that have emerged or become more significant since 2001, we took several steps. To obtain information on the problems with business operations at Los Alamos, we reviewed and analyzed reports by DOE’s Office of Inspector General and external reviews done by Price- Waterhouse Coopers, Ernst and Young, and others. We also reviewed and analyzed the April 2003 report from the Deputy Secretary of Energy and the NNSA Administrator on the business problems and Los Alamos and the recommendations to the Secretary of Energy. In addition, we interviewed officials with the Los Alamos laboratory, the NNSA Los Alamos site office, and the University of California Laboratory Administration Office on the status of corrective actions, and we reviewed documentation and reports they provided. To obtain information on the continuing problems with emergency management at the Lawrence Livermore laboratory, we reviewed reports of DOE’s Office of Independent Oversight and Performance Assurance, the annual performance measures in the contract for the Lawrence Livermore laboratory, and NNSA’s annual evaluations of contractor performance. In addition, we interviewed NNSA officials at the Lawrence Livermore site office and reviewed documents provided by them. We also reviewed the Fiscal Year 2004 Energy and Water Development Appropriations Act and corresponding House of Representatives and Conference Reports to understand the requirement to compete DOE contracts that had been awarded more than 50 years ago without competition.

To determine the remaining challenges, if any, that NNSA and the University face in sustaining improvements in mission support activities, we reviewed and analyzed Office of Management and Budget Circular A- 123, Management Accountability and Control; GAO’s Standards for Internal Control in the Federal Government; and DOE Office of Inspector General and Performance and Accountability reports. We also reviewed and analyzed University, laboratory, and independent reports obtained during site visits and during interviews with officials to identify other areas of concern and any potential barriers to implementing and sustaining the improvement efforts. We interviewed officials with the University Laboratory Administration Office, the Los Alamos and Lawrence Livermore laboratories, and the NNSA site offices to obtain their views on remaining challenges the laboratories face and the need for improved oversight of the laboratories. In particular, we discussed DOE’s oversight policies, NNSA’s proposed oversight policies, and challenges to improving oversight. We also reviewed documents relevant to oversight issues, such as NNSA’s draft policy letter on contractor assurance and oversight and the investigative report on the space shuttle Columbia accident, which was prepared for the National Aeronautics and Space Administration. Finally, for additional insight into remaining challenges, we analyzed the results of public meetings on DOE and NNSA oversight of nuclear safety held by the Defense Nuclear Facilities Safety Board.

We conducted our review from May 2003 through February 2004 in accordance with generally accepted government auditing standards.

Comments from the Department of Energy

GAO Contacts and Staff Acknowledgments

GAO Contacts

Acknowledgments

In addition to the individuals named above, Carole Blackwell, Doreen Feldman, Terry Hanford, Jonathan McMurray, Jill Peterson, Robert Sanchez, and Stan Stenersen made key contributions to this report.

GAO’s Mission

The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs