Nuclear Waste: Further Actions Needed to Increase the Use of Innovative Cleanup Technologies
Published: 1998-09-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the Department of Energy's (DOE) Office of Environmental Management's (EM) efforts to deploy innovative cleanup technologies, focusing on: (1) the extent to which innovative technologies developed by the Office of Science and Technology (OST) have been deployed at DOE sites and how this rate of deployment compares with the rates of other government organizations that develop environmental technologies; (2) what progress EM has made in overcoming obstacles to deploying innovative technologies at DOE cleanup sites; and (3) what EM is doing to increase the deployment of innovative technologies.
section_title: What GAO Found
paragraphs: GAO noted that: (1) OST has initiated 713 technology development projects and has reported that 152 projects have been deployed one or more times, for an overall deployment rate of 21 percent; (2) GAO found many errors in the office's deployment data and estimates that EM has deployed between 88 and 130 of these projects, for an overall deployment rate of 12 to 18 percent; (3) OST overstated its deployment information because it had not previously maintained comprehensive deployment data; compiled the data rapidly in response to congressional requests; and lacked procedures for compiling the data; (4) in comparison with the deployment rates of other programs that demonstrate environmental technologies--the Environmental Protection Agency's Superfund Innovative Technology Evaluation Program and the Department of Defense's Environmental Security Technology Certification Program--OST's deployment rate for projects at comparable stages of development falls between the rates of these two programs; (5) however, comparisons of OST's deployment rate with the rates of other organizations' programs must be viewed with caution because no organization is fully comparable with OST, and the deployment rate is not the only possible measure of success for research and development programs; (6) as DOE's Environmental Management program has matured, its waste cleanup sites have made progress in overcoming some obstacles to implementing innovative technologies; (7) other obstacles that are internal to the operations of EM and its OST continue to slow the use of innovative technologies, including the lack of: (a) involvement by technology users in the development of cleanup technologies by OST; and (b) technical assistance by OST to help sites select and implement technologies; (8) after congressional hearings in May 1997, EM initiated changes in its organization and processes to increase the deployment of innovative technologies; (9) some of these initiatives address the internal obstacles limiting deployment; (10) however, the office has not yet improved developer-user cooperation in individual technology development projects; (11) OST does not consistently and rigorously use its existing decisionmaking process for managing the development of innovative technologies; and (12) EM has yet to determine how it will: (a) provide technical assistance to sites in selecting and implementing innovative technologies; and (b) make modifications to completed technologies to meet sites' specific needs and conditions.

section_title: Letter
paragraphs: As requested, this report addresses the deployment rate for technologies developed by the Office of Science and Technology in the Department of Energy’s (DOE) Environmental Management program and compares this rate with those of other similar organizations. The report also addresses obstacles to the use of innovative technologies at DOE’s cleanup sites and the Environmental Management program’s efforts to overcome these obstacles.
As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 15 days after the date of this letter. At that time, we will provide copies of the report to interested congressional committees, the Secretaries of Energy and Defense, the Administrator of the Environmental Protection Agency, the Director of the Office of Management and Budget, and other interested parties. We will also make copies available to others on request.
subsections: section_title: Introduction
paragraphs: Nearly a decade ago, the Department of Energy (DOE) embarked on a mission to deal with the environmental legacy of the Cold War. This DOE mission, which is expected to continue for many years, involves a number of activities, with the most ambitious and far-ranging being the cleanup of the hazardous and radioactive contaminants that resulted from the production of nuclear weapons at DOE facilities. The challenges of this task are technical, institutional, and economic. For example, thousands of tons of radioactive waste must be treated and put into permanent storage; contaminated soil must be stabilized; contaminated water must be treated; and nuclear reactors and materials-processing facilities must be decontaminated, decommissioned, and demolished. In some cases, no safe and effective technology is currently available to address the more complex contamination problems.
In June 1998, DOE estimated that it will be very expensive, about $150 billion, to clean up the legacy of the Cold War. However, DOE has also stated that the cost of cleaning up its contaminated facilities and sites can be significantly reduced through the use of innovative cleanup technologies. It supports the development of these technologies through its Office of Science and Technology (OST), within the Office of Environmental Management (EM).
subsections: section_title: OST Established to Develop Innovative Technologies
paragraphs: In 1989, DOE established EM to clean up and restore its contaminated facilities and sites in compliance with federal and state environmental laws and regulations. The Congress also directed the Secretary of Energy to establish “a program of research for the development of technologies useful for (1) the reduction of environmental hazards and contamination resulting from defense waste, and (2) environmental restoration of inactive defense waste disposal sites.”
In response, DOE established the Office of Technology Development within EM to develop innovative technologies to support the waste cleanup and restoration efforts of EM’s program offices—the Offices of Waste Management, Environmental Restoration, and Nuclear Material and Facility Stabilization. The Office of Technology Development was renamed the Office of Science and Technology in 1994, when basic science research for waste cleanup was added to its responsibilities. OST’s projects are intended to produce technologies that could accelerate cleanups, reduce costs, enable cleanup activities for which there are no existing cost-effective technologies, or reduce risks to cleanup workers. From fiscal year 1990 through fiscal year 1998, the Congress appropriated approximately $2.5 billion for OST’s development of innovative waste cleanup technologies, and OST has initiated over 700 projects. OST’s budget for technology development activities in fiscal year 1998 is about $220 million. OST requested a total of $180.5 million for technology development activities for fiscal year 1999.
OST develops technology at DOE’s national laboratories, private companies under contract to OST, and universities. Although OST is responsible for technology development, DOE waste sites are responsible for selecting the technologies they will use, with the review and approval of the Environmental Protection Agency (EPA) and state agencies that regulate DOE’s cleanups, and with input from the public involved with the site.
To serve sites’ needs for cleanup technology, OST is organized into five major remediation and waste management problem areas (termed “focus areas”). OST first established focus areas in 1994 in order to better serve the cleanup sites by concentrating technology resources on each of the major cleanup problems DOE faces. OST currently has the following five focus areas:
Mixed Waste Characterization, Treatment, and Disposal. Known as “mixed waste,” this focus area addresses the large inventory of mixed, low-level, and transuranic waste;
Radioactive Tank Waste Remediation. Known as “tanks,” this focus area addresses the hundreds of large storage tanks containing over 100 million gallons of radioactive waste;
Subsurface Contaminants. This focus area addresses hazardous and radioactive contaminants in soil and groundwater and the remediation challenges posed by numerous DOE landfills;
Deactivation and Decommissioning. This focus area addresses the deactivation, decommissioning, and disposal of aging and contaminated DOE weapons complex facilities; and
Plutonium Stabilization and Disposition. This focus area addresses the over 20 tons of excess plutonium that must be stabilized.
OST has established a lead field office to manage each focus area. For example, the Savannah River site manages the Subsurface Contaminants Focus Area. EM has also established site technology coordination groups in each of its field offices to identify sites’ technology needs, provide information to OST and its focus areas, and communicate information about OST’s technology development projects to the cleanup sites.
In 1994, the Assistant Secretary for EM established the Environmental Management Advisory Board (EMAB) to provide the Assistant Secretary with information, advice, and recommendations on issues confronting the EM program, including advice on the development and deployment of innovative technology for waste cleanup. EMAB has about 25 members from industry; academia; and private, federal, tribal, state, and local environmental groups. EMAB has been very active in studying OST and recommending improvements in its operations.
section_title: Use of Innovative Technology May Have Benefits, Including the Reduction of Cleanup Costs
paragraphs: DOE believes it will be very costly and take many years to clean up its waste sites if only conventional technology is used. For example, using the conventional method of removing contaminants from groundwater can involve pumping and treating the water for 30 years or more. In addition, no technology exists to address some cleanup problems. For example, no technology exists for some aspects of removing and treating the radioactive waste now in large tanks at several major DOE facilities. Furthermore, some cleanup activities could be dangerous or impossible for cleanup workers unless innovative technologies, such as remote robotic devices to clean inside radioactive waste tanks, are used.
Those in the Congress and in DOE who led the effort to establish OST believed that the use of innovative technology would reduce the cost of waste cleanup. For example, in 1995, DOE estimated that it would cost between $200 billion and $350 billion and take another 75 years to complete the cleanup. However, DOE also estimated that the use of new technologies could reduce cleanup costs by a minimum of $9 billion to as much as $80 billion, depending on the cleanup scenario. More recently, in 1997, the Army Corps of Engineers reviewed cost savings estimates developed by OST for 37 of its technology projects and concluded that these 37 projects could potentially save about $20 billion over the use of conventional technology. DOE believes that cleanup costs could significantly exceed current estimates if innovative technology is not used.
section_title: GAO Has Reported in the Past on Problems in Managing Technology Development and Deployment
paragraphs: We have issued a number of reports and testified on the operation and management of EM’s technology program. Among other things, we have identified obstacles to the deployment of innovative technology at DOE’s cleanup sites. In 1992, we reported that EM had not established key management tools, such as cost estimates and schedules, and decision points for evaluating technology projects. In January 1993, EM implemented a management plan for the technology program that incorporated our recommendations. The program established cost estimates and schedules for projects. EM also developed decision points (called gates) and related requirements for evaluating projects and making “go/no-go” decisions.
In 1994, we reported that officials at DOE cleanup sites may not be familiar with innovative technologies and may fear that using new technologies may lead DOE to miss cleanup deadlines if the technology fails to perform as expected. In response to our report, OST took several actions, including establishing the site technology coordination groups discussed earlier, to improve communication on sites’ technology needs and the capabilities of newly developed technologies. In addition, to help ensure that development activities were concentrated on the most pressing cleanup needs, EM restructured its technology development program into the focus areas.
In 1996, we reported that EM had not coordinated technology development to prevent duplication of effort, particularly between OST and the Office of Waste Management, which together had 60 projects to develop equipment to melt and immobilize waste. A key reason for the duplication was EM’s lack of a comprehensive list of technology development projects. EM subsequently developed a list of its technology projects. We also found that more technology projects were being started at the sites where the focus areas were physically headquartered. In following up on this situation in 1997, we found that this concentration had decreased.
In 1997, we testified before the Subcommittee on Oversight and Investigations, House Committee on Commerce, that OST appeared to have made some improvements in its project management, but we had continuing concern about the extent of use of OST-developed technologies at DOE’s waste cleanup sites and the validity of OST data on deployments and expected cost savings. OST had also proposed a new initiative, Accelerated Site Technology Deployment (ASTD), to facilitate the use of its technologies. We expressed several concerns about the likely effectiveness of this initiative, which provides funding to DOE sites for the first use of an innovative technology. OST provided a total of approximately $26 million to 14 ASTD projects in fiscal year 1998.
section_title: Objectives, Scope, and Methodology
paragraphs: The Chairman and Ranking Minority Member of the House Committee on Commerce and the Chairman and Ranking Minority Member of its Subcommittee on Oversight and Investigations asked us to review EM’s Office of Science and Technology. Specifically, we were asked to determine (1) to what extent innovative technologies developed by OST have been deployed (used) at DOE sites and how this rate of deployment compares with the rates of other government organizations that develop environmental technologies; (2) what obstacles exist to deploying innovative technologies at DOE sites; and (3) what EM is doing to overcome obstacles to deploying innovative technologies.
To determine the extent to which OST-developed technologies have been deployed at DOE sites, we obtained deployment information on OST’s projects from an OST management information system. This information provided project names and numerical identifiers, research stage, deployment sites (if any), and other project information, as of January 1998. We also obtained information about OST’s use and definition of the term “deployment” and OST’s procedures for entering and updating the information in this system. In order to assess the accuracy of OST’s deployment data, we used a random sample of the projects that OST listed as deployed, and we verified the claimed deployments with site operations officials. Upon finding a significant error rate, we used our sample results to estimate a range for the actual number of OST project deployments. The methodology for our verification is described in appendix I.
To compare the rate of deployment for OST’s technologies with the rates of other government organizations that develop environmental technologies, we used database searches and contacts with federal agency officials to identify federal government programs that develop environmental technologies. We contacted the eight government programs whose research and development work was most comparable to OST’s in mission and scope, and two of these programs—EPA’s Superfund Innovative Technology Evaluation Program and the Department of Defense’s Environmental Security Technology Certification Program—were able to provide deployment data for comparison with OST’s data. We also contacted two private sector organizations that develop environmental technologies but found that they did not maintain deployment data. Because the two federal programs providing deployment data conduct technology demonstrations but not earlier phases of research and development, we identified OST projects that had reached a similar stage of maturity to provide an equitable group for comparison. (See app. II for a detailed discussion of these two programs.)
To identify obstacles that exist to deploying innovative technologies at DOE sites, we first reviewed past reports on this subject by GAO, DOE, and advisory groups to DOE. In order to obtain more current and specific information about obstacles to deployment and EM’s progress in overcoming them, we visited five DOE sites: Hanford (Washington State), Savannah River (South Carolina), Oak Ridge (Tennessee), Fernald (Ohio), and Lawrence Livermore National Laboratory (California). We selected these sites to provide varied perspectives: The first three sites are among the largest DOE cleanup sites; Fernald is far along in its cleanup efforts and represents a medium-sized cleanup effort; and Lawrence Livermore has a smaller cleanup effort and budget. For the site visits, we identified OST-developed technologies that were either selected for use at the site, considered for use but not selected, or potentially applicable to the site’s cleanup problems. We identified technologies to discuss with site officials from our meetings with managers of OST’s focus areas, records maintained by EM’s Office of Environmental Restoration, and discussions with headquarters EM officials. The technologies were judgmentally selected to provide coverage of (1) EM’s various cleanup challenges and the related OST focus areas and (2) innovative technologies selected or not selected for use.
We discussed with EM field personnel and contractor staff the obstacles they faced to using particular technologies, the ways they addressed and overcame these obstacles (for deployed technologies), and the reasons they did not select the technologies. We discussed a total of 30 OST-developed technologies with one or more of the five sites and obtained documentation on related selection decisions. We analyzed this information to identify (1) commonly cited obstacles to deployment and (2) the means by which sites overcame these obstacles in those cases in which OST-developed technologies were selected or in use.
In order to identify EM’s actions to overcome deployment obstacles, we reviewed a memo from the Assistant Secretary for EM that directed a number of actions to increase deployment, and we obtained information about the status and results of these actions. We also interviewed OST managers to identify additional actions under way within OST and obtained related documentation. To assess the adequacy of these actions, we compared the EM and OST actions to the obstacles to deployment that we had identified.
We provided a draft of this report to DOE for its review and comment, and a draft of chapter 2 and appendix II to the Department of Defense and EPA for their review and comment. DOE’s and the Department of Defense’s written comments and our responses are included in appendixes III and IV.
We performed our review from August 1997 through September 1998 in accordance with generally accepted government auditing standards.
section_title: EM Has Deployed Less Than 20 Percent of OST’s Projects
paragraphs: While OST has initiated 713 technology development projects, we estimate that EM has deployed between 88 and 130 of these projects, for an overall deployment rate of 12 to 18 percent. In contrast, OST has reported that 152 projects have been deployed one or more times, for an overall deployment rate of 21 percent. OST’s overstated deployment information is the result of several factors, including its rapid compilation of deployment data in response to congressional requests and the lack of a formal definition of what constitutes a deployment.
Most organizations we contacted, including some private technology developers, did not track deployment data comparable to OST’s. We contacted eight government programs and two private sector programs engaged in environmental technology research and found only two that could provide data on deployment. In comparing data from these two organizations and OST, we found that OST’s deployment rate was close to that of the 2-year-old Environmental Security Technology Certification Program (ESTCP) in the Department of Defense and somewhat lower than the rate for the 12-year-old Superfund Innovative Technology Evaluation (SITE) program in EPA. However, it is important to recognize that the value of deployment rate comparisons with other organizations is limited. To assess the overall performance of a research and development (R&D) program like OST, other measures in addition to deployment would be relevant.
subsections: section_title: EM Has Deployed About 88 to 130 of OST’s Technology Projects
paragraphs: OST developed deployment data in response to a November 1996 request from the Subcommittee on Oversight and Investigations, House Committee on Commerce. As of January 1998, OST’s database showed that EM had initiated 713 technology development projects since OST’s inception. On the basis of our verification and analysis of these data, using a 95-percent confidence interval, we estimate that EM actually deployed from 88 to 130 projects, to achieve an overall deployment rate of 12 to 18 percent for the 713 projects. In contrast, OST has reported that 152 of the 713 technology projects initiated since its inception have been deployed. Thus, according to OST’s data, about 1 in 5 OST technologies have been deployed one or more times, for an overall deployment rate of 21 percent.
We also found that OST had overstated the number of deployment instances reported for each technology project. OST’s database listed a total of 283 deployment instances for the 152 projects claimed as deployed. We estimate that of the 283 deployment instances claimed by OST, only 137 to 216 have actually occurred. Table 2.1 lists OST’s data, the error rates we found, and our estimates of actual deployments based on the error rate found in our sample of 30 projects.
section_title: Several Factors Have Led to OST’s Inaccurate Deployment Data
paragraphs: OST’s inaccurate deployment data resulted from several factors. Specifically, OST compiled deployment data quickly, in response to a congressional request that came 7 years after the program’s inception, because it had not previously maintained comprehensive data. In addition, the lack of a formal definition for deployment led to differing understandings among the focus area personnel responsible for compiling the data. Finally, OST has begun only recently to establish procedures for entering and updating project data. If such procedures had been in place early on, they would have uncovered the need to formalize the definition of deployment.
subsections: section_title: Initial Compilation of Deployment Data
paragraphs: Some inaccuracy in OST’s deployment data may have been due to the fact that the data were compiled quickly. OST prepared deployment data in response to a November 1996 request from the Chairman of the House Committee on Commerce. Previously, OST had not maintained comprehensive deployment data on its projects. Instead, OST tended to focus its performance measures on completed demonstrations. For the November 1996 request, OST gathered deployment data for its projects over a period of several months and provided the information to the Chairman in April 1997. At that time, OST reported that 150 projects had been deployed and an additional 41 projects had been selected for use in the future (for a total of 191 past and future deployments).
section_title: Lack of a Formal Definition for Deployment
paragraphs: Another reason for OST’s inaccurate data has been the lack of a formal definition of deployment, leading to different understandings among the focus area personnel who collected the deployment data about what should be counted as a deployment. According to OST managers, while gathering data to respond to the Committee’s request, OST headquarters officials told focus area personnel to refer to an earlier definition of implementation for the meaning of deployment but did not distribute new written guidance. This definition, which OST had formalized and distributed in April 1996, defined implementation to mean that the technology was used or selected for use to meet specified user performance measures (e.g., completion of an assessment or treatment of waste for disposal). However, officials of the Subsurface Contaminants Focus Area provided us with a definition of deployment that they received along with the instructions for responding to the Committee’s data request. This definition stated that the number of deployments means the number of “hot” demonstrations (that is, demonstrations in radioactive environments) and that deployment site means the location of a hot demonstration.
We found that OST focus area personnel entering the data frequently regarded demonstrations as deployments. For example, OST counted as a deployment the use of a characterization technology called Laser Ablation/Mass Spectroscopy at the Pacific Northwest National Laboratory in Washington State. In response to our questions, site contractor officials stated the technology’s use at the laboratory had been a demonstration in which data derived from the laser technology were compared with data derived from a conventional technology. At this time, the site cannot rely upon the laser technology to accomplish its goals for characterization.
section_title: Status of OST Efforts to Improve Data Quality
paragraphs: While OST has issued a definition of deployment and is taking other steps to improve data quality, written procedures for data verification have not yet been developed. In August 1998, the Acting Deputy Assistant Secretary for OST issued a memo that formally defined deployment. The definition appropriately emphasized that deployments must accomplish site objectives, such as the completion of assessments, cleanups, or the treatment and disposal of wastes. The memo stated that this definition is to be used for performance measurement. OST has also completed a data verification effort for those projects considered deployed during fiscal year 1997. It used verification by site personnel and other data sources to improve the accuracy of this portion of the data.
According to OST officials, the office intends to continue similar verification efforts in the future. However, these data verification plans are not reflected in OST’s draft procedures for its database, which do not specify a method of data verification. The procedures, drafted in January 1998, identify OST’s focus areas as responsible for entering data and ensuring their quality and completeness. These procedures also require that the data on ongoing projects be updated at least once per quarter (every 3 months). However, the draft procedures do not identify any means by which the data are to be verified or spot-checked for accuracy. While site technology coordination groups can comment on the deployments listed in the database, the procedures do not state any requirement for data review and concurrence by these groups or other site officials. The Acting Deputy Assistant Secretary for OST told us that OST plans to obtain further advice about verification methods and then develop written procedures.
OST has not yet determined whether, or to what extent, to verify data from the years prior to 1997 because of the time and resources involved. According to the Acting Deputy Assistant Secretary for OST, the office is seeking clarification from the House Committee on Commerce on the degree of accuracy or certainty needed.
section_title: Deployment Rate for OST Technologies Falls Between the Rates of Two Other Environmental Technology Programs
paragraphs: We compared the deployment rate for OST’s technologies with the deployment rates for technologies sponsored by EPA’s SITE program and the Department of Defense’s ESTCP. The SITE program is engaged solely in the environmental technology demonstration and implementation stages of R&D. Similarly, ESTCP demonstrates and validates technologies and funds environmental technologies that have progressed to the stage at which field demonstrations are warranted. Taking into account the limitations of this comparison, OST’s deployment rate for projects at comparable stages of development falls between the rates of the two organizations that provided data, as shown in table 2.2. (App. II discusses in detail how we developed each comparison.)
section_title: Deployment Rate Comparisons Between OST and Other Environmental Technology Developers May Have Limited Usefulness
paragraphs: Comparisons of OST’s deployment rate with the rates of other organizations must be viewed with caution when assessing how well EM is doing in deploying OST-developed technologies. We found few organizations that engage in the range of environmental research OST performs, and no organization we contacted routinely tracked deployment data on its projects. Data provided by the two organizations differed widely in source and composition. Finally, many individuals we contacted question whether a deployment rate is a sufficient benchmark for successful R&D.
subsections: section_title: A Small Number of Environmental Technology Development Programs Maintain Deployment Data
paragraphs: Most organizations we contacted, including some private technology developers, did not track deployment data comparable to OST’s. Of the eight government programs and two private sector programs engaged in environmental technology research we contacted, only the SITE program and ESTCP could provide data on deployment. Even these two programs needed to compile their information so that it could be expressed as deployment rates. Table 2.3 shows the entities that we contacted.
Furthermore, we found that only one of the other government programs listed in table 2.3 engaged in nearly the full range of environmental R&D that OST performs. OST’s R&D includes basic science research, applied research and engineering development, field testing and demonstration, and implementation by the end user (commercialization). Most of the governmental organizations we contacted performed either the early stages of R&D or the later stages, but not both. Technology development efforts undertaken at the early stages have more unknowns and are likely to involve a greater risk of failure than efforts at the later stages. Since we would expect performance results to differ for each stage, meaningful comparisons can only be made among projects or programs that are at similar stages of R&D maturity.
Two organizations provided us with very different types of data. EPA’s SITE program had accumulated survey data on the number of contracts their technology vendors had obtained over about 8 years. We agreed that a contract for use could be considered deployment of the technology. As to be expected, the survey responses were less than 100 percent, unlike the OST and ESTCP data, which include all of these agencies’ technology projects. Therefore, the data from EPA’s SITE program are incomplete, and the deployment rate for SITE could actually be higher. The Department of Defense’s ESTCP provided a description of the transition (deployment) status for all of its projects from the program’s first 2 years of existence. Since ESTCP is a relatively new program, its deployment data are based on a limited number of projects and may be less representative of the program’s future performance. We did not verify the accuracy of these organizations’ deployment data, but we reviewed their available project summaries and believe the organizations’ approaches were reasonable responses to our request. Nevertheless, differences in how the programs defined deployment, and whether they counted incomplete projects, will affect computed rates.
section_title: Deployment Rate May Be an Incomplete Measure of OST’s Performance
paragraphs: As we have previously reported, measuring the performance of R&D programs is difficult. Performance measures used in other federal R&D programs include the scientific peer review of projects, numbers of patents issued, and studies of publications. Recent R&D management literature suggests that certain measures, such as the number of patents issued, are best suited to earlier stages of research, while outcome measures, such as deliverables and customer satisfaction, are more relevant for later-stage research. In this context, a deployment rate measure would be most useful when applied to more mature projects. At the same time, program managers need to assess how successful the program has been at selecting early-stage projects with high potential for future payoff.
Officials in a number of programs we contacted told us that deployment has only recently been raised as a possible performance measure. Furthermore, programs performing earlier stages of R&D were less likely to have any deployment data. Developers of later-stage technologies believed that the deployment rate is an incomplete performance measure, and that cost savings or some measure of dollar impact should also be used to evaluate program success. EM is considering developing a performance measure that would assess cost savings from the use of innovative technologies.
EM and OST recognize that deployment is not the only relevant measure of success in technology development. We reviewed performance measures established for OST for fiscal years 1994 through 1997 and found that completing demonstrations of technologies and the number of technologies made available for use—that is, number that have completed development—were the main performance measures used. In fiscal year 1998, OST’s performance measures are (1) demonstrate 35 new technologies, (2) make 40 alternative technologies available for use with cost and engineering data, and (3) perform 49 deployments of new technologies. As described in more detail in chapter 4, performance measures for deploying innovative technologies are also being applied to EM’s field operations offices in fiscal year 1998, and OST is considering developing additional performance measures for its focus areas that address technologies in various stages of development.
section_title: Some Obstacles Continue to Slow Deployment of Innovative Technologies
paragraphs: As EM’s cleanup program has matured, several of the obstacles to using innovative technologies reported previously by us, EM, and others have been addressed. For instance, DOE sites and their regulators have improved their working relationships, and, in cases where innovative technologies were selected, DOE sites have found ways to address regulator concerns about whether these technologies will achieve required objectives. However, some obstacles, internal to EM and OST program operations, continue to slow the deployment of innovative technologies, and, in some cases, have led OST to spend millions of dollars for technologies that the cleanup sites do not want. The most significant and continuing of these internal obstacles has been EM’s and OST’s failure to involve users sufficiently in the design and development of technology targeted for use at the cleanup sites. As a result, OST has developed generic technologies that do not meet site-specific needs or that require modification to make them usable by the site. However, EM has not clearly defined responsibilities and funding sources for modifying technologies among OST and potential technology users. Furthermore, OST still has no clearly defined role in helping sites select the appropriate technology and infrequently provided technical assistance in the cases we reviewed.
Several factors contribute to these problems. First, prior to 1996, OST had not comprehensively assessed users’ technology needs and linked these needs with technology development efforts. Second, OST has not fully implemented its system for monitoring, and if necessary, modifying or terminating ongoing technology development projects—a system that would require interaction with technology users.
subsections: section_title: Sites Have Overcome Obstacles to Using Some Innovative Technologies
paragraphs: DOE’s field and contractor staff face a number of challenges when attempting to use an innovative cleanup technology. Past reports by us, EM, and advisory groups have catalogued the challenges: the perceived risks of exceeding projected costs or failing to meet time schedules; the need to convince regulators and stakeholders of the advantages of innovative technology; and technical problems, including the need to modify a technology to make it fit a specific situation. However, as the EM technology program has matured and site personnel, regulators, and stakeholders have become more aware of the benefits of using some innovative technologies, some obstacles have diminished in importance. Furthermore, when the use of a new technology is clearly and significantly advantageous, cleanup sites make a strong effort to overcome any obstacles to its use.
Specifically, when regulators and stakeholders are concerned about a technology’s effectiveness, sites have provided additional data or testing and, occasionally, modified technology to satisfy some concern or implemented an innovative technology in phases to obtain performance data. For example, according to Hanford officials, using a new technology to encapsulate certain carbon-based waste would be much less costly than incinerating it. State regulations, however, called for incinerating such waste. Nonetheless, Hanford persisted and obtained a waiver from the state to encapsulate the waste. At Oak Ridge, DOE and its contractors wanted to use a frozen soil barrier to contain a relatively small pool of water that had been contaminated with reactor waste. However, regulators and stakeholders were skeptical that this innovative technology would work and be cost effective. Oak Ridge demonstrated the technology to obtain cost and performance data and provided this information to regulators and stakeholders. The technology has since gained wide acceptance by these groups.
Some technology may have to be modified to satisfy regulatory concerns. For example, Hanford officials wanted to test an innovative technology for cleaning up contaminated soil, which they believed was better than current methods. However, regulators were concerned about the possible expulsion of carbon tetrachloride contaminants into the air. Hanford officials convinced the regulators to allow them to experiment with the new technology by offering to add a filter to the equipment to catch any contaminants. The modification was a low-cost and easy addition to the equipment.
In some cases, sites implement a technology in phases to obtain performance data and to assure themselves and convince regulators and stakeholders of the technology’s viability. For example, OST funded the development of a robotics device, called Houdini, that could help clean up waste in tanks. Oak Ridge, with the help of the manufacturer, adapted Houdini to help clean up radioactive waste stored at the bottom of the site’s large tanks. However, because Houdini had never been used to clean up radioactive waste, no information was available on the device’s performance and reliability. Oak Ridge therefore had to implement Houdini in phases using nonradioactive “cold” testing; followed by treatability tests in a lower radiation environment; and finally, “hot” testing on the radioactive waste in its tanks.
Field officials also told us that the projects in which OST and an EM operating group get involved as a joint venture seem to work well. In these cases, OST provides funding and some technical assistance, and the operating group also provides funding and implements the project. If there are also partners from industry, they further enhance the chances for success. For example, at Hanford, OST and EM’s environmental restoration group are participating in a large project to demonstrate a number of technologies that can be used to put Hanford’s old, shutdown reactors into safe interim storage. Hanford officials were convinced that if the demonstrated technologies were successful, the time needed to prepare the reactors for storage could be cut by 7 years.
The demonstration project started in 1996, with contributions totaling about $8 million from OST and about $16 million from the environmental restoration group. However, the project did not have the extra money to make needed refinements and modifications to technologies. Consequently, Hanford officials suggested partnering with private contractors who would assume the risk and cost of getting the technologies to perform. OST’s Deactivation and Decommissioning Focus Area, which routinely works with the private sector, helped to bring about this partnership with private contractors. As of July 1998, the project had successfully demonstrated 20 technologies and deployed 13 of them at Hanford’s C Reactor, two other Hanford reactors, and a number of other DOE reactors throughout the complex. In addition, the technologies have been transferred to the commercial reactor sector and will be used to help put the nuclear power plant in Chernobyl, Ukraine, into safe storage.
section_title: OST Has Developed Innovative Technologies Without Sufficient User Involvement
paragraphs: Despite the progress that has been made, some obstacles internal to EM and OST operations continue to slow the deployment of innovative technologies. In particular, OST has developed technologies that tend to be generic solutions to cleanup problems and, if usable at all, have to be modified to fit a site’s specific problem. These problems occur in part because OST had not, until 1996, comprehensively assessed the technology needs of the cleanup sites and has not involved potential technology users in the development of technology that might be used to address specific cleanup problems. Without user involvement, there have been no identified customers for some of the technology that OST has sponsored. For example, of the 107 technologies that OST has completed, 31 technologies, costing $71 million to develop, have not been used by cleanup sites.
subsections: section_title: Cleanup Sites Charge That Technology Does Not Meet Their Needs
paragraphs: According to EM field and contractor personnel responsible for waste cleanup, in many cases, OST technologies do not meet their needs. They said that OST has many times assumed that “one-size-fits-all” and therefore has developed generic solutions to cleanup problems. However, these solutions either do not fit a site’s specific needs or must be modified before they can be used. For example, Fernald workers needed portable equipment that would allow them to characterize contamination within buildings without climbing ladders to obtain samples from contaminated areas. OST said that, although its laser-induced fluorescence imaging equipment had not been field-tested, the equipment had been designed to meet needs such as Fernald’s. However, when Fernald workers attempted to use the equipment, they found that it was not ready for field use. It was cumbersome (not really “portable”) and light interfered with measurement readings. As a result, the equipment was not usable and was returned to the manufacturer for modifications. Consequently, Fernald personnel continued to take samples from the contaminated building areas by hand. Although they realized that the OST equipment had not been thoroughly tested before they tried it, Fernald officials said, they believed that if OST had involved them in the design and development of the equipment, the problems would have been avoided, or at least identified and corrected earlier.
Similarly, officials at DOE’s Hanford site tried two OST technologies that promised to support faster remediation of contaminated soil but had to reject them because they were not designed to work in Hanford’s arid soil. The officials said that the concept for faster remediation of contaminated soil was attractive and probably would have been acceptable to Hanford’s regulators, but the generic design of the technologies did not meet Hanford’s specific needs.
Furthermore, some site officials said that they would like to use some OST technologies, but the technologies require modification to fit the site’s situation. They pointed out that it is not clear who should make and pay for these modifications. For example, a project manager at DOE’s Savannah River Site told us that he would like to use more innovative technology in his projects, but it is unclear who is responsible for making site-specific modifications, and his program does not have funding to make such modifications. At Hanford, officials were interested in using OST’s Electrical Resistance Tomography to help detect leaks in their high-level radioactive waste tanks. (Hanford has 67 known or suspected leaking tanks.) However, a Hanford official said that the technology needed substantial fine-tuning to make it work on the Hanford tanks, and no funding was available to pay for this. He said that it was unclear who is responsible for funding modifications to OST technologies.
When only minor, inexpensive modifications are required, site representatives said that they have made and usually paid for them. But other technologies that are of interest to sites would require more extensive and more expensive modifications. Without a clear policy on who is responsible for modifying the technology and paying for the modification, the sites are likely to reject the innovative technology and select a known alternative.
section_title: User Needs for Technology Not Fully Assessed Until Recently
paragraphs: Until its reorganization in 1994, OST did not involve the cleanup sites in identifying technologies that need to be developed and did not conduct comprehensive needs assessments until 1996 and 1997. Therefore, most of the technologies developed through OST were not based on a comprehensive assessment of the technology needs of those responsible for cleaning up DOE waste sites. Instead, OST consulted with its developers at the national laboratories in deciding which technologies it would sponsor to solve sites’ cleanup problems. These technical solutions, according to potential technology users, tend toward the “one-size-fits-all” development philosophy.
We reported in 1994 that technology needs had not been comprehensively identified to allow prudent research decisions nor had various environmental program offices in headquarters and in the field worked together effectively to identify and evaluate all of the possible technology solutions available. In 1995, and again in 1996, the Environmental Management Advisory Board (EMAB) told the Assistant Secretary of Environmental Management that the lack of a comprehensive assessment linking identified needs with technology development efforts was a “primary barrier” to technology deployment. EMAB said that technology development and deployment must be linked together as a single system.
Site technology coordination groups, established in 1994, made early attempts to assess the needs of potential technology users. However, because OST considered data from these early surveys unreliable, it and the site groups developed guidance and worksheets for a more comprehensive assessment, which the site technology coordination groups carried out in October 1996. In October 1997, an updated needs assessment and a database that matches technology needs with appropriate existing technology or the future efforts of technology developers was completed, according to the director of OST’s Office of Technology Systems.
section_title: OST Has Not Involved Users in Technology Development
paragraphs: In addition to not involving the cleanup sites in identifying technology needs, OST has not sufficiently involved users in designing technologies and monitoring their development to help ensure that they meet users’ needs. In 1992, we recommended that EM institute a technology development management system with explicit decision points at which the technology would be assessed to determine whether development should continue or be terminated. OST established its “Technology Investment Decision Model” (called the “gates system”) to do this. The gates system satisfies our 1992 recommendation and was intended to be “a user-oriented decision-making process for managing technology development and for linking technology-development activities with cleanup operations.” However, OST has not fully implemented the gates system and thus cannot be certain that appropriate technology is developed to meet the needs of DOE’s cleanup sites.
Under OST’s gates system, the focus areas are to assess a technology’s development at six stages, from basic research through implementation. At each stage, the focus area is to make a go/no-go decision, with input from potential users. The critical decision points include the following:
Gate 1: Entrance Into Applied Research Stage. To pass through gate 1 and enter this stage, a proposed technology must be shown to address national interests and priority environmental needs. EM guidance states that if a technology does not address a specific need, it should not pass through gate 1.
Gate 2: Entrance Into Exploratory Development Stage. To pass through gate 2 and enter this stage, a technology has to be linked with the specific needs of an identified user.
Gate 3: Entrance Into Advanced Development Stage. To pass through gate 3 and enter this stage, the technology must be able to meet an identified user’s specific performance requirements. In addition, it must be documented that the research to develop the technology is expected to produce results consistent with the user’s time frame for deployment and implementation.
Gate 4: Entrance Into Engineering Development Stage. To pass through gate 4 and enter this stage, the technology must be shown to meet the user’s specific needs in a timely manner. In addition, it must be documented that the proposed innovative technology will be more cost-effective than current methods or other emerging technology.
Gate 5: Entrance Into Demonstration Stage. To pass through gate 5 and enter this stage, the identified user for the technology must make a commitment to deploy the technology if it meets performance requirements. In addition, the user must agree to share in the cost of and the responsibility for demonstrating the technology.
Gate 6: Entrance Into Implementation Stage. To pass through gate 6 into implementation, the technology must successfully complete a “real world” demonstration, either at a DOE site or another location, using actual waste streams and/or anticipated operating conditions. In addition, it must be documented that the technology has proven to be viable, cost-effective, and applicable to the users’ needs.
As this discussion of the gates system shows, OST’s focus areas must identify a user for the technology in the early stages of development. Furthermore, this user must stay involved throughout the development process to ensure that the technology will meet the needs and implementation schedule of the user.
OST, however, has not fully implemented its gates system to involve potential users in the assessment of technology that it is developing, and, in some cases, OST has not identified an end user for the technology. Furthermore, a review by EM and EMAB representatives, completed in late 1997, revealed that OST’s focus areas do not consistently use the gates system and do not consistently involve potential technology users in technology development decisions.
EMAB has pointed out in numerous reports that OST has failed to rigorously apply the gates system. EMAB has stated that OST should use the gates system to identify and terminate technologies that have no identified customer, are not cost-effective, or have limitations that may increase the risk of failure when used. According to the Chairman of EMAB’s Committee on Technology Development and Transfer, OST officials told him that they did not rigorously apply the gates system because it yielded results that OST and technology developers at the laboratories did not like—that is, indicating that some technology projects should be terminated.
Similarly, representatives of one of OST’s focus areas told us that OST does not rigorously use the gates system because it would force OST to terminate technologies that have no identified customer, do not meet users’ needs, are technically limited, or have some other fault. The manager of the Subsurface Contaminants Focus Area told us that his focus area had rigorously applied the gates system and terminated some technologies, which led to a confrontation with the laboratories developing the technologies.
The Director of OST’s Office of Technology Systems told us that the gates system was never fully implemented because staff were confused by other evaluations and OST’s reorganization into focus areas, which were taking place at the time the gates system was instituted. He said that the gates system was currently not being used but would be reinstituted in the future. According to the Acting Deputy Assistant Secretary of OST, the criteria of the gates system are still valid, but when focus areas tried to use the gates system, their approach was inappropriate and did not work. Specifically, he stated that focus areas set up panels to periodically review their projects according to the gate criteria. Instead, the gates system was intended to be used on an ongoing basis, so that the focus areas could determine whether requirements for the various stages of technology development, including user involvement, had been met. According to this official, OST’s intent was that technology developers and technology users have frequent interaction.
section_title: OST Has Not Provided Some Needed Technical Assistance
paragraphs: OST has not fulfilled its role of providing technology users with the technical advice and assistance that they need to identify solutions to cleanup problems and to help implement those solutions. Focus areas’ ability to provide technical help varies widely, although this was a principal mission when these groups were established in 1994. Some site officials responsible for cleanup told us that they are reluctant to try new technologies without a reliable source for advice and assistance, but some are reluctant to seek help from the focus areas because they do not trust the focus areas’ abilities.
subsections: section_title: Focus Areas’ Capability to Provide Technical Assistance Varies
paragraphs: EM established the focus areas in part to provide technology users with technical advice and assistance. However, EMAB has consistently noted the lack of technical knowledge in some focus areas and suggested that this problem be addressed. Similarly, we found that cleanup sites are skeptical of the technical expertise of some focus areas and rarely call upon them for assistance.
EMAB believes that the focus areas need to become experts not only in OST-sponsored technology but also in other domestic and foreign technology that might help solve waste cleanup problems. EMAB reported in January 1998 that some focus areas do not know the state-of-the-art technology for their area. The Chairman of EMAB’s Committee on Technology Development and Transfer told us that the Tanks Focus Area and the Deactivation and Decommissioning Focus Area seem capable, but he said that EMAB is concerned about the capability of the Subsurface Contaminants Focus Area, which has the largest workload by type of waste problem.
During our visits to five cleanup sites, we found that the sites infrequently sought technical assistance from OST and its focus areas. Site officials said that technical assistance would be helpful in deploying new technologies, but some are not convinced of the focus areas’ technical expertise. Furthermore, they preferred to go directly to a vendor for technical assistance because the vendor was much more knowledgeable than OST.
section_title: Sites Are Reluctant to Allow OST a Role in Major Technology Decisions
paragraphs: In 1994, we recommended that OST be given a formal role in sites’ selections of technologies to solve cleanup problems. For example, OST could formally take part in sites’ feasibility studies to identify and analyze technologies that could potentially solve a specific waste cleanup problem and to help a site decide which technology to use. However, some site officials told us that OST and its focus areas are not familiar enough with their sites’ waste cleanup problems and appropriate solutions. They said that our recommendation was not taken because site officials are skeptical of OST’s ability to provide quality technical advice and assistance and therefore are reluctant to allow OST more of a role in selecting cleanup technologies for their sites.
The Acting Deputy Assistant Secretary for OST told us that he is aware of this problem and has directed the focus areas to become more technically competent and supportive. He said that providing technical assistance should be routine for the focus areas; they should be out in the field providing this help, not waiting in the office for the sites to call them. He emphasized that if the focus areas are not able to provide expert technical assistance, he will look to other groups, perhaps the national laboratories, to provide needed technical assistance.
section_title: EM’s Efforts to Increase Deployment Do Not Fully Address Cooperation Between Technology Developers and Users
paragraphs: EM management devoted little attention to the deployment of innovative technologies until a congressional oversight hearing in May 1997 criticized EM’s performance in deploying technology. Following the hearing, the Assistant Secretary of EM issued a memorandum in July 1997, directing OST and other EM offices to initiate specified actions designed to facilitate technology deployment. Some of these actions have already been completed, and the remainder were to be completed by September 30, 1998. These actions establish responsibilities, require the development of performance measures for technology deployment, establish the Technology Acceleration Committee of upper-level EM and field managers, require sites to develop deployment plans, and continue the Accelerated Site Technology Deployment program that funds individual projects. OST has additional initiatives under way, including establishing technology-user steering committees and developing multiyear plans for technology development.
However, EM’s efforts only partially address the internal obstacles limiting deployment. On the positive side, EM has established deployment performance measures for field sites and required sites to develop deployment plans. Users’ involvement in developing overall plans and priorities for OST’s work is also improving. On the other hand, although the initiatives provided for upper management attention through the Technology Acceleration Committee, the future of this Committee is uncertain because of the departure of EM’s Assistant Secretary, who established it. According to EM officials, a broader executive committee addressing EM issues may take its place. EM did not carry out its plans to include deployment in the annual performance expectations of its senior managers, considering their membership in the then-active Technology Acceleration Committee to be sufficient to hold managers accountable. In addition, EM has not yet improved developer-user cooperation for individual projects. Specifically, EM’s initiatives do not require OST to use its existing decision process for technology development (the gates system), which would require user involvement at various stages in the development process. Furthermore, EM has yet to determine how it will provide deployment assistance to cleanup sites to (1) more routinely provide technical assistance in selecting and implementing innovative technologies and (2) make modifications to completed technologies to better meet sites’ needs when it is cost-effective to do so.
subsections: section_title: EM Has Established Initial Deployment Performance Measures for Field Sites and Is Developing Additional Measures
paragraphs: In a July 1997 memo, EM’s Assistant Secretary stated that technology deployment is the responsibility of all senior EM management, including the managers of EM’s operating groups, OST, and field offices. EM management had not previously emphasized technology deployment, and this was the first formal assignment of responsibility for deployment. The Assistant Secretary also directed that performance measures based on technology deployment be established for those groups involved with deployment efforts and be included in the performance expectations for senior managers. In response, EM has instituted or is planning performance measures addressing the deployment of innovative technologies at several levels: (1) DOE field sites undergoing cleanup, (2) contractors that manage the DOE field sites, and (3) OST and its focus areas. Field sites were also required to submit deployment plans addressing both their overall approach to utilizing innovative technologies and their plans to achieve deployments in specific cleanup projects. EM continues to refine its performance measures and has asked EMAB for advice about improving performance measures at the various levels to help increase deployment.
subsections: section_title: Performance Measures and Deployment Plans for Field Sites
paragraphs: In responding to our written inquiry to EM management in March 1998, the Acting Deputy Assistant Secretary for OST stated that “in analyzing the most appropriate and optimum way” to accelerate technology deployment, EM management concluded that deployment goals can best be achieved by holding those at the point of implementation of new technology—the field sites—responsible for deployment. EM has established two indicators to measure field sites’ efforts to use innovative technology to clean up waste sites: (1) the number of technologies deployed annually and (2) life-cycle cost savings resulting from the use of innovative technology. For the present, annual targets for the number of deployments are based on the amount of annual EM funding a site receives. EM established a target that requires field offices to agree to deploy one new technology for every $100 million in annual funding that they receive. For example, DOE’s Oak Ridge site will receive about $600 million in EM funding in fiscal year 1998 and is therefore expected to use six new technologies a year in its effort to clean up nuclear waste. For fiscal year 1998, field sites have agreed to deploy a total of 49 new technologies, which can be from OST or other sources. OST believes that the majority of these new technologies will be ones that it has sponsored.
Field sites must also submit site-specific deployment plans for innovative technologies. The plans, most of which were submitted in May and June 1998, describe the sites’ overall approaches to deploying innovative technologies, such as processes for identifying deployment opportunities and involving regulators. The plans also specify opportunities to deploy innovative technologies in the sites’ cleanup projects. For instance, the plans describe the schedule for technology deployments, projected benefits from using the technologies, and funding requirements.
In the future, EM may establish performance targets for field sites that are based on the amount of savings that would be produced by using innovative, rather than conventional, technology over the life of a project. These measures were not established in fiscal year 1998 because EM lacked a standard methodology for calculating cost savings. However, in March 1998, EM completed a draft of a standardized process for calculating these savings.
section_title: Performance Measures for DOE’s Contractors
paragraphs: The need for contract incentives for the use of innovative technologies has been broadly recognized by EM managers in headquarters and the field. Each of the five sites that we visited had used performance measures addressing deployment for the site’s management contractor. Some sites have experimented with different approaches to determine which measures work best. For example, at Savannah River, DOE tried performance-based incentives for its contractor in 1995 and 1996 that were based on the number of innovative technologies used and the associated cost savings; then, in 1997 and 1998, it switched to incentives based on the cost savings achieved—regardless of whether conventional or innovative technologies were used. According to DOE’s Assistant Manager for Environmental Quality at Savannah River, over half the cost savings that the contractor achieved in environmental restoration in 1997 came from the use of innovative technologies, and he believes that the cost savings measure has worked the best in providing incentives for using innovative technologies. At Lawrence Livermore National Laboratory, which participates in a number of OST technology projects, the contractor’s performance measures address both using innovative technologies in the laboratory’s cleanup activities and supporting their use at other sites.
section_title: Performance Measures for OST
paragraphs: OST’s performance will also be measured on the basis of technology development and related deployment. For example, OST’s performance goals for fiscal year 1998 include demonstrating 35 new technologies; finishing the development of 40 “alternative” technologies; and, along with the cleanup sites, taking responsibility for the 49 deployments of technology to be used in waste cleanup projects.
According to the Acting Deputy Assistant Secretary for OST, several additional performance measures are under consideration for OST’s focus areas to help ensure that the technologies still in development are “deployable” when they are completed. These measures include whether the focus areas’ projects address high-priority technology needs and whether end users consider the technologies under development to be viable solutions to their needs.
section_title: Further Improvements Planned for Performance Measures
paragraphs: In a June 1998 meeting, EMAB presented its analysis, prepared at EM’s request, of how EM should improve performance measures for technology development and deployment. Among other things, EMAB emphasized that the use of performance measures must be supported by EM’s leadership and that performance measures for EM’s technology research, development, and deployment must be integrated with similar measures for site cleanup programs. EMAB also suggested that EM’s Technology Acceleration Committee review and improve existing research and development performance measures. The Acting Deputy Assistant Secretary for OST told us that EMAB’s advice would be considered in designing additional performance measures for OST’s focus areas. As of September 1998, EM was still in the process of identifying and improving performance measures to help ensure that cost-effective innovative technologies are used for waste cleanup.
section_title: EM Is Increasing User Input to OST’s Planning and Priority Setting
paragraphs: EM has established a mechanism—a user steering committee for each of OST’s focus areas—to engage technology users in setting overall plans and priorities for the work of the focus areas. The committees include the senior managers of DOE field sites (such as sites with tank waste for the Tanks Focus Area) and headquarters officials appropriate to the focus area. These committees are to work on budgeting, planning, and setting directions for the R&D investments of the focus areas. The committees are modeled after the practice of the Tanks Focus Area, which set up such a committee in 1996. The committees for the other focus areas began organizing in February 1998.
Among other things, user steering committees will help focus areas develop their multiyear program plans. OST is initiating these 5-year plans to manage and measure focus areas’ performance under the requirements of the Government Performance and Results Act of 1993. OST plans to complete the first set of plans by December 31, 1998, and to develop the plans annually for the upcoming 5 years. In addition, at their meetings in the spring of 1998, the user steering committees provided input to the focus areas’ proposed fiscal year 2000 budgets.
section_title: Continued Upper Management Attention to Deployment Is Not Ensured
paragraphs: While the EM and OST initiatives have begun to address internal barriers to the deployment of innovative technologies, continued attention by EM’s upper management to deployment is not ensured. The attention may not continue because (1) the future of the Technology Acceleration Committee is uncertain and (2) deployment measures have not been included in the contracts of EM’s senior managers.
subsections: section_title: Upper Management Committee’s Future Is Uncertain
paragraphs: In response to the July 1997 memo by EM’s Assistant Secretary, the Technology Acceleration Committee, composed of senior-level managers from EM headquarters and the field, was organized and met in September 1997. This Committee’s purpose is to “provide corporate leadership to ensure an aggressive effort to deploy alternative and more effective technologies through full integration of the technology development and user organizations.” According to the Committee’s draft charter, it would meet at least once per quarter. The Committee met again in January 1998, but has not met since. According to the Acting Deputy Assistant Secretary of OST, the Committee has been inactive because it reported directly to EM’s Assistant Secretary, who left the Department in January 1998. According to the Acting Assistant Secretary for EM, EM is considering establishing a broader executive committee of senior managers to address EM issues, including the deployment of innovative technologies.
To date, the Technology Acceleration Committee has increased communication among OST, EM line offices, and field offices. It has discussed issues such as clarifying deployment responsibilities, involving technology users throughout the technology development process, and improving incentives for contractors. The Committee also directed the establishment of user steering committees for focus areas. Because the user steering committees have members from EM’s headquarters and field offices, we believe that the existence of the Technology Acceleration Committee facilitated this innovation. The Acting Deputy Assistant Secretary of OST agreed with the importance of the Committee but thought that a broader executive committee of senior officials could address technology deployment and other EM issues.
Even with these improvements, unresolved issues affecting technology deployment still exist and could benefit from the attention of EM’s upper management. As noted above, EMAB suggested that the Technology Acceleration Committee review and improve R&D performance measures. In addition, the site-specific deployment plans state that a number of issues need to be resolved, such as learning the possible effects of EM’s increased use of fixed-price contracting and private financing (referred to as “privatization”) on the use of innovative technologies.
For example, the deployment plan of the Ohio field office raises privatization as a policy issue requiring guidance from headquarters, stating that most fixed-price bidders will use technologies with which they are familiar. As a result, the plan states, technologies that were developed at considerable expense may not be deployed because of bidders’ reluctance to assume a risk of failure. In our visits to field sites, we observed instances in which the use of OST-developed technologies was uncertain because EM planned to solicit fixed-price bids for cleanup work and the technology selected would depend on the choice of the private firm winning the contract. For instance, the Houdini robot was designed for retrieving radioactive wastes from silos at the Fernald site. However, when EM decided to solicit fixed-price bids for waste retrieval from Fernald’s silos, the Houdini robot was instead used in the radioactive waste tanks at Oak Ridge. Fernald had not yet received bids at the time of our visit, and environmental remediation officials told us that the companies bidding for this work will define which waste retrieval tools they would use—Houdini might or might not be included.
section_title: EM Has Not Established Deployment Performance Expectations in Its Senior Managers’ Contracts
paragraphs: In his July 1997 memo, the Assistant Secretary for EM stated that, beginning in October 1997, performance expectations for EM’s senior managers in headquarters and the field would be developed to require the deployment of alternative and more effective technologies. However, the Acting Deputy Assistant Secretary for OST, in response to our written inquiry to EM management, stated that technology-related performance measures would not be included in senior managers’ performance contracts and that senior managers are held responsible for technology deployment through their membership in the Technology Acceleration Committee. However, as noted above, this Committee has not met since January 1998, and its future is uncertain.
section_title: EM and OST Initiatives Do Not Fully Address User Input to Technology Projects and User Need for Deployment Assistance
paragraphs: EM’s and OST’s current efforts and initiatives only partially address the internal obstacles to deployment that were discussed in chapter 3. Specifically, the new initiatives do not reinforce the need for OST’s focus areas to use the technology development gates system and do not provide for OST’s deployment assistance to help sites select new waste-cleanup technologies or modify existing technologies for site use.
subsections: section_title: Initiatives Do Not Stipulate the Use of the Gates System
paragraphs: Although EM’s initiatives involve users in setting the overall plans and priorities of OST’s focus areas, they do not fully address the need for detailed user input on individual technology projects. The Acting Deputy Assistant Secretary for OST told us that the focus areas need to use OST’s existing gates system to obtain user input into the design and development of cleanup technology. Furthermore, he said that it is necessary to use this system to help prevent the development of technologies that do not meet sites’ needs, a problem discussed in chapter 3. However, in contrast to these statements of support for the gates system, we found that EM’s new initiatives do not require its use nor identify an alternative means to ensure that technology developers and users communicate and cooperate about individual technology development projects.
section_title: EM Has Not Fully Addressed Deployment Assistance to Cleanup Sites
paragraphs: EM and OST initiatives have not fully addressed two areas that must be considered when deploying innovative technologies: (1) providing technical assistance to sites on innovative technologies and (2) modifying completed technologies for use at specific sites. One potential vehicle for providing deployment assistance—OST’s new Accelerated Site Technology Deployment program—has not increased technical assistance in most cases and did not have the benefit of information that EM now has that EM could use to improve its priority setting for deployment assistance.
subsections: section_title: Technical Assistance for Sites Is Not Yet Well-Defined
paragraphs: EM and OST have not yet identified sources of expertise and procedures or developed a policy for routinely providing technical assistance on innovative technologies to DOE sites. OST recognizes that focus areas should more frequently provide technical assistance to sites when they are selecting and beginning to implement technologies and that this assistance should address innovative technologies developed by other sources as well as by OST. EMAB has questioned whether the focus areas currently have the expertise needed to provide such assistance. The Acting Deputy Assistant Secretary for OST acknowledged that the focus areas vary in their degree of expertise and ability to provide technical assistance. He noted that the Tanks Focus Area works closely with one of the national laboratories, which can provide in-depth expertise, and stated that the other focus areas need to develop a roster of technical experts who can be consulted for particular site cleanup problems that the focus areas cannot solve. Furthermore, the Acting Deputy Assistant Secretary stated that performance measures that encourage focus areas to provide technical assistance will be needed.
Some initial steps have been taken to involve OST in selecting technology for environmental restoration sites. In fiscal year 1998, the Office of Environmental Restoration began including OST in its processes for providing sites with information and support for technology selection decisions. OST is contributing funding and the technical support of its focus area staff to this program. However, OST does not have a similar involvement with EM’s Office of Waste Management or Office of Nuclear Material and Facility Stabilization.
EM lacks a policy on whether OST should provide technical assistance for major cleanup actions routinely or only if requested by a site. While the management-level Technology Acceleration Committee reached an “understanding” that the focus areas’ role should include technical support to end users for deployment, the Committee did not identify resources, procedures, or policies for such technical assistance. According to the Acting Deputy Assistant Secretary for OST, policies and procedures for providing technical assistance will be one of the elements addressed in the business system redesign currently under way in OST, and procedures may be completed by the end of 1998.
section_title: EM Has Not Addressed Need to Modify Some Completed Technologies
paragraphs: The initiatives do not address a barrier to deployment that we discussed in chapter 3—the lack of a mechanism and resources for modifying completed technologies for use at specific sites. In fact, none of the initiatives, action plans, or meetings of the Technology Acceleration Committee even raise this issue. Officials at three of the five sites we visited told us that OST sometimes considers its technology development work completed before technologies are ready for specific applications in the field. The Acting Deputy Assistant Secretary for OST agreed that this is a problem and told us that, while the Tanks Focus Area develops technologies fully to the point of use, technologies from the other focus areas were not always ready for field use. For example, he stated that the Mixed Waste Focus Area had not tested its thermal treatment technologies on actual radioactive waste. The Acting Deputy Assistant Secretary stated that sites and focus areas should work together to enable and jointly fund the first use of an OST-developed technology.
While joint OST and site support for deployment has occurred for some projects—including the reactor safe-storage project at Hanford and the use of the Houdini robot in tanks at Oak Ridge that are described in chapter 3—EM lacks an overall policy, procedure, and designation of responsibilities for situations in which OST-developed technologies may require modification for site use. Nor has EM identified resources for this purpose, except to the extent that some projects under the Accelerated Site Technology Deployment program may address this need. According to the Director of OST’s Office of Technology Systems, focus areas consider any funding needs for technology modifications when requested by sites. He noted that such requests would compete for limited funding with the focus areas’ technology projects.
EM has data that could be used to identify OST technologies that might have additional cost-effective deployments. Sites’ Accelerating Cleanup plans, issued in draft in June 1997 and most recently submitted in June 1998, provide a comprehensive compilation of sites’ technology needs, as well as detailed information on each cleanup project across the DOE complex. OST has developed a database, called a linkage table, that identifies links between its completed and ongoing projects and the sites’ technology needs. EM could identify OST-developed technologies that could provide cost-effective solutions to sites’ needs and set priorities for deployment assistance to cleanup projects, including technical assistance and technology modifications, if needed. OST’s Deactivation and Decommissioning Focus Area has already used this database to contact potential technology users at the sites and inquire whether the focus area can provide assistance. However, OST has not required its focus areas to do this.
section_title: OST’s Accelerated Site Technology Deployment Program Does Not Fully Meet Sites’ Needs for Deployment Assistance
paragraphs: One potential vehicle for deployment assistance is OST’s Accelerated Site Technology Deployment (ASTD) program, begun in fiscal year 1998. OST funded 14 ASTD projects at 12 sites to deploy innovative technologies in cleanup projects. The approximately $26 million that OST provided to site projects in fiscal year 1998 resulted in an additional investment of about $708 million from the sites over the life of the projects. OST identified potential ASTD projects through site proposals and competitively evaluated the proposals to select projects to fund. Selection criteria included the technical merit of the approach, interest in deploying the technologies at multiple locations, and commitment of additional funding by the site.
While ASTD may be helping these selected projects in addressing obstacles to deployment, the program has not fostered interaction among technology developers and users in many instances. For example, we found that OST’s focus areas provided technical assistance to only 5 of the 14 ASTD projects, and national laboratory personnel who had helped to develop some of the technologies provided technical assistance to 2 additional ASTD projects. It should also be noted that technical assistance and technology modifications on a smaller scale than the current ASTD projects may be appropriate in some cases.
section_title: Conclusions and Recommendations
paragraphs: Having spent more than $2 billion and 9 years on over 700 innovative cleanup technology projects, EM and OST recognize that the cleanup program can only benefit from these efforts if the innovative technologies that have been developed are successfully deployed. To promote deployment, EM and OST have initiated a number of actions aimed at improving the relationship between technology developers in OST and the users at EM’s cleanup sites. However, we are concerned that the committees and processes that EM and OST are now creating will be ineffective if they are not accompanied by more fundamental changes in how EM conducts technology development and deployment.
We believe that EM and OST need to take three relatively straightforward actions to increase the deployment of existing innovative technologies First, OST must make sure that it has adequate technical expertise to assist users in evaluating and implementing innovative technologies that it and others have developed. The focus areas are the logical source for this expertise; however, if they are unable to meet this need, other centers of expertise, possibly in the national laboratories, need to be developed.
Second, we continue to believe that OST staff, equipped with the appropriate expertise, need to be formally involved in evaluating and selecting technologies for use at the cleanup sites. We believe that the program’s experience has shown that without a specific requirement to bridge the gap between developers and users, each party will continue to operate in its own environment, with users deploying only those technologies with which they are familiar, and OST developing technologies that are generic and not designed for specific situations.
Third, existing innovative technologies could be implemented, as we found repeatedly, if they could be modified or fine-tuned to address a specific site cleanup problem. Information now exists from sites’ Accelerating Cleanup plans and OST’s linkage tables to identify technologies that can be modified to fit specific situations. However, such modification takes money, and without specific action by EM management, neither users nor developers are likely to provide these funds on their own. For example, if OST uses its funds to fine-tune an existing technology, it is reducing the funds available for its other missions. Similarly, users can logically view the use of their funds to modify a technology as taking away resources that they need for other cleanups. However, EM’s experience, for example, from the project for safe storage of the C reactor at the Hanford site or from the ASTD program, has shown that successful deployment can occur if both parties make a financial commitment.
Additional technology development will be needed to address technology problems for which no cost-effective solution exists, such as high-level waste tanks at Hanford. To ensure the deployment of technologies that are currently under development or will be developed, EM does not need additional processes and procedures. Rather, it needs to rigorously and consistently apply its current gates system. Consistent use of this system by focus areas would help ensure that technology developers and users communicate and cooperate throughout the development of individual technologies, and that, if technologies are not living up to their potential or there is not adequate commitment from users, the project can be terminated and the funds redirected to more productive uses.
Ensuring that these actions are taken consistently will require the commitment of top management in the EM program. The Technology Acceleration Committee is a sound idea; however, it has already missed a planned meeting, and we are concerned that it could easily slip into disuse. We believe that continuing a committee of senior EM managers is a key element in ensuring that top management is focused on formulating policy for technology deployment. An additional important element is the establishment of performance measures that hold EM’s top managers accountable for technology deployment. While EM has made clear to field managers that they are responsible for deploying innovative technologies, this commitment needs to be reflected throughout the organization if additional innovative technologies are to be successfully deployed.
Finally, with an increased emphasis on deployment, EM will need more accurate data than it currently has on deployment efforts. A verification effort similar to the one we undertook will be needed to provide valid data on future deployments. On the other hand, we recognize that improving data on prior deployments may not be cost-effective. Therefore, reporting existing data as estimates could lend more credibility to the data and the overall program. In addition, EM has recognized that deployment is not the only relevant measure of success in technology development. EM’s recent efforts to develop additional performance measures for the entire program are a step in the right direction.
subsections: section_title: Recommendations
paragraphs: To increase the deployment of existing technologies and ensure that technologies developed in the future are used, we recommend that the Secretary of Energy direct the Assistant Secretary for Environmental Management to direct the Deputy Assistant Secretary for the Office of Science and Technology to establish centers of expertise for innovative technologies by using existing focus areas or another approach if needed and require that a representative from one of these centers participate in the technology selection process on each cleanup project; direct the cleanup programs and OST to (1) use existing data to identify OST-developed technologies that can be cost-effectively modified to meet sites’ needs and (2) identify funds to modify these technologies if needed; direct that the gates system be used rigorously and consistently as a decision-making tool for managing technology development projects and as a vehicle for increasing developer-user cooperation; use their annual performance expectations to hold EM headquarters managers responsible for increasing the deployment of innovative technology; and implement a system to verify the accuracy of future deployment data and label any existing data that have not been verified as an estimate.
section_title: Agency Comments and Our Evaluation
paragraphs: Overall, DOE agreed with the recommendations in our report. In doing so, DOE offered information regarding actions it had taken or intended to take that it believed were responsive to our recommendations. However, DOE’s responses to two of the recommendations suggest that the actions described would not be fully responsive to these recommendations. DOE’s comments are included as appendix III.
In response to our recommendation that OST establish centers of expertise and include a representative from one of these centers in the technology selection process, DOE indicated a willingness to act on our recommendation but offered few specifics, especially with respect to involving OST in the technology selection process. In 1994, we also recommended that OST be given a formal role in the technology selection process. During our current review, we found that this recommendation had not been implemented primarily because site officials were skeptical about OST’s ability to provide quality technical advice and were therefore reluctant to allow OST more of a role in selecting cleanup technologies. We believe that it will take more specific actions by OST, beyond the generalized user steering committees cited in its response, to develop credible expertise and thus gain a role in the technology selection process.
In response to our recommendation that DOE rigorously and consistently use the gates system as a decision-making tool for managing technology development, DOE also agreed with the recommendation but noted that it had incorporated the gates system into its system of peer review. While we recognize the value of peer review as a mechanism for obtaining independent technical judgments about projects OST is pursuing, we note that peer review can occur infrequently over the life of a project and after significant decisions are made. Therefore, we do not believe that peer review is a substitute for focus area managers using a disciplined decision-making system that involves users throughout the technology development and deployment process.