Cooperative Threat Reduction: DOD Has Improved Its Management and Internal Controls, but Challenges Remain
Published: 2005-06-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Section 3611 of the National Defense Authorization Act for Fiscal Year 2004 mandates that GAO assess the Department of Defense's (DOD) internal controls for the Cooperative Threat Reduction (CTR) program and their effect on the program's execution. In addressing the mandate, we assessed DOD's management and internal controls over implementing CTR projects since 2003 by using the control standards for the federal government as criteria. In response to the mandate, we focused on those management and internal control areas considered most relevant to CTR project implementation: (1) building a management structure, (2) risk assessments, (3) performance measures, (4) program reviews, (5) communications, and (6) project monitoring. The Congress also mandated that GAO describe the status of DOD's implementation of legislative mandates covering the CTR program.
section_title: What GAO Found
paragraphs: Through the CTR program, DOD provides assistance to help the former states of the Soviet Union secure and eliminate their weapons of mass destruction. Since 2003, DOD has improved its management and internal controls over the CTR program. Prior to 2003, DOD had problems managing the program and ensuring that the program was meeting its objectives. These inadequacies became apparent in 2003 following two project failures in Russia that cost the CTR program almost $200 million, including the never used liquid rocket fuel disposition facility. Following these incidents, DOD implemented a more structured approach to managing the CTR program. In July 2003, DOD filled vacancies in the office responsible for managing the program, providing a level of leadership and oversight that did not previously exist. Once in place the new leadership made important improvements to the program's internal controls in the areas of organizational structure, risk assessments, performance measures, program reviews, and communication. For example, DOD now assesses and balances risks with project requirements and measures project performance at each phase. DOD also conducts semi-annual meetings to review commitments and responsibilities of CTR-recipient governments and to minimize risk. Although enhancing its internal controls helps mitigate the risks that stem from having to rely on the cooperation of CTR-recipient governments, DOD can never fully eliminate the project risks associated with recipient governments' cooperation. Furthermore, while DOD's enhancements are an improvement over previous internal controls, current mechanisms do not include a separate review of CTR projects upon their completion. As such, DOD lacks a system for evaluating projects upon their completion and applying lessons learned to future projects.

section_title: Letter
paragraphs: Since 1992, Congress has authorized the Department of Defense (DOD) to  provide more than $5 billion for the Cooperative Threat Reduction (CTR)  program to help the former states of the Soviet Union secure and eliminate  their weapons of mass destruction and prevent their proliferation. Through  the CTR program, the United States has supported activities such as  eliminating nuclear missiles, building storage facilities for nuclear  materials, eliminating chemical weapons, securing biological pathogens,  and employing former weapons scientists. Recent project failures in  Russia, however, have raised congressional concerns about the program’s  management. Specifically, from 1993 through 2003, DOD spent nearly $200  million to construct a liquid rocket fuel disposition facility that was never  used and to design a solid rocket motor elimination facility that was never  constructed. In 2003, as a result of these incidents, DOD began to revise its  management and internal controls over the CTR program. (For more  detailed information on these CTR project failures, see app. I.)
The Deputy Assistant to the Secretary of Defense for Chemical  Demilitarization and Threat Reduction within the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) is  responsible for developing CTR implementation plans and making sure that  CTR contractual obligations are met. Concurrently, the CTR Policy Office,  within the Office of the Under Secretary of Defense for Policy, is  responsible for developing and coordinating CTR policy guidance and  defining CTR program objectives. The Defense Threat Reduction Agency  (DTRA) reports to the Deputy Assistant to the Secretary of Defense for  Chemical Demilitarization and Threat Reduction and oversees the  execution of CTR projects on a daily basis.
Section 3611 of the National Defense Authorization Act for Fiscal Year 2004  mandates that GAO assess DOD internal controls for the CTR program and  their effect on the program’s execution. The mandate specifies that our  analysis focus on controls intended to ensure that projects are being  executed consistent with the program’s objectives. In addressing the  mandate, we evaluated DOD’s management and internal controls for  implementing CTR projects since 2003 by using the applicable control  standards for the federal government. To respond to the mandate, we  identified those management and internal control areas most relevant to  CTR project implementation: (1) program management, (2) risk  assessments, (3) performance measurement, (4) program reviews, (5) communications, and (6) project monitoring. Congress also mandated  that we describe the status of DOD’s implementation of legislative  mandates covering the CTR program. (See app. II for information on CTR  legislative mandates.)
In reviewing DOD’s management and internal controls for implementing  the CTR program, we collected and analyzed DOD documents and  developed a semi-structured interview guide and questioned 30 DOD  officials responsible for managing and implementing the CTR program. We  met with other DOD officials, reviewed DOD documents including DOD  acquisition management guidance, and analyzed legislation. In addition, we  traveled to Russia and Kazakhstan to observe CTR project implementation  and to obtain information from American, Russian, and Kazakhstani  government officials and contractor personnel. To determine the reliability  of the data we used in this report, we reviewed relevant agency documents  and obtained information from agency officials to ensure that the data used  are sufficiently reliable for our work. We performed our work from April  2004 through May 2005 in accordance with generally accepted government  auditing standards. (See app. III for more details on our scope and  methodology.)
subsections: section_title: Background
paragraphs: Congress created the CTR program in 1991 to help the states of the former  Soviet Union secure and eliminate their weapons of mass destruction and  prevent their proliferation. Through the CTR program, the United States  has supported activities to eliminate nuclear missiles, build a storage  facility for nuclear materials, eliminate chemical weapons, secure  biological pathogens, and employ former weapons scientists. As of January  2005, the CTR program has assisted in the elimination of about 570  intercontinental ballistic missiles and nearly 30 nuclear powered ballistic  missile submarines. In 2004, Congress authorized DOD to expand the scope  of the CTR program to countries outside the former Soviet Union. For  example, beginning in 2005, CTR assistance will help Albania destroy its  chemical weapons stockpile.
Figure 1 shows the DOD management structure for the CTR program.  Within the Office of the Under Secretary of Defense for Policy, the CTR  Policy Office is responsible for developing and coordinating policy  guidance, defining program objectives for the CTR program, and  negotiating agreements with CTR recipients. The CTR Policy Office works  with the office of the Under Secretary of Defense for Acquisition,  Technology, and Logistics through the Deputy Assistant to the Secretary of  Defense for Chemical Demilitarization and Threat Reduction. DTRA  reports to the Assistant to the Secretary of Defense for Nuclear and  Chemical and Biological Defense Programs. The Deputy Assistant to the  Secretary of Defense for Chemical Demilitarization and Threat Reduction  provides strategic implementation guidance on and oversight of CTR  projects, and interacts daily with DTRA on CTR matters. Within DTRA, the  Cooperative Threat Reduction (CT) directorate manages the program’s  daily operations. The directorate is organized into five program areas:  Biological Weapons Proliferation Prevention, Chemical Weapons  Elimination, Nuclear Weapons Safety and Security, Strategic Offensive  Arms Elimination, and Weapons of Mass Destruction (WMD) Proliferation  Prevention. (For a more detailed description of these program areas, see  app. IV.) The directorate is located at Ft. Belvoir, Virginia, and several  DTRA offices throughout the former Soviet Union provide in-country  support for CTR program implementation.
At the beginning of the program in 1992, DOD primarily purchased and  provided equipment such as cranes, cutting tools, and vehicles to recipient  countries. As the program matured, CTR assistance provided more  services, such as hiring U.S. contractors who helped recipient countries  dismantle nuclear delivery systems and missiles. Currently, CTR provides  most assistance to recipient countries through contracts with American  firms. DOD executes, manages, and reviews the contracts according to  DOD and federal acquisition requirements. Specifically, in 2001, the CTR  program began using special contracts with prime contractors who, with  their teams of supporting subcontractors, implement the majority of CTR  projects in the recipient countries. These five contractors are known as  CTR Integrating Contractors. DTRA has also contracted with the Science  Applications International Corporation’s Threat Reduction Support Center  (TRSC). TRSC staff provide support to CTR program and project managers  in the areas of operations, logistics, engineering, financial, and program  management.
section_title: DOD Has Improved Its Management and Internal Controls over the CTR Program
paragraphs: Since 2003, DOD has improved its management and internal controls over  the CTR program. Prior to 2003, DOD’s internal controls over the program  were limited and did not ensure that CTR program objectives were being  met. Following two project failures in Russia, DOD implemented a series of  new measures in 2003 that provided a more structured approach to  managing the CTR program. Most importantly, in July 2003, DOD filled  vacancies within AT&L, the office responsible for ensuring that DTRA’s  implementation of CTR projects was meeting cost, schedule, and  performance goals. After DOD filled these positions, the new leadership  worked closely with DTRA officials to introduce important enhancements  to the program’s internal controls. For example, DOD adopted several new  methods to assess and mitigate the risks involved in cooperating with CTR- recipient governments. Although these methods attempt to reduce risk to  an acceptable level, DOD cannot fully mitigate the risks involved in  working jointly with CTR-recipient governments. While DOD’s  enhancements are an improvement over the previous management and  internal controls for the program, CTR procedures do not include final  reviews of CTR projects upon their completion. As such, DOD has no  mechanism for assessing the success of completed projects and applying  lessons learned to future projects.
subsections: section_title: Improved Program Management and Internal Controls Allow for Improved Implementation of CTR Program
paragraphs: Beginning in 2003, DOD implemented several new and enhanced  management processes to allow program managers to better assess the  progress of CTR projects and address program implementation  weaknesses to reduce the risk of program failures. For example, DOD filled  vacant AT&L positions; developed specific guidance for project managers  on reporting objectives, schedules, and cost estimates; and improved  communication within the program and with recipient countries. (For a  comparison of DOD’s CTR internal controls with selected control  standards for the federal government, see app. V.) DOD developed a  training course that all CTR project and program managers are required to  complete, which provides detailed instruction on incorporating the new  requirements of the internal control framework into all CTR projects.   According to 24 of the 30 CTR program, policy, and acquisition officials  responding to our structured interview, the new framework has helped  improve CTR project implementation. For example, CTR officials stated  that now the program management review system is more rigorous and  project managers know what is expected of them in reporting on the cost,  schedule, and performance of their projects.
subsections: section_title: Key Leadership and Oversight Vacancies Filled for CTR Program
paragraphs: In July 2003, DOD filled AT&L vacancies, closing a critical gap in the  department’s ability to ensure that the CTR program was meeting cost,  schedule, and performance goals. Previously, DOD had not been carrying  out its own management plans for ensuring that CTR projects were meeting  stated goals. Specifically, in May 1994, the Deputy Secretary of Defense  approved a plan to strengthen the implementation of CTR projects. Under  this plan, the CTR policy office was responsible for negotiating agreements  with recipient countries, establishing policy guidance, working on the CTR  budget, and notifying Congress of developments in the program. After CTR  policy approved a project and signed an agreement to begin work, AT&L  was responsible for developing detailed implementation plans, monitoring  ongoing work, and ensuring that work was meeting cost, schedule, and  performance goals. However, DOD left several AT&L positions vacant until  2003, leaving a critical gap in oversight over the CTR program. The CTR  policy office began managing daily CTR project activities to fill this  leadership gap. However, according to the director of the policy office, staff  in that office were not qualified to manage the activities of the program  because they were not familiar with DOD acquisition guidelines nor did  they have the technical expertise necessary to manage CTR programs.
According to a 2004 DOD Inspector General (IG) report on the  management of the CTR program, if the AT&L positions had been filled,  those officials might have identified some of the risks involved in the two  failed CTR projects that cost DOD nearly $200 million.
Since the AT&L positions were filled in July 2003, the office now  participates in CTR program planning and review, overseeing program  review meetings, and providing guidance on issues such as performance  measurement and reporting requirements. The Deputy Assistant Secretary  of Defense for Chemical Demilitarization and Threat Reduction attends  informal monthly meetings with CTR program managers to be updated on  the status of projects and other management issues. He also serves as the  program reviewer for several CTR projects, making him responsible for  overseeing the cost, schedule, and performance of each of those projects  and approving them at the end of each project phase. For example, in July  2004, he approved a biological weapons proliferation prevention project’s  acquisition program baseline and authorized the program manager to move  the project into the demonstration phase. CTR officials stated that it is now  clear who they need to report to and when.
section_title: DOD Uses Several New Methods to Assess CTR Project Risk
paragraphs: DOD uses several new methods to assess and mitigate risks associated  with CTR projects. DOD identifies a senior official responsible for ensuring  the potential risks to meeting objectives are evaluated for each project,  requires stakeholders on each project to meet regularly to conduct specific  risk management activities, and implements each project in three phases.  According to DOD’s risk management guide, risk is defined as a measure of  the potential inability of a program to achieve its overall program  objectives within defined cost, schedule, and technical constraints.
DOD’s approach to assessing program risks was limited prior to 2003. In  September 1996, we reported that the CTR multiyear plan did not indicate  whether program officials had omitted risk and contingencies from project  cost estimates. In addition, a 2003 DOD IG report found that DOD did not  identify risks or have adequate controls in place to mitigate risk when managing projects. According to a CTR official, CTR program and project  managers periodically included risk assessments in planning their projects,  but did not include actions to control the risks identified if problems  occurred. The DOD IG reported that the CTR program management’s  failure to fully assess project risks contributed to DOD spending nearly  $200 million on projects in Russia to construct a liquid rocket fuel  disposition facility that was never utilized and to design a solid rocket  motor elimination facility that was never constructed.
In an effort to improve assessments of the risks associated with CTR  projects, DOD began designating an official, known as the Milestone  Decision Authority (MDA), to be responsible for ensuring that project  managers, with assistance from project stakeholders, assess the risks to  meeting project objectives and formulate plans to mitigate these risks.  MDAs are assigned to projects based on several factors, including the  project’s risk and expected cost. According to an AT&L official, the Deputy  Assistant to the Secretary of Defense for Chemical Demilitarization and  Threat Reduction is usually assigned as the MDA for high-cost or high-risk  projects. For projects with less risk or expense, the MDA is usually the  director of the DTRA/CT directorate. MDAs review the risks identified by  the project managers and evaluate the plans they have developed to  mitigate these risks.
In addition, DOD instituted periodic stakeholders meetings to assess and  minimize risks associated with CTR projects and to discuss major project  issues and milestones. In these meetings, project managers present  assessments of potential risks that could impact their ability to meet  project objectives. For example, a risk identified for the Russian SS-24  missile elimination project was that political or economic developments in  Russia might unexpectedly affect the project’s costs. After the project  managers present their assessments, the stakeholders provide input to  address these risks and consider additional problems that may arise during  project implementation. According to CTR management officials, this team  approach to risk assessment ensures consensus early in each phase of the  project. It has resulted in more informed decision making because  stakeholders meet regularly to receive updates on project status and make  decisions on the next phase of project implementation based on the facts  presented during those meetings. Of the 30 DOD and CTR officials we  interviewed using our structured interview guide, 9 said that this new  process of stakeholder involvement was one of the most important new  internal controls for the CTR program.
Furthermore, DOD now uses a new phased-contract approach that divides  each CTR project into three phases. These phases can vary according to  project, but usually include phases covering project development, project  execution, and project maintenance, according to a CTR official. This  approach helps to minimize risk by allowing managers to make the  appropriate changes, delay, or stop a project if a problem occurs. For  example, in 2003, in the development phase of a Ukrainian SS-24 missile  elimination project, DOD decided not to proceed with the project because  the risks associated with the missile destruction method that the  Ukrainians wanted to use were too high. Project managers are required to  develop exit criteria for each project phase that clearly state under what  conditions the project will be permitted to move into the next phase and  under what conditions DOD will stop the project. For example, for a CTR  project tasked with eliminating Russia’s SS-25 missiles, one of the exit  criteria for moving into the project’s maintenance phase is that DOD  complete negotiations on the contract to maintain the missile elimination  facility that is being constructed.
section_title: DOD Has New Guidelines for Reporting Project Performance
paragraphs: In 2003, DOD devised and implemented new guidelines that provide CTR  project managers with written instructions on developing and reporting  project objectives, schedules, and cost estimates. According to the internal  control guidelines for the federal government, it is important for an  organization to establish measures to gauge its performance on critical  activities and determine if the organization is meeting its objectives. CTR  program area and project managers we interviewed stated that prior to  2003 there were no established procedures for developing performance  measures, evaluating project performance, or reporting (either orally or in  writing) on project implementation to management. In addition, project  plans were not comprehensive and lacked established baselines against  which to measure performance.
According to CTR project managers, the current guidance on performance  measurement is clearer and more consistent than in the past. For example,  in a training course required for all CTR program and project managers,  project managers are instructed on developing measures for how, when, in  what sequence, and at what cost specific project tasks will be completed.  Our fieldwork included a site visit to a CTR project in Russia that had  developed such measures. One measure used to gauge performance on that  project is whether the elimination of Russian SS-24 missiles complies with  arms control treaty requirements. For each measure, project managers  develop objectives – the indicator’s desired outcome – and thresholds – the  minimum acceptable performance for that measure. For example, one  objective for the SS-24 missile elimination project is to eliminate Russia’s  SS-24 missiles by March 2008. However, if the missiles cannot be eliminated  by then, they must be eliminated by the threshold date of August 2008.  (Figure 2 shows the elimination of an SS-24 engine.)
If the threshold is not met at the end of a particular project phase, the  project manager and DOD management officials may consider stopping the  project. When an indicator is in danger of not being met, the project  manager is required to submit a warning report to the project’s MDA to  ensure that management is aware of potential delays and that the project  manager is addressing the problem. If the indicator is not met, DOD  management officials may stop the project until a plan is in place to bring  the indicator up to the threshold level.
section_title: DOD Has More Systematic and Consistent Tools to Review Programs
paragraphs: In 2003, DOD introduced a new process to review projects and programs to  provide a more systematic and consistent structure to management’s  review of CTR projects. According to the internal control guidelines for the  federal government, program reviews are important for program  management because they provide comparisons of actual performance to  planned or expected results and help management assess its programs.  Program reviews lacked the detail that allowed senior management to  evaluate projects and risks consistently. However, according to CTR  program managers we interviewed, before 2003 there was no standardized  guidance to assist program managers on developing program reviews or  implementing their programs. For example, CTR program area and project  managers did not receive any guidance on how to report on the daily  management of program operations or on the type of information that  status reports should include.
Under the new program review system, the designated MDA conducts  reviews of a project’s cost, schedule, and performance objectives. During  program reviews, which take place periodically throughout the course of a  project, project managers report to their MDAs on the status of their  projects and whether the objectives are being met. In addition, these  review meetings are more detailed than they were before the new system  was in place. For example, a project review in 2004 for a CTR project  tasked with installing nuclear detection devices in Uzbekistan included  details on the project’s schedule over the next 3 years, with specific dates  for completion of certain milestones. It also included a detailed breakdown  of funding for the project over the next 3 years and a thorough discussion  of project risks. The information was not included in the project’s 2003  review. According to several CTR project managers, the new program  review system has resulted in more consistently conducted project  evaluations. Of the 30 DOD officials we interviewed, 19 said the program  review process, conducted by the MDA, was one of the most important new internal controls for the CTR program. They reported that, with the  introduction of the MDA, program reviews are occurring at the same  intervals for each project and that project managers report cost, schedule,  and performance data in the same format to their MDAs during the reviews.  Through the course of work we reviewed copies of various MDA project  review documents.
section_title: Communication Is More Structured
paragraphs: According to DOD officials, communication within the CT directorate and  among the DOD offices involved in the CTR program, has improved with  the introduction of new internal controls. DOD also has improved its  external communications with CTR-recipient countries. Internal control  guidelines for the federal government state that communication  mechanisms should exist within an organization to allow the easy flow of  information down, across, and up the organization. However, before 2003,  internal communications within the CTR program office were not clear,  according to DOD officials. For example, all CTR stakeholders were not  present during project development meetings nor were they involved in  early decision making about project risks. Communications between DOD  and CTR recipient governments also were not clear. DOD assumed, without  getting written documentation that CTR recipient countries would carry  out the responsibilities and commitments to which they agreed.
Since 2003, communication among the DOD offices working on the CTR  program has improved. Stakeholders on specific projects meet more  frequently now than in the past to discuss project issues and problems.  Project managers involve stakeholders in the earliest stages of project  development on through to the final phase of project completion to assure  that stakeholders and managers have regular opportunities to learn about  project developments and provide input on project implementation. This  system has now been institutionalized and all CTR project managers are  instructed in a new training course to convene meetings with stakeholders  throughout the life of their projects. In addition, new reporting  requirements help ensure that all stakeholders are informed of project  developments. All of the 30 DOD officials we interviewed said that they are  required to report on the cost, schedule, and performance of their  programs and projects periodically, including daily, weekly, monthly, and  quarterly. For example, DOD now requires program managers to submit  monthly project status reports to ensure that potential problems are  documented and stakeholders are informed of them. In addition, 28 of the  30 DOD officials in our structured interview reported that the amount of  communication within the CT directorate allows them to effectively  implement their projects. Project managers are in frequent contact with  contractors implementing projects in recipient countries. We observed a  meeting in Russia between a CTR project manager and the Russian  contractors implementing the project he manages. During the meeting, they  negotiated revisions to a new contract and discussed the project’s status.  The project manager makes similar trips at least once a month to the  project site to oversee progress and meet with the contractors. Other  project managers we interviewed in Russia and in the U.S. stated that they  hold weekly phone conferences with contractors, exchange emails, and  make regular visits to project sites.
Contracting officials in Russia stated that they hold weekly telephone  conferences with their CTR project managers and contact them regularly  when project implementation issues arise. We observed such a weekly  telephone communication during our visit to the International Science and  Technology Center in Moscow. Contractors also submit monthly written  reports. Project managers also are in daily contact with their program  managers and CT directorate management. According to a CTR official, at  quarterly program review meetings, program and project managers present  detailed information, both orally and in writing, on the status of their  projects to all involved stakeholders.
DOD has also improved its external communications with CTR-recipient  countries. DOD and recipient government officials now consistently share  more detailed information on project developments and issues of concern.  CTR management officials and program and project managers are in  frequent contact with their recipient government counterparts throughout  project implementation. In 2004 CTR teams made 165 trips, compared with  70 trips in fiscal year 2001, to meet recipient government officials and  improve their monitoring of CTR projects. Russian government officials  working on CTR projects stated that they communicate with CTR officials  continually and meet regularly with the director of the CT directorate. They  also hold weekly teleconferences with project managers, and project  managers visit project sites regularly. While traveling with CTR project  managers in Russia and Kazakhstan, we observed extensive discussions of  important issues during site visits and meetings with contractors and  recipient government officials.
Furthermore, DOD has introduced and updated its controls to ensure that  commitments made by the CTR program and recipient governments are  regularly documented and discussed. These controls also are a means to  ensure that each party is held accountable for its responsibilities. In 2003,  DOD began using Joint Requirements and Implementation Plans (JRIP) to  document the commitments and responsibilities agreed to by each party  involved in project implementation. For example, a requirements plan for a  CTR project tasked with eliminating a specific type of Russian nuclear  missile states that one of DOD’s responsibilities in implementing the  project is to design and construct storage facilities for the missiles to be  eliminated. One of the Russian government’s responsibilities on the same  project is to provide DOD with a schedule for the delivery of the missiles to  the proper facility for elimination. If either party fails to meet its  obligations as articulated in the document, the other party can stop  progress on the project. For example, DOD officials halted new  construction from March to June 2004 at the CTR-funded chemical  weapons destruction facility at Shchuch’ye until the Russian government  stopped insisting on unnecessary design changes for the construction of a  boiler house on the site. To further enhance communication between CTR  program officials and CTR recipient countries, DOD also holds biannual  meetings where officials from both sides meet to review and discuss  project implementation and revise plans when necessary. According to  CTR management officials and JRIP documents we reviewed, these  meetings provide a regular forum for discussion that was not previously  available and have improved communication between DOD officials and  CTR-recipient governments.
section_title: Inherent Risks Remain in Working with Recipient Governments
paragraphs: DOD faces significant challenges in collaborating with CTR-recipient  governments to jointly implement projects and ensure that assistance is  used to meet program objectives. Successful implementation of CTR  projects requires the cooperation of recipient governments, but DOD  cannot fully mitigate the risks involved in working jointly with these  governments. First, working with CTR-recipient governments often  involves lengthy negotiations to reach agreements on various issues  throughout a project’s implementation. This can delay U.S. funded efforts  to help secure or dismantle weapons of mass destruction by months or  years. Second, risks to the project can increase when implementation  begins before the necessary agreements are in place. Third, after  agreements are reached and implementation is under way, additional risk is introduced by the control environment within the recipient governments.  For instance, if a recipient government has a poor control environment risk  increases that the agreed to objectives and conditions will not be met.
In cooperating with CTR-recipient governments, DOD must negotiate a  variety of agreements that can require lengthy negotiations. The highest  level of agreements, called umbrella agreements, provide an overall legal  framework for U.S. and CTR-recipient countries’ cooperation in  implementing projects. Implementing agreements outline the types and  amounts of assistance to be provided for specific CTR projects. For  instance, projects to eliminate strategic nuclear arms, including strategic  bombers, missiles, and related equipment are conducted under the  Strategic Nuclear Arms Elimination Implementing Agreement signed by  DOD and the Ukrainian Ministry of Defense in December 1993.  Agreement amendments update the annual amount of funding that CTR  will provide for a specific project within a recipient country. For example,  the December 2004 agreement amendment for biological weapons  proliferation projects with the government of Kazakhstan provides for  $30 million in CTR funding during fiscal years 2004 and 2005. The recipient  governments must sign agreements or agreement amendments before  projects can begin and funding can be provided or increased, but this may  take time and delay projects, according to CTR officials. According to a  CTR program area manager, the Russian government took more than 18  months to sign an implementing agreement for nuclear weapons  transportation and security projects because it did not want to reveal the  location of nuclear weapons storage sites that the government planned to  close. In 2004, the government of Kazakhstan took more than 6 months to  sign the annual agreement amendment for biological weapons proliferation  projects. According to CTR contractors and officials at Kazakhstani  biological research facilities, the government’s delay slowed efforts to  improve the security and safety of biological pathogens at their institutes.  For CTR biological weapons proliferation prevention projects in Russia,  however, DOD has no implementing agreement. These projects are  implemented through the International Science and Technology Center in  Moscow. Until it can conclude a biological threat reduction implementing  agreement with the Russian government, DOD has limited the types of  projects it initiates in Russia.
Risks to CTR projects can increase when DOD begins implementation  before the necessary agreements are in place with CTR recipient  governments. After more than 10 years, Russia and DOD have yet to  negotiate a transparency agreement that would allow U.S. personnel access  to the CTR-funded fissile material storage facility at Mayak to ensure that it  is being used as intended. DOD designed and built the facility to provide  centralized, safe, secure, and ecologically sound storage for weapons-grade  fissile material from dismantled Russian nuclear warheads. In December  2003, DOD completed the CTR-funded Mayak facility at a cost of about  $335 million, and the Russian government assumed full responsibility for  its operation and maintenance. Although the Russian government has  pledged its commitment to transparency, it has not signed an agreement  with DOD. Therefore, the United States has no reasonable assurance that  Russia will only use the facility to store materials from dismantled nuclear  weapons and not reuse the materials. According to CTR program officials,  the Russian government may soon begin storing nuclear materials at the  Mayak facility without an agreement in place. We first raised concerns  about the lack of a transparency agreement for the Mayak facility in 1994.  Later, in April 1999, we voiced concerns that the United States still lacked clear assurances that Russia would use the Mayak facility in a manner  consistent with all U.S. national security objectives for the project.
Furthermore, two CTR project failures in Russia illustrate the  consequences of DOD not having the necessary agreements in place (see  app. I for additional information). In the early 1990s, DOD agreed to assist  Russia in constructing a facility to dispose of liquid missile propellant,  known as heptyl, which had been drained from intercontinental and  submarine-launched ballistic missiles. DOD spent nearly $95 million over  10 years to build a facility to destroy the heptyl, only to learn in January  2002 that Russia had diverted the heptyl to its commercial space program,  rather than storing it for eventual destruction. As a result, the facility was  never used. The DOD IG reported in 2002 that CTR program officials  negotiated a weak implementing agreement with the Russian government.  Specifically, the agreement did not require the Russian government to  provide the heptyl or provide access for CTR program officials to inspect  the heptyl storage facilities and verify the quantities present.
Similarly, DOD had agreed in the early 1990s to build a facility in Russia to  dispose of solid rocket motors from dismantled missiles. DOD spent almost  $100 million over nearly 10 years to design the facility, despite the concerns  of local residents about the possible environmental impact. In January  2003, Russian officials notified DOD that the regional government had  denied the land allocation permit necessary to begin construction due to  the opposition from local residents. As a result, DOD never began  construction on the facility. The DOD IG found that the implementing  agreement for the design of the solid rocket motor elimination facility at  Votkinsk failed to specify Russian responsibilities for the project. Primarily,  the Russian government was to obtain the necessary land allocation  permits. CTR officials accepted in good faith that Russia would help  implement program objectives and therefore assumed that they did not  need to document the Russian government’s responsibilities. In addition,  despite local protests against construction of the facility from the beginning of the project, DOD project managers did not identify land  allocation as a potential risk until April 2002.
Even after DOD concludes appropriate agreements, however, risks still  may exist due to the control environment of the recipient governments. For  instance, if a recipient government has a poor control environment risk  increases that the agreed to objectives and conditions will not be met. A  good control environment requires that an organization’s structure clearly  defines key areas of authority and responsibility. When the Russian  government reorganized in early 2004, it was uncertain which agencies and  officials were in charge of working with DOD. While the names of some of  the agencies had merely changed, other agencies were subsumed into  larger organizations or completely dissolved. According to CTR program  officials, the reorganization had a significant impact on program  implementation. For example, the CTR Policy Office is renegotiating its  implementing agreements to reflect the new Russian government entities.  CTR projects also experienced delays when the Russian government  reorganized the committee that granted tax exemptions and resolved  customs issues for all CTR assistance entering Russia. Work on the CTR- funded chemical weapons destruction facility in Russia was delayed until  needed equipment was cleared through customs. Furthermore, CTR  recipient governments may not provide adequate access to project sites or  may pursue priorities that compete with CTR program objectives. DOD’s  inability to gain access to all sites where CTR assistance is provided has  been an issue since the CTR program began in 1992. The U.S. government  has been concerned with its ability to examine the use of its CTR-provided  assistance, while CTR-recipient countries have security concerns regarding  U.S. access to sensitive sites. For example, as we reported in March 2003,  DOD had made only limited progress installing security upgrades at  Russian nuclear weapons storage sites and former biological weapons  facilities because Russia would not provide DOD access to several sites.  Since March 2003, Russia has granted DOD access to some nuclear weapon  storage sites, and continues to restrict access to some former biological  weapons facilities.
section_title: DOD Does Not Have a Mechanism to Review CTR Projects Once They Are Completed
paragraphs: While CTR program officials monitor the progress of ongoing projects,  DOD has no mechanism to monitor and evaluate the results of completed  projects in relation to their meeting program objectives. According to  internal control standards, monitoring should assess the quality of project  performance over time. Conducting program evaluations, such as  reviewing completed CTR projects, may be warranted after major changes  in management plans. DOD does not conduct final evaluations of  completed CTR projects and currently has no mechanism to document  lessons learned and apply them to future project planning and  implementation. At its inception, the CTR program primarily provided  equipment to recipient countries, but now the vast majority of assistance is  provided through contracted services. Although the program has shifted to  funding costly, complex, and sometimes high-risk projects that can last for  many years, DOD has not expanded the scope of its project monitoring  process to include evaluations of the efficiency and effectiveness of CTR  projects upon their completion.
In June 2001, we recommended that DOD conduct such evaluations to  improve DOD’s overall program oversight. In response, DOD agreed to  periodically assess the efficiency and effectiveness of CTR assistance,  including contracted services. However, DOD lacks a final review process  to assess the efficiency and effectiveness of completed CTR projects. As of  June 2005, DOD had completed 77 projects, but program officials did not  evaluate and record what went well during a project’s implementation and  what could have been improved to better meet program objectives. While  CTR officials discuss ongoing individual projects performance through the  MDA process, senior CTR management officials acknowledged that  projects are not evaluated upon their completion and such information is  not shared program wide in a systematic manner. As such, it is difficult to  apply lessons learned to future CTR projects as they are being planned and  implemented and avoid past mistakes. Officials stated that conducting final  evaluations could further improve their management of the CTR  program,especially as the program expands into countries outside the  former Soviet Union. Since DOD does not assess the efficiency and  effectiveness of projects as they are completed, it cannot apply the lessons  learned from such evaluations to new and ongoing projects in a systematic  way.
section_title: Conclusion
paragraphs: Since 1992, CTR assistance has helped the states of the former Soviet  Union eliminate and protect their weapons of mass destruction. Although  the CTR program has helped reduce the threat that these weapons could be  stolen or misused, incidents such as the heptyl disposition and solid rocket  motor elimination projects demonstrated significant problems with DOD’s  program management. In the aftermath of these incidents, DOD has  worked to revamp its CTR program management to achieve greater  assurance that projects are implemented according to program objectives.  By standardizing its management approach and applying it consistently  across all CTR program areas, DOD is improving its management of the  CTR program. DOD has greater assurance that all stakeholders, including  recipient governments, are involved in project implementation. CTR  program and project managers have clearer guidance on how to conduct  their work and report on it. Furthermore, DOD has made progress in more  clearly articulating and documenting its cooperative arrangements with  CTR recipient countries, as well as holding recipient governments more  accountable for implementing the CTR projects in their respective  countries.
These improved controls cannot eliminate the risks inherent in the  program, but the goal is to mitigate risk to an appropriate level given the  circumstances. Most significantly, the success of the CTR program requires  the cooperation of recipient governments. Good internal controls help  mitigate the risks from having to rely on recipient governments to sign  agreements, provide access, and support project implementation. Still,  governments can change their project goals, deny access to U.S.  contractors and officials, or withhold permits to allow work to proceed.  DOD’s more robust internal controls have helped minimize the impact of  these actions, but they cannot guarantee a project’s success. The U.S.  government remains concerned about its ability to determine how CTR- provided assistance is being used, while CTR recipient countries continue  to have security concerns regarding U.S. access to their sensitive facilities  and sites.
In addition, while DOD has made progress over the past 2 years in  improving its management of the CTR program, it still does not review the  overall performance of projects upon their completion. As projects are  completed, assessing and documenting lessons learned will allow DOD to  further improve CTR project implementation. As the CTR program  completes more projects and the program begins to expand beyond the  former Soviet Union, such a mechanism will become more important to  overall program management.
section_title: Recommendation for Executive Action
paragraphs: We recommend that the Secretary of Defense conduct performance  reviews upon the completion of CTR projects. Such reviews would provide  a mechanism for documenting lessons learned and applying them to future  project planning and implementation.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided comments on a draft of this report, which are reproduced in  appendix VI. DOD concurred with our recommendation that reviews of  completed CTR projects should be conducted to document and apply  lessons learned. DOD also provided technical comments, which we have  incorporated where appropriate.
We are providing copies of this report to the Secretary of Defense and other  interested congressional committees. We will also make copies available to  others upon request. In addition, this report will be available on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-8979 or christoffj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix VII.
section_title: Two Project Failures Cost the CTR Program Nearly $200 Million
paragraphs: By 2003, two CTR program project failures caused DOD to reassess its  management of the program. In the early 1990s, DOD agreed to assist  Russia in constructing a facility to dispose of liquid missile propellant,  known as heptyl, and build a solid rocket motor disposition facility.  However, DOD terminated these projects after spending nearly $200  million over almost a decade.
In the case of the heptyl disposition facility, DOD spent more than $95  million over 10 years on the facility at Krasnoyarsk, Russia, that was never  used. In 1993, the Russian government asked for CTR assistance to dispose  of heptyl from intercontinental and submarine-launched ballistic missiles  that were being destroyed in compliance with arms control agreements. At  the time, Russian government officials claimed that existing heptyl storage  facilities were full and that they needed a way to dispose of the propellant,  according to DOD officials. DOD officials also stated that Russian officials  had told them that the heptyl could not be used for the Russian commercial  space program. However, when CTR officials were ready to test the almost  completed facility in January 2002, officials from the Russian Aviation and  Space Agency revealed that nearly all of the heptyl had been diverted to the  commercial space program. In February 2003, the Deputy Secretary of  Defense approved the dismantlement and salvage of the no-longer needed  heptyl disposition facility.
According to the DOD IG, a variety of inadequate management controls  contributed to the heptyl project failure. The IG reported that AT&L was  not assuming its role in managing the CTR program by providing input and  direction for projects. Rather, the CTR Policy Office, which had little  experience in following DOD acquisition guidelines, establishing  milestones, and identifying risks, was managing daily CTR project  activities. Because AT&L was not performing adequate oversight of the program, CTR program officials negotiated an implementing agreement  without specific programmatic commitments from the Russian government  and did not thoroughly identify the risks associated with eliminating the  heptyl. Specifically, the agreement did not require the Russian government  to provide the heptyl or provide access for CTR program officials to inspect  the heptyl storage facilities and verify the quantities present. CTR officials  accepted in good faith that Russia would provide the heptyl and therefore  assumed that they did not need to document or oversee the Russian  government’s responsibilities. In assessing the risks of the heptyl project,  CTR project officials failed to identify the possibility that the Russian  government would use the heptyl for other purposes and therefore  developed no mitigation plan.
A second project failed in January 2003. After spending almost 10 years to  design the facility at Votkinsk to destroy solid rocket motors, CTR program  officials ended the project at a cost of almost $100 million. In the early  1990s, Russia had requested CTR assistance to destroy motors from  dismantled missiles in compliance with an arms control agreement.  Originally, the facility was to be located at Perm, but pending construction  of the facility generated environmental opposition from local residents.  The facility was thus moved to Votkinsk in February 1998, where local  residents concerned with the environmental impact of the facility also  began protests. Still, CTR program officials continued with the design of  the facility, remaining optimistic that the regional government would issue  the required permits regardless of opposition. Officials from the Russian  Aviation and Space Agency told CTR program officials in July 2002 that  land for the facility would be allocated no later than September 2002. In a  January 2003 letter, however, Russian officials notified DOD that the  regional government had denied the land allocation permit due to the  opposition from local residents.
Inadequate management practices also contributed to the failure of the  solid rocket motor disposition project at Votkinsk. As with the failed heptyl  project, the DOD IG reported that AT&L did not assume its management  role in overseeing the CTR program. The CTR Policy Office was managing  daily CTR project activities. The implementing agreement for the Votkinsk  project failed to specify Russian responsibilities, such as obtaining the  necessary land allocation permits. In addition, despite the local  environmental protests against construction of the facility from the  beginning, project managers did not identify land allocation as a risk until  April 2002. Furthermore, the contracting processes that were in place  contained no mechanism to terminate the project when costs increased  and the schedule was delayed. DTRA awarded the project contract for the  complete design and construction of the facility rather than contracting in  phases so that possible CTR program losses could be minimized.
section_title: Legislative Mandates Covering the CTR Program
paragraphs: As required by section 3611 of the National Defense Authorization Act for  Fiscal Year 2004, we reviewed the status of DOD’s implementation of  legislative mandates covering the CTR program. Since 1992, Congress has  passed 25 pieces of legislation that guide CTR project activities.  Specifically, Congress has established a series of (1) requirements that  must be met before DOD can fund CTR projects, (2) conditions on CTR  expenditures, and (3) reporting requirements on the CTR program and  project implementation. Figure 3 illustrates the types of congressional  legislation covering the CTR program from fiscal year 1992 to 2004 and  includes those legislative requirements that have lapsed. Over the years,  DOD has mostly complied with these requirements, except for several  occasions when it was late in providing required reports to Congress.  Legislation has recently been proposed that would repeal some DOD  requirements.
Congress has established a variety of requirements that must be met before  DOD can fund CTR projects. For example, in establishing the CTR program  in 1991, Congress required that CTR assistance provided to the countries of  the former Soviet Union could not be expended until the President certified  to Congress that the recipient governments were committed to reducing  their weapons arsenals. According to CTR officials, verifying CTR program  compliance with legislation can be a time-consuming process and may  delay the implementation of projects, but they cannot spend CTR funds  unless all legislative conditions are met. DOD officials involved with  managing the CTR program recognize that Congress is exercising its  oversight responsibilities over the CTR program.
Congress has also placed limits or conditions on how DOD can spend CTR  money. For instance, in Congress placed conditions on CTR money to  prohibit spending in certain areas, such as conventional weapons  destruction and housing for retired or current members of CTR-recipient  countries’ military forces. Also in 2000, Congress halted CTR funding for construction of the Russian chemical weapons destruction facility at  Shchuch’ye until fiscal year 2004 when it granted a waiver.
Congress also requires DOD to submit reports on overall program  implementation, as well as specific projects. Since the beginning of the  CTR program, DOD has mostly complied with its congressional reporting  requirements. However, as we previously reported, from 1994 through 1999  DOD was late in providing its annual report, which accounts for CTR  assistance. Specifically, DOD was 16 months late in submitting its report  for 1997 and more than 10 months late in submitting its report for 1998.  Beginning in fiscal year 2001, the reporting requirement to account for CTR  assistance became part of the annual CTR report. For fiscal years 2002  through 2004, DOD provided its annual CTR report to Congress late.  However, DOD provided its annual report to Congress for fiscal years 2005  and 2006 mostly on time.
Recently, some members of Congress have introduced bills that may lesson  the legislative burden on the CTR program. In February 2005, Senator  Lugar introduced the Nunn-Lugar Cooperative Threat Reduction Act of  2005. The bill, among other actions, would repeal some of the restrictions  that Congress had previously placed on the CTR program. If enacted the bill would remove (1) a Presidential certification requirement for all CTR  recipient countries to receive CTR assistance and (2) the funding  constraints placed on the construction of the CTR-funded chemical  weapons destruction facility in Russia. In February 2005, members of the  House of Representative introduced the Omnibus Nonproliferation and  Anti-nuclear Terrorism Act of 2005. This bill also includes a provision for  the repeal of the same restrictions outlined in the bill introduced by  Senator Lugar.
section_title: Scope and Methodology
paragraphs: To assess DOD’s management and internal controls over the CTR program,  we collected and analyzed DOD documents, including CTR project plans,  briefings, annual reports, and milestone decision authority memorandums.  We also obtained and analyzed all legislation passed since 1992 that covers  the CTR program. We applied the internal standards as described in GAO’s  Standards for Internal Control in the Federal Government. We focused  on those controls most relevant to the CTR program, including  organizational structure, risk assessments, performance measures,  program reviews, communications, and monitoring of projects. We also  reviewed DOD acquisition management guidance as contained in the  Defense Threat Reduction Agency’s Instruction 5000.01 for our  assessment of CTR management controls. Using the federal government  standards and DOD’s guidance, we developed and tested a semi-structured  interview guide that included questions regarding DOD’s internal controls  for the CTR program. We included steps in the development and  administration of the semi-structured interview guide to minimize errors  resulting from the respondents’ interpretation of the questions or from  differences in information available to respondents answering the  questions. We pretested the instrument with three DOD officials. In  addition, an internal survey specialist reviewed our semi-structured  interview guide. We modified the interview guide to reflect the questions  and comments from the pretests and internal review. We used the semi- structured interview guide to interview 30 DOD officials responsible for  managing and implementing the CTR program. We also held meetings with  17 other officials. Specifically, we met with officials from the CTR Policy  Office, AT&L, and DTRA’s Business and Cooperative Threat Reduction (CT)  directorates. Within CT, we obtained information from the director, deputy  director, program and project managers from all five program areas, and  officials from the Program Integration office. In addition, we met with  officials from DTRA offices in Moscow and Almaty and the Threat  Reduction Support Center in Springfield, Virginia. We traveled to the  Russian Federation to observe CTR projects involving strategic offensive  arms elimination and biological weapons proliferation prevention. We met  with Russian officials at the Federal Space Agency, the Federal Agency for  Industry, and the Federal Atomic Energy Agency. We also visited the  Republic of Kazakhstan to observe CTR-funded projects involving  biological weapons proliferation prevention. While in Russia and  Kazakhstan, we met with representatives from all five CTR Integrating  Contractors to obtain information on their roles in implementing CTR  projects. We also reviewed our prior work on the CTR program.
Although information about funding for the CTR program and the  program’s accomplishments is used for background purposes only, we  assessed the reliability of these data by reviewing relevant agency  documents and obtaining information from agency officials. We  determined that the data used were sufficiently reliable for the purposes of  this report.
We performed our work from April 2004 through May 2005 in accordance  with generally accepted government auditing standards.
section_title: Current CTR Program Areas
paragraphs: Since 1992, Congress has authorized DOD to provide more than $5 billion  for the CTR program to help the former states of the Soviet Union,  including Russia, Ukraine, Belarus, Kazakhstan, Uzbekistan, Azerbaijan,  Moldova, and Georgia, secure and eliminate their weapons of mass  destruction and prevent their proliferation. As of April 2005, DOD has  obligated about $4.5 billion in support of the CTR program. Of this  obligated amount, about $2.7 billion funds projects are being implemented  under CTR’s five program areas of biological weapons proliferation  prevention, chemical weapons elimination, nuclear weapons safety and  security, strategic offensive arms elimination, and weapon of mass  destruction proliferation prevention initiative, as shown in figure 4. The  remaining obligations cover completed CTR projects or other program  support areas.
section_title: DOD’s Current Management and Internal Controls for the CTR Program Compared with Internal Control Standards
paragraphs: In managing the CTR program, standards for internal controls in the federal  government provide an overall framework for DOD to establish and  maintain management controls and identify and address major  performance challenges and areas at risk for mismanagement. The five  overall standards for internal control are control environment, risk  assessment, control activities, information and communications, and  monitoring. Each standard contains numerous factors that an  organization’s management can use to evaluate its internal controls. For  example, under the control environment standard, there are about 30  factors listed such as whether an agency’s organizational structure has  appropriate and clear internal reporting requirements. For this report, we  focused on those factors most relevant to CTR program implementation.  The scope of our work thus covered factors such as organizational  structure, risk assessments, performance measures, program reviews,  communications, and monitoring of projects. Table 1 describes the factors  selected in reviewing DOD’s current internal controls for the CTR program.
section_title: Comments from the Department of Defense
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Dave Maurer, Beth Hoffman León,  Josie Sigl, Stephanie Robinson, Nima Patel Edwards, Stacy Edwards, Lynn  Cothern, Judy Pagano, and Mark Dowling contributed to this report. Etana  Finkler also provided assistance.