Defense Acquisitions: Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach
Published: 2010-02-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: By law, GAO is directed to assess the annual progress the Missile Defense Agency (MDA) made in developing and fielding the Ballistic Missile Defense System (BMDS). GAO also assessed MDA's progress toward improving accountability and transparency in agency operations, management processes, and its acquisition strategy. To accomplish this, GAO reviewed asset fielding schedules, test plans and reports, as well as pertinent sections of Department of Defense (DOD) policy to compare MDA's current level of accountability with that of other DOD programs. GAO's fiscal year 2009 assessment of MDA's cost, schedule, and performance progress is more limited than previous assessments because MDA removed key components of schedule and performance goals from its annual report of goals. In addition, though it had committed to, MDA did not report total cost estimates in 2009.
section_title: What GAO Found
paragraphs: Fiscal year 2009 was an unprecedented year of transition for MDA as it experienced its first change of administration, its third MDA Director, shifts in plans for missile defense in Europe as well as a shift in focus for technology development from intercepting missiles during the boost phase to the early intercept phase. Such changes present new challenges for MDA but also opportunities to strengthen acquisition approaches. (1) Progress: MDA achieved several accomplishments. For example, MDA revised its testing approach to better align tests with modeling and simulation needs and undertook a new targets development effort to resolve longstanding problems supplying sufficient and reliable targets. The agency also demonstrated increased levels of performance for some elements through flight and ground testing. Fiscal year 2009 testing indicates an increased level of interoperability among multiple elements, improving both system-level performance and advancing the BMDS models and simulations needed to predict performance. In addition, the agency delivered 83 percent of the assets it planned to deliver by the end of fiscal year 2009. (2) Challenges: While there was progress, all BMDS elements had delays in conducting tests, were unable to accomplish all planned objectives, and experienced performance challenges. Poor target performance continued to be a problem, causing several test delays and leaving several test objectives unfulfilled. The test problems also precluded MDA from gathering key knowledge and affected development of advanced algorithms and homeland defense. These test problems continued to affect the models and simulations used to assess the overall performance of the BMDS. Consequently, comprehensive assessments of its capabilities and limitations are still not possible. MDA also redefined its schedule baseline, eliminating goals for delivering integrated capabilities so we were not able to assess progress in this area. Despite these problems, MDA proceeded with production and fielding of assets. (3) Transparency, Accountability, and Oversight: In 2009, the significant adjustments MDA made to its acquisition approach--terminating the block structure; reducing, eliminating, or not reporting key baselines; and terminating its capability declaration process--and adjustments to the material reported to Congress reduced the transparency and accountability MDA had begun to build. However, MDA is beginning to implement several initiatives--including the adoption of key principles of DOD acquisition regulations--that could improve transparency and accountability and lay the foundation needed for oversight. If these initiatives are implemented in accordance with knowledge-based acquisition principles, an opportunity exits to improve the BMDS acquisition by ensuring MDA programs begin with realistic, transparent plans and baselines. While these initial steps hold promise, they will take time to fully implement and once implemented they will need to be sustained over time and consistently applied.

section_title: Letter
paragraphs: Fiscal year 2009 was an unprecedented year of transition for the Missile  Defense Agency (MDA) as it experienced its first change of administration,  its third MDA Director, its third Ballistic Missile Defense System (BMDS)  acquisition strategy, and a reduction of approximately $1 billion from its  annual budget request. Established in 2002 and directed by the President  to “deploy a set of initial missile defense capabilities beginning in 2004,”  MDA has developed and deployed missile defense capabilities while  struggling to provide the transparency and accountability necessary for  the Department of Defense’s (DOD) largest single acquisition program— spending between approximately $7 billion and $9.5 billion per year.
Since 2002, Congress has directed GAO to assess MDA’s annual fiscal year  cost, schedule, testing, and performance progress in developing the  BMDS. We have delivered assessments of MDA’s progress covering fiscal  years 2003 through 2008 and are currently mandated to continue delivering  assessments through fiscal year 2013. According to this mandate we are  required to assess MDA’s fiscal year progress against the annual goals it is  required to report to Congress each February. However, as agreed with  your staff, this year we are providing a more limited assessment of MDA’s  progress than we have in the past because MDA removed key components  of the schedule and performance goals it had previously reported from its  annual report of goals. MDA also did not report the total acquisition cost  goals that it had committed to include in this year’s report, nor did it  report top-level test goals. In addition, the agency did not report its annual  goals until August 2009—only a month and a half before the end of fiscal  year 2009.
This report provides an assessment of MDA’s cost, schedule, testing, and  performance progress in fiscal year 2009 as well as problems and  challenges in these areas. We also report on the progress MDA made in  improving accountability and transparency over the past year. Although  prior reports have included an assessment of MDA’s cost progress on its  prime contracts, we will report on this analysis in more detail in a separate  report. In addition, given the number, scope and breadth of changes made  to missile defense this year, we discuss those changes as well as their  implications on acquisitions.
In order to provide some measure of MDA’s progress, given the late and  limited goals established by MDA, we had to determine alternative  comparison points and we discussed these with MDA and our  congressional clients. We assessed MDA’s progress in testing by  comparing the tests conducted against the goals as presented in its  October 2008 Integrated Master Test Plan and the annually submitted  budget justification documents. In terms of performance, we assessed the  capabilities demonstrated through the tests. For schedule, we compared  progress against fiscal year 2008 goals, budget justification material, and  the September 2008 execution master fielding schedule.
To assess progress during fiscal year 2009, we examined the  accomplishments of eight BMDS elements that MDA is currently  developing and fielding: the Aegis Ballistic Missile Defense (Aegis BMD);  Airborne Laser (ABL); BMDS Sensors; Command, Control, Battle  Management, and Communications (C2BMC); Ground-based Midcourse  Defense (GMD); Space Tracking and Surveillance System (STSS); Targets  and Countermeasures; and Terminal High Altitude Area Defense  (THAAD). These elements collectively account for about 76 percent of  MDA’s research and development budget. We also examined MDA’s  Fiscal Year 2009 BMDS Accountability Report, Program Execution  Reviews, test plans and reports, and production plans. We interviewed  officials within program offices and within MDA functional directorat such as the Directorate for Advanced Technology. In addition, we  discussed the elements’ test programs and results with the BMDS  Operational Test Agency and DOD’s Office of the Director, Operational  Test and Evaluation.
To follow up on the progress MDA made to improve transparency and  accountability, we held discussions with officials in MDA’s Directorate Business Operations to discuss MDA’s block structure termination. In  addition, we reviewed pertinent DOD policies to compare MDA’s current  level of accountability with that of other DOD programs. We interviewed  officials from the Office of the Under Secretary of Defense for Acquis Technology and Logistics to discuss its role in conducting quarterly  execution reviews as well as the oversight role of the Missile Defe Executive Board. Lastly, we met with officials involved in MDA’s  Integration Synchronization Group to discuss how the agency is managing  es. Our scope and methodology is  and reporting against its internal baselin discussed in more detail in appendix II.
We conducted this performance audit from April 2009 to February 2010 in  accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: MDA’s BMDS is being designed to counter ballistic missiles of all rang short, medium, intermediate, and long. Since ballistic missiles have  different ranges, speeds, sizes, and performance characteristics, MDA is  employing an integrated and layered architecture to provide multiple  opportunities to destroy ballistic missiles before they can reach their  targets. The system’s architecture includes networked space-based  sensors as well as ground- and sea-based radars, ground- and sea-based  interceptor missiles, and a command and control, battle management, and  communications network providing the warfighter with the necessary  communication links to the sensors and interceptor missiles.
A possible engagement scenario to defend against an intercontinental  ballistic missile would occur as follows:  Infrared sensors aboard early-warning satellites detect the hot plume  of a missile launch and alert the command authority of a possible  attack.
Upon receiving the alert, land-  or sea-based radars are directed to  track the various objects released from the missile and, if so designed,  to identify the warhead from among spent rocket motors,  countermeasures, and debris.
When the trajectory of the missile’s warhead has been adequately  established, an interceptor—consisting of a kill vehicle mounted atop a  booster—is launched to engage the threat. The interceptor boosts  itself toward a predicted intercept point and releases the kill vehicle.
The kill vehicle uses its onboard sensors and divert thrusters to detect ead. With a combined closing  identify, and steer itself into the warh speed of approximately 10 kilometers per second (22,000 miles per  hour), the warhead is destroyed above the atmosphere through a to kill” collision with the kill vehicle.
Some interceptors use sensors to steer themselves into the inbound  ballistic missile. Inside the atmosphere, weapon systems kill the ballistic missile using a range of mechanisms, such as direct collision between the  interceptor missile and the inbound ballistic missile, or using the  combined effects of a blast fragmentation warhead (heat, pressure, and  shrapnel) in cases where a direct hit does not occur.
In the August 2009 BMDS Accountability Report, MDA presents the BMD performance from the perspectives of homeland defense and  regional/theater capabilities. Homeland defense uses the capabilities of  s  Ground-based Interceptors (GBI), Aegis BMD assets, and BMDS radar against the threat from intercontinental and intermediate-range ballist ic  missiles, while regional and theater defense use Aegis BMD Standard  Missile-3 (SM-3) and THAAD interceptors with mobile radars against  threats from medium-range missiles and short-range ballistic missiles.
Table 1 provides a brief description of eight BMDS elements that are  currently under development by MDA.
section_title: Significant Adjustments Made to Missile Defense in Fiscal Year 2009
paragraphs: The new administration proposed significant changes to the BMDS  program in 2009 including program terminations and changes to some of  the BMDS elements we reported on in the past, as well as changes to plans  for missile defense in Europe. Administration proposals culminated in  reductions of approximately $1 billion from MDA’s budget request for  fiscal year 2010. In the spring of 2009, the Secretary of Defense  recommended termination of the Multiple Kill Vehicle element. Originally  designed as an optional warhead for all midcourse interceptors, MDA  terminated the Multiple Kill Vehicle element because of feasibility issues  raised about this technology, which was still in its early stages of  development, as well as a decision to refocus MDA’s resources on new  technologies aimed at early intercept of ballistic missiles. MDA also  terminated its Kinetic Energy Interceptor element because of technical  issues, its incompatibility with operational infrastructures, and delays  during development. It was originally designed as a mobile land-based  missile defense system to destroy medium, intermediate, and  intercontinental ballistic missiles during the boost and midcourse phases  of their flight. The ABL program was also significantly affected by the  Secretary of Defense’s proposal to designate it as a technology program  and cancel the plans for the purchase of a second aircraft that would have  provided an operational capability. In addition, MDA requested increased  funding for the Aegis BMD and THAAD programs for fiscal year 2010  following administration recommendations. MDA plans to use these funds  to move both elements toward meeting full funding policies, to increase  production for Aegis BMD and THAAD interceptors, to increase the  interceptor production rate and number of THAAD batteries, and to  increase the number of Aegis BMD ships.
MDA is also responding to the new administration’s shift in its approach to  European missile defense. In September 2009, DOD altered its approach to  European defense, which originally focused on GBIs from the GMD  element and a large fixed radar as well as transportable X-Band radars,  and is now focusing on providing defenses against long-range threats to  the United States and short-, medium-, and intermediate-range Iranian  threats to Europe. This new “Phased, Adaptive Approach” consists  primarily of Aegis BMD sea-based and land-based systems and  interceptors, as well as various sensors to be deployed over time as the  various capabilities are matured. According to DOD, this new approach  offers a number of improvements over the previous architecture, such as  providing missile defenses sooner with greater flexibility to meet evolving  threats, providing more opportunities to involve close allies, and delivering  greater capability to defend against a large number of threat missiles.
In addition, during fiscal year 2009, MDA transitioned to a new Director  and the agency’s development effort was rebalanced to focus more on  regional/theater missile defense. This rebalancing included shifting  technology development efforts from boost-phase intercept technologies  to early intercept technologies (or ascent phase). MDA officials state that  because early intercept technology initiates intercept as early as possible  to execute a shoot-look-shoot tactic and defeat a threat before  countermeasures are deployed, it will ultimately reduce the number of  interceptors required to defeat a raid of threat missiles and save on the  costs of maintaining a significant number of expensive interceptors to  destroy advanced countermeasures in a later phase of a threat missile’s  flight. According to the MDA Director, this technology will force the  deployment of countermeasures early in flight where they are less  effective.
In June 2009, MDA also began to change its acquisition management  strategy. From its inception in 2002 to December 2007, MDA managed the  acquisition of missile defense capabilities by organizing the development  effort into 2-year increments known as blocks. Each block was intended to  provide the BMDS with capabilities that enhanced the development and  overall performance of the system. The first 2-year block—Block 2004— fielded a limited initial capability that included early versions of the GMD,  Aegis BMD, Patriot Advanced Capability-3, and C2BMC elements as well  as various sensors. The agency’s second 2-year block—Block 2006— culminated on December 31, 2007, and fielded additional BMDS assets. On  December 7, 2007, according to MDA in response to recommendations  from GAO, MDA’s Director announced a new acquisition management  strategy to better communicate its plans and goals to Congress. The  agency’s new approach was based on fielding capabilities that address  particular threats as opposed to a biennial time period. This approach  divided fielding capabilities into five blocks.
The capabilities-based five-block approach included several positive  changes, including the commitment by DOD to establish total acquisition  costs and unit cost for selected block assets, including in a block only  those elements or components that will be fielded during the block, and  abandoning the practice of deferring work from one block to another.  MDA was still transitioning to this new capabilities-based block approach  when the MDA Director terminated it in June 2009—a year and a half after  it was created. According to MDA, the agency terminated the capability- based block structure to address the explanatory statement accompanying  the Department of Defense Appropriations Act, 2009, which stated that  MDA’s “justification materials should no longer be presented in the Block  format, but rather by fiscal year for each activity within the program  element.” The agency has decided that it will manage the BMDS as a single  integrated program and is in the process of determining how it will  implement changes to its acquisition management strategy.
section_title: Progress Made in Fiscal Year 2009
paragraphs: In fiscal year 2009, MDA achieved several noteworthy accomplishments.  For example, MDA revised its testing approach to better align tests with  modeling and simulation needs and is undertaking a new targets  development effort to resolve long-standing problems supplying sufficient  and reliable targets. The agency also demonstrated increased levels of  performance for some of its BMDS elements through flight and ground  testing. MDA testing achievements during the year indicate an increased  level of interoperability among multiple elements, improving both system- level performance and advancing the validation of BMDS models and  simulations needed to predict performance. In addition, the agency  delivered most of the assets as planned by the end of fiscal year 2009.
subsections: section_title: MDA Revised Its Testing Approach
paragraphs: In fiscal year 2009, MDA revised its testing approach in response to GAO  and DOD concerns. In March 2009 we reported that MDA’s Integrated  Master Test Plan—its test baseline—was not effective for management  and oversight because it was revised frequently, only extended through the  following fiscal year and was not well integrated with other key aspects of  testing such as target acquisitions. Most of the annual revisions to the test  baseline were occurring either because MDA changed the substance of  tests, changed the timing of tests, or added tests to the baseline. In other  instances, MDA canceled planned tests which also affected the test  baseline. In addition, the BMDS Operational Test Agency identified several  limitations in the previous BMDS test program, including unaccredited  models and simulations, flight test artificialities, and inadequate modeling  of some environmental conditions.
Members of Congress also expressed concern with MDA’s test approach.  For example, in the fiscal year 2008 National Defense Authorization Act  conference report, conferees noted that MDA failed to ensure an adequate  testing program and that its test and targets program needed to be  managed in a way that fully supported high-priority near-term programs.
MDA extensively revised its test plan in fiscal year 2009 to address many  of these concerns. For example, the new Integrated Master Test Plan  bases test scenarios on modeling and simulation needs and extends the  test baseline through 2015, which allows for better estimation of target  needs, range requirements, and test assets. As part of the revised test plan,  MDA scheduled dedicated periods of developmental and operational  testing, during which the system configuration will remain fixed to allow  the warfighter to carry out training, tactics, techniques, and procedures for  developmental and operational evaluation. Additionally, the new test plan  will provide sufficient time after test events to conduct a full post-test  analysis. These improvements are important because BMDS performance  cannot be fully assessed until models and simulations are accredited and  validated and the test program cannot be executed without meeting its  targets needs.
section_title: MDA Began New Target Development Effort
paragraphs: In July 2009, MDA also initiated a new target acquisition strategy to  address recurring target performance issues and increases in target costs.  According to the Director of MDA, this new target approach is based on  streamlining a set of classes of targets to increase quality control of an  inventory of identical targets that represent general threat characteristics  to account for intelligence uncertainties. He further stated that a goal of  the new target acquisition strategy is to minimize the number of targets  needed to emulate specific threats and establish backup targets, which  will be available in 2012.
Targets have been a recurring cause of flight test delays, cancellations, and  failures since 2006. In the past, we reported that the THAAD program was  unable to achieve its first intercept attempt in 2006 because the target  malfunctioned. The program also experienced target anomalies in 2007  that precluded the completion of two radar characterization tests. During  the same year, the GMD program experienced long-term effects on its  flight test schedule when it was unable to achieve all primary test  objectives because of a target failure. We also reported in March 2009 that  the Aegis BMD program was unable to conduct an intercept because the  target was not available. In addition, in its January 2009 report to the  defense committees, MDA acknowledged target availability and reliability  problems and reported its plan for a new target acquisition strategy to  address these issues and improve costs, quality, and reliability.
In revising its target acquisition strategy, MDA solicited input from  industry in an effort to better understand possible new target solutions  that might be available to improve cost, quality, and performance. To  leverage industry capability and promote a more competitive contract  environment, MDA decided to use multiple contractors with multiple  contracts instead of a single prime contract, increasing its flexibility to  respond to changing program requirements. The agency plans to award a  new contract for each class of target needed to execute the BMDS test  plan. MDA will begin making decisions on contract awards and new target  designs over the next year. According to program officials, MDA originally  planned to issue five requests for proposals for new contracts in fiscal year  2010 and one additional request in fiscal year 2011. However, to reflect  changes in the test plan and subsequent changes to the acquisition  strategy, the program now plans to issue two requests for proposals in  fiscal year 2010 and one in fiscal year 2011. The Targets and  Countermeasures program anticipates that the first targets will be  delivered under the new strategy in fiscal year 2012, and the first  intercontinental ballistic missile target is expected to be delivered in fiscal  year 2013.
MDA also made progress in several ongoing target development efforts  that could enhance the ability to test the BMDS. During fiscal year 2009,  the Targets and Countermeasures program made progress in developing  four new targets—the LV-2 target, Aegis Readiness Assessment Vehicle-C  target, a new medium range target, and the Extended-Long Range Air  Launched Target. Each target adds a new capability to MDA’s target  portfolio. For example, the LV-2 target provides the potential for  significantly expanding the intermediate range payload and range  performance over current inventory capabilities. The Aegis Readiness  Assessment Vehicle-C target provides a new, low-cost capability as it is  designed to contribute additional separating and maneuvering capabilities  in short- and medium-range targets. MDA’s new Medium Range Target  provides improved kill assessment capability at this range. In addition,  the new Extended-Long Range Air Launched Target is a medium-range  target that provides a greater range capability than previous air-launched  targets and adds the ability to deploy associated objects—a capability not  currently available in other similar target types. MDA expects each of  these targets to be ready for use in flight tests in fiscal year 2010.
section_title: MDA Demonstrated Some Improved Performance through Testing
paragraphs: In fiscal year 2009, MDA conducted several ground tests and flight tests  demonstrating improved performance in several areas of the BMDS  including element-level functionality, theater and regional performance,  and interoperability. Table 2 identifies key test events achieved in fiscal  year 2009 for each element.
In June 2009, the ABL program successfully completed its first two  tracking tests against boosting missile targets. These tests marked the first  time ABL demonstrated a complete low-power engagement sequence  against a boosting target. In addition, the ABL was able to demonstrate its  ability to fire its high energy laser in an airborne environment during a  flight test in August 2009. During this test, the laser was fired into a  calorimeter on board the aircraft to capture the laser’s energy and  measure performance characteristics of the laser’s beam.
The Aegis BMD program also demonstrated increased levels of element  performance through Navy fleet exercises and developmental tests. For  example, Aegis BMD demonstrated, for the first time, its capability to  destroy a ballistic missile in the terminal phase of flight using Standard  Missile-2 Block IV missiles while simultaneously conducting a mission  using the Standard Missile-2 Block IIIA missile against a cruise missile  target. The program also conducted successful developmental component  tests for the next generation of the Aegis BMD interceptor—the SM-3  Block IB. Developmental testing will continue into 2010. In addition, the  program successfully demonstrated that the latest software release of the  Aegis BMD system had the capability to support the program’s next  generation interceptor during simulated SM-3 Block IB engagements.
The C2BMC program also satisfied multiple test objectives and increased  its capability in fiscal year 2009. The program participated in many system- level tests during the year that enabled it to demonstrate multiple  capabilities, including improved situational awareness and sensor  management. During testing, C2BMC used multi-sensor correlation and  provided integrated situational awareness for weapons release decisions.
GMD, for the first time, used information from multiple sensors to develop  and successfully conduct an intercept of a live target during a flight test. In  December 2008, target information from four different sensors and  satellite data were input into the GMD fire control system to develop an  intercept plan. The involvement of multiple sensors provides better  information to develop an engagement. In addition, GMD made progress in  addressing BMDS Operational Test Agency concerns regarding the  formatting, tracking, and accounting of messages from GMD sensors. For  example, MDA added test instrumentation to collect data for  regional/theater tests communications. However, the agency still faces  ongoing challenges assessing timeliness with the exchange of messages at  the strategic level. According to BMDS Operational Test Agency officials,  they continue to work with MDA to resolve this issue. Key to the  integration and functionality of the BMDS is communications and message  traffic. The timely reception of messages from sensors to weapon systems  is key to support decisions and achieve effective intercepts. In March 2009,  we reported that these data management problems prevented the analysis  of the timeliness of message data, according to BMDS Operational Test  Agency officials.
The STSS program successfully completed the ground testing and  integration of components to support the launch of its two demonstration  satellites in September 2009. These satellites will use onboard infrared  sensors to detect, track, and discriminate ballistic missiles throughout  their trajectories.
THAAD also demonstrated improved element-level functionality when it  successfully launched a salvo of two THAAD interceptors to intercept a  separating target inside the earth’s atmosphere. The primary interceptor  hit the target and the second interceptor hit the largest remaining piece of  target debris seconds later.
section_title: BMDS Demonstrated Improved Theater and Regional Performance and an Increased Level of Interoperability
paragraphs: Regional and theater BMDS assets—Aegis BMD and THAAD—succeeded  in demonstrating improved interoperability in fiscal year 2009. For  example, during a THAAD intercept test, Aegis BMD tracked a target and  provided the information to THAAD’s fire control. As a result, the missile  was successfully engaged by THAAD. Additionally, during this test, the  forward-based radar supporting THAAD was also able to discriminate the  threat reentry vehicle from other objects and provide the information to  support the engagement. According to program officials, the THAAD  element reported that C2BMC provided accurate and timely status  information for the BMDS as well as situational awareness of the test to  the warfighter.
MDA also demonstrated interoperability for BMDS elements during  several ground tests in fiscal year 2009. For example, during one ground  test—GTD-03—MDA successfully demonstrated simultaneous theater and  regional capabilities using operational BMDS hardware and actual  communications between them. In addition, MDA demonstrated  simultaneous BMDS capabilities to conduct training while the BMDS  network remained operational during this test. This capability allows MDA  to conduct development activities while maintaining readiness to engage  in missile defense operations. This ground test also allowed several BMDS  elements to demonstrate that they could successfully exchange data with  other elements. Additionally, in December 2008 numerous elements  worked together to support system-level post-flight reconstruction needed  to validate BMDS models and simulations. This system-level post-flight  reconstruction for flight test FTX-03 was the first ever and was highly  successful because different MDA groups achieved the same results,  according to MDA officials.
section_title: MDA Made Progress Developing and Integrating Models and Simulations
paragraphs: MDA took significant steps forward in fiscal year 2009 in developing the  modeling and simulation tools necessary to understand BMDS  performance against strategic and theater/regional threats. Because the  potential combinations of BMDS configurations, intercept scenarios, and  missile threats are too numerous for ground and flight testing, assessing  overall BMDS performance depends upon the use of models and  simulations to understand the capabilities and limitations of the system.  Such an end-to-end system-level simulation brings together the capabilities  of various element models in order to analyze how the BMDS integrated  and fielded radars, communication networks, and interceptors perform  during scenarios. However, to work effectively these models and  simulations need to be anchored to data from ground and flight tests and  validated by independent evaluators—the BMDS Operational Test  Agency—in order to have confidence in their results. Moreover, the  system-level simulation itself is expected to change over time as additional  models become available to represent the evolving BMDS configuration.
In March 2009 we reported that MDA experienced several problems in its  overall modeling and simulation program, which negatively affected the  2007 performance assessment and led to the cancellation of the 2008  performance assessment. Performance Assessment 2007 was  unsuccessful primarily because of inadequate flight and ground test data  for verification and validation to support accreditation and a lack of  common threat and environment input data among element models. MDA  officials canceled their 2008 performance assessment efforts in April 2008  because of developmental risks associated with modeling and simulations,  focusing instead on testing and models for Performance Assessment 2009.
In fiscal year 2009, MDA made some progress integrating the individual  element models and simulations for Performance Assessment 2009. A  leading accomplishment was the development of a system-level simulation  for regional and theater scenarios in addition to existing strategic  scenarios for a more complete analysis of BMDS performance.  Performance Assessment 2007 only included homeland defense scenarios  against strategic threats. One of MDA’s goals for the performance  assessment is the integration of models that communicate like the  networked BMDS. As of October 2009, Performance Assessment 2009  achieved interactive communications among the element models and  simulations. In addition, MDA achieved consistency in representing the  threat missile and post-intercept data among all models and scenarios,  which was also a weakness of Performance Assessment 2007. Finally, the  BMDS Operational Test Agency observed that conducting Performance  Assessment 2009 is helping to build confidence in BMDS-level simulation  capability for the subsequent Performance Assessment 2010.
section_title: MDA Delivered Many Assets as Planned in Fiscal Year 2009
paragraphs: In fiscal year 2009, MDA met many of its delivery goals. Four MDA  elements—Aegis BMD, GMD, Sensors, and C2BMC—were scheduled to  deliver a total of 41 assets and capabilities in fiscal year 2009. MDA  delivered 34 of these assets or 83 percent. Table 3 outlines BMDS asset  deliveries in fiscal year 2009.
Aegis BMD planned to install the Aegis Weapons System 3.6.1 software on  20 ships and deliver 10 SM-3 missiles in fiscal year 2009. The program met  its goal to deliver the 10 missiles and began to deliver additional rounds,  initially designated for 2010, ahead of schedule. However, the program fell  behind on its goal of installing the 3.6.1 software on 20 ships, delivering 18  by the end of the fiscal year 2009. Aegis BMD officials pointed out that all  ship sets were available but because of real-world national security  situations, these ships were not available for installations in fiscal year  2009. Nonetheless, one of the remaining ships was completed in December  2009 and another will be completed by March 2010. In fiscal year 2009,  Aegis BMD also delivered an additional ship set with the next generation  Aegis BMD Weapon System, 4.0.1, for a total of 19 ship deliveries.
The GMD program also partially met its delivery goals in fiscal year 2009.  The program delivered an additional silo at Vandenberg Air Force Base as  planned, but lagged in its GBI deliveries. For example, in fiscal year 2009,  GMD emplaced three interceptors that were initially planned for fiscal  year 2008 and only one of the three interceptors planned for fiscal year  2009.
The Sensors program met most of its delivery goals, successfully fielding a  new near-term discrimination algorithm, activating an additional AN/TPY-2  radar site, and delivering an additional AN/TPY-2 radar. However, it fell  short of meeting all of its delivery goals for the fiscal year. Although the  program completed the construction for the Thule radar site ahead of  schedule in fiscal year 2008, it was unable to deliver Thule radar  communications and upgrades as planned in fiscal year 2009. These  activities have been delayed until fiscal year 2010.
Finally, C2BMC delivered four additional C2BMC Web browsers, five work  stations, and an additional combatant command suite. Additionally, the  program office rolled out the Global Engagement Manager suite and added  four work stations that support it. However, it was unable to meet its  schedule baseline goal of an additional fielding and site activation to  declare its next spiral operational. This was due to major program  restructures needed to accelerate C2BMC capabilities for other BMDS  elements as well as programmatic changes to fulfill warfighter requests  and meet new administration direction.
section_title: Significant Challenges Remain in Developing the BMDS
paragraphs: While there was progress in addressing concerns about test planning and  target development as well as in delivering assets, all BMDS elements  experienced delays in conducting tests, were unable to accomplish all  planned objectives, and experienced performance challenges. Poor target  performance continued to be a problem causing several test delays and  leaving several test objectives unfulfilled. The test problems also  precluded the agency from gathering key knowledge through tests  specified by the MDA Director that were originally planned to be  completed in fiscal year 2008. MDA’s efforts to develop advanced  algorithms and its efforts to demonstrate homeland defense were also  affected by target issues. These shortfalls in testing continued to delay  validation of the models and simulations used to assess the overall  performance of the BMDS. Consequently, comprehensive assessments of  the capabilities and limitations of the BMDS are still not possible. MDA  also redefined its schedule baseline, eliminating goals for delivering  integrated capabilities so we were not able to assess MDA’s progress in  this key area.
subsections: section_title: MDA Experienced Testing Delays and Shortfalls Across the Board in Fiscal Year 2009
paragraphs: During fiscal year 2009, although several tests showed progress in  individual elements and some system-level capabilities, all BMDS elements  experienced test delays and shortfalls in part because of problems with  the availability and performance of target missiles. None of the elements  conducted all planned tests as scheduled and none achieved all planned  objectives. Table 4 outlines BMDS test and target issues in fiscal year 2009.
Two BMDS elements—ABL and C2BMC—experienced delays in achieving  fiscal year 2009 test events. For example, ABL experienced delays in  development and ground testing that resulted in the delay of its first full  flight test demonstration until fiscal year 2010. Additionally, C2BMC was  unable to conduct testing needed to further develop its next spiral  capability because of BMDS-level delays in developing the models and  simulations needed to conduct this testing. Major program restructures  needed to accelerate C2BMC capabilities for other BMDS elements and  programmatic changes to fulfill warfighter requests and meet new  administration direction also contributed to C2BMC’s inability to conduct  planned fiscal year 2009 testing.
As noted in table 4, targets affected the BMDS test program for four  elements in fiscal year 2009. The Aegis BMD, GMD, Sensors, and THAAD  test program were affected by either target availability or target reliability  and performance issues. In fiscal year 2009, targets contributed to a test  cancellation and test delays and prevented elements from completing tests  or achieving all test objectives. One test for Aegis BMD—FTM-15—was  originally projected to use the new Flexible Target Family’s LV-2 target in  fiscal year 2008, but because of qualification difficulties, the target was  unavailable and the test was not conducted. This test was planned as the  first Aegis BMD SM-3 engagement against an intermediate-range target. It  was also expected to verify interoperability of Aegis BMD, a Sensors radar,  and C2BMC. As of December 2009, MDA had canceled the test and  planned to combine several of the FTM-15 objectives with those in a future  flight test in 2013—FTM-23. However, as of February 2010, the Director of  MDA stated that the test is being rescheduled for 2011. Test  documentation was not provided for our review so it remains unclear  whether the test will include the original test objectives, target, and BMDS  hardware and software configurations.
The GMD and Sensors programs were also unable to complete all planned  objectives because of a target failure during an intercept test. During a  December 2008 flight test—FTG-05—the target failed to release planned  countermeasures. A similar target failure was experienced in a prior 2008  test—FTX-03—and MDA’s risk assessments leading up to the FTG-05 test  could not determine the root cause of the failure. These risk assessments  determined that a similar failure would be “likely” and the consequences  “severe” if MDA proceeded with the test in December 2008, even after  taking mitigation steps. According to the Defense Contract Management  Agency, the cost to execute FTG-05 exceeded $210 million. This was the  last planned flight test using this type of target. As a result of the target  failure, GMD was unable to assess the Capability Enhancement-I kill  vehicle against countermeasures. According to the July 2009 Integrated  Master Test Plan, this test is now planned to be conducted in the third  quarter of fiscal year 2011—nearly 4 years after this configuration  completed fielding.
The GMD program had to delay its second planned fiscal year 2009  intercept test—FTG-06—to fiscal year 2010 because pretest analysis raised  concerns that the target may not perform as required. This test was  important because it was planned as the first test of GMD’s enhanced  version of the kill vehicle called the Capability Enhancement II  exoatmospheric kill vehicle. This test was also designed to demonstrate a  long-flight time for the GBI and GMD’s capability against  countermeasures. In early 2009, MDA altered the target to present a more  representative threat. Since MDA did not have modeling data to represent  the new characteristics of the target, MDA officials were concerned about  the target’s expected performance and decided to delay the test. In  January 2010, MDA conducted FTG-06. However, all test objectives were  not met as the GBI failed to intercept the target as planned. According to  an MDA official, a Failure Review Board was convened to investigate the  test results, but its investigation is expected to take months to complete.
As we reported in March 2009, THAAD program officials had to reschedule  the planned fiscal year 2008 BMDS-level event, FTT-10, into fiscal year  2009 because of a target malfunction. THAAD successfully completed  this test event in fiscal year 2009. In addition, a Short Range Air Launch  Target planned for use in a third quarter fiscal year 2009 THAAD flight test FTT-11, had a component failure and subsequently needed to be  requalified. This failure caused the THAAD program to modify its plannedflight test objectives and move the test into fiscal year 2010, also resulting  in delays to a subsequent test—FTT-12. FTT-11 was conducted in  December 2009 but could not be completed due to failure of the target  missile. The air-launched target was successfully deployed from a  transport aircraft, but the target’s rocket motor did not ignite. The THAAD  interceptor was not launched and test objectives were not achieved.  According to the Director of MDA, the Failure Review Board was  concluding its investigation of the root cause of this failure. The board’s  report was not available during our audit.
Target reliability and failures in fiscal year 2009 also prevented several  elements from achieving all planned objectives. In March 2009, Aegis BMD  experienced target difficulties when two refurbished lower-cost Army  targets for a short-range mission fell short of their expected trajectory.  One target was outside the intercept control area and Aegis BMD was not  able to fire the interceptor because of safety limitations. In the second test,  the target, while short of its expected trajectory, fell in the intercept  control area and was successfully intercepted.
section_title: It Will Be Several Years Before New Target Acquisition Approach Addresses Problems
paragraphs: It will be several years before MDA’s new approach to target development  and acquisitions will be fully implemented because most targets needed  through fiscal year 2011 are already under contract and will not be  affected by the new strategy. The activities under existing contracts will  not be complete until 2013. Moreover, MDA’s implementation of a new  acquisition management strategy does not necessarily mean that any  particular target currently being used, such as the LV-2, will be phased out  of the test program. MDA could decide to continue to use an existing  target under the new strategy, and as a result, some existing target missiles  could continue to be procured under new contracts.
MDA has not presented a complete business case for proceeding with a  new target acquisition management strategy. A complete business case  includes establishing top-level cost, schedule, and performance baselines  available internally and externally for oversight. It is the essential first  step in any acquisition program because it sets the stage for acquisition  and execution. Program officials told us that they would have cost,  schedule, and performance baselines finalized and documented as part of  the decision to proceed with new contract awards. These baselines,  however, will be very detailed and spread across multiple documents and  therefore are unsuitable for internal and external oversight. The officials  further stated that they do not intend to establish top-level cost, schedule,  and performance baseline measures similar to approved program  baselines that are established for DOD’s major defense acquisitions to  provide accountability. In September 2008, we reported that MDA had  difficulty in developing and supplying new targets in part because a sound  business case was not developed before significant decisions were made.  In that report we recommended that MDA develop cost, schedule and  performance baselines as part of an effort to establish a sound business  case for each new class of target under development.
As part of the new target development efforts, MDA also developed a new  cost model. However, because the cost model and test baseline are  continually updated, the Targets and Countermeasure program continues  to lack solid cost baselines against which progress can be measured.  According to the Director of MDA, the agency will continue to update its  cost model as the Integrated Master Test Plan changes, noting that where  the technical content of the test plan remains constant, cost, schedule, and  performance baselines can be measured from year to year. However, as  we reported in March 2009, the Integrated Master Test Plan changes  frequently. In fact, the latest approved version is dated July 2009, and  according to MDA’s Director, a revised version of the Integrated Master  Test Plan is expected in March 2010, which limits the baseline’s stability to  approximately 8 months and limits our ability to measure MDA’s progress  against a cost baseline.
MDA’s ability to develop an accurate cost baseline is also affected by the  lack of historical data available for targets or for other similar missiles.  Program officials said that they are now collecting more useful cost data  for new contracts by requiring more detailed cost reporting from their  contractors. This approach will allow program officials to gather more  complete and accurate data over time to make the new cost model a more  powerful cost estimating tool.
section_title: Testing and Targets Problems Affected Planned Progress of Overall BMDS Development
paragraphs: The inability of MDA to successfully conduct its test plan precluded the  agency from collecting critical information needed for key decisions and  significantly affected development of advanced algorithms and homeland  defense capabilities.
subsections: section_title: Key Director’s Knowledge Points Delayed Again
paragraphs: In fiscal year 2009, MDA was unable to accomplish any of the Director’s  knowledge points that were to be achieved through tests. Several of these  tests were originally planned for fiscal year 2008, but were delayed into  2009 and then again delayed into fiscal years 2010 and 2011. Table 5 shows  the original test date and MDA’s current estimate for obtaining the  necessary knowledge.
Target issues continued to affect MDA’s ability to fully develop algorithms  needed for discrimination capability. In March 2009, we reported that  multiple elements experienced test failures which caused delays in  collecting data needed to develop discrimination capability. For example,  in 2007, two THAAD radar characterization tests were unsuccessful  because of target anomalies. These tests were designed with  characteristics needed for radar observation in support of advanced  discrimination algorithm development. However, target problems  prevented an opportunity for the radar to exercise all of the planned  algorithms, causing a loss of expected data. Similarly, in a 2008 sensor  characterization test, the target failed to release its countermeasures,  which prevented the sensors from collecting expected data. Consequently,  MDA was unable to fully develop discrimination algorithms as planned.
In fiscal year 2009, MDA continued to be unable to develop its advanced  algorithms as planned as key tests that were designed to reduce the  maturation risk were affected by targets. For example, the Sensors and  GMD elements were unable to collect data to develop their advanced  algorithms when the target failed to release countermeasures and present  the expected scene complexity during FTG-05. The subsequent delay to  the next intercept test—FTG-06—until January 2010 has also reduced the  data MDA had expected in fiscal year 2009 for the development of  discrimination capability. Additionally, target unavailability caused MDA  to delay a THAAD test—FTT-11—from fiscal year 2009. This test was  designed to provide data for the development of advanced algorithms for  the THAAD radars. The test was conducted in fiscal year 2010 but could  not be completed because the target malfunctioned during deployment.  According to the Director of MDA, the Failure Review Board was  concluding its investigation of the root cause of this failure. The board’s  report was not available during our audit.
Likewise, GMD continues to experience delays demonstrating increased  interceptor performance for homeland defense as the two aforementioned  tests—FTG-05 and FTG-06—were not conducted as planned. As we  testified in February 2009, MDA had expected to conduct seven GMD  interceptor flight tests from the start of fiscal year 2007 through the first  quarter of fiscal year 2009. However, MDA was able to conduct only two,  which, according to the Director of Operational Test and Evaluation, has  limited the complete sets of information necessary for validating ground-  based interceptor models. MDA also delayed the other planned flight test,  FTG-06, beyond fiscal year 2009 because of target issues and an anomaly  with a component of the Sea-Based X-band radar. As of June 2009, MDA  estimated this test to cost over $236 million while the Defense Contract  Management Agency estimated the cost to exceed $310 million. These  costs are likely understated because they do not include all of the cost  increases of delaying the test first to September 2009, nor do they include  any cost increases of further delaying the test until the second quarter of  fiscal year 2010.
Although the Aegis BMD missile—SM-3 Block IA—capability against an  intermediate range ballistic missile is not a requirement, MDA has planned  for years and invested millions of dollars in a plan to test the Aegis BMD  system and SM-3 Block IA interceptor against this type of threat. At the  start of fiscal year 2009, Aegis BMD officials intended to conduct this test  in the third quarter of fiscal year 2009. However, as of December 2009,  MDA had canceled the test and planned to combine several objectives  with those in a future flight test in 2013. As of February 2010, the Director  of MDA stated that the test is being rescheduled for 2011. Test  documentation was not provided for our review, so it remains unclear  whether the test will include the original test objectives, target, and BMDS  hardware and software configurations.
section_title: New Test Plan Not Yet Complete, Not Fully Tied to Resources, and Not Synchronized with Other Management Baselines
paragraphs: MDA’s new July 2009 test plan was intended to provide stability; however,  program officials already anticipate major revisions and alterations.  According to MDA officials, budget decisions and the presidential decision  to implement a European phased, adaptive approach, drove changes to the  test and targets program. For example, the new strategy for European  missile defense will primarily utilize Aegis BMD interceptors as opposed to  GMD interceptors. Tests in support of developing this capability have not  yet been added to the test plan. The Director of MDA stated that his  agency is coordinating with the Office of the Director, Operational Test  and Evaluation and with the BMDS Operational Test Agency to address  these changes. According to the Director of MDA, flight and ground testing  to support phases one through four of the Phased Adaptive Approach will  be baselined in the March 2010 Integrated Master Test Plan, but the test  plan was not available for our review during our audit.
One way MDA’s new testing approach was intended to provide stability is  that it was structured to slow the spiral development fielding process,  allowing the warfighter to gain confidence in the BMDS before fielding  decisions are made. However, BMDS Operational Test Agency officials  told us that changes to hardware and software configurations need to  follow the process jointly agreed to with MDA, noting that changes to the  operational baseline should not occur until the appropriate developmental  tests and operational tests have been completed. After the adoption of the  new test plan through October 2009, MDA continued to incorporate  software changes as updates to the operational baseline. According to  Operational Test Agency officials, most of the proposed and approved  software changes had not been through system-level testing and  immediately made future test configurations in the Integrated Master Test  Plan invalid. Changes made without full system-level testing, could result  in possible adverse effects to the BMDS and the warfighter’s ability to use  the system effectively. The BMDS Operational test Agency continues to  work with MDA on these issues. BMDS Operational Test Officials told us  that they have seen improvements since October 2009, noting that there  has been an increase in early coordination and presentation of data to  support interim releases of software and hardware. According to these  officials, these improvements coupled with the new warfighter and MDA-  accepted approach for testing—allowing developmental testing to occur  before operational testing and before new capabilities are delivered to the  Warfighter—will likely resolve issues encountered with frequent changes  to software and hardware.
We testified in February 2009 that the success of MDA’s new approach to  testing hinges on providing sufficient resources, among other factors.  However, these resource challenges continue to affect the test plan  because MDA’s new test plan was not fully resourced when it was  approved in July 2009. In addition, BMDS Operational Test Agency  officials also raised concerns that the Integrated Master Test Plan is not  currently resourced to support the necessary personnel to analyze the  tests or the performance assessment.
Until the new development efforts are fully reflected in the test plan, MDA  will also not be able to fully integrate that plan with other key aspects of  testing and development, such as the acquisition of targets. The test plan is  one of six management baselines MDA uses to track program progress.  However, MDA determined that these baselines consist of a disparate set  of non-integrated business processes. More importantly, MDA  acknowledged that there is inconsistent management, configuration  control, integration, and synchronization of existing manual processes.  MDA is developing new business tools to automate the integration of these  baselines and projects. While it will take several years for the agency to  integrate these baselines using those tools and synchronize them with  other key testing and development efforts, the initial capability to  automatically integrate cost, schedule, and performance baselines will be  available in early fiscal year 2011.
section_title: Overall BMDS Performance Cannot Be Assessed Because MDA Models Are Not Sufficiently Mature
paragraphs: MDA models and simulations have not matured sufficiently to assess  overall BMDS performance and may not fully mature until 2016, instead of  2011as we reported last year. According to the BMDS Operational Test  Agency, it could not project which models and simulations could be  accredited for Performance Assessment 2009. It expects to make its  determination in July 2010 at the earliest. Further, functionality shortfalls  diminished the usable scope and integration issues have delayed the  execution of Performance Assessment 2009 by at least 6 months. As a  result, the BMDS Operational Test Agency did not use the Performance  Assessment 2009 data in its 2009 annual operational assessment as it had  once intended. According to these officials, because of the known  limitations and the changes to the BMDS operational configuration that  will occur in 2010, the BMDS Operational Test Agency also will not be able  to use the results as part of its 2010 annual operational assessment.
MDA officials acknowledged that their primary challenge for the next  several years will be obtaining enough flight test data to anchor and  accredit the models. Moreover, the BMDS Operational Test Agency is still  concerned about the effect on the validation of models due to artificialities  in flight tests, particularly for GMD. The BMDS Operational Test Agency  believes that the validation of models will improve as artificialities in flight  tests are reduced. Another unresolved modeling and simulation weakness  in the testing program has been addressing different weather conditions.  MDA, in concert with the BMDS Operational Test Agency, is addressing  modeling deficiencies with respect to weather conditions, but specific  plans to resolve this weakness were not available during our audit. Finally,  the BMDS Operational Test Agency anticipates that deficiencies in  modeling the BMDS communications system at the regional and theater  levels that exist in Performance Assessment 2009 will improve in the  subsequent Performance Assessment 2010.
section_title: Progress in Delivering Integrated Capabilities Could Not Be Assessed
paragraphs: In 2008, we assessed MDA’s capability delivery progress against its  integrated capability schedule goals and found that many slipped to 2009.  We are no longer able to assess MDA’s progress in delivering integrated  capabilities because, in fiscal year 2009, the agency eliminated integrated  capability delivery goals from its schedule baseline. In its most recent  BMDS Accountability Report, MDA redefined its schedule baseline to  consist solely of hardware and software deliveries spread across fiscal  years. MDA assigned schedule metrics to asset deliveries on an element  level only and removed key schedule measures—engagement sequence  groups—from its August 2009 BMDS Accountability Report that tracked  integrated block capability deliveries and provided a means for assessing  the readiness of BMDS capabilities, integration, and functionality. Thus,  MDA provided no information about its progress and plans to deliver  integrated BMDS capabilities.
MDA previously identified its capability delivery schedule goals and  baselines within the block structure, in terms of assets and engagement  sequence groups made available for fielding in a particular timeframe.  Under this capabilities-based five-block acquisition management strategy,  some blocks contained schedule baselines for deliveries of significant  increments of capabilities against particular threats, culminating in the full  capability declaration at a projected date. According to MDA, engagement  sequence groups created manageable combinations of system  configurations and provided a structure to assess BMDS performance.  Because MDA presented early, partial and full capability delivery dates for  individual engagement sequence groups, engagement sequence groups  served as baseline to measure the schedule of integrated capability  deliveries.
MDA officials told us that the agency eliminated engagement sequence  groups as measures of integrated capability deliveries to address  warfighter concerns. According to MDA officials, the warfighter did not  assess engagement sequence groups since they were organized in a way  that did not align with warfighter operations, tactics, and procedures.  During our audit, MDA had not replaced these previously reported  integrated capability delivery baselines with new metrics. However,  according to the Director of MDA, the agency is working to develop new  baselines and schedules from which progress can be measured. In  addition, agency officials told us that MDA is transitioning to an  incremental BMDS capability delivery concept. However, MDA did not  provide a definition of incremental BMDS capability deliveries or define  them as schedule goals in the August 2009 BMDS Accountability Report.  MDA also did not identify anticipated delivery dates for its performance  metrics; however, the Director of MDA stated that developmental  baselines are anticipated to be developed, reviewed and approved by the  third quarter of fiscal year 2010. Furthermore, major MDA documents  designed to communicate MDA’s BMDS schedule are not synchronized.  Although MDA officials told us that they have recently synchronized the  Integrated Master Schedule with the Integrated Master Test Plan, the two  documents’ schedule still does not correspond to the BMDS Master Plan.  The Integrated Master Test Plan will be revised in February 2010,  rendering all three documents again unsynchronized with MDA’s  acquisition strategy and programmatic decisions.
While it has eliminated its externally reported integrated capability  declaration goals, MDA continues to internally track capability  declarations for at least two of its assets—the Sea-based X-band radar and  the Shariki AN/TPY-2 radar—whose capability declarations slipped again  in fiscal year 2009. The Sea-based X-band radar partial capability  declaration appears to have slipped from fiscal year 2009 to fiscal year  2010, while full capability will be declared with less knowledge than  initially planned. According to MDA officials, the agency was planning for  a partial capability declaration in June 2009, following successful  execution of four test events—GTI-03, FTX-03, FTG-05, and GTD-03—and  analysis. However, these events slipped over the course of the year, and  according to MDA, the partial capability declaration was delayed to fiscal  year 2010. According to the Director of MDA, the capability declaration is  currently planned to occur after analysis can include both FTG-06 and a  test—CD-03—planned for September 2010. It remains unclear what effect  the problems encountered in FTG-06 will have on the declaration decision.
The Shariki radar was designated by MDA to reach a full capability  declaration by December 2008, but that was subsequently delayed to July  2009. The radar was to undergo the military mission capability assessment,  in which the warfighter verifies the radar’s readiness for full operational  use by the services in the context of the present BMDS architecture. To  date, the full capability declaration has not been made. Consequently, the  date for the full mission capability has not been determined. Furthermore,  as with the Sea-based X-band radar, the decision has not been made as to  whether the Shariki radar capability declaration process will continue  under the original plan or migrate to the new approach.
section_title: BMDS Assets Still Being Delivered Despite Developmental Problems and Test Delays
paragraphs: Despite testing delays, developmental problems, and the continued  inability to complete the Director’s test-related knowledge points, MDA  proceeded with manufacturing, production, and fielding of BMDS assets  prior to operational testing and evaluation.
The Aegis BMD program intends to execute a contract modification in the  second quarter of 2010 to acquire 18 operationally configured SM-3 Block  IB missiles, used for testing and fielding. These 18 SM-3 Block IB missiles  were originally justified in the fiscal year 2010 budget request as needed  for flight testing and for delivery to the fleet as operational assets.  According to MDA’s September 2009 SM-3 Block IB utilization plan, 2  missiles are to be used for flight tests, 10 are to be used for fleet  deployment, and 6 are to be used for either fleet proficiency training or  fleet deployment. However, MDA is proceeding with the contract  modification even though flight testing of a fully integrated prototype for  this missile type in an operational environment will not have occurred.  The first flight test—FTM-16—that could demonstrate some performance  of the missile is currently scheduled for the third quarter of fiscal year  2011. In addition, the program is still maturing several critical  technologies, such as the throttleable divert and attitude control system,  and developmental testing of these technologies will not be complete until  after the manufacturing decision for these 18 missiles. The manufacturing  decision is also scheduled to occur almost a year before the manufacturing  readiness review—currently scheduled for the second quarter of fiscal  year 2011. Consequently, approval for production of this missile is  scheduled before the results of developmental testing to demonstrate that  the technologies and design are fully mature, before the first flight test  demonstrates the system functions as intended, and before the readiness  to begin manufacturing has been assessed—all of which increases the risk  of costly design changes while production is underway. The Director of  MDA and the Assistant Secretary of the Navy for Research, Development  and Acquisition approved a developmental baseline in January 2010 that  set production criteria and projected an initial production decision for 74  SM-3 Block IB missiles in the third quarter of fiscal year 2011.
GMD continues to manufacture and field the Capability Enhancement II  exoatmospheric kill vehicle prior to having it verified through  operationally realistic flight testing. In March 2009, we reported that MDA  had planned to conduct an intercept test to assess Capability  Enhancement II exoatmospheric kill vehicle in the first quarter of fiscal  year 2008—months before emplacing any interceptors with this  configuration. However, developmental problems with the new  configuration’s inertial measurement unit and problems with the target  delayed the first flight test with the Capability Enhancement II  configuration—FTG-06—until the fourth quarter of fiscal year 2009. This  test was again delayed because of modeling uncertainties with the target  and failures experienced with the Sea-Based X-Band radar during testing.  GMD officials stated that they do not plan to adjust deliveries of the  Capability Enhancement II exoatmospheric kill vehicle because of the test  delay. However, MDA officials told us that they will not add Capability  Enhancement II to the operational baseline until after FTG-06 has been  conducted. As previously noted, FTG-06 was conducted in January 2010  but was unsuccessful. According to the July 2009 revised Integrated  Master Test Plan, the next planned intercept test with a similar  configuration as FTG-06—a three-stage booster and a Capability  Enhancement II exoatmospheric kill vehicle—is not scheduled to take  place until at least fourth quarter fiscal year 2012. If MDA delivers  Capability Enhancement II exoatmospheric kill vehicle units as currently  scheduled, it will have delivered all of the Capability Enhancement II  exoatmospheric kill vehicles that are currently under contract before the  test is conducted.
MDA’s concurrent approach to developing and fielding assets has led to  concerns about the performance of some fielded assets. In March 2009, we  reported that MDA had initiated a refurbishment program in 2007 to  replace questionable parts and that some improvements had already been  introduced into the manufacturing flow. However, according to program  officials, they discovered additional problems during early refurbishments  causing the program to expand its effort. Additionally, as MDA continues  to manufacture ground based interceptors, it is discovering additional  process and design issues, and the corrective actions are being  incorporated into the refurbishment program. The program has three  categories for refurbishments—minimal, moderate, and extensive—with  the cost of each vehicle going through refurbishment varying from vehicle  to vehicle. MDA originally estimated that the cost for extensive  refurbishment of an individual interceptor could reach as high as $24  million.
section_title: MDA Lacks Controls and Mechanisms Needed to Establish Transparency, Accountability, and Oversight
paragraphs: MDA continues to face challenges with transparency, accountability, and  oversight controls and mechanisms. In establishing MDA in 2002, the  Secretary of Defense directed the agency to develop the BMDS as a single  program using a capabilities-based, spiral upgrade approach to quickly  deliver a set of integrated defensive capabilities. To accomplish this  mission, MDA was granted exceptional flexibility in setting requirements  and managing the acquisition. This flexibility allowed MDA to begin  delivering an initial defensive capability in 2004, but at the expense of  transparency and accountability.
Since our first MDA report in 2004, we have repeatedly found that MDA’s  approach for building its cost, schedule, and performance goals hindered  transparency and limited accountability of the BMDS development effort.  Specifically in April 2004, we reported that MDA’s goals did not provide a  reliable and complete baseline for accountability purposes and decision  making because these goals varied year to year, did not include all  associated costs, and were based on assumptions about performance that  were not explicitly stated. These conclusions still hold true for several  aspects of the BMDS acquisition strategy. For example, MDA’s goals  change continuously, cost baselines have yet to be established, and some  details regarding performance goals are still not explicitly stated. Since  2004, we have also made recommendations to develop baselines and  report variances to those baselines to promote a higher level of  transparency and accountability for the agency; to adjust its block strategy  to ensure that it was knowledge-based and aligned with agency goals; and  to strengthen oversight by, for example, having the Missile Defense  Executive Board (MDEB) consider the extent to which MDA could adapt  and adopt aspects of DOD’s standard acquisition policies to enhance  oversight.
Members of Congress have also expressed concerns regarding the block  strategy, acquisition management strategy, accountability, and oversight of  MDA. For example, in 2007, the House Appropriations Committee directed  MDA to “develop a system-wide plan to report according to the spirit of  existing acquisition laws to improve accountability and transparency of its  program.” More recently, in the National Defense Authorization Act for  Fiscal Year 2008, Congress required MDA to establish acquisition cost,  schedule, and performance baselines for each system element that has  entered the equivalent of the systems development and demonstration  phase of acquisition or is being produced or acquired for operational  fielding. MDA is not yet fully compliant with this requirement. However,  officials indicated that they are working toward fulfilling this requirement,  but the expected date for full compliance was unknown at the time of our  audit.
While MDA has committed to take actions to address concerns about  accountability and transparency, it has made limited progress in  implementation, as shown in table 6.
MDA’s termination of its capabilities-based block approach in June 2009  marked the third acquisition management strategy for the BMDS in the last  3 years and effectively reduced transparency and accountability for the  agency. As previously noted, MDA has organized the development of the  BMDS using two different block approaches in the past—(1) sequential 2- year blocks of BMDS-wide integrated capabilities and (2) five capabilities- based blocks of different MDA elements against particular threats.  Changing the block structure is problematic because each time the block  structure is changed, the connection is obscured between the old block  structure’s scope and resources and the new block structure’s rearranged  scope and resources. This makes it difficult for decision makers to hold  MDA accountable for expected outcomes and clouds transparency of the  agency’s efforts.
In March 2008, we reported that the agency’s capabilities-based block  approach had begun to provide improvements to transparency and  accountability, but as we recommended, transparency and accountability  could have been further improved with MDA’s development and reporting  of full acquisition cost estimates as well as independent verification of  those costs.
subsections: section_title: New MDA Initiatives Provide Opportunity to Increase Transparency and Accountability
paragraphs: Although key controls and mechanisms needed to establish a sound  acquisition process for MDA are still lacking, MDA has initiatives  underway that could improve the transparency, accountability, and  oversight of the acquisition of the BMDS. In June 2009, the MDA Director  testified before the Senate Armed Services Committee that MDA is  responding to the Weapon System Acquisition Reform Act of 2009  through the establishment of acquisition milestone decisions. These  decisions are designed to ensure appropriate competitive acquisition  strategies. He further noted that as the Acquisition Executive for the initial  phases of missile defense, he is implementing milestone review and  baseline reporting processes that are closely aligned with the principles of  DOD’s acquisition policies, commonly referred to as the DOD 5000 series.  He further noted that he recognized the need to incorporate the tenets of  the DOD 5000 series to ensure that programs are affordable, are justified  by the warfighter, and demonstrate acceptable risk through a milestone  review process overseen by the MDEB. He also stated that MDA intends to  separate the management of its technology and development programs.  The Director testified that under his authority, potential programs that  may provide technological or material solutions for MDA will undergo a  Milestone “A” decision to determine if they should become programs.  These technology-based programs will be managed by knowledge points  and incubated until maturity, at which time MDA along with the service  acquisition executive will be able to make a Milestone “B” decision as to  whether the program should be converted to a development program. He  explained that the Under Secretary of Defense for Acquisition, Technology  and Logistics will make Milestone “C” production decisions regarding the  programs.
We were only able to obtain limited insight into these initiatives because  the agency only determined how they will be implemented at the end of  our audit and was just beginning to implement them. In regards to the  milestone decisions, the Director of MDA indicated that the agency is  undertaking a baseline phase review process. The agency is transitioning  to managing the six developmental baselines at the project element level.  These baselines will be approved in developmental baseline reviews and  managed through quarterly performance element reviews. MDA has  identified three phases of development where baselines are approved— technology development, product development, and initial production  phases—which may ensure that the appropriate level of knowledge is  obtained before acquisitions move from one phase to the next. Approval of  the product development and initial production baselines will be jointly  reviewed by the Director of MDA and the respective service acquisition  executive. In addition, while our draft was being reviewed by MDA, the  Director of MDA provided us with initial information regarding the  definition of these new phases and the process for establishing cost,  schedule, or performance baselines. Based on our initial briefing on MDA’s  new process, it may include many of the necessary elements of a sound  business case—such as establishing top-level cost, schedule, and  performance measures that are available internally and externally for  oversight.
Although we were unable to fully evaluate MDA’s new initiatives, these  initiatives do offer an opportunity for the agency to increase transparency  and accountability if they are implemented in accordance with knowledge- based acquisition principles, leading to the establishment of sound  business cases and realistic baselines.
Over the past 10 years, we have conducted extensive research on  successful programs and have found that successful defense programs  ensure that their acquisitions begin with realistic plans and baselines prior  to the start of development. We have previously reported that the key  cause of poor weapon system outcomes, at the program level, is the  consistent lack of disciplined analysis that would provide an  understanding of what it would take to field a weapon system before  system development begins. We have reported that there is a clear set of  prerequisites that must be met by each program’s acquisition strategy to  realize successful outcomes. These prerequisites include the following:    Establishing a clear, knowledge-based, executable business case for  the product. An executable business case is one that provides  demonstrated evidence that (1) the identified needs are real and  necessary and can best be met with the chosen concept and (2) the  chosen concept can be developed and produced within existing  resources—including technologies, funding, time, and management  capacity. Knowledge-based acquisition principles and business cases  combined are necessary to establish realistic cost, schedule and  performance baselines. Without documented realistic baselines there  is no foundation to accurately measure program progress.
Separating technology development activities from product  development activities. As noted earlier, the Director of MDA plans to  separate technology development and product development for the  BMDS. The process of developing technology culminates in  discovery—the gathering of knowledge—and must, by its nature, allow  room for unexpected results and delays. When immature technologies  are brought onto the critical path of product development programs  too early, they often cause long delays in an environment where large  workforces must be employed; complex tools, plants, and facilities  must be operated; long and expensive supplier networks must be paid;  and the product itself must sometimes be redesigned once the final  form of the technologies is known. Ensuring that only mature  technologies are brought into product development is a key step for  successful programs.
Employing early systems engineering discipline in order to develop  realistic cost and schedule estimates prior to development start.  Early systems engineering provides the knowledge a product  developer needs to identify and resolve performance and resource  gaps before product development begins, either by reducing  requirements, deferring them to the future, or increasing the estimated  cost for the weapon system’s development. Requirements that are too  risky given the state of technology and design should not be allowed  into this expensive environment. MDA’s Director noted that he has  taken steps to enhance systems engineering by designating a senior  executive position to establish engineering policy, ensure the  disciplined practice of systems engineering fundamentals, and develop  the systems engineering competencies of the missile defense  workforce; creating knowledge centers; and increasing the number of  recent engineering school graduates. While these initiatives hold  promise for the future, they could provide further enhancements if  they are used as the foundation to develop realistic cost and schedule  estimates for the BMDS.
These practices could address MDA’s past problems of initiating programs  and beginning system development based on limited systems engineering  knowledge. These programs depended on critical technologies that were  immature and not ready for product development or production. The  Director of MDA acknowledged the importance of changing MDA’s  acquisition approach to adopt knowledge-based acquisition processes.
section_title: Conclusions
paragraphs: In order to respond to a presidential directive to deliver a missile defense  capability in a rapid manner, MDA has been given unprecedented funding  and decision-making flexibility. This flexibility has allowed concurrent  development, testing, manufacturing and fielding and enabled MDA to  quickly develop and field the first increment of capability in 2005.  However, while this approach has expedited the fielding of assets, it also  resulted in less transparency and accountability than is normally present  in a major weapon program. Since the program’s inception, MDA’s lack of  baselines and its management of the BMDS with high levels of uncertainty  about requirements and program cost estimates effectively set the missile  defense program on a path to an undefined destination at an unknown  cost. Across the agency, these practices left programs with limited  knowledge and few opportunities for crucial management oversight and  decision making concerning the agency’s investment and the warfighter’s  continuing needs. At the program level, these practices contributed to  quality problems affecting targets acquisitions, which in turn, hampered  MDA’s ability to conduct tests as planned.
As MDA transitions to new leadership, a new acquisition strategy, a new  test strategy, and a shift in emphasis toward early intercept capabilities,  the agency has an opportunity to chart a course that enables transparency  and accountability as well as flexibility, and it appears committed to doing  so. Importantly, the Director of MDA has begun new initiatives in  accordance with guiding principles of DOD’s acquisition policies, which  already embrace knowledge-based practices and sound management  controls. The Director of MDA intends to apply these new policies to each  element or appropriate portions of the elements, as is currently done  across DOD, in order to provide a foundation for the Congress and others  to assess progress and hold senior leadership accountable for outcomes.
These initial steps are promising, but it will take time to fully implement  them and once implemented they will need to be sustained and the tools  consistently used in order to establish accountability. If this is done  effectively, with baselines set at a program level, MDA can respond to  strategic changes affecting the overall configuration of the system without  losing basic knowledge about cost, schedule, and performance. Such  actions do not have to result in a slower or more burdensome acquisition  process. In the past, weapon programs often rushed into systems  development before they were ready, in part because DOD’s acquisition  process did not require early formal milestone reviews and programs  would rarely be terminated once underway. Over time, in fact, these  changes could help programs replace risk with knowledge, thereby  increasing the chances of developing weapon systems within cost and  schedule targets while meeting user needs.
section_title: Recommendations for Executive Action
paragraphs: As MDA implements its initiatives to improve transparency, accountability,  and oversight, and begins efforts to manage and oversee MDA at the  element level, we recommend that the Secretary of Defense direct MDA to  take the following eight actions:    Establish cost, schedule, and performance baselines for the acquisition  of each new class of target when it is approved by the Director prior to  proceeding with acquisition and report those baselines to Congress.
Obtain independent Cost Assessment and Program Evaluation cost  estimates in support of these cost baselines.
Ensure that program acquisition unit costs for BMDS assets are  reported in the BMDS Accountability Report, to provide Congress with  more complete and comprehensive information by including  development costs.
Update DOD’s Plan to Enhance the Accountability and Transparency  of the Ballistic Missile Defense Program to reflect MDA’s current  initiatives and include dates for fulfilling each commitment.
Report top-level test goals for each element, or appropriate portions  thereof, to Congress in the next BMDS Accountability Report.
Develop and report to Congress in the annual BMDS Accountability  Report a measure for schedule baseline goals that incorporates  delivering integrated capabilities to the warfighter.
Develop and report to Congress in the annual BMDS Accountability  Report the dates at which performance baselines will be achieved.
Report to Congress variances against all established baselines.
Several of these actions, such as establishing cost, schedule, and  performance baselines, have been recommended in prior GAO reports or  addressed in legislation. This report, however, restates these  recommendations in the context of changes made to the missile defense  program, for example, the deletion of the block structure and increased  focus on elements.
We further recommend that the Secretary of Defense direct MDA to take  the following two actions:    Delay the manufacturing decision for SM-3 Block IB missiles intended  for delivery to the fleet as operational assets until after (1) the critical  technologies have completed developmental testing, (2) a successful  first flight test demonstrates that the system functions as intended, and  (3) the successful conclusion of the manufacturing readiness review.
Ensure that developmental hardware and software changes are not  made to the operational baseline that disrupt the assessments needed  to understand the capabilities and limitations of new BMDS  developments.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report. These  comments are reprinted in appendix I. DOD also provided technical  comments, which were incorporated as appropriate.
DOD fully concurred with 9 of our 10 recommendations, including our  recommendation to establish cost, schedule, and performance baselines  for the acquisition of each new class of target when it is approved by the  MDA Director prior to proceeding with acquisition and report those  baselines to Congress. In response to our recommendation, DOD  commented that MDA has already established and the Director has  approved cost, schedule, and performance baselines for the acquisition of  each new class of target.  The department noted that these baselines are  contained in multiple documents and will be brought together in a Target  Program Baseline prior to contract award. However, MDA should ensure  that the Target Program Baseline establishes top-level cost, schedule, and  performance baseline measures similar to approved program baselines  that are established for DOD’s major defense acquisitions and available for  internal and external oversight. It is unclear whether MDA will make its  Target Program Baseline available internally for oversight and report it to  Congress as we recommended.
DOD partially concurred with our recommendation that the Secretary of  Defense direct MDA to delay the manufacturing decision for SM-3 Block  IB missiles intended for delivery to the fleet as operational assets until  after (1) the critical technologies have completed developmental testing,  (2) a successful first flight test demonstrates that the system functions as  intended, and (3) the successful conclusion of the manufacturing  readiness review. In response to this recommendation, DOD stated that  manufacturing of SM-3 Block IB missiles to support testing is under way,  but the production decision for SM-3 Block IB missiles used for fleet  operation is planned to occur after criteria listed in our recommendation  have been met. However, during our review, we found that the 18 SM-3  Block IB missiles in question were originally justified in the fiscal year  2010 budget request as needed for “flight testing and for delivery to the  fleet as operational assets.” In addition, Aegis BMD Program Office  responses related to this matter indicate that these missiles will be used  operationally if a security situation requires it. Furthermore, according to  MDA’s September 2009 SM-3 Block IB utilization plan briefed and  approved by the MDA Acquisition Strategy Board, only 2 of these missiles  are specifically designated for flight tests, while 10 are to be used for fleet  deployment and 6 are to be used for either fleet proficiency training or  fleet deployment. Based on this information, the contract modification to  acquire these 18 SM-3 Block IB missiles will take place before the critical  technologies are fully matured at the conclusion of FTM-16—the first SM-3  Block IB end-to-end flight test of a fully integrated, production- representative prototype. Thus, we maintain that approval for  manufacturing of these 18 SM-3 Block IB missiles—the majority of which  will be deployed to the fleet—is scheduled to occur before the results of  developmental testing to demonstrate that the technologies and design are  fully mature, before the first flight test demonstrates the system functions  as intended, and before the readiness to begin manufacturing has been  assessed—all of which increase the risk of costly design changes and  retrofit.
We are sending copies of this report to the Secretary of Defense and to the  Director of MDA. The report also is available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. GAO staff who made major contributions to this  report are listed in appendix III.
section_title: Appendix I: Comments from the Department of Defense
section_title: Appendix II: Scope and Methodology
paragraphs: To examine the progress that eight Missile Defense Agency (MDA)  elements have made during fiscal year 2009 toward schedule, testing, and  performance baselines, we developed data collection instruments that  were completed by each element’s program office. These instruments  collected detailed information on prime contracts, design reviews, test  schedules and results, element performance, noteworthy progress, lessons  learned, and challenges facing the elements during the fiscal year. In  addition, we reviewed individual element Program Execution Reviews,  test plans and reports, production plans, staffer day briefings, and other  requirements documents. We held interviews with officials in each  element’s program office and followed up on the information we received  with MDA’s Agency Operations Office; the Department of Defense’s (DOD)  Office of the Director, Operational Test and Evaluation; and MDA’s  Ballistic Missile Defense System (BMDS) Operational Test Agency.
To further review individual element and BMDS-level performance  progress during the fiscal year, we met with officials in MDA’s Modeling  and Simulation Directorate at the Missile Defense Integration and  Operations Center, individual element program offices, and MDA’s BMDS  Operational Test Agency to discuss modeling and simulations plans and  procedures as well as other performance metrics. We also reviewed DOD  and MDA policies, memos, and flight test plans related to modeling and  simulations. In addition, we reviewed various elements’ verification,  validation, and accreditation plans, MDA performance briefings, and  validation, verification, and accreditation plans for MDA’s BMDS  Performance Assessment 2009.
We assessed MDA’s testing and target development progress by reviewing  MDA’s Integrated Master Test Plans, Integrated Master Schedule, target  acquisition plan, and target business case analysis. In addition, we met  with officials in the Targets and Countermeasures Program Office to  obtain information on MDA’s acquisition management strategy including  plans for cost, schedule, and testing. We also met with MDA’s testing  directorate, MDA’s BMDS Operational Test Agency, and DOD’s Office of  the Director of Test and Evaluation to discuss the progress, challenges,  and lessons learned during fiscal year 2009 testing.
To analyze MDA’s changing acquisition approach and the agency’s  progress in addressing issues related to transparency, accountability, and  oversight, we interviewed officials from the Office of the Under Secretary  of Defense for Acquisition, Technology and Logistics; MDA’s Agency  Operations Directorate; MDA’s Advanced Technology Directorate; and  MDA’s Office of Quality, Safety, and Mission Assurance Directorate. We  also reviewed various MDA statements and documents related to MDA’s  block structure. We reviewed DOD acquisition system policy and various  DOD directives to gain insight into other DOD systems’ accountability and  oversight mechanisms. We also analyzed MDA’s acquisition directives and  Missile Defense Executive Board briefings to examine MDA’s current level  of oversight. In addition, we reviewed MDA budget estimate submission  justifications, Integrated Master Test Plans, the Ballistic Missile Defense  Master Plan, the BMDS Accountability Report, and prior reports that  outlined the agency’s baselines and goals.
Our work was performed both at MDA headquarters in Arlington, Virginia  and at various program offices located in Huntsville, Alabama. In Arlington  we met with officials from the Aegis Ballistic Missile Defense Program  Office; Airborne Laser Program Office; Command, Control, Battle  Management, and Communications (C2BMC) Program Office; MDA’s  Agency Operations Office; DOD’s Office of the Director, Operational Test  and Evaluation; and the Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics. In Huntsville, Alabama we  interviewed officials from the Ground-based Midcourse Defense (GMD)  Program Office, the Sensors Program Office, the Terminal High Altitude  Area Defense Project Office, the Targets and Countermeasures Program  Office, the Advanced Technology Directorate, and the Office of the  Director for BMDS Tests. We met with officials from the Missile Defense  Integration and Operations Center at Schriever Air Force Base in Colorado  Springs, Colorado, to discuss the C2BMC and Space Tracking and  Surveillance System elements as well as to receive further information on  MDA’s models and simulations. Additionally, we interviewed Raytheon  officials in Tucson, Arizona, to discuss the Kinetic Energy Interceptor,  GMD, and Aegis BMD elements’ status.
In December 2007, the conference report accompanying the National  Defense Authorization Act for Fiscal Year 2008 noted the importance of  DOD and MDA providing information to GAO in a timely and responsive  manner to facilitate the review of ballistic missile defense programs.  During the course this audit, we experienced significant delays in  obtaining information from MDA. During the audit, MDA did not always  provide GAO staff with expeditious access to requested documents and  articles of information, which delayed some audit analysis and contributed  to extra staff hours. Of the documents and information we requested, we  received approximately 24 percent within the 10 to15 business day  protocols that were agreed upon with MDA. Pre-existing documentation  took MDA on average about 28 business days to provide and many pre- existing documents took 40 business days or more to be provided to GAO.  Notwithstanding these delays, we were able to obtain the information  needed to satisfy our objectives in accordance with generally accepted  government auditing standards.
We conducted this performance audit from April 2009 to February 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, David Best, Assistant Director;  LaTonya Miller; Ivy Hübler; Tom Mahalek; Steven Stern; Meredith Allen  Kimmett; Wiktor Niewiadomski; Kenneth E. Patton; Karen Richey; Robert  Swierczek; and Alyssa Weir made key contributions to this report.