Missile Defense: Actions Needed to Improve Transparency and Accountability
Published: 2011-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since 2002, Congress has directed GAO to assess the Missile Defense Agency's (MDA) annual fiscal year cost, schedule, testing, and performance progress in developing the Ballistic Missile Defense System (BMDS). This year's report specifically assesses MDA's progress in (1) delivering missile defense assets as scheduled (2) improving accountability and transparency over the past year (3) implementing the European Phased Adaptive Approach (4) implementing changes to the Ground-based Midcourse Defense program (5) implementing the targets revised acquisition strategy identified in 2009, and (6) testing the BMDS and developing its modeling and simulations to assess performance. To accomplish this, GAO reviewed MDA's progress reports to the Congress, pertinent Department of Defense (DOD) policies and reports including a DOD assessment and plan related to the Ground-based Midcourse Defense system.
section_title: What GAO Found
paragraphs: In 2010, MDA made progress in delivering assets as well as increasing transparency and accountability. While many significant, positive steps were taken, GAO also found issues limiting the extent to which cost, schedule, and system performance can be tracked. Stabilizing the new acquisition approach, improving execution and increasing transparency are key steps for DOD. Asset Delivery: In 2010, MDA was able to meet or exceed its delivery goals for several MDA activities, such as missile defense upgrades to Aegis ships. However, the agency was unable to meet all of its goals for Terminal High Altitude Area Defense, a system used to defend against targets in their last phases of flight. Transparency and Accountability: MDA finalized a new process in which detailed baselines were set for several missile defense systems. As a result of the new process, its 2010 progress report to the Congress is more comprehensive than it was in 2009. Although the information in MDA's progress reports to the Congress increased, GAO found its unit and life-cycle cost baselines had unexplained inconsistencies and documentation for six baselines had insufficient evidence to be a high-quality cost estimate. As a result, GAO could not evaluate cost progress. European Phased Adaptive Approach for Missile Defense: The September 2009 shift in focus for European missile defense represents a significant change in U.S. policy and a substantial investment for DOD. However, DOD has not fully implemented a management process that synchronizes European missile defense acquisition activities and ensures transparency and accountability. Without key management and oversight processes, there is a limited basis for oversight, and there is a risk that key components will start production before demonstrating system performance. In the past, similar deficiencies in missile defense acquisition oversight have led to rework, cost increases, delays, and doubts about delivered capabilities. Ground-based Midcourse Defense (GMD): While the GMD system---which is primarily designed to engage longer-range targets in the midcourse range of flight--has demonstrated a limited capability, DOD has not yet determined the system's full capabilities and limitations. In January and December 2010, GMD experienced two flight test failures. In addition, GMD is just beginning to take actions necessary to sustain the capability through 2032. Targets Acquisition, Testing, and Performance: MDA made a targets acquisition decision in 2010 in response to a target failure. This decision was not consistent with its 2009 acquisition plan which envisioned competitive contract awards that would reduce reliance on its prime contractor. The cost of this action remains unknown. Also, as in previous years, failures and delays in testing have continued to delay validation of models and simulations used to assess BMDS performance.
section_title: What GAO Recommends
paragraphs: GAO makes 10 recommendations for MDA to strengthen its resource, schedule and test baselines, facilitate baseline reviews, and further improve transparency and accountability. GAO is also making a recommendation to improve MDA's ability to carry out its test plan. In response, DOD fully concurred with 7 recommendations. It partially concurred with 3, contending that its current actions are sufficient and that the test recommendation is also not affordable. GAO continues to believe that additional action is needed.

section_title: Letter
paragraphs: The Missile Defense Agency’s (MDA) mission is to develop an integrated  and layered Ballistic Missile Defense System (BMDS) to defend the United  States, its deployed forces, allies, and friends. In order to meet this  mission, MDA is developing a highly complex system of systems—land,  sea-, and space-based sensors, interceptors and battle management. Since  its initiation in 2002, MDA has been given a significant amount of flexibility  in executing the development and fielding of the BMDS. To enable MDA to  field and enhance a missile defense system quickly, the Secretary of  Defense in 2002 delayed the entry of the BMDS program into the  Department of Defense’s (DOD) traditional acquisition process until a  mature capability was ready to be handed over to a military service for  production and operation. To meet a presidential directive to deliver an  initial capability by 2004, the program has concurrently developed and  fielded assets. While this approach helped MDA rapidly deploy an initial  capability, it has also meant that MDA has fielded some assets whose  capability is uncertain. Moreover, we have reported that MDA’s approach  has limited transparency and accountability for DOD’s largest single  acquisition program—spending from approximately $7 billion to $9.5  billion per year. More specifically, there has been limited understanding  about baseline costs, schedules, requirements, and system effectiveness.
Last year, we reported that MDA was in the process of transitioning to new  leadership, a new acquisition strategy, a new test strategy, and a shift in  emphasis toward early intercept capabilities. Given the breadth and scope  of these changes, we concluded that the agency had an opportunity to  chart a course that enables transparency and accountability while  retaining its desired flexibility, and it appeared that MDA was committed  to doing so. Importantly, the Director of MDA had begun new initiatives in  accordance with guiding principles of DOD’s acquisition policies, which  already embrace knowledge-based practices and sound management  controls. The Director of MDA intended to apply these new policies to  each element or appropriate portions of the elements, as is currently done  across DOD, in order to provide a better foundation for Congress and  others to assess progress and hold senior leadership accountable for  outcomes.
Since 2002, Congress has directed GAO to assess MDA’s annual fiscal year  cost, schedule, testing, and performance progress in developing the  BMDS. We have delivered assessments of MDA’s progress covering fiscal  years 2003 through 2009 and are currently mandated to continue our  assessments through fiscal year 2013. According to this mandate, we are  required to assess MDA’s fiscal year progress against the annual goals it is  required to report to Congress each February in the BMDS Accountability  Report (BAR). This year’s report specifically assesses MDA’s progress in  (1) delivering missile defense assets as scheduled (2) improving  accountability and transparency over the past year (3) implementing the  European Phased Adaptive Approach (PAA) (4) implementing changes to  the Ground-based Midcourse Defense (GMD) program (5) implementing  the targets revised acquisition strategy identified in 2009, and (6) testing  the BMDS and developing its modeling and simulations to assess  performance.
To assess all six areas of MDA’s progress, we examined the  accomplishments of nine BMDS elements and supporting efforts that MDA  is currently developing and fielding: the Aegis Ballistic Missile Defense  (Aegis BMD); Aegis Ashore; BMDS Sensors; Command, Control, Battle  Management, and Communications (C2BMC); GMD; Precision Tracking  and Surveillance System (PTSS); Space Tracking and Surveillance System  (STSS); Targets and Countermeasures; and Terminal High Altitude Area  Defense (THAAD). We reviewed individual element responses to GAO data  collection instruments, which detailed key accomplishments for fiscal year  2010. The results of these reviews are presented in detail in appendixes to  this report and are also integrated as appropriate in our findings related to  progress in delivering assets and implementing new initiatives. We also  sought to examine MDA’s Baseline Execution Reviews of each element’s  progress, but were unable to assess the supporting backup materials as  they were not made available to us until February 2011—the very end of  the audit.
To assess whether MDA elements delivered assets as scheduled, we  examined the 2009 BAR and compared it to the 2010 version, looking for  similarities and differences between the two. We also reviewed responses  to GAO data collection instruments, which detailed key accomplishments  for fiscal year 2010, including some asset deliveries, as well as any delayed  asset deliveries   To follow up on the progress MDA made to improve transparency and  accountability, we reviewed pertinent DOD policies to compare MDA’s  current level of accountability with that of other DOD programs. We also  assessed the cost, schedule, and test baselines included in the 2010 BAR.  In addition, we held discussions with officials in MDA’s Operations  Directorate to discuss the new phased adaptive approach. Lastly, we met  with officials in MDA’s Acquisition Directorate to discuss how the agency  is establishing and managing against its internal baselines.
For our assessment regarding the implementation of the management  process to synchronize acquisitions for the European PAA, we synthesized  management and oversight principles from the Office of Management and  Budget, DOD, MDA, and GAO best acquisition practices for large  acquisition efforts similar to European PAA. We then compared European  PAA acquisition efforts to these principles. We also reviewed DOD and  MDA documentation related to European PAA. We also requested the  European PAA cost estimate that was completed in fall of 2009. However,  we did not receive the cost estimate until February 2011—the very end of  our audit—and therefore we could not assess it for this report. We met  with MDA and Office of the Secretary of Defense officials. We visited the  U.S. European Command and U.S. Strategic Command. We met with  officials from MDA directorates and element program offices as well as  the Offices of the Under Secretary of Defense for Policy and the Under  Secretary of Defense for Acquisition, Technology and Logistics and visited  contractor facilities.
In addition, we are including the acquisition-related parts of our final  analysis performed in response to a congressional mandate that we review  a DOD assessment and plan related to the GMD system, that included  issues related to acquisition, sustainment and refurbishment. For DOD’s  Assessment and Plan for the GMD program, we reviewed the reports  provided by DOD; analyzed sustainment, refurbishment and test plans and  program schedules; and assessed budget documents and program reviews.  We interviewed officials from MDA, U.S. Strategic Command, U.S.  Northern Command, and the Army.
To assess BMDS testing and target development progress, we reviewed  technical baselines in the BAR, MDA’s Integrated Master Test Plans, the  target business case analysis, target contracts, and other documents  related to target planning and acquisitions. We also interviewed officials  within program offices and within MDA functional directorates, such as  the Directorates for Engineering and Testing. In addition, we discussed the  elements’ test programs and test results with the BMDS Operational Test  Agency and DOD’s Office of the Director, Operational Test and Evaluation.  We held discussions with the BMDS Operational Test Agency to follow up  on BMDS models and simulations. Our scope and methodology for each of  the six objectives is discussed further in appendix XIV.
As we agreed to with your staff, we do not include an analysis of MDA’s  earned value management (EVM) reporting this year. In prior years, in the  absence of full cost baselines for elements, we assessed the EVM progress  of individual contracts. We issued our findings on MDA’s EVM progress  during fiscal year 2009 in July 2010 and found data reliability issues with 2  of the 14 contracts. Because of these issues with the GMD and Targets  and Countermeasures programs’ EVM data, we were unable to report cost  progress for these two contracts which amounted to half of the total  budgeted contract costs for MDA prime contracts we reviewed. We plan to  assess EVM data on MDA prime contracts in the future once MDA has had  time to address our recommendations to improve data reliability. In our  July report, MDA stated that it intended to take a key step in addressing  our recommendations by conducting a major review of the GMD  program’s EVM data by the end of September 2010. Because MDA  provided documentation of its actions at the end of our audit—in February  2011— leaving no time for review, we will assess whether the steps MDA  has taken sufficiently address GMD's data reliability issues in next year's  report.
We conducted this performance audit from March 2010 to March 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: MDA’s BMDS is being designed to counter ballistic missiles of all ranges— short, medium, intermediate, and long. Since ballistic missiles have  different ranges, speeds, sizes, and performance characteristics, MDA is  employing an integrated and layered architecture to provide multiple  opportunities to destroy ballistic missiles before they can reach their  targets. The system’s architecture includes space-based and airborne  sensors as well as ground- and sea-based radars; ground- and sea-based  interceptor missiles; and a command and control, battle management, and  communications system to provide the warfighter with the necessary  communication links to the sensors and interceptor missiles.
A typical engagement scenario to defend against an intercontinental  ballistic missile would occur as follows:   Infrared sensors aboard early-warning satellites detect the hot plume  of a missile launch and alert the command authority of a possible  attack.
Upon receiving the alert, land- or sea-based radars are directed to  track the various objects released from the missile and, if so designed,  to identify the warhead from among spent rocket motors, decoys, and  debris.
When the trajectory of the missile’s warhead has been adequately  established, an interceptor—consisting of a kill vehicle mounted atop a  booster—is launched to engage the threat. The interceptor boosts  itself toward a predicted intercept point and releases the kill vehicle.    The kill vehicle uses its onboard sensors and divert thrusters to  detect, identify, and steer itself into the warhead. With a combined  closing speed of up to 10 kilometers per second (22,000 miles per  hour), the warhead is destroyed above the atmosphere through a “hit  to kill” collision with the kill vehicle.    Some interceptors use sensors to steer themselves into the inbound  ballistic missile. Inside the atmosphere, these systems kill the ballistic  missile using a range of mechanisms such as direct collision between  the missile and the inbound ballistic missile or killing it with the  combined effects of a blast fragmentation warhead (heat, pressure,  and grains/shrapnel) in cases where a direct hit does not occur.
Table 1 provides a brief description of nine BMDS elements and  supporting efforts currently under development by MDA.
subsections: section_title: Acquisition Strategy for Missile Defense
paragraphs: MDA has employed varied strategies to acquire and deploy missile defense  systems. From its inception in 2002 through 2007, MDA developed missile  defense capability in biennial increments, known as blocks. These 2-year  blocks were each built on preceding blocks and enhanced the  development and capability of the BMDS. However, there was little  visibility into baseline costs and schedules associated with systems that  comprised the blocks or how the blocks addressed particular threats. In  response to recommendations from GAO, in December 2007, MDA  announced a new block structure that was intended to improve the  program’s transparency, accountability, and oversight. The new blocks  were not based on biennial time periods, but instead focused on fielding  capabilities that address particular threats. Because the new block  structure was not aligned to regular time periods, multiple blocks were  under way concurrently. The capabilities-based five-block approach  included several positive changes, including the commitment by DOD to  establish total acquisition costs and unit costs for selected block assets,  including in a block only those elements or components of elements that  would be fielded during the block and abandoning the practice of  deferring work from one block to another.
MDA was still transitioning to this new capabilities-based block approach  when the MDA Director terminated it in June 2009—a year and a half after  it was created. According to MDA, the agency terminated the capability- based block structure to address congressional concerns. The agency then  began to manage the BMDS as a single integrated program but planned to  report on cost, schedule, and performance issues by each element within  the program. At the time of our 2010 report, MDA was in the process of  determining how it would implement changes to its acquisition  management strategy.
section_title: Shift in Approach to European Missile Defense
paragraphs: Another significant change that took place in 2009 was the new  administration’s shift in its approach to European missile defense. DOD  altered its approach to European defense, which originally focused on  ground-based interceptors (GBI) from the GMD element and a large fixed  radar as well as transportable X-Band radars, in order to provide defenses  against long-range threats to the United States and short-, medium-, and  intermediate-range Iranian threats to Europe. The new European PAA  consists primarily of Aegis BMD sea-based and land-based systems and  interceptors, as well as various sensors to be deployed over time as the  various capabilities are matured. According to DOD, this new approach  offers a number of improvements over the previous architecture, such as  providing missile defenses sooner with greater flexibility to meet evolving  threats, providing more opportunities to involve close allies, and delivering  greater capability to defend against a large number of threat missiles. In  essence, the new policy marks a fundamental change from an emphasis on  a global integrated system to a policy that bundles capability and tailors it  for different regions—the PAA—with the European PAA being the first of  these regional approaches. According to DOD, it intends to use the  department’s existing processes for managing missile defense acquisitions  and the existing BMDS element-based acquisition approach for missile  defense system elements—not one specific to each PAA—to approve  system acquisitions for each PAA.
The European PAA policy announced by the President articulates a  schedule for delivering four phases of capability to defend Europe and  augment current protection of the U.S. homeland in the following  timeframes: Phase 1 in 2011, Phase 2 in 2015, Phase 3 in 2018, and Phase 4  in 2020. DOD’s phased schedule for the European PAA comprises multiple  elements and interceptors to provide increasingly integrated ballistic  missile defense capability. The policy is structured around the phased  deployment of increasingly capable variants of the SM-3 interceptor,  together with sensors, command and control, and other capabilities. It is  projected that each successive phase will deliver additional capability with  respect to both threat missile range and raid size.
Table 2 outlines the plans and estimated delivery timeframes associated  with each European PAA phase.
However, many aspects of the European PAA have not yet been  determined. DOD has thus far committed to using two land-based Aegis  Ashore facilities and at least one Army Navy/ Transportable Radar  Surveillance—Model 2 (AN/TPY-2) radar. Additionally, each European  PAA phase could have as many as three Aegis BMD ship patrol areas, but  DOD has not yet committed to a specific number of ships or SM-3  interceptors for each phase. DOD also has not yet committed to the  specific type or number of the other elements and interceptors that will be  part of the European PAA phases.
section_title: MDA Met Most of Its Fiscal Year 2010 Asset Delivery Goals
paragraphs: MDA increased inventories for Aegis, GMD, Sensors, and THAAD in fiscal  year 2010. The Aegis, GMD, and Sensors elements all either met or  exceeded their delivery goals, but THAAD was unable to meet all of its  2010 delivery goals. Table 3 shows the asset deliveries that we were able to  track and record based on the agency’s 2009 BAR and other information  we obtained directly from each program element.
In fiscal year 2010, Aegis BMD planned to deliver two additional ships to  the fleet with the Aegis Weapons System software as well as 17 SM-3  missiles. The program reported that it met its goal of delivering two  additional ships with the Aegis Weapon System software as planned,  which completed the 20-ship fleet. In addition, the program delivered 26  SM-3 Block IA missiles in fiscal year 2010—9 more than originally planned.
Furthermore, MDA redelivered 2 missiles that resolved quality problems  discovered in 2009.
Meanwhile, GMD met its delivery goals, delivering three interceptors as  planned. The program also delivered two additional interceptors in fiscal  year 2010 that were originally scheduled to be delivered in fiscal year 2009.  The program also delivered an interceptor that was used during a fiscal  year 2010 flight test. Lastly, the BMDS Sensors program successfully  delivered one AN/TPY-2 radar to THAAD as planned.
The THAAD program partially met its goal to deliver its first battery in  fiscal year 2010. While some assets of its first battery were delivered,  which included an X-band radar, a fire control and communications  system, and some ground support equipment, none of the battery’s 24  interceptors or 3 launchers were delivered as planned. The interceptors  are experiencing delays because of production issues stemming from  design and qualification issues related to a safety system in the  interceptor.  The launchers are experiencing delays to completing the  government acceptance process because of issues during production  associated with manufacturing challenges, parts obsolescence, design  changes, and manufacturing defects. Late qualification and design changes  have also delayed the acceptance process further. THAAD plans to  complete the interceptors by the end of the fourth quarter of fiscal year  2011. The government acceptance process for the launchers should be  complete by the third quarter of fiscal year 2011.
section_title: MDA Undertook Steps to Improve Transparency and Accountability
paragraphs: Throughout 2010, MDA continued to make significant changes to its key  acquisition processes in order to increase transparency and accountability  and reflect acquisition best practices. Specifically, MDA assessed each  program to determine its acquisition phase (technology development,  product development, or production), and for the first time, developed and  reported detailed baselines for each element, including resource baselines  outlining key costs for several BMDS components. While the steps taken  are significant and positive, our analysis identified shortcomings in cost,  schedule, and testing baselines that limit the extent to which cost and  schedule growth as well as system performance can be tracked.
subsections: section_title: New Review Process Results in More Detailed Baselines, but Gaps Remain
paragraphs: In our last report, we recommended that in addition to procurement unit  costs, the unit costs for BMDS assets reported in the 2010 BAR be  broadened to include development costs so that all costs were included in  its unit costs. We also recommended that MDA develop and report to  Congress a measure for schedule baseline goals that incorporates  delivering integrated capabilities to the warfighter and the dates at which  performance baselines will be achieved. In addition, we recommended  that top-level test goals for each element, or appropriate portions thereof,  be reported to Congress in the 2010 BAR. A final recommendation called  for MDA to report variances against all established baselines. DOD  concurred with all of these recommendations.
In 2010, MDA made progress in implementing these recommendations by  finalizing a new baseline phase review process in which the agency set  detailed baselines for several BMDS elements, or portions of elements, for  the first time. Specifically, MDA established resource, schedule, test,  operational capacity, technical, and contract baselines for several  BMDS components. It reported these to Congress in its June 2010 BAR.  MDA identified three phases of development where baselines are  approved—technology development, product development, and initial  production phases—and specified the key knowledge that is needed at  each phase. MDA officials stated that they expect that aligning the  development efforts with the phases will help to ensure that the  appropriate level of knowledge is obtained before the acquisitions move  from one phase to the next. In another key step, approval of the product  development and initial production baselines will be jointly reviewed by  the Director of MDA and the respective service acquisition executive, as a  number of missile defense systems are expected to eventually transition to  the military services for operation. In addition, in regard to these new  phases, the agency established a process for approving baselines. As a  result of MDA’s new baseline phase review process, its 2010 BAR is more  comprehensive than its 2009 BAR.
This year, we specifically assessed the new resource, schedule, and test  baselines set by MDA. While the material reported was more  comprehensive and provided more transparency into resources, schedule,  and testing than in previous years, we identified shortcomings that limit  the ability to track cost and schedule growth and performance.
subsections: section_title: Resource Baseline
paragraphs: The resource baseline is the expected investment needed that leads to the  delivery of a BMDS product. MDA presented unit cost baselines for  specific assets of each element as well as baselines for cost components of  the element including development costs, production and deployment  costs, and military construction costs. As a result, there can be multiple  baselines for each element. This was the first time that MDA reported  program acquisition unit costs (which include development costs in its  calculation) and baselines for portions of, and in some cases all, life-cycle  costs derived from life cycle cost estimates.
Unit cost is the cost divided by the quantity. It is usually reported in  two ways: (1) average procurement unit cost—procurement cost  divided by procurement quantity—and (2) program acquisition unit  cost—acquisition cost (which includes both development and  procurement) divided by total quantity.
MDA’s baselines for life cycle costs are based on life cycle cost  estimates. A life cycle cost estimate should encompass all past (or  sunk), present, and future costs for every aspect of the program,  regardless of funding source, and can be used to support budgetary  decisions, key acquisition decision points, milestone reviews, and  investment decisions. The elements of a program’s life cycle cost  include the costs of research, development, test, and evaluation;  production and deployment; military construction; operation and  sustainment; and disposal. MDA’s life cycle cost baselines in many  cases included sunk costs and costs from the future years defense plan  through completion. In addition, MDA included a time-phased estimate  of the costs including sunk costs, costs through the future years  defense plan, and “to complete” costs.
MDA provided 12 cost baselines for elements’ efforts under our review,  including Aegis Ashore; Aegis BMD SM-3 Block IB missiles and associated  software upgrades for ship sets; future Aegis BMD ship software upgrades;  two capability deliveries for AN/TPY-2; C2BMC Spiral 6.4; GMD; two  capability deliveries for the Sea Based X-band (SBX) radar; the current  THAAD capability configuration; and the Targets and Countermeasures  program short-range, medium-range, intermediate-range, and  intercontinental ballistic missile target efforts.
While the amount of information presented has increased, we found that  the unit cost baselines and the baselines for portions of, and sometimes  all, life cycle costs in the BAR did not provide clear, consistent, and  complete information, which affected its usefulness for oversight.
Parts of the 2010 resource baselines were unclear and inconsistent  because MDA used particular terms that did not always appear to  accurately describe the costs and baselines it was reporting. For  example, MDA presented “life cycle costs” in several resource  baselines but the use of this term is misleading because in some cases  the reported totals appeared to only partially reflect all the main life  cycle cost categories. Specifically, it is unclear if disposal costs and, in  some cases, operation and sustainment costs were accounted for in all  of the life cycle costs. No explanation is provided in the BAR to  explain why some of the cost categories appear to be missing or to  describe how the use of the term life cycle cost is appropriate for what  was reported.
The resource baselines for three of the four Targets and  Countermeasures resource baselines were incomplete because a large  percentage of sunk costs were excluded. The BAR, however, does not  disclose that the sunk costs were excluded nor does it explain why  MDA decided to exclude them.
MDA officials told us that they mistakenly baselined operation and  sustainment costs in the 2010 BAR and intend to remove those  baselines in the 2011 BAR. Choosing not to baseline these costs, in  some cases, excludes a large portion of costs from the total cost  baseline MDA is reporting. Further, officials also told us that they do  not plan to report variances in future BARs for any of the remaining  development, production and deployment, or military construction  costs that are baselined. Reporting variances is important because  deviations from the baseline plan provide valuable insight into  program risk and its causes to decision makers. It is unclear why the  agency will not report variances against these baselines.
MDA was also unclear and incomplete in its reporting of operation and  sustainment costs. In four of five cases, the operation and sustainment  costs reported in the BAR are only those costs for which MDA is  responsible and do not include the costs borne by the services.  MDA’s BAR does not explain that the operation and sustainment costs  represented are only a portion of those costs that will be required for  the program.
Although MDA presented average procurement unit costs for 10 BMDS  assets in its 2009 BAR, it only reported on 6 of the 10 in its 2010 BAR.  MDA also did not identify variances for any of these unit costs or  explain why these variances were not provided, although it could have  reported variances in 3 of the 6 cases.
MDA was also unclear and inconsistent in how it reported target unit  costs—by choosing not to report average procurement or program  acquisition unit costs. Instead the program reported average unit costs  and, separately, recurring costs without defining these costs or  explaining their purpose the BAR. By reporting only the average unit  costs for targets, MDA is not reporting the full unit costs to develop  and procure targets that would normally be captured in program  acquisition unit costs as it reported for other baselines.
Details on these inconsistencies, exclusions, and issues with clarity are  outlined in appendix II.
Finally, we sought to determine whether the cost baselines reported in the  BAR were supported by high-quality—that is, reliable—cost estimates.  According to the GAO Cost Estimating and Assessment Guide, the cost  estimate is part of a total systems analysis and is a critical element in any  acquisition process to help decision makers evaluate resource  requirements at milestones and other important decision points. Cost  estimates establish and defend budgets and drive affordability analysis.  The guide identifies the 12 steps necessary for developing a high-quality  cost estimate, which we used to assess MDA’s life cycle cost estimates to  determine whether they were (1) comprehensive, (2) well documented, (3)  accurate, and (4) credible. According to MDA cost officials, they  developed their cost estimates based on the methodology in our guide.  Our methodology and the full details of our assessment are reported in  appendix II.
While MDA was able to provide life cycle cost estimates for all 12 of the  BAR life cycle cost baselines reported, we found that 6 of the estimates did  not match the costs that were reported in the BAR—specifically those for  Aegis Ashore, Aegis BMD SM-3 Block IB missile and associated software  upgrades for ship sets, C2BMC Spiral 6.4, GMD, SBX capability deliveries,  and current THAAD capability configuration estimates. We did not assess  those estimates further because they did not support the BAR baselines.  For the remaining 6, we found that the documentation that MDA provided  contained insufficient evidence to meet the characteristics of a high- quality cost estimate. Specifically, the estimates were not comprehensive,  lacked documentation, were not completely accurate, or were not  sufficiently credible. See figure 1 for the results of our review of the 6  estimates.
The 6 estimates were not fully comprehensive because in some cases they  did not provide enough detail to enable us to determine if all government  and contractor costs had been included, or, in other cases, did not include  the historical data necessary to back up given assumptions. The estimates  were not well documented in some cases because the estimates lacked  documentation necessary to recreate the estimate, or, in some instances,  did not provide enough information on how cost factors were derived.  None of the estimates met the standards for accuracy because, in some  cases the estimates did not provide a confidence level making it  impossible to determine if the estimate was unbiased and based on most  likely costs, did not provide access to the detailed calculations used in the  estimate, or discrepancies existed on costs among several of the  documents provided. Finally, costs associated with the estimates lacked  credibility because none of the 6 estimates performed a cost sensitivity  analysis, provided evidence that cost driver cross checks were performed,  or completed independent cost estimates.
As discussed above, one of the criteria for a credible cost estimate is  having an independent cost assessment. Other DOD major defense  acquisition programs are required to perform an independent cost  estimate before advancing through major milestones. Because of the  flexibilities granted in 2002, MDA is not required to obtain independent  cost estimates for its programs, and has not otherwise required their  programs to do so. The Office of the Director for Cost Assessment and  Program Evaluation (CAPE) has performed independent cost estimates for  the Aegis BMD program as well as portions of other MDA programs in the  past including the STSS and GMD programs. More recently, the CAPE  performed an independent cost estimate of THAAD procurement costs.  MDA has committed to work with the CAPE to develop cost estimates for  other MDA elements in the future. MDA’s plans to work with the CAPE to  develop cost estimates for more MDA elements will contribute to the  credibility of its estimates.
Although MDA established life cycle cost baselines as part of its resource  baseline for the first time in fiscal year 2010, cost progress cannot be  independently evaluated until MDA reports baselines that are supported  by reliable, high-quality cost estimates and addresses the issues noted in  appendix II.
section_title: Schedule Baseline
paragraphs: The schedule baseline, as presented in the 2010 BAR, is a timeline for key  product development milestones and tasks, such as key decision points  and product deliveries. For the first time, MDA reported detailed schedule  baselines, including information about the program phase decision points  and capability deliveries, element knowledge points, models and  simulations, and product and capability development efforts. In addition,  some schedule baselines also include scheduled flight and ground test  information, as well as fielding and field support events. For example, the  GMD schedule baseline shows fielding and field support deliveries, such  as hardware deliveries, which includes new and refurbished GBI  deliveries; facilities, such as silo construction; and software upgrades.  However, the amount and type of information presented varies from  element to element.
While the amount of information presented is a significant improvement  over that in the 2009 BAR, we found that the asset delivery schedule  reported in the 2010 BAR was not comparable to the asset delivery  schedule in the 2009 BAR. The 2009 BAR included a comprehensive list of  planned asset deliveries, as well as schedule delays and accelerations.  However, the 2010 BAR did not include a comprehensive list of planned  asset deliveries nor did it report variances against the delivery plans it set  in 2009. As a result, we could not track and analyze the timing and number  of asset deliveries using the BAR. Instead, we had to request detailed  information from each program element. The lack of variance reporting on  asset deliveries from 2009 to 2010, coupled with the absence of planned  asset deliveries limits the usefulness of the BAR for oversight.
section_title: Test Baseline
paragraphs: MDA’s 2010 BAR includes a schedule of major flight tests, ground tests,  key modeling and simulation events and the primary goals associated with  those tests and events. The test events found in the BAR are further  detailed and explained in MDA’s test baseline known as the Integrated  Master Test Plan (IMTP). The IMTP is the management tool that MDA uses  to establish and document test requirements for the BMDS test program.  We reported last year that MDA extensively revised the IMTP to address  concerns that the plan was not effective for management and oversight  because it was revised frequently, only extended through the following  fiscal year and was not well integrated with other key aspects of testing  such as target acquisitions. While the new IMTP still represents a  significant improvement in test planning and transparency for MDA,  frequent revisions to the test baseline due to test failures and/or target  availability, hinders external oversight of MDA’s test baseline and the  funding allocated for testing.
More specifically, the revised IMTP extended the test baseline through  2015 and better laid out MDA’s test plan by establishing and documenting  the test requirements for the BMDS with specific focus on collecting the  data needed for verification, validation, and accreditation of the BMDS  models and simulations. Also, under the new plan, each test was designed  to test certain criteria, so that, according to MDA test officials, a failed test  did not necessarily require alteration to the IMTP.
Despite the agency’s effort to revamp its test plans, MDA’s test plan  remains success-oriented. For example, MDA officials told us that they do  not plan for test failures when developing the test schedule and that there  is no flexibility to absorb test failures. Despite experiencing several  failures in recent years and conducting several retests, the agency does not  build in contingencies for retests in its test plans. Moreover, according to  program officials, there are no risk reduction flights planned for newly  developed targets and MDA does not budget for extra flight test  interceptors. Accordingly, having a sufficient schedule and funding margin  to account for actual recent experience remains key to a more successful  and stable test program.
Because the IMTP is success-oriented, any problems encountered with the  plan necessitates a change in the plan and frequent changes to those plans  make it difficult to maintain effective oversight of MDA’s test program  using the test baselines reported in the BAR. For example two intercept  tests that were planned for the THAAD program in fiscal year 2010 did not  take place because of a target failure in a previous flight test.  Additionally, MDA had to alter GMD’s planned flight test schedule for  fiscal years 2010 and 2011 in order to conduct a retest of a prior test— FTG-06—that failed January 2010. According to the Director, Operational  Test and Evaluation, this retest will likely delay the previous flight test  program by at least 6 months. Also, MDA canceled an additional future  flight test in order to conduct the retest, thereby reducing the amount of  intercept data that will be collected.
These frequent changes make it more difficult to track testing progress  and system performance. Moreover, it is unclear what happens to funding  planned for tests within a certain fiscal year when the test is canceled or  moved to a subsequent fiscal year. Each year, as part of its budget request,  MDA justifies its flight test funding request based on a planned set of  specific tests. However, because MDA is unable to execute its test plan as  presented in the BAR and the budget and does not identify test cost  variances in the BAR or the budget, the connection between the  justification for funds and the actual execution is broken. For example,  MDA requested funding for a GMD flight test—FTG-09. According to the  fiscal year 2010 budget request, this test was scheduled to be conducted in  fiscal year 2011 as a salvo test. However, because of the noted failure in  FTG-06, this test was canceled. Consequently, reviewing budget  documents and the BAR does not enable us to determine how much was  spent on the FTG-09 test prior to its deletion. Until MDA establishes or  reports a test baseline that is both relatively stable and synchronized with  other key BMDS management documents, such as the budget, as well as  identifies test cost variances, decision-makers will continue to make  decisions based on unstable data that are not effective for oversight or  useful for timely decisions.
Lastly, as in prior years, target availability also continued to affect MDA’s  test plans in fiscal year 2010. Since 2006, we have reported that target  availability has delayed and prompted modifications to planned test  objectives. This trend continued in 2010. For example, because of target  availability, five tests scheduled for fiscal year 2010 were canceled  because of a moratorium on air launches of targets. Additionally, target  shortfalls have contributed to delays in flight tests, reduced the number of  flight tests, and altered test objectives. According to MDA officials, targets  were the main driver of the revisions of the test schedule in the latest  IMTP. For additional information regarding target issues and their effect  on BMDS testing, see appendix XII.
Congress established new requirements for Missile Defense baselines, in  December 2010, in the National Defense Authorization Act for Fiscal Year  2011.  Several of the shortcomings in the 2010 BAR we discuss above are  addressed in the law as MDA is now required to submit annual reports on  acquisition baselines—to include schedules, cost estimates, and test  baselines— starting February 15, 2011. MDA’s 2010 BAR was approved  prior to this law. However, we plan to assess the agency’s 2011 BAR  against the law’s criteria in our 2012 report.
section_title: Acquisitions Supporting European PAA Face Synchronization, Transparency, and Accountability Challenges
paragraphs: The shift in focus for European missile defense represents the most  comprehensive shift in U.S. Ballistic Missile Defense policy since 2002  when the Secretary of Defense created MDA and the President directed  the fielding of an initial set of missile defense capabilities by 2004. To  assist Congress in its review of this policy shift and its implications, we  were asked to assess DOD’s efforts to implement the phased adaptive  approach for Ballistic Missile Defense in Europe. In December 2010, we  issued a correspondence on acquisition management and near-term  development risks related to the European PAA. In January 2011, we  reported on (1) the extent to which DOD has developed guidance and  addressed management of cost and schedule in the planning and  implementation of the European PAA and (2) the extent to which DOD  planning for the European PAA is informed by operational performance  data.
In our January report on the European PAA, we recommended actions to  enable effective oversight of the European PAA, including establishment  of a life cycle cost estimate to help assess affordability and an integrated  schedule to help identify European PAA implementation risks. With  respect to acquisition management for the European PAA, our December  2010 report found that DOD has not fully implemented a management  process that synchronizes European PAA acquisition activities and  ensures transparency and accountability. At the time of our review, DOD  had made progress in acquisition planning for technology development  and systems engineering and testing and partial progress in defining  requirements and identifying stakeholders but had not yet developed a  European PAA acquisition decision schedule or an overall European PAA  investment cost. We found that the limited visibility into the costs and  schedule for the European PAA and the lack of some key acquisition  management processes reflect the oversight challenges with the  acquisition of missile defense capabilities that we have previously  reported. The consequences of these issues have included limited means  of independently assessing progress and a limited basis for oversight and  the department entering into production before fully demonstrating  system performance, leading to rework, cost increases, delays, and  uncertainties about delivered capabilities. We concluded that for the  European PAA, the flexibility desired by DOD is not incompatible with  appropriate visibility into key aspects of acquisition management. As DOD  proceeds with the European PAA acquisition activities, it is important for  Congress and the President to have assurance that the European PAA  policy is working as intended and that acquisition activities are cost- effective.
In response to our December 2010 report, DOD stated that it disagreed  with GAO’s approach to assess the European PAA as what it termed a  “near-distinct” element of the BMDS, rather than DOD’s decision to  employ the department’s existing processes. As we have consistently  reported, the department’s existing processes for developing and acquiring  missile defense have transparency and accountability issues that limit  oversight and preclude assessing overall progress and limit opportunities  for constructive action to put programs in a better position to succeed.  The principles we used to assess the European PAA are embedded in the  department’s acquisition guidance as well as the Office of Management  and Budget’s guidance for capital programming across federal agencies,  particularly as it relates to bringing together and synchronizing multiple  development efforts. Although we understand that the European PAA is a  policy approach, not an acquisition program, it nonetheless represents an  arrangement of significant investments, it requires a high degree of  coordination and technical integration, and the progress implementing the  European PAA continues to be of congressional interest. Thus, we  reported that we continue to believe these acquisition management  principles serve as a useful, appropriate and beneficial standard by which  to assess the department’s approach to managing European PAA  acquisitions.
subsections: section_title: European PAA Acquisition Schedules Are Highly Optimistic
paragraphs: We have consistently reported that a sound, executable acquisition has  firm requirements, mature technologies, and a strategy that provides  sufficient time and money for design activities before making the decision  to start system development and demonstration or to transition to  production. The administration’s European PAA policy committed DOD to  a schedule before the scope of system development effort was fully  understood. DOD is working to develop plans for implementation within  the constraints of policy time frames; however, system schedules are  highly optimistic in technology development, testing, production, and  integration, leaving little room for potential delays. As efforts to meet near- term commitments unfold, the schedule for delivering capabilities may be  difficult to achieve and resources needed may grow. (See table 4.)
section_title: New GMD Strategy Curtails Planned Operational Quantities, and It May Take Years to Understand Capabilities, Limitations, and Sustainability
paragraphs: Another significant change made for 2010 included a reduction in the  number of planned emplaced GBIs from 44 to 30. In making this reduction,  the Secretary of Defense stated that 30 GBIs provide an adequate near- term defensive capability for the United States, while allowing for testing  and resolution of problems with interceptor technology. According to the  Ballistic Missile Defense Review, over the past few years, MDA  accelerated development of the homeland defense by simultaneously  developing and fielding assets. This step was taken largely in order to  facilitate the deployment of GBIs in time to coincide with the expected  deployment of intercontinental ballistic missiles by North Korea. That  projected intercontinental ballistic missile deployment has not yet  occurred and consequently, DOD concluded that it is unnecessary to  continue to accept the same level of programmatic risk in the GMD  program going forward. Rather, the department plans to focus homeland  defense on maintaining the current level of capability and developing  future, proven capabilities to enhance homeland defense should a new  threat emerge while still maintaining current GMD capabilities.
We were mandated by the Congress in 2009 to examine a DOD assessment  and plan related to the GMD system, that included issues related to  acquisition, sustainment and refurbishment. We found that while the GMD  system has demonstrated a limited capability, DOD has not yet determined  the system’s full capabilities and limitations. According to the Director of  Operational Test and Evaluation’s fiscal year 2010 assessment, ground  tests support characterization of GMD performance and test results  suggest that GMD provides a capability to defend the United States against  limited, uncomplicated, long-range ballistic missile threats. However, lack  of sufficient data for comprehensive model and simulation validation and  accreditation continues to preclude a full performance evaluation of GMD.  Even though GMD continued to further develop its capability in fiscal year  2010, the GMD program has been unable to successfully achieve all of its  testing objectives since 2005. Testing shortfalls continued in fiscal year  2010 when GMD failed to achieve an intercept during Flight Test Ground- based Interceptor (FTG)-06. As we previously reported, FTG-06 was  planned as the first test of GMD’s enhanced version of the kill vehicle  called the Capability Enhancement II (CE-II). Additionally, we noted that  MDA stated that the CE-II interceptors would not be declared operational  until the satisfactory completion of FTG-06. As a result of the failure in  FTG-06, MDA inserted FTG-06a, which also failed to achieve an intercept  in December 2010.
Despite testing delays, shortfalls, and failures, MDA has continued to  deliver assets. For example, neither the Capability Enhancement I (CE-I)  or CE-II Exoatmospheric Kill Vehicle (EKV) has verified its capability  against countermeasures—a capability it expected to demonstrate for the  CE-I in 2008. Failures in both the target and the interceptors over several  years have prevented collecting this information. Moreover, the CE-II EKV  has not successfully intercepted a target. Despite these shortfalls all CE-I  EKVs have been delivered and the manufacturing and emplacements of  the CE-II EKV continued in 2010. According to the Director, MDA, based  on the issues that arose in FTG-06a, he directed the agency to stop taking  deliveries of any more completed CE-II EKVs. MDA officials stated that the  agency is allowing the contractor to continue to work on those  components of the EKV that would not be factors in the FTG-06a flight test  anomaly in order to keep the production line moving and to allow a rapid  recovery of deliveries once the failure investigation team determines likely  cause or causes and develops either design changes or mitigations.  However, based on our assessment of the GMD developmental flight test  plan that was scheduled to continue until 2021, even with this delay, most  CE-II EKVs will be manufactured and delivered prior to completing the  developmental flight tests and validating their capability.
Delays in collecting data to fully understand capabilities and limitations  also hinder GMD’s efforts to adequately plan sustainment efforts. In  fiscal year 2010, the Director of MDA stated that the GMD program is  expected to be in service until 2032. However, in certain cases, the agen is just beginning to collect necessary sustainment data that would suppo the service life. In addition, the collection of this data is incomplete and, many cases, will not be completed for many years. Table 5 lists several  activities that are necessary to inform sustainment efforts and their status.  Appendix IX provides additional details regarding GMD sustainment  efforts.
section_title: Decisions on Acquisition of Targets in Fiscal Year 2010
paragraphs: MDA initiated a new targets acquisition plan in 2009 which called for  separate contract awards for different classes of targets, including  medium-range, intermediate-range, and intercontinental targets. This plan  was undertaken in response to GAO findings regarding cost, schedule and  performance problems with the previous targets acquisition approach.
MDA conducted a business case analysis in order to identify a new  acquisition plan that could address these problems. This new approach  was to seek separate, competitive contract awards for different classes of  targets, allowing MDA to benefit from greater opportunities for  competition and reducing the role of the prime contractor. In fiscal year  2010, as part of this strategy, MDA issued two solicitations: one for  medium-range targets, and another for intermediate-range targets.  However, the agency canceled the solicitation for medium-range targets  after receiving proposals. According to MDA officials, the proposals  received were more expensive than anticipated. MDA officials cited  higher-than-expected costs for the proposals received on the medium- range solicitation as the primary reason it was canceled, but the  information they provided did not show detailed analysis of the effect of  the cancellation on the cost and schedule assumptions of the business  case. The cost effects of this change remain unknown. According to  targets program officials, they may issue a new competitive solicitation for  medium-range targets, but they have not yet done so. MDA also issued a  draft solicitation for intercontinental-range targets in September 2010, but  canceled the solicitation in February 2011 before it was finalized.
Separately, MDA also issued a new undefinitized contract action in the  second quarter of fiscal year 2010, which asked the incumbent prime  contractor to build five new medium-range air-launched targets. MDA  officials stated that this action was in response to the target failure in the  FTT-11 flight test, and was not related to the cancellation of the medium- range solicitation. Undefinitized contract actions authorize contractors to  begin work before reaching a final agreement on contract terms. The  current “not-to-exceed” level for the contract action is $496 million. MDA  has allowed this undefinitized contract action to continue for an extended  period—over 290 days as of January 2011. According to MDA officials,  the delay in definitization is due to changes in its requirements for the  targets, and they anticipate definitization in July 2011, by which time the  contract action will have remained undefinitized for about 450 days. The  extended use of undefinitized contract actions has previously been  identified by GAO and others as risky for the government. Because  contracting officers normally reimburse contractors for all allowable costs  they incur before definitization, contractors bear less risk and have little  incentive to control costs during this period. The government also risks  incurring unnecessary costs as requirements may change before the  contract is definitized. According to MDA officials, acquiring these new  medium-range air-launched targets through an undefinitized action was  necessary in order to meet the testing schedule. Documentation provided  by the agency showed only limited analysis prior to the decision to award  the undefinitized contract action. Officials recognized that this approach  would add to the cost of targets compared to the Air Force contract  vehicle they had previously used, but because the action remains  undefinitized, the agency does not yet know the final price of these targets.  For more detailed information regarding targets during fiscal year 2010,  see appendix XII.
section_title: Overall Performance of the BMDS Still Cannot Be Assessed Due to Models and Simulations Challenges and Shortfalls in Testing
paragraphs: For the eighth year, we are unable to report on the overall performance of  the BMDS because MDA models and simulations have not matured  sufficiently and may not be fully mature until 2017. Because the potential  combinations of BMDS configurations, intercept scenarios, and missile  threats are too numerous for ground and flight testing, assessing overall  BMDS performance depends upon the use of models and simulations to  understand the capabilities and limitations of the system. Such an end-to- end system-level simulation brings together the capabilities of various  element models in order to analyze how the BMDS integrated and fielded  radars, communication systems, and interceptors perform during  scenarios. To work effectively, these models and simulations need to be  anchored to data from ground and flight tests and validated by  independent evaluators—the BMDS Operational Test Agency (OTA)—in  order to have confidence in their results. Moreover, the system-level  simulation itself is expected to change over time as additional models  become available to represent the evolving BMDS configuration. Until  models and simulations are accredited and validated, BMDS performance  cannot be fully assessed.
In fiscal year 2010, MDA began execution of its revamped IMTP to collect  the data needed to accredit the models and simulations used for assessing  BMDS performance. The agency was able to conduct several BMDS  element tests that generated data needed to accredit models and  simulations as well as assess performance for individual BMDS elements.  For example, in June 2010 during a GMD booster verification test, C2BMC  was able to collect data for accreditation of its models and simulations  while demonstrating command and control of a key BMDS radar. In  addition, in August 2010, MDA and the BMDS OTA conducted a ground  test referred to as Ground Test Integrated-04b (GTI-04b), to demonstrate  functionality, interoperability, and performance of the BMDS and to  characterize BMDS element capabilities. According to the Director of  Operational Test and Evaluation, this test provided the most accurate  representation to date of the BMDS including GMD for characterization of  performance as well as insight into GMD functionality, interoperability,  and performance within the BMDS. Test results suggested that GMD  provides a capability to defend the United States against limited long- range ballistic missiles with uncomplicated, emerging threat reentry  vehicles.
While some individual BMDS elements’ models and simulations provided  insight into their interoperability and performance within the BMDS,  BMDS OTA officials told us that current BMDS system-level modeling and  infrastructure are inadequate to assess the overall performance of the  BMDS. BMDS OTA officials told us that the current IMTP focuses on data  needed to accredit element-level models, but it does not address system- level integrated performance. In an August 2009 memorandum to MDA, the  BMDS OTA and the Joint Functional Component Command for Integrated  Missile Defense, referred to as the Warfighter, identified testing  requirements that are needed to assess BMDS performance. More  importantly, in this memorandum both the BMDS OTA and the Warfighter  alerted MDA that there were a number of system-level model,  infrastructure, and stimuli limitations that adversely affected the OTA’s  ability to assess BMDS performance. The Warfighter and the BMDS OTA  require testing to reflect realistic representations of strategic and  regional/theater defense designs with the full complement of fielded  BMDS assets. The BMDS OTA and the Warfighter noted that as the BMDS  matures and fielded assets increase, modeling and simulation capabilities  and laboratory representations of BMDS assets must keep pace to  maintain operational realism. However, they stated that current laboratory  and model and simulation limitations prevent full asset representation and  fail to test the loading that would occur on the communication links as  well as the complexity of a full regional/theater conflict on command and  control nodes. This limitation severely affects event execution and the  ability of the Warfighter to make informed decisions relative to the  operational readiness and capability of BMDS elements.
In addition, as with previous years, failures and delays in testing have  continued to delay the validation of the models and simulations used to  assess the overall performance of the BMDS. GMD, Sensors and THAAD  were able to conduct several key flight tests demonstrating element  performance; however, each also failed to achieve all planned fiscal year  objectives. Additionally, the C2BMC element successfully participated in  multiple element tests and completed some ground tests needed to declare  a new software configuration operational. Aegis BMD and STSS were  unable to conduct either of their planned developmental flight tests for  fiscal year 2010. For specific details regarding each element’s annual  testing accomplishments, delays, and failures, see the individual element  appendixes.
section_title: Conclusions
paragraphs: Last year we noted that as MDA transitioned to new leadership, a new  acquisition strategy, a new test strategy, and a shift in emphasis toward  early intercept capabilities, the agency had an opportunity to chart a  course that enables transparency and accountability as well as flexibility.  We also noted that the Director of MDA had begun new initiatives in  accordance with guiding principles of DOD’s acquisition policies, which  already embrace knowledge-based practices and sound management  controls. This year we began to see the fruit of those efforts as MDA  applied these new policies and began to provide a better foundation for  Congress and others to assess progress and hold senior leadership  accountable for outcomes. However, while it is undoubtable that some  progress has been made in terms of implementing new acquisition reviews  and reporting detailed baselines, there remain critical gaps in both the  material reported and particularly the quality of the underlying cost  estimates needed to establish baselines.
In addition, the department has undertaken another series of significant  changes to the acquisition focus of MDA. The decision in 2010 to shift the  focus of acquisitions from strategic to regional missile defense brings with  it a host of significant challenges for MDA and the department. Ensuring  that these changes are properly analyzed, efforts are properly  synchronized and the resulting decisions are communicated and justified  will require significant attention from top departmental leaders.
Lastly, fundamental to accountability and oversight is being able to  establish a sound plan and to track actual performance against that plan.   For a variety of reasons, including a lack of baselines and changing goals  and methods for tracking progress, we have not been able to report with  any confidence or consistency on how the missile defense program has  performed against its plans or expectations these past 8 years. Thus, it is  imperative that the latest policy initiatives be sustained in a way that  provides consistent, complete and reliable reporting of progress and  variances against those baselines from year to year. The baseline structure  MDA established in 2010 is a good start and represents a reasonable basis  for reporting progress and variances in the future, and thus should be  safeguarded for several years.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense direct MDA to undertake the  following 10 actions to strengthen its baselines, facilitate external and  independent reviews of those baselines, ensure effective oversight of the  BMDS, and further improve transparency and accountability of its efforts.  Several of these recommendations relate to how DOD may meet the new  statutory requirements to establish and report baselines under the Ike  Skelton National Defense Authorization Act for Fiscal Year 2011.
To improve clarity, consistency, and completeness of the baselines  reported to Congress, the Secretary of Defense should ensure that MDA  takes the following actions:   1.  For resource baselines:  a)  Provide more detailed explanations and definitions of information  included in resource baselines; particularly operations and support  costs and unit cost calculations.  b)  Label cost estimates to appropriately reflect the content reported  and explain any exclusions.  c)  Include all sunk costs in all of its cost estimates and baselines.   d)  Obtain independent cost estimates for each baseline.  e)  In meeting new statutory requirements to include a cost estimate in  each acquisition baseline for a program element, take steps to  ensure these cost estimates are high quality, reliable cost estimates  that are documented to facilitate external review.
2.  For schedule baselines:  a)  In meeting new statutory requirements to report on an acquisition  baseline including a comprehensive schedule, include a  comprehensive list of actual versus planned quantities of assets  that are or were to be delivered each fiscal year.  b)  In meeting new requirements to report variances on acquisition  baselines, report on variances of these quantities by fiscal year and  the reasons for these differences.
3.  For test baselines:  a)  In meeting new statutory requirements to report variances between  reported acquisition baselines, also report variances between the  test plan as presented in the previous acquisition baseline and the  test plan as executed that explain the reason for any changes.  b)  Report the cost effects of those test changes in either the BAR or  the budget justification documentation.
In order to stabilize the test plan and ensure the test baseline can absorb  test failures and test delays  and remains executable, the Secretary of  Defense should ensure that the Missile Defense Agency:  4. Includes sufficient schedule and resource margin, including spare test  assets and targets, based on recent test experience and forecasted testing  demands.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report. These  comments are reprinted in appendix I. DOD also provided technical  comments, which were incorporated as appropriate.
DOD fully concurred with 7 of our 10 recommendations, including our  recommendations to take steps to ensure its cost estimates are high- quality, reliable cost estimates that are documented to facilitate external  review. In response to this recommendation, DOD stated that MDA will  follow the GAO Cost Estimating and Assessment Guide for each program  reported in the BAR. In doing so, MDA Cost Estimating and Analysis  Directorate plans to form a cost assessment group to formally assess each  cost estimate to ensure they are well documented, comprehensive,  accurate, and credible. DOD also fully concurred with our  recommendation to report the cost effects of its test changes in either the  BAR or budget justification documentation, stating that MDA will provide  the cost effects as part of the individual program resource baselines  starting in the 2012 BAR.
DOD partially concurred with our recommendation that the Secretary of  Defense direct MDA to include all sunk costs in all of its cost estimates  and baselines. DOD stated that MDA will continue to report sunk costs in  most of its acquisition programs except targets, where it will continue to  report unit costs in the same manner as the 2010 and 2011 BARs.  According to DOD, because each target is inherently a test article and no  two are identical, there are always variable non-recurring costs associated  with each article. Also, due to the extensive reuse of previous strategic  missile components in the targets program, including all sunk costs does  not reflect MDA program costs accurately. Accordingly, MDA will use the  costs incurred/planned during the future years defense plan to calculate  unit costs. These unit costs will not include any sunk costs. We continue to  believe that all costs, including all sunk costs incurred for individual target  efforts, should be reported to the maximum extent possible, regardless of  the agency or service that originally incurred those costs. To the extent  that a sunk cost, such as the original cost of previous strategic missile  components, can not be calculated, that fact should be explained when the  costs are reported. Moreover, DOD’s method of reporting and baselining  target unit costs will not reflect even MDA program costs accurately  because it excludes MDA-incurred sunk costs. Excluding these costs,  which can run into the hundreds of millions of dollars, precludes  understanding of the full investment required to develop and procure  targets, as well as limiting insight into developmental progress. In addition,  the incomplete costs reported will be changed every two years because, as  DOD noted, MDA uses only the costs for the future years defense plan to  calculate the target unit cost baselines. The future years defense plan  covers a six year period of time, but changes every two years to a new six  year period of time. Calculating unit costs this way provides neither the  full cost nor the full quantities of targets as it removes any costs or  quantities prior to fiscal year 2010. We therefore continue to believe that  including these costs will aid departmental and congressional decision  makers as they make difficult choices of where to invest limited resources.
DOD also partially concurred with our recommendation that the Secretary  of Defense direct MDA to report, in its annual reports on acquisition  baselines, variances in quantities of assets by fiscal year that are or were  to be delivered each fiscal year and the reasons for these differences. DOD  stated that a change in an asset quantity in a fiscal year may drive a  reportable BMDS schedule variance and noted that any changes in asset  quantities would also be reflected in unit costs and in the technical  baseline. In a separate response, DOD stated that in meeting new statutory  requirements, MDA will report a comprehensive list of actual versus  planned assets that are or will be delivered each fiscal year. MDA, in its  2009 BAR, included a comprehensive list of actual versus planned  quantities of assets that were to be delivered each fiscal year as its  schedule baseline, identified the criteria for reporting schedule variances,  and also planned to provide the rationale for any variances. MDA is now  statutorily required to establish an acquisition baseline for each element  that includes a comprehensive schedule, including delivery and fielding  schedules. In addition, because DOD has stated that they will include  actual versus planned quantities when meeting annual baseline reporting  requirements, we continue to believe that additionally reporting the  rationale for these variances, as MDA had previously committed to, would  benefit departmental and congressional decision makers as they meet  their obligations to manage and oversee this effort.
DOD also partially concurred with our recommendation that the Secretary  of Defense direct MDA to include sufficient schedule and resource margin,  including spare test assets and targets, based on recent test experience  and forecasted testing demands. DOD responded that maintaining  cost/schedule margins and additional targets is not cost effective. Instead,  DOD stated that MDA will conduct an event-oriented test and acquisition  decision-based program and update the test plan frequently to reflect the  status of test execution. In addition, DOD noted that MDA will use a  “rolling spare” approach to targets. We remain concerned about MDA’s  inability to successfully execute those plans. MDA changed its approach to  testing 2 years ago, after struggling for years to successfully execute the  test plan, instituting a more rigorous planning effort. However, despite  increased attention to defining and resourcing its test plan, it has been  unable to stabilize and execute the plan successfully.  Without a change in  its approach, MDA is almost certain to continue to set and reset test plans  that are immediately unexecutable soon after they are reset. This  approach incurs significant hidden or unplanned costs as testing is  delayed and test plans are restructured. As we testified February 25, 2009,  the success of MDA’s test planning hinges on properly resourcing the tests  with sufficient time, funding and reliable targets. Our current  recommendation, based on the experience of the last 2 years, is designed  to help stabilize the test plan and reduce the frequency and effort required  to reset the test plan while gathering the knowledge MDA has decided it  needs as efficiently as possible. Until MDA adds sufficient cost and  schedule margin, including additional interceptors and targets, it will  continue to achieve less testing, at a greater cost than planned. We  therefore continue to believe that the department and Congress would  benefit if test plans incorporate reasonable cost and schedule margins.
Lastly, DOD identified 35 “factual errors" in its technical comments.  However, upon review we found that 7 of the 35 were not errors of fact,  but rather different conclusions; 6 we determined were errors in DOD's  official comments, 12 required additional substantiation that was not  provided or were material previously requested but could not be assessed  during this review because the extensive delay left no time for analysis;  and lastly 10 we determined were actual errors of facts and we therefore  made the appropriate changes in those cases.
We are sending copies of this report to the Secretary of Defense and to the  Director of MDA. The report also is available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Major contributors to this report are listed in appendix XV.
section_title: Appendix I: Comments from the Department of Defense
section_title: Appendix II: Assessment of MDA Resource Baselines
paragraphs: We reviewed 12 resource baselines for Ballistic Missile Defense System  (BMDS) elements’ efforts as presented in the June 2010 BMDS  Accountability Report (BAR) including:    Aegis Ashore;    Aegis Ballistic Missile Defense (BMD) Standard Missile 3 (SM-3) Block  IB missiles and associated software upgrades for ship sets;  future Aegis BMD ship software upgrades;  two capability deliveries for Army Navy/Transportable Radar  Surveillance (AN/TPY-2);    Command and Control, Battle Management, and Communications  (C2BMC) Spiral 6.4;    Ground-based Midcourse Defense (GMD);  two capability deliveries for the Sea Based X-Band (SBX) Radar;  current Terminal High Altitude Area Defense (THAAD) capability  configuration; and    Targets and Countermeasures program short-range ballistic missile;    Targets and Countermeasures program medium-range ballistic missile;    Targets and Countermeasures program intermediate-range ballistic    Targets and Countermeasures program intercontinental ballistic  missile target efforts.
The Missile Defense Agency (MDA) presented baselines for both unit costs  and portions of, and in some cases all, life cycle costs in the 2010 BAR.  MDA presented unit cost baselines for specific assets of each element as  well as baselines for cost components of the element including  development, production and deployment, and military construction costs.  As a result, there can be multiple baselines for each element. MDA  expressed the majority of its unit costs as program acquisition unit cost  (PAUC) or average procurement unit cost (APUC). The APUC is the total  procurement funds divided by the fielded quantity, and was included as  part of MDA’s unit cost baselines for the first time in the 2009 BAR. The  PAUC is the total cost for the development and procurement of the  acquisition program divided by the total quantity, and was included for  most of the elements for the first time in the 2010 BAR in response to GAO  recommendations. MDA’s 2010 BAR also presented, for the first time, a  baseline for portions of, and in some cases all, life cycle costs based on life  cycle cost estimates for several BMDS components.
Other major defense acquisition programs are required by statute to  describe program goals in an acquisition program baseline that, as  implemented by the Department of Defense (DOD), has been approved by  a higher-level DOD official prior to the program’s initiation. The baseline,  derived from the best estimates of cost, schedule, and performance  requirements, provides decision makers with the program’s total cost for  an increment of work, unit costs for assets to be delivered, the date that an  initial operational capability will be fielded, and the weapon’s intended  performance parameters. The baseline, according to DOD acquisition  guidance, includes cost parameters that should reflect realistic cost  estimates of the total program and/or increment, and should contain cost  parameters for major elements of program life-cycle cost. The elements of  a program’s life-cycle cost include research, development, test, and  evaluation; production and deployment; military construction; operation  and sustainment; and disposal costs. Once approved, major defense  acquisition programs are required to measure their program against the  baseline. The threshold application of these laws and policies related to  baselines are typically triggered by a program’s entry into engineering and  manufacturing development (formerly system development and  demonstration)—a phase in the DOD acquisition cycle during which the  weapon system is designed and then demonstrated in tests. The BMDS has  not formally entered into the DOD acquisition cycle, therefore this  requirement does not yet apply.
subsections: section_title: The Resource Baselines Reported in the BAR Do Not Provide Clear, Consistent and Complete Information
paragraphs: Parts of the 2010 resource baselines were unclear because MDA used  particular terms that did not always appear to accurately describe the  costs and baselines it was reporting. For example, MDA presented “life  cycle costs” in several resource baselines but it was unclear in the BAR  whether these costs cover the full range of cost categories or cost  components normally included in a life cycle cost estimate. In addition,  although MDA presented baselines for some of the costs, and, in other  cases, all of the costs included in its life cycle costs, program officials told  us that they mistakenly baselined operation and sustainment costs in the  2010 BAR and intend to remove those baselines in the 2011 BAR. Officials  also told us that they do not plan to report variances in future BARs for  any of the remaining development and production costs that are baselined.  The operation and sustainment costs that MDA did report in the BAR were  incomplete and unclear because, except in one case, they did not include  the costs borne by the services and there was no explanation in the BAR  for these exclusions. The resource baselines also included many  inconsistencies that were unexplained in the BAR. For example, MDA did  not report unit cost variances that could have been reported and MDA  presented alternate versions of unit costs without explaining how these  unit costs were constructed. There also appeared to be instances of  underestimating life cycle costs when we compared the time-phased  estimate to the life cycle cost baseline. It was unclear why these  inconsistencies existed for the life cycle cost baselines because MDA did  not explain in the BAR how any portions of the life cycle costs were  baselined.
Although MDA termed one section of its resource baseline “life cycle  costs,” the use of the term is misleading because in some cases the  reported totals appeared to only partially reflect all the main life cycle cost  categories. These categories normally include research and development,  production and deployment, operation and sustainment, and disposal  costs. Life cycle should be thought of as a “cradle to grave” approach to  managing a program throughout its useful life. This usually entails  identifying all cost elements that pertain to the program from initial  concept all the way through operations, support, and disposal. For some  of the elements, not all of these cost categories were identifiable in the  figures reported and as a result it appears that the totals reported were  mislabeled as life cycle costs. Specifically, it appeared that the Aegis  Ashore and future Aegis BMD ship software upgrades elements may not  have included operation and sustainment costs. It also appeared that none  of the 12 baselines reported disposal costs. No explanation was given in  the BAR for the exclusion of either of these cost categories or why the  term life cycle cost is appropriate for what was reported.
In another instance, the information reported in three of the four Targets  and Countermeasures resource baselines was not complete because sunk  costs were excluded from the life cycle cost baselines. The short-range  ballistic missile effort excluded more than $400 million in sunk costs from  its baseline, to leave less than 43 percent of its total estimated costs  baselined. The intermediate-range ballistic missile effort also removed  more than $900 million in sunk costs from its baselined development  costs, choosing to baseline less than half of the total estimated costs. MDA  did note in the BAR that costs beyond fiscal year 2015 were excluded from  the Targets and Countermeasures baselines because the quantities of  targets were uncertain in the years beyond 2015. The BAR, however, does  not inform the reader that the sunk costs were excluded nor does it  explain why MDA decided to exclude them. Agency officials told us that  sunk costs were excluded from the development baseline because sunk  costs did not fit the definition of average unit cost for the targets.  However, this response still does not explain why these costs were  excluded from the life cycle cost baselines. Excluding these costs in the  baseline means that the full costs through 2015 for these efforts are  underestimated and, therefore, are not transparent to decision makers  external to the agency.
MDA officials told us that they mistakenly reported operation and  sustainment costs as baselined for several elements and stated that they  do not intend to report variances for the remaining baselined portions of  life cycle costs, despite indications in the 2010 resource baselines that  variances would be reported in the future. For the current THAAD  capability configuration and the Aegis BMD SM-3 Block IB missiles and  associated software upgrades for shipsets, agency officials said that  reporting the operation and sustainment costs associated with these  elements as a baseline was an error. Although the agency intends to  continue to report operation and sustainment costs, they do not consider  those costs baselined and will not report them as part of the baseline in  the future. Choosing not to baseline these costs, in some cases, removes a  large portion of costs from the total cost baseline MDA is reporting. See  table 6 for details on the operation and sustainment costs as a percentage  of total costs MDA will not baseline in the future.
Further, the operation and sustainment costs that MDA reported in the  BAR were incomplete and unclear because, except in one case, they did  not include the costs borne by the services and there was no explanation  in the BAR for these exclusions. By excluding the services’ portion of  these costs, the operation and sustainment costs reported in the BAR for  four out of five of the elements’ efforts are underestimated since they only  include those costs that MDA will be responsible for paying. Only the  operation and sustainment costs reported for the two capability deliveries  for the SBX Radar included both the costs borne by MDA and the service.
The BAR does not explain these differences or exclusions in reporting its  operation and sustainment costs.
In addition, MDA officials stated that although MDA will continue to  report development, production and deployment, and military  construction costs as baselined, none of these baselines will report  variances. They stated that the resource baselines are represented solely  by unit cost baselines—not by any baselines for life cycle costs. Officials  told us that there is no technical reason that prevents MDA from reporting  variances for development, production and deployment, or military  construction costs. It is also unclear why the agency will not report  baselines and variances for operation and sustainment costs. Reporting  variances is important because deviations from the baseline plan give  management at all levels information about where corrective actions are  needed to bring the program back on track. While variances are often  perceived as something bad, they provide valuable insight into program  risk and its causes and can empower management to make decisions  about how to best handle risks.
Although MDA presented APUC unit costs in its 2009 BAR, it only reported  on six of the ten APUCs presented in last year’s BAR. In addition, it did not  present variances for any of these unit costs or explain why these  variances were not provided. Of these six, MDA could have reported  variances on three according to the agency’s own definition of a  variance—a cost increase of 10 percent or more when compared to the  original resource baseline for a BMDS product. In the case of the THAAD  fire control, the unit cost increased by more than 20 percent. According to  MDA, the agency did not report on APUC unit costs for four of the assets  presenting APUCs in the 2009 BAR because it considered these programs  to be complete and the 2009 BAR to be a final report. However, one of the  four assets not reporting an APUC in the 2010 BAR is the Aegis BMD SM-3  Block IA missile for which, during fiscal year 2010, the agency delivered an  additional 26 missiles. It is unclear why the agency did not provide an  updated APUC for this asset in the 2010 BAR since it was not complete  before fiscal year 2010. In addition, the 2010 BAR did not explain MDA’s  decision to stop reporting these four assets. MDA officials also stated that  no variances were reported because the agency considers the 2010 BAR  resource baselines to be the first established unit cost baselines. However,  the agency first reported APUC costs in 2009, and the 2009 BAR explicitly  refers to these as “unit cost baselines.” Further, the 2009 BAR states that  the agency would report any variances to these APUC baselines in the  2010 BAR. See table 7 for a comparison of the 2009 BAR APUC unit costs  to the 2010 BAR APUC unit costs and associated unreported variances.
MDA was also unclear and inconsistent in how it reported target unit costs  and did not define or explain these inconsistencies in the BAR. While MDA  reported APUC and PAUC for most of its unit cost baselines, it chose to  instead report average unit costs for its 4 Targets and Countermeasures  program efforts presented in the BAR—short-range, medium-range,  intermediate-range, and intercontinental ballistic missile targets. It also  separately reported the non-recurring costs to develop these targets. There  was no explanation in the BAR as to how the targets’ average unit costs  were constructed or why the targets required a different reporting method  from the rest of the BMDS element assets. MDA officials told us that since  Targets and Countermeasures is procuring targets and is not a  developmental acquisition program, that reporting APUC and PAUC was  not the best method for reporting unit costs for targets. However, MDA is  investing significant amounts to develop the targets it needs. MDA  reported in the 2010 BAR that it is planning to invest $2.4 billion to develop  and procure targets between fiscal years 2010 and 2015.
By reporting only the average unit costs for targets, which officials told us  was the same as the average recurring costs, MDA is not reporting the full  unit costs to develop and procure targets.  In analyzing the information in  the 2010 BAR, we determined that in only 4 of the 12 separate target unit  costs reported did the BAR provide enough information to calculate PAUC  costs. For example, while MDA reported that the new extended medium  range ballistic missile average unit cost was $29 million, if it had included  all reported development costs, it could have reported a PAUC of $60  million for these targets. In addition, the planned new intermediate range  ballistic missile reported an average unit cost of $40 million, but had it  included all reported development costs, it could have reported a PAUC of  $109 million. For 8 of the 12 targets presenting average unit costs,  including the LV-2 intermediate range target, insufficient information was  presented in the BAR to calculate a PAUC.
The 2010 BAR’s life cycle cost baselines also lacked clarity and  consistency in how some costs were accounted for, giving the appearance  that some of the baselines were underestimated. MDA presented both a  time-phased estimate in then-year dollars and a life cycle cost baseline in  base year dollars, without providing any details in the BAR on how these  costs were converted. For example, sunk costs presented in the time- phased estimates were sometimes converted to base year dollars in the life  cycle cost baseline and other times not converted. Four of the life cycle  costs baselines contained sunk costs that were not converted. Not  converting these costs to base year dollars means these costs were  underestimated in the life cycle cost baseline. However, it is not possible  to assess by how much these costs are underestimated since, in some  cases, life cycle cost estimates did not include details on the span of years  included in sunk costs. Without knowing which years are covered or being  able to account for the costs per year, we were unable to verify the  estimates for sunk costs.
section_title: Life Cycle Cost Baselines Are Not Based on High-Quality Cost Estimates
paragraphs: We assessed MDA’s life cycle cost estimates using the GAO Cost  Estimating and Assessment Guide that is based on best practices in cost  estimating and which identifies key criteria for establishing high quality  cost estimates. Our analysis of the 12 life cycle cost estimates that  formed the basis of the 12 life cycle cost baselines in the 2010 BAR found  that only 6 of these estimates matched the costs that were reported in the  BAR. The 6 estimates that did not match the BAR baselines—Aegis  Ashore, Aegis BMD SM-3 Block IB missile and associated software  upgrades for shipsets, C2BMC Spiral 6.4, GMD, SBX capability deliveries,  and current THAAD capability configuration—were therefore excluded  from further analysis. Of the remaining 6 life cycle cost estimates, none  fully met the 4 characteristics of high-quality cost estimates as established  by the GAO Cost Estimating and Assessment Guide.
According to the GAO Cost Estimating and Assessment Guide, the cost  estimate is part of a total systems analysis and is a critical element in any  acquisition process to help decision makers evaluate resource  requirements at milestones and other important decision points. Cost  estimates establish and defend budgets and drive affordability analysis.  The guide identifies 12 steps necessary for developing a high-quality cost  estimate. According to the guide, documentation is a pervasive  requirement throughout the 12 steps needed to establish a reliable cost  estimate. The detailed documentation called for in the guide addresses the  purpose of the estimate, the program background and system description,  its schedule, the scope of the estimate (in terms of time and what is and is  not included), the ground rules and assumptions, all data sources,  estimating methodology and rationale, the results of the risk analysis, and  a conclusion about whether the cost estimate is reasonable. A cost  estimate is much more than just a single number. It is a compilation of  many lower-level cost element estimates that span several years, based on  the program schedule. Therefore, a good cost estimate—while taking the  form of a single number—is supported by detailed documentation that  describes how it was derived and how the expected funding will be spent  in order to achieve a given objective. This documentation should include  enough detail that a cost analyst unfamiliar with the program could  recreate it quickly with the same result.
Our research has identified a number of best practices that are the basis of  effective program cost estimating and should result in reliable and valid  cost estimates that management and key decision makers can use for  making informed decisions. These four characteristics of a high-quality  and reliable cost estimate are comprehensive, well documented, accurate,  and credible. A cost estimate is considered:    Comprehensive: when it accounts for all possible costs associated with  a project, details all cost-influencing ground rules and assumptions, is  technically reasonable, is structured in sufficient detail to ensure that  costs are neither omitted nor double-counted, and the estimating  teams’ composition is commensurate with the assignment;    Well-documented: when supporting documentation for the estimate is  accompanied by a narrative explaining the process, sources, and  methods used to create the estimate and contains the underlying data  used to develop the estimate;    Accurate: when the estimate is not overly conservative or too  optimistic and is based on an assessment of the costs most likely to be  incurred; and    Credible: when the estimate has been cross-checked with independent  cost estimates, the level of confidence associated with the point  estimate has been identified, and a sensitivity analysis has been  conducted—that is, the project has examined the effect of changing  one assumption related to each project activity while holding all other  variables constant in order to identify which variable most affects the  cost estimate.
Although MDA cost officials stated that MDA followed the GAO Cost  Estimating and Assessment Guide for building cost estimates and  baselines, the documentation that MDA provided for the 6 life cycle cost  estimates contained insufficient evidence to meet the characteristics of a  high-quality cost estimate. In our analysis, we used ratings to describe how  much of the criteria the documentation met toward being comprehensive,  well-documented, accurate, and credible. The ratings we used were as  follows: “Met” means that the program provided documentation that  satisfied the criterion; “Substantially met” means that the program  provided the majority of the documentation to satisfy the criterion;  “Partially met” means that the program provided documentation satisfying  part of the criterion; “Minimally met” means that the program provided  documentation satisfying a minor part of the criterion; and “Not met”  means that the program did not provide documentation that satisfied the  criterion. See figure 2 for the results of our review of the 6 estimates.
The 6 estimates were not fully comprehensive because they did not  provide enough detail to enable us to determine if all government and  contractor costs had been included, and it was unclear if the cost estimate  was derived from a detailed work breakdown structure allowing us to  check if all of the life cycle costs were fully accounted for. In addition, 4 of  the estimates did not include the historical data necessary to back up  given assumptions. The estimates were not well documented in 5 cases  because the estimates lacked documentation necessary to be able to  recreate the estimates, and further did not include an electronic cost  model to provide the information necessary to understand how each  estimate was created. The remaining estimate was not considered well  documented because, in some instances, it did not provide enough  information on how cost factors were derived, and cited methodology but  did not include the backup analysis to support it. None of the estimates  met the standards for accuracy. For example, four of the estimates did not  provide a confidence level in the estimate making it impossible to  determine if the estimate was unbiased and based on most likely costs, did  not provide access to the detailed calculations used in the estimate, or did  not provide the historical data to allow us to check the reliability of the  sources or how data is verified for accuracy. The other two cost estimates  lacked accuracy because discrepancies existed on costs among several of  the documents provided or the costs did not match the summary estimates  reported in the BAR. Finally, costs associated with the estimates lacked  credibility because none of the six estimates performed a cost sensitivity  analysis, provided evidence that cost driver cross checks were performed,  or completed independent cost estimates. In addition, MDA performed a  cost and risk uncertainty analysis on only one of the six estimates, without  which the estimate lacks the level of confidence associated with achieving  the cost estimate.
As discussed above, one of the criteria for a credible cost estimate is  having an independent cost assessment. Other DOD major defense  acquisition programs are required to perform an independent cost  estimate before advancing through major milestones. Because of the  flexibilities granted in 2002, MDA is not required to obtain independent  cost estimates for its programs, and has not otherwise required their  programs to do so. The Office of the Director for Cost Assessment and  Program Evaluation (CAPE) has performed independent cost estimates for  the Aegis BMD program as well as portions of other MDA programs in the  past including the STSS and GMD programs. More recently, the CAPE  performed an independent cost estimate of THAAD procurement costs.  MDA has committed to work with the CAPE to develop cost estimates for  other MDA elements in the future. MDA’s plans to work with the CAPE to  develop cost estimates for more MDA elements will contribute to the  credibility of its estimates.
section_title: Appendix III: Aegis Ballistic Missile Defense (Aegis BMD) with Standard Missile-3 Interceptor
paragraphs: Activities in fiscal year 2010—including the new European  PAA—brought new challenges to the Aegis BMD portfolio.
MDA restructured Aegis BMD into a program executive office  budget request, Aegis BMD is poised to be MDA’s largest  single investment in a BMDS element. with four programs, including Aegis BMD 4.0.1/SM-3 IB, Aegis  Ashore, and Aegis BMD 5.1/ SM-3 IIA.
MDA requested $2.16 billion for fiscal year 2011, an 18  demonstrate performance of Aegis BMD 3.6.1 with SM-3  Block IA for European PAA Phase I. percent increase from last year’s request.
MDA added BMD 3.6.1 capability to two ships. DOD  in development of the Throttleable Divert Attitude Control  System (TDACS), an interceptor component. established Aegis Ashore, which will be based on Aegis 5.0  and the SM-3 IB interceptor.
MDA held several program reviews for Aegis BMD 4.0.1,  software and is a key dependency of the new Aegis Ashore  program.
subsections: section_title: Progress of Aegis 4.0.1 with SM-3 Block IB Limited by Delay in Development of TDACS, an Interceptor Component
paragraphs: Aegis 4.0.1 with SM-3 IB has executed more slowly than expected in fiscal  year 2010. At the last execution review of the fiscal year, progress toward  verifying the SM-3 IB engagement capability required action, and 6 of the  14 development phase exit criteria tracking program execution were  assessed as not on track, including those related to requirements,  affordability, design, manufacturing, and weapon system safety. The  slower pace primarily reflects delays in SM-3 IB interceptor development.
Aegis 4.0.1, with the SM-3 IB interceptor, is expected to have increased  discrimination, engagement coordination, threat missile range capability  and raid capacity. Technology development of the interceptor’s  Throttleable Divert Attitude Control System (TDACS) is following a high- risk path due to continuing resolution of issues discovered during sub- assembly hazard testing. The TDACS issues relate to the operational  suitability and expected lifetime of the interceptor. The first intercept  flight test, FTM-16, was moved into the third quarter fiscal year 2011 to  allow time to investigate and resolve the issues. FTM-16 is critical to  demonstrating the interceptor performance, as well as being required to  certify the Aegis combat system. According to the Director, MDA, the  flight test was rescheduled to allow time to complete qualification tests.  Design verification and qualification tests validate component  performance, reliability and producibility. MDA has since determined the  root cause of the TDACS problem and identified two design  improvements. However, due to continuing delays redesigned TDACS  components will not be included in the interceptor manufactured for FTM- 16; instead, it will be manufactured using a new process control to  partially mitigate the issue. Further, MDA has since decided to conduct the  flight test in the fourth quarter of fiscal year 2011 without completing the  qualification tests as originally planned. The TDACS qualifications and  verifications leading up to the FTM-16 intercept test will be limited to the  environmental conditions expected during the FTM-16 event. Following  FTM-16, the design changes to the TDACS will require the full set of design  and qualification testing.
We reported in February 2010 that planned interceptor production would  precede knowledge of interceptor performance, and recommended that  MDA delay a decision to produce interceptors to follow successful  completion of developmental testing, a flight test, and manufacturing  readiness review. We reported again in December that the SM-3 IB test  schedule was not synchronized with planned production and financial  commitments. This schedule had become even more compressed as a  result of TDACS redesign, and planned requalification. As a result, MDA  recently deferred planned interceptor production decisions to follow  redesign efforts, the manufacturing readiness review, and an additional  flight test; steps that could better inform those production decisions.
While MDA characterized the first 30 interceptors as being test rounds,  half remain unassigned to a specific test. Furthermore, of those  interceptors assigned to a test, some may be produced earlier than  necessary since they deliver 1 to 2 years prior to the scheduled test.  Program officials note the unassigned “test” rounds will provide  information on reliability, maintainability, and supportability, and verify  cost estimates and production processes.
section_title: Aegis BMD 5.0 Features Highly Integrated Hardware and Software and Is a Key Dependency of the New Aegis Ashore Program
paragraphs: Aegis BMD 5.0 will not provide new mission capability; instead it will  leverage the Navy’s Aegis modernization effort, which transitions the  cruisers’ and destroyers’ computers and displays from military standard to  commercial-off-the-shelf components.  The modernization effort will  increase the number of cruisers and destroyers that have the potential to  be BMD capable from 27 to 84, and the installation of Aegis 5.0 in  conjunction with the modernization will add the BMD capability. Once  Aegis 5.0 is available, Aegis ships with version 3.6.1 may be upgraded  directly to 5.0, instead of undergoing an interim installation of 4.0.1  (estimated at $52 million per ship). Although it does not add new mission  capability, the migration into an open architecture environment requires  significant modification and testing of 8 of 10 major components of the  Aegis weapons system. Execution reviews already show signs of schedule  compression and interdependencies of multiple efforts increase schedule  risk. As the European PAA’s new Aegis Ashore program is highly  dependent on the scheduled delivery of Aegis 5.0, delays could have  significant consequences for providing European BMD capability as  planned (see appendix IV).
section_title: Appendix IV: Aegis Ashore with Standard Missile-3 Block IB (European PAA Phase II)
paragraphs: Appendix IV: A Missile-3 Block IB (European PAA Phase II)
MDA established the Aegis Ashore program office in early    Aegis Ashore was announced in September 2009 as part of  the European PAA.
MDA held developmental baseline and baseline execution    Technical analysis supporting the selection of Aegis Ashore  for the European PAA is unclear.
MDA prepared two program cost estimates; although the  planned, independent cost estimate was not completed.   Highly concurrent acquisition activity results in significant cost,  schedule, and performance risk.
European Phased Adaptive Approach (Romania in 2015 and  Poland in 2018). components require modification for a land-based  configuration.
MDA awarded the contract to design new Aegis Ashore    Deckhouse and vertical launching system (VLS) enclosures  system components, including deckhouse and enclosures for  component systems.
DOD began expansion of the Pacific Missile Range Facility to  production commitments. support testing and development for the European PAA.
Background and Overview  Aegis Ashore is MDA’s future land-based variant of the ship-based Aegis  BMD. It is expected to track and intercept ballistic missiles in their  midcourse phase of flight using Standard Missile-3 (SM-3) interceptor  variants as they come available. Key components include the Aegis SPY-1  radar, command and control system, vertical launching system (VLS), SM- 3 missiles and removable enclosures for the systems to facilitate  worldwide deployment. DOD expects to deploy the first Aegis Ashore with  the SM-3 block IB in fiscal year 2015 as part of the European Phased  Adaptive Approach (PAA).
subsections: section_title: Technical Analysis Supporting Selection of Aegis Ashore for European PAA Is Unclear
paragraphs: According to the Director of MDA, the idea of Aegis Ashore preceded the  new European PAA policy. Earlier in 2008 and 2009 MDA had been  studying alternatives to the Arrow 3 program—a joint US-Israeli program  designed for Israeli self-defense against short-range ballistic missiles.   MDA’s analysis had considered several land-based SM-3 options, and had  concluded that an Aegis with SM-3 was the preferred option. It is unclear  how assumptions and analysis related to Arrow-3 supported—and what if  any additional technical analysis was conducted to support—the selection  of Aegis Ashore for the European PAA.
While MDA does not yet follow DOD’s standard acquisition processes, a  robust consideration of alternatives is a key first step in that process and  is intended to assess the operational effectiveness, costs, and risks of  alternative weapon system solutions for addressing a validated warfighting  need. We reported in September 2009 that DOD often did not effectively  consider a broad range of alternatives for addressing a warfighting need or  assess technical and other risks associated with each alternative. Without  a sufficient comparison of alternatives and focus on technical and other  risks, reviews of alternatives may identify solutions that are not feasible  and decision makers may approve programs based on limited knowledge.  While many factors can affect cost and schedule outcomes, we found that  programs that had a limited assessment of alternatives tended to have  poorer outcomes than those that had more robust assessments. The Office  of Cost Assessment and Program Evaluation is studying alternatives to  Aegis Ashore.
section_title: Highly Concurrent Acquisition Activity and Development Uncertainties Result in Significant Cost, Schedule and Performance Risk
paragraphs: A knowledge-based acquisition approach is a cumulative process in which  certain knowledge is acquired by key decision points before proceeding. In  other words, demonstrating technology maturity is a prerequisite for  moving forward into system development, during which the focus should  be on design and integration. The President’s announcement of the  European PAA on September 17, 2009 officially began Aegis Ashore, and  MDA placed Aegis Ashore directly into the third acquisition development  phase—product development. MDA officials note that this decision was  due to the understanding that Aegis Ashore would be a modification of  proven Aegis BMD capabilities. Yet Aegis Ashore has not yet completed  some of the activities MDA outlines for its first two development phases  (materiel solutions analysis and technology development), and is  executing others concurrently with activities in the third development  phase (product development). For example, although MDA’s acquisition  oversight process identifies the following top-level tenets for phase review  entry/entrance criteria prior to entering the Product Development phase,  Aegis Ashore had not: obtained an independent cost estimate, prepared a  life cycle cost estimate, demonstrated critical technologies in the  operational environment (land), or ensured interoperability and  integration with the larger BMDS. MDA’s knowledge points—typically  identified during the first MDA acquisition phase—identify information  required to make key decisions (e.g. program funding decisions,  technology selections, capability declarations, program continuation, or  the selection of an alternative course of action) and manage program risk.  MDA’s knowledge points for Aegis Ashore were completely redefined less  than 3 months after being established.
DOD’s commitment to field Aegis Ashore by 2015 has resulted in  significant schedule compression for the program, even as MDA  discovered issues that broadened the scope of development and design,  placing the program at increased risk of cost growth and schedule delay.  According to the Director, MDA, Aegis Ashore development is not a high  risk because it is based on the existing Aegis BMD system. However, while  Aegis BMD has demonstrated performance at sea, these demonstrations  used the currently fielded 3.6.1 version of Aegis BMD with the SM-3 IA  interceptor, not the newer variant of the Aegis operating system and new  interceptor that Aegis Ashore will use. Aegis Ashore is dependent on next- generation versions of Aegis systems—Aegis 4.0.1 and Aegis 5.0—as well  as the new SM-3 IB interceptor, all of which are currently under  development (see appendix III). Moreover, a series of changes are required  to further modify these new variants of Aegis BMD for use on land with  Aegis Ashore. These modifications include changes to the VLS;  suppression or disabling of certain features used at sea; design,  integration, and fabrication of a new deckhouse enclosure for the radar,  and potential changes to the SM-3 IB interceptor. Changes to those  existing Aegis BMD components that will be reused for Aegis Ashore may  reduce their maturity in the context of the new Aegis Ashore program, and  new features will require testing and assessment to demonstrate their  performance. MDA plans to conduct both ground and flight tests prior to  deployment, however these tests will not occur prior to making  production decisions.
section_title: Various Aegis Components Require Modification for a Land-Based Configuration
paragraphs: Aegis Ashore expects to leverage the existing shipboard Aegis Combat  System—comprised of 32 sensors, communications, weapons, and  countermeasures.  However only 11 of these will be reused for Aegis  Ashore; the remaining 21 will need to be suppressed or otherwise disabled,  including the software that accounts for a ship’s pitch and yaw. While  officials note that current land-based testing processes for portions of the  Aegis Combat System involve similar suppression, the Aegis Ashore land- based configuration is unique and must still be demonstrated through  operational testing.
The program office assessed both the SPY-1 radar and the VLS as flight  proven through successful mission conditions, reflecting the assessment  of the radar and VLS currently at sea. However, these systems will operate  on land, and it is unclear whether the radar’s spectrum supportability is  fully understood or accounted for. Also, the VLS will be modified to  address the differing protection, safety, and environmental requirements  of its new land-based environment. Finally, MDA may modify the SM-3  Block IB for Aegis Ashore; the extent of these changes remains unknown.
section_title: Deckhouse and VLS Enclosures Design Not Complete
paragraphs: Both the radar and the VLS will be configured in removable enclosures  that have not yet been designed or tested. Also, Aegis Ashore’s new  deckhouse—instead of being integrated on a ship deck with the VLS and  the ship’s hull, mechanical, and electrical systems—will be configured  apart from the VLS and these will require standalone power generation.  The deckhouse design also requires that it be removable in order to  facilitate shipment. However not all requirements are fully known, and  although neither the deckhouse requirements or design are stable, the  contract for Aegis Ashore deckhouse fabrication and integration was  awarded prior to preliminary or critical design reviews for the overall  Aegis Ashore system. The potential for rework because design starts  before requirements are complete was noted as a continuing risk in the  last program review of the fiscal year, and we have previously reported  that starting fabrication prior to achieving design stability can lead to  costly modifications later in the process due to rework.
section_title: Aegis Ashore Flight Tests Reduced from Previous Plan, Not Timed to Inform Production Commitments
paragraphs: The number of planned developmental flight test events has been reduced  since the Aegis Ashore program began and they are not timed to inform  production decisions. In MDA’s February 2010 test plan Aegis Ashore was  scheduled to participate in 7 developmental flight test events, 5 of which  were intercept events. The current plan is for 4 test events, 2 of which are  intercepts. The first intercept is now scheduled more than a year later than  previously planned (see figure 3).
MDA officials indicate that the current plan is sufficient to collect data on  critical variables and to evaluate weapon system performance in the Aegis  Ashore configuration. We have previously reported that repetition of  intercept-related objectives is important to build confidence in intercept  capability.  MDA plans to make production commitments for the first  operational Aegis Ashore and its interceptors by early fiscal year 2012. The  first intercept flight test with a target is planned for the second half of  fiscal year 2014, at which point the design will have been finalized, the  Aegis Ashore deckhouse and components built, and Aegis Ashore  construction and interceptor production will be well under way.
section_title: Appendix V: Airborne Infrared (ABIR)
paragraphs: borne Infrared (ABIR)
The ABIR program demonstrated the ability of infrared  sensors aboard Unmanned Aerial System (UAS) to observe  ballistic missiles in flight at long distances and improved  sensor performance.   The ABIR acquisition approach focuses on exploring the use  of currently available technology.    MDA delayed fielding ABIR from fiscal year 2015 to fiscal year  2019.
The ABIR program completed alternatives analysis and  selected Multi-Spectral Target System-class infrared sensor  and MQ-9 Reaper ABIR experiments and demonstration  efforts.
The ABIR program partnered with Air Force, Navy, and  combatant commanders to develop risk reduction tests and  preliminary operational concept. intercept capabilities.    MDA’s plan to field ABIR in fiscal year 2019 is inconsistent  with justification for developing the program.
The program participated in five ABIR tests demonstrating  missile-tracking accuracy.
Background and Overview  ABIR is being designed to track enemy ballistic missile raids soon after  launch with airborne sensors forward in the theater in order to enable  early intercept attempts. MDA began exploring a similar concept of an  operational unmanned aerial system (UAS) infrared missile detection  system with the Airborne Infrared System (AIRS) in fiscal year 2002. In  fiscal year 2009, MDA began assessing new platform and sensor  alternatives and decided to conduct ABIR experiments and demonstration  efforts to determine the usefulness of UASs to perform missile defense  missions. In addition, MDA plans to have an operational ABIR program  fielded by fiscal year 2019 as part of Phase III of the Phased Adaptive  Approach (PAA) for the defense of Europe and the United States.
subsections: section_title: The ABIR Acquisition Approach Focuses on Exploring the Use of Currently Available Technology
paragraphs: MDA is following a different acquisition approach for ABIR than it has  with some of its other acquisition development programs, according to  program officials.  Rather than developing a system from the ground up,  MDA is using existing technology to demonstrate the usefulness of UAS  infrared sensors to perform early missile tracking. ABIR experiments and  demonstrations will inform the requirements for an operational system. To  facilitate this process, MDA will utilize experiments and demonstrations to  meet dedicated knowledge points, such as verifying the infrared sensor’s  performance, assessing missile raid size handling, and launching an  interceptor on ABIR sensor tracks. For the experiments and  demonstration efforts, MDA and the Air Force jointly conducted an  analysis of alternatives to select the infrared sensor and decided on the  Multi-Spectral Target System-class sensor, an infrared sensor with built-in  integration to UASs that was used in earlier risk-reduction activities for  ABIR. Program officials stated that MDA collaborated with the Air Force  and determined that the MQ-9 Reaper, a medium-to-high altitude, long  endurance UAS, best fit both MDA’s and the Air Force’s needs. While ABIR  will use the Multi-Spectral Target System-class sensor and the MQ-9  Reaper, according to program officials, the operational system will be  selected through industry competition. To that end, program officials  stated they are developing the prototypes so that they are not necessarily  tied to a specific UAS or sensor platform to enhance future competition.
section_title: MDA Delayed Planned Fielding of ABIR from Fiscal Year 2015 to Fiscal Year 2019
paragraphs: MDA decided to delay fielding ABIR to fiscal year 2019 in order to further  explore the usefulness of forward based airborne sensors to the BMDS,  follow a more knowledge-based acquisition approach, and alleviate  funding constraints. MDA initially planned to deliver six ABIR platforms  by fiscal year 2015 and participate in Phase II of the PAA. However, MDA  decided to delay fielding to fiscal year 2019 and participate in Phase III of  the PAA to allow MDA more time to experiment and demonstrate the  usefulness of forward-based UASs for early intercept missile tracking.  Technical development challenges exist, such as achieving a three  dimensional track of boosting missiles within a tactically significant  timeframe and operationally realistic setting as well as tracking mass raid  sizes. Officials from the Office of the Under Secretary of Acquisition,  Technology and Logistics have stated concerns about the Precision  Tracking Space System’s (PTSS) planned capability to accomplish similar  objectives. MDA plans to conduct experiments and progressively  demonstrate key knowledge points in order to determine the usefulness of  ABIR to the BMDS.
In addition, although DOD has defined an initial operational concept for  ABIR, DOD has identified challenges in determining a concept of  operations for ABIR. DOD’s initial operational concept is based on  persistent coverage of a threat area by ABIR platforms for a limited  duration, also referred to as a surge role. The intent is to maintain non- stop pre-determined orbits during periods of heightened tension, and  thereby provide missile launch coverage. According to MDA officials, MDA  does not plan to acquire UASs specifically for ABIR. Instead, MDA plans  on adding missile-tracking capabilities to pre-existing UASs. The UASs  that serve the ABIR role will be able to perform normal assignments and  tasks when not being used for the BMDS mission. Program officials also  stated that they are exploring the potential of using a detachable sensor  pod or sensor processing “kit” to enhance the flexibility of the ABIR  operational system. However, DOD officials have identified operational  challenges, such as operating UASs in controlled airspaces, developing a  sensor that can interface with multiple UASs, and matching availability of  UASs to meet operational needs of an effective ABIR system.
By allowing more time for ABIR experiments and demonstrations, MDA is  following a more knowledge-based acquisition approach. Program officials  stated that in order to meet timelines in the initial fiscal year 2015  schedule, the program schedule included a lot of concurrency where  production decisions would occur before adequate testing had been  conducted. As a result, MDA decided, in part, to delay fielding ABIR until  fiscal year 2019 to ensure operationally realistic testing occurs before new  capabilities are deployed—a key tenet of the 2010 Ballistic Missile Defense  Review. For years, we have reported that MDA has allowed concurrent  development, testing and fielding of BMDS elements at the expense of cost  and performance. By delaying ABIR fielding to allow time to experiment  and demonstrate planned capabilities, MDA is following a more  knowledge-based acquisition approach.
In addition, MDA recognized budgetary constraints as a challenge to  fielding an operational ABIR system by fiscal year 2015. According to  MDA, accelerating ABIR and integrating the capability into Phase II of the  PAA in fiscal year 2015 would have required significant funding and put  the program on a high-risk path. MDA stated that the current 2018  capability delivery allows more time for sensor technology development,  integration and testing of sensor capabilities on an unmanned aircraft, and  integrating ABIR into the PAA Phase III architecture.
section_title: Delays in Fielding ABIR Prolong Sensor Coverage Gaps for Additional Early Intercept Capabilities
paragraphs: Delays in fielding ABIR prolong sensor coverage gaps to track missiles  early after launch and delay plans for additional early intercept  capabilities. MDA discovered there were sensor coverage gaps in its ability  to acquire and track large ballistic missile raid sizes, intercept ballistic  missiles earlier in its trajectory, assess intercept attempts in real-time, and  launch additional interceptors if necessary. Currently, the sensor systems  of the BMDS consist of radar sensors, such as Sea Based X-band Radar  (SBX) and Army-Navy/Transportable Radar Surveillance (AN/TPY-2).  ABIR is being designed to fill these sensor coverage gaps by tracking large  missile raid sizes soon after launch to enable earlier intercepts.
Although MDA plans to deploy a forward-based AN/TPY-2 radar that can  perform early missile tracking in southern Europe by fiscal year 2011, the  radar has limitations in providing early intercept capabilities. The radar  has limitations in its ability to search for missile launches while  simultaneously tracking missiles already in flight. According to program  officials, one of the primary benefits of ABIR is its ability to free the  AN/TPY-2 radar from its search roles, which substantially increases  AN/TPY-2’s tracking capabilities. Utilizing additional sensors, such as  ABIR and PTSS, will increase capabilities for sensor coverage for early  missile intercepts.
section_title: MDA’s Plan to Field ABIR by Fiscal Year 2019 Is Inconsistent with Justification for Developing the Program
paragraphs: ABIR will not be fielded sooner than PTSS, which is inconsistent with a  major justification for developing the program. MDA has stated that a  major reason for the ABIR program is that it could provide infrared  tracking sooner than fielding a space-based infrared system. Infrared  tracking is a necessary capability, according to MDA, because it can fill  gaps in sensor coverage that currently exist in the BMDS. In 2002, the  Senate Armed Services Committee recognized that MDA had no near-term  plans to provide infrared tracking to missile defense systems. Plans to  develop a space-based infrared system had been delayed so the committee  urged MDA to aggressively pursue an airborne infrared program, the AIRS  program, to fill sensor coverage gaps. According to program officials, MDA  conducted some airborne data collection activities under the AIRS  program but did not test infrared sensors with UASs until fiscal year 2010  under the ABIR program. MDA currently plans to field an operational  ABIR system by fiscal year 2019, near the same time PTSS will be fielded.  Since developing ABIR to fill the sensor coverage gaps until an operational  space-based sensor could be fielded is a major justification for the  program, MDA’s current plan to field both systems at the same time is  inconsistent with that justification. MDA officials noted that while both  ABIR and PTSS are infrared systems, they are not duplicative systems  because they provide different capabilities. For example, ABIR is capable  of acquiring and tracking ballistic missiles earlier than PTSS and has the  flexibility to deploy to multiple combat theaters to help meet combatant  commander needs.
section_title: Appendix VI: Airborne Laser Test Bed (ALTB)
paragraphs: Laser Test Bed (ALTB)
After nearly 14 years of development and more than $5  billion, the airborne laser successfully conducted its first lethal  destruction of a target in February 2010.
A subsequent attempt at lethal demonstration was conducted  in September 2010, but was unsuccessful due to corrupted  beam control software. experiments throughout fiscal year 2010 and into early fiscal  year 2011.    MDA is working with the Director of Defense Research and  Engineering and High Energy Laser Joint Technology Office  to shape the future of the ALTB.
subsections: section_title: Test Bed’s First Lethal Demonstration Was Successful
paragraphs: In February 2010, MDA demonstrated that the ALTB could successfully  destroy a short-range threat-representative ballistic missile during the  boost phase. This test marked the first directed energy intercept  demonstration against a liquid-fueled boosting missile target from an  airborne platform.  The entire engagement occurred within 2 minutes of  the target missile launch, while the target’s rocket motors were still  thrusting.
Less than 1 hour later, a solid-fuel missile was launched, but the system  did not destroy the target because the laser shut down prematurely.  During this engagement, the system detected increased laser light levels,  which prompted the system to abort the test. Historically, these types of  aborts are due to scattering of laser light which could prevent the system  from successfully destroying the target missile.
section_title: Subsequent Flight Tests Were Unsuccessful
paragraphs: In September 2010, after overcoming a number of technical problems  during ground and flight test preparations, MDA attempted to destroy  another target using the ALTB, but failed to destroy the target. The ALTB  successfully detected and tracked the target, but corrupted beam control  software steered the high energy laser slightly off center. The ALTB safety  system detected this shift and shut down the high energy laser.
MDA conducted another flight test in October 2010 against a solid-fuel  missile. However, while the system seems to have successfully acquired  and tracked the plume or rocket exhaust of the target, it never transitioned  to active tracking. As a result, the laser was not fired. The laser incorrectly  reported that it was not ready and the safety default aborted the  engagement. The program’s failure review board determined that the  cause of the failure was a single micro-switch on a valve that incorrectly  reported that the valve was closed. Neither the program’s failure review  board nor the contractor working with the valve and micro-switch  component vendors was able to determine the root cause of the part  failure. The board noted that it would take an extensive test program to  attempt to determine the root cause and/or design a replacement micro- switch to determine or eliminate the root cause. However, it  recommended corrective actions which were implemented by the  program. According to program officials, the valves were replaced and  new software procedures were implemented to ensure that this type of  problem would not cause a system abort in the future.
section_title: The Future of the ALTB
paragraphs: MDA is working with the Director of Defense Research and Engineering  and High Energy Laser Joint Technology Office to shape the future of the  ALTB. The Secretary of Defense directed a joint study of the ALTB that  was to be completed in June 2010. However, as of February 2011, the  report was not available. Agency officials also told us that MDA has  commissioned a Scientific Review Committee and an Executive Steering  Group to develop a science and technology plan for the ALTB. The  committee and steering group, consist of subject matter experts from the  Air Force Research Laboratory, the High Energy Laser Joint Technology  Office, and the Director of Defense Research and Engineering.
The Director of Defense Research and Engineering has also been tasked  with annually assessing the ALTB for high power laser research and  development. MDA will maintain the airborne laser aircraft as a potential  test bed for ground and flight tests to characterize high energy laser  research pending the results of the Director of Defense Research and  Engineering study.
section_title: Appendix VII: BMDS Sensors
paragraphs: MDA completed first version of common software    AN/TPY-2 is a major component of Phases I and II of the  development qualification which increases the flexibility and  interchangeability of the AN/TPY-2 radar for forward-based  and terminal-based modes.
BMDS Sensors participated in ground and flight tests with  other elements, including GMD and THAAD.
Joint Chiefs of Staff made the decision to deploy an AN/TPY- 2 radar to U.S. Central Command in fiscal year 2011.
Transition of Thule Upgraded Early Warning Radar (UEWR)  sustainment to the Air Force was completed in fiscal year  2011. band Radar (SBX) to the Navy in fiscal year 2012.    The SBX program experienced a significant failure during  FTG-06 and attempted to demonstrate a correction in FTG- 06a.    DOD has not yet made a decision to upgrade Cobra Dane, an  important part of the BMDS sensor suite.
subsections: section_title: AN/TPY-2 Is a Major Component in Phases I and II of the Phased Adaptive Approach
paragraphs: MDA currently plans to deploy an AN/TPY-2 (forward-based mode) radar  in southern Europe in 2011 as part of Phase I of the Phased, Adaptive  Approach (PAA). The addition of the radar will significantly increase the  size of the area that can be defended with Aegis BMD. Radar tracks from  AN/TPY-2 will be used to provide early tracking information to enhance  homeland defense assets. MDA will deploy AN/TPY-2 to southern Europe  as part of the PAA to give the BMDS a continuous tracking and  discrimination capability with more opportunities to engage the target,  when coupled with other sensors, resulting in a greater probability for a  successful intercept.
section_title: Although AN/TPY-2 Will Provide Early Intercept Capabilities, Additional Sensors May Be Needed to Enhance Early Intercept Capabilities
paragraphs: Although AN/TPY-2 will provide missile coverage that will provide early  intercept capabilities, the radar also has limitations in its capabilities. The  AN/TPY-2 radar, like most other radars, is susceptible to adverse weather  conditions. In addition, X-band radars can become overwhelmed due to  their high detection sensitivity as exhibited by AN/TPY-2 during a 2009  event. In addition, forward-basing AN/TPY-2 radars come with diplomatic  challenges to develop host nation agreements, have significant operation  and sustainment costs, and need multi-layered force protection. Lastly, the  radar has inherent capability limitations that necessitate the need for  additional sensors.
AN/TPY-2 capability limitations can be mitigated, according to MDA  officials, by utilizing additional infrared sensor systems, such as the future  PTSS and ABIR programs. MDA is developing PTSS, a space-based  infrared sensor satellite constellation, and ABIR, infrared sensors used  aboard unmanned aircraft systems, to detect and track missile launches.  Infrared sensors can help provide early missile launch acquisition and  tracking while avoiding many of the limitations of radars. For example,  according to MDA budget documentation, a small constellation of PTSS  satellites can provide coverage equivalent to 50 AN/TPY-2 radars or 20  SBX radars. MDA plans to field a PTSS constellation by fiscal year 2018  and ABIR system by fiscal year 2019. Any delays in fielding additional  infrared sensors to augment the AN/TPY-2 in southern Europe could  significantly affect the planned capability to defend the United States and  Europe from ballistic missile attack.
section_title: MDA Is on Track to Deliver 13 Operational AN/TPY-2 Radars by Fiscal Year 2017 for European Missile Defense, Forward-Based Missions, and THAAD Deployment
paragraphs: MDA plans to produce 14 radars by fiscal year 2017, the first of which is  developmental, and the other 13 of which will be operational. MDA has  currently delivered 4 AN/TPY-2 radars, all of which were delivered on  schedule. Table 8 describes MDA’s current plans to deliver AN/TPY-2  radars.
MDA recently decided to speed up production of radar #8 to be fielded  with THAAD Battery #3 to help meet warfighter demands. Although DOD  has not determined where THAAD batteries will be deployed, MDA has  stated that future THAAD batteries may be deployed to Europe to defend  critical assets against ballistic missile threats on a contingency basis. In  addition, although the remaining radars to be produced (radars #8-14) are  designated for THAAD batteries, MDA officials stated that if the need for  an additional forward-based AN/TPY-2 arose, these radars may be  redesignated for forward-based use.
section_title: MDA Plans to Hand over Operational Control of SBX to the Navy in Fiscal Year 2012
paragraphs: MDA made progress toward fielding SBX and plans to hand over  operational control of SBX to the Navy in fiscal year 2012. During fiscal  year 2010, MDA completed a critical Navy inspection and certification  necessary to hand over operational control of SBX. Planned operational  areas include positions in the northern, western, and middle Pacific  Ocean. According to MDA, the Missile Defense Executive Board approved  a recommendation from the Navy to utilize a more flexible approach  allowing SBX to port at multiple locations rather than establishing a  dedicated port for SBX. U.S. Strategic Command will exercise combatant  commander authority of SBX and delegate operational control to the Navy  via Pacific Command. Although the Navy will operate SBX, MDA will  maintain the development responsibility for improving radar capabilities.
section_title: The SBX Program Experienced a Significant Failure during FTG-06 and Attempted to Demonstrate a Correction in FTG-06a
paragraphs: During the GMD flight test FTG-06 in January 2010, SBX experienced a  significant failure.  According to program officials, as the primary sensor  for the flight test, SBX’s task was to track the missile target, an  intermediate-range missile, and provide a qualified track on the target to  GMD. During the flight test, SBX initially performed as expected but then  experienced a failure which prevented it from establishing a reportable  track on the target. According to MDA, high sensitivity coupled with a  large number of presented objects could cause the SBX processor to  become overwhelmed as exhibited in FTG-06. MDA also states that  software changes were incorporated into SBX to mitigate this problem.  Currently, computer models cannot replicate the situation SBX  experienced during FTG-06 but can replicate the effect SBX experienced.
To address the failure in FTG-06, program officials developed iterative  software corrections. The first software correction, Spiral 1, has already  been developed by the program office and was tested during four targets  of opportunity. During three of the tests, computer simulators mimicked  the effects experienced during FTG-06 to test the software corrections.  According to program officials, these tests, in addition to other ground  tests, served as risk reduction leading up to a re-test of SBX objectives in  FTG-06a.
MDA tested those software corrections in FTG-06a, which was conducted  in December 2010. This test was deemed a failure by MDA because GMD  was unable to intercept the target. However, the performance of SBX  during the flight test is unknown at this time because the test report for  this event was unavailable during our audit. SBX served as an inline sensor  for the test and was tasked to track the target over the horizon,  discriminate the warhead, and send a missile track of the target to GMD.
section_title: Transition of Thule UEWR Sustainment to the Air Force Was Completed in Fiscal Year 2011
paragraphs: In fiscal year 2011, MDA transitioned Thule to the Air Force after  successfully transitioning sustainment of both Beale and Fylingdales to the  Air Force in 2009. According to program officials, transitioning the UEWRs  is a process during which MDA gradually hands over control of the  UEWRs to the Air Force. The official transfer of the UEWRs will occur in  fiscal year 2012 when all software upgrades are completed. The most  recent software build to add information assurance upgrades that is  currently under development will be deployed in fiscal year 2011.  According to MDA officials, once certified for missile defense, the addition  of Thule UEWR to the BMDS reduces sensor coverage gaps and provides  larger defended areas for homeland defense.
section_title: DOD Has Not Yet Made a Decision to Upgrade Cobra Dane, an Important Part of the BMDS Sensor Suite
paragraphs: MDA and the Director, Operational Test and Evaluation believe that both  Clear and Cobra Dane are vital sensors for GMD. Both radars provide data  to GMD to enable ballistic missile intercepts. However, Cobra Dane will  soon be in need of large-scale hardware upgrades to continue to operate,  according to program officials. Although MDA has upgraded UEWRs in the  past for BMDS functions, those upgrades involved small hardware and  software processing additions to the radars. MDA currently plans to  upgrade Clear to bring it up to the current configuration of the other  UEWRs in order to efficiently manage all the UEWRs. According to an  official in the Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics, a decision to upgrade Cobra Dane will be a  departmental decision, which will be made after consultation with  interagency partners.
DOT&E officials have emphasized the importance of keeping Cobra Dane  a part of the BMDS sensor suite. According to DOT&E officials, Clear will  not provide all necessary sensor coverage to take the place of Cobra Dane.  For GMD to have the necessary sensor coverage, it must rely on both Clear  and SBX to be available. However, since SBX may be unavailable for  missile tracking at times, GMD would also need Cobra Dane in order to  have robust sensor coverage. DOT&E officials stated Cobra Dane is also  an advantageous sensor for GMD because of its unique geographic  location to detect a launch from North Korea.
section_title: Appendix VIII: Command and Control, Battle Management and Communications (C2BMC)
paragraphs: Command and Control, Battle  ment and Communications (C2BMC)
The C2BMC program demonstrates control of multiple  AN/TPY-2 radars during ground tests.
C2BMC participates in several elements’ flight tests including  in one region and better communicate and manage BMDS  capabilities.
THAAD, GMD, and BMDS Sensors.   Spiral 6.4 has experienced multiple delays and adjustments to  content since fiscal year 2008.    C2BMC completed all ground testing in fiscal year 2010 to  make progress toward declaring Spiral 6.4 operational    C2BMC plans for new server at U.S. Central Command in  fiscal year 2011.
subsections: section_title: Spiral 6.4 Adds Capability to Control More Than One Radar in One Region and Better Communicate and Manage BMDS Capabilities
paragraphs: The next upgrade is Spiral 6.4, which is planned for operational use in the  third quarter of fiscal year 2011. Spiral 6.4 is the first release of upgraded  software since Spiral 6.2 became operational in December 2007. Spiral 6.4  offers several upgrades to Spiral 6.2 including the Global Engagement  Management, which will add the capability to manage multiple AN/TPY-2  radars in a region to acquire and discriminate threat tracks, then forward  the best track to BMDS weapon system fire controls. Spiral 6.4 will also  improve C2BMC’s capability to send situational awareness data to  European Command in addition to the tri-node server which includes U.S.  Strategic Command, Northern Command, and Pacific Command. The  program intends to declare Spiral 6.4 operational for the tri-node in the  third quarter of fiscal year 2011. Concurrent with Spiral 6.4 fielding, the  program is preparing for the release of its next planned spiral capability  release, Spiral 8.2, in fiscal year 2015.
section_title: Spiral 6.4 Has Experienced Multiple Delays and Adjustments to Content since Fiscal Year 2008
paragraphs: The plan for Spiral 6.4 has changed significantly from when MDA expected  to release a single configuration for the tri-node in fiscal year 2008. Since  then, MDA has modified its releases to break-up Spiral 6.4 into smaller  increments of capabilities and for other combatant commands. During  fiscal year 2009, the program was replanned to accelerate capabilities from  Spiral 8.2 to Spiral 6.4 adding the capability for C2BMC to manage multiple  radars and to enhance the BMDS’ ability to perform early intercept. The  following year, in February 2010, the program decided to release its Spiral  6.4 multi-radar configuration in two separate releases—one for the tri- node and then later for European Command. MDA added European  Command to address requirements for the European phased adaptive  approach. In addition, the program intended to later deliver a separate  upgrade to enhance the BMDS’ ability to perform early intercept to both  the tri-node and European Command as well. Just 4 months later, MDA  decided to delay this early intercept capability until Spiral 8.2.
The C2BMC has experienced multiple delays in declaring Spiral 6.4  operational. It was originally planned to go operational in fiscal year 2008.  However, in 2008, the program delayed it to the first quarter of fiscal year  2010 due to technical issues in both the current operational capability,  Spiral 6.2, and the developing capability, Spiral 6.4. These delays were  necessary because throughout fiscal year 2008, Spiral 6.2 required more  fixes than anticipated for warfighter-identified deficiencies. In order to fix  the deficiencies the contractor delayed Spiral 6.4 work so that it could  apply those resources to the Spiral 6.2 problems. Specifically, experience  with testing the AN/TPY-2 with Spiral 6.2 identified the need to improve  radar track processing in Spiral 6.4. This, along with changes that needed  to be incorporated to meet new information assurance requirements  pushed Spiral 6.4 operational fielding to the first quarter of fiscal year  2010. Then during fiscal year 2009, the program added another year of  development, delaying Spiral 6.4 to the first quarter of fiscal year 2011  when it accelerated Spiral 8.2’s capabilities to control multiple radars,  support early intercept, and support deployment of Spiral 6.2 capability to  European Command. In June 2010, the program again delayed the tri-node  server release of Spiral 6.4 another 6 months to the third quarter of fiscal  year 2011 because changes to the BMDS-level testing plan delayed the  ground testing necessary to declare the spiral operational.
section_title: C2BMC Completed All Ground Testing in Fiscal Year 2010 to Make Progress toward Declaring Spiral 6.4 Operational
paragraphs: During fiscal year 2010, the C2BMC program was able to successfully  complete ground testing as part of the series needed to declare Spiral 6.4  operational. MDA completed its first test of the initial C2BMC Global  Engagement Management capabilities during the second quarter of fiscal  year 2010. It was the first in a series of ground tests necessary to declare  Spiral 6.4 operational. This test successfully demonstrated the capability  of Global Engagement Manager to manage operations of multiple AN/TPY- 2 radars. This was the first major system-level event using Spiral 6.4.
In addition, MDA completed a second ground test in the series during the  fourth quarter of fiscal year 2010. This integrated ground test  demonstrated system track forwarding between C2BMC and Aegis BMD as  well as supported C2BMC element-level verification. It also tested mission  functionality with GMD, AN/TPY-2, THAAD, Aegis, and Patriot. Full  analysis results of this testing are expected in fiscal year 2011.
section_title: C2BMC Plans for New Server at Central Command in Fiscal Year 2011
paragraphs: During fiscal year 2010, MDA made the decision to install an operational  Spiral 6.2 server at Central Command during the second quarter of fiscal  year 2011. Program officials told us that this capability was urgently  needed in Central Command, so the agency made Spiral 6.2 operational  immediately rather than waiting until Spiral 6.4 was available.
section_title: Appendix IX: Ground-based Midcourse Defense (GMD)
paragraphs: Appendix IX:  Defense (GMD)
The program completed the planned fielding of 30 GBIs.   GMD’s expected service life is until 2032.
The GMD element continued the manufacturing and delivery  of the enhanced version of the kill vehicle known as the  Capability Enhancement II (CE-II) EKVs.   To date, DOD has spent over $35 billion on this system and is  planning on spending approximately another $6 billion over  the future years defense plan.
The program failed to successfully intercept a target during its  only intercept test in fiscal year 2010. This was the first test  designed to assess the capability of the enhanced CE-II  variant.
In response to the President’s 2002 directive to deploy an  initial set of missile defense capabilities by 2004, MDA  accelerated GMD developmental activities and emplaced five  Ground-based interceptors (GBIs) by 2004.
The GMD element conducted a non-intercept flight test of its  two-stage GBI, which was originally designed for a European  site.   To date, GMD has fielded 30 GBIs: 20 with the original kill  vehicle known as the Capability Enhancement I (CE-I) and 10  with the enhanced kill vehicle known as the Capability  Enhancement II (CE-II).    GMD has only successfully conducted 3 intercepts with the  operationally configured GBI.    The CE-II EKV has failed in both flight test attempts to  intercept a target.    GMD is expected to award a new development and  sustainment contract in fiscal year 2011, expected value of  $600 million per year, not to exceed 10 years.
subsections: section_title: 2010 Congressionally Mandated Reports
paragraphs: The National Defense Authorization Act for Fiscal Year 2010 mandated  the Secretary of Defense to (1) conduct an assessment of the Ground- based Midcourse Defense element and future options for GMD and (2)  establish a plan for the GMD element to cover the period of the future  years defense program. Additionally, the act mandated the Comptroller  General to review the assessment and plan and provide the results to  Congress. We briefed the Congress in October 2010 and agreed to include  the acquisition-related findings in this report. Those findings are included  below.
The two DOD reports were issued in May 2010 and covered numerous  topics including, but not limited to the following key areas:    The capabilities as of the date of the assessment and any planned  capabilities.
The plan to maintain the operational effectiveness of the GMD element  over the course of its service live, including sustainment efforts.
The plan for flight testing the GMD element, including aging and  surveillance tests to demonstrate the continuing effectiveness for the  system over its service life.
The number of GBI missiles necessary for operational assets, test  assets, and spare missiles.
The following sections reflect GAO’s findings on each of these topics.
section_title: GMD Fielded a Limited Defensive Capability in 2004
paragraphs: The integrated capabilities of the GMD element are expressed as  performance baselines with associated performance metrics reported in  the BAR; however, DOD lacks the information needed to validate them.  DOD established performance metrics such as the Probability of  Engagement Success, Launch Area Denied, and Defended Area. To be  credible, these metrics must to be informed by models that are anchored  by flight and ground test data, but test delays and failures have provided  less data than is needed for evaluation. For example, at the end of fiscal  year 2009, the data necessary for validation, verification, and accreditation  of GMD models was only 8 percent complete. Moreover, U. S. Northern  Command officials stated that they do not have confidence in the values  provided by MDA for the official metric of effectiveness (Probability of  Engagement Success) due to the low number of tests and the use of  specification data rather than test data.
Although uncertainties remain about the performance of GMD’s  capabilities, a 2010 annual review of BMDS capability conducted by U.S.  Strategic Command highlighted an increasing confidence in GMD.  Additionally, the Director, Operational Test and Evaluation recently  concluded in its annual assessment that test results suggested GMD  provides a capability against limited, emerging, uncomplicated threats.  However, the lack of sufficient data for comprehensive model and  simulation validation and accreditation continues to preclude a full  evaluation of GMD performance.
section_title: DOD’s Sustainment Program Critical to Ensure Effectiveness of the GMD System through 2032
paragraphs: GMD’s sustainment program is expected to maintain and support an  effective defensive system over the expected 20-year lifecycle, through  2032; however the decision to defer reliability and sustainment efforts  coupled with delays in collecting data has hindered GMD’s efforts to  adequately plan sustainment efforts. Although the program has been  fielding interceptors since 2004, the Director, MDA stated that the decision  to maintain the GMD system until 2032 is based on the fact that the  expected lifespan for the interceptors is 20 years and the last GBI is  planned to be fielded in 2012 resulting in the 2032 timeframe.
The GMD sustainment program is comprised of multiple efforts. Key  among them is the Stockpile Reliability Program Plan, which is intended to  monitor the health of fielded interceptors, identify reliability degraders,  and mitigate obsolescence issues. The Stockpile Reliability Program Plan  was not finalized by GMD until September 30, 2010 and still requires  additional data in order for it to be fully implemented. For example, the  reports states that the “overall approach to accomplishing the GBI  surveillance and stockpile reliability program is to analyze data from GBI  development, sustainment, and flight testing activities throughout the  lifecycle of the deployed interceptor fleet.” However, developmental  testing to get that data will not be completed until at least 2021. GMD  intercept tests conducted to date have already led to major hardware or  software changes to the GBIs—not all of which have been verified through  flight testing.
section_title: Formal Aging and Surveillance Testing Beginning; Data Incomplete
paragraphs: Aging and surveillance data, a key part of the Stockpile Reliability  Program Plan, for a certain limited set of GBI components are being  gathered by GMD contractors and other government organizations  including the Crane Division of the Naval Surface Warfare Center and the  U.S. Army Aviation & Missile Research, Development & Engineering  Center. However, testing will not be completed for many years. Although  Crane has been testing since 2008, a completion date has not been  determined because it is based on the sufficiency of data received from  ongoing component testing and funding. In addition, according to Crane  officials, the Aging and Surveillance Test Program does not cover all  Capability Enhancement-I (CE-I) component parts and there are no  current efforts to test any Capability Enhancement-II (CE-II) specific  components.
Comprehensive aging and surveillance testing is hampered by limited  availability of spare components for the booster and the EKV.  For  example, according to Crane officials, they have had to obtain components  for testing from engineering units or returns. In addition, artificially aged  testing has not been conducted on the Inertial Measurement Unit because  there are insufficient numbers of components available for testing.  However, according to the DOD, natural aging of the inertial measurement  unit is being assessed.
section_title: Valuable, but Limited Data Provided by Current Maintenance Built-in Test
paragraphs: Maintenance Built-in Test (MBIT) testing provides trend data for the  status of the interceptor and determines the interceptor’s health, although  the assessment is primarily pass/fail.  According to program officials,  MBIT testing is conducted quarterly on emplaced interceptors in the silos.  However, an April 2007 report, developed by a team that included the  prime contractor, warned that a lack of improvements to the MBIT may  compromise GMD’s ability to make refurbishment decisions over the  service life of fielded GBIs.
GMD has developed enhanced MBIT testing to provide a better insight into  the health of the GBIs. For example, enhanced MBIT enables program  officials to gather more extensive and higher fidelity telemetry data for  components of the interceptor and develop detailed graphical trends in the  health and status for GBIs. According to contractor officials, enhanced  MBIT would add value to monitoring the GBI fleet.  However, currently  only one silo is equipped with the enhanced MBIT capability and  according to MDA officials, there are no plans to upgrade existing silos or  to incorporate enhanced MBIT capability in new silos being constructed.
section_title: Lack of Spare Component Parts Makes Planning for Refurbishments, Repairs, and Unexpected Failures Difficult
paragraphs: MDA lacks sufficient spare parts for EKV refurbishments and repairs.  According to MDA officials, besides the initial refurbishment kits that  were on contract, there were no other spare component parts ordered for  CE-I and CE-II EKVs. Consequently, there are no spare components for  CE-I refurbishments and only minimal spares for the CE-II EKVs.  Certain  EKV components are specifically fitted for either the CE-I or CE-II  configuration, therefore these components are not interchangeable. The  lack of spare components could lead to difficult choices if components  that are no longer available fail. For example, contractor officials stated  that EKVs might need to be used as a source for parts resulting in a  reduction in available inventory.
MDA plans to purchase a specific amount of kits for scheduled  refurbishments, but GMD officials stated that the amount of additional  spare parts needed for unexpected failures and repairs has not been  determined.  It is planned that as part of the development and sustainment  contract, which MDA expects to award in fiscal year 2011, the contractor  will define repair and spare hardware requirements for each GBI.  However, it is unclear when that will occur and at what cost.
Lower-tier GBI suppliers began completing delivery followed by a break in  production starting in 2007. All remaining third- and fourth-tier GBI  suppliers are expected to complete their deliveries in fiscal year 2010 with  two exceptions—Aerojet and Rockwell Collins. Future GBI  procurements—five additional GBIs plus additional hardware to support  GBI scheduled maintenance activities—will require funding to restart  manufacturing lines. However, the full cost to restart and requalify  vendors is unknown. Although estimates for fiscal year 2010 and 2011  funding were provided, MDA has not yet developed a comprehensive cost  estimate for restart and requalification activities for all vendors involved in  GBI production beyond fiscal year 2011. Under the current plan, any  additional GBI purchases beyond that year will likely incur redesign and  development costs due to parts obsolescence and GBI purchases after  fiscal year 2013 will incur manufacturing line restart costs for third- and  fourth-tier GBI suppliers.
section_title: Reliability, Availability, and Maintainability Is Key for Developing and Fielding Weapon Systems
paragraphs: Reliability, Availability, and Maintainability (RAM) analysis provides the  probability that an item will perform its required function under stated  conditions for a specified period of time, measures failure rates, sets  corrective and preventative maintenance requirements, and determines  maintenance conditions and procedures. According to DOD’s Guide for  Achieving Reliability, Availability, and Maintainability, during system  development, the most important RAM activity is to identify potential  failure mechanisms and to make design changes to remove them.  Achieving specified levels of RAM for a system is important for many  reasons; specifically because of the effect RAM has on readiness, system  safety, and mission success. Additionally, RAM problems slow the  development and fielding of systems, drive up total cost, and degrade  operational readiness and mission accomplishment. However, according  to a June 2010 MDA briefing on GMD’s RAM program, RAM was not  designed into components nor were complete data collected to inform  RAM prior to 2007. Additionally, a comprehensive RAM program was not  developed until 2008.
section_title: Difficulty in Conducting GMD Flight Testing Delays Planned Knowledge
paragraphs: Although MDA altered the GMD flight test tempo to one intercept per year,  it is unclear if it can still be executed successfully. Since 2004, we have  continually reported that GMD has experienced difficulty in conducting its  annual flight test plan, a difficulty that continued in 2010 as well.   According to MDA’s fiscal year 2010 budget request, the GMD program  expected to conduct two tests utilizing its two-stage booster in fiscal year  2010. However, the program modified this plan to include one intercept  test (FTG-06) that utilized the enhanced kill vehicle known as the CE-II  EKV and one booster verification test for the two-stage design. As we  previously reported, FTG-06 was important because it was planned as the  first intercept test of this version of the EKV, and because it was  designed to demonstrate a long-flight time for the GBI and GMD’s  capability against countermeasures. MDA officials told us that they will  not add the CE-II EKV to the operational baseline until after the successful  completion of an intercept test. One such test, FTG-06, was conducted in  January 2010, but was unsuccessful: the CE-II EKV failed to achieve an  intercept of the target. MDA chartered a failure investigation team to  investigate and determine the root cause of the failure. In light of the  failure, the GMD program attempted to conduct a similar test in December  2010—FTG-06a—but that test was also unsuccessful due to a failure wit the CE-II EKV.
The costs to conduct each GMD flight test are in the hundreds of millions.  For example, as we previously reported, MDA projected, prior to FTG-06,  that the test would cost over $236 million while the Defense Contract  Management Agency estimated that the cost was likely to exceed $310  million. These costs were likely understated because they did not include  all of the cost increases of delaying the test first to September 2009, nor  did they include any cost increases of further delaying the test to fiscal  year 2010 or for investigating the failure. Additionally, as noted above  because FTG-06 failed MDA inserted FTG-06a which was structured to  capture the same objectives.  It is therefore likely that the cost to conduct  the FTG-06a test was similarly expensive.
It should be noted that although the program failed to successfully  complete its intercept tests in fiscal year 2010, MDA was able to conduct a  non-intercept flight test of its two-stage GBI (designated BVT-01) in June  2010. Although all flight test objectives were achieved, an EKV anomaly  was experienced that might affect system performance.
GMD testing delays and shortfalls have had three major consequences for  the program. First, delays in validating capability results in a reduce level  of knowledge needed to inform GMD models. Second the production and  fielding of the CE-II EKVs has gotten ahead of testing. Currently, even  though the CE-II has failed both intercept attempts, 12 of the 23 CE-II GBIs  have already been delivered. Additionally, according to the March 2010  plan, all CE-IIs under contract are expected to be delivered prior to fully  validating its capability. Third, GMD’s current flight test plan assumes  success and any flight test failures could require difficult decisions, such  as a reduction in planned developmental flight tests or the use of an  interceptor designated as an operational spares or for stockpile reliability  testing. For example, FTG-09, a salvo test planned for fiscal year 2011, was  canceled in order to re-conduct FTG-06. Although the flight test objectives  were moved to FTG-06a and FTG-08, this removes an opportunity to  collect performance data needed to develop GMD models and to assess  the capability of the EKV. Additionally, as we previously reported, the  repetition of intercept-related objectives is important to build confidence  in the intercept capability. Consequently, the delays in validating  capability results in a reduced level of knowledge needed to inform the  GMD program leaving a significant amount of data still needing to be  gathered to fully assess GMD capability. For example, the CE-I and CE-II  capability against countermeasures still is not validated, although MDA  has been attempting to obtain it since 2008 and EKV performance against  its main threat class—an intercontinental ballistic missile—will not occur  until at least 2014.
section_title: Planned Inventory of 52 GBIs Needed through 2032 Lacks Analysis
section_title: Appendix X: Precision Tracking Space System (PTSS)
paragraphs: ppendix X: Precision Tracking Space System  A (PTS  S)
MDA conducted architecture studies and system engineering,    Many aspects of the PTSS acquisition approach are  including developing system concepts and determining  capability needs. consistent with GAO-recommended best practices.    However, MDA has developed an optimistic schedule to field  an operational constellation by fiscal year 2018.    MDA faces significant design challenges in developing PTSS  but has mitigation plans in place to address them.    Delays in fielding PTSS would significantly affect U.S. and  European missile defense.
subsections: section_title: Many Aspects of PTSS’s Acquisition Approach Are Consistent with GAO Best Practices
paragraphs: In alignment with GAO best practices, MDA plans to build two prototype  satellites to define the system performance and focus on cost-effective  production in an industrial environment. The prototype design and  operation will be heavily informed by the on-orbit STSS demonstration  satellites. Past GAO work identified Department of Defense (DOD)  programs that did not regularly test production-representative prototypes  before committing to production which led to cost growth and schedule  delays. We have recommended that DOD develop fully-capable prototypes  to demonstrate that the system can be built efficiently and production and  postproduction costs are minimized.
Appendix X: Precision Tracking Space System  (PTSS) used in commercial satellite sensors without significantly altering form, fit  or function. In addition, the Director, MDA stated that a key hallmark of  the PTSS satellites will be their relatively small size and simplistic design.  For example, rather than including a dedicated acquisition sensor to  identify ballistic missile launches, PTSS will rely on other existing DOD  infrared sensors to establish a ballistic missile launch, thus removing the  need for an acquisition sensor and simplifying the overall design of the  satellite.
MDA also plans to follow an incremental path toward meeting user needs  by using currently available technology to deliver near-term capabilities  while maintaining the flexibility to add capabilities later. For example,  combatant commanders identified a need for global coverage of missile  threats to the homeland. To address the need, PTSS will initially be able to  track regional medium- and intermediate-range ballistic missile threats,  with the potential to handle future intercontinental ballistic missile threats  to the United States. MDA officials stated that they also have the flexibility  to increase the total constellation size by building and emplacing  additional satellites to further add to system capabilities, such as  increasing raid handling size. MDA plans to confine PTSS’s program  objectives to track missile launches for the BMDS and exclude additional  program objectives to ensure that the design remains relatively simple. For  example, the program currently plans to exclude observation of space  objects, also known as space situational awareness, as a program  objective as it could complicate the design and scope of the program. MDA  officials stated that PTSS could potentially serve other functions, such as  space situational awareness, during times when the satellite is not actively  being used to serve the BMDS.
section_title: MDA Has Developed an Optimistic PTSS Schedule
paragraphs: Appendix X: Precision Tracking Space System  (PTSS)
System (STSS), a program that will heavily inform PTSS. After years of  research and development efforts by the Air Force, MDA began  refurbishing and developing two demonstration STSS satellites in 2002 and  launched the satellites in fiscal year 2009, totaling about 8 years. PTSS  intends to develop and launch two prototype satellites in approximately 5  years. MDA stated that it has conducted three studies in recent years and  concluded that using current technologies for the payload design may  enable a rapid PTSS acquisition approach. However, it should be noted  that the program is not yet far enough along in development to determine  whether MDA’s current acquisition plans will enable it to develop and  deploy the operational constellation faster than other DOD space  programs.
section_title: MDA Faces Significant PTSS Design Challenges but Has Mitigation Plans in Place to Address Them
paragraphs: Although MDA plans to follow many GAO-recommended best practices to  use mature technologies, MDA faces significant design challenges.  Officials in the Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics (AT&L) have identified key technical design  challenges to PTSS that will benefit from long lead analysis and  prototyping activities, including payload design and characterization. Key  design risks include developing sufficient sensitivity to detect and track  post boost ballistic missiles, tracking large raids of ballistic missiles, and  mass-producing payloads. According to AT&L officials, the program plans  to address these design challenges through the use of computer modeling  and simulations and refining payload design models. Modeling and  simulation is verified and validated by on-orbit test and activities of STSS.  Although the STSS program has experienced delays in obtaining this data,  MDA officials stated that they are confident that STSS will collect the data  in time to sufficiently inform PTSS.
MDA also faces significant design challenges integrating missile launch  cues from other DOD infrared sensors into the BMDS in order to cue PTSS  to track the detected missile launch. Since missile defense applications  run on a timescale of seconds to minutes, the overriding challenges for  receiving launch cues are obtaining on-demand tasking priority for DOD  infrared sensors and processing the cues on highly compressed early  intercept timelines. For example, to address challenges with integrating  launch cues, MDA is pursuing a developmental initiative dedicated to  incorporating DOD infrared sensor cues within the BMDS. To obtain  priority for DOD infrared sensors, AT&L officials state that the main risk  reduction effort is to develop a joint concept of operations for their use.
Appendix X: Precision Tracking Space System  (PTSS)
MDA also faces challenges integrating C2BMC with PTSS, including  processing multiple PTSS missile tracks with other sensor tracks  simultaneously and filtering out false alarms without significant  processing delays. MDA plans to mitigate challenges by measuring and  planning for processing delays and simulating PTSS and C2BMC  interactions.
In addition, STSS, a program that heavily informs PTSS, has encountered  significant problems operating in a high radiation environment in space.  This environment is similar to the environment PTSS will operate in.  During fiscal year 2010, a radiation particle collided with the STSS  spacecraft processor while program officials were conducting efforts to  fully calibrate the STSS satellites. Although program officials fully restored  the affected processor, the event delayed the STSS satellites reaching full  operating capability. MDA plans to mitigate environmental radiation issues  by incorporating lessons learned from STSS radiation issues into the PTSS  program. For example, PTSS program officials participated in STSS review  meetings to determine root causes and solutions for radiation issues that  arose during satellite calibration. In addition, MDA officials stated that the  radiation environment will be considered as a key factor during the  selection of PTSS satellite parts. PTSS also plans to use newer satellite  processors than STSS that have improved capabilities to shield against  radiation.
section_title: Delays in Fielding PTSS Would Significantly Affect U.S. and European Missile Defense
paragraphs: Appendix X: Precision Tracking Space System  (PTSS)
AN/TPY-2 radar to southern Europe for Phase I of the PAA for the defense  of Europe and the United States. Although the AN/TPY-2 radar can  perform early missile tracking, the radar has inherent capability  limitations. It is therefore necessary to have additional sensors in order to  enhance early launch missile tracking capabilities. Delays in the PTSS  programs would prolong robust early intercept capabilities, resulting in  increased vulnerability to ballistic missile attacks.
section_title: Appendix XI: Space Tracking and Surveillance System (STSS)
paragraphs: XI: Space Tracking and Surveillance STSS)
Demonstration satellites progressed towards full capability    STSS  has  experienced  several  on-orbit  issues  following  launch.
STSS successfully tracked six missile targets.
STSS performed two-dimensional and three-dimensional  acquisition sensor missile tracks.
STSS performed a track of an intercontinental ballistic missile. significant schedule delays.    The program is at risk of not completing all data collections to  inform PTSS.    Despite on-orbit issues, STSS has overcome significant  challenges and has reached full capability performance.
subsections: section_title: MDA’s Decision to Develop PTSS Extended STSS’s On- Orbit Activities 3 Years beyond Its Planned Mission Life
paragraphs: Appendix XI: Space Tracking and Surveillance  System (STSS)
Committee on Appropriations related to a 2010 Defense Appropriation bill  stated that the demonstration satellites were built with payloads that  would demonstrate capabilities required under the former SBIRS-Low  program and that the PTSS constellation MDA is pursuing is very different  from the original SBIRS-Low concept and the STSS demonstration  satellites. The committee stated that while dedicated flight tests for STSS  could provide useful data to MDA, there was sufficient funding in the  budget request to demonstrate the STSS capability for which they were  built.
STSS’s revised on-orbit activities are planned to continue 3 years past the  satellites’ planned mission life. As a result of MDA’s decision to cancel  dedicated flight tests, program officials altered their planned on-orbit  activities for STSS to support PTSS by collecting risk reduction data  during BMDS test flights and targets of opportunity. In fiscal year 2010,  STSS successfully tracked six missile targets over the last year producing  very significant risk reduction data for PTSS, according to program  officials. Although MDA has developed a plan to collect a majority of risk  reduction data by fiscal year 2013, the satellites will not complete the  PTSS risk reduction data collections until fiscal year 2016—3 years after  the satellites’ planned mission life ends.
section_title: STSS has Experienced Several On-Orbit Issues Following Launch
paragraphs: Since launch of the demonstration satellites in September 2009, STSS has  experienced over twenty-two on-orbit issues. For example, shortly after  launch, one of the satellites was unable to autonomously stabilize itself  during launch deployment. The operations team had to manually rotate the  satellite to a safe sun pointing configuration. An anomaly review indicated  that one of the satellite alignment numerical matrices was missing a  “comma,” causing the satellite to fire its stabilization thrusters  inaccurately—a potentially catastrophic failure. According to program  officials, good launch preparation enabled the team to successfully  mitigate the situation. Program officials stated that prior to launch, they  had practiced several contingency circumstances including the exact  scenario which occurred. According to program officials, whereas most  satellite programs only set up a few, if any, mobile command sites, the  STSS program set up multiple mobile command sites to monitor and  control the satellites throughout post-launch operations. Had the team  been unable to control the satellite, program officials said it was possible  that they could have lost the satellite entirely.
Appendix XI: Space Tracking and Surveillance  System (STSS)
The most significant on-orbit issue STSS experienced delayed the program  by over 3 months. Less than 2 weeks after launch, program officials  discovered problems with the Attitude Determination and Control System  when one of the satellites lost lock on its alignment stars. The satellite was  conducting a system bake-out, a 3-week process in which the satellite  heats up so any remaining moisture in the satellite evaporates, when the  error occurred, forcing the satellite to exit the system bake-out and rotate  to a safe-sun pointing position. Program officials uploaded corrective  software updates to the demonstration satellites about 5 months after the  problem was identified. Since the corrective software was uploaded,  program officials stated they have not experienced any additional  problems with the Attitude Determination and Control System.
The satellites have also experienced challenges operating in a high  radiation environment. About 11 months after launch, one of the  demonstration satellite’s processors was struck by an atomic particle  resulting in a temporary memory issue. According to program officials,  they have fully restored the affected processor but anticipate that such  radiation events will happen approximately 4 times per year. Although the  program took measures to protect the satellites from radiation damage,  inherent risks remain while the satellites continue to operate in a high  radiation environment.
Program officials are also investigating the root cause of telemetry  problems with one of the demonstration satellites. While conducting a  payload software upgrade, the satellite experienced an anomaly that  prevented telemetry reporting of the cryocoolers, a system that ensures  the sensors are kept at an appropriately cool temperature range. The lack  of telemetry reporting prevented the sensor from operating, resulting in  over a 2-week delay. Program officials made the decision to use backup  hardware while they investigated the root cause and now believe they  have a better understanding of the problem.
section_title: On-Orbit Issues and Aggressive Scheduling Led to Significant Schedule Delays
paragraphs: Program officials stated that at launch, the program had an aggressive, no  schedule margin plan that was ultimately delayed by approximately 11  months. Following launch of the demonstration satellites, program  officials planned to conduct initial satellite check-out and sensors  calibration to achieve full capability performance of the satellites within 4  months. At that point, the satellites would then begin collecting critical  risk reduction data to inform PTSS. However, due to on-orbit issues and  aggressive scheduling, the satellites did not reach full capability  performance until December 2010, approximately 15 months after launch.
Appendix XI: Space Tracking and Surveillance  System (STSS)
STSS on-orbit issues resulted in significant delays in the program’s  schedule. Half of the 22 on-orbit issues the program has experienced to  date resulted in schedule delays, ultimately delaying the satellites from  reaching full capability performance. Because of on-orbit issues, MDA  revised their test program and test schedule to focus on achieving fully  calibrated performance and participating in interim BMDS-level flight  tests.
section_title: The Program Is at Risk of Not Completing All Data Collections to Inform PTSS
paragraphs: The program is at risk of not completing all data collections before the  satellites’ mission life is over, although the majority of data collection will  occur by fiscal year 2013. The STSS program expects to fulfill all risk  reduction data collection with the conclusion of flight test FTG-17 in fiscal  year 2016, resulting in a 7-year mission life for the demonstration satellites.  The STSS demonstration satellites were designed to have a 4-year mission  life and are therefore at risk of not completing all data collections before  the satellites go inoperable. Program officials anticipate collecting a large  majority of risk reduction data by fiscal year 2013, within the satellites’  planned mission life.
STSS faces challenges coordinating flight tests to be able to collect data  when targets will be able to fly within the satellites’ viewing window.  According to program officials, in order for STSS to view a target during a  flight test, the test has to be coordinated among STSS program officials  and testing officials when the STSS satellites will be in a position to view  the target. However, competing requirements of other participating  elements may require some tests to be conducted without STSS’s  participation. For example, STSS was unable to collect data in flight tests  FTG-06, FTL-01, and FTG-06a because the target did not fly within the  satellite’s view. Delays in collecting full risk reduction data necessary for  PTSS could result if STSS is unable to collect data during planned flight  tests.
Appendix XI: Space Tracking and Surveillance  System (STSS) million in cost overruns, program officials assert that insufficient funding  due to several years of budget cuts may prevent the STSS program from  collecting all the data required for PTSS risk reduction.
section_title: Despite On-Orbit Issues, STSS Has Overcome Significant Challenges and Reached Full Capability Performance
paragraphs: Although the STSS demonstration satellites have encountered several on- orbit issues, the STSS program has minimized the affect of these issues  and reached full capability performance in November 2010. Program  officials stated that both demonstration satellites have healthy hardware  and the remaining issues are understood by the program office. Program  officials stated that the satellites are incrementally improving through  testing and operations and are currently exceeding sensor measurement  requirements as well. Although the satellites were delayed in reaching full  capability performance in fiscal year 2010, that capability has now been  achieved and both satellites routinely participate in BMDS flight tests and  PTSS data collection events. The satellites have been successfully  participating in flight tests and achieved several important  accomplishments including establishing a three-dimensional track of a  ballistic missile, tracking an intercontinental ballistic missile, and tracking  a medium-range missile early after launch through the midcourse phase.  Program officials are confident that on-orbit issues have not disrupted risk  reduction data collection to inform PTSS and that, the majority of  necessary data for PTSS will be collected by fiscal year 2013 according to  the program’s current schedule.
section_title: Appendix XII: Targets and Countermeasures
paragraphs: Thirteen targets delivered and launched.
There was one failure of a target, on the FTT-11 flight test.
There were successful first launches of three new target  types: LV-2, JUNO, and ARAV-C.
Solicitations for medium- and intermediate-range targets were  issued.
Medium-range solicitation was canceled after bids were  received; no contract was awarded.
An undefinitized contract action with the prime contractor for  medium-range targets was issued.
subsections: section_title: MDA Had a Number of Successful Target Flights in Fiscal Year 2010, Including New Targets
paragraphs: In fiscal year 2010, MDA delivered 13 targets. The targets launched during  the year supported tests of several different BMDS elements, including the  THAAD, GMD, and Patriot systems. Three of the targets launched were  new developments.
MDA made important progress in target development during fiscal year  2010, launching three new targets. After a series of delays and cost  increases, the long-range 72-inch FTF target, referred to as the LV-2, was  launched for the first time in January 2010.  According to MDA officials,  the target performed properly, although the interception by the GMD  system was unsuccessful. Two short-range targets, the JUNO and the  ARAV-C, were also successfully flown for the first time, in tests of the  Patriot system and Aegis BMD radar, respectively.
section_title: FTT-11 Failure Caused Delays and Cost Increases
paragraphs: A target failure in the THAAD FTT-11 test resulted in additional costs,  delayed some other launches, and led to changes in MDA’s acquisition  approach for air-launched targets. That short-range air-launched target  was the only unsuccessful launch in fiscal year 2010. The target failed to  initiate after it was dropped from the aircraft, falling into the ocean, and as  a result, FTT-11 was aborted. A subsequent failure review board  investigation identified the rigging of cables to the missile in the aircraft as  the immediate cause of the failure. These cables are attached to the target  while it remains inside the aircraft, and are detached as the target is  dropped from the aircraft. However, the investigation identified  shortcomings in internal processes at the contractor as the underlying  cause.
The agency took several actions in response to the failure and the review  board’s findings. Through the Air Force Space and Missile Systems Center,  the holder of the underlying contract, it issued a cure notice requiring the  contractor to complete specific steps by certain dates. If the contractor  fails to do so, MDA may terminate or descope the delivery orders. In  addition, the government told the contractor that it would halt further  obligations of funding until the first step called for in the cure notice— personnel and organizational changes—was taken. MDA is in the process  of a joint review and "return to flight" process with the Space and Missile  Systems Center. According to MDA, the agency also incurred a total of  $96.4 million in costs because of the failure, including the loss of the target  itself and associated logistics ($38.5 million), the “return to flight” test and  related activities ($29 million), delays to other flight tests ($19.8 million),  and increased mission assurance work ($9.1 million).
section_title: Targets Remained a Source of Delays for BMDS Testing
paragraphs: Targets continued to be a source of delays for MDA’s BMDS testing  program. We have previously reported that problems with the availability  and reliability of targets have caused delays in MDA’s testing of BMDS  elements, and they continued to cause delays in fiscal year 2010.  According to MDA, 8 additional tests that had been planned during fiscal  year 2010 were delayed past the end of the year or canceled. Five of these  delayed tests were a result of the air launch moratorium put in place after  the failure of FTT-11. The remaining 3 delays and cancellations were  caused by factors outside the targets program, such as congressional  action modifying BMDS elements.
In addition, MDA has changed its planned schedule of BMDS tests in  response to target availability. According to MDA officials, target  availability was a driving factor in determining the testing schedule for the  new version 10.2 of its Integrated Master Test Plan (IMTP). In this  revision, the agency delayed 11 tests compared to IMTP 10.1; 5 of those  tests were delayed for more than a year. MDA also accelerated 10 tests, 6  of which were accelerated by one fiscal year quarter. In addition, MDA  stated that the test program has been restructured and the IMTP revised in  support of the PAA.
section_title: Decisions on Acquisition of Targets Made in Fiscal Year 2010
paragraphs: intercontinental-range targets in September 2010, also as part of its 2009  acquisition plan, but it was later canceled as well.
Separately, MDA awarded a new undefinitized contract action to its  incumbent prime targets contractor. This action, signed in April 2010,  asked the prime contractor to build a new type of medium-range air- launched target. The contract action initially included three targets; the  quantity was then increased to five targets in September 2010. MDA  officials stated that this new acquisition was to obtain a second  procurement source for air-launched targets following the FTT-11 failure  and was not in response to the cancellation of the medium-range  solicitation.
According to MDA officials, pursuing new air-launched targets through an  undefinitized action was necessary to meet the test schedule. The  extended use of undefinitized contract actions has previously been  identified by GAO and others as risky for the government. Because  contracting officers normally reimburse contractors for all allowable costs  they incur before definitization, contractors bear less risk and have little  incentive to control costs during this period. The government also risks  incurring unnecessary costs as requirements may change before the  contract is definitized. previously reported that conducting work through the prime contractor  had increased target costs.  Likewise, in comparing the prime contractor  to other options prior to awarding the contract action, MDA estimated that  the prime contractor’s management and fees would add about $9 million  per target compared to the Air Force contract vehicle they had previously  used.  MDA still does not know what the total cost will be for these  targets—as of January 2010, the contract action had remained  undefinitized for over 290 days, despite regulations that require such  contract actions to provide for definitization within 180 days.  The current  not-to-exceed amount for this contract action is $496 million. According to  MDA officials, the delay in definitization is due to changes in its  requirements for the targets, and they anticipate definitization in July 2011,  by which time the contract action will have remained undefinitized for  about 450 days.
section_title: Key Targets Acquisition Decisions in Fiscal Year 2011 Will Shape Program Outcomes
paragraphs: The Targets and Countermeasures program will make several key  acquisition decisions in fiscal year 2011 that will shape outcomes several  years into the future, as well as decisions related to contractor return to  flight activities in the wake of the FTT-11 failure. (See table 10.) year 2011. A goal of this contract is to build an inventory of targets that  can be used to meet testing needs as they arise.
Likewise, MDA may also make a decision on its strategy for procuring  medium-range targets.  Following the cancellation of the solicitation in  2010, MDA officials reported that they are considering options for  procurement of future medium-range targets.  The agency cited both a new  solicitation and the use of existing contracts as options they are  considering.  Use of existing contracts—as MDA has already done for five  targets under the undefinitized contract action with its prime contractor— would reduce planned opportunities for competition and not be consistent  with the acquisition approach the targets program initiated in 2009.
Finally, the return to flight process for MDA’s air-launch contractor will  also take place during fiscal year 2011.  Following the December 2009  failure of FTT-11 and the subsequent failure review board report, MDA  halted flight tests that use the contractor’s systems or components, and  has been working with the firm to address problems. The results of the  “return to flight” test, a target launch that is presently scheduled for the  third quarter of fiscal year 2011, will inform MDA decisions regarding  targets currently on contract with the firm, as well as potential future  contract awards.
section_title: Appendix XIII: Terminal High Altitude Area Defense (THAAD)
paragraphs: HAAD)
FTT-11 was declared a no-test due to a target failure.
The program successfully conducted FTT-14 to prove out  objective software for THAAD batteries. delivery of the first two batteries.    The THAAD program is currently mitigating production issues.
The program completed ground component deliveries for  THAAD Battery 1 and 2.   Conditional release of THAAD Batteries to the Army is  delayed 6 months.
The program successfully completed qualification on  interceptor ignition safety system.   Successful THAAD flight test proves out objective software for  batteries during fiscal year 2010.    Targets issues continued to delay test schedule and  achievement of knowledge points.
subsections: section_title: THAAD Interceptor Design and Qualification Issues Have Delayed Delivery of the First Two Batteries
paragraphs: Appendix XIII: Terminal High Altitude Area  Defense (THAAD)
2011. In addition, discoveries during a recent ground test have led to  further design changes which are not expected to be completed until the  second or third quarter of fiscal year 2011.
MDA’s delays in incorporating the requirement for the optical block, an  ignition safety system to prevent inadvertent launches, led to the design  and qualification issues. In 2003, an Army safety review board identified  the requirement to install an optical block on the THAAD interceptor’s  initiation system. However, MDA did not modify the development contract  to include this requirement until 2006. In addition, MDA awarded a  production contract in December 2006 before a design for the optical  block was selected. To date, 7 years after the optical block requirement  was identified, program and contractor officials told us that full recurring  costs have not been determined and the requirement has not been defined  on the production contract. Although the safety switch itself had been  qualified, the program’s first qualification of the integrated design in early  fiscal year 2010 failed due to contamination. After making changes to the  manufacturing process, the part was able to pass qualification in  September 2010.
Although the program recently completed efforts to requalify the optical  block component, failures during recent in-processing testing on the  integrated design have again led the program to make minor design  changes and delayed production of the first interceptor for a THAAD  battery another quarter. The program was on-track to produce the first  interceptor for THAAD Battery #1 in the first quarter of fiscal year 2011,  but due to corrective actions to a circuit in the integrated design, the  program won’t be able to produce its first interceptor for a THAAD battery  until the second quarter of fiscal year 2011. This extra quarter is in  addition to the more than a year delay from the original planned date.  According to program officials, the program plans to develop another  design of the optical block that is more producible for use on THAAD  battery interceptors in the future.
section_title: The THAAD Program Is Currently Mitigating Production Issues
paragraphs: In addition to completing its optical block design qualification, the THAAD  program office is pursuing mitigation plans on other production risks  including production rates and production gaps.
Appendix XIII: Terminal High Altitude Area  Defense (THAAD) program has only demonstrated producing one flight sequencing assembly  every 2 months. A variety of issues have prevented the subcontractor from  demonstrating the necessary production rate including design changes and  producibility issues associated with the optical block as well as availability  of optical block parts from the contractor that produces it. However,  according to the program office, the main issue that led to such a low level  of production was a time-consuming testing procedure during the  production process of the flight sequencing assembly. A new process has  been devised to significantly speed up this testing and program officials  are projecting that, with this process in place, production rates will  increase to one per week—meeting the four per month required rates. The  program hopes to demonstrate this new production rate as production  resumes.
Appendix XIII: Terminal High Altitude Area  Defense (THAAD) program plans to negotiate these costs with the future contract but the  effect of these gaps on cost is not yet known.
section_title: Conditional Release of THAAD Batteries to the Army Is Delayed 6 Months
paragraphs: The THAAD program has delayed its conditional release of batteries to the  Army from September 2010 until April 2011 because of ongoing safety  issues with interceptor components. For the release to occur, the Army  must certify that the batteries are safe, suitable, and logistically supported.  According to program officials, the Army Safety Review Board will  perform safety testing on the integrated design of the optical block to be  completed in February 2011. These results are needed before the release  board can make its decision. However, the review board would need to  perform repeat testing on 21 integrated designs of the optical block in  order to achieve full materiel release, so THAAD will continue with  conditional materiel release after passing Army safety testing on 3 units of  the integrated designs.
In addition, program officials told us that THAAD is also accepting  conditional materiel release because of its inability to incorporate another  safety feature into the initial production units. This safety feature, called a  thermally initiated venting system (TIVS) prevents explosion of the boost  motor in the event that the canister holding the interceptor heats up to a  certain temperature. The THAAD program provided an authorization to  proceed on the requirement for TIVS in June 2007—more than 6 months  after the production contract was signed. The scope of the work for TIVS  on the production contract has yet to be authorized by the THAAD  program. A developmental test of the early TIVS design resulted in a test  failure. The program has planned corrective actions which will be tested  during an upcoming verification test in the second quarter of fiscal year  2011. The THAAD program will not be able to start producing interceptors  with this safety feature until it has produced more than half of its  production quantity for the first two batteries. Because it cannot deliver  interceptors with this safety feature to meet the requirement for full  materiel release up front, according to program officials, the Army will  accept conditional material release of THAAD and the THAAD program  will cut-in the TIVS design in the production process during the fourth  quarter of fiscal year 2011.
Appendix XIII: Terminal High Altitude Area  Defense (THAAD)
section_title: Successful THAAD Flight Test Proves Out Objective Software for Batteries during Fiscal Year 2010
paragraphs: Despite test delays over the past 2 years due to targets issues, the THAAD  program was able to conduct a flight test in June 2010 that successfully  demonstrated the complete objective software for the THAAD battery.  After reorganizing its testing schedule due to address issues with air- launched targets, the program was able to accelerate the next test that did  not use the air-launched targets, FTT-14, from the second quarter of fiscal  year 2012 to the third quarter of fiscal year 2010. The program changed  FTT-14’s objectives to demonstrate component objective software for the  THAAD battery in order to meet an important knowledge point for the  program. Also during the test, the program was able to successfully  demonstrate THAAD’s performance against a simulated mass raid  scenario.
section_title: Targets Issues Continued to Delay Test Schedule and Achievement of Knowledge Points
paragraphs: Since 2009, the THAAD program has experienced multiple delays and test  restructures that have affected the program’s ability to achieve knowledge  points. The THAAD program was unable to conduct planned tests FTT-11  and FTT-12 in fiscal year 2009 due to target availability issues and an  agency-wide restructuring of its testing schedule. Both of these tests were  slated to meet key knowledge points to demonstrate the AN/TPY-2  (THAAD mode) radar’s advanced discrimination capabilities and to  demonstrate the component objective software for the THAAD battery,  respectively. These tests were delayed into fiscal year 2010. However, in  December 2009, FTT-11 resulted in a “no test” due to a target failure  related to air-launched targets. This target failure led MDA to suspend all  tests involving air-launched targets (see appendix XII for a detailed  discussion of target issues). The THAAD program then had to reorganize  its testing plans delaying FTT-12 and FTT-13, accelerating FTT-14, and  changing its test objectives to meet the knowledge point originally planned  for FTT-12 to prove out THAAD battery objective software. Although the  program was able to successfully demonstrate this knowledge point, it  won’t be able to demonstrate the AN/TPY-2 (THAAD mode) radar’s  advanced discrimination capability until the fourth quarter of fiscal year  2011, nearly 2 years after it was originally planned. The third knowledge  point to demonstrate a flight test intercept against a medium-range  ballistic missile target to be met during FTT-13, has been delayed from the  second quarter of fiscal year 2011 until the third quarter of fiscal year 2012  because of the lack of availability of a medium range air-launched target.  See table 11 for how THAAD’s flight test objectives have changed.
Appendix XIII: Terminal High Altitude Area  Defense (THAAD)
section_title: Appendix XIV: Scope and Methodology
paragraphs: To assess all six areas of MDA’s progress, we examined the  accomplishments of nine BMDS elements that MDA is currently  developing and fielding: the Aegis Ballistic Missile Defense (Aegis BMD);  Aegis Ashore; BMDS Sensors; Command, Control, Battle Management, and  Communications (C2BMC); Ground-based Midcourse Defense (GMD);  Precision Tracking and Surveillance System (PTSS); Space Tracking and  Surveillance System (STSS); Targets and Countermeasures; and Terminal  High Altitude Area Defense (THAAD).  We developed data collection  instruments that were completed by most elements’ program offices and  reviewed the individual element responses. These instruments collected  detailed information on prime contracts, test schedules and results,  element performance, noteworthy progress, lessons learned, and  challenges facing the elements during the fiscal year. We also performed  detailed reviews of the progress made within selected Missile Defense  systems, or elements. The results of these reviews are presented in detail  in appendixes to this report and are also integrated as appropriate in our  findings related to progress in delivering assets and implementing new  initiatives. We also examined the cost/resource, schedule and test  baselines as presented in the BMDS Accountability Report (BAR), test  plans and reports, and production plans. We sought to examine MDA’s  Baseline Execution Reviews of each element’s progress, but were unable  to asses the supporting backup materials as they were not made available  to us until February 2011—the very end of the audit.
To assess whether MDA elements delivered assets as scheduled, we  examined the 2009 BMDS Accountability Report, and compared it to the  2010 version, looking for similarities and differences between the two. We  also reviewed responses to GAO data collection instruments, which  detailed key accomplishments for fiscal year 2010, to include some asset  deliveries, as well as any delayed asset deliveries.
To follow up on the progress MDA made to improve transparency and  accountability, we held discussions with officials in MDA’s Operations  Directorate to discuss the new phased adaptive approach as well as the  status of transparency and accountability issues the agency initiated in  early 2010. In addition, we reviewed pertinent DOD policies to compare  MDA’s current level of accountability with that of other DOD programs.
We interviewed officials from the Office of the Under Secretary of Defense  for Acquisition Technology and Logistics to discuss the oversight role of  the Missile Defense Executive Board. We also met with officials in MDA’s  Acquisition Directorate to discuss how the agency is establishing and  managing its internal baselines. We also reviewed various MDA statements  and documents related to MDA’s new phased adaptive approach. We  reviewed DOD acquisition system policy and various DOD directives to  gain insight into other DOD systems’ accountability and oversight  mechanisms. We also analyzed MDA’s acquisition directives and Missile  Defense Executive Board briefings to examine MDA’s current level of  oversight. In addition, we reviewed MDA’s Fiscal Year 2010 BMDS  Accountability Report, Integrated Master Test Plans, MDA budget estimate  submission justifications, the Ballistic Missile Defense Master Plan, and  prior reports that outlined the agency’s baselines and goals.
To assess the management process to synchronize acquisitions for the  European Phased Adaptive Approach, we synthesized management and  oversight principles from the Office of Management and Budget, DOD, the  Missile Defense Agency (MDA), and GAO best acquisition practices for  large acquisition efforts similar to EPAA. We then compared EPAA  acquisition efforts to these principles. To determine the status of those  efforts, we reviewed DOD and MDA documentation related to EPAA,  including the Ballistic Missile Defense Review (BMDR) Report and MDA’s  BMDS Accountability Report (BAR). We also requested the European PAA  cost estimate that was completed in fall of 2009. However, we did not  receive the cost estimate until near the end of our review and therefore we  could not perform and include the related analysis in our review. We met  with MDA and Office of the Secretary of Defense officials. We also visited  the U.S. European Command and U.S. Strategic Command. In addition, to  identify near-term development risks, we reviewed MDA’s BMDS  acquisition documentation, including the integrated master test plan and  budget documents. We met with officials from MDA directorates and  element program offices as well as the Offices of the Under Secretary of  Defense for Policy and the Under Secretary of Defense for Acquisition,  Technology, and Logistics and visited contractor facilities. We also used  prior GAO work regarding best acquisition practices to assess those risks. reports; however, in certain cases MDA was unable to provide supporting  documentation or the documentation provided was determined to be  insufficient to assess. Specifically, we analyzed sustainment,  refurbishment and test plans and program schedules. We also assessed  budget documents, and program reviews. We interviewed officials from  MDA, U.S. Strategic Command, U.S. Northern Command, and the Army.  We also interviewed officials from DOD’s Office of the Director, Test and  Evaluation, and the Space and Missile Defense Command. We met with  GMD contractors, Boeing and Raytheon, and met with officials from the  Naval Warfare Center, Crane Division to discuss aging and surveillance  testing. Our analysis covered data ranging from January 2002 through  September 2010.
We assessed MDA’s testing and target development progress by reviewing  the technical baselines in the BAR, MDA’s Integrated Master Test Plans,  the target business case analysis, target contracts, and other documents  related to target planning and acquisitions. In addition, we met with  officials in the Targets and Countermeasures Program Office to obtain  information on the target acquisition strategy including plans for cost,  schedule, and testing as well as to discuss the progress, challenges, and  lessons learned during fiscal year 2010 testing. We also interviewed  officials within program offices and within MDA functional directorates,  such as the Directorates for Engineering and Testing. In addition, we  discussed the elements’ test programs and test results with the BMDS  Operational Test Agency and DOD’s Office of the Director, Operational  Test and Evaluation. We held discussions with the BMDS Operational Test  Agency to follow up on BMDS models and simulations. future once MDA has had time to address our recommendations to  improve data reliability. In our July report, MDA stated that it intended  to  take a key step in addressing our recommendations by conducting a major review of the GMD program’s EVM data by the end of September 2010.  Because MDA provided documentation of its actions at the end of our  audit—in February 2011—leaving no time for review, we will assess  whether the steps MDA has taken sufficiently address GMD's data  reliability issues in n ext year's report.
As noted above, during the course this audit, we experienced significant  delays in obtaining information from MDA. During the audit, MDA did not  always provide GAO staff with expeditious access to requested documents  and articles of information, which delayed some audit analysis and  contributed to extra staff hours. Notwithstanding these delays, we were  able to obtain the information needed to satisfy our objectives in  accordance with generally accepted government auditing standards.  However, we were not given access until Late February 2011 to the  September 2009 European Phased Adaptive Approach cost estimate, the  backup material for the Baseline Execution Reviews, or the actions MDA  took to respond to our previous recommendations on Earned Value  Management. As a result, we were unable to assess this material as part of  this year's annual review. We intend to review this material as part of our  next mandated annual assessment. subcontractor providing medium-range targets. We also met with Coleman  Aerospace officials in Orlando, Florida to discuss their work on short- range targets and the actions they are taking in response to a December  2009 target failure. We met with Lockheed Martin official in Troy, Alabama  to discuss their role in developing THAAD interceptors.
We conducted this performance audit from March 2010 to March 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix XV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, David Best, Assistant Director;  LaTonya Miller; Ivy Hübler; Teague Lyons; Steven Stern; Gwyneth  Woolwine; Meredith Allen Kimmett; Brian Tittle; Letisha Antone; Kenneth  E. Patton; Karen Richey; Hai Tran; Robert Swierczek; Nicolaas Cornelisse;  Matthew S. Spiers; Rebecca Guerrero; and Alyssa Weir made key  contributions to this report.