Defense Acquisitions: Missile Defense Program Instability Affects Reliability of Earned Value Management Data
Published: 2010-07-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: By law, GAO is directed to assess the annual progress the Missile Defense Agency (MDA) made in developing and fielding the Ballistic Missile Defense System (BMDS). GAO issued its latest assessment of MDA's progress covering fiscal year 2009 in February 2010. This report supplements that assessment to provide further insight into MDA's prime contractor performance for fiscal year 2009. Prime contractors track earned value management (EVM) by making comparisons that inform the program as to whether the contractor is completing work at the cost budgeted and whether the work scheduled is being completed on time. Our analysis of contractor EVM data included examining contract performance reports for 14 BMDS contracts, reviewing the latest integrated baseline reviews, performing extensive analysis of data anomalies, and conducting interviews with Defense Contract Management Agency (DCMA) officials--the independent reviewers of MDA contractor EVM systems.
section_title: What GAO Found
paragraphs: Unlike GAO's reports in previous years, GAO was unable to analyze the EVM data for all MDA contracts. GAO determined that the data for the Ground-based Midcourse Defense (GMD) and Targets and Countermeasures programs were not sufficiently reliable to include in our report because of instability in these programs' baselines. When the baseline on which the work is performed and measured against is no longer representative of the program of record, program managers and other decision makers lose the ability to develop constructive corrective action plans to get the program on track. Specifically, without reliable EVM data, GAO was unable to identify significant performance drivers or forecast future cost and schedule performance. Because the two contracts associated with these programs represent half of the budgeted cost at completion for the 14 contracts GAO reviewed, GAO also determined it was not appropriate in this report to aggregate total projected underruns or overruns of the remaining 12 prime contracts as GAO has in prior reports. The GMD prime contractor performance data was not sufficiently reliable to use as the basis for analysis because the contractor was unable to update its baseline to include numerous changes to the program and modifications to the contract. Despite three large restructures since 2007 totaling over $2 billion, the GMD program has not conducted an integrated baseline review since December 2006. DOD acquisition policy states that an integrated baseline review is to be conducted within 6 months after contract award, exercise of contract options, or a major modification to an existing contract. The Director, MDA has taken extra steps to gain insight into the contractor's performance. Further, he intends to report EVM information to Congress annually. Similarly, the EVM data for the Targets and Countermeasures contractor is also not sufficiently reliable to use in our analysis. DCMA identified several issues with the stability of the Targets and Countermeasures program baseline including a large amount of schedule and quantity changes to planned flight tests and over 20 contract changes to the scope of work or corrective actions to quality issues for one of the delivery orders over the course of a year. Because the contractor has not been able to update the established budget in the baseline, the cost performance reports do not reflect an appropriate baseline against which to measure cost and schedule progress. Nine of the remaining twelve contracts experienced cost overruns for fiscal year 2009 mostly because of issues with maturing technologies, immature designs, or other technical issues. For example, the Airborne Laser contractor experienced a failure in some of the system's optics which required it to develop and procure new high power optics, delaying the test schedule and increasing program cost.
section_title: What GAO Recommends
paragraphs: GAO recommends that MDA resolve prime contractor EVM data reliability issues by the beginning of fiscal year 2011. If, by this time, MDA has not resolved these issues, the Secretary of Defense should provide a report to Congress on the steps MDA is taking to resolve them. DOD concurred with our recommendation.

section_title: Letter
paragraphs: In 2002, the President of the United States directed the Department of  Defense (DOD) to begin fielding an initial Ballistic Missile Defense System  (BMDS) capable of defending the U.S. homeland, deployed troops, friends,  and allies against ballistic missiles of all ranges in all phases of flight. The  Missile Defense Agency (MDA), established to develop and deploy these  missile defense capabilities, began delivering an initial capability by  concurrently developing and fielding assets. MDA is the DOD’s largest  single acquisition program—spending from approximately $7 billion to  $9.5 billion per year.
To more effectively manage complex investments such as these, in  December 1996 the Under Secretary of Defense for Acquisition and  Technology signed a memorandum announcing DOD’s adoption of  industry standards for earned value management (EVM) systems for use  on defense acquisitions. EVM is a project management approach that, if  implemented appropriately, provides objective reports of project status,  produces early warning signs of impending schedule delays and cost  overruns, and provides unbiased estimates of anticipated costs at  completion.
This report provides an in-depth analysis of MDA’s prime contractor fiscal  year 2009 EVM cost and schedule progress. Congress directed GAO in its  fiscal year 2002, 2007, and 2008 National Defense Authorization Acts, to  assess the cost, schedule, testing, and performance progress that MDA is  making in developing the BMDS. We have delivered assessments of  MDA’s progress covering fiscal years 2003 through 2009, issuing our latest  assessment of fiscal year 2009 in February 2010. As we reported in our  February 2010 report, although our annual assessments usually include  analysis of EVM data for MDA contractors, this year we are reporting on  this information separately.
To assess progress during fiscal year 2009, we examined contract  performance reports for 14 BMDS contracts that were managed by eight  BMDS program elements. However, we were only able to report our  analysis of EVM data for 12 of these 14 contracts in appendix II due to  concerns with data reliability. The data for the Ground-based Midcourse  Defense and the Targets and Countermeasures contracts were not  sufficiently reliable for inclusion in our analysis. The 14 contracts we  reviewed EVM data for are:     Aegis Ballistic Missile Defense (BMD) weapon system software;   two Aegis BMD SM-3 Block IA contracts for   a fourth lot of 27 missiles and   another lot of 24 missiles;    Aegis BMD SM-3 Block IA and IB missile technology development and    Airborne Laser (ABL);   three Sensors’ contracts    BMDS radars,    Army Navy/Transportable Radar Surveillance—Model 2 (AN/TPY- 2) radar #7, and     Thule upgraded early warning radar;     Command and Control, Battle Management, and Communications    Ground-based Midcourse Defense (GMD);     Space Tracking and Surveillance System (STSS);     Targets and Countermeasures; and   two Terminal High Altitude Area Defense (THAAD) contracts for    development and   fire unit fielding production.
Because we were only able to analyze EVM data for 12 of 14 contracts, this  report does not provide a BMDS-level analysis as we have provided in  previous years, again, because of data reliability concerns. To perform our  review of the EVM data for 14 contracts, we included several checks to  ensure data reliability including reviewing the most current  documentation on contractor performance data systems for each of the  contractor sites, reviewing the latest integrated baseline review (IBR) for  each contract, and following up with each program office to track how  identified risks are being addressed. In addition, we checked the reliability  of the performance data by consulting with earned value experts who  provided us with tools to perform extensive analysis to independently  review the data. We followed up on the results of this analysis tool with  the program office and further reviewed its responses with the Defense  Contract Management Agency (DCMA), which independently reviews all  of the MDA contractor EVM systems. Our scope and methodology is  discussed in more detail in appendix III.
We conducted this performance audit from February 2010 to July 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: MDA’s mission is to develop an integrated and layered BMDS to defend the  United States, its deployed forces, allies, and friends. In order to meet this  mission, MDA is developing a highly complex system of systems—land-,  sea- and space-based sensors, interceptors and battle management. Since  its initiation in 2002, MDA has been given a significant amount of flexibility  in executing the development and fielding of the BMDS. To enable MDA to  field and enhance a missile defense system quickly, the Secretary of  Defense in 2002 delayed the entry of the BMDS program into the DOD’s  traditional acquisition process until a mature capability was ready to be  handed over to a military service for production and operation. Because  MDA does not follow the traditional acquisition process, it has not yet  triggered certain statutory and regulatory requirements that other major  defense acquisition programs are required to adhere to.
For example, other major defense acquisition programs are required to  establish the total scope of work and total cost baselines as part of their  entry into the formal acquisition cycle. Title 10 United States Code (U.S.C.)  section 2435 requires a baseline description for major defense acquisition  programs, however the requirement to establish a baseline is not triggered  until a system enters into system development and demonstration.  DOD  has implemented this requirement with the acquisition program baseline in  its acquisition policy. Because the BMDS has not yet formally entered the  acquisition cycle, it has not yet been required to meet the minimum  requirements of section 2435. Therefore, because of the Secretary of  Defense’s decision to delay entry of the BMDS system into the acquisition  cycle, MDA is not required to establish the full scope of work or total cost  baselines. Since we began annual reporting on missile defense in 2004, we  have been unable to assess overall progress on cost. As a result, one of the  only tools available for us to use in assessing BMDS costs is the costs  reported on individual contracts.
MDA employs prime contractors to accomplish different tasks that are  needed to develop and field the BMDS. Prime contractors receive the bulk  of funds MDA requests each year and work to provide the hardware and  software for elements of the BMDS.
Table 1 provides a brief description of eight BMDS elements and the prime  contracts associated with these elements currently under development by  MDA.
Each BMDS program office’s prime contractor provides monthly earned  value reports which provide insight into the dollar gained or lost for each  dollar invested. These Contract Performance Reports compare monthly  progress to the existing cost or schedule performance baseline to reveal  whether the work scheduled is being completed on time and if the work is  being completed at the cost budgeted. For example, if the contractor was  able to complete more work than scheduled and for less cost than  budgeted, the contractor reports a positive schedule and cost variance, or  “underrun”. Alternatively, if the contractor was not able to complete the  work in the scheduled time period and spent more than budgeted, the  contractor reports both a negative schedule and cost variance, or  “overrun”. The results can also be mixed by, for example, completing the  work ahead of schedule (a positive schedule variance) but spending more  than budgeted to do so (a negative cost variance).
We also used contract performance report data to base projections of  likely overrun or underrun of each prime contractor’s budgeted cost at  completion. Our projections of overruns or underruns to the budgeted cost  at completion are based on the assumption that the contractor will  continue to perform in the future as it has in the past. In addition, since the  budgeted cost at completion provides the basis for our projected overruns,  we also provide it for each contract we assessed in appendix II.
In addition, as part of the yearly system compliance verification process,  DCMA conducts a periodic surveillance of contractor EVM systems to  determine initial and continuing compliance of those management systems  with government accepted standards. Surveillance (routine evaluation  and assessment) of the EVM systems is mandatory for all contracts that  require EVM systems compliance.  Surveillance ensures that the  contractor is meeting contractual terms and conditions and is in  compliance with applicable policies and regulations. DCMA has primary  responsibility for surveillance of the prime contractor and sub-tier  suppliers with EVM requirements. According to a DCMA Earned Value  Management Center official responsible for leading system surveillance, at  the completion of the assessment, the DCMA Earned Value Management  Center submits to the contracting officer a status of the contractor’s EVM  system compliance, including all supporting data to that effect.
If deficiencies are found during the course of the surveillance process, it is  the surveillance team’s responsibility, working through DCMA’s Earned  Value Management Center, to issue a written corrective action request.  The purpose of a corrective action request is to formally notify the  contractor that a documented course of action in the form of a corrective  action plan is needed to bring the EVM system in compliance with  government accepted EVM system guidelines. Corrective action requests  range in severity from Level I to Level IV where, according to a DCMA  Earned Value Management Center official responsible for leading system  surveillance, Level I is for non-compliance with the Defense Federal  Regulation Acquisition Supplement clauses in the contract that can be  corrected immediately and for which no special management attention is  required, and Level IV identifies issues where cost, schedule, technical  performance, resources, or management process issues have unfavorably  affected the supplier's EVM so that it is incapable of reporting meaningful  EVM across multiple programs or multiple sites; and these issues have not  been corrected. Level III and IV corrective action requests may trigger  formal reviews such as post award review for cause, compliance reviews,  or other system validation reviews and may result in suspension or  revocation of EVM systems certification.
section_title: EVM Data for the GMD and Targets and Countermeasures Programs Are Not Sufficiently Reliable
paragraphs: For GMD and Targets and Countermeasures, we determined that the EVM  data were not sufficiently reliable to analyze these contracts’ cost and  schedule performance because of instability in these programs. Without  reliable EVM data, we are unable to identify significant performance  drivers or forecast future cost and schedule performance. Further, when  the baseline on which the work is performed and measured against is no  longer representative of the program of record, program managers and  other decision makers lose the ability to develop constructive corrective  action plans to get the program on track. These reliability issues affect  MDA’s oversight of contractor progress and both MDA and GAO’s ability  to report this progress to external parties and Congress. MDA officials  were aware that significant changes were not reflected in the baselines for  these two elements and have been conducting more extensive oversight to  compensate, but did not alert us to this issue during the course of our  audit. The Director, MDA has acknowledged the importance of EVM and  to address some of these issues he has enacted quarterly reviews of each  of the program’s baselines. Further, he intends to report EVM information  to Congress annually.
According to DCMA officials, there were several issues associated with the  Boeing EVM system for GMD. One of the main issues was the contractor’s  inability to maintain a consistent performance measurement baseline.  With numerous changes to the program and modifications to the contract,  the contractor experienced difficulty incorporating these changes into the  baseline in order to measure performance against this new work. For  example, although the GMD program experienced a $1.3 billion dollar  restructure in 2007, another major restructure beginning in fiscal year 2008  for over $500 million that was completed in fiscal year 2009, and a third  in fiscal year 2010 for over $380 million, the GMD program has not  conducted an IBR since December 2006.DOD’s acquisition policy states  that an IBR is to be conducted within 6 months after contract award,  exercise of contract options, or major modifications to a contract.DCMA  officials told us that the GMD program had an IBR underway following the  restructure that began in fiscal year 2008 and completed in fiscal year  2009, but in May 2009 the program was again redirected and the baseline  review was cancelled.
The Director, MDA explained that some of the GMD program’s baseline  instability from frequent restructures was related to the changing GMD  role in European defense. The February 2007 budget request for fiscal year  2008 included an approach to European defense focused on GBIs from the  GMD element and a large fixed radar as well as transportable X-Band  radars. In September 2009, the administration altered its approach to  European defense and instead constructed a defense system to consist  primarily of Aegis BMD sea-based and land-based systems and  interceptors, as well as various sensors to be deployed over time as the  various capabilities are matured. The Director told us that these European  capability requirements changes drastically affected the GMD program as  a significant amount of work had to be restructured.
During these three to four years of GMD baseline instability, the Director,  MDA told us that MDA took steps to gain additional insight into the  contractor’s progress. The program held added reviews in the absence of  IBRs to understand planned near-term effort and how well they were  executing against those plans. In addition, the Director told us that the  program held monthly focus sessions during which the joint government  and contractor teams briefed the status of progress and risks. The Director  acknowledged that these insights are necessary to understand the meaning  of the near-term EVM data. However, without the benefit of a documented  IBR after multiple larger restructures to the program or being made aware  of MDA’s added reviews, we do not have sufficient confidence in the GMD  program performance measurement baseline to reliably analyze the  existing EVM data.
Boeing and MDA are taking steps to address problems with the reliability  of the contractor’s EVM data. The contractor had planned to deliver a  performance measurement baseline by May 2010 and the GMD program is  planning to conduct a series of IBRs on the remaining prime and major  subcontractor effort beginning in July 2010. In addition, the contractor is  taking initiatives to put a performance measurement baseline in place as  quickly as possible and is providing additional training for its management  and control account managers in charge of EVM. The Director, MDA told  us that MDA was changing how its future contracts for the GMD program  are being structured to be more receptive to modifications. This new  contract structure will include dividing the work into delivery orders so  that modifications will be reflected at a delivery order level instead of  affecting a larger contract. These steps may help resolve the EVM issues;  however we cannot determine the full effect of these steps until further  evaluation after their full implementation.
Similarly, we have determined that the EVM data for the Targets and  Countermeasures contractor, Lockheed Martin Space Systems, are not  sufficiently reliable for inclusion in our analysis. Based on discussions  with and reports issued by DCMA, the Targets and Countermeasures  contractor was unable to update its baseline because of numerous  program changes. In September 2007, when the delivery order for the  launch vehicle-2 was approximately 60 percent complete, Lockheed Martin  signaled that its baseline was no longer valid by requesting a formal  reprogramming of the effort to include an overrun in its baseline for this  delivery order. MDA allowed the contractor to perform a schedule  rebaseline and remove schedule variances – but did not provide any more  budget for the recognized overrun in the performance measurement  baseline. As a result, DCMA reported that the performance indicators for  this delivery order, needed to estimate a contract cost at completion, were  unrealistic. According to the Director, MDA did not believe the contractor  had justified that there was a scope change warranting additional budget  in the performance measurement baseline. He said he believed doing so  would mask problems the contractor was experiencing planning and  executing the contract which he identified as the issue as opposed to  changes in the contract’s scope. According to the Director, one example of  the issues the contractor was experiencing on this delivery order included  a failure rate of 64 percent on production qualification components. MDA  has since completed the work on this delivery order and begun managing  follow-on target production on a newly established delivery order.
In addition, during fiscal year 2009 DCMA identified several issues with the  stability of the Targets and Countermeasures program baseline. For  example, program changes since fiscal year 2008 on one delivery order  included over 20 contract changes to the scope of work or corrective  actions to quality issues. In addition, the schedule and quantity of planned  flight tests changed significantly. During the fiscal year, DCMA submitted a  corrective action request for noncompliance with incorporating authorized  changes in a timely manner although the contractor was able to close this  issue before the end of the reporting period. Because of the instability in  the baseline and the contractor’s inability to update the baseline with  these frequent changes, we determined the cost performance reports for  2009 do not reflect an appropriate baseline against which to measure cost  and schedule progress.
According to the Director, MDA, the agency has undertaken a major effort  to stabilize the Targets and Countermeasures program. MDA has  established a new target acquisition strategy to address recurring target  performance issues and increases in target costs. In this new strategy, the  agency will buy generic targets in larger lots that are not tied to a  particular test instead of smaller lots. This effort should also help increase  MDA’s flexibility to respond to changing program requirements. In  addition, the Director, MDA told us that the Director of Engineering at  MDA will define target requirements instead of the program manager  which should also help create more stability.
subsections: section_title: Despite Non-Compliance Ratings for MDA Prime Contractor EVM Systems, Most Were Sufficiently Reliable for GAO Review
paragraphs: During the course of our review, we found that DCMA assessed 7 of the 14  contractors’ EVM systems as noncompliant in fiscal year 2009. DCMA also  rated 3 of the 14 contractors systems as unassessed. We reviewed the  basis for the noncompliance and unassessed ratings and determined that  only the GMD and Targets and Countermeasures contractor EVM issues  affected the reliability of the data for our purposes. See table 2 for the  DCMA compliance ratings for the 14 MDA prime contracts’ EVM systems  and GAO’s reliability assessment.
Five EVM systems besides the GMD and Targets and Countermeasures  contractor EVM systems were rated as noncompliant by DCMA during the  fiscal year but did not lead to GAO to conclude that the EVM data were not  sufficiently reliable. In order to judge the reliability of the data, we  reviewed the significance of any open corrective action request(s) that  triggered a noncompliance rating and its impact on the contractor’s ability  to judge cost and schedule performance against a baseline. During the  course of our audit, we interviewed DCMA representatives at each of the  contractor sites to understand the basis for the noncompliance  determination and to gain information to help us assess the reliability of  the data.
For example, the EVM system of the STSS contractor Northrop Grumman  was deemed noncompliant because of two low-level corrective action  requests related to issues with other contracts that did not materially  affect the performance baseline for the STSS contract we assessed. Also,  the C2BMC’s contractor Lockheed Martin Information Systems & Global  Services received a rating of noncompliant during 2009 because of a  corrective action request that stated that major subcontractor efforts were  not specifically identified, assigned, or tracked in the organizational  breakdown structure. However, after the noncompliant rating was given,  DCMA reversed its decision and decided to close the corrective action  without requiring the contractor to change its methods.
In addition, although DCMA was unable to assess two EVM systems during  2009 for Lockheed Martin Mission Systems and Sensors under the Aegis  BMD weapon system contract, and Lockheed Martin Space Systems  Company under the two THAAD contracts, we determined that the  reasons for the unassessed rating did not lead to issues with data  reliability. According to the DCMA EVM specialist responsible for  monitoring the Aegis BMD weapon system, the Aegis BMD weapon system  contractor was unassessed because some of the accounting guidelines  could not be assessed in time for the compliance rating. In addition, the  THAAD contractor was not assessed because, according to DCMA,  although the contractor had addressed the open corrective action  requests, DCMA did not have the resources to review and document the  effectiveness of those actions in order to close these items before the end  of the rating assessment period. However, subsequent to the closing of the  rating assessment period, the contractor’s actions were deemed sufficient  by DCMA to fix the unresolved issues and the corrective action requests  were closed.
section_title: BMDS Prime Contractors Aggregate Analysis Not Appropriate Due to Data Reliability Issues
paragraphs: We are unable in this year’s report to aggregate total projected underruns  or overruns in our analysis of the remaining 12 prime contracts because  we had to exclude the GMD and Targets and Countermeasures programs  due to data reliability issues. The GMD and Targets and Countermeasures  prime contracts’ budgeted costs at completion total nearly $16 billion  dollars or half of the total 14 contracts’ budgeted cost at completion. By  removing such a large portion of data from our analysis, we determined  that it is inappropriate to perform any aggregate analysis. More detail is  provided for each of the contractors responsible for the remaining twelve  BMDS contracts’ cost and schedule performance in appendix II.
Nine of the remaining 12 contracts experienced cost overruns for fiscal  year 2009. Most of the overruns were because of issues with maturing  technologies, immature designs, or other technical issues. For example,  the ABL contractor experienced a failure in some of the system’s optics  which required it to develop and procure new high power optics, delaying  the test schedule and increasing program cost. In addition, the THAAD  development contractor expended more funds than expected for redesigns  on the missile’s divert and attitude control system assembly, correcting  issues with its boost motor, and making changes on the design of its  optical block—a safety system to prevent inadvertent launches. Also, the  contractor experienced cost overruns on extended testing and redesigns  for its prime power unit in the radar portion of the contract.
Contractors were able to perform within their fiscal year 2009 budgeted  costs for three contracts—the Aegis BMD SM-3 contract for a fourth lot of  27 SM-3 Block IA missiles and contract for another lot of 24 SM-3 Block IA  missiles, and the BMDS radars contract. The Aegis BMD SM-3 contractor  attributed underruns in both of these lots of Block IA missiles to  production efficiencies since the contractor has been building Aegis BMD  SM-3 Block I and IA missiles for nearly 6 years. The BMDS radars  contractor improved cost performance during the fiscal year through  efficiencies in the software development and systems engineering.
section_title: Conclusions
paragraphs: Because MDA has not established cost baselines, prime contractor EVM  data provides one of the only tools to understand MDA’s cost and schedule  progress, particularly for purposes of external oversight. At present that  tool cannot be used effectively for two major contractors because their  data are not sufficiently reliable. While MDA is taking action to stabilize its  programs and thereby improve the reliability of its EVM data, any  additional delays into fiscal year 2011 could affect future fiscal years’  oversight. Moreover, until the data are sufficiently reliable, MDA, GAO and  Congress lose the valuable insights into contractor performance that EVM  provides, including an understanding of significant drivers to performance,  the ability to forecast future cost and schedule performance, and the  ability to develop constructive corrective action plans based on these  results to get programs that have encountered problems back on track.
section_title: Recommendation for Executive Action
paragraphs: We recommend the Secretary of Defense direct MDA to resolve prime  contractor data reliability issues by the beginning of fiscal year 2011 and, if  MDA has not resolved the data reliability problems, determine the barriers  preventing resolution and provide a report to Congress on:  the steps MDA is taking to make its contractor data sufficiently  reliable,    how the data reliability issues affect MDA’s ability to provide  oversight of its contractors, and  the effect these issues have on MDA’s ability to report contractor  progress to others, including Congress.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report. These  comments are reprinted in appendix I. DOD also provided technical  comments, which were incorporated as appropriate.
DOD concurred with our recommendation to resolve prime contractor  EVM data reliability issues by 2011; however, DOD stated that MDA  considers its fiscal year 2009 prime contractor performance data to be  reliable. It should be noted that, while MDA has undertaken extra  measures to gain insight into and compensate for the program instability  effects on its EVM data, the insights gained by MDA are not available to  external organizations which depend on the EVM data to analyze and  forecast trends.  Without the benefit of MDA's extra measures and added  reviews, we maintain that the prime contractor fiscal year 2009 EVM data  are not sufficiently reliable for analysis. Although we agree that MDA will  likely have better insight into the reliability of its contractor performance  data once it completes its comprehensive Integrated Baseline Review  process and verifies data reliability through joint surveillance of the  contractor’s EVM system as stated in the DOD response, we are retaining  the recommendation to ensure that these corrective steps are  implemented in time to improve the reliability of the EVM data by the  beginning of the next fiscal year.
We are sending copies of this report to the Secretary of Defense, the  Director, MDA, and Office of Management and Budget. The report also is  available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. GAO staff who made major contributions to this  report are listed in appendix IV.
section_title: Appendix I: Comments from the Department of Defense
section_title: Appendix II: BMDS Prime Contractor Fiscal Year 2009 Cost and Schedule Performance
paragraphs: To determine if they are executing the work planned within the funds and  time budgeted, each prime contractor provides monthly reports detailing  cost and schedule performance. The contractor tracks earned value  management (EVM) by making comparisons that inform the program as to  whether the contractor is completing work at the cost budgeted and  whether the work scheduled is being completed on time and then reports  this information on Contract Performance Reports. For example, if the  contractor was able to complete more work than scheduled and for less  cost than budgeted, the contractor reports a positive schedule and cost  variance, or “underrun”. Alternatively, if the contractor was not able to  complete the work in the scheduled time period and spent more than  budgeted, the contractor reports both a negative schedule and cost  variance, or “overrun”. The results can also be mixed by, for example,  completing the work ahead of schedule (a positive schedule variance) but  spending more than budgeted to do so (a negative cost variance).
We provide two kind of variances in our individual contract assessments  pertaining to overruns or underruns either cumulatively over the life of the  contract or during the fiscal year. Cumulative variances are the overruns  or underruns the contractor has earned since the contract began. In order  to calculate fiscal year variances, we determined the contractor’s  cumulative variances at the end of September 2008 and subtracted them  from the cumulative variances at the end of September 2009. Fiscal year  2009 variances give us an idea of the contractor’s performance trends  during the fiscal year. A contractor may have cumulative overruns but  underrun its fiscal year budgeted cost or schedule by improving its cost  performance over the course of the fiscal year.
In our graphs, positive fiscal year variances (underrunning cost or  schedule) are indicated by increasing performance trend lines and  negative fiscal year variances (overrunning cost or schedule) are shown by  decreasing performance trend lines. In our notional example in Figure 1,  the positive slope of the cost variances line indicates that the contractor is  underrunning fiscal year budgeted cost. Specifically, the contractor began  the fiscal year with a negative cumulative cost variance of $7.0 million but  ended the fiscal year with a negative cumulative cost variance of $1.0  million. That means that the contractor underran its fiscal year budgeted  costs by $6.0 million and therefore has a positive $6.0 million fiscal year  cost variance. Alternately, the cumulative schedule variance is decreasing  during the fiscal year indicating that the contractor was unable to  accomplish planned fiscal year work and therefore has a negative fiscal  year schedule variance. In this case, the schedule performance declined  during the fiscal year from $5.0 million down to $2.0 million. Therefore,  the contractor was unable to accomplish $3.0 million worth of work  planned during the fiscal year.
The individual points on Figure 1 also show the cumulative performance  over the entire contract up to each month. Points in a month that are  above $0 million represent a positive cumulative variance (underrunning  cost or schedule) and points below $0 million represent a negative  cumulative variance (overrunning cost or schedule). In our notional  example, the contractor ended the fiscal year with a negative cumulative  cost variance of $1.0 million. This means that since the contract’s  inception, the contractor is overrunning its budgeted cost by $1.0 million.
Alternately, the contractor ended the fiscal year with a positive cumulative  schedule variance of $2.0 million. That means that over the life of the  contract, the contractor has been able to accomplish $2.0 million more  worth of work than originally planned.
Besides reporting cost and schedule variances, we also used contract  performance report data to base projections of likely overrun or underrun  of each prime contractor’s budgeted cost at completion. Our projections of  overruns or underruns to the budgeted cost at completion are based on  the assumption that the contractor will continue to perform in the future  as it has in the past. Our projections are based on the current budgeted  costs at completion for each contract we assessed, which represents the  total planned value of the contract as-of September 2009. However, the  budgeted costs at completion, in some cases, have grown significantly  over time. For example, the Airborne Laser (ABL) contractor reported  budgeted costs at completion totaling about $724 million in 1997, but that  cost has since grown to about $3.7 billion.
Our assessment only reveals the overrun or underrun since the latest  adjustment to the budget at completion. It does not capture, as cost  growth, the difference between the original and current budgeted costs at  completion. As a result, comparing the underruns or overruns for Missile  Defense Agency (MDA) programs with cost growth on other major defense  acquisition programs is not appropriate because MDA has not developed  the full scope of work and total cost baselines that other major defense  acquisition programs have.
subsections: section_title: Aegis BMD Contractors Experienced Mixed Performance during the Fiscal Year
paragraphs: The Aegis Ballistic Missile Defense (BMD) program employs two prime  contractors for its two main components—Lockheed Martin Mission  Systems and Sensors for the Aegis BMD Weapon System and Raytheon for  the Aegis BMD Standard Missile-3 (SM-3). During fiscal year 2009, the  Aegis BMD SM-3 Block IA and IB missile technology development and  engineering contract experienced declining cost and schedule  performance, the Aegis BMD SM-3 contract for a fourth lot of 27 Block IA  missiles had increasing cost and schedule performance, and the Aegis  Weapon System and Aegis BMD SM-3 contractor for another lot of 24 SM-3  Block IA missiles experienced mixed performance.
subsections: section_title: Aegis BMD Weapon System
paragraphs: Although the Aegis Weapon System contractor overran fiscal year 2009  budgeted costs by $0.2 million, it was able to accomplish $1.7 million more  worth of work than originally anticipated. The fiscal year 2009 cost  overrun is attributed to unplanned complexity associated with developing  radar software. During the fiscal year, the decline in cost performance and  subsequent recovery is partially attributed to annual technical instruction  baseline updates. These baseline updates occur over the course of a sixty  day period during which varying performance data occurs. At the end of  this period, there is a jump in performance as the contractor earns two  months worth of performance. Some of the cost savings from April  through September 2009 are the result of a planned flight test being  cancelled during the fiscal year and the contractor not spending intended  funds on pre-flight test, flight test, and post-flight test activities. The  favorable schedule variance was driven by completion of some technical  instruction efforts. Figure 2 shows cumulative variances at the beginning  of fiscal year 2009 along with a depiction of the contractor’s cost and  schedule performance throughout the fiscal year.
Considering prior performance on the Aegis Weapon System contract  since it began performance in October 2003, the contractor is $0.2 million  over budget and has been unable to accomplish $6.7 million worth of  work. The small negative cost variance was driven primarily by radar  software development issues, including a significant redesign not included  in the original baseline. In addition, the engineering test and evaluation  portion of the radar software is experiencing an increase in the lines of  code that also accounts for some of the budget overrun. The unfavorable  $6.7 million in schedule variances are attributed to the engineering test  and evaluation portion of the radar software for which builds and  capabilities are being delivered later than originally planned. If the  contractor continues to perform as it did through September 2009, our  analysis projects that at completion in June 2010, the work under the  contract could cost about $0.2 million more than the budgeted cost of $1.5  billion.
section_title: Aegis BMD SM-3 for 27 Block IA Missiles
paragraphs: The Aegis BMD SM-3 contractor for a fourth lot of 27 Block IA missiles  underran its budgeted fiscal year 2009 cost and schedule by $0.5 million  and $5.8 million respectively. The program attributed its cost and schedule  underruns to efficiencies in producing Aegis BMD SM-3 Block I and IA  missiles since the contractor has been building these missiles for nearly 6  years. Additionally, the program reported that the contract incentivizes the  contractor to deliver missiles ahead of schedule for maximum incentive  fee which further encouraged the contractor to accomplish $5.8 million  more worth of work then originally planned during the fiscal year. See  figure 3 for an illustration of cumulative cost and schedule variances  during the course of the fiscal year.
Considering prior years’ performance since the contract began in May  2007, the contractor is performing under budgeted cost with a favorable  cumulative cost variance of $3.9 million but is behind schedule on $1.3  million worth of work. The cost underruns are primarily driven by  implemented efficiencies, material transfers, and program management  adjustments within the solid divert and attitude control system; a decrease  in rework and more efficiencies realized with the seeker; and underruns in  engineering efforts associated with the third stage rocket motor. The $1.3  million in schedule overruns are attributed to late delivery of parts as the  result of some equipment failures. If the contractor continues to perform  as it did through September 2009, our analysis projects that at completion  in December 2011, the work under the contract could cost about $5.2  million less than the budgeted cost of $233.8 million.
section_title: Aegis BMD SM-3 for 24 Block IA Missiles
paragraphs: As of September 2009, the Aegis BMD SM-3 contractor for another lot of 24  Block IA missiles had underrun its fiscal year budget by $4.2 million and  was behind in completing $3.7 million worth of work. The contractor  attributes its cost underrun to efficiencies in program management and  systems engineering because of its experience in building SM-3 Block I  and IA missiles. The $3.7 million in schedule overruns resulted from the  contractor planning the baseline to a more aggressive schedule than the  contractual missile delivery schedule requires. The contractor plans in this  way because it is incentivized to deliver missiles 2 months ahead of  schedule. As a result, negative schedule variances have occurred as the  contractor is pushing to deliver missiles early. Figure 4 shows both cost  and schedule trends during fiscal year 2009.
Cumulatively, since the contract began in February 2008, the contractor is  underrunning its contract’s budgeted cost by $1.4 million but is behind on  $2.1 million worth of work. The contractor attributes the cost underrun to  labor efficiencies and reduced manpower within the seeker design as well  as a slower-than-planned ramp-up of some engineering efforts. The  schedule delays are mainly driven by non-delivery of parts for the first  stage rocket motor and late deliveries of parts associated with the third  stage rocket motor. If the contractor continues to perform as it did  through September 2009, our analysis projects that at completion in  December 2011, the work under the contract could cost from $15.3 million  less to $1.9 million more than the budgeted cost of $192.6 million.
section_title: Aegis BMD SM-3 Block IA and IB Technical Development and Engineering
paragraphs: For the majority of the fiscal year, the Aegis BMD SM-3 Block IA and IB  Technical Development and Engineering contractor experienced a  negative downward trend in cost and schedule performance. The program  attributes its fiscal year cost overrun of $44.6 million to engineering  development on its Aegis BMD SM-3 Block IB throttleable divert and  attitude control system being more difficult than planned. The $29.4  million of unaccomplished work during the fiscal year was due to late  receipt of materials that drove delays in some of the hardware testing. See  figure 5 for trends in the contractor’s cost and schedule performance  during the fiscal year.
Cumulatively, since the contract began in December 2007, the program  also has unfavorable cost and schedule variances of $51.2 million and  $40.0 million, respectively. Drivers of the $51.2 million in cost overruns are  throttleable divert and attitude control system engineering and hardware  major submaterial price increases in support of design reviews and  demonstration unit. In addition, quality issues added to cost overruns as  the contractor experienced unanticipated design changes to the nozzle  resulting from foreign object debris issues. The $40.0 million worth of  work that the contractor was unable to achieve was driven by several  issues, including late receipt of hardware and late production-level  drawings. In addition, delays in testing for attitude control system  thrusters and a quality issue that led to the contractor receiving  nonconforming hardware also contributed to unaccomplished work. If the  contractor continues to perform as it did through September 2009, our  analysis projects that at completion in December 2010, the work under the  contract could cost from $94.0 million to $194.8 million more than the  budgeted cost of $588.9 million.
section_title: ABL Cost and Schedule Performance Declined during Fiscal Year 2009
paragraphs: The ABL contractor, Boeing, experienced cost growth and schedule delays  throughout the fiscal year. The contractor overran budgeted fiscal year  2009 cost and schedule by $10.2 million and $14.9 million respectively. The  major drivers of fiscal year negative variances were technical issues and  the addition of some testing that was not originally anticipated. For  example, a fire suppression system failed to meet performance  requirements for the laser flight test which limited the scope of the testing,  added an unscheduled ground test and flight tests to ensure that the  system worked properly, and increased costs. In addition, the contractor  experienced a failure in some of the system’s optics which required it to  develop and procure new high-power optics and ultimately delayed the  test schedule and increased program cost. Lastly, because of issues  discovered during beam control/fire control flights, the program scheduled  additional unplanned beam control flights to accomplish the necessary  objectives. The contractor experienced a continuing cost and schedule  performance decline, as seen in figure 6.
The contractor’s cumulative cost variance is over budget by $95.0 million  and behind schedule by $38.5 million from when the contract began in  November 1997. The program attributes these variances to optics issues  that have affected delivery and installation and caused test program  delays. If the contractor continues to perform as it did through September  2009, our analysis projects that at completion in February 2010, the work  under the contract could cost from $98.0 million to $116.8 million more  than the budgeted cost of $3.7 billion.
section_title: C2BMC Overrunning Cumulative Cost and Schedule
paragraphs: The Command and Control, Battle Management, and Communications  (C2BMC) contractor, Lockheed Martin Information Systems & Global  Services, is currently overrunning budgeted costs for the agreement since  it began performance in February 2002 by $29.5 million and has a  cumulative schedule variance of $4.2 million. According to program  officials, the main drivers of the cumulative variances are associated with  the Part 4 and Part 5 portions of the agreement. The Part 4 effort, which  began in January 2006 and finished December 2007, was for the  completion of several spiral capabilities, the upgrade for spiral suites, and  implementation of initial global engagement capabilities at its operations  center. The Part 5 effort, which began in January 2008 and is still ongoing,  covers operations and sustainment support for fielded C2BMC; deliveries  of spiral hardware, software, and communications; and initiated  development of initial global engagement capabilities. MDA and the  contractor anticipate being able to cover cost overruns on the agreement  with the nearly $39 million in management reserve set aside by the  contractor.
Part 5 accounts for nearly $10.4 million of the $29.5 million in negative  cumulative cost variances. These budgeted cost overruns are driven by  increased technical complexity of Spiral 6.4 development, and more  support needed than planned to address requests from the warfighter for  software modifications. The $4.2 million of unaccomplished work on the  agreement is driven by efforts in the Part 5 portion of the agreement,  including delays in system level tests, late completion of C2BMC interface  control document updates, and unexpected complexity of algorithm  development and network design. See figure 7 for an illustration of  cumulative cost and schedule performance during fiscal year 2009.
The contractor overran its fiscal year 2009 budgeted cost by $5.2 million  but is $2.9 million ahead of schedule. The drivers of the unfavorable fiscal  year cost variance of $5.2 million are complexities associated with Spiral  6.4 development, additional design excursions, and additional costs to  address system modifications requested by the warfighter. The contractor  achieved a favorable fiscal year schedule variance largely because of gains  in the month of September 2009. During this month, the contractor  performed a replan of its work content and a future spiral’s scope was  removed from the Part. This replan eliminated approximately $10 million  in schedule variances for labor and materials because the work was no  longer to be performed. If the contractor continues to perform as it did  through September 2009, our analysis projects that at completion in  December 2011, the work under the agreement could cost from $26.5  million to $33.1 million more than the budgeted cost of $1.0 billion.
section_title: Sensors Contractor Experienced Mixed Performance during the Fiscal Year
paragraphs: This year we are reporting on three contracts under the Sensors  program—the Ballistic Missile Defense System (BMDS) Radars contract  on which we have reported in prior years, the Terminal High Altitude Area  Defense (THAAD) fire unit radar #7 contract, and the Thule radar contract.  During fiscal year 2009, the Sensors’ contractor, Raytheon, experienced  declining cost and schedule performance on the Thule radar and Army  Navy/Transportable Radar Surveillance—Model 2 (AN/TPY-2) radar #7  contracts, but had favorable cost and schedule performance on the BMDS  Radars contract.
subsections: section_title: BMDS Radars
paragraphs: Throughout fiscal year 2009, the BMDS Radars contractor exhibited  improved cost and schedule performance. The contractor was able to  perform $5.8 million under budgeted cost and $3.5 million ahead of  schedule for the fiscal year. The drivers of the contractor’s improved cost  performance are efficiencies in the software development and systems  engineering. The contractor reports that the improved schedule  performance is due to software schedule improvement as well as  completion of manufacturing and integration testing on one of the radars.  The variances, depicted in figure 8, represent the BMDS Radars  contractor’s cumulative cost and schedule performance over fiscal year  2009.
Since the contract began in March 2003, the BMDS Radars contractor is  under budget by $27.8 million but is behind on accomplishing $6.1 million  worth of work. The favorable cost variance of $27.8 million is driven by  the use of less manpower than planned and the benefit of lessons learned  from previous radar software builds. The unfavorable $6.1 million of  unaccomplished work was driven by the late start on restructuring the  latest software release and rework and subcomponent delays with one of  the radars. If the contractor continues to perform as it did through  September 2009, our analysis projects that at completion in August 2010,  the work under the contract could cost from $31.3 million to $43.0 million  less than the budgeted cost of $1.2 billion.
section_title: AN/TPY-2 #7 Radar
paragraphs: The AN/TPY-2 radar #7 contractor experienced unfavorable fiscal year  2009 cost and schedule variances of $4.3 million and $15.2 million,  respectively. As of September 2009, the AN/TPY-2 radar #7 contract had  overrun its budgeted cost by $1.9 million but was ahead in completing $9.0  million worth of work. Contributors to the cumulative cost overruns  included supplier quality issues that required an increase in supplier  quality support that was not in the original baseline. In addition, the  program’s prime power unit purchase orders were over budgeted cost  because the budgeted cost for four of the prime power units was  prematurely established before the design of the first prime power unit  was finalized. These delays caused some uncertainty in the final  production costs until the design was finalized. As of August 2009, the  contractor was working to develop a cost model and establish a true unit  cost price per prime power unit. Trends in cost and schedule performance  during the fiscal year are depicted in figure 9.
Cumulatively, since the contract began in February 2007, the AN/TPY-2  Radar #7 contractor has completed $9.0 million worth of work ahead of  schedule on this contract by executing work ahead of the contract  baseline plan in some areas, including obtaining materials for equipment  supporting radar operation. If the contractor continues to perform as it did  through September 2009, our analysis projects that at completion in April  2010, the work under the contract could cost from $0.3 million less to  $36.9 million more than the budgeted cost of $172.5 million.
section_title: Thule Radar
paragraphs: The Thule radar contractor overran fiscal year 2009 budgeted costs by $0.4  million and was unable to accomplish $0.8 million worth of work. The  contractor attributes the cost overruns to exceeding planned engineering  efforts in order to proactively work on issues prior to equipment delivery  and ship readiness. The unfavorable schedule performance is due to the  contractor expending some if its positive schedule variance in 2008 and  from being behind schedule on the implementation of information  assurance requirements. Figure 10 shows cumulative variances at the  beginning of fiscal year 2009 along with a depiction of the contractor’s  cost and schedule performance throughout the fiscal year.
The Thule radar contractor, since it began performance in April 2006, is  underrunning budgeted costs by $2.5 million and overrunning schedule by  $0.2 million. Underruns in hardware, manufacturing, and facility design,  construction, and installation drove the $2.5 million in cost underruns. If  the contractor continues to perform as it did through September 2009, our  analysis projects that at completion in September 2010, the work under  the contract could cost from $1.4 million to $2.8 million less than the  budgeted cost of $101.9 million.
section_title: STSS Maintained Schedule Performance, but Cost Performance Continued to Decline during the Fiscal Year
paragraphs: During fiscal year 2009, the Space Tracking and Surveillance System  (STSS) contractor, Northrop Grumman, was able to accomplish $0.1  million more worth of work than originally anticipated, but overran  budgeted costs by $72.6 million. The contractor reports that the favorable  schedule variances are due to completed space vehicle 1 and 2 shipment,  setups and validations, and launch. In addition, the contractor overran  budgeted fiscal year costs because of additional support required to  support launch operations including addressing hardware anomalies,  payload integration, procedure development, and launch site activities.  Additional support was also required to support the delays to the launch  date beyond the original plan. See figure 11 for an illustration of the  cumulative cost and schedule variances during fiscal year 2009.
Despite the small gains in schedule variances during the fiscal year, the  contractor maintains cumulative negative cost and schedule variances of  $391.8 million and $17.7 million respectively from the contract’s inception  in August 2002. Drivers of the $391.8 million in contract cost overruns are  for labor resources exceeding planned levels and unanticipated difficulties  related to space vehicle environment testing, hardware failures and  anomalies, and program schedule extension. In addition, space vehicle-1  testing, rework, hardware issues, and sensor testing anomaly resolution as  well as space vehicle-2 anomalies and testing have also contributed to the  unfavorable cost variances. System test and operations and program  management experienced cost overruns because of launch date schedule  extensions. Lastly, ground labor resources exceeded planned levels  because of the unanticipated need for a new ground software build and  ground acceptance and verification report activities. The contractor has  been unable to accomplish $17.7 million worth of work on the contract  because of launch schedule delays, delays in verification of system  requirements caused by late space segment deliveries, and tasks slipping  in response to fiscal year 2009 funding reductions. If the contractor  continues to perform as it did through September 2009, our analysis  projects that at completion in September 2010, the work under the  contract could cost from $620.9 million to $1.6 billion more than the  budgeted cost of $1.6 billion.
section_title: THAAD Development Contract Overran Cost and Schedule While THAAD Fire Unit Fielding Production Contract Experienced Underruns
paragraphs: This year we report on two THAAD contracts—the development contract  and the fire unit fielding production contract. As the contractor for both of  these contracts, Lockheed Martin Space Systems Company was  overrunning budgeted cost and schedule on the THAAD development  contract but remained under cost and ahead of schedule on the THAAD  fire unit fielding production contract.
subsections: section_title: THAAD Development
paragraphs: During fiscal year 2009, the THAAD development contractor overran its  budgeted cost by $33.1 million but was ahead on completing $7.4 million  worth of work. The fiscal year cost overruns are mainly in the missile,  launcher, and radar portions of the contract. The missile experienced  overruns on divert and attitude control system assembly redesigns,  correcting issues with its boost motor, and making changes on the design  of its optical block—a safety system to prevent inadvertent launches. The  contractor spent more than expected during the fiscal year on the  launcher portion of the contract, investing in labor and overtime to  recover schedule. Lastly, the prime power unit in the radar portion of the  contract required extended testing and redesign, which also contributed to  fiscal year costs.
Despite fiscal year cost overruns, the contractor was able to accomplish  $7.4 million more worth of work than originally anticipated also in the  missile and launcher portions of the contract. The schedule variance  improved in the missile portion because of completion of missile  qualification work. The contractor was also able to complete software  activities and resolve hardware design and qualification issues in the  launcher. See figure 12 for trends in the contractor’s cost and schedule  performance during the fiscal year.
Although the contractor made some schedule gains during the fiscal year,  overall the contractor since it began performance in June 2000 is behind  on $9.1 million worth of work. The radar’s portion of unfavorable schedule  variance is driven by delays to THAAD flight test missions during fiscal  year 2009. In addition, the fire control’s software qualification testing had  to be extended because of the number of software changes and because  the welding on the fire control power distribution unit’s chassis failed  weld inspection and was subsequently unusable which contributed to the  unfavorable schedule variance. The launcher experienced design delays  and quality issues that led to nonconformances in delivered hardware.  This hardware subsequently required investigation and rework, which also  added unexpected work to the schedule. Lastly, the program was unable  to accomplish work in the missile component’s flight sequencing assembly  component because qualification tests were delayed due to failures with  the optical block switch.
The unfavorable fiscal year cost variances added to the overall cost  overruns of $261.9 million. The contractor attributes overruns to the  missile, launcher, and radar portions of the contract. The missile’s  unfavorable cost variance is driven by unexpected costs in electrical  subsystems, propulsion, and divert and attitude control systems. Also  contributing are issues associated with the optical block, range safety,  communications systems, and boost motors. The launcher has  experienced cost growth because of inefficiencies that occurred during  hardware design, integration difficulties, quality issues leading to delivered  hardware nonconformances, and ongoing software costs being higher than  planned because of rework of software to correct testing anomalies. These  problems resulted in schedule delays and higher labor costs to correct the  problems. In addition, cooling and power issues with the radar have  contributed to overruns with the prime power unit. Numerous fan motor  control system redesigns and retrofits for the cooling system drove costs  by the supplier. Inexperience with building a prime power unit and a  limited understanding of the true complexity and risks associated with the  system led to significant cost growth and delivery delays. If the contractor  continues to perform as it did through September 2009, our analysis  projects that at completion in January 2011, the work under the contract  could cost from $267.2 million to $287.4 million more than the budgeted  cost of $4.8 billion.
section_title: THAAD Fire Unit Fielding Production
paragraphs: The THAAD fire unit fielding production contractor overran fiscal year  2009 budgeted cost and schedule by $4.7 million and $10.7 million,  respectively. The fiscal year cost and schedule overruns were caused  primarily by the missile and fire control components. Unfavorable missile  cost and schedule variances were the result of hardware failures  associated with components of the inertial measurement unit,  communications transponder, and the boost motor causing delays and  rework. In addition, the fire control portion of the contract experienced  overruns because of unplanned engineering design changes and labor  associated with fire control hardware and issues identified during testing.  These changes were made to the hardware and deliveries already  completed. See figure 13 for an illustration of cumulative cost and  schedule variances during the course of the fiscal year.
Despite fiscal year overruns, the fire unit production contractor continues  to underrun its total contract cost and schedule. The contractor, since it  began performance in December 2006, is currently $6.1 million under  budgeted costs and has completed $11.3 million more worth of work than  originally anticipated. The cost underruns are primarily due to a slow  start-up on fire unit fielding level of effort activities. Schedule variances  are not reported on level of effort activities, so delaying these activities  would save on costs without affecting reported schedule. However, these  false positive cost variances will erode over time once the work gets  accomplished. When planned level of effort work is not performed, EVM  metrics are distorted because they show cost savings for work that has not  yet been accomplished. However, once the work is finished, large  unfavorable cost variances will be revealed since the program will need to  expend funds to accomplish the work for which it has already received  credit. In addition, the program reports its favorable schedule variances  are due to the transfer of excess interceptor hardware from the  development contract to the fire unit fielding contract. Although the  favorable schedule variance from this transfer of hardware is nearly $23.0  million, offsets occurred from delayed interceptor build activity driven by  multiple supplier hardware issues and schedule delays because of issues  with the boost motor including unplanned replacement of motor cases,  delayed case fabrication, and slowed operations caused by a safety  incident at a production facility. If the contractor continues to perform as  it did through September 2009, our analysis projects that at completion in  August 2011, the work under the contract could cost from $1.3 million to  $17.9 million less than the budgeted cost of $604.4 million. However, it  should be noted that the projection of the estimated cost at completion  may also be overestimated because it is based on current cost  performance that is inflated because of level of effort activities and  schedule performance which are inflated by transfers of materials from  another contract.
section_title: Appendix III: Scope and Methodology
paragraphs: To examine the progress Missile Defense Agency (MDA) prime  contractors made in fiscal year 2009 in cost and schedule performance, we  examined contractor performance on 14 Ballistic Missile Defense System  (BMDS) element contracts. In assessing each contract, we examined  contract performance reports from September 2008 through October 2009  for each contract, including the format 1 variance data report, cost and  schedule variance explanations included in the format 5, and format 2  organizational category variance totals where available. We performed  extensive analysis on the format 1 of the contract performance reports in  order to aggregate the data and verify data reliability.
To ensure data reliability, we performed a series of checks based on  consultation with earned value experts and in accordance with GAO  internal reliability standards. We began by tracking which earned value  management (EVM) systems that produced the contract performance  reports were compliant with American National Standards Institute  standards in 2009 by reviewing the certification documentation. We  received this documentation through the Defense Contract Management  Agency (DCMA), which performs independent EVM surveillance of MDA  contractors. We then reviewed the latest integrated baseline review out- briefs for the BMDS elements’ contracts to examine the earned value- related risks that were identified during the review and followed up with  the program office to see which, if any, risks were still open action items.
To further review the contract performance report format 1 data, we  performed basic checks on the totals from contract performance report  format 1 to ensure that they matched up with organizational totals from  the contract performance report format 2, where available. This check  enabled us to review whether the earned value data were consistent  across the report. In addition, we obtained a spreadsheet tool from GAO  internal earned value experts to perform a more extensive check of the  data. Using this tool, we ran various analyses on the data we received to  search for anomalies. We then followed up on these anomalies with the  program offices that manage each of the 14 BMDS element contracts. We  reviewed the responses with GAO EVM experts and further corroborated  the responses with DCMA officials.
We used contract performance report data in order to generate our  estimated overrun or underrun of the contract cost completion by using  formulas accepted by the EVM community and printed in the GAO Cost  Estimating and Assessment Guide. We generated multiple formulas for  the projected contract cost at completion that were based on how much of  the contract had been completed up to September 2009. The ranges in the  estimates at completion are driven by using different efficiency indices  based on the program’s completion to adjust the remaining work  according to the program’s past cost and schedule performance. The idea  in using the efficiency index is that how a program has performed in the  past will indicate how it will perform in the future. In close consultation  with earned value experts, we reviewed the data included in the analysis  and made adjustments for anomalous data where appropriate.
We conducted this performance audit from February 2010 to July 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, David Best, Assistant Director;  Meredith Kimmett; LaTonya Miller; Karen Richey; Robert Swierczek;  Alyssa Weir, and John A. Krump made key contributions to this report.