Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals
Published: 2008-03-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: By law, GAO annually assesses the Missile Defense Agency's (MDA) progress in developing and fielding a Ballistic Missile Defense System (BMDS). Funded at $8 billion to nearly $10 billion per year, it is the largest research and development program in the Department of Defense (DOD). The program has been managed in 2-year increments, known as blocks. Block 2006, the second BMDS block, was completed in December 2007. GAO assessed MDA's progress in (1) meeting Block 2006 goals for fielding assets, completing work within estimated cost, conducting tests, and demonstrating the performance of the overall system in the field, and (2) making managerial improvements to transparency, accountability, and oversight. In conducting the assessment, GAO reviewed the assets fielded; contractor cost, schedule, and performance; and tests completed during 2007. GAO also reviewed pertinent sections of the U.S. Code, acquisition policy, and the charter of a new missile defense board.
section_title: What GAO Found
paragraphs: MDA made progress in developing and fielding the BMDS during Block 2006 but fell short of meeting its original goals. Specifically, it fielded additional assets such as land-based interceptors and sea-based missiles and upgraded other assets, including Aegis BMD-equipped ships. It also met most test objectives, with a number of successful tests conducted. As a result, fielded capability has increased. On the other hand, it is difficult to assess how well BMDS is progressing relative to the funds it has received because fewer assets were fielded than originally planned, the cost of the block increased by at least $1 billion, some flight tests were deferred, and the performance of the fielded system remains unverified. In particular, GAO could not determine the full cost of Block 2006 because MDA continued to defer budgeted work into the future, where it is no longer counted as a Block 2006 cost. Also making cost difficult to assess is a work planning method--referred to as level of effort--used by contractors that does not link time and money with what is produced. When not appropriately used, level-of-effort planning can obscure work accomplished, portending additional cost in the future. MDA is working to minimize the use of this planning method--a needed step as contractors overran their fiscal year 2007 budgets. Performance of the fielded system is as yet not verifiable because too few tests have been conducted to validate the models and simulations that predict BMDS performance. Moreover, the tests that are done do not provide enough information for DOD's independent test organization to fully assess the BMDS' suitability and effectiveness. GAO has previously reported that MDA has been given unprecedented funding and decision-making flexibility. While this flexibility has expedited BMDS fielding, it has also made MDA less accountable and transparent in its decisions than other major programs, making oversight more challenging. MDA, with direction from Congress, has taken several steps to address these concerns. MDA implemented a new way of defining blocks--its construct for developing and fielding BMDS increments--that should make costs more transparent. For example, under the newly-defined blocks, MDA will no longer defer work from one block to another. Accountability should also be improved as MDA will, for the first time, estimate unit costs for selected assets and report variances from those estimates. DOD also chartered a new board with more BMDS oversight responsibility than its predecessor, although it does not have approval authority for some key decisions made by MDA. Finally, MDA will begin buying certain assets with procurement funds like other programs. This will benefit transparency and accountability, because procurement funding generally requires that assets be fully paid for in the year they are bought. Previously, MDA, with Congressional authorization, was able to pay for assets incrementally over several years. Additional steps could be taken to further improve oversight. For example, MDA has not yet estimated the total cost of a block, and therefore, cannot have its costs independently verified--actions required of other programs to inform decisions about affordability and investment choices. However, MDA does plan to estimate block costs and have them verified at some future date.

section_title: Letter
paragraphs: In 2002, the President of the United States directed the Department of  Defense (DOD) to begin fielding an initial Ballistic Missile Defense System  (BMDS) capable of defending the U.S. homeland, deployed troops, friends,  and allies against ballistic missiles of all ranges in all phases of flight.   Since the 1980s, DOD has spent more that $100 billion on the development  and early fielding of this system and it estimates that continued  development will require an additional $50 billion between fiscal years  2008 and 2013.
The Missile Defense Agency (MDA) has been tasked to carry out the  President’s direction and is developing and fielding the BMDS, DOD’s  largest research and development program.  MDA placed an initial set of  missile defense components in the field by December 2005.  These  components are collectively referred to as Block 2004.  Recently, MDA  delivered its second increment of capability—Block 2006—which includes  additional components as well as performance enhancements.  Current  plans call for the continuation of BMDS development for many years, with  the system eventually including a diverse collection of land-, air-, sea-, and  space-based assets located around the globe.
In its fiscal year 2002 and 2006 National Defense Authorization Acts,  Congress directed GAO to assess the cost, schedule, testing, and  performance progress that MDA is making in developing the BMDS. We  have delivered assessments covering fiscal years 2003, 2004, 2005, and  2006. This report assesses Block 2006 and gives special attention to the  progress made during fiscal year 2007 toward Block 2006 goals.  It also  follows up on BMDS program transparency, accountability, and oversight  issues addressed in our March 2007 report.
To assess progress during Block 2006, we examined the accomplishments  of all nine BMDS elements that MDA is developing and fielding.  These  elements include the Airborne Laser (ABL); Aegis Ballistic Missile Defense  (Aegis BMD); BMDS Sensors; Command, Control, Battle Management, and  Communications (C2BMC); Ground-based Midcourse Defense (GMD);  Kinetic Energy Interceptors (KEI); Multiple Kill Vehicles (MKV); Space  Tracking and Surveillance System (STSS); and Terminal High Altitude  Area Defense (THAAD). These elements collectively account for about 77  percent of MDA’s research and development budget. In assessing BMDS  progress, we examined documents such as Program Execution Reviews,  test plans and reports, production plans, and Contract Performance  Reports. We also interviewed officials within program offices and within  MDA functional directorates, such as the Directorate for System  Engineering. In addition, we discussed each element’s test program and its  results with DOD’s Office of the Director, Operational Test and Evaluation.
In following up on transparency, accountability, and oversight issues  raised in our March 2007 report, we held discussions with officials in  MDA’s Directorate of Business Operations to determine whether its new  block structure improved accountability and transparency of the BMDS.   In addition, we reviewed pertinent sections of the U.S. Code to compare  MDA’s current level of accountability with federal acquisition laws. We  also interviewed officials from the Office of the Under Secretary of  Defense for Acquisition, Technology, and Logistics and DOD’s Joint Staff  to discuss the oversight role of the new Missile Defense Executive Board.  Additionally, we reviewed the Board’s charter to identify its oversight  responsibility.  Our scope and methodology is discussed in more detail in  appendix IV.
We conducted this performance audit from May 2007 to March 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: The Missile Defense Agency’s mission is to develop an integrated and  layered BMDS to defend the United States, its deployed forces, allies, and  friends. The BMDS is expected to be capable of engaging all ranges of  enemy ballistic missiles in all phases of flight. This is a challenging  expectation, requiring a complex combination of defensive components— space-based sensors, surveillance and tracking radars, advanced  interceptors, and a battle management, command, control, and  communications component—that work together as an integrated system.
A typical scenario to engage an intercontinental ballistic missile (ICBM)  would unfold as follows:  Infrared sensors aboard early-warning satellites detect the hot plume of  a missile launch and alert the command authority of a possible attack.
Upon receiving the alert, land- or sea-based radars are directed to track  the various objects released from the missile and, if so designed, to  identify the warhead from among spent rocket motors, decoys, and  debris.
When the trajectory of the missile’s warhead has been adequately  established, an interceptor—consisting of a kill vehicle mounted atop a  booster—is launched to engage the threat. The interceptor boosts itself  toward a predicted intercept point and releases the kill vehicle.
The kill vehicle uses its onboard sensors and divert thrusters to detect,  identify, and steer itself into the warhead. With a combined closing  speed on the order of 10 kilometers per second (22,000 miles per hour),  the warhead is destroyed above the atmosphere through a “hit to kill”  collision with the kill vehicle.
To develop a system capable of carrying out such an engagement, MDA,  until December 2007, executed an acquisition strategy in which the  development of missile defense capabilities was organized in 2-year  increments known as blocks. Each block was intended to provide the  BMDS with capabilities that enhanced the development and overall  performance of the system. The first 2-year block—Block 2004—fielded a  limited initial capability that included early versions of the GMD, Aegis  BMD, Patriot Advanced Capability-3, and C2BMC elements. The agency’s  second 2-year block—Block 2006—culminated on December 31, 2007, and  fielded additional BMDS assets. Block 2006 also continued the evolution of  Block 2004 by providing improved GMD interceptors, enhanced Aegis  BMD missiles, upgraded Aegis BMD ships, a Forward-Based X-Band- Transportable radar, and enhancements to C2BMC software.  On  December 7, 2007, MDA’s Director approved a new block construct that  will be the basis for all future development and fielding.  Table 1 provides  a brief description of all elements currently being developed by MDA.
section_title: More Capability Fielded but Less than Planned and at Higher Cost
paragraphs: MDA made progress in developing and fielding the BMDS during 2007.  Additional assets were fielded and/or upgraded, several tests met planned  objectives, and other development activities were conducted. On the other  hand, fewer assets were fielded than originally planned, the cost of the  block increased, some flight tests were deferred, and the performance of  fielded assets could not be fully evaluated.
subsections: section_title: Block 2006 Improves BMDS Capability
paragraphs: During Block 2006, MDA increased its inventory of BMDS assets while  enhancing the system’s performance. The agency fielded 14 additional  Ground-based interceptors, 12 Aegis BMD missiles designed to engage  more advanced threats, 4 new Aegis BMD destroyers, 1 new Aegis BMD  cruiser, as well as 8 C2BMC Web browsers and 1 C2BMC suite. In addition,  MDA upgraded half of its Aegis BMD ship fleet, successfully conducted  four Aegis BMD and two GMD intercept tests, and completed a number of  ground tests to demonstrate the capability of BMDS components.  Considering assets fielded during Blocks 2004 and 2006, MDA, by  December 31, 2007, had cumulatively fielded a total of 24 Ground-based  interceptors, 2 upgraded early-warning radars, an upgraded Cobra Dane  surveillance radar, 1 Sea-based X-band radar, 2 Forward-Based X-Band  Transportable radars, 21 Aegis BMD missiles, 14 Aegis BMD destroyers,  and 3 Aegis BMD cruisers.  In addition, MDA had fielded 6 C2BMC suites;  46 warfighter enterprise workstations with situational awareness; BMDS  planner and sensor management capabilities; 31 C2BMC Web browsers, 13  with laptop planners; and redundant communications node equipment to  connect BMDS elements worldwide.
section_title: MDA Does Not Meet Original Goals, but Meets or Exceeds Most Revised Goals
paragraphs: In March 2005, MDA submitted to Congress the number of assets it  planned to field during Block 2006. However, increasing costs, technical  challenges, and schedule delays prompted the agency to reduce the  quantity of planned assets. Consequently, in March 2006, shortly after  submitting its fiscal year 2007 budget, MDA notified Congress that it was  revising its Block 2006 Fielded Configuration Baseline. Although MDA did  not meet its original block fielding goals, it was able in nearly all instances  to meet or exceed its revised goals. Of the four elements delivering assets  during Block 2006, one—Sensors—was able to meet its original goal.  However, two elements—GMD and C2BMC—were able to exceed their  revised fielding goals. Table 2 depicts the goals and the number of assets  fielded.
Although GMD did not meet its original goal of fielding up to 15  interceptors and partially upgrading the Thule early warning radar, the  element was able to surpass its revised goal of fielding 12 interceptors.  By  December 31, 2007, the GMD element fielded 14 interceptors—2 more than  planned.  To achieve its revised goal, the element’s prime contractor added  a manufacturing shift during 2007 and extended the number of hours that  certain shifts’ personnel worked. These actions allowed the contractor to  more than double its interceptor emplacement rate.
Last year, we reported that MDA delayed the partial upgrade of the Thule  early-warning radar—one of GMD’s original goals—until a full upgrade  could be accomplished. According to DOD, the full upgrade of Thule is  the most economical option and it meets DOD’s desire to retain a single  configuration of upgraded early warning radars. The Thule early warning  radar upgrade is being accomplished by two separate contract awards.   Raytheon was awarded a contract in April 2006 to develop and install  prime mission equipment; while Boeing was expected to receive a contract  in January 2008 to integrate the equipment into the BMDS ground  communication network.
In March 2005, MDA included three C2BMC suites as part of its fielding  goal for Block 2006. These suites were to be fielded at U.S. European  Command, U.S. Central Command, and another location that was to be  identified later. Faced with a $30 million reduction in C2BMC’s fiscal year  2006 budget, MDA in March 2006 revised this goal to replace the 3 suites  with 3 less expensive Web browsers. However, by the end of Block 2006,  MDA found an innovative way to increase combatant commands’  situational awareness and planning capability. In 2005, the C2BMC  program conducted a network load analysis and concluded that situational  awareness and planning capability—equivalent to that provided by a suite- -could be gained by combining Web browsers and planners. To prove that  this approach would work, MDA fielded 4 Web browsers and one planner  at the U.S. European Command. MDA learned that this combination of  hardware, fielded in the quantities needed to meet a command’s needs and  connected to an existing server, provided the situational awareness and  planning capability of a suite at less cost. MDA extended this approach by  fielding one Web browser and one planner at four other locations—U.S.  Forces Japan; U.S. Forces Korea; the Commander of U.S. Strategic  Command; and the Commander of the Space and Missile Defense  Command. In addition, MDA fielded one suite at U.S. Pacific Command.
The Aegis BMD element was able to meet its revised block goals for only  one of its two components. The program upgraded all planned ships, but  fielded three fewer Aegis BMD Standard Missile-3s (SM-3) than planned.  The program did not meet its revised missile goal because three U.S  missiles were delayed into 2008 to accommodate an unanticipated  requirement to deliver three missiles to Japan. Figure 1 below depicts the  location of current BMDS assets.
section_title: Block Costs Have Increased, but Full Cost Is Not Known
paragraphs: MDA’s Block 2006 program of work culminated with higher than  anticipated costs. In March 2007, we reported that MDA’s cost goal for  Block 2006 increased by approximately $1 billion because of greater than  expected GMD operations and sustainment costs and technical problems.  During fiscal year 2007, some prime contractors performing work for the  BMDS overran their budgeted costs. To stay within its revised budget,  MDA was forced to reduce the amount of work it expected to accomplish  during the block. The full cost of the block cannot be determined because  of the deferral of work from one block to another. In addition, some MDA  prime contractors too often employ a planning methodology that has the  potential to obscure the time and money that will be needed to produce  the outcomes intended. If the work does not yield the intended results,  MDA could incur additional future costs. While MDA struggled to contain  costs during Block 2006, the agency awarded two contractors a large  percentage of available fee for performance in cost and/or program  management although the contractor-reported data showed declining cost  and schedule performance. Both award fee plans for these contractors  direct that cost and schedule performance be considered as factors in  making the evaluation. While these factors are important, MDA’s award fee  plans provide for the consideration of many other factors in making award  fee determinations.
subsections: section_title: Five Contractors Overran 2007 Budgets
paragraphs: To determine if contractors are executing the work planned within the  funds and time budgeted, each BMDS program office requires its prime  contractor to provide monthly Earned Value Management reports detailing  cost and schedule performance. If more work was completed than  scheduled and the cost of the work performed was less than budgeted, the  contractor reports a positive schedule and cost variance. However, if the  contractor was unable to complete all of the work scheduled and needed  more funds to complete the work than budgeted, the contractor reports a  negative schedule and cost variance. Of course, the results can be mixed.
That is, the contractor may have completed more work than scheduled but  at a cost that exceeded the budget.
As shown in table 3 below, the contractors for the nine BMDS elements  collectively overran their fiscal year 2007 budgets by approximately $166  million.  We estimate that at completion, the cumulative overrun in the  contracts could be between about $1.3 billion and $1.9 billion. Our  predictions of final contract costs were developed using formulas  accepted within the cost community and were based on the assumption  that the contractor will continue to perform in the future as it has in the  past. It should also be noted that some contracts include more than Block  2006 work. For example, the STSS contract includes work being  accomplished in anticipation of future blocks. Our analysis is presented in  table 3 below. Appendix II provides further details on the cost and  schedule performance of the contractors outlined in the table.
Technical problems and software issues caused several BMDS elements to  overrun their fiscal year 2007 budgeted costs. In addition, 4 of the 10  contracts we reviewed contained some kind of replanning activity during  fiscal year 2007 and the ABL contract was partially rebaselined.  Contractors may replan when they conclude that the current plan for  completing the effort remaining on the contract is unrealistic. A replan can  include reallocating the remaining budget over the rest of the work,  realigning the schedule within the contractually defined milestones, and  setting either cost or schedule variances to zero or setting both to zero. A  rebaseline is similar, but it may also add additional time and/or funding for  the remaining work.
The ABL contractor was overrunning both its fiscal year 2007 budget and  schedule early in the year. Although by year’s end it appears that the  contractor recovered, the contractor would have continued to overrun  both its budget and its schedule if most of the contract had not been  rebaselined. The contractor realized cost and schedule growth as it  worked to solve software integration problems in the Beam Control/Fire  Control component and dealt with a low-power laser needed for flight  tests that was not putting enough energy on the target. After encountering  these problems, the ABL contractor did not have sufficient schedule or  budget to complete the remaining contract work. Therefore, in May 2007,  the program allowed the contractor to rebaseline all of the remaining work  devoted to developing, integrating, flight testing, and delivering the ABL  prototype. The rebaselining effort added about $253 million to the contract  and extended the contract’s period of performance by more than a year.
The THAAD prime contractor’s cost overrun of $91.1 million was primarily  caused by technical problems related to the element’s missile, launcher,  radar, and test components. Missile component cost overruns were caused  by higher than anticipated costs in hardware fabrication, assembly, and  support touch labor for structures, propulsion, and other subassembly  components. Additionally, design issues with the launcher’s missile round  pallet and the electronics assembly that controls the launcher caused the  contractor to experience higher than anticipated labor and material costs.  The radar component ended the fiscal year with a negative cost variance  as more staff was required than planned to resolve hardware design issues  in the radar’s prime power unit. The contractor also experienced negative  cost variances with the system test component because the Launch and  Test Support Equipment required additional set-up time at the flight test  range.
The STSS contractor’s $67.7 million fiscal year 2007 cost variance is  primarily attributed to problems that occurred during thermal vacuum  testing of the first satellite. Since the satellites are legacy hardware built  under a former program, there are no spares available for testing. As a  result, the contractor needed to handle the parts carefully to avoid damage  to the hardware, increasing the time devoted to the test. Further test  delays occurred when a number of interface issues surfaced during testing  and when the cause of component problems could not be easily traced to  their source. The program office believes that the cost variance would  have been less if design engineers had been available during testing.  Because engineers were not present to quickly identify the cause of  component problems, a time-consuming analysis of each problem was  needed.
section_title: Total Block 2006 Costs Cannot be Determined
paragraphs: In March 2007, we reported that a full accounting of Block 2006 costs was  not possible because MDA has the flexibility to redefine block outcomes.  That is, MDA can delay the delivery of assets or other work activities from  block to block and count the work as a cost of the block during which the  work is performed, even though the work does not benefit that block. For  example, MDA deferred some Block 2004 work until Block 2006 so that it  could use the funds appropriated for that work to cover unexpected cost  increases caused by technical problems recognized during development,  testing, and production. With the deferral of the work, its cost was no  longer counted as a Block 2004 cost, but as a Block 2006 cost. As a result,  Block 2004’s cost was understated and Block 2006’s cost is overstated.  Because MDA did not track the cost of the deferred work, the agency  could not make an adjustment that would have matched the cost with the  correct block.
The cost of Block 2006 was further blurred as MDA found it necessary to  defer some Block 2006 work until a future block.  For example, when the  STSS contractor overran its fiscal year 2007 budget because of testing  problems, the program did not have sufficient funds to launch the  demonstration satellites in 2007 as planned.  The work is now scheduled  for 2008.  The consequence of deferring Block 2004 work to Block 2006  and Block 2006 to 2008 is that the full cost of Block 2006 cannot be  determined.
section_title: Contractors’ Planning Methodology Has the Potential to Obscure Block Cost
paragraphs: Some MDA prime contractors too often employ a planning methodology  that has the potential to obscure the time and money that will be needed  to produce the outcomes intended. Contractors typically divide the total  work of a contract into small efforts in order to define them more clearly  and to ensure proper oversight. Work may be planned in categories  including (1) level of effort (LOE) —work that contains tasks of a general  or supportive nature and do not produce a definite end product—or (2)  discrete work—work that has a definable end product or event.
Level of effort work assumes that if the staff assigned to the effort spend  the planned length of time, they will attain the outcome expected.   According to earned value experts and the National Defense Industrial  Association, while it is appropriate to plan such tasks as supervision or  contract administration as LOE, it is not appropriate to plan tasks that are  intended to result in a product, such as a study or a software build, as LOE  because contractors do not report schedule variances for LOE work.  Therefore, when contractors incorrectly plan discrete work as LOE,  reports that are meant to allow the government to assess contractor cost  and schedule performance may be positive, but the government may not  have full insight into the contractor’s progress. The greater the percentage  of LOE, the weaker the link between inputs (time and money) and  outcomes (end products), which is the essence of earned value analysis.
Essentially, depending on the magnitude of LOE, schedule variances at the  bottom line can be understated. The significant amount of BMDS work  being tracked by LOE may have limited our assessment of the contractors’  performance. That is, the contractor’s performance may appear to be more  positive than it would be if work had been correctly planned. In such  cases, the government may have to expend additional time and money to  achieve the outcomes desired. MDA Earned Value Management officials  agreed that some BMDS prime contractors incorrectly planned discrete  work as LOE, but the agency is taking steps to remedy this situation so  that they can better monitor the contractors’ performance.
While it is not possible to state with certainty how much work a contractor  should plan as LOE, experts within the government cost community, such  as Defense Contract Management Agency officials, agree that LOE levels  over 20 percent warrant investigation. According to MDA, many of its  prime contractors plan a much larger percentage than 20 percent of their  work as LOE. Table 4 presents the percentage of work in each BMDS  prime contract that is categorized as LOE.
The Aegis BMD SM-3, MKV, ABL, and C2BMC contractors planned more  than half of certain work as LOE. In several instances, MDA Earned Value  Management officials and program office reviewers agreed that some of  the LOE work could be redefined into discrete work packages. For  example, from January through December 2007, the C2BMC contractor  planned 73 percent of its work as LOE. This included activities such as  software development and integration and test activities that result in two  definable products—software packages and tests. At the direction of the  C2BMC Program Office, the C2BMC contractor redefined some contract  work, including software development and integration and test activities,  as discrete, reducing the amount of LOE on the contract to 52 percent.
The Aegis BMD element also reported a high percentage of LOE for its  Standard Missile-3 contract, particularly considering that its products— individual missiles—are quite discrete. In August 2007, the element  reported that the contractor had planned 73 percent of the contract work  as LOE. The portion of the work that contained this amount of LOE was  completed in March 2007 with an underrun of $7.2 million. Although the  contractor reported an underrun for this work upon its completion, the  high percentage of LOE may have, over the contract period, distorted the  contractor’s actual cost and schedule performance. Similarly, it is  important to note that the amount of LOE for the SM-3 work that is  currently ongoing is considerably less. Program officials told us that prior  to the commencement of this segment of work, the MDA Earned Value  Management Group and program officials recommended that the program  minimize the amount of LOE on its contracts.  Currently, only 18 percent  of the SM-3 contract is considered LOE.
section_title: Two Contractors Awarded Fee That Does Not Reflect Earned Value Performance
paragraphs: MDA uses award fees to encourage its contractors to perform in an  innovative, efficient, and effective way in areas considered important to  the development of the BMDS. Because award fees are intended to  motivate contractor performance for work that is neither feasible nor  effective to measure objectively, award fee criteria and evaluations tend to  be subjective. Each element’s contract has an award fee plan that  identifies the performance areas to be evaluated and the methodology by  which those areas will be assessed. An award fee evaluation board—made  up of MDA personnel, program officials, and officials from key  organizations knowledgeable about the award fee evaluation areas–– judges the contractor’s performance against specified criteria in the award  fee plan. The board then recommends to a fee determining official the  amount of fee to be paid.  MDA’s Director is the fee-determining official  for all BMDS prime contracts that we assessed.
During fiscal year 2007, MDA awarded approximately 95 percent, or $606  million, of available award fee to its prime contractors. While the cost,  schedule, and technical performance of several contractors appeared to be  aligned with their award fee, two contractors were rated as performing  very well in the cost and/or program management elements and received  commensurate fees even though earned value management data showed  that their cost and schedule performance was declining.  On the other  hand, MDA did not award any fee to the THAAD contractor for its  management of contract cost during a time when earned value data  showed steadily increasing costs. Although DOD guidance discourages the  use of earned value performance metrics in award fee criteria, MDA  includes this as a factor in several of its award fee plans. The agency  considers many factors in rating contractors’ performance and making  award fee determinations, including consideration of earned value data  that shows cost, schedule, and technical trends.  In addition, MDA has  begun to revise its award fee policy to align agency practices more closely  with DOD’s current policy that better links performance with award fees.
The ABL and Aegis BMD weapon system contractors received a large  percentage of the 2007 award fee available to them for the cost and/or  program management element. MDA rated the ABL contractor’s  performance in cost and program management elements as “very good,”  awarding the contractor 88 percent of the fee available in these  performance areas. According to the award fee plan, one of several factors  that is considered in rating the contractor’s performance as very good is  whether earned value data indicates that there are few unfavorable cost,  schedule, and/or technical variances or trends. During the February 2006  to January 2007 award fee period, earned value data shows that the  contractor overran its budget by more than $57 million and did not  complete $11 million of planned work. Similarly, the Aegis BMD weapon  system contractor was to be rated as to how effectively it managed its  contract’s cost. The award fee plan for this contractor also directs that  earned value be one of the factors considered in making such an  evaluation. During the fee period that ran from October 2006 through  March 2007, MDA rated the contractor’s cost management performance as  outstanding and awarded 100 percent of the available fee. Earned value  data during this time period indicates that the contractor overran its  budget by more than $6 million. MDA did not provide us with more  detailed information as to other factors that may have influenced its  decision as to the amount of fee awarded to the ABL and Aegis BMD  Weapon System contractors.
MDA recognizes that there is not always a good link between the agency’s  intentions for award fees and the amount of fee being earned by its  contractors. In an effort to rectify this problem, the agency released a  revised award fee policy in February 2007 to ensure its compliance with  recent DOD policies that are intended to address award fee issues  throughout the Department. Specifically, MDA’s policy directs that every  contract’s award fee plan include:    Criteria for each element of the award fee that is specific enough to  enable the agency to evaluate contractor performance and to  determine how much fee the contractor can earn for that element. The  criteria is to clearly define the performance that the government  expects from the contractor for the applicable award fee period and  the criteria for any one element must be distinguishable from criteria  for other elements of the award fee;    An emphasis on rewarding results rather than effort or activity; and    An incentive to meet or exceed agency requirements.
Additionally, MDA’s policy calls for using the Award Fee Advisory Board  to not only make award fee recommendations to the fee determining  official, but to also biannually report to MDA’s Director as to whether  award fee recommendations are consistent with DOD’s Contractor  Performance Assessment Report—a report that provides a record, both  positive and negative, on a given contract for a specific period of time.  Appendix II of this report provides additional information on BMDS prime  contracts and award fees.
section_title: Most Test Objectives Achieved Despite Schedule Delays
paragraphs: During 2007, several BMDS programs experienced setbacks in their test  schedules.  The Aegis BMD, THAAD, ABL, STSS, and C2BMC elements  experienced test delays, but all were able to achieve their primary test  objectives. GMD, on the other hand, experienced a schedule delay caused  by an in-flight target anomaly that prevented full accomplishment of one  major 2007 test objective. The remaining three elements—MKV, KEI, and  Sensors—were able to execute all scheduled activities as planned.
subsections: section_title: Several Elements Met Key Test Objectives
paragraphs: The Aegis BMD, THAAD, C2BMC, ABL, and STSS elements continued to  achieve important test objectives in 2007, although some tests were  delayed. Aegis BMD proved its capability against more advanced threats,  while THAAD proved that it could intercept both inside and outside of the  atmosphere. C2BMC completed a number of software and system-level  tests. The ABL and STSS programs saw delays in important ground tests,  but ABL was able to begin flight testing its beam control/fire control  component using a low-power laser in 2007 and STSS completed thermal  vacuum testing of both satellites by the end of the year. However, the  delays in the ABL and STSS programs may hold up their incorporation into  the BMDS during future blocks.
Although the Aegis BMD program encountered some test delays, it was  able to achieve all fiscal year 2007 test objectives. In December 2006, the  program stopped a test after a crew member changed the ship’s doctrine  parameters just prior to target launch, preventing the ship’s fire control  system from conducting the planned engagement. During this test event,  the weapon system failed to recognize the test target as a threat, which  prevented the SM-3 missile from launching. Also, according to program  officials, the system did not provide a warning message which contributed  to the mission being aborted prematurely and prevented the Aegis BMD  program from meeting its test objectives. However, 4 months later, the  same flight test event was successfully completed and all test objectives  were met. During that event, the program was able to demonstrate that the  Aegis BMD could simultaneously track and intercept a ballistic missile and  an anti-ship cruise missile. In June 2007, the program successfully  completed its first flight test utilizing an Aegis BMD destroyer to intercept  a separating target, and in November, the program conducted its first test  that engaged two ballistic missile targets simultaneously. During the last  test, Aegis missiles onboard an Aegis BMD cruiser successfully intercepted  two short-range non-separating targets and achieved all primary test  objectives outlined for this event.
The THAAD program expected to complete four flight tests prior to the  end of fiscal year 2007 but was only able to complete three.  Two tests  successfully resulted in intercepts of short-range ballistic missiles at  different levels of the atmosphere.  The third test successfully  demonstrated component capability in a high-pressure environment and  was the lowest altitude interceptor verification test to date.  However, the  fourth test was delayed, initially due to target availability driven by late  modifications to the target hardware configuration.  Additionally, during  pre-flight testing, the contractor found debris in the interceptor. This  caused the interceptor to be returned to the factory for problem  investigation. While the problem was corrected and the interceptor was  returned to the test range in only 7 days, the test was rescheduled because  the test range was not available before the end of fiscal year 2007.
During fiscal year 2007, the C2BMC program completed BMDS-level  ground and flight tests, successfully achieving its test objectives of  verifying the capabilities and readiness of a new software configuration.   The software is designed to provide the BMDS with improved defense  planning capability, including better accuracy and speed; a new  operational network; and additional user displays.  Because of the integral  nature of the C2BMC product, problems encountered in some elements’  test schedules have a cascading effect on C2BMC’s test schedule.  Even  though this limited C2BMC testing, a review of the integrated and  distributed ground test data resulted in the decision to field the software  in December 2007.
ABL achieved most of its test objectives during fiscal year 2007, but  experienced delays during Block 2006 that deferred future BMDS program  decisions. The program experienced a number of technical problems  during fiscal year 2006 that pushed some planned activities into fiscal year  2007. One such activity was the execution of the program’s first of four key  knowledge points—a ground test to demonstrate ABL’s ability to acquire  and track a target while performing atmospheric compensation. The test  was conducted in December 2006, 3 ½ months later than planned. At the  culmination of the test, program officials noted two problems. First, the  system’s beam control/fire control software was not integrated as  anticipated. In addition, the energy that the low-power laser placed on the  target during the test was not optimal. According to program officials,  both of these issues were resolved before the system began flight testing  the full beam control/fire control component in February 2007. However,  the delays caused the program to further postpone a key lethality  demonstration—a demonstration in which the ABL will attempt to shoot  down a short-range ballistic missile—until last quarter of fiscal year 2009.  This demonstration is important to the program because it is the point at  which MDA will decide the program’s future.
Although the ABL program experienced some setbacks with its first key  knowledge point, it was able to meet all objectives for each subsequent  knowledge point. In addition to the first knowledge point, the program  planned to demonstrate three additional knowledge points during fiscal  year 2007. The second knowledge point was contingent upon completion  of the first. To demonstrate the achievement of the two knowledge points,  the contractor performed a flight test that showed the low-power laser  was integrated and the beam control/fire control functioned sufficiently to  perform target tracking and atmospheric compensation against an  airborne target board. The third knowledge point was completed three  months ahead of the planned 2007 schedule and demonstrated that ABL’s  optical subsystem was adequate to support its high-power laser system.  The fourth knowledge point–the completion of a series of flight tests to  demonstrate the performance of the low-power laser system in flight—was  completed in August 2007.
Delays in the STSS test program, along with funding shortages, postponed  the planned 2007 launch of the program’s demonstration satellites. The  STSS program is integrating two demonstration satellites with sensor  payloads from legacy hardware developed under a former program. The  use of legacy hardware has complicated the test program because spares  needed for testing are not available. In order to preserve the condition of  the legacy components, the program must exercise caution in handling the  components to prevent damage, which has caused delays in testing.  Additionally, a thermal vacuum test on the first space vehicle, to assess the  ability of the satellite to operate in the cold vacuum of space, took twice as  long as scheduled, due to a number of interface issues. Although the  program was able to complete the integration and test of both  demonstration satellites in 2007—major objectives for the program—funds  were not available to launch the satellites as planned. Program officials  believe that the satellites could be launched as early as April 2008 and as  late as July 2008, 1 year later than originally scheduled. According to the  program office, there is no margin in the 2008 budget, so any unexpected  issues could put the 2008 launch date at risk. The delays in launching the  STSS demonstration satellites do not impact MDA’s Block 2006 fielding  plans as the satellites are intended to demonstrate a surveillance and  tracking capability and do not provide any operational capability during  the block. However, the delay in launching the demonstration satellites is  causing a delay in MDA’s ability to initiate development of an operational  constellation, which may delay a BMDS global midcourse tracking  capability.
Despite delays in hardware and software testing and integration, other  parts of the STSS program have proceeded according to schedule. Lessons  learned from the thermal vacuum test for the first satellite’s sensor  payload facilitated the completion of thermal vacuum testing of the second  satellite’s payload in November 2007. Additionally, command and control  capabilities of the ground segment were demonstrated and the second part  of the acceptance test of STSS ground components was completed in  September 2007.
section_title: Test Delays Prevent GMD from Achieving All 2007 Objectives
paragraphs: A target anomaly prevented the GMD element from achieving all 2007  objectives. The GMD program planned to conduct three flight tests–two  intercept attempts and one radar characterization test—but was only able  to conduct the radar test and one intercept test. The radar characterization  test was conducted in March 2007. The target was launched from  Vandenberg Air Force Base and was successfully tracked by the SBX radar  and the radar of two Aegis BMD ships. During the test, officials indicated  the SBX exhibited some anomalous behavior, yet was able to collect target  tracking data and successfully transmit the information to the C2BMC  element and the GMD fire control system at DOD’s Missile Defense  Integration and Operations Center. No live interceptor was launched.
However, an intercept solution was generated and simulated interceptor  missiles were “launched” from Fort Greely, Alaska. To address anomalous  behavior, MDA adjusted software and performance parameters of the SBX  radar.
In May 2007, the program attempted an intercept test, but a key  component of the target malfunctioned. For that reason, the weapon  system did not release the Ground-based interceptor and program officials  declared the flight test a “no test” event. To date, program officials have  not determined the root cause of the malfunction. In September 2007, the  program successfully conducted a re-test and achieved an intercept of the  target using target tracking data provided by the Beale upgraded early  warning radar. MDA test officials told us that aging target inventory could  have contributed to the target anomaly. The officials explained that some  targets in MDA’s inventory are more than 40 years old and their reliability  is relatively low. Target officials told us that they are taking preventive  actions to avoid similar anomalies in the future.
The time needed to complete the first 2007 intercept delayed GMD’s  second planned intercept attempt until at least the second quarter of fiscal  year 2008. The delayed test was to have determined whether the SBX radar  could provide data in “real time” that could be used by the GMD fire  control component to develop a weapon task plan. Although the weapon  task plan was not developed in real time during 2007, GMD was able to  demonstrate that the SBX radar could plan an engagement when the target  was live but the interceptor was simulated.
section_title: Three BMDS Elements Completed Activities as Planned
paragraphs: During 2007, the KEI program redefined its development efforts and  focused on near-term objectives. Also, the MKV program redefined its  strategy to acquire multiple kill capability.  Once redefined, these  programs conducted all planned activities as scheduled and each was able  to meet all planned objectives. In addition, the Sensors program  successfully completed all planned tests.
In June 2007, MDA directed the KEI program to focus on two near-term  objectives—the development of its booster and its 2008 booster flight test.  Some work, such as development of the fire control and communications  and mobile launcher, was deferred into the future. During fiscal year 2007,  the KEI program conducted all planned test activities, including booster  static fire tests that demonstrated the rocket motor’s performance in  induced environments and wind tunnel tests that gathered data to validate  aerodynamic models for the booster flight controls.
MKV officials redefined their acquisition strategy by employing a parallel  path to develop multiple kill vehicles for the GMD and KEI interceptors  and the Aegis BMD SM-3 missile. MDA initiated the MKV program in 2004  with Lockheed Martin.  In 2007, the MKV program added Raytheon as a  second payload provider.  According to program officials, the two payload  providers may use different technologies and design approaches, but both  adhere to the agency’s goal of delivering common, modular MKV payloads  for integration with all BMDS midcourse interceptors.  In fiscal year 2007,  Lockheed Martin successfully conducted static fire tests of its Divert  Attitude Control System as planned. Additionally, Raytheon, funded with  excess KEI funds made available when that program was replanned, began  concept development.  Raytheon did not have any major test activities  scheduled for the fiscal year.
During 2007, the Sensors program focused on testing FBX-T radars that  were permanently emplaced and newly produced. After the first FBX-T  was moved from its temporary location in Japan to its permanent location  in Shariki, Japan, various ground tests and simulations were conducted to  ensure its interoperability with the BMDS. The program also delivered a  second FBX-T to Vandenberg Air Force Base, where its tracking capability  is being tested against targets of opportunity.  According to program  officials, a decision has not been made as to where the second FBX-T  radar will be permanently located.
section_title: BMDS Achievement of Performance Goals Remains Uncertain
paragraphs: As we reported in March 2007, MDA altered its original Block 2006  performance goals commensurate with the agency’s reductions in the  delivery of fielded assets. However, insufficient data exists to fully assess  whether MDA achieved its revised performance goals. The performance of  some fielded assets is also questionable because parts have not yet been  replaced that were identified by auditors in MDA’s Office of Quality,  Safety, and Mission Assurance as less reliable or inappropriate for use in  space. In addition, tests of the GMD element have not included target suite  dynamic features and intercept geometries representative of the  operational environment in which GMD will perform its mission and  BMDS tests only allow a partial assessment of the system’s effectiveness,  suitability, and survivability.
MDA uses a combination of simulations and flight tests to determine  whether performance goals are met. Models and simulations are needed to  predict performance because the cost of tests prevents the agency from  conducting sufficient testing to compute statistical probabilities of  performance. The models and simulations that project BMDS capability  against intercontinental ballistic missiles present several problems. First,  the models and simulations that predict performance of the GMD element  have not been accredited by an independent agency. According to the  Office of the Director, Operational Test and Evaluation without accredited  models, GMD’s performance cannot be predicted with respect to (1)  variations in threat parameters that lie within the bounds of intelligence  estimates, (2) stressing ground-based interceptor fly-outs and  exoatmospheric kill vehicle engagements, and (3) variations in natural  environments that lie within meteorological norms. Second, too few flight  tests have been completed to ensure the accuracy of the models’ and  simulations’ predictions. Since 2002, MDA has only completed two end-to- end tests of engagement sequences that the GMD element might carry out.  While these tests provide some evidence that the element can work as  intended, MDA must test other engagement sequences, which would  include other GMD assets that have not yet participated in an end-to-end  flight test. For example, MDA has not yet used the Sea-based X-band radar  as the primary sensor in an end-to-end test. Additionally, officials in the  Office of the Director, Operational Test and Evaluation told us that MDA  needs more flight tests to have a high level of confidence that GMD can  repeatedly intercept incoming ICBMs. Further testing is also needed to  demonstrate that Aegis BMD can provide real-time, long-range  surveillance and tracking data for the GMD element. In March 2006, we  reported that the cancellation of a GMD flight test prevented MDA from  exercising Aegis BMD’s long-range surveillance and tracking capability in  a manner consistent with an actual defensive mission. Program officials  informed us that the Aegis BMD is capable of performing this function and  has demonstrated its ability to surveil and track ICBMs in several  exercises. However, MDA has not yet shown that Aegis BMD can  communicate this data to GMD during a live intercept engagement and  that GMD can use the data to prepare a weapon task plan for actual— rather than simulated––interceptors. Officials in the Office of the Director  for Operational Test and Evaluation told us that having Aegis BMD  perform long-range surveillance and tracking during a live engagement  would provide the data needed to more accurately gauge performance.
Similarly, MDA has not yet proved that the FBX-T radar can provide real- time, long-range surveillance and tracking data for the GMD element. On  several occasions, MDA has shown that the FBX-T can acquire and track  targets of opportunity, but the radar’s data has not yet been used to  develop a weapon system task plan for a GMD intercept engagement.  Because the radar’s permanent location in Japan does not allow MDA to  conduct tests in which the FBX-T is GMD’s primary fire control radar, the  Director, Operational Test and Evaluation, in 2006 recommended that  prior to emplacing a second FBX-T at its permanent location that MDA test  the radar’s capability to act as GMD’s primary sensor in an intercept test.
Confidence in the performance of the BMDS is also reduced because of  unresolved GMD technical and quality issues. The GMD element has  experienced the same anomaly during each of its flight tests since 2001.  This anomaly has not yet prevented the program from achieving any of its  primary test objectives, but to date neither its source nor solution has  been clearly identified or defined. Program officials plan to continue their  assessment of test data to identify the anomaly’s root cause and have  implemented design changes to mitigate the effects and reduce risks  associated with the anomaly. The reliability of emplaced GMD interceptors  raises further questions about the performance of the BMDS. Quality  issues discovered by auditors in MDA’s Office of Quality, Safety, and  Mission Assurance nearly 3 years ago have not yet been rectified in all  fielded interceptors. According to the auditors, inadequate mission  assurance and quality control procedures may have allowed less reliable  parts or parts inappropriate for use in space to be incorporated into the  manufacturing process, thereby limiting the reliability and performance of  some fielded assets. The program has strengthened its quality control  processes and is taking several steps to mitigate similar risks in the future.   These steps include component analysis of failed items, implementing  corrective action with vendors, and analyzing system operational data to  determine which parts are affecting weapon system availability.  MDA has  begun to replace the questionable parts in the manufacturing process and  to purchase the parts that it plans to replace in fielded interceptors.   However, it will not complete the retrofit effort until 2012.
Additionally, test officials told us that although the end-to-end GMD test  conducted during 2007 demonstrated that for a single engagement  sequence military operators could successfully engage a target, the target  represented a relatively unsophisticated threat because it lacked specific  target suite dynamic features and intercept geometry.  Other aspects of the  test were more realistic—such as closing velocity and fly-out range—but  these were relatively unchallenging.  While the test parameters may be  acceptable in a developmental test, they are not fully representative of an  operational environment and do not provide high confidence that GMD  will perform well operationally.
Finally, because BMDS assets are being fielded based on developmental  tests, which are not always representative of the operational environment,  operational test officials have limited test data to determine whether all  BMDS elements/components being fielded are effective and suitable for  and survivable on the battlefield.  MDA has added operational test  objectives to its developmental test program, but many of the objectives  are aimed at proving that military personnel can operate the equipment.  In  addition, limited flight test data is available for characterizing the BMDS’  capability against intercontinental ballistic missiles.  Up until 2007, the  overall lack of data limited the Office of the Director of Operational Test  and Evaluation, in annual assessments, to commenting on the operational  realism of tests and recommending other tests needed to characterize  system effectiveness and suitability.  In 2007, tests provided sufficient  information to partially quantify the effectiveness and suitability of the  BMDS' midcourse capability (Aegis BMD and GMD) and to fully  characterize a limited portion of the BMDS' terminal capability (PAC-3).   However, according to the Office of the Director of Operational Test and  Evaluation, further testing that incorporates realistic operational  objectives and verification, validation, and accreditation of models and  simulations will be needed before the performance, suitability, and  survivability of the BMDS can be fully characterized.
section_title: Efforts Underway to Improve BMDS Management and Oversight
paragraphs: Since its initiation in 2002, MDA has been given a significant amount of  flexibility in executing the development of the BMDS. While the flexibility  has enabled MDA to be agile in decision making and to field an initial  capability relatively quickly, it has diluted transparency into MDA’s  acquisition processes, making it difficult to conduct oversight and hold the  agency accountable for its planned outcomes and costs. As we reported in  2007, MDA operates with considerable autonomy to change goals and  plans, which makes it difficult to reconcile outcomes with original  expectations and to determine the actual cost of each block and of  individual operational assets. In the past year, MDA has begun  implementing two initiatives—a new block construct and a new executive  board–to improve transparency, accountability, and oversight. These  initiatives represent improvements over current practices, although they  provide for less oversight than statutes provide for other major defense  acquisition programs. In addition, Congress has directed that MDA’s  budget materials, after 2009, request funds using the appropriation  categories of research, development, and evaluation, procurement,  operations and maintenance, and military construction, which should  promote accountability for and transparency of the BMDS.
subsections: section_title: New Block Construct Offers Improvements, but Does Not Address All Issues
paragraphs: In 2007, MDA redefined its block construct to better communicate its  plans and goals to Congress. The agency’s new construct is based on  fielding capabilities that address particular threats as opposed to the  biennial time periods that were the agency’s past approach to  development and fielding. MDA’s new block construct makes many  positive changes. These include establishing unit cost for selected block  assets, including in a block only those elements or components that will be  fielded during the block, and abandoning the practice of deferring work  from block to block.
subsections: section_title: Features of New Block Construct
paragraphs: Table 5 illustrates MDA’s new block construct for fielding the BMDS.
MDA’s new block construct provides a means for comparing the expected  and actual unit cost of assets included in a block. As we noted in our fiscal  year 2006 report, MDA’s past block structure did not estimate unit costs  for assets considered part of a given block or categorize block costs in a  manner that allowed calculations of expected or actual unit costs. For  example, the expected cost of Block 2006 GMD interceptors emplaced for  operational use was not separated from other GMD costs. Even if MDA  had categorized the interceptors’ cost, it would have been difficult to  determine the exact cost of these interceptors because MDA acquires and  assembles components into interceptors over several blocks and it has  been difficult to track the cost of components to a specific group of  interceptors. Under the new block construct, MDA expects to develop unit  costs for selected block assets—such as THAAD interceptors—and  request an independent verification of that unit cost from DOD’s Cost  Analysis Improvement Group.  MDA will also track the actual unit cost of  the assets and report significant cost growth to Congress.  However, MDA  has not yet determined for which assets a unit cost will be developed and  how much a unit cost must increase before that increase is reported to  Congress.
The new construct also makes it clearer as to which assets should be  included in a block. Under the agency’s prior block construct, assets  included in a given block were sometimes not planned for delivery until a  later block. For example, as we reported in March 2007, MDA included  costs for ABL and STSS as part of its Block 2006 cost goal although those  elements did not field or plan to field assets during Block 2006. Agency  officials told us those elements were included in the block because they  believed the elements could offer some emergency capability during the  block timeframe.
Finally, the new block construct should improve the transparency of each  block’s actual cost. Under its prior construct, MDA deferred work from  one block to another; but it did not track the cost of the deferred work so  that it could be attributed to the block that it benefited. For example, MDA  deferred some work needed to characterize and verify the Block 2004  capability until Block 2006 and counted the cost of those activities as a  cost of Block 2006. By doing so, it understated the cost of Block 2004 and  overstated the cost of Block 2006. Because MDA did not track the cost of  the deferred work, the agency was unable to adjust the cost of either block  to accurately capture the cost of each. MDA officials told us that under its  new block construct, MDA will no longer transfer work, along with its  cost, to a future block. Rather, a block of work will not be considered  complete until all work that benefits a block has been completed and its  cost has been properly attributed to that block.
section_title: Block Construct Does Not Resolve All Issues
paragraphs: Although improvements are inherent in MDA’s new block construct, the  new construct will not dispel all transparency and accountability  concerns.  MDA has not yet estimated the full cost of a block.  Also, MDA  has not addressed whether it will transfer assets produced during a block  to a military service for production and operation at the block’s  completion, or whether MDA will continue its practice of concurrently  developing and fielding BMDS elements and components.
According to its fiscal year 2009 budget submission, MDA does not plan to  initially develop a full cost estimate for any BMDS block.  Instead, when a  firm commitment can be made to Congress for a block of capability, MDA  will develop a budget baseline for the block.  This budget will include  anticipated funding for each block activity that is planned for the 6 years  included in DOD’s Future Years Defense Plan.  MDA officials told us that if  the budget for a baselined block changes, MDA plans to report and explain  those variations to Congress. At some future date, MDA does expect to  develop a full cost estimate for each committed block and is in discussions  with DOD’s Cost Analysis Improvement Group on having the group verify  each estimate; but documents do not yet include a timeline for estimating  block cost or having that estimate verified.  For accountability, other DOD  programs are required to provide the full cost of developing and producing  their weapon system before system development and demonstration can  begin. Until the cost of a block of BMDS capability is fully known, it will  be difficult for decision makers to compare the value of investing in a  block of BMDS capability to the value of investing in other DOD programs  or to determine whether the block of capability that is being initiated will  be affordable over the long term.
The new block construct does not address whether the assets included in  a block will be transferred at the block’s completion to a military service  for production and operation.  Officials representing multiple DOD  organizations recognize that the transfer criteria established in 2002 are  neither complete nor clear given the BMDS’s complexity. Without clear  transfer criteria, MDA has transferred the management of only one  element—the Patriot Advanced Capability-3—to the military for  production and operation. Joint Staff officials told us that for all other  elements, MDA and the military services have been negotiating the  transition of responsibilities for the sustainment of fielded elements—a  task that has proven arduous and time consuming. Although MDA  documents show that under its new block construct the agency should be  ready at the end of each block to deliver BMDS components that are fully  mission-capable, MDA officials could not tell us when MDA’s Director will  recommend that management of components, including production  responsibilities, be transferred to the military. MDA officials maintain that  even though a particular configuration of a weapon could be fully mission- capable, that configuration may never be produced because it could be  replaced by a new configuration.  Yet, by the block’s end, a transfer plan  for the fully mission-capable configuration will have been drafted,  developmental ground and flight tests will be complete, elements and  components will be certified for operations, and doctrine, organization,  training, material, leadership, personnel, and facilities are expected to be  in place.
Another issue not addressed under MDA’s new block construct is whether  the concurrent development and fielding of BMDS elements and/or  components will continue. Fully developing a component or element and  demonstrating its capability prior to production increases the likelihood  that the product will perform as designed and can be produced at the cost  estimated. To field an initial capability quickly, MDA accepted the risk of  concurrent development and fielding during Block 2004.  For example, by  the end of Block 2004, the agency realized that the performance of some  Ground-based interceptors could be degraded because the interceptors  included inappropriate or potentially unreliable parts. MDA has begun the  process of retrofitting these interceptors, but work will not be completed  until 2012. Meanwhile there is a risk that some interceptors might not  perform as designed. MDA also continued to accept this risk during Block  2006 as it fielded assets before they were fully tested.  MDA has not  addressed whether it will accept similar performance risks under its new  block construct or whether it will fully develop and demonstrate all  elements/components prior to fielding.
section_title: New Executive Board Offers Improved, but Not Full, Oversight
paragraphs: In March 2007, the Deputy Secretary of Defense established a Missile  Defense Executive Board (MDEB) to recommend and oversee  implementation of strategic policies and plans, program priorities, and  investment options for protecting the United States and its allies from  missile attacks.  The MDEB was also to replace existing groups and  structures, such as the Missile Defense Support Group (MDSG). However,  while it has some oversight responsibilities, the MDEB was not established  to provide full oversight of the BMDS program and it would likely be  unable to carry out this mission even if tasked to do so.  The MDEB will  not receive some information that the Defense Acquisition Board relies  upon to make program recommendations, and in other cases, MDA does  not plan to seek the MDEB’s approval before deciding on a course of  action. In addition, there are parts of the BMDS program for which there  will be no baseline against which progress can be measured, which makes  oversight difficult.
subsections: section_title: Functions of the New Board
paragraphs: According to its charter, the MDEB is vested with more responsibility than  its predecessor, the MDSG. When the MDSG was chartered in 2002, it was  to provide constructive advice to MDA’s Director. However, the Director  was not required to follow the advice of the group. According to a DOD  official, although the MDSG met many times initially, it did not meet after  June 2005. This led, in 2007, to the formation of the MDEB.  This board’s  mission is to review and make recommendations on MDA’s  comprehensive acquisition strategy to the Deputy Secretary of Defense. It  is also to provide the Under Secretary of Defense, Acquisition, Technology  and Logistics, with a recommended strategic program plan and a feasible  funding strategy based on “business case” analysis that considers the best  approach to fielding integrated missile defense capabilities in support of  joint MDA and warfighter objectives  The MDEB will be assisted by four standing committees. These  committees, which are chaired by senior-level officials from the Office of  the Secretary of Defense and the Joint Staff, could play an important  oversight role as they are expected to make recommendations to the  MDEB, which in turn will recommend courses of action to the Under  Secretary of Defense for Acquisition, Technology, and Logistics (USD  AT&L) and the Director, MDA, as appropriate. The following table  identifies the chair of each standing committee as well as key committee  functions.
The MDEB will not have access to all information normally available to  oversight bodies.  For other major defense acquisition programs, the  Defense Acquisition Board must approve the program’s progress through  the acquisition cycle. Further, before a program can enter the System  Development and Demonstration phase of the cycle, statute requires that  certain information be developed.  This information is then provided to the  Defense Acquisition Board. However, in 2002, the Secretary of Defense  allowed MDA to defer application of the defense acquisition system that  among other things require programs to follow a defined acquisition cycle  and obtain approval before advancing from one phase of the cycle to  another. Because MDA does not follow this cycle, it does not enter System  Development and Demonstration and it does not trigger the statutes  requiring the development of information that the Defense Acquisition  Board uses to inform its decisions. For example, most major defense  acquisition programs are required by statute to obtain an independent  verification of program cost prior to beginning system development and  demonstration, and/or production and deployment.  Independent life- cycle cost estimates provide confidence that a program is executable  within estimated cost and along with other DOD-wide budget demands.  Although MDA plans to develop unit cost for selected block assets and  request that DOD’s Cost Analysis Improvement Group verify the unit  costs, the agency does not initially plan to develop a block cost estimate  and, therefore, cannot seek an independent verification of that cost. In  addition, even when MDA estimates block costs, the agency will not be  required to obtain an independent verification of that cost, because, as  noted earlier, the BMDS program operates outside of DOD’s acquisition  cycle. Although not required, MDA officials told us that they have initiated  discussions with the Cost Analysis Improvement Group on independent  verifications of block cost estimates.
Statute also requires an independent verification of a system’s suitability  for and effectiveness on the battlefield before a program can proceed  beyond low-rate initial production. After the test is completed, the  Director for Operational Test and Evaluation assesses whether the test  was adequate to support an evaluation of the system’s suitability and  effectiveness for the battlefield, whether the test showed the system to be  acceptable, and whether any limitations in suitability and effectiveness  were noted. However, a comparable assessment of the BMDS assets being  produced for fielding will not be available to the MDEB.  As noted earlier,  the limited amount of testing completed, which has been primarily  developmental in nature, and the lack of verified, validated, and accredited  models and simulations prevent the Director of Operational Test and  Evaluation from fully assessing the effectiveness,  suitability, and  survivability of the BMDS in annual assessments.
MDA will also make some decisions without approval from the MDEB or  any higher level DOD official. Although the charter of the MDEB includes  the mission to make recommendations to MDA and the Under Secretary of  Defense for AT&L on investment options, program priorities, and MDA’s  strategy for developing and fielding an operational missile defense  capability, the MDEB will not have the opportunity to review and  recommend changes to BMDS blocks. According to a briefing on the  business rules and processes for MDA’s new block structure, the decision  to initiate a new block of BMDS capability will be made by MDA’s  Director. Also cost, schedule, and performance parameters will be  established by MDA when technologies that the block depends upon are  mature, a credible cost estimate can be developed, funding is available,  and the threat is both imminent and severe. The Director will inform the  MDEB as well as Congress when a new block is initiated, but he will not  seek the approval of either.
Finally, there will be parts of the BMDS program that will be difficult for  the MDEB to oversee because of the nature of the work being performed.  MDA plans to place any program that is developing technology in a  category known as Capability Development. These programs, such as ABL,  KEI, and MKV, will not have a firm cost, schedule, or performance  baseline. This is generally true for technology development programs in  DOD because they are in a period of discovery, which makes schedule and  cost difficult to estimate. On the other hand, the scale of the technology  development in BMDS is unusually large, ranging from $2 billion to about  $5 billion dollars a year—eventually comprising nearly half of MDA’s  budget by fiscal year 2012. The MDEB will have access to the budgets  planned for these programs over the next 5 or 6 years, each program’s  focus, and whether the technology is meeting short-term key events or  knowledge points. But without some kind of baseline for matching  progress with cost, the MDEB will not know how much more time or  money will be needed to complete technology maturation. MDA’s  experience with the ABL program provides a good example of the  difficulty in estimating the cost and schedule of technology development.
In 1996, the ABL program believed that all ABL technology could be  demonstrated by 2001 at a cost of about $1 billion. However, MDA now  projects that this technology will not be demonstrated until 2009 and its  cost has grown to over $5 billion. While the uncertainties of technology  development must be recognized, some organizations suggest ways to  establish a baseline appropriate for such efforts. For example, the Air  Force Research Laboratory suggested a methodology to estimate a  technology’s cost once analytical and laboratory studies physically  validate analytical predictions of separate elements of the technology.
section_title: MDEB Could Play Key Role In Proposal to Return BMDS to Standard Requirements Process
paragraphs: In an effort to further improve oversight, the Joint Requirements Oversight  Council proposed a plan to transition the BMDS into standard DOD  processes. In August 2007, the Vice Chairman of the Joint Chiefs of Staff  and Joint Requirements Oversight Council Chairman requested the Deputy  Secretary of Defense approve a proposal to return MDA to the Joint  Capabilities Integration and Development System process and direct the  Joint Requirements Oversight Council to validate BMDS capabilities. The  Vice Chairman believed that the council should exercise oversight of MDA  in order to improve Department-wide capability integration. More  specifically, he noted that:  In 2002, the Secretary of Defense exempted the BMDS program from  the traditional requirements generation process to expedite fielding the  system as soon as practicable.
Now that an initial capability for homeland defense has been deployed,  there is no longer the same need for flexibility provided by the  requirements exemption.
The current process, with MDA exempted, does not allow the Joint  Requirements Oversight Council to provide appropriate military advice  or to validate missile defense capabilities. Without this change, there is  increasing potential that MDA-fielded systems will not be synchronized  with other air and missile defense capabilities being developed.
The current process hinders the military departments’ ability to plan  and program resources for fielding and sustainment of MDA-developed  systems.
In responding to the proposal, the Acting Under Secretary of Defense for  AT&L recommended that the Deputy Secretary of Defense delay his  approval of the Joint Staff’s proposal until the MDEB could review the  proposal and provide a recommendation. However, he agreed that more  Joint Requirements Oversight Council involvement was necessary for the  BMDS, although he was not sure that returning BMDS to standard DOD  processes was the appropriate solution to the agency’s oversight issues.  Instead, he noted that the Deputy Secretary of Defense recently  established the MDEB to recommend and oversee the implementation of  strategic policies and plans, program priorities, and investment options for  the BMDS. He stated that since the MDEB is tasked with determining the  best means of managing the BMDS throughout its life cycle, it should  consider the Joint Staff’s proposal.
section_title: MDA Directed to Use Procurement Funding for the First Time
paragraphs: In an effort to improve the transparency of MDA’s acquisition processes,  Congress has directed that MDA’s budget materials delineate between  funds needed for research, development, and evaluation, procurement,  operations and maintenance, and military construction. Using  procurement funds will mean that MDA generally will be required to  adhere to congressional policy that assets be fully funded in the year of  their purchase, rather than incrementally funded over several years. The  Congressional Research Service reported in 2006 that “incremental  funding fell out of favor because opponents believed it could make the  total procurement costs of weapons and equipment more difficult for  Congress to understand and track, create a potential for DOD to start  procurement of an item without necessarily stating its total cost to  Congress, permit one Congress to ‘tie the hands’ of future Congresses, and  increase weapon procurement costs by exposing weapons under  construction to uneconomic start-up and stop costs.” Our analysis of  MDA developed costs, which are presented in table 7, also shows that  incremental funding is usually more expensive than full funding, in part,  because inflation decreases the buying power of the dollar each year.
The National Defense Authorization Act for Fiscal Year 2008 directed MDA  to submit a plan to transition from using research and development funds  exclusively to using procurement, operations and maintenance, military  construction, and research and development funds by March 1, 2008.  However, it allowed MDA to continue to use research and development  funds in fiscal year 2009 to incrementally fund previously approved missile  defense assets.  The act also directed that beginning in fiscal year 2009, the  MDA budget request include, in addition to RDT&E funds, military  construction funds and procurement funds for some long lead items such  as those required for the third and fourth THAAD fire units and Aegis BMD  SM-3 Block 1A missiles. MDA did not request long lead funding for either  THAAD or SM-3 missiles in its fiscal year 2009 budget because MDA has  slipped the schedule for procuring fire units 3 and 4 by one year, and the  National Defense Authorization Act for Fiscal Year 2008 was not signed in  time to allow MDA to adjust its budget request for SM-3 missiles.
Congress also provided MDA with the authority to use procurement funds  for fiscal years 2009 and 2010 to field its BMDS capabilities on an  incremental funding basis, without any requirement for full funding.  Congress has granted similar authority to other DOD programs. In the  conference report accompanying the Fiscal Year 2008 National Defense  Authorization Act, the conferees indicated that if MDA wishes to use  incremental funding after fiscal year 2010, DOD must request additional  authority for a specific program or capability. Conferees cautioned DOD  that additional authority will be considered on a limited case-by-case basis  and that future missile defense programs will be funded in a manner more  consistent with other DOD acquisition programs.
Since 2002, MDA has been granted the flexibility to incrementally fund the  fielding of its operational assets with research and development funds. In  some cases, the agency spreads the cost of assets across 5 to 7 budget  years. After reviewing the agency’s incremental funding plan for future  procurements of THAAD fire units and Aegis BMD missiles, we analyzed  the effect of fully funding these assets using present value techniques and  found that the agency could save about $125 million by fully funding their  purchase and purchasing them in an economical manner. Our analysis is  provided in table 7. In addition, more detailed analysis is available in  appendix III.
According to our analysis, fully funding the THAAD and Aegis BMD assets  will, in all instances, save MDA money. For example, full funding would  save the THAAD program approximately $104 million and the Aegis BMD  program nearly $22 million. In addition, by providing funds upfront, the  contractors should be able to arrange production in the most efficient  manner.
section_title: Conclusions
paragraphs: By the end of Block 2006, MDA posted a number of accomplishments for  the BMDS, including fielding more assets, conducting several successful  tests, and progressing with developmental efforts. As a result, fielded  capability has increased. On the other hand, some problems continue that  make it difficult to assess how well the BMDS is progressing relative to the  funds it has received and the goals it has set for those funds.  First, under  the proposed block construct, MDA plans to develop a firm baseline for  each block and have it independently reviewed.  However, MDA has not  yet developed estimates for full block costs, so the initial baseline  incorporates the budget for each block only through DOD’s Future Years  Defense Plan.  Second, while MDA expects to estimate unit costs and track  increases, it is unclear as to what criteria will be used for reporting  variances to Congress. Third, while MDA has gotten some contractors to  lower the portion of work planned as level of effort, a substantial amount  of work remains so planned. Fourth, while it may not be reasonable to  expect the same level of accountability for technology development efforts  as it is for development and production of systems, the high level of  investment—up to half of its budget—MDA plans to make in technology  development warrants some mechanism for reconciling the cost of these  efforts with their progress. Finally, MDA fields assets before development  testing is complete and without conducting operational testing. We have  previously recommended that MDA return to its original non-concurrent,  knowledge-based approach to developing, testing, and fielding assets.  Short of that, the developmental testing that is done provides the primary  basis for the Director of Operational Test and Evaluation to assess  whether a block of BMDS capability is suitable and effective for the  battlefield. So far, BMDS testing has not yielded sufficient data to make a  full assessment.
section_title: Recommendations for Executive Action
paragraphs: To build on efforts to improve the transparency, accountability, and  oversight of the missile defense program, we recommend that the  Secretary of Defense direct:    MDA to develop a full cost for each block and request an independent  verification of that cost;    MDA to clarify the criteria that it will use for reporting unit cost    MDA to examine a contractor’s planning efforts when 20 percent or  more of a contract’s work is proposed as level of effort;    MDA to investigate ways of developing a baseline or some other  standard against which the progress of technology programs may be  assessed; and    MDA and the Director of Operational Test and Evaluation to agree on  criteria and incorporate corresponding scope into developmental tests  that will allow a determination of whether a block of BMDS capability  is suitable and effective for fielding.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report.  These  comments are reprinted in appendix I.  DOD also provided technical  comments, which we incorporated as appropriate.
DOD concurred with three of our five recommendations—developing a  full cost estimate for each block and requesting an independent  verification of that cost, clarifying criteria for reporting unit cost variances  to Congress, and examining contractors’ planning efforts when 20 percent  or more of a contract’s work is proposed as level of effort.  The  Department indicated that MDA has already taken steps to develop new  cost models aligned with its new block structure and met with DOD’s Cost  Analysis Improvement Group to initiate the planning process for the  independent verifications of MDA’s cost estimates.  The cost estimates will  extend until block completion and will not be limited by a 6-year Future  Years Defense Plan window.  MDA is also working to establish criteria for  reporting unit cost variances and to incorporate them into an MDA  directive.  Finally, MDA has made a review of prime contractors’ work  planning efforts part of the Integrated Baseline Review process and the  Defense Contract Management Agency has agreed to continuously validate  the appropriateness of each contractor’s planning methodology as part of  its ongoing contract surveillance.
DOD partially concurred with our recommendation that MDA investigate  ways of developing a baseline or some other standard against which the  progress of technology programs may be assessed.  DOD observed that  MDA uses knowledge points, technology readiness levels, and engineering  and manufacturing readiness levels in assessing the progress of its  technology programs and that it will continue to investigate other methods  of making such assessments.  While we recognize their value, these  methods typically assess progress in the short term and do not provide an  estimate of the remaining cost and time needed to complete a technology  program.  Because MDA must balance its efforts to improve the existing  BMDS while developing new capability, DOD and MDA need to ensure  that only the most beneficial technology programs in terms of  performance, cost, and schedule are pursued.  This will require an  understanding of not only the benefit to be derived from the technology,  but also an understanding of the cost and time needed to bring the  technology to fruition.
DOD also partially concurred with our last recommendation that MDA and  the Director of Operational Test and Evaluation (DOT&E) agree on criteria  and additional scope for developmental tests that will allow a full  determination of the effectiveness and suitability of a BMDS block for  fielding.  DOD noted that it is MDA’s mission to work with the warfighter,  rather than DOT&E, to determine that the BMDS is ready for fielding, but  that MDA will work closely with DOT&E to strengthen the testing of  BMDS suitability and effectiveness.  We agree that DOT&E is not  responsible for fielding decisions, but its mission is to ensure that weapon  systems are realistically and adequately tested and that accurate  evaluations of operational effectiveness, suitability, and survivability are  available for production decisions. MDA improved the operational realism  of testing in 2007 and for the first time DOT&E considered tests at least  partially adequate to make an assessment of the BMDS.  However, a full  assessment is not yet possible and we continue to recommend that MDA  and DOT&E take steps to make as full a BMDS evaluation as possible.  In  doing so, MDA and DOT&E can work cooperatively to reduce the number  of unknowns that will confront the warfighter when the system is required  operationally and improve the likelihood that the BMDS will perform as  needed in the field.
We are sending copies of this report to the Secretary of Defense and to the  Director, MDA. We will make copies available to others upon request. In  addition, the report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you, or your staff have any questions concerning this report, please  contact me at (202) 512-4841. Contact Points for our offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. The major contributors are listed in appendix V.
section_title: Appendix I: Comments from the Department of Defense
section_title: Appendix II: MDA Contracts
subsections: section_title: Overview of MDA Prime and Support Contracts
paragraphs: The Missile Defense Agency (MDA) employs prime contractors and  support contractors to accomplish different tasks that are needed to  develop and field the ballistic missile defense system. Prime contractors  receive the bulk of funds MDA requests each year and work to provide the  hardware and software for elements of the Ballistic Missile Defense  System (BMDS). Support contractors provide a wide variety of useful  services, such as special knowledge and skills not available in the  government and the capability to provide temporary or intermittent  services.
subsections: section_title: Prime Contracts
paragraphs: MDA has prime contracts with four defense companies—Boeing,  Raytheon, Lockheed Martin, and Northrop Grumman—to develop  elements of the BMDS. All current contracts and agreements are cost  reimbursement type that provide for payment of reasonable, allowable,  and allocable incurred costs to the extent provided in the contract or  agreement. The contracts also provide fee for the contractor performing  the work, but the amount earned depends on many variables, including the  type of cost contract, contractor performance, technical risk, and  complexity of the requirement. All of the cost reimbursement contracts  used for the BMDS elements include cost plus award fee aspects. Cost  plus award fee contracts provide for a fee consisting of a base fee—fixed  at the inception of the contract that may be zero—and an award amount  based upon a subjective evaluation by the government, meant to  encourage exceptional performance.  It should be noted that some award  fee arrangements include objective criteria such as Key Performance  events.
The Multiple Kill Vehicle (MKV) contract and Command, Control, Battle  Management and Communications (C2BMC) Other Transaction  Agreement differ somewhat from the other elements' contracts. The MKV  prime contractor was awarded an indefinite delivery/indefinite quantity  cost reimbursement contract. This type of contract allows MDA to order  services as they are needed through a series of task orders. Without having  to specify a firm quantity of services (other than a minimum or maximum  quantity), the government has greater flexibility to align the tasks with  available funding. The C2BMC element operates under an Other  Transaction Agreement with cost reimbursement aspects.  These types of  agreements are not always subject to procurement laws and regulations  meant to safeguard the government. MDA chose the Other Transaction  Agreement to facilitate a collaborative relationship between industry,  government, federally funded research and development centers, and  university research centers.
DOD requires that all contractors awarded cost reimbursement contracts  or other agreements of $20 million or greater implement an Earned Value  Management System (EVMS) to integrate the planning of work scope,  schedule, and resources, and to provide insight into their cost and  schedule performance. To implement this system, contractors examine the  totality of the work directed by the contract and break it into executable  work packages. Each work package is assigned a schedule and a budget  that is expected to enable the work’s completion. On a monthly basis, the  contractor examines initiated work packages to determine whether the  work scheduled for the month was performed on time and within budget.  If more work was completed than scheduled and the cost of the work  performed was less than budgeted, the contractor reports a positive  schedule and cost variance. However, if the contractor was unable to  complete all of the work scheduled and needed more funds to complete  the work than budgeted, the contractor reports a negative schedule and  cost variance. Of course, the results can be mixed. That is, the contractor  may have completed more work than scheduled but at a cost that  exceeded the budget. The contractor details its performance to MDA each  month in Contract Performance Reports. These reports also identify the  reasons that negative or positive variances are occurring. Used properly,  the earned value concept allows program managers to identify problems  early so that steps can be taken before the problems increase the  contract’s overall cost and/or schedule.
In the course of subdividing the total work of the contract into smaller  efforts, contractors plan work according to its type. Included in these  classifications are discrete work—work that is expected to produce a  product, such as a study, lines of software code, or a test—and work  considered to be level of effort (LOE). LOE is work that does not result in  a product, but is of a general or supportive nature. Supervision and  contract administration are examples of work that do not produce  definable end products and are appropriately planned as LOE.
section_title: Some BMDS Contracts Have a High Percentage of Level of Effort Work
paragraphs: Several contracts for BMDS systems have relatively high proportions of  work planned as LOE. When work is incorrectly planned as LOE, the  contractor’s performance becomes less transparent because earned value  does not recognize schedule variances for such work. Rather, it is  assumed that the time budgeted for an LOE effort will produce the  intended result. Although an LOE work package will report cost variances,  those variances will only be measured against how much the program  intended to spend at certain time intervals. If LOE were to be used on  activities that could otherwise be measured discretely, the project  performance data could be favorably distorted and contractors and  program managers might not be able to discern the value gained for the  time spent on the task. Specifically, the program’s Contract Performance  Reports would not indicate whether or not the work performed produced  the product expected. By losing early insight into performance, the  program could potentially need to spend more time and money to  complete the task.
Since earned value management is less suited for work that is not intended  to produce a specific product, or work that is termed LOE, the Standard  for Earned Value Management Systems Intent Guide instructs that  although some amount of LOE activity may be necessary, it must be held  to the lowest practical level. In addition, earned value experts such as  Defense Contract Management Agency officials agree that if a contractor  plans more than 20 percent of the total contract work as LOE, the work  plan should be examined to determine if work is being properly planned.
Although the amount of LOE should be minimized, some BMDS prime  contracts have a relatively high percentage of LOE. As figure 2 illustrates,  the MKV contractor planned much of the work for task orders open during  fiscal year 2007 as LOE. Contractors for Aegis BMD SM-3 and C2BMC also  planned a high percentage of their work as LOE. Both MDA’s Earned Value  Management Group and program office reviewers encouraged the SM-3  and C2BMC contractors to reduce their LOE percentages. By the end of  the fiscal year, the SM-3 and C2BMC contractors had reduced the amount  of work planned as LOE.
In December 2006, the Aegis BMD SM-3 contractor completed work to  develop and produce initial Block 1A missiles with 73 percent of this work  categorized as LOE—well above the 15 percent that the Aegis BMD SM-3  program reports as its industry standard. Although we have reported that  the contractor completed this segment of work ahead of cost but slightly  behind schedule, it is difficult to assess whether this represents the  contractor’s actual performance. The high percentage of LOE associated  with this work may have limited our assessment and distorted whether the  work completed was in all respects the work planned. Subsequently, the  contractor initiated procurement of long lead materials to produce an  additional 20 Block 1A missiles before work packages were developed.   Once work packages were developed, only 18 percent of the work was  planned as LOE.
The C2BMC program was able to reduce the percentage of work planned  as LOE, but the program continues to encourage further reductions.  During fiscal year 2007, the C2BMC contractor replanned its work and  reduced the amount of work planned as LOE from 73 to 52 percent. This  change was implemented after two closely related reviews suggested the  percentage of LOE work was too high. Both the program office and its  contractor acknowledge the high level of LOE and have made plans to  limit it in future work.
As noted in figure 2, the MKV contractor considered all work being  completed under two task orders—Task Orders 4 and 5—as LOE. The  primary objective of Task Order 4 is to update the program plan and  complete the systems engineering effort necessary to integrate the MKV  warhead into the BMDS to the extent required for the systems  requirements review. Both the system concept review, completed in July  2006, and the system requirements review, scheduled for December 2008,  are major milestones. However, the contractor did not plan these  milestone reviews as products. According to program officials, Task Order  4 will be reevaluated in February 2008 to reduce the amount of LOE and  recognize more work as discrete.  The MKV program also planned 100  percent of Task Order 5 work as LOE. Under this task order, the  contractor was to design a prototype propulsion system, assemble and  integrate the hardware for the prototype, and perform a static hot fire test  of the integrated system. This effort culminates in hardware—a tangible  end product—that is expected to exhibit certain performance  characteristics during the static hot fire test. The contractor could have  categorized this task order, at least in some part, as discrete work since  the work was expected to deliver a defined product with a schedule that  could slip or vary. Because the contractor categorized all of this task order  as LOE, the program lost its ability to gauge performance and to make  adjustments that might prevent contract cost growth.
section_title: Prime Contractors Exceed Fiscal Year 2007 Budgeted Costs but Make Gains on Schedule
paragraphs: We analyzed Fiscal Year 2007 Contract Performance Reports for MDA’s 10  prime contracts and determined that collectively the contractors overran  budgeted costs by nearly $170 million but were ahead of schedule by  nearly $200 million. However, the percentage of work planned as LOE  should be scrutinized before accepting this as the contractors’ actual  performance because a high percentage of LOE, as noted above, can  potentially distort the contractors’ cost and schedule performance. The  cumulative performance of one contractor is also distorted because it  rebaselined part of its work. Rebaselining is an accepted EVM procedure  that allows a contractor to reorganize all or part of its remaining contract  work, add additional time or budget for the remaining effort, and, under  some circumstances, set affected cost and/or schedule variances to zero.  When variances are set to zero, the cumulative performance of the  contractor appears more positive than it is.
Four of the 10 contracts we reviewed also contained some kind of  replanning activity during fiscal year 2007. Contractors may replan when  they conclude that the current plan for completing the effort remaining on  the contract is unrealistic. A replan can consist of any of the following:  reallocating the budget for the remaining effort within the existing  constraints of the contract, realigning the schedule within the  contractually defined milestones, and setting cost and/or schedule  variances to zero. During the course of replanning a contract, the  contractor must provide traceability to previous baselines as well as  ensure that available funding is not exceeded.
subsections: section_title: Aegis BMD Contractors Perform within Budgeted Costs
paragraphs: The Aegis BMD program awarded two prime contracts for its major  components, the Aegis BMD Weapon System and the Standard Missile-3.  During the fiscal year, the contractors completed all work at less cost than  budgeted. Both contractors ended the year with positive cumulative cost  variances, but negative cumulative schedule variances. Based on our  analysis, we project that if the contractors continue to perform at the same  level, the weapon system contractor could underrun its budget by between  $8.8 million and $17.7 million, while the SM-3 contractor could complete  its work on 20 Block 1A missiles for $7.4 million to $11.1 million less than  budgeted.
The weapon system contractor’s fiscal year 2007 cost performance  resulted in a positive cost variance of $7.7 million. The positive variance  was realized as two software packages required less effort than  anticipated and were completed earlier than expected. Combined with its  performance from earlier periods, the contractor finished the year with a  cumulative positive cost variance of $7 million. This upward trend is  depicted in figure 3.
The contractor produced a $3.8 million unfavorable schedule variance in  fiscal year 2007. The contractor reported that the unfavorable cumulative  variance was caused in part by a delay in receiving component materials  for the radar’s processor.
During fiscal year 2007, the Aegis SM-3 contractor closed out work related  to missile development and initial production of Block 1A missiles and  began new work in February 2007 to manufacture an additional 20 Block  1A missiles. In performing the new work, the contractor underran its cost  budget by $6.2 million, but failed to complete $4.0 million of planned work.  The Aegis BMD SM-3 contractor’s cumulative cost and schedule variances  are highlighted in figure 4.
The positive cost variance can be attributed to several factors including  cost efficiencies realized from streamlining system engineering resources  and lower than planned hardware costs. Our analysis predicts that if the  SM-3 contractor continues to perform as it did through September 2007, it  will underrun its budgeted costs for the 20 Block 1A missiles by between  $7.4 million and $11.1 million. The contractor’s negative cumulative  schedule variance of $4 million for the 20 missiles was primarily caused by  delayed qualification testing and integration of hardware components.
section_title: Despite Rebaselining, ABL’s Cumulative Cost and Schedule Performance Remain Negative
paragraphs: In May 2007, MDA allowed ABL’s contractor to rebaseline one part of its  contract after the work associated with a key knowledge point could not  be completed on schedule. Because the contractor did not achieve this  knowledge point as planned, the program was forced to postpone its  lethality demonstration until August 2009. Technical issues including  weapon system integration, beam control/fire control software  modifications, and flight testing discoveries, all contributed to the delay in   completing the knowledge point for the program. To provide funds and  time to support the delay in the lethality demonstration, the program  extended the contract’s period of performance by approximately 1 year  and increased the contract’s ceiling cost by $253 million. Once the new  baseline was incorporated, the contractor was able to complete fiscal year  2007 with positive cost and schedule variances of $3.7 million and $24.2  million, respectively. Figure 5 depicts the contractor’s cumulative cost and  schedule performance.
As shown in figure 5 above, the ABL contractor was not able to overcome  the negative cost and schedule variances of prior years and ended the  fiscal year with an unfavorable cumulative cost variance of $74.2 million  and an unfavorable cumulative schedule variance of $25.8 million. We  estimate that, at completion, the contract could overrun its budget by  between $95.4 million and $202.5 million.
section_title: C2BMC Program Incurs Negative Cumulative and Fiscal Year Variances
paragraphs: During fiscal year 2006, the C2BMC contractor did not report earned value  because it was working on a replan of its Block 2006 increment of work  (known as Part 4). Following the definitization of the Part 4 replan in  November 2006, the C2BMC contractor resumed full EVM reporting with  the first submittal covering February 2007 data.  As part of the replan, the  contractor adjusted a portion of its Part 4 work and set cost and schedule  variances to zero in an effort to establish a baseline commensurate with  the contractor’s replanning efforts. However, even with the adjustment,  the C2BMC program ended fiscal year 2007 with negative fiscal year cost  and schedule variances of $11.1 million and $1.5 million, respectively.  Figure 6 shows the contractor’s cumulative performance in fiscal year  2007.
The unfavorable fiscal year cost variance was largely due to adding staff to  support a software release; while the unfavorable fiscal year schedule  variance was attributable to delays in hardware delivery, initiation of a  new training system, and completing training material for the new system.  Added to prior year negative variances, the C2BMC contractor reported  cumulative negative cost and schedule variances of $14.5 million and $3.5  million, respectively. The contractor completed Part 4 work in December  2007 and reported an overrun of $9.9 million.
section_title: GMD Contractor Maintained Negative Cost and Schedule Variances Throughout the Fiscal Year
paragraphs: The GMD prime contractor’s cost performance improved significantly in  fiscal year 2007.  The contractor experienced a budget overrun of $22.1  million for the fiscal year following budget overruns in both fiscal years  2005 and 2006 that exceeded $300 million.  Program officials attribute this  turnaround in performance to several factors, including rigorous  management of the contract’s estimate at completion, quality initiatives,  and joint efforts by the contractor and program office to define scope,  schedule, and price of change orders.  The cumulative cost variance at the  end of fiscal year 2007 was over $1 billion.  We estimate that at completion  the contract, with a target price of $15.54 billion, could exceed its  budgeted cost by between $1.06 billion and $1.4 billion.
The contractor was able to complete $84.9 million more work than  scheduled for fiscal year 2007, but could not overcome poor performance  in earlier years and ended the year with a negative cumulative schedule  variance of $52.9 million. Figure 7 illustrates both cost and schedule trends  in GMD fiscal year 2007 performance.
The unfavorable fiscal year cost variance is primarily attributable to the  EKV. During fiscal year 2007, the EKV contractor experienced negative  cost variances as it incurred additional labor costs to recover delivery  schedules, manufacturing schedule delays, hardware manufacturing  problems, and embedded software development and system integration  problems.  With 18 percent of the EKV work remaining, the negative  trends on this component could continue.
As we reported last year, the contractor was in the process of developing a  new contract baseline to incorporate the updated scope, schedule, and  budget that the contractor was working toward. In September 2006, phase  one of the new baseline, covering fiscal year 2006-2007 efforts, had been  implemented and validated through the Integrated Baseline Review of the  prime contractor and its major subcontractors. Phase two of the review  was completed in December 2006.  Subsequent to the reviews, fiscal year  2007 ground and flight tests were replanned to reflect a contract change  that added additional risk mitigation effort to one planned flight test and  added a radar characterization system test.
section_title: KEI Replan Significantly Affects Cost and Schedule Variances
paragraphs: The KEI contractor replanned its work in April 2007 when MDA directed  the program to focus, in the near term, on two main objectives: booster  vehicle development and the 2008 booster flight developmental test. Prior  to the replan, the KEI program was developing a land-mobile capability  with fire control and communications and mobile launcher components.  Although the contractor’s primary objectives are now focused around the  booster segment of work, it is still performing some activities related to  the fire control and communications component. During fiscal year 2007,  the contractor incurred a positive cost variance of $2.1 million and a  negative schedule variance of $7.5 million. Combined with variances from  earlier fiscal years, the cumulative cost variance is a positive $5.7 million  and the cumulative schedule variance is a negative $12.8 million. Figure 8  illustrates KEI’s cumulative performance over the course of the fiscal year.
KEI’s fiscal year favorable cost variance primarily results from completing  work on the fire control and communications component, as well as  systems engineering and integration with fewer staff than planned. We  were unable to estimate whether the total contract is likely to be  completed within budgeted cost since the contract is only 10 percent  complete and trends cannot be developed until at least 15 percent of the  contract is completed.
Work related to the interceptor’s booster and systems engineering and  integration contributed to KEI’s cumulative negative fiscal year schedule  variance of $7.5 million. The contractor reports that the booster work was  understaffed, which caused delays in finalizing designs that, in turn,  delayed procurement of subcomponents and materials and delayed  analysis and tests. While the reduction in staff for systems engineering and  integration work reduced costs for the contractor, it also delayed  completion of the weapon system’s scheduled engineering, flight, and  performance analysis products.
section_title: Information Not Available to Assess All MKV Work
paragraphs: We could evaluate only two of five MKV task orders open during fiscal  year 2007 because the contractor did not report sufficient earned value  data to make an assessment of the other three meaningful. MDA awarded  the MKV contract in January 2004 and has since initiated eight task orders  through its indefinite delivery/indefinite quantity contract. During fiscal  year 2007, the program worked on five of these task orders—Task Orders  4, 5, 6, 7, and 8. We evaluated the contractor’s cost and schedule  performance for Task Orders 5 and 6 only. Of the three task orders that we  did not evaluate, the contractor began reporting full earned value on two  so late in the fiscal year that little data was available for analysis. In the  third case, the contractor’s reports did not include all data needed to make  a cost and schedule assessment.
In June 2006, MDA issued Task Order 5 which directed the design,  assembly, and integration of the hardware for a prototype propulsion  system, and a static hot fire test of the integrated prototype. Because the  contractor planned all activities for this task order as level of effort, the  contractor reported zero schedule variance. Contract Performance  Reports show that in preparation for the hot fire test in August 2007, the  program discovered anomalies indicative of propellant contamination in  the prototype’s propulsion system.  These anomalies led to multiple  unplanned propellant tank anomaly investigations, which contributed to  the unfavorable $2.3 million cost variance for the fiscal year. Additionally,  during the hot fire test, one of the thrusters in the propulsion system’s  divert and attitude control component experienced anomalies due to  foreign object contamination. This anomaly led to unplanned  investigations which also contributed to increased costs. Figure 9 below  depicts the unfavorable cumulative cost variance of $2.7 million and  cumulative schedule variance of zero reported for Task Order 5. Based on  our analysis, we predict the contractor will overrun its contract costs by  between $2.6 million and $2.9 million.
MKV’s objective for Task Order 6 is to manufacture a prototype seeker  capable of acquiring, tracking, and discriminating objects in space. The  program plans to demonstrate the prototype seeker, which is a component  of a carrier vehicle, through testing in 2009. In contrast to Task Order 5,  the contractor correctly planned the bulk of Task Order 6 as discrete work  and has been reporting the work’s cost and schedule status since March  2007. During this time, the contractor has completed 37 percent of the  work directed by the task order at $0.3 million less than budgeted. The  contractor was also able to complete $0.9 million more work than planned.  See Figure 10 for an illustration of cumulative cost and schedule variances  for this task order.
The program attributes its favorable fiscal year cost and schedule  variances for Task Order 6 to the early progress made on interface  requirements, hardware procurements, component drawings, and the  prototype seeker’s architecture. Because detailed designs for the seeker  are derived from models, the program is anticipating some rework will be  needed as the designs are developed, processed, and released. Although  program officials are expecting some degradation in cumulative cost and  schedule variances to occur, the program does not expect an overrun of  the contract’s budgeted cost at completion. Based on the contractor’s  performance to date, we predict, at contract completion, the contractor  will underrun costs by between $0.8 million and $2.5 million.
section_title: Sensors’ FBX-T Contractor Remains Ahead of Cost and on Schedule
paragraphs: The Sensors contractor’s performance during fiscal year 2007 resulted in a  positive cost variance of $3.9 million and an unfavorable schedule variance  of $8.8 million. Added to variances from prior years, the contractor is  reporting cumulative positive cost and schedule variances of $24.1 million  and $17.8 million, respectively. The contractor’s performance in 2007,  suggests that at completion the contract will cost from $22.0 million to  $46.8 million less than budgeted. The variances, depicted below in figure  11 represent the Sensors contractor’s cumulative cost and schedule  performance over fiscal year 2007.
The contractor has reported favorable schedule and cost variances since  the contract’s inception because the program was able to leverage the  hardware design of the THAAD radar to reduce development timelines and  it implemented manufacturing efficiencies to reduce manufacturing costs.  However, during fiscal year 2007, the contractor experienced a negative  schedule variance as it struggled to upgrade software expected to provide  an increased capability for the FBX-T radar.
section_title: STSS Contractor’s Cumulative Cost and Schedule Variances Remain Unfavorable
paragraphs: After replanning a portion of its work in October 2006, the STSS  contractor in fiscal year 2007 experienced an unfavorable cost variance of  $67.7 million and a favorable schedule variance of $84.7 million. Combined  with performance from earlier periods, the contractor is reporting  cumulative negative cost and schedule variances of $231.4 million and  $19.7 million, respectively. Figure 12 shows both cost and schedule trends  during fiscal year 2007.
During the fiscal year, the contractor was able to accomplish a significant  amount of work ahead of schedule after a replan added additional time for  planned work efforts. However, the contractor was unable to overcome  the negative schedule variances incurred in prior years.
Delays in hardware and software testing as well as integration issues  contributed to fiscal year 2007’s negative cost variance. We did not  estimate the cost of the STSS contract at completion. The contract  includes not only the effort to develop and launch two demonstration  satellites (the Block 2006 capability) but also effort that will benefit future  blocks. Block 2006 work is about 86 percent complete, while work on  future blocks is about 16 percent complete.
section_title: THAAD Contractor Improves Schedule, but Costs Continue to Be a Problem
paragraphs: The THAAD contractor overran its fiscal year 2007 budgeted costs by $91.1  million but accomplished $19.0 million more work than scheduled.  Cumulatively, the contractor ended the year with an unfavorable cost  variance of $195.2 million and a negative schedule variance of $9.1 million,  as shown by figure 13.
The THAAD prime contractor’s cost overrun of $91.1 million was primarily  caused by technical problems related to the element’s missile, launcher,  radar, and test components. Missile component cost overruns were caused  by higher than anticipated costs in hardware fabrication, assembly, and  support touch labor as well as subcontractor material costs for structures,  propulsion, and other sub-assembly components.  Additionally, design  issues with the launcher’s missile round pallet and the electronics  assembly that controls the launcher caused the contractor to experience  higher than anticipated labor and material costs. More staff than planned  was required to resolve hardware design issues in the radar’s prime power  unit, causing the radar component to end the fiscal year with a negative  cost variance.  The contractor also experienced negative cost variances  with the system test component because the Launch and Test Support  Equipment required additional set-up time at the flight test range.
THAAD’s prime contractor fared better in performing scheduled work. It  was able to reduce its negative cumulative schedule variance over the  course of the fiscal year because subcontracted missile items were  delivered early and three flight tests were removed from the test program  to accommodate target availability and budget constraints, allowing staff  more time to work on current efforts.
The contractor projects an overrun of $174 million at contract completion,  while we estimate that the overrun could range from $227.2 million to  $325.8 million. To achieve its projection, the contractor needs to complete  $1.04 worth of work for every dollar spent. In contrast, during fiscal year  2007, the contractor achieved an average of $0.82 worth of work for each  dollar spent. Therefore, it seems unlikely that the contractor will be able  to achieve its estimate at completion.
section_title: MDA Relies on Award Fees to Motivate Contractor Performance
paragraphs: Like other DOD programs, MDA has not always effectively used award  fees to encourage contractors toward exceptional performance but it is  making efforts to revise its award fee policy to do so. Over the course of  fiscal year 2007, the agency sometimes rolled over large percentages of  award fee—in most cases for work that was moved to later periods but  also for one contractor that exhibited poor performance.  In addition,  some award fee plans allow fee to be awarded to contractors for merely  meeting the requirements of their contract. For two contractors, MDA  awarded fee amounts that were linked to very good or outstanding work in  the cost and/or program management performance elements. During their  award fee periods, the contractors’ earned value data showed declines in  cost and/or schedule variances, although there are several other factors  considered when rating contract performance. However, in June 2007,  MDA issued a revised draft of its award fee guide in an effort to more  closely link the amount of award fees earned with the level of contractor  performance.
In an effort to encourage its defense contractors to perform in an  innovative, efficient, and effective way in areas considered important to  the development of the BMDS, MDA offers its contractors the opportunity  to collectively earn billions of dollars through monetary incentives known  as award fees. Award fees are intended to motivate exceptional  performance in subjective areas such as technical ingenuity, cost, and  schedule. Award fees are appropriate when contracting and program  officials cannot devise predetermined objective targets applicable to cost,  technical performance, or schedule.
Currently, all 10 of the contracts we assessed for BMDS elements utilize  award fees in some manner to incentivize their contractor’s performance.  Each element’s contract has an award fee plan that identifies the  performance areas to be evaluated and the methodology by which those  areas will be assessed. At the end of each period, the award fee evaluation  board, made up of MDA personnel, program officials, and officials from  key organizations knowledgeable about the award fee evaluation areas,  begins its process. The board judges the contractor’s performance and  recommends to a fee determining official the amount of fee to be paid. For  all BMDS prime contracts we assessed, the fee determining official is the  MDA Director. Table 1 provides a summary of the award fee process.
section_title: DOD Has Not Linked Award Fees to Acquisition Outcomes
paragraphs: GAO has found in the past that DOD has not always structured and  implemented award fees in a way that effectively motivates contractors to  improve performance and achieve acquisition outcomes. Specifically,  GAO cited four issues with DOD’s award fee processes. GAO reported that  in many evaluation periods when rollover—the process of moving  unearned available award fee from one evaluation period to the next—was  allowed, the contractor had the chance to earn almost the entire unearned  fee, even in instances when the program was experiencing problems.  Additionally, DOD guidance and federal acquisition regulations state that  award fees should be used to motivate excellent contractor performance  in key areas. However, GAO found that most DOD award fee contracts  were paying a significant portion of the available fee from one evaluation  period to the next for what award fee plans describe as “acceptable,  average, expected, good, or satisfactory” performance. Furthermore, DOD  paid billions of dollars in award fees to programs whose costs continued  to grow and schedules increased by many months or years without  delivering promised capabilities to the warfighter. GAO also found that  some award fee criteria for DOD programs were focused on broad areas— such as how well the contractor was managing the program—instead of  criteria directly linked with acquisition outcomes—such as meeting cost  and schedule goals, and delivering desired capabilities. All of these DOD  practices contribute to the difficulty in linking elements of contractor  performance considered in award fee criteria to overall acquisition  outcomes and may lessen the motivation for the contractor to strive for  excellent performance.
subsections: section_title: Similar Problems Found in MDA Award Fee Practices
paragraphs: We assessed all award fee plans for the BMDS elements and fiscal year  2007 award fee letters for 9 of the 10 contractors. Our review revealed that  during 2007 MDA experienced some of the same award fee problems that  were prevalent in other DOD programs. MDA did not roll fee forward  often, but when it did the contractor was, in one case, able to earn 100  percent of that fee. Also, MDA allowed another contractor to earn the  unearned portion of fiscal year 2007 award fee in the same period through  a separate pool composed of the unearned fee but tied to other  performance areas. In two other instances, MDA awarded fee amounts  that were linked to very good or outstanding work in the cost and/or  program management performance element. However, during the award  fee periods, earned value data indicates that these two contractors’ cost  and/or schedule performance continued to decline. Although DOD  guidance discourages use of earned value performance metrics in award  fee criteria, MDA includes this as a factor in several of its award fee plans.   MDA considers many factors in rating contractors’ performance and  making award fee determinations, including considerations of earned  value data that shows cost, schedule, and technical trends. Table 9  provides the award fee MDA made available to its contractors, as well as  the fee earned during fiscal year 2007.
MDA is awarding some BMDS contractors a large percentage of the fees  rolled over from a prior period. The agency’s award fee plans allow the fee  determining official, at his discretion, to rollover all fee that is not  awarded during one period to a future period. For example, in accordance  with MDA’s award fee policy, the fee determining official may consider  award fee rollover when a slipped schedule moves an award fee event to  another period, it is the desire of the fee determining official to add greater  incentive to an upcoming period, and when the contractor improves  performance to such a great extent that it makes up for previous  shortfalls. During fiscal year 2007, MDA rolled fee forward for 3 of the 8  contractors for which award fee letters were available. Table 10 presents a  synopsis of this data.
As noted in table 10, MDA rolled over a large percentage of the fee that  was not earned by the THAAD contractor during fiscal year 2007. During  its last award fee period in fiscal year 2007, the THAAD contractor did not  earn any of the fee associated with cost management. The award fee letter  cited unfavorable cost variances and a growing variance projected at  completion of the contract as the reasons for not awarding any of the fee  for cost management. However, the fee determining official decided to roll  100 percent of that portion of the unearned fee to a rollover pool tied to  minimizing cost overruns. Fee will be awarded from this pool at the end of  the contract. By rolling the fee forward, MDA provided the contractor an  additional opportunity to earn fee from prior periods. Rolling over fee in  this instance may have failed to motivate the contractor to meet or exceed  expectations.
The award fee plan for the GMD contract allowed the contractor to not  only rollover fee, but to earn all unearned fee in the same period. During  the fiscal year, the GMD contractor earned 97.7 percent of the $330 million  dollars in award fees tied to performance areas outlined in the award fee  plan. However, the award fee plan made provisions for the contractor to  earn the unearned $7.5 million by creating a separate pool funded solely  from this unearned portion and awarding the fee for performance in other  areas. In this instance, the contractor did not have to wait to earn rolled  over fees in later award fee periods—it was able to receive the unearned  portion in the same period despite not meeting all of the criteria for its  original objectives. GMD officials told us that this fee incentivized the  contractor to achieve added objectives.
In contrast, the fee determining official handled rollover of fee on the ABL  contract in accordance with DOD’s new policy. According to ABL’s award  fee plan, MDA was to base its 2007 award fee decision primarily on the  outcome of three knowledge points. During this period, the contractor  completed two of the knowledge points, but could not complete a third.  To encourage the contractor to complete the remaining knowledge point  in a timely manner, the fee determining official rolled over only 35 percent  of the fee available for the event.
subsections: section_title: MDA Award Fee Plans Allow Contractors to Earn Fee for Satisfactory or Lower Performance
paragraphs: All of the award fee plans we assessed allowed MDA to award fees for  satisfactory ratings—that is, work considered to meet most of the  requirements of the contract. Some award fee plans even allow fee for  marginal performance or performance considered to meet some of the  requirements of the contract. By paying for performance at the minimum  standards or requirements of the contract, the intent of award fees to  provide motivation for excellence above and beyond basic contract  requirements is lost. While the definitions of satisfactory or marginal  differed from element to element, the award fee plans allotted roughly  more than 50 percent award fee to contractors performing at these levels.  According to the award fee plans, MDA allows between 51 and 65 percent  of available fee for work rated as marginal for the C2BMC and KEI  contractor and no less than 66 percent of available fee for satisfactory  performance by the ABL contractor. MDA’s practice of allowing more than  50 percent of available fee for satisfactory or, even, marginal performance  illustrates why DOD in April 2007 directed that no more than 50 percent of  available fee be given for satisfactory performance on all contract  solicitations commencing after August 1, 2007.
section_title: Not All Award Fees Reflect Performance as Reported by the Earned Value Management System
paragraphs: Earned value is one of several factors that according to the award fee  plans for the ABL and Aegis BMD Weapon System contractors will be  considered in rating the contractors’ cost and program management  performance. During a good part of fiscal year 2007, earned value data for  both contractors showed that they were overrunning their fiscal year cost  budgets. In addition, the ABL contractor was not completing all scheduled  work. Even considering these variances, MDA presented the contractors  with a significant portion of the award fee specifically tied to cost and/or  program management. In contrast, the THAAD contractor also  experienced downward trends in its cost variance during its last award fee  period in fiscal year 2007, but was not paid any of the award fee tied to  cost management.
The ABL and Aegis BMD Weapon System contractors received a large  percentage of the 2007 award fee available to them for the cost and/or  program management element. According to ABL’s award fee plan, one of  several factors that is considered in rating the contractor’s performance as  “very good” is whether earned value data indicates that there are few  unfavorable cost, schedule, and/or technical variances or trends. During  the award fee period that ran from February 2006 to January 2007, MDA  rated the contractor’s cost and program management performance as very  good and awarded 88 percent of the fee available for these areas of  performance. Yet, earned value data indicates that the contractor overran  its budget by more than $57 million and did not complete $11 million of  planned work. Similarly, the Aegis BMD weapon system contractor was to  be rated in one element of its award fee pool as to how effectively it  managed its contract’s cost. Similar to ABL’s award fee plan, the weapon  system contractor’s award fee plan directs that earned value data be one  of the factors considered in evaluating cost management. During the fee  period that ran from October 2006 through March 2007, MDA rated the  contractor’s performance in this area as outstanding and awarded the  contractor 100 percent of the fee tied to cost management. Earned value  data for this time period indicates that the contractor overran its budget by  more than $6 million. MDA did not provide us with more detailed  information as to other factors that may have influenced its decision as to  the amount of fee awarded to the ABL and Aegis BMD contractors.
In another instance, MDA more closely linked earned award fee to  contractor performance. The THAAD contractor continued to overrun its  2007 cost budget, and was not awarded any fee tied to the cost  management element during its last award fee period in fiscal year 2007.  The award fee decision letter cites several examples of the contractor’s  poor cost performance including cost overruns and an increased projected  cost variance at contract completion. These and other cost management  issues led the fee determining official to withhold the $9.8 million to be  awarded on the basis of cost management.
section_title: MDA Is Making Changes to Its Award Fee Policy
paragraphs: MDA has made efforts to comply with DOD policy regarding some of  GAO’s recommendations and responded to the DOD issued guidance by  releasing its own revised award fee policy in February 2007. According to  the policy, every contract’s award fee plan is directed to include:    a focus on developing specific award fee criteria for each element of    an emphasis on rewarding results rather than effort or activity, and   an incentive to meet or exceed MDA requirements.
Additionally, the directive calls for using the Award Fee Advisory Board,  established to make award fee recommendations to the fee determining  official, to biannually review and report to the Director on the consistency  between MDA’s award fees and DOD’s Contractor Performance  Assessment Report—which provides a record, both positive and negative,  on a given contract for a specific period of time. MDA’s directive also  requires program managers to implement MDA’s new award fee policy at  the earliest logical point, which is normally the beginning of the next  award fee period.
MDA is currently constructing a revised draft of its award fee guide that  addresses the rollover and rating scale issues from DOD’s March 2006 and  April 2007 memorandums. In the latest draft, MDA limits rollover to  exceptional cases and adopts the Under Secretary’s limitation of making  only a portion of award fee available for rollover. MDA’s latest draft of the  guide also makes use of the latest ratings scale, referencing the Under  Secretary’s April 2007 direction, and applies the usage of the new scales to  contract solicitations beginning after July 31, 2007.
section_title: MDA Tightens Controls on Unpriced Changes and Task Orders
paragraphs: MDA sometimes finds that events such as funding changes, technology  advances, and concurrent development and deployment of the BMDS arise  that make changes to the contract’s provisions or terms necessary. MDA  describes contract changes that are within the scope of the contract but  whose final price, or cost and fee, the agency and its contractor have not  agreed upon as unpriced changes. MDA has followed the FAR in  determining how quickly the agency should reach agreement on such  unpriced changes’ price, or cost and fee. According to the FAR, an  agreement should be reached before work begins if it can be done without  adversely affecting the interest of the government. If a significant cost  increase could result from the unpriced change, and time doesn’t permit  negotiation of a price, the FAR requires the negotiation of a maximum  price unless it is impractical to do so. In 2007, MDA began applying  tighter limits on definitization of price.
MDA also issues unpriced task orders. MDA uses this term to describe  task orders issued under established contract ordering provisions, such as  an indefinite delivery/indefinite quantity contract, for which a definitive  order price has not yet been agreed upon. MDA has followed the FAR  requirements that task orders placed under an  indefinite  delivery/indefinite quantity contract must contain, at least, an estimated  cost or fee.
During Block 2006—January 1, 2006 through December 31, 2007—MDA  authorized 137 unpriced changes and task orders with a value of more  than $6 billion. Consistent with the FAR requirements noted above, of the  total 137 unpriced changes and unpriced task orders, 61 percent of these— totaling $5.9 billion—were not priced for more than 180 days. Agreement  on the price of several was not reached for more than a year, and  agreement on the price of one was not reached for more than two and a  half years. Table 11 below shows the value of unpriced changes and task  orders issued on behalf of each BMDS element and the number of days  after the contractor was authorized to proceed with the work before MDA  and its contractor agreed to a price, or cost and fee, for the work.
Realizing that unpriced changes and unpriced task orders may greatly  reduce the government’s negotiation leverage and typically result in higher  cost and fee for the overall effort, MDA, in February 2007, issued new  contract guidance that required tighter limits on the timeframes for  reaching agreement on price, or cost and fee. The agency now applies  some of the Defense Federal Acquisition Regulation Supplement  guidelines established for undefinitized contract actions to unpriced  changes and unpriced task orders. Undefinitized contract actions are  different from MDA’s unpriced changes or unpriced task orders in that  they are contract actions on which performance is begun before  agreement on all contract terms, including price, or cost and fee, is  reached. A contract modification or change will not be considered an  undefinitized contract action if it is within the scope and under the terms  of the contract. MDA has elected to follow some of the stricter  undefinitized contract action guidelines because the agency believes the  guidelines will lead to better cost results. Similar to the undefinitized  contract action guidelines, the agency’s new guidelines require that MDA’s  unpriced changes and unpriced task orders be definitized within 180 days,  that the contractor be given a dollar value that it cannot exceed until price  agreement is reached, and that approval for the unpriced change or task  order be obtained in advance. MDA’s new policy also, to the maximum  extent practicable, limits the amount of funds that a contractor may be  given approval to spend on the work before agreement is reached on price  to less than 50 percent of the work’s expected price.
section_title: Support Contracts
paragraphs: MDA officials maintain that support contracts provide necessary  personnel and are instrumental in developing the BMDS quickly. The  agency contracts with 45 different companies that provide the majority of  the personnel who perform a variety of tasks. Table 12 illustrates the  broad categories of job functions that MDA support contractors carry out.
Last year we reported that MDA had 8,186 approved personnel positions.  This number has not changed appreciably in the last year. According to  MDA’s manpower database, about 8,748 personnel positions—not  counting prime contractors—currently support the missile defense  program. These positions are filled by government civilian and military  employees, contract support employees, employees of federally funded  research and development centers (FFRDC), researchers in university and  affiliated research centers, as well as a small number of executives on loan  from other organizations. MDA funds around 95 percent of the total 8,748  positions through its research and development appropriation. Of this 95  percent, 2,450, or about 29 percent, of the positions are set aside for  government civilian personnel. Another 60 percent, or 5,005 positions, are  allotted for support contractors. The remaining 11 percent are positions  either being filled, or expected to be filled, by employees of FFRDCs and  university and affiliated research centers that are on contract or under  other types of agreements to perform missile defense tasks. MDA officials  noted that nearly 500 of the 8,748 personnel positions available were  currently vacant. Table 13 shows the staffing levels within the BMDS  elements.
Support contractors in MDA program and functional offices may perform  tasks that closely support those tasks described in the FAR as inherently  governmental. According to the FAR, tasks such as determining agency  policy and approving requirements for prime contracts should only be  performed by government personnel. Contract personnel that, for  example, develop statements of work, support acquisition planning, or  assist in budget preparation are carrying out tasks that may closely  support tasks meeting this definition. Having support contractors  perform these tasks may create a potential risk that the contractors may  influence the government’s control over and accountability for decisions.  MDA officials told us that when support contractors perform tasks that  closely support those reserved for government employees the agency  mitigates its risk by having knowledgeable government personnel provide  regular oversight or final approval of the work to ensure that the data  being generated is reasonable.
section_title: Appendix III: Incremental Funding
subsections: section_title: Incremental Versus Full-funding of BMDS Assets
paragraphs: In the tables below we provide more information comparing the cost of  purchasing THAAD and Aegis BMD assets incrementally versus fully- funding the assets. Table 14 presents MDA’s incremental funding plans for  THAAD fire units 3 and 4, 48 Aegis BMD (SM-3) missiles to be produced  during Blocks 2012 and 2014, and 19 shipsets intended to improve the  performance of Aegis BMD ships. Tables 15 through 17 present our  analysis of the cost of purchasing these same assets with procurement  funds and following Congress’ full-funding policy.
section_title: Appendix IV: Scope and Methodology
paragraphs: To examine the progress MDA made in fiscal year 2007 toward its Block  2006 goals, we examined the accomplishments of nine BMDS elements.  The elements included in our review collectively accounted for 77 percent  of MDA’s fiscal year 2007 research and development budget request. We  evaluated each element’s progress in fiscal year 2007 toward Block 2006  schedule, testing, performance, and cost goals. In assessing each element  we examined Program Execution Reviews, test plans and reports,  production plans, Contract Performance Reports, and MDA briefing  charts. We developed data collection instruments that were completed by  MDA and each element program office. The instruments gathered detailed  information on completed program activities including tests, prime  contracts, and estimates of element performance. To understand  performance issues, we talked with officials from MDA’s Deputy for  Engineering and Program Director for Targets and Countermeasures, each  element program office, as well as the office of DOD’s Director,  Operational Test and Evaluation. To assess each element’s progress  toward its cost goals, we reviewed Contract Performance Reports and,  when available, the Defense Contract Management Agency’s analyses of  these reports. We applied established earned value management  techniques to data captured in Contract Performance Reports to  determine trends and used established earned value management formulas  to project the likely costs of prime contracts at completion. We also  interviewed MDA officials within the Deputy for Acquisition Management  office to gather detailed information regarding BMDS prime contracts. We  reviewed 10 prime contracts for the 9 BMDS elements and also examined  fiscal year 2007 award fee plans, award fee letters, and gathered data on  the number of and policy for unpriced changes and unpriced task orders.  We became familiar with sections of the Federal Acquisition Regulation  and Defense Federal Acquisition Regulation Supplement dealing with  contract type, contract award fees, and undefinitized contract actions.  To  develop data on support contractors, we held discussions with officials in  MDA’s Office of Business Operations.  We also collected data from MDA’s  Pride database on the numbers and types of employees supporting MDA  operations.
In assessing MDA’s accountability, transparency, and oversight, we  interviewed officials from the Office of the Under Secretary of Defense’s  Office for Acquisition, Technology, and Logistics and Joint Staff officials.  We also examined a Congressional Research Service report, U.S. Code,  DOD acquisition system policy, the MDEB Charter, and various MDA  documents related to the agency’s new block structure.
In determining whether MDA would save money if it fully funded THAAD  and Aegis BMD assets rather than funding them incrementally, we used  present value techniques to restate dollars that MDA planned to expend  over a number of years to the equivalent number of dollars that would be  needed if MDA fully funded the assets in the fiscal year that incremental  funding was to begin.  We also considered whether MDA would need to  acquire long lead items for the assets and stated those dollars in the base  year that their purchase would be required.  We then compared the total  cost of incrementally funding the assets, as shown in MDA’s funding plans,  to the fully funded cost that our methodology produced.
To ensure that MDA-generated data used in our assessment are reliable,  we evaluated the agency’s management control processes. We discussed  these processes with MDA senior management. In addition, we confirmed  the accuracy of MDA-generated data with multiple sources within MDA  and, when possible, with independent experts. To assess the validity and  reliability of prime contractors’ earned value management systems and  reports, we interviewed officials and analyzed audit reports prepared by  the Defense Contract Audit Agency. Finally, we assessed MDA’s internal  accounting and administrative management controls by reviewing MDA’s  Federal Manager’s Financial Integrity Report for Fiscal Years 2003, 2004,  2005, 2006, and 2007.
Our work was performed primarily at MDA headquarters in Arlington,  Virginia. At this location, we met with officials from the Aegis Ballistic  Missile Defense Program Office; Airborne Laser Program Office;  Command, Control, Battle Management, and Communications Program  Office; BMDS Targets Office, and MDA’s Agency Operations Office. We  also met with DOD’s Office of the Director, Operational Test and  Evaluation and the Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics in Washington, DC. In addition, in  Huntsville, Alabama, we met with officials from the Ground-based  Midcourse Defense Program Office, the Terminal High Altitude Area  Defense Project Office, the Kinetic Energy Interceptors Program Office,  the Multiple Kill Vehicle Program Office, and BMDS Tests Office. We also  met with Space Tracking and Surveillance System officials in Los Angeles,  California.
We conducted this performance audit from May 2007 to March 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the individual named above, Barbara Haynes, Assistant  Director; LaTonya Miller; Sigrid McGinty; Michele R. Williamson; Michael  Hesse; Steven Stern; Meredith Allen Kimmett; Kenneth E. Patton; and  Alyssa Weir made key contributions to this report.