Space Acquisitions: DOD's Goals for Resolving Space Based Infrared System Software Problems Are Ambitious
Published: 2008-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 1996, DOD initiated the Space Based Infrared System (SBIRS) to replace the nation's current missile detection system, and to provide expanded missile warning capability. Since then, SBIRS has been restructured several times to stem cost increases and schedule delays, including revising program goals in 2002, 2004, and 2005. These actions were partly due to the challenges of developing sophisticated technologies and software. In 2007, SBIRS had a major setback when flight software for the first satellite underwent testing and failed, a failure caused by design issues. DOD developed a plan for resolving these issues, and revised its cost and schedule goals. GAO has assessed (1) the approach used to mitigate the problems, and (2) the cost and schedule risks and challenges of that approach. To conduct our work, GAO has contacted, met with, and performed detailed work at numerous DOD and contractor offices; and reviewed technical documents on flight software.
section_title: What GAO Found
paragraphs: To mitigate the SBIRS flight software problems, DOD has assessed various alternatives and developed a way to implement the software redesign and oversee its development. In April 2008, DOD approved the redesign effort, which addressed problems with the original design that affected the timing of stored programs, distribution of control between processors, and failure at the hardware interface level. Six review teams comprised of 70 personnel in all evaluated the designs to ensure the technical solutions, development approach, and readiness of test facilities were adequate. DOD and its contractor are now implementing the simplified architecture, developing new software, and testing elements critical to the integration and test of systems. DOD is also improving its program oversight and better managing the SBIRS development, by acting on the recommendations of an Independent Program Assessment; addressing weaknesses in management responsibility, accountability and organizational structure; and establishing a central execution team. DOD has estimated that the SBIRS program will be delayed by 15 months and cost $414 million in funding to resolve the flight software problems, but these estimates appear optimistic. For example, confidence levels--based on the program's ability to develop, integrate, and test software in time to meet the schedule goal--have been assessed as low. Further, the review teams who approved the designs to start coding software report that the program's aggressive schedule is a major challenge because it allows "little margin for error." DOD has also introduced risk by granting waivers to streamline the software development processes to meet the aggressive schedule. These allow the program to deviate from disciplined processes in order to compress the schedule and meet the goal. In addition, some software elements are behind schedule, and thousands of software activities and deliverables remain to be integrated. Delay by these other programs could create unintended consequences for the SBIRS launch goal. If DOD should need additional time or encounter problems beyond what was planned for, more funds will be needed and launch of the first satellite in December 2009 could be jeopardized.

section_title: Letter
paragraphs: In 1996, the Department of Defense (DOD) initiated the Space Based  Infrared System (SBIRS), a satellite missile warning system, to replace the  nation’s current missile detection system and to provide expanded  capabilities to support intelligence, surveillance, and reconnaissance.  Since its inception, SBIRS has been burdened by underestimated software  and technical complexities, poor oversight, and other problems that have  resulted in cost overruns and years in schedule delays. DOD had expected  to field SBIRS by 2004 at a cost of $4.2 billion; however, SBIRS is now  estimated to cost over $10.4 billion, and the first satellite launch is  expected in 2009—a 7-year delay.
In 2006, you requested that we review the SBIRS program. In response, we  reported on an array of problems the program was still facing, particularly  with respect to software development, the expenditure of management  reserves, and deferred requirements. Subsequent to our work, SBIRS  experienced another major setback in January 2007 when the flight  software for the first satellite underwent testing and failed. The flight  software controls and monitors the satellite’s health and status and is  considered a critical component of the satellite. In April 2007, DOD  determined that the software failure was caused by design issues that  affected the timing of stored programs, among other problems. DOD also  developed a plan for resolving the issues, and associated cost and  schedule goals.
Given the importance of flight software to the first SBIRS satellite and its  cost and schedule impact on the SBIRS program, we agreed to follow up  on our work and assess the software management, development, and  mitigation efforts. Specifically, we (1) identified DOD’s approach to  mitigate the SBIRS flight software problems, and (2) assessed the cost and  schedule risks and challenges of that approach.
To conduct our work for this report, we contacted the Office of the  Secretary of Defense (OSD), Air Force, and contractor offices. We also  conducted detailed work and held discussions with both the Air Force and  Lockheed Martin on their efforts to manage, mitigate, and redesign the  flight software that is to operate, control, and monitor the satellite’s  health, status, and safety. We reviewed technical software plans,  assessments, analyses, and independent reviews pertaining to the flight  software’s redesign, and held discussions with key Air Force and  contractor officials on various aspects of the flight software development  for SBIRS. In addition, we drew from our body of past work on weapon  systems acquisitions practices and used disciplined software practices as  criteria. We conducted this performance audit from April 2008 to August  2008 in accordance with generally accepted auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. Appendix I further discusses our scope and  methodology.
subsections: section_title: Background
paragraphs: DOD initiated the SBIRS program to meet all military infrared surveillance  requirements through a single, integrated system, and to provide better  and timelier data to the Unified Combatant Commanders, U.S. deployed  forces, U.S. military strategists, and U.S. allies. SBIRS is to replace the  existing infrared system, the Defense Support Program, which has  provided early missile warning information since the 1970s. The SBIRS  program was originally conceived as having high- and low-orbiting space- based components and a ground segment for mission-data processing and  control to improve current capabilities. In 2001, the SBIRS Low  component was transferred from the Air Force to the Missile Defense  Agency and renamed the Space Tracking and Surveillance System. The Air  Force continued developing SBIRS High (herein referred to as “SBIRS”).  It, along with its associated ground segment, is one of DOD’s highest  priority space programs.
The SBIRS program originally consisted of four satellites to operate in  geosynchronous earth orbit (GEO), plus one spare, an infrared sensor  placed on two host satellites in highly elliptical orbit (HEO)—known as  “HEO sensors”—and a ground segment for mission-data processing and  control.
The SBIRS GEO satellite is designed to support two infrared sensors—a  scanning sensor and a staring sensor. The first GEO satellite is commonly  referred to as GEO 1. Figure 1 shows the GEO satellite that is to operate in  space.
As a result of past technical and program difficulties experienced during  sensor and satellite development, the SBIRS program has encountered  cost and schedule increases. These difficulties have led DOD to  restructure the program multiple times, including revising program goals  in 2002, 2004, and 2005. For example, in 2002, the program faced serious  problems with software and hardware design progress and, in the  Conference Report accompanying the National Defense Authorization Act  for Fiscal Year 2002, conferees recommended cutting advance  procurement funding due to concerns about program developments and  the unclear status of the SBIRS program. At that time, the first satellite  launch slipped from 2002 to 2006. In late 2005, SBIRS was restructured for  a third time which stemmed from a 160 percent increase in estimated unit  cost, triggering a fourth Nunn-McCurdy breach, which again postponed  the delivery of promised capabilities to the warfighter.
subsections: section_title: Flight Software
paragraphs: The flight system software is expected to control the GEO satellite’s  mission critical functions and activities. Unlike other software programs  that can be deferred and uploaded to the satellite after launch, the flight  software cannot be deferred because it is critical to the satellite’s  operation and function. The flight software is expected to operate, control,  and monitor the GEO satellite’s health, status, and safety. Based on the  original design, the flight software was to operate on two of four computer  processors onboard the satellite and perform important functions and  operations, such as telemetry, thermal control, power management, and  fault detection activities. Figure 2 shows a simplified diagram of the  original flight software design.
section_title: Origin and Chronology of Flight Software Events
paragraphs: In 1996, development of the flight software began as an independent  research and development project by Lockheed Martin—referred to as  reusable flight software (RFSW)—to be used for multifunctional “bus”  purposes. In 2004, the RFSW was provided to the SBIRS program for  development as the flight system software to operate, control, and monitor  the GEO satellite’s health, status, and safety. At that time, the software  needed to address 1261 requirements in order to satisfy the specific flight  software system needs for the GEO satellite. From 2005 to 2006, the Air  Force and Lockheed Martin conducted detailed requirements reviews that  resulted in the delivery of flight software that was integrated into the  satellite’s computers.
In January 2007, the flight software underwent testing in a space  representative environment called thermal vacuum testing and  experienced a higher number of unexpected and unexplained failures. By  April 2007, in additional tests, the number of problems escalated well  beyond what was expected. At this time, Lockheed Martin notified DOD of  the seriousness of the problem. From April 2007 to July 2007, the Air Force  and Lockheed Martin analyzed the problems and developed two options:    modify the existing software or redesign the software by simplifying the architecture, developing more  software, and increasing the robustness of the fault management  system.
The Air Force chose to redesign the software architecture and began its  work with Lockheed Martin on detailed software designs from September  2007 to December 2007. In March 2008, the new design underwent  Incremental Design Review Block 1 and was approved by the program  review board for the revised cost and schedule baseline. In April 2008, six  independent review teams examined the Block 2 design during the  Systems Engineering & Incremental Design Review and authorized the Air  Force and Lockheed to proceed with formal software coding under the  redesign.
section_title: DOD Is Taking Steps to Mitigate Software Problems, Including Initiatives to Improve Program Oversight
paragraphs: To mitigate the software problems, DOD has assessed various alternatives  and developed an approach for implementing the software redesign effort  and overseeing its development. DOD and the SBIRS contractor are taking  steps to address problems, among others, with the original software  architecture. DOD has redesigned the architecture, and is in the midst of  developing additional software, and testing elements critical to the  integration and test of systems. DOD has also undertaken several  initiatives to improve its program oversight and to help it better manage  the development, including addressing weaknesses in program  management responsibility, accountability, and other areas.
subsections: section_title: Steps Have Been Undertaken to Address Poor Software Architecture
paragraphs: To address the software’s poor architectural design that ultimately  resulted in the unexpected loss of telemetry and commanding for  extended periods and unexpected hardware errors, a trade study was  conducted by Lockheed Martin to examine options for redesign. Table 1  shows the trade study options considered, and recommendations made.
As indicated in table 1, the trade study recommended a simplified  architecture that places all the software applications on a single processor,  processor “A”, rather than using distributed applications because it  represents the best fit with system designs. Lockheed Martin officials  stated that the simplified software architecture will address a number of  areas that were problematic with the original design, such as the timing of  stored programs that failed during thermal vacuum tests. Among other  elements, the new design will involve the development of additional  software that will also increase the robustness of the fault management  system.
subsections: section_title: Major Redesign Approved for Coding Software
paragraphs: Approved in April 2008, the new designs have undergone numerous  reviews, the last of which was subjected to comprehensive and detailed  examination involving six independent review teams. Teams comprised of  personnel—from the Office of the Under Secretary of Defense for  Acquisition, Technology, and Logistics; Aerospace Corporation, a federally  funded research and development center (FFRDC); Lockheed Martin  Corporate; Air Force Space and Missiles Systems Center Wing; and the  Software Engineering Institute—evaluated the technical solutions,  development approach, and readiness of the test facilities, among other  elements.
The objective of the design review was to authorize the start of formal  software coding. For the incremental design review, independent review  teams were provided detailed information about software issues on the  original design, including the severity of the issues and the status of each.  Other information included DOD’s approach in managing risk, resolution  of critical issues, disposition of deficiency reports, requirements volatility,  and integration with ground systems. Technical data included diagrams of  the simplified architecture, operating system interface design, and lines of  software code that would be impacted from earlier designs. Other  information about the software included designs of subsystems,  schematics, integration and delivery schedules, and productivity and sizing  estimates.
section_title: Progress Is Being Made to Develop Software and Conduct Tests
paragraphs: DOD is making progress to develop needed software and conduct tests of  elements that are critical to the first satellite system, called GEO 1. For  example, in June 2008, DOD held a design review on software for the fault  management system that elicited concurrence from external stakeholders  to proceed with coding activities. At the same time, they held a space  technical interchange meeting that provided consensus on the  methodology and a plan for complete space vehicle testing, including the  flight software. In July 2008, Lockheed Martin delivered 63,000 of the  projected 67,000 source lines of code for the space vehicle and ground  software integration effort, including a database that provided data so that  development efforts could continue on ground software and testing  activities.
According to Lockheed Martin, software development efforts followed a  disciplined process, except in those cases where waivers were requested  and granted by the software engineering process group. Figure 2 shows  Lockheed Martin’s process for developing and qualifying flight software.
section_title: Qualification
paragraphs: DOD has taken steps to fund critical test bed resources that are needed to  adequately test, model, analyze, and simulate software functions as a  means to reduce integration and test risks, in response to lessons learned  from the failed software that identified the need to add and upgrade their  simulation and test bed resources. For example, an evaluation of the  software problems found several contributory factors that prevented them  from identifying the software problems earlier. These include:  test beds that had matured in parallel with the flight software and  hardware, making it difficult to distinguish between test bed and  software issues;     oversubscription of test beds and lack of simulation resources that  precluded them from checking out high-risk areas (timing, and stored  programs); and  insufficient modeling of timing, and analysis of stored program  implementation, which might have shed light earlier on lack of  robustness.
In May 2008, the additional test bed and simulator was brought online and  is currently in use.
section_title: Actions Have Been Undertaken to Address Program Weaknesses, and Improve Oversight of GEO Development
paragraphs: DOD and Lockheed Martin have undertaken several initiatives to address  areas of program risk, such as efforts to improve oversight of GEO 1 and  flight software development. These include acting on recommendations  made in an Independent Program Assessment (IPA) that was conducted to  ensure the validity of the technical, cost, and schedule baselines. As part  of the assessment, the IPA study assessed contractor performance,  evaluated program risk areas, and made recommendations on where  program improvements could be made. In November 2007, officials from  the Air Force, Lockheed Martin, and Aerospace Corporation reported the  IPA findings. Table 2 shows the IPA findings, recommendations, and status  of implementation efforts.
As indicated in table 2, the Air Force and Lockheed Martin have taken  actions to address areas of risk. Among others, these actions included  deliberately emphasizing the software development process where  adherence to process disciplines was lacking, and enhancing the  interaction between cost and schedule functions where the Air Force  organization structure was found to be flawed because it did not mirror  the contractor’s more traditional approach where these functions are  combined for better program control.
To improve the oversight and management of the GEO 1 satellite and  software development, the Air Force and Lockheed Martin established a  dedicated execution team with a focus on overseeing the test, integration,  and assembly of software and hardware, and ensuring delivery of the GEO  1 satellite. The execution team is a joint effort that includes the Air Force,  Lockheed Martin, and Aerospace Corporation. As part of the management  approach, the execution team is responsible for conducting daily meetings  to review “inch stone” metrics and to resolve issues. The execution team  also meets weekly with the Executive Program Management leadership to  provide early insight on issues and resolve organizational weaknesses, and  conduct monthly reviews with senior executives to provide consistent  communication and allow opportunity for guidance. According to DOD  officials, the execution team not only improved oversight of software  development and management of the GEO 1 effort, but also addressed  weaknesses identified in the IPA study. For example, these weaknesses  included, among others, the need to fix the program’s responsibility,  accountability, and authority disconnects. Officials reported that the  execution team helped alleviate the strained relationships that had existed  between the Air Force and Lockheed Martin where adversarial  relationships and morale problems were evident.
section_title: DOD’s Plan for Resolving the Software Problem Is Optimistic
paragraphs: While DOD has estimated that the SBIRS program will be delayed by 15  months and cost $414 million to resolve the software problems, those  estimates appear too optimistic, given the cost and schedule risks  involved. For example, SBIRS contractors’ report low confidence that  software can be produced in time to meet the December 2009 satellite  launch goal. Further, DOD and the contractor face significant challenges  and risks that could result in more time and money being required to meet  program goals, to include the bypassing of some disciplined software  practices that add risk to cost and schedule. Finally, as of August 2008,  DOD reported that SBIRS was already behind schedule on some software  development efforts, and thousands of activities remain that must be  integrated and tested across various systems, with cost and schedule  implications, if problems or unintended consequences occur.
subsections: section_title: Low Confidence That Software Can Be Produced to Meet Cost and Schedule Goals
paragraphs: A major concern is the infeasibility of producing the software in time to  meet the estimated launch goal. For example, technical contractors— Aerospace Corporation, Galorath Inc., and Lockheed Martin—estimated  the confidence to be “low” that software can be developed within the tight  time frames. These estimates are based on widely accepted models  (System Evaluation and Estimation of Resources, Software Estimating  Model, and Risk Assessment) that take into account the effective size of  the software, staffing of the effort, complexity, volatility of software  requirements, and integration and risk of anticipated rework and failure in  system tests. Using DOD’s self-imposed baseline schedule goal, software  productivity estimates show very low confidence levels that the schedule  goal can be met. Table 3 shows the confidence in meeting the GEO 1  launch goal in December 2009 (various models used).
As indicated in table 3, one estimate shows only a 5 percent confidence  that the software can be produced in time to meet the schedule goal, while  the other estimate shows a less than 10 percent confidence level.  Lockheed’s own software productivity estimate shows a 50 percent  confidence level in meeting the December 2009 launch schedule, but its  estimate assumes (1) a higher productivity than has been demonstrated,  and (2) the software will require less effort, which has not been the  program’s experience. According to DOD’s Cost Analysis Improvement  Group, if productivity on software does not materialize, or problems occur  during testing and integration beyond what was marginally planned for,  then it could cost an additional $400 million for each year of schedule  slippage.
section_title: Major Challenge and Risks to the Redesign and Development Effort Still Exist
paragraphs: Based on an April 2008 review of the revised software designs and  software development approach, the independent review teams— comprised of personnel from the Office of the Under Secretary of Defense  for Acquisition, Technology, and Logistics; Aerospace Corporation;  Lockheed Martin Corporate; Air Force Space and Missiles Systems Center  Wing; and the Software Engineering Institute—concluded that the  program should proceed with formal software coding, but also expressed  concern about the ambitious schedule. Specifically, the review teams cited  the program’s aggressive schedule as a major challenge because it allows  “little margin for error” and concluded the program faces high risk of not  meeting the schedule. Table 4 shows the weaknesses and risks to software  development.
Although the Air Force and Lockheed Martin are committed to the effort  and have built in a 120-day margin to fix unexpected and unforeseeable  problems, a computer engineer from the Defense Contract Management  Agency who is familiar with the program believes that the margin is  insufficient because the planned schedule considers only routine  development activities, and that additional time will likely be needed to  address any unanticipated problems.
subsections: section_title: Bypassing Disciplined Software Practices Adds Risk
paragraphs: Further, to meet the cost and schedule goals, the program is using  approaches that will increase program risk. These risks stem from  waivers, which were requested by Lockheed Martin, as specified by  software provisions in the program’s software development process. In  following the SBIRS Software Development Plan, for Flight Software  System 1.5, waivers were generated and approved by a software  engineering process group so that developers could deviate from the  established processes. These deviations from the disciplined development  process allowed the program to shortcut important processes in order to  meet the ambitious schedule goal, rather than follow a disciplined process  to develop software. For example, a waiver was granted for software  design to be done in parallel with the software specification activity.  However, according to DOD, the risk is that requirements could be  rejected and that rework may be required in coding or design. Another  waiver was granted for software unit integration testing to be done in  parallel with formal unit testing. According to DOD, the risk is that formal  unit testing may find problems that were not identified during prior  informal (developer) unit testing, thereby necessitating possible rework.
section_title: Cost and Schedule Goals Are at Risk Because Some Software Elements Are Behind Schedule, and Complex Integration and Other Activities Remain
paragraphs: Some of the flight software’s elements are already behind schedule and a  significant amount of activities remain to be done, posing concern to DOD.  For example, DOD reported that, as of August 2008, the software  qualification test case and script development effort was already a month  behind schedule. Also, final delivery of the Block 2 flight software is now  forecasted to be at least 2 weeks late. Other problems that could set back  SBIRS are the thousands of integration and coordination activities that  must take place as they ramp up. For example, Lockheed Martin reports  that the schedule has more than 14,500 tasks that will occur, beginning in  January 2008, across multiple systems. This means that the flight software  test activities and integration efforts must all be integrated in a “single- flow” without consequence across a broad spectrum of systems, such as  integration with ground, space, and database systems, among others.  Software experts, independent reviewers, and government officials  acknowledged that the aggressive schedule, when combined with the  significant amount of work that remains, is the biggest challenge facing the  program.
Still, there are external factors that could create schedule impacts for  meeting the SBIRS schedule goal. For example, DOD reports that the GEO  1 satellite launch could be affected by other satellites scheduled to launch  prior to the SBIRS launch. Essentially, these launch activities use the same  launch range resources that will be required to launch the GEO 1 satellite,  and delays in any of these events could create unintended consequences to  the SBIRS GEO 1 launch goal.
section_title: Conclusions
paragraphs: Given the technical complexity of the program and SBIRS’ poor program  history, it is unwise for DOD to pursue such ambitious goals for resolving  the flight software problem. More than 12 years after its inception, the  SBIRS program continues to face major challenges that have proven  technically challenging and substantially more costly than originally  envisioned. The testing failure of the flight software is further proof that  sophisticated technology and inherent complexities related to software  continue to be underestimated. To its credit, DOD has instilled greater  discipline by involving outside experts, regaining control of development  activities, and dealing with the poor relationships that had existed for  some time. To ensure that such steps can lead to success, adherence to  disciplined software practices should be made a priority over steps or  measures taken to compress the schedule for the sake of meeting the self- imposed launch goal. Prioritizing such disciplines will improve efforts to  acquire a better product, increase executability of the program, and  reduce program risk. In turn, establishing goals that are synchronized with  such priorities will allow DOD to achieve expectations and program  deliverables with greater reliability. Essentially, these will position the  leadership to better direct investments by establishing goals with greater  confidence that they can be achieved.
section_title: Recommendations for Executive Action
paragraphs: To better ensure that SBIRS can meet the cost and schedule goals for  resolving the flight software problems as well as launch the first satellite  on schedule, we recommend that the Secretary of Defense   revise the cost and schedule estimates based on more realistic  assumptions to increase the confidence of success, and  require that the contractor make adherence to disciplined software  practices a priority to reduce program risk.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. DOD  partially concurred with our recommendation to revise the cost and  schedule estimates based on more realistic assumptions, and concurred  with our recommendation to require the contractor to make adherence to  disciplined practices a priority. DOD’s comments appear in appendix II.
In its comments, DOD partially concurred with the recommendation that  the cost and schedule estimates be revised based on more realistic  assumptions to increase the confidence of success. DOD noted that the  current goals are executable on the basis of available management reserve  and schedule margin. In the event that the program encounters any  unforeseeable problems that may cause further delays, DOD stated that  Congress has approved an additional $45 million in funding to mitigate any  future launch delays. The department pointed out that OSD is working  with the SBIRS program to hold a more specific review of the flight  software. Based on the results of this review, DOD stated it would  consider them in any decision to modify the cost and schedule estimates.  DOD expects these assessments to be complete by the end of the 2008  calendar year.
As indicated in our report, SBIRS has been restructured several times  because it underestimated the technical complexity and inherent  challenges associated with software, among other technical elements.  Neither the software assessment conducted to determine the confidence  of producing software nor the independent reviewers who examined the  redesign approach indicated that the current goals were executable.  Rather, as we noted, software experts, independent reviewers, as well as  the government officials we interviewed expressed concern over the  aggressive schedule and questionable schedule margin, which the Defense  Contract Management Agency believes is insufficient. Moreover, as we  previously reported and noted in this report, the expenditure of  management reserves has been particularly problematic because these  funds were being rapidly spent. Further, while OSD’s plan to assess  software and its willingness to revise the cost and schedule goals appear  plausible, we believe this approach falls well short of a more reasonable  approach to increase the confidence of success for the reasons we cited.  In light of the program’s risks, poor performance history, and technical  challenges expected during integration, we maintain that establishing  goals that are based on more realistic assumptions would place DOD in a  better position to achieve cost and schedule goals with greater confidence.
DOD concurred with the second recommendation stating that adherence  to disciplined software development processes improves the quality and  predictability of the software development while reducing the amount of  rework. DOD further states that the program office and the contractor  jointly accepted two process waivers to streamline the process, but that  these waivers have had no adverse impact on the software development  effort. In order to keep the focus on quality software deliveries, DOD  noted that the program would disapprove any waivers which might  compromise the team’s ability to complete the development.
We are encouraged by DOD’s efforts to adhere to disciplined software  processes to improve the quality and predictability of development. In this  endeavor, DOD states that it would disapprove any waivers that could  compromise the development effort. However, it is unclear exactly what  criteria DOD will use to determine whether a waiver will compromise  development efforts. Without this, there is no mechanism to ensure that  any waivers that are granted will not have a material effect on software  development.
We also received technical comments from DOD which have been  addressed in the report, as appropriate.
We are sending copies of this report to the Secretary of Defense; the Office  of the Under Secretary of Defense for Acquisition, Technology and  Logistics; the Secretary of the Air Force; and the Director, Office of  Management and Budget. Copies will also be made available to others on  request. In addition, the report will be made available at no charge on the  GAO Web site at http://www.gao.gov.
If you, or your staff, have any questions concerning this report, please  contact me at (202) 512-4589. Contact points for our offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. The major contributors are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To identify the Space Based Infrared System’s (SBIRS) approach to mitigate  the flight software problems, we reviewed the plans and alternatives the  Department of Defense (DOD) put in place to mitigate the software  problem. We also interviewed Air Force, Defense Contract Management  Agency, and Lockheed Martin officials who were responsible for  management and oversight of the software development effort. We also  examined technical reports, studies, and analyses about the factors that  contributed to the flight software problems, as well as planning documents  and alternatives that were considered in fixing the software problem.
To assess the cost and schedule risks and challenges of the way forward, we  held discussions with both the DOD and Lockheed Martin on their efforts to  assess the program risks and challenges, including their approach to  manage, mitigate, and redesign the flight software that is to operate, control  and monitor the satellite’s health, status, and safety. We also reviewed  schedules, risk reports, analyses, program assessments, and independent  review reports pertaining to the flight software’s redesign, and selected  assessments by independent sources that were used, in part, as basis for  selecting December 2009 as the launch goal for the GEO 1 satellite. We also  interviewed Air Force and contractor officials responsible for developing  and executing the redesign, including a contractor hired for their expertise  in estimating software productivity.
We conducted this performance audit at the Office of the Secretary of  Defense, Washington D.C.; Space and Missile Systems Center, Los Angeles  Air Force Base, California; and Lockheed Martin and the Defense Contract  Management Agency, Sunnyvale, California from April to August 2008 in  accordance with generally accepted auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. In addition, we drew from our body of past  work on weapon systems acquisition practices and disciplined software  practices. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: Contact
section_title: Acknowledgments
paragraphs: In addition to the individual named above, Arthur Gallegos, Assistant  Director; John M. Ortiz Jr.; Claire A. Cyrnak; Madhav S. Panwar; Bob S.  Swierczek; and Alyssa B. Weir made key contributions to this report.
section_title: Related GAO Products
paragraphs: Space Acquisitions: Major Space Programs Still at Risk for Cost and  Schedule Increases. GAO-08-552T. Washington, D.C.: March 4, 2008.
Space Acquisitions: Space Based Infrared System High Program and Its  Alternative. GAO-07-1088R. Washington, D.C.: September 12, 2007.
Defense Acquisitions: Assessments of Selected Weapon Programs.   GAO-07-406SP. Washington, D.C.: March 30, 2007.
Space Acquisitions: Actions Needed to Expand and Sustain Use of Best  Practices. GAO-07-730T. Washington, D.C.: April 19, 2007.
Space Acquisitions: DOD Needs to Take More Action to Address  Unrealistic Initial Cost Estimates of Space Systems. GAO-07-96 (DATE?)
Space Acquisitions: Improvements Needed in Space Systems  Acquisitions and Keys to Achieving Them. GAO-06-626T. Washington,  D.C.: April 6, 2006.
Space Acquisitions: Stronger Development Practices and Investment  Planning Needed to Address Continuing Problems. GAO-05-891T.  Washington, D.C.: July 12, 2005.
Defense Acquisitions: Stronger Management Practices Are Needed to  Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393.  Washington, D.C.: March 1, 2004.
Defense Acquisitions: Risks Posed by DOD’s New Space Systems  Acquisition Policy. GAO-04-0379R. Washington, D.C.: January 29, 2004.
Defense Acquisitions: Improvements Needed in Space Systems  Acquisition Policy to Optimize Growing Investment in Space. GAO-04- 253T. Washington, D.C.: November 18, 2003.
Best Practices: Better Matching of Needs and Resources Will Lead to  Better Weapon System Outcomes. GAO-01-288. Washington, D.C.:   March 8, 2001.