Geostationary Operational Environmental Satellites: Steps Remain in Incorporating Lessons Learned from Other Satellite Programs
Published: 2006-09-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Oceanic and Atmospheric Administration (NOAA) plans to procure the next generation of geostationary operational environmental satellites, called the Geostationary Operational Environmental Satellites-R series (GOES-R). This new series is considered critical to the United States' ability to maintain the continuity of data required for weather forecasting through the year 2028. GAO was asked to (1) determine the status of and plans for the GOES-R series procurement, and (2) identify and evaluate the actions that the program management team is taking to ensure that past problems experienced in procuring other satellite programs are not repeated.
section_title: What GAO Found
paragraphs: NOAA is nearing the end of the preliminary design phase of its GOES-R system--which was estimated to cost $6.2 billion and scheduled to have the first satellite ready for launch in 2012. It expects to award a contract in August 2007 to develop this system. However, according to program officials, NOAA's plans for the GOES-R procurement could change in the near future. Recent analyses of the GOES-R program cost--which in May 2006 the program office estimated could reach $11.4 billion--have led the agency to consider reducing the scope of requirements for the satellite series. NOAA officials estimated that a decision on the future scope and direction of the program could be made by the end of September 2006. NOAA has taken steps to implement lessons learned from past satellite programs, but more remains to be done. Prior satellite programs--including a prior GOES series, a polar-orbiting environmental satellite series, and various military satellite programs--often experienced technical challenges, cost overruns, and schedule delays. Key lessons from these programs include the need to (1) establish realistic cost and schedule estimates, (2) ensure sufficient technical readiness of the system's components prior to key decisions, (3) provide sufficient management at government and contractor levels, and (4) perform adequate senior executive oversight to ensure mission success. NOAA has established plans to address these lessons by conducting independent cost estimates, performing preliminary studies of key technologies, placing resident government offices at key contractor locations, and establishing a senior executive oversight committee. However, many steps remain to fully address these lessons. Until it completes these activities, NOAA faces an increased risk that the GOES-R program will repeat the increased cost, schedule delays, and performance shortfalls that have plagued past procurements.

section_title: Letter
paragraphs: Operational geostationary environmental satellites play a critical role in  Operational geostationary environmental satellites play a critical role in  our nation’s weather forecasting. These satellites—which are managed by  our nation’s weather forecasting. These satellites—which are managed by  the National Oceanic and Atmospheric Administration (NOAA)—provide  the National Oceanic and Atmospheric Administration (NOAA)—provide  critical information on atmospheric, oceanic, climatic, and solar  critical information on atmospheric, oceanic, climatic, and solar  conditions that help meteorologists observe and predict global and local  conditions that help meteorologists observe and predict global and local  weather events. They also provide the best means to identify severe storm  weather events. They also provide the best means to identify severe storm  conditions, such as hurricanes and tornadoes, and to track the movement  conditions, such as hurricanes and tornadoes, and to track the movement  and intensity of these storms once they develop.  and intensity of these storms once they develop.
NOAA, with the aid of the National Aeronautics and Space Administration  NOAA, with the aid of the National Aeronautics and Space Administration  (NASA), is planning to procure the next generation of geostationary  (NASA), is planning to procure the next generation of geostationary  satellites, called the Geostationary Operational Environmental Satellites-R  satellites, called the Geostationary Operational Environmental Satellites-R  series (GOES-R). The GOES-R series is to replace the current series of  series (GOES-R). The GOES-R series is to replace the current series of  satellites which will likely begin to reach the end of their useful lives in  satellites which will likely begin to reach the end of their useful lives in  approximately 2012. This new series is expected to mark the first major  approximately 2012. This new series is expected to mark the first major  technological advance in GOES instrumentation since 1994. It is also  technological advance in GOES instrumentation since 1994. It is also  considered critical to the United States’ ability to maintain the continuity  considered critical to the United States’ ability to maintain the continuity  of data required for weather forecasting through the year 2028.  of data required for weather forecasting through the year 2028.
This report responds to your request that we review NOAA’s management  This report responds to your request that we review NOAA’s management  of the GOES-R program. Specifically, we were asked to (1) determine the  of the GOES-R program. Specifically, we were asked to (1) determine the  status of and plans for the GOES-R series procurement, and (2) identify  status of and plans for the GOES-R series procurement, and (2) identify  and evaluate the actions that the program management team is taking to  and evaluate the actions that the program management team is taking to  ensure that past problems experienced in procuring other satellite  ensure that past problems experienced in procuring other satellite  programs are not repeated.  programs are not repeated.
To determine GOES-R status and plans, we reviewed program documents  To determine GOES-R status and plans, we reviewed program documents  on the planned acquisition schedules, cost estimates, and system  on the planned acquisition schedules, cost estimates, and system  requirements, and interviewed program officials. To assess the program  requirements, and interviewed program officials. To assess the program  office’s actions to address lessons learned from past satellite programs, we  office’s actions to address lessons learned from past satellite programs, we  analyzed lessons learned from past satellite programs—including a prior  analyzed lessons learned from past satellite programs—including a prior  GOES satellite series (called GOES I-M), the National Polar-orbiting  Operational Environmental Satellite System (NPOESS), and selected  military satellite programs. We analyzed program management documents,  including data on a critical instrument’s development, to determine and  evaluate plans for addressing past lessons. We also interviewed program  officials from NOAA and NASA.
We conducted our work at NOAA and NASA offices in the Washington,  D.C., metropolitan area between December 2005 and August 2006, in  accordance with generally accepted government auditing standards.  Appendix I contains further details on our objectives, scope, and  methodology.
subsections: section_title: Background
paragraphs: Since the 1960s, geostationary and polar-orbiting environmental satellites  have been used by the United States to provide meteorological data for  weather observation, research, and forecasting. NOAA’s National  Environmental Satellite Data and Information Service (NESDIS) is  responsible for managing the civilian geostationary and polar-orbiting  satellite systems as two separate programs, called GOES and the Polar  Operational Environmental Satellites, respectively.
Unlike polar-orbiting satellites, which constantly circle the earth in a  relatively low polar orbit, geostationary satellites can maintain a constant  view of the earth from a high orbit of about 22,300 miles in space. NOAA  operates GOES as a two-satellite system that is primarily focused on the  United States (see fig. 1). These satellites are uniquely positioned to  provide timely environmental data to meteorologists and their audiences  on the earth’s atmosphere, its surface, cloud cover, and the space  environment. They also observe the development of hazardous weather,  such as hurricanes and severe thunderstorms, and track their movement  and intensity to reduce or avoid major losses of property and life.  Furthermore, the satellites’ ability to provide broad, continuously updated  coverage of atmospheric conditions over land and oceans is important to  NOAA’s weather forecasting operations.
To provide continuous satellite coverage, NOAA acquires several satellites  at a time as part of a series and launches new satellites every few years.  Three satellites—GOES-11, GOES-12, and GOES-13—are currently in orbit.
Both GOES-11 and GOES-12 are operational satellites, while GOES-13 is in  an on-orbit storage mode. It is a backup for the other two satellites should  they experience any degradation in service. The others in the series,  GOES-O and GOES-P, are planned for launch over the next few years.  NOAA is also planning a future generation of satellites, known as the  GOES-R series, which are planned for launch beginning in 2012.
Each of the operational geostationary satellites continuously transmits  raw environmental data to NOAA ground stations. The data are processed  at these ground stations and transmitted back to the satellite for broadcast  to primary weather services both in the United States and around the  world, including the global research community. Raw and processed data  are also distributed to users via ground stations through other  communication channels, such as dedicated private communication lines  and the Internet. Figure 2 depicts a generic data relay pattern from the  geostationary satellites to the ground stations and commercial terminals.
subsections: section_title: A Brief History of Prior GOES Series
paragraphs: To date, NOAA has procured three series of GOES satellites and is in the  planning stages to acquire a fourth one (see table 1).
In 1970, NOAA initiated its original GOES program based on experimental  geostationary satellites developed by NASA. While these satellites  operated effectively for many years, they had technical limitations. For  example, this series of satellites was “spin-stabilized,” meaning that the  satellites slowly spun while in orbit to maintain a stable position with  respect to the earth. As a result, the satellite viewed the earth only about 5  percent of the time and had to collect data very slowly, capturing one  narrow band of data each time its field-of-view swung past the earth. A  complete set of sounding data took 2 to 3 hours to collect.
subsections: section_title: GOES I-M Series
paragraphs: In 1985, NOAA and NASA began to procure a new generation of GOES,  called the GOES I-M series, based on a set of requirements developed by  NOAA’s National Weather Service, NESDIS, and NASA, among others.  GOES I-M consisted of five satellites, GOES-8 through GOES-12, and was a  significant improvement in technology from the original GOES satellites.  For example, GOES I-M was “body-stabilized,” meaning that the satellite  held a fixed position in orbit relative to the earth, thereby allowing for  continuous meteorological observations. Instead of maintaining stability  by spinning, the satellite would preserve its fixed position by continuously  making small adjustments in the rotation of internal momentum wheels or  by firing small thrusters to compensate for drift. These and other  enhancements meant that the GOES I-M satellites would be able to collect  significantly better quality data more quickly than the older series of  satellites.
section_title: GOES-N Series
paragraphs: In 1998, NOAA began the procurement of satellites to follow GOES I-M,  called the GOES-N series. This series used existing technologies for the  instruments and added system upgrades, including an improved power  subsystem and enhanced satellite pointing accuracy. Furthermore, the  GOES-N satellites were designed to operate longer than its predecessors.  This series originally consisted of four satellites, GOES-N through GOES- Q. However, the option for the GOES-Q satellite was cancelled based on  NOAA’s assessment that it would not need the final satellite to continue  weather coverage. In particular, the agency found that the GOES satellites  already in operation were lasting longer than expected and that the first  satellite in the next series could be available to back up the last of the  GOES-N satellites. As noted earlier, the first GOES-N series satellite— GOES-13—was launched in May 2006. The GOES-O and GOES-P satellites  are currently in production and are expected to be launched in July 2008  and July 2011, respectively.
section_title: Planned GOES-R Series
paragraphs: NOAA is currently planning to procure the next series of GOES satellites,  called the GOES-R series. This series will consist of four satellites, GOES- R through GOES-U, and is intended to provide the first major  technological advance in instrumentation since the first satellite of the  GOES I-M series was launched in 1994.
section_title: GOES-R Program—An Overview
paragraphs: NOAA is planning for the GOES-R program to improve on the technology  of prior GOES series, in terms of both system and instrument  improvements. The system improvements are expected to fulfill more  demanding user requirements and to provide more rapid information  updates. Table 2 highlights key system-related improvements GOES-R is  expected to make to the geostationary satellite program.
The instruments on the GOES-R series are expected to significantly  increase the clarity and precision of the observed environmental data.  NOAA plans to acquire five different types of instruments. The program  office considers two of the instruments—the Advanced Baseline Imager  and the Hyperspectral Environmental Suite—to be most critical because  they will provide data for key weather products. Table 3 summarizes the  planned instruments and their expected capabilities.
The program management structure for the GOES-R program differs from  past GOES programs. Prior to the GOES-R series, NOAA was responsible  for program funding, procurement of the ground elements, and on-orbit  operation of the satellites, while NASA was responsible for the  procurement of the spacecraft, instruments, and launch services. NOAA  officials stated that this approach limited the agency’s insight and  management involvement in the procurement of major elements of the  system.
Alternatively, under the GOES-R management structure, NOAA has  responsibility for the procurement and operation of the overall system— including spacecraft, instruments, and launch services. NASA is  responsible for the procurement of the individual instruments until they  are transferred to the overall GOES-R system contractor for completion  and integration onto the spacecraft. Additionally, to take advantage of  NASA’s acquisition experience and technical expertise, NOAA located the  GOES-R program office at NASA’s Goddard Space Flight Center. It also  designated key program management positions to be filled with NASA  personnel (see fig. 3). These positions include the deputy system program  director role for advanced instrument and technology infusion, the project  manager for the flight portion of the system, and the deputy project  manager for the ground and operations portion of the system. NOAA  officials explained that they changed the management structure for the  GOES-R program in order to streamline oversight and fiduciary  responsibilities, but that they still plan to rely on NASA’s expertise in  space system acquisitions.
section_title: Satellite Programs Often Experience Technical Problems, Cost Overruns, and Schedule Delays
paragraphs: Satellite programs are often technically complex and risky undertakings,  and as a result, they often experience technical problems, cost overruns,  and schedule delays. We and others have reported on a historical pattern  of repeated missteps in the procurement of major satellite systems,  including the National Polar-orbiting Operational Environmental Satellite  System (NPOESS), the GOES I-M series, the Space Based Infrared System  High Program (SBIRS-High), and the Advanced Extremely High Frequency  Satellite System (AEHF). Table 4 lists key problems experienced with  these programs and is followed by a summary of each program.
NPOESS is being developed to combine two separate polar-orbiting  environmental satellite systems currently operated by NOAA and the  Department of Defense (DOD) into a single state-of-the-art environment  monitoring system. A tri-agency program office—comprised of officials  from DOD, NOAA, and NASA—is responsible for managing this program.  Within the program office, each agency has the lead on certain activities.  NOAA has overall program management responsibility for the converged  system and for satellite operations; DOD has the lead on the acquisition;  and NASA has primary responsibility for facilitating the development and  incorporation of new technologies into the converged system.
Since its inception, the NPOESS program has encountered cost overruns  and schedule delays. Specifically, within a year of the contract award, the  program cost estimate increased by $1.2 billion, from $6.9 billion to $8.1  billion, and the expected availability of the first satellite was delayed by 20  months. We reported in September 2004 that these cost increases and  schedule delays were caused, in part, by changes in the NPOESS funding  stream. Subsequently, in November 2005, we reported that problems in  the development of a critical sensor would likely cause program costs to  grow by at least another $3 billion and the schedule for the first launch  would likely be delayed by almost 3 years. The senior executive oversight  committee for NPOESS was expected to make a decision in December  2005 on the direction of the program—which involved increased costs,  delayed schedules, and reduced functionality. We urged this committee to  make a decision quickly so that the program could proceed. However, in  late November 2005, the NPOESS program’s anticipated cost growth  triggered a legislative requirement forcing DOD to reassess its options and  to recertify the program. In June 2006, DOD decided to reduce the  system’s capabilities and number of satellites from six to four, and  announced that the newly-restructured program was estimated to cost  $11.5 billion and the launch of the first satellite had been delayed by at  least 4 years from the time the contract was awarded.
NPOESS’ problems involved a number of factors, including unrealistic  cost and schedule estimates, insufficient technical maturity of critical  sensors at a key development milestone, poor performance at multiple  levels of contractor and government management, insufficient executive  oversight, and excessive award fee payments to the contractor.  Specifically, in 2003, an Air Force cost group performed an independent  cost estimate for NPOESS and found that, based on actual outcomes from  historical programs similar to NPOESS, the program office underestimated  contract costs by almost $1 billion. This group also concluded that the  program office underestimated the required time needed to integrate the  sensors onto the spacecraft by almost 80 percent. Despite the differences  in planned cost and schedule, the program office moved forward with its  own estimates—and, in turn, established unrealistic budgets that led, in  part, to the eventual restructuring of the program.
Further, an independent review team charged with assessing the NPOESS  program found that the program management office did not sufficiently  validate the subcontractors’ design work on various sensors. As a result,  the sensors were approved to move into production before they reached a  sufficient level of technical maturity. This resulted in unexpected technical  problems during sensor production.
We also reported that the development issues on one critical sensor were  attributed, in part, to the subcontractor’s inadequate project management.  Specifically, after a series of technical problems, internal review teams  sent by the prime contractor and the program office found that the  sensor’s subcontractor had deviated from a number of contract,  management, and policy directives set out by the main office and that both  management and process engineering were inadequate. Neither the  contractor nor the program office recognized the underlying problems in  time to fix them. Further, an independent review team reported that the  program management office did not have the technical system engineering  support it needed to effectively manage the contractor. In addition, the  program office and contractor set aside less than 10 percent of their  budgets in management reserve—an amount which was insufficient to  effectively deal with these technical problems. With just 2 years into the  contract, the prime contractor had spent or allocated over 90 percent of its  reserves.
The involvement of the NPOESS executive leadership committee was also  inconsistent and indecisive—it wavered from frequent heavy involvement  to occasional meetings with few resulting decisions. In the 32-month  period from May 2003 through December 2005, the committee met  formally six times. Despite mounting evidence of the seriousness of the  critical sensor problems, the committee did not effectively challenge the  program manager’s optimistic assessments, and from May 2003 through  December 2004, convened only twice to consider the program’s status.
In May 2006, the Department of Commerce’s (Commerce) Inspector  General reported that the NPOESS award fee structure was not an  effective system for promoting high-quality performance by the contractor.  Despite the significant delays and cost overruns on the program, the  contractor received about 84 percent of the available fee pool for the first  six billing periods.
subsections: section_title: GOES I-M Series
paragraphs: In its development of the GOES I-M series, NOAA experienced severe  technical challenges, massive cost overruns, and risky schedule delays.  The overall development cost of the program was over three times greater  than planned, escalating from $640 million to approximately $2 billion.  Additionally, the launch of the first satellite of this series, which had been  planned for July 1989, did not occur until April 1994. This nearly 5-year  schedule delay left NOAA in danger of losing geostationary satellite  coverage, although no gap in coverage occurred. We reported that these  problems were caused by a number of factors, including insufficient  technical readiness of the satellite design prior to contract award,  unrealistic cost and schedule estimates, and inadequate management by  NOAA and NASA.
Specifically, NOAA and NASA did not require any engineering analyses to  be completed prior to the award of the GOES I-M contract. As a result,  both agencies were unable to anticipate the level of complexity of NOAA’s  requirements (related to the satellite’s pointing accuracy) or the  contractor’s approach to meeting those requirements. This unanticipated  design complexity led to additional analyses, redesigns, and  remanufacture of parts, which resulted in increased costs and schedule  delays. Additionally, the lack of adequate understanding of the system  prior to contract award also prevented program officials from establishing  realistic cost and schedule estimates for the program.
The inadequate management of the GOES I-M program—by both the  government and contractor—played a significant part in its cost increases  and program delays. Specifically, NASA and NOAA made the decision to  forgo preliminary studies of the system because of fiscal constraints and  pressure to launch the first satellite as quickly as possible. This decision  was compounded by NASA’s limited technical support in the areas of  optics, satellite control systems, and thermal engineering. Additionally,  both the prime contractor and major subcontractor had little experience in  directing the design of complex weather instruments. The subcontractor  had also noted that it was not prepared for GOES I-M. For example, the  instruments were expected to meet manufacturing and testing standards  that the subcontractor had never experienced before. We recommended  Congress consider directing NASA and NOAA to report on their progress  in resolving these problems and the timeframe and cost for achieving  proposed solutions. Further, we recommended that funds for the  production and testing of the satellites be withheld until a favorable  solution was identified and reported to Congress.
section_title: Space Based Infrared System High Program
paragraphs: SBIRS-High satellites are being developed to replace DOD’s older missile  warning satellites. In addition to missile warning and missile defense  missions, the satellites are also expected to perform technical intelligence  and battlespace characterization missions. After the program was initiated  in 1994, it faced cost, scheduling, and technology problems. SBIRS-High  had experienced schedule slips of at least 6 years and cost increases that  have triggered legislative requirements to reassess and recertify the  program several times—most recently in 2005. While DOD’s total program  cost estimate was initially about $3.9 billion, it is now $9.9 billion—nearly  a 150 percent unit cost increase. DOD is currently reexamining this  program, potential alternatives, and cost estimates.
Our reviews have attributed past problems on the SBIRS-High program to  serious hardware and software design problems, insufficient oversight of  contractors, and technology challenges. Further, an independent review  team chartered by DOD reported that a root cause of these problems was  that system requirements were not well understood by DOD when the  program began. Specifically, the requirements-setting process was often ad  hoc, many decisions on requirements were deferred to the contractor, and  the program was too immature to enter system design and development.  As a result, there was too much instability on the program after the  contract award—leading DOD to undertake four major replanning efforts.  We made multiple recommendations to improve this program, including  commissioning an independent task force to assess the development  schedule, the stability of the program design, and software development  practices, and to provide guidance for addressing the program’s  underlying problems. In addition, we recommended that DOD establish a  mechanism for ensuring that the knowledge gained from the assessment  was used to determine whether further programmatic changes were  needed to strengthen oversight, adjust cost and schedule estimates, and  address requirements changes.
section_title: Advanced Extremely High Frequency Satellite System
paragraphs: AEHF is a satellite system intended to be DOD’s next generation of high- speed, protected communication satellites and to replace the existing  Milstar system. In 2003, we reported that cost estimates developed by the  Air Force for this program increased from $4.4 billion in January 1999 to  $5.6 billion in June 2001 for five satellites. Moreover, DOD would not  meet its accelerated target date for launching the first satellite in  December 2004. To minimize costs, DOD then decided to purchase three  satellites with options to purchase the fourth and fifth—which brought the  program cost to $4.7 billion. Despite this action, AEHF costs grew to about  $6.1 billion—an increase of more than 15 percent over the baseline  estimate, which triggered legislative requirements to assess and certify the  program. Schedule slippages for launching this communication system  have now stretched to over 3 years.
A number of factors contributed to cost and schedule overruns and  performance shortfalls. First, in the early phases of the AEHF program,  DOD substantially and frequently altered requirements—resulting in major  design modifications that increased costs by millions of dollars. For  instance, a new requirement for additional anti-jamming protection led to  a cost increase of $100 million and an added set of requirements for  training, support, and maintainability that cost an additional $90 million.  Second, based on a satellite constellation gap caused by the failure of a  Milstar satellite, DOD accepted a high-risk schedule that turned out to be  overly optimistic and highly compressed—leaving little room for error and  depending on a chain of events taking place at certain times. Third, AEHF  allocated 4 percent of its budget to management reserve—which was an  inadequate amount to cover unforeseen problems for the duration of the  program. Between December 2002 and June 2005, the contractor had  depleted about 86 percent of its reserves with 5 years remaining on the  contract. Lastly, at the time DOD decided to accelerate the program, it did  not have the funding needed to support the activities or the manpower  needed to design and build the satellites more quickly. The lack of funding  also contributed to schedule delays, which in turn, caused more cost  increases. We made a number of recommendations to improve this  program and others, including implementing processes and policies that  stabilize requirements and addressing shortfalls in staff with science and  engineering backgrounds. These recommendations were made to assure  that DOD had an investment strategy in place that would better match  resources to requirements.
section_title: GOES-R Procurement Activities Are Under Way, but System Requirements and Cost Estimates May Change
paragraphs: NOAA is nearing the end of the preliminary design phase on its GOES-R  program and plans to award a contract for the system’s development in  August 2007; however, because of concerns with potential cost growth,  NOAA’s plans for the GOES-R procurement could change in the near  future. To date, NOAA has issued contracts for the preliminary design of  the overall GOES-R system to three vendors and expects to award a  contract to one of these vendors to develop the system in August 2007. In  addition, to reduce the risks associated with developing new instruments,  NASA has issued contracts for the early development of one critical  instrument and for the preliminary designs of four other instruments. The  agency plans to award these contracts and then turn them over to the  contractor responsible for the overall GOES-R program. However, this  approach is under review and NOAA may wait until the instruments are  fully developed before turning them over to the system contractor. Table 5  provides a summary of the status of contracts for the GOES-R program.
According to program documentation provided to the Office of  Management and Budget in 2005, the current life cycle cost estimate for  GOES-R is approximately $6.2 billion (see table 6). However, program  officials reported that this estimate is over 2 years old and is under review.
NOAA is tentatively planning to launch the first GOES-R series satellite in  September 2012. The development of the schedule for launching the  satellites was driven by a requirement that the satellites be available to  back up the last remaining GOES satellites (GOES-O and GOES-P) should  anything go wrong during the planned launches of these satellites. Table 7  provides a summary of the planned launch schedule for the GOES-R  series.
Commerce is scheduled to make a major acquisition decision before the  end of this year. Commerce will decide whether or not the GOES-R series  should proceed into the development and production phase in December  2006. Program officials reported that the final request for proposal on the  GOES-R contract would be released upon completion of this decision  milestone.
However, NOAA’s plans for the GOES-R procurement could change in the  near future because of concerns with potential cost growth. Given its  experiences with cost growth on the NPOESS acquisition, NOAA recently  asked program officials to recalculate the total cost of the estimated $6.2  billion GOES-R program. In May 2006, program officials estimated that the  life cycle cost could reach $11.4 billion. The agency then requested that  the program identify options for reducing the scope of requirements for  the satellite series. Program officials reported that there are over 10 viable  options under consideration, including options for removing one or more  of the planned instruments. The program office is also reevaluating its  planned acquisition schedule based on the potential program options.  Specifically, program officials stated that if there is a decision to make a  major change in system requirements, they will likely extend the  preliminary design phase, delay the decision to proceed into the  development and production phase, and delay the contract award date.  NOAA officials estimated that a decision on the future scope and direction  of the program could be made by the end of September 2006.
section_title: The GOES-R Program Office Has Taken Steps to Address Past Lessons Learned, but Significant Actions Remain
paragraphs: NOAA has taken steps to apply lessons learned from problems  encountered on other satellite programs to the GOES-R procurement. Key  lessons include (1) establishing realistic cost and schedule estimates,       (2) ensuring sufficient technical readiness of the system’s components  prior to key decisions, (3) providing sufficient management at government  and contractor levels, and (4) performing adequate senior executive  oversight to ensure mission success. NOAA has established plans designed  to mitigate the problems faced in past acquisitions; however, many  activities remain to fully address these lessons. Until it completes these  activities, NOAA faces an increased risk that the GOES-R program will  repeat the increased cost, schedule delays, and performance shortfalls that  have plagued past procurements.
subsections: section_title: NOAA Is Taking Steps to Improve the Reliability of Cost and Schedule Estimates, but Key Steps Remain in Reconciling Cost Estimates
paragraphs: We and others have reported that space system acquisitions are strongly  biased to produce unrealistically low cost and schedule estimates in the  acquisition process. For example, we testified last July on the continued  large cost increases and schedule delays being encountered on military  space acquisition programs—including NPOESS, SBIRS-High, and AEHF.  We noted that during program formulation, the competition to win funding  is intense and has led program sponsors to minimize their program cost  estimates. Furthermore, a task force chartered by DOD to review the  acquisition of military space programs found that independent cost  estimates and government program assessments have proven ineffective in  countering this tendency. NOAA programs face similar unrealistic  estimates. For example, the total development cost of the GOES I-M  acquisition was over three times greater than planned, escalating from  $640 million to $2 billion. The delivery of the first satellite was delayed by  5 years.
NOAA has several efforts under way to improve the reliability of its cost  and schedule estimates for the GOES-R program. NOAA’s Chief Financial  Officer has contracted with a cost-estimating firm to complete an  independent cost estimate, while the GOES-R program office has hired a  support contractor to assist with its internal program cost estimating. The  program office is re-assessing its estimates based on preliminary  information from the three vendors contracted to develop preliminary  designs for the overall GOES-R system. Once the program office and  independent cost estimates are completed, program officials intend to  compare them and to develop a revised programmatic cost estimate that  will be used in its decision on whether to proceed into system  development and production. In addition, NOAA has planned for an  independent review team—consisting of former senior industry and  government space acquisition experts—to provide an assessment of the  program office and independent cost estimates for this decision milestone.  To improve its schedule reliability, the program office is currently  conducting a schedule risk analysis in order to estimate the amount of  adequate reserve funds and schedule margin needed to deal with  unexpected problems and setbacks. Finally, the NOAA Observing System  Council submitted a prioritized list of GOES-R system requirements to  the Commerce Undersecretary for approval. This list is expected to allow  the program office to act quickly in deleting lower priority requirements in  the event of severe technical challenges or shifting funding streams.
While NOAA acknowledges the need to establish realistic cost and  schedule estimates, several hurdles remain. As discussed earlier, the  agency is considering reducing the requirements for the GOES-R program  to mitigate the increased cost estimates for the program. Therefore, the  agency’s efforts to date to establish realistic cost estimates cannot be fully  effective in addressing this lesson until this uncertainty is resolved. NOAA  suspended the work being performed by its independent cost estimator  until a decision is made on the scope of the program. Further, the agency  has not yet developed a process to evaluate and reconcile the independent  and program office cost estimates once final program decisions are made.  Without this process, the agency may lack the objectivity necessary to  counter the optimism of program sponsors and is more likely to move  forward with an unreliable estimate. Until it completes this activity, NOAA  faces an increased risk that the GOES-R program will repeat the cost  increases and schedule delays that have plagued past procurements.
section_title: NOAA Is Conducting Preliminary Studies in Order to Avoid Technical Problems in Later Acquisition Phases, but Steps Remain in Determining Components’ Technical Maturity
paragraphs: Space programs often experience unforeseen technical problems in the  development of critical components as a result of having insufficient  knowledge of the components and their supporting technologies prior to  key decision points. One key decision point is when an agency decides on  whether the component is sufficiently ready to proceed from a preliminary  study phase into a development phase; this decision point results in the  award of the development contract. Another key decision point occurs  during the development phase when an agency decides whether the  component is ready to proceed from design into production (also called  the critical design review). Without sufficient technical readiness at these  milestones, agencies could proceed into development contracts on  components that are not well understood and enter into the production  phase of development with technologies that are not yet mature. For  example:    On the GOES I-M series, NOAA and NASA did not require engineering  analyses prior to awarding the development contracts in order to  accelerate the schedule and launch the first satellite. The lack of these  studies resulted in unexpected technical issues in later acquisition  phases—including the inability of the original instrument designs to  withstand the temperature variations in the geostationary orbit.
Both the NPOESS and SBIRS-High programs committed funds for system  development before the design was proven and before the technologies  had properly matured. For instance, at the critical design review milestone  for a key NPOESS sensor, the program office decided that the sensor was  ready to proceed into production even though an engineering model had  not been constructed. This sensor has since faced severe technical  challenges that directly led to program-wide cost and schedule overruns.
To address the lesson learned from the GOES I-M experience, in 1997,  NOAA began preliminary studies on technologies that could be used on  the GOES-R instruments. These studies target existing technologies and  assessed how they could be expanded for GOES-R. The program office is  also conducting detailed trade-off studies on the integrated system to  improve its ability to make decisions that balance performance,  affordability, risk, and schedule. For instance, the program office is  analyzing the potential architectures for the GOES-R constellation of  satellites—the quantity and configuration of satellites, including how the  instruments will be distributed over these satellites. These studies are  expected to allow for a more mature definition of the system  specifications.
NOAA has also developed plans to have an independent review team  assess project status on an annual basis once the overall system contract  has been awarded. In particular, this team will review technical,  programmatic, and management areas; identify any outstanding risks; and  recommend corrective actions. This measure is designed to ensure that  sufficient technical readiness has been reached prior to the critical design  review milestone. The program office’s ongoing studies and plans are  expected to provide greater insight into the technical requirements for key  system components and to mitigate the risk of unforeseen problems in  later acquisition phases.
However, the progress currently being made on the only instrument  currently under development—the Advanced Baseline Imager—has  experienced technical problems and could be an indication of more  problems to come in the future. These problems relate to, among other  things, the design complexity of the instrument’s detectors and  electronics. As a result, the contractor is experiencing negative cost and  schedule performance trends. As of May 2006, the contractor incurred a  total cost overrun of almost $6 million with the instrument’s development  only 28 percent complete. In addition, from June 2005 to May 2006, it was  unable to complete approximately $3.3 million worth of work. Unless risk  mitigation actions are aggressively pursued to reverse these trends, we  project the cost overrun at completion to be about $23 million. (See app. II  for further detail on the Advanced Baseline Imager’s cost and schedule  performance.)
While NOAA expects to make a decision on whether to move the  instrument into production (a milestone called the critical design review)  in January 2007, the contractor’s current performance raises questions as  to whether the instrument designs will be sufficiently mature by that time.  Further, the agency does not have a process to validate the level of  technical maturity achieved on this instrument or to determine whether  the contractor has implemented sound management and process  engineering to ensure that the appropriate level of technical readiness can  be achieved prior to the decision milestone. Until it does so, NOAA risks  making a poor decision based on inaccurate or insufficient information— which could lead to unforeseen technical problems in the development of  this instrument.
section_title: Efforts to Strengthen Government and Contractor Management are Under Way, but Significant Work on Program Controls Remains
paragraphs: In the past, we have reported on poor performance in the management of  satellite acquisitions. The key drivers of poor management included  inadequate systems engineering and earned value management  capabilities, unsuitable allocation of contract award fees, inadequate  levels of management reserve, and inefficient decision-making and  reporting structure within the program office. The NPOESS program office  lacked adequate program control capabilities in systems engineering and  earned value management to effectively manage the contractor’s cost,  schedule, and technical performance. Furthermore, Commerce’s Inspector  General reported that NOAA awarded the NPOESS contractor excessive  award fees for a program plagued with severe technical problems and a  consistent failure to meet cost and schedule targets. Additionally, on  SBIRS-High, the program management office had fewer systems engineers  than other historical space programs. As a result, the program did not have  enough engineers to handle the workload of ensuring that system  requirements properly flowed down into the designs of the system’s  components. Further, the NPOESS and AEHF programs had less than 5  percent of funds allocated to management reserve at the start of the  system’s development and spent or allocated over 85 percent of that  reserve within 3 years of beginning development. On GOES I-M, NOAA  found that it did not have the ability to make quick decisions on problems  because the program office was managed by another agency.
NOAA has taken numerous steps to restructure its management approach  on the GOES-R procurement in an effort to improve performance and to  avoid past mistakes. These steps include:    The program office revised its staffing profile to provide for government  staff to be located on-site at prime contractor and key subcontractor  locations.
The program office plans to increase the number of resident systems  engineers from 31 to 54 to provide adequate government oversight of the  contractor’s system engineering, including verification and validation of  engineering designs at key decision points (such as the critical design  review milestone).
The program office has better defined the role and responsibilities of the  program scientist, the individual who is expected to maintain an  independent voice with regard to scientific matters and advise the  program manager on related technical issues and risks.
The program office also intends to add three resident specialists in earned  value management to monitor contractor cost and schedule performance.
NOAA has work under way to develop the GOES-R contract award fee  structure and the award fee review board that is consistent with our recent  findings, the Commerce Inspector General’s findings, and other best  practices, such as designating a non-program executive as the fee- determining official to ensure objectivity in the allocation of award fees.
NOAA and NASA have implemented a more integrated management  approach that is designed to draw on NASA’s expertise in satellite  acquisitions and increase NOAA’s involvement on all major components of  the acquisition.
The program office reported that it intended to establish a management  reserve of 25 percent consistent with the recommendations of the Defense  Science Board Report on Acquisition of National Security Space  Programs.
While these steps should provide more robust government oversight and  independent analysis capabilities, more work remains to be done to fully  address this lesson. Specifically, the program office has not determined  the appropriate level of resources it needs to adequately track and oversee  the program and the planned addition of three earned value management  specialists may not be enough as acquisition activities increase. By  contrast, after its recent problems and in response to the independent  review team findings, NPOESS program officials plan to add 10 program  staff dedicated to earned value, cost, and schedule analysis. An insufficient  level of established capabilities in earned value management places the  GOES-R program office at risk of making poor decisions based on  inaccurate and potentially misleading information. Finally, while NOAA  officials believe that assuming sole responsibility for the acquisition of  GOES-R will improve their ability to manage the program effectively, this  change also elevates NOAA’s risk for mission success. Specifically, NOAA  is taking on its first major system acquisition and an increased risk due to  its lack of experience. Until it fully addresses the lesson of ensuring an  appropriate level of resources to oversee its contractor, NOAA faces an  increased risk that the GOES-R program will repeat the management and  contractor performance shortfalls that have plagued past procurements.
section_title: NOAA Has Established a Senior Executive Committee to Perform Critical Oversight of the GOES-R Program
paragraphs: We and others have reported on NOAA’s significant deficiencies in its  senior executive oversight of NPOESS. The lack of timely decisions and  regular involvement of senior executive management was a critical factor  in the program’s rapid cost and schedule growth. The senior executive  committee was provided with monthly status reports that consistently  described in explicit detail the growing costs and delays attributable to the  development of a key instrument. Despite mounting evidence of the  seriousness of the instrument’s problems, this committee convened only  twice between May 2003 and December 2004 to consider the program’s  status.
NOAA formed its program management council in response to the lack of  adequate senior executive oversight on NPOESS. In particular, this council  is expected to provide regular reviews and assessments of selected NOAA  programs and projects—the first of which is the GOES-R program. The  council is headed by the NOAA Deputy Undersecretary and includes  senior officials from Commerce and NASA. The council is expected to  hold meetings to discuss GOES-R program status on a monthly basis and  to approve the program’s entry into subsequent acquisition phases at key  decision milestones—including contract award and critical design  reviews, among others. Since its establishment in January 2006, the  council has met regularly and has established a mechanism for tracking  action items to closure.
The establishment of the NOAA Program Management Council is a  positive action that should support the agency’s senior-level governance of  the GOES-R program. In moving forward, it is important that this council  continue to meet on a regular basis and exercise diligence in questioning  the data presented to it and making difficult decisions. In particular, it will  be essential that the results of all preliminary studies and independent  assessments on technical maturity of the system and its components be  reviewed by this council so that an informed decision can be made about  the level of technical complexity it is taking on when proceeding past  these key decision milestones. In light of the recent uncertainty regarding  the future scope and cost of the GOES-R program, the council’s  governance will be critical in making those difficult decisions in a timely  manner.
section_title: Conclusions
paragraphs: Procurement activities are under way for the next series of geostationary  environmental satellites, called the GOES-R series—which is scheduled to  launch its first satellite in September 2012. With the GOES-R system  development contract planned for award in August 2007, NOAA is  positioning itself to improve the acquisition of this system by  incorporating the lessons learned from other satellite procurements,  including the need to establish realistic cost estimates, ensure sufficient  government and contractor management, and obtain effective executive  oversight. However, further steps remain to fully address selected lessons.  Specifically, NOAA has not yet developed a process to evaluate and  reconcile the independent and government cost estimates. In addition,  NOAA has not yet determined how it will ensure that a sufficient level of  technical maturity will be achieved in time for an upcoming decision  milestone, or determined the appropriate level of resources it needs to  adequately track and oversee the program using earned value  management. Until it completes these activities, NOAA faces an increased  risk that the GOES-R program will repeat the increased cost, schedule  delays, and performance shortfalls that have plagued past procurements.
Recent concerns about the potential for cost growth on the GOES-R  procurement have led the agency to consider reducing the scope of  requirements for the satellite series. A decision on the future scope and  direction of the program could by made by the end of September 2006.  Once the decision is made, it will be important to move quickly to  implement the decision in the agency budgets and contracts.
section_title: Recommendations for Executive Action
paragraphs: To improve NOAA’s ability to effectively manage the procurement of the  GOES-R system, we recommend that the Secretary of Commerce direct its  NOAA Program Management Council to take the following three actions:    Once the scope of the program has been finalized, establish a process for  objectively evaluating and reconciling the government and independent  life cycle cost estimates.
Perform a comprehensive review of the Advanced Baseline Imager, using  system engineering experts, to determine the level of technical maturity  achieved on the instrument, to assess whether the contractor has  implemented sound management and process engineering, and to assert  that the technology is sufficiently mature before moving the instrument  into production.
Seek assistance from an independent review team to determine the  appropriate level of resources needed at the program office to adequately  track and oversee the contractor’s earned value management. Among  other things, the program office should be able to perform a  comprehensive integrated baseline review after system development  contract award, provide surveillance of contractor earned value  management systems, and perform project scheduling analyses and cost  estimates.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the  Department of Commerce (see app. III). In the department’s response, the  Deputy Secretary of Commerce agreed with our recommendations and  identified plans for implementing them. Specifically, the department noted  that it plans to establish a process for reconciling government and  independent cost estimates and to evaluate the process and results with an  independent team of recognized senior experts in the satellite acquisition  field. The department also noted that an independent review team is  planning to perform assessments of the technical maturity of the  Advanced Baseline Imager and the extent to which the program  management structure and reporting process will provide adequate  oversight of the GOES-R system acquisition. Additionally, the department  expressed concern regarding our use of a cost estimate that they  considered to be premature and misleading. During the course of our  review, NOAA provided us with a cost estimate that was later determined  by agency officials to be inaccurate and was subsequently corrected. We  have incorporated the revised cost estimate of $11.4 billion for the overall  GOES-R program to ensure that all cost estimates reported at this time are  accurate.
The department provided additional technical corrections, which we have  incorporated as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to interested  congressional committees, the Secretary of Commerce, the Administrator  of NASA, the Director of the Office of Management and Budget, and other  interested parties. In addition, this report will be available at no charge on  our Web site at http://www.gao.gov.
If you have any questions on matters discussed in this report, please  contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) determine the status of and plans for the  Geostationary Operational Environmental Satellites-R series (GOES-R)  procurement and (2) identify and evaluate the actions that the project  management team is taking to ensure that past problems experienced in  procuring other satellite programs are not repeated. To accomplish these  objectives, we focused our review on the National Oceanic and  Atmospheric Administration’s (NOAA) GOES-R program office, the  organization responsible for the overall GOES-R program.
To determine the status of and plans for the GOES-R series procurement,  we reviewed various program office plans and management reports such  as acquisition schedules, cost estimates, and planned system  requirements. Furthermore, we conducted interviews with NOAA and  National Aeronautics and Space Administration (NASA) officials to  determine key dates for future GOES-R acquisitions efforts and  milestones, and potential changes in program scope, cost, and schedule.
To identify the steps the GOES-R project management team is taking to  ensure that past problems experienced in procuring other satellite series  are not repeated, we analyzed our past body of work on major space  system acquisitions, including the Advanced Extremely High Frequency  satellites, the GOES I-M satellites, the National Polar-orbiting Operational  Environmental Satellite System, and the Space Based Infrared System  High program in order to identify key lessons. We also analyzed findings  from other government reports on satellite procurements, such as by the  Defense Science Board–Air Force Scientific Advisory Board Joint Task  Force and the Department of Commerce’s Office of Inspector General. We  assessed relevant management documents, such as cost reports and  program risk plans. Our evaluation included the application of earned  value analysis techniques to data from contractor cost performance  reports over an 11-month period (from June 2005 to May 2006). We also  conducted interviews with agency officials to identify and to evaluate the  adequacy of the actions taken to address these lessons.
We obtained comments on a draft of this report from officials at the  Department of Commerce and incorporated these comments as  appropriate. We performed our work at NOAA and NASA offices in the  Washington, D.C., metropolitan area between December 2005 and August  2006 in accordance with generally accepted government auditing  standards.
section_title: Appendix II: Current Shortfalls in Contractor Performance on Key Instrument Development
paragraphs: The development of one of the critical GOES-R instruments, the Advanced  Baseline Imager (ABI), is experiencing technical challenges and, as a  result, the contractor is missing cost and schedule targets. Despite the  uncertainty regarding the future scope of the GOES-R program, it is  expected that the requirements for this instrument will not change.
Contractor-provided data from June 2005 to May 2006 indicates that ABI’s  cost performance is experiencing a trend of negative variances. Figure 4  shows the 11-month cumulative cost variance for the ABI contract. As of  May 2006, the contractor has incurred a total cost overrun of almost $6  million with ABI development only 28 percent complete. This information  is useful because trends tend to continue and can be difficult to reverse  unless management attention is focused on key risk areas and risk  mitigation actions are aggressively pursued. Studies have shown that, once  work is 15 percent complete, the performance indicators are indicative of  the final outcome.
Based on contractor performance from June 2005 to May 2006, we  estimated that the current ABI instrument contract—which is worth  approximately $360 million—will overrun its budget by between $17  million and $47 million. Our projection of the most likely cost overrun will  be about $23 million. The contractor, in contrast, estimates about a $7  million overrun at completion of the ABI contract. Given that the  contractor has 72 percent of work remaining and has already accumulated  a cost overrun of $5.9 million, the likelihood that the contractor will meet  its estimated projection is small.
Our analysis also indicates that the contractor has been unable to meet its  planned schedule targets. Figure 5 shows the 11-month cumulative  schedule variance for the ABI contract. From June 2005 to May 2006, the  contractor was unable to complete approximately $3.3 million worth of  scheduled work. The contractor reported that its incorporation of revised  subcontractor budgets resulted in the fluctuations in schedule  performance data prior to March 2006. The current inability to meet  contract schedule performance could be a predictor of future rising costs,  as more spending is often necessary to resolve schedule overruns.
According to contractor-provided documents, the cost and schedule  overruns were primarily caused by design complexity issues experienced  in the development of the instrument’s detectors and the electronics  design for the cryocooler and the unplanned time and manpower  expended to resolve these issues. Other significant cost and schedule  drivers include software issues on the scanner and supplier quality issues  on some parts.
section_title: Appendix III: Comments from the Department of Commerce
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
paragraphs: David A. Powner, (202) 512-9286 or pownerd@gao.gov.
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Carol Cha, Neil Doherty, Nancy  Glover, Kush Malhotra, Colleen Phillips, and Karen Richey made key  contributions to this report.