Geostationary Operational Environmental Satellites: Progress Has Been Made, but Improvements Are Needed to Effectively Manage Risks
Published: 2007-10-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Commerce's National Oceanic and Atmospheric Administration (NOAA), with the aid of the National Aeronautics and Space Administration (NASA), plans to procure the next generation of geostationary operational environmental satellites, called the Geostationary Operational Environmental Satellites-R series (GOES-R). This series is considered critical to the United States' ability to maintain the continuity of data required for weather forecasting through the year 2028. GAO was asked to (1) assess the status and plans for GOES-R, and (2) evaluate whether NOAA is adequately mitigating key technical and programmatic risks. To do so, GAO analyzed contractor and program data and interviewed officials from NOAA and NASA.
section_title: What GAO Found
paragraphs: NOAA has made progress in planning its GOES-R procurement--which is estimated to cost $7 billion and scheduled to have the first satellite ready for launch in 2014--but cost and schedules are likely to grow. Specifically, the agency completed preliminary design studies of GOES-R and recently decided to separate the space and ground elements of the program into two separate development contracts. However, this change in the GOES-R acquisition strategy has delayed a decision to proceed with the acquisition. Further, independent estimates are higher than the program's current cost estimate and convey a low level of confidence in the program's schedule. Independent studies show that the estimated program could cost about $2 billion more, and the first satellite launch could be delayed by 2 years. As NOAA works to reconcile the independent estimate with its own program office estimate, costs are likely to grow and schedules are likely to be delayed. To address cost, schedule, and technical risks, the GOES-R program has established a risk management program and has taken steps to mitigate selected risks. For example, as of July 2007, the program office identified the lack of an integrated master schedule to be its highest priority risk and established plans to bring this risk to closure. However, more remains to be done to fully address GOES-R risks. Specifically, the program has multiple risk watchlists that are not always consistent and key risks are missing from the watchlists, including risks associated with unfilled executive positions, limitations in NOAA's insight into NASA's deliverables, and insufficient funds for unexpected costs--called management reserves. As a result, the GOES-R program is at risk that problems will not be identified or mitigated in a timely manner and could lead to program cost overruns and schedule delays.

section_title: Letter
paragraphs: Operational geostationary environmental satellites play a critical role in  Operational geostationary environmental satellites play a critical role in  our nation’s weather forecasting. These satellites—which are managed by  our nation’s weather forecasting. These satellites—which are managed by  the National Oceanic and Atmospheric Administration (NOAA)—provide  the National Oceanic and Atmospheric Administration (NOAA)—provide  critical information on atmospheric, oceanic, climatic, and solar  critical information on atmospheric, oceanic, climatic, and solar  conditions that help meteorologists observe and predict global and local  conditions that help meteorologists observe and predict global and local  weather events. They also provide a means to identify severe storm  weather events. They also provide a means to identify severe storm  conditions, such as hurricanes and tornadoes, and to track the movement  conditions, such as hurricanes and tornadoes, and to track the movement  and intensity of these storms once they develop.  and intensity of these storms once they develop.
NOAA, with the aid of the National Aeronautics and Space Administration  NOAA, with the aid of the National Aeronautics and Space Administration  (NASA), is planning to procure the next generation of geostationary  (NASA), is planning to procure the next generation of geostationary  satellites, called the Geostationary Operational Environmental Satellites-R  satellites, called the Geostationary Operational Environmental Satellites-R  series (GOES-R). The GOES-R series is to replace the current series of  series (GOES-R). The GOES-R series is to replace the current series of  satellites which will likely begin to reach the end of their useful lives in  satellites which will likely begin to reach the end of their useful lives in  approximately 2014. This new series is expected to mark the first major  approximately 2014. This new series is expected to mark the first major  technological advance in GOES instrumentation since 1994. It is also  technological advance in GOES instrumentation since 1994. It is also  considered critical to the United States’ ability to maintain the continuity  considered critical to the United States’ ability to maintain the continuity  of data required for weather forecasting through the year 2028.  of data required for weather forecasting through the year 2028.
This report responds to your request that we review NOAA’s management  This report responds to your request that we review NOAA’s management  of the GOES-R program. Specifically, we were asked to (1) assess the  of the GOES-R program. Specifically, we were asked to (1) assess the  status and revised plans for the GOES-R procurement, and (2) evaluate  status and revised plans for the GOES-R procurement, and (2) evaluate  whether NOAA is adequately mitigating key technical and programmatic  whether NOAA is adequately mitigating key technical and programmatic  risks facing the GOES-R program.  risks facing the GOES-R program.
To assess GOES-R status and revised plans, we reviewed program  documents on the revised acquisition strategy, system requirements, and  cost estimates, and interviewed officials from NOAA, NASA, and the  contractor for a critical GOES-R instrument. To evaluate whether NOAA is  adequately mitigating key technical and programmatic risks, we analyzed  risk management plans and also met with internal technical experts to  determine the key risks that could negatively affect the program’s ability  to acquire the GOES-R system. We also interviewed agency officials to  evaluate the adequacy of NOAA’s mitigation efforts.
We conducted our work at NOAA and NASA offices in the Washington,  D.C., metropolitan area and at ITT Corporation in Fort Wayne, Indiana. We  performed our work from April 2007 to September 2007, in accordance  with generally accepted government auditing standards. Appendix I  contains further details on our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: Since the 1960s, geostationary and polar-orbiting environmental satellites  have been used by the United States to provide meteorological data for  weather observation, research, and forecasting. NOAA’s National  Environmental Satellite Data and Information Service (NESDIS) is  responsible for managing the civilian geostationary and polar-orbiting  satellite systems as two separate programs, called GOES and the Polar  Operational Environmental Satellites, respectively.
Unlike polar-orbiting satellites, which constantly circle the earth in a  relatively low polar orbit, geostationary satellites can maintain a constant  view of the earth from a high orbit of about 22,300 miles in space. NOAA  operates GOES as a two-satellite system that is primarily focused on the  United States (see fig. 1). These satellites are uniquely positioned to  provide timely environmental data to meteorologists and their audiences  on the earth’s atmosphere, its surface, cloud cover, and the space  environment. They also observe the development of hazardous weather,  such as hurricanes and severe thunderstorms, and track their movement  and intensity to reduce or avoid major losses of property and life.  Furthermore, the satellites’ ability to provide broad, continuously updated  coverage of atmospheric conditions over land and oceans is important to  NOAA’s weather forecasting operations.
To provide continuous satellite coverage, NOAA acquires several satellites  at a time as part of a series and launches new satellites every few years.  Three satellites—GOES-11, GOES-12, and GOES-13—are currently in orbit.  Both GOES-11 and GOES-12 are operational satellites, while GOES-13 is in  an on-orbit storage mode. It is a backup for the other two satellites should  they experience any degradation in service. The others in the series,  GOES-O and GOES-P, are planned for launch over the next few years.
NOAA is also planning the next generation of satellites, known as the  GOES-R series, which are planned for launch beginning in 2014.  Each of the operational geostationary satellites continuously transmits  raw environmental data to NOAA ground stations. The data are processed  at these ground stations and transmitted back to the satellite for broadcast  to primary weather services and the global research community here in  the United States and abroad. Raw and processed data are also distributed  to users via ground stations through other communication channels, such  as dedicated private communication lines and the Internet. Figure 2  depicts a generic data relay pattern from the geostationary satellites to the  ground stations and commercial terminals.
subsections: section_title: A Brief History of Prior GOES Series
paragraphs: To date, NOAA has procured three series of GOES satellites and is in the  planning stages to acquire a fourth one (see table 1).
In 1970, NOAA initiated its original GOES program based on experimental  geostationary satellites developed by NASA. While these satellites  operated effectively for many years, they had technical limitations. For  example, this series of satellites was “spin-stabilized,” meaning that the  satellites slowly spun while in orbit to maintain a stable position with  respect to the earth. As a result, the satellite viewed the earth only about 5  percent of the time and had to collect data very slowly, capturing one  narrow band of data each time its field of view swung past the earth. A  complete set of sounding data for GOES-4 through GOES-7 took 2 to 3  hours to collect.
subsections: section_title: GOES I-M Series
paragraphs: In 1985, NOAA and NASA began to procure a new generation of GOES,  called the GOES I-M series, based on a set of requirements developed by  NOAA’s National Weather Service, NESDIS, and NASA, among others.  GOES I-M consisted of five satellites, GOES-8 through GOES-12, and was a  significant improvement in technology from the original satellites. For  example, GOES I-M was “body-stabilized,” meaning that the satellite held a  fixed position in orbit relative to the earth, thereby allowing for  continuous meteorological observations. Instead of maintaining stability  by spinning, the satellite would preserve its fixed position by continuously  making small adjustments in the rotation of internal momentum wheels or  by firing small thrusters to compensate for drift. This and other  enhancements meant that the GOES I-M satellites would be able to collect  significantly better quality data more quickly than the older series of  satellites.
section_title: GOES-N Series
paragraphs: In 1998, NOAA began the procurement of satellites to follow GOES I-M,  called the GOES-N series. This series used existing technologies for the  instruments and added system upgrades, including an improved power  subsystem and enhanced satellite pointing accuracy. Furthermore, these  satellites were designed to operate longer than their predecessors. This  series originally consisted of four satellites—GOES-N through GOES-Q.  However, the option for the GOES-Q satellite was cancelled based on  NOAA’s assessment that it would not need the final satellite to continue  weather coverage. In particular, the agency found that the satellites  already in operation were lasting longer than expected and that the first  satellite in the next series could be available to back up the last of the  GOES-N satellites. As noted earlier, the first GOES-N series satellite— GOES-13—was launched in May 2006. The GOES-O and GOES-P satellites  are currently in production and are expected to be launched in July 2008  and July 2011, respectively.
section_title: Planned GOES-R Series
paragraphs: NOAA is currently planning to procure the next series of GOES satellites— called the GOES-R series—which is intended to provide the first major  technological advance in instrumentation since the launch of the first  satellite of the GOES I-M series in 1994. The original plans for this series  included four satellites—GOES-R through GOES-U. However, in  September 2006, NOAA reduced the scope of the program to consist of a  minimum of two satellites and removed a technically complex instrument  because it was estimated that program costs could almost double the  original cost estimate.
section_title: GOES-R Program—An Overview
paragraphs: NOAA plans for the GOES-R program to improve on the technology of  prior series, in terms of both system and instrument improvements. The  system improvements are expected to fulfill more demanding user  requirements and to provide more rapid information updates. Table 2  highlights key system-related improvements GOES-R is expected to make  to the geostationary satellite program.
In addition to the system improvements, the instruments on the GOES-R  series are expected to significantly increase the clarity and precision of the  observed environmental data. NOAA originally planned to acquire six  different types of instruments. Furthermore, two of these instruments— the Advanced Baseline Imager and the Hyperspectral Environmental  Suite—were considered to be the most critical because they would  provide data for key weather products. Table 3 summarizes the originally  planned instruments and their expected capabilities. providing higher resolution and faster coverage.
Expected to continuously monitor lightning activity over the United States and provide a  more complete dataset than previously possible. Key features include detecting lightning strikes as an indicator of severe storms and providing a new capability to GOES that only previously existed on polar satellites.
Expected to provide information on the general level of geomagnetic activity, monitor  current systems in space, and permit detection of magnetopause crossings, sudden  storm commencements, and substorms.
Space Environmental In-Situ Suite (SEISS)  Expected to provide information on space weather to aid in the prediction of particle  precipitation, which causes disturbance and disruption of radio communications and  navigation systems.  Key features include measuring magnetic fields and charged particles, providing improved heavy ion detection, adding low energy electrons and protons,  and  enabling early warnings for satellite and power grid operation, telecom services,  astronauts, and airlines.
Expected to provide coverage of the entire dynamic range of solar X-ray features, from  coronal holes to X-class flares, as well as estimate the measure of temperature and  emissions. Key features include providing images of the sun and measuring solar output to monitor solar storms and providing improved imager capability.
SIS development work was divided into two separate acquisitions, the Solar Ultra Violet Imager  (SUVI) and the Extreme Ultraviolet and X-Ray Irradiance Suite (EXIS).
More recently, however, NOAA reduced the scope of the GOES-R program  because of expectations of higher costs. In May 2006, the program office  projected that total costs, which were originally estimated to be $6.2  billion, could reach $11.4 billion. We reported that this led NOAA to reduce  the scope and technical complexity of the baseline program. Specifically,  in September 2006, NOAA reduced the minimum number of satellites from  four to two, cancelled plans for developing the Hyperspectral  Environmental Suite, and estimated the revised program would cost $7  billion. Table 4 provides a summary of the timeline and scope of these key  changes.
In our September 2006 report, we made a number of recommendations to  improve the agency’s ability to effectively manage GOES-R based on  lessons learned from other satellite procurements. In particular, we  recommended that NOAA establish a process for objectively evaluating  and reconciling government and independent cost estimates once the  scope of the program was finalized. We also recommended that the agency  perform a comprehensive review of the Advanced Baseline Imager to  determine the level of technical maturity achieved prior to moving the  instrument into production and to seek assistance in determining the  appropriate levels of resources needed at the program office to adequately  track and oversee the contractor’s earned value management data. The  agency agreed with our recommendations and has completed a  comprehensive review of the Advanced Baseline Imager. Actions on other  recommendations are under way. For example, NOAA has established a  team to conduct a line-by-line reconciliation of the government and  independent cost estimates. Additionally, NOAA plans to have four earned  value specialists on staff by the end of 2007 and has requested that NASA  determine the appropriate level of resources needed to perform its earned  value management functions.
section_title: GOES-R Program Office Structure
paragraphs: NOAA is solely responsible for GOES-R program funding and overall  mission success. However, since it relies on NASA’s acquisition  experience and technical expertise to help ensure the success of its  programs, NOAA implemented an integrated program management  structure with NASA for the GOES-R program (see fig. 3). Within the  program office, there are two project offices that manage key components  of the GOES-R system. These are called the flight and operations project  offices. The flight project office oversees the spacecraft, instruments, and  launch services. The operations project office oversees the ground  elements and on-orbit operations of the satellites. The project manager for  the flight project office and the deputy project manager for operations  project office are designated to be filled with NASA personnel.  Additionally, NOAA has located the program office at NASA’s Goddard  Space Flight Center.
section_title: Planned GOES-R Acquisition Strategy
paragraphs: NOAA’s acquisition strategy was to award contracts for the preliminary  design of the GOES-R system to several vendors who would subsequently  compete for the contract to be the single prime contractor responsible for  overall system development and production. As such, in October 2005,  NOAA awarded contracts for the preliminary design of the overall GOES-R  system to three vendors.
In addition, to reduce the risks associated with developing technically  advanced instruments, NASA awarded contracts for the preliminary  designs for five of the originally planned instruments. NASA expected to  subsequently award development contracts for these instruments and to  eventually turn them over to the prime contractor responsible for the  overall GOES-R program.
section_title: GOES-R Preliminary Design Studies Are Completed, but Key Program Changes Have Been Made and Cost and Schedule Estimates Are Likely to Grow
paragraphs: NOAA has completed preliminary design studies of its GOES-R  procurement. In addition, the agency recently decided to separate the  space and ground elements of the program into two separate contracts to  be managed by NASA and NOAA, respectively. However, this change has  delayed a key decision to proceed with the acquisition, which was planned  for September 2007. Further, independent estimates are higher than the  program’s current $7 billion cost estimate and convey a low level of  confidence in the program’s schedule for launching the first satellite by  2014. As NOAA works to reconcile the independent estimate with its own  program office estimate, costs are likely to grow and schedules are likely  to be delayed.
subsections: section_title: Progress Has Been Made on GOES-R Procurement Activities
paragraphs: NOAA and NASA have made progress on GOES-R. The program office has  completed preliminary design studies of the overall GOES-R system and  has initiated development work on most of the planned instruments.  Specifically, the NOAA-issued contracts for the preliminary design of the  overall GOES-R system to three vendors have ended, and the designs have  been completed.
In addition, after completing preliminary designs on five of the originally  planned instruments, NASA awarded development contracts for three of  them. Further, the most critical of these instruments—the Advanced  Baseline Imager—has completed a major development milestone. In  February 2007, it passed a critical design review gate and NASA approved  the contractor to begin production of a prototype model. See table 5 for  the summary of the contracts on GOES-R and its component instruments.
section_title: NOAA Revised its Acquisition Strategy
paragraphs: NOAA recently made a number of key changes in how it plans to acquire  the GOES-R system. Originally, NOAA planned to award and manage a  single prime contract for the acquisition and operation of the integrated  system. However, an independent review team assessed the program and  found that this approach was risky. It recommended that NOAA split the  acquisition effort into two separate contracts for the space and ground  segments and have NASA manage the space segment. The independent  review team concluded that there was less risk in continuing with this  approach than there would be if NOAA took on a new and expanded role.
In March 2007, Commerce approved NOAA’s decision to implement these  recommendations. The agency revised its acquisition strategy to include  two separate contracts—the space segment and the ground segment. The  two contracts are expected to be awarded in May 2008 and August 2008,  respectively. The space segment is to be managed by a NASA-led flight  project office. As such, NASA is to be responsible for awarding and  managing the space segment contract, delivering the flight-ready  instruments to the space segment contractor for integration onto the  satellites, and overseeing the systems engineering and integration. NOAA  is to be responsible for the ground segment contract, which is to be  managed by the NOAA-led operations project office.
The revised acquisition strategy has delayed NOAA’s plans to complete a  key decision milestone on whether to proceed with GOES-R development  and production in September 2007. Once this decision is made, the final  requests for proposals on the system segments are to be released. The  agency could not provide a timeframe for when this key decision  milestone would take place.
section_title: GOES-R Cost Estimates Are Likely to Grow and Schedule Estimates Are Likely to Slip
paragraphs: NOAA’s current estimate that the life cycle cost of the GOES-R program  would be $7 billion is likely to grow, and its estimate that the first satellite  would be launched in December 2014 is likely to slip. Consistent with best  practices in cost estimating, in May 2007, NOAA had two different cost  estimates completed for the current GOES-R program—one by its program  office and one by an independent cost estimating firm. The program office  estimated with 80 percent confidence that the program would cost $6.9  billion. The independent estimating firm estimated with 80 percent  confidence that the program would cost $9.3 billion.
A comparison of the two cost models shows that the independent  estimator has about a 20 percent level of confidence that the program can  be completed for $6.9 billion. Further, the independent estimator  concluded that the program office estimate significantly understated the  risk of cost overruns. Other major differences between the two estimates  are contained in government costs and in the space and ground segments.  In commenting on a draft of this report, NOAA officials noted that one of  the differences between the estimates is the inflation rate. The  independent estimator assumed a higher inflation rate than the rate that  NOAA and NASA typically use. NOAA officials noted that if the  independent estimate was adjusted to NOAA’s inflation rate, the program’s  cost estimate with 80 percent confidence would be $8.7 billion. However,  we believe that the value of an independent estimate is that it does not  necessarily use the same assumptions as the program office. By offering  alternative assumptions, the independent estimate provides valuable  information for government officials to consider when revising program  cost estimates.
Program officials are reconciling the two different cost estimates and plan  to establish a new program cost estimate to be released in conjunction  with the President’s fiscal year 2009 budget in February 2008. Program  officials were unable to provide us information on the reconciled estimate  until it is released. Nonetheless, the revised cost estimate will likely be $1  billion more than the current $7 billion.
Regarding schedule, NOAA’s current plan to launch the first GOES-R  series satellite in December 2014 could be delayed.  This schedule was  driven by a requirement that the satellites be available to back up the last  remaining GOES satellites (GOES-O and GOES-P) should anything go  wrong during the planned launches of these satellites (see table 6).  However, as part of its cost estimate, the independent estimator  performed a schedule risk analysis. The independent estimator determined  that there was less than a 50 percent chance that the first satellite would  be ready for launch by December 2014 and that a later date would be more  realistic. The estimator determined that it had 50 percent confidence that  the first satellite would launch by October 2015 and 80 percent confidence  that the satellite would launch by March 2017. A delay of this magnitude  could affect the continuity of GOES data should the agency experience  problems with the predecessor satellites.
section_title: NOAA is Taking Steps to Address Key Risks, but More Remains to Be Done
paragraphs: To address cost, schedule, and technical risks, the GOES-R program  established a risk management program and has taken steps to identify  and mitigate selected risks. However, more remains to be done to fully  address a comprehensive set of risks. Specifically, the program has  multiple risk watchlists and they are not always consistent. Further, key  risks are missing from the risk lists, including risks associated with  unfilled executive positions, limitations in NOAA’s insight into NASA’s  deliverables, and insufficient funding for unexpected costs (called  management reserve) on a critical sensor. As a result, the GOES-R  program is at increased risk that problems will not be identified or  mitigated in a timely manner and that they could lead to program cost  overruns and schedule delays.
subsections: section_title: GOES-R Has a Risk Management Program and Is Taking Measures to Address Selected Risks
paragraphs: The GOES-R program office established a risk management program and  is tracking and mitigating selected risks. Risk management is a leading  management practice that is widely recognized as a key component of a  sound system development approach. An effective risk management  approach typically includes identifying, prioritizing, and mitigating risks,  and escalating key risks to the attention of senior management.
In accordance with leading management practices, the GOES-R program  identifies risks, assigns a severity rating to risks, tracks these risks in a  database, plans response strategies for each risk in the database, and  reviews and evaluates these risks during monthly program risk  management board meetings. Programwide and project-specific risks are  managed by different offices. The program office identifies and tracks  programwide risks—those that affect the overall GOES-R program.
NASA’s flight project office and NOAA’s operations project office manage  risks affecting their respective aspects of the program. Further, the  program office briefs senior executives on top program and project risks  on a monthly basis.
subsections: section_title: GOES-R Program Office Identified and Is Working to Mitigate Programwide Risks
paragraphs: As of July 2007, the program office identified three program risks affecting  the overall GOES-R program (see fig. 4). These risks include the  development of the integrated master schedule, the ability to secure  authorization to use a key frequency band to meet the space-to-ground  communication data link requirements for the GOES-R system, and the  final approval of the GOES-R mission requirements from the NOAA  Deputy Undersecretary.
NOAA is working to mitigate and close program risks that it is tracking.  For example, the program office recently closed the risk associated with  GOES-R requirements because it had sufficiently defined and obtained  approval of these requirements. As another example, the program office  considers the lack of an integrated master schedule to be its highest  priority risk. Program officials reported that completion of the integrated  master schedule is driven by the completion of the intermediate schedules  for the ground segment and the space-to-ground interdependencies. Key  program staff members, including a resident scheduler, meet on a weekly  basis to resolve outstanding design issues and hone these schedules.  Program officials reported that the intermediate schedules are near  completion and that they plan to have the integrated master schedule  completed in Fall 2007. They expect to remove this issue from the risk  watchlist at that time.
section_title: NASA Identified Flight Segment Risks and Is Working to Mitigate Them
paragraphs: As of July 2007, the NASA flight project office identified four risks  affecting instrument development, all of which are classified as medium  risk (see fig. 5). The top three risks pertain to the advanced imaging  instrument, ABI—including issues on timely and quality subcontractor  delivery of a critical part, stray light negatively impacting the performance  of the optical system, and meeting specified performance requirements on  image navigation and registration. The fourth priority risk pertains to the  improvement of subcontractor quality assurance on a key sensor for the  Space Environmental In-Situ Suite.
NASA is working to mitigate the flight segment risks that it is tracking. For  example, the ABI contractor, among other things, plans to complete a key  simulation review before the end of the year (called the structural thermal  optical performance analysis) to evaluate whether the instrument can  meet its expected performance parameters for image navigation and  registration. NASA also recently conducted a vendor facility assessment of  the SEISS subcontractor to determine whether adequate quality assurance  improvements had been made to be compliant with contract requirements.  These actions are expected to help mitigate the risk.
section_title: NOAA Identified Risks in its Operations Segment and Is Working to Mitigate Them
paragraphs: As of July 2007, the NOAA operations project office identified five risks  impacting the management and development of the ground system and  operations, including one that is identified as a medium risk (see fig. 6).  These risks include, among other things, inadequate definition of flight and  operations project interdependencies, algorithm development  responsibilities, and the adequate definition of coordination requirements  between the space and ground segments to ensure that the two requests  for proposals are consistent.
NOAA is working to mitigate the ground system and operations risks that  it is tracking. For example, for the highest priority risk regarding schedule  interdependencies, key staff from both the flight and operations projects  meet weekly in order to identify and synchronize project schedules. The  project office expects to close this risk in Fall 2007.
section_title: Multiple Watchlists Are Not Consistent, Making it Difficult to Prioritize and Manage Risks
paragraphs: While GOES-R has implemented a risk management process, its multiple  risk watchlists are not consistent in areas where there are  interdependencies between the lists, which makes it difficult to effectively  prioritize and manage risks at the appropriate organizational levels. Sound  risk management practices call for having a consistent prioritization  approach and for significant problems to be elevated from the component  level to the program level. This is because an issue affecting a critical  component could have severe programmatic implications and should be  identified, tracked, and overseen at the program level. In addition,  program executives should be briefed regularly on the status of key risks.
However, on the GOES-R program, the risks identified on the multiple risk  lists are inconsistent in areas where there are interdependencies between  the lists. These interdependencies include situations where a risk is raised  by one project office and affects the other project office, but is not  identified by the other project office or elevated to the program level risk  list. They also include situations where a risk identified by a project office  has programwide implications, but is not elevated to the program level  risk list. For example, the operations project office identified schedule  interdependencies between the flight and operations project offices as a  medium criticality risk, but neither the flight project office nor the  program identified this risk even though it is relevant to both. As another  example, the operations project office identified the ground procurement  schedule as a major issue in its briefing to senior management, but this  risk was not identified on its own or on the programwide risk lists.
In addition, while the three offices brief senior management about their  key risks on a monthly basis, selected risks may not be accurately  depicted in these briefings because of the inconsistencies among the risk  watchlists. For example, both the flight and operations project offices  identified technical development issues as minor to moderate risk areas,  but the program office did not identify this item as a risk and, when it  briefed senior management, it noted that technical development was in  good shape. Figure 7 depicts examples of inconsistencies among risk lists  and briefings to senior management.
The lack of consistency in managing risks in areas where there are  interdependencies makes it difficult to ensure that all identified risks are  appropriately prioritized and managed. This situation hampers the  program office’s ability to identify and mitigate risks early on and to  anticipate and manage the impact of risks on other areas of the program.
section_title: Important GOES-R Management Risks Are Missing from the Program Watchlist
paragraphs: To be effective, a risk management program should have a comprehensive  list of risks. However, several key risks that impact the GOES-R  procurement and merit agency attention are not identified in the  program’s risk lists. These risks include (1) key leadership positions that  need to be filled, (2) NOAA’s limited insight into NASA’s deliverables, and  (3) insufficient management reserves (held by the program and a key  instrument contractor). At the conclusion of our review, program officials  stated that they are aware of these issues and working to monitor them or  address them, as warranted. Nevertheless, until these and other  programwide risks are identified and addressed as part of a  comprehensive risk management program, there is increased likelihood  that issues will be overlooked that could affect the acquisition of the  GOES-R system.
subsections: section_title: Key GOES-R Leadership Positions Need to be Filled
paragraphs: The two senior GOES-R program positions—the system program director  and deputy system program director—are currently filled by NASA and  NOAA personnel in an acting capacity until they can be permanently filled  by NOAA. In addition, the acting system program director is not able to  work full time in this role because she is also on a special assignment as  the NESDIS Deputy Assistant Administrator for Systems. NOAA reported  that it plans to fill the deputy system program director role in the near  future, but noted that it could take more than 6 months to fill the system  program director role. Given the approach of the development phase of  the GOES-R acquisition and the competing priorities of the acting system  program director, it is especially important that these key leadership  positions be filled quickly. At the conclusion of our review, agency  officials stated that they are aware of this issue and working to fill the  positions, but they did not believe the issue warranted inclusion on the  program level risk watch list. However, without the senior level attention  inherent in a sound risk management program, it is not clear that NOAA is  sufficiently focused on the importance of establishing knowledgeable and  committed program executives, or in moving quickly to fill these critical  positions.
section_title: NOAA’s Insight into NASA’s Program Elements Is Limited
paragraphs: NOAA’s March 2007 decision to adopt an acquisition management  approach similar to prior GOES procurements could make the agency  vulnerable to repeating some of the problems experienced in the past. In  particular, our work on the GOES I-M series found that NOAA did not have  the ability to make quick decisions on problems because portions of the  procurement were managed by NASA. In fact, NOAA officials originally  intended to depart from this approach as a lesson they learned from the  GOES I-M acquisition, because it limited the agency’s insight and  management involvement in the procurement of major elements of the  system.
The established NOAA/NASA interagency agreements require NASA to  submit monthly contractor cost performance reports to NOAA and to alert  NOAA should cost and schedule performance drop below certain  thresholds. NASA is currently submitting the required reports and has  alerted NOAA on major cost and schedule changes. However, these  interagency agreements do not contain provisions that enable NOAA to  ensure that the data and reports are reliable and that they accurately  depict contractor performance. To do so would entail NOAA having the  ability and means to question and validate data, such as by having direct  access to the contractor.
NASA and NOAA officials reported that the two agencies are working  together with an unparalleled level of transparency and noted that NOAA  program staff have access to contractor data and can bring any questions  with the data to the relevant NASA staff. However, they acknowledged  that this process is not documented and were not able to demonstrate that  NOAA staff had questioned contract data and that NASA had facilitated  obtaining answers to the questions. By not identifying and mitigating this  risk on its program risk list, NOAA increases the likelihood that the   GOES-R program will repeat the management and contractor shortfalls  that plagued past GOES procurements.
section_title: Recent Changes on a Key Instrument Have Reduced Program Management Reserve Funds and Limited Contractor Reserve Funds Leave GOES-R Vulnerable to Future Cost Increases
paragraphs: A recent modification to the critical ABI instrument contract increased its  cost, thereby reducing the amount of management reserve funds held by  the program office for unexpected expenses. In September 2006, we  reported that ABI was experiencing technical challenges, resulting in cost  and schedule overruns. Since then, the contractor continued missing cost  and schedule targets—a trend that continued until February 2007. At that  time, NASA modified the contract to implement a revised baseline cost  and schedule. The added cost of this modification was funded using  management reserve funds held by the GOES-R program office. As a  result, the amount of reserve held by the program office dropped below 25  percent—a level that NOAA reported it intended to establish as a lesson  learned from other satellite acquisitions. As of July 2007, the program’s  reserve level was at about 15 percent. Program officials stated that their  revised goal is to maintain between 10 and 15 percent in reserve at the  program level. While maintaining a 10 to 15 percent management reserve is  on par with other major satellite acquisitions, the depletion of  management reserves this early in the GOES-R acquisition raises concerns  that there will be insufficient reserves during the challenging development,  integration, and testing phases to come.
In addition, the contractor for the ABI instrument has a very low level of  reserve funding for unexpected costs, which means that any unexpected  problems will likely lead to cost growth on the overall GOES-R program.  As of May 2007, the contractor was holding less than 1 percent of funding  in reserve to cover unexpected costs associated with the 40 percent of  work left to be completed. As such, there is a risk that the new baseline  could fail due to inadequate reserves to finish the program. This would  likely have a diminishing effect on the reserve held by the GOES-R flight  project and the program office to cover the costs of a second revised  baseline plan. Our prior work on system acquisitions has shown  inadequate reserves to be an indicator of poor management performance  that could lead to cost overruns. Considering that GOES-R has not yet  entered the development and production phase, it will be critical for  NOAA’s senior executive management to aggressively manage this risk. By  not identifying, mitigating, and tracking this risk in a programwide risk list,  the GOES-R program runs an increased risk that unanticipated issues on  the ABI instrument will lead to programwide cost overruns and schedule  delays.
section_title: Conclusions
paragraphs: Over the last year, NOAA has completed preliminary design studies of its  GOES-R system and has made a number of key changes to the program.  Current program plans call for a two-satellite program—estimated to cost  about $7 billion—with launch of the first satellite in December 2014.  However, independent studies show that the program’s cost could  increase by about $2 billion more and the first launch could be delayed by  at least 2 years.
NOAA has taken steps to identify and address key risks but more could be  done to effectively manage risks from a programwide perspective. In  particular, the program has multiple risk watchlists that are not consistent  in areas where there are interdependencies and key risks have not been  elevated for programwide attention. Also, several risks that warrant  NOAA’s attention have not been placed on any watchlist. Specifically, the  top two leadership positions are only temporarily filled; NOAA does not  have the ability and means to obtain insight into NASA contracts in order  to validate contractor performance data; and insufficient management  reserves to handle unexpected problems on a critical instrument and at  the program level are likely to affect overall program costs when any  unexpected problems arise. Until NOAA manages and addresses a  comprehensive set of program risks, the agency’s ability to effectively  manage the GOES-R acquisition will be significantly weakened and could  lead to substantial program overruns and delays.
section_title: Recommendations for Executive Action
paragraphs: To improve NOAA’s ability to effectively manage the procurement of the  GOES-R system, we recommend that the Secretary of Commerce direct  the Undersecretary of Commerce for Oceans and Atmosphere to take the  following two actions:    Ensure that the GOES-R program office manages, mitigates, and  reports on risks using a program-level risk list that is reconciled with  and includes risks from its flight and operations project offices that  could impact the overall program.
Include the following risks on the programwide risk list, develop plans  to mitigate them, and report to senior executives on progress in  mitigating them:    unfilled or temporary GOES-R program leadership positions,  insufficient program insight on NASA contract performance, and  insufficient management reserve on the critical Advanced Baseline  Imager instrument and at the GOES-R program level.
section_title: Agency Comments and Our Evaluation
paragraphs: The Department of Commerce provided written comments on a draft of  this report (see app. II). In the department’s response, the Secretary of  Commerce agreed with our recommendations to use a program level risk  list and to add selected risks to its list. The department reported that  NOAA has established a consolidated programwide risk list that is to be  used to evaluate risks during monthly internal and external reviews.  Further, NOAA acknowledges the risks associated with unfilled leadership  positions and insufficient management reserves and is working to mitigate  these risks. However, the department disagreed with our recommendation  to manage and mitigate the risk that NOAA has insufficient insight into  NASA’s contracts. The department cited an unparalleled level of  transparency between the two agencies and listed multiple regular  meetings that the two agencies hold to ensure close coordination. While  an improved working relationship between the two agencies is critical,  NOAA has not provided any evidence that it has been able to effectively  question and validate data on NASA’s contractor performance. Given the  past problems that NOAA experienced in obtaining insight into NASA’s  contracts and the importance of this interagency relationship to the  success of the GOES-R program, we believe that this issue should be  managed and monitored as a risk.
NOAA also requested that we acknowledge its effort to reconcile its  program estimate with the independent estimate and reflect a 20 percent  possibility that the program could cost $1 billion more than the current  estimate of $7 billion, rather than $2 billion more. We acknowledge this in  our report, however, the reconciliation effort is not complete and NOAA  did not provide us with a reconciled estimate.
In addition, NOAA provided detailed comments on its cost estimates,  which are bulleted below and include our response.
NOAA asserts that the independent cost estimate of $9.3 billion at  the 80 percent level could only be realized in an extreme situation;  and that based on its own cost model, there is a 20 percent  probability this extreme situation will occur. The independent  estimator’s $9.3 billion figure is not an extreme situation. Rather, it  is at the 80 percent level, meaning that there is a 20 percent chance  that the cost could be even higher.
NOAA expressed concern with our use of preliminary independent  cost estimates and schedule projections that are more conservative  than the current independent analyses indicate. However, NOAA  did not provide us its current independent analyses. Further, these  current independent analyses were developed through NOAA’s  negotiations with the independent estimator to modify selected  assumptions. We believe that such negotiations could reduce the  independence of the estimate and, in turn, diminish its value as a  conservative view by experts outside the program office.
NOAA commented that its reconciled cost estimate is within 4  percent of the independent estimate’s mean (50 percent) value, and  that this difference is well within a standard margin of error for  cost estimates at this phase of the program. While we are not  prescribing a specific level of risk that NOAA should assume, it  should be noted that at the 50 percent confidence level, there is a  50 percent chance of being under or over budget. Throughout our  report, we have chosen to report the independent estimate at the  80 percent confidence level.
NOAA also commented that the schedule difference between the  independent estimate and the program estimate is currently 11  months and that the program estimate is considered reasonable by  the independent estimator. However, the agency compared its  projected launch date (December 2014) to the independent  estimator’s projected launch at 50 percent confidence (October  2015), instead of at its 80 percent confidence (March 2017). As  such, our report accurately depicts the results of this analysis at  the 80 percent confidence level.
In the department’s comments, NOAA also disagreed that the ABI  instrument contractor’s low level of management reserve would likely lead  to cost growth on the overall GOES-R program. In particular, it noted that  the flight project office is expected to maintain a reserve of 20 percent of  the “cost to go” (the remaining funds) of the overall space segment, which  includes ABI. NOAA stated that, as of August 2007, this reserve was at 20  percent. However, with the contractor’s management reserve at 1 percent  and 40 percent of work remaining, we continue to believe that ABI’s  situation will likely result in the early depletion of reserves for the overall  program. Furthermore, since ABI is the first major initiative in this  program and much remains to be done, it is important to ensure that  sufficient management reserve is conserved for other key components as  the program enters the development phase.
The Department also suggested technical comments which we  incorporated as appropriate.
We will be sending copies of this report to interested congressional  committees, the Secretary of Commerce, the Administrator of NASA, the  Director of the Office of Management and Budget, and other interested  parties. In addition, this report will be available at no charge on our Web  site at http://www.gao.gov.
If you have any questions on matters discussed in this report, please  contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to assess the status and revised plans for the  Geostationary Operational Environmental Satellites-R series (GOES-R)  procurement and to evaluate whether the National Oceanic and  Atmospheric Administration (NOAA) is adequately mitigating key  technical and programmatic risks facing the GOES-R program.
To assess the status and the revised plans for the GOES-R procurement,  we evaluated various programmatic and technical plans, management  reports, and other program documentation. We reviewed the acquisition  strategy, cost estimates performed by the program office and an  independent estimator, planned system requirements, and monthly  executive-level management briefings. We also analyzed earned value  management data obtained from a key instrument contractor to assess  performance against cost and schedule targets. Furthermore, we  interviewed agency officials from NOAA, the National Aeronautics and  Space Administration (NASA), and ITT Corporation to determine key  dates for future GOES-R acquisition efforts and milestones and progress  made on current instrument development efforts.
To evaluate whether NOAA is adequately mitigating key technical and  programmatic risks facing the GOES-R program, we analyzed relevant cost  reports and risk management plans to identify key risks that could  negatively affect the program’s ability to acquire the GOES-R system and  the mitigation plans for these risks. We also met with technical experts in  system acquisitions and relied on our past body of work on other major  satellite acquisitions (such as the previous GOES series and the National  Polar-orbiting Operational Environmental Satellite System programs) to  identify other key risks. We applied lessons learned and also interviewed  agency officials to evaluate the adequacy of NOAA’s mitigation efforts.
We conducted our work at NOAA and NASA offices in the Washington,  D.C., metropolitan area and at ITT Corporation (the contractor for the  most critical GOES-R instrument) in Fort Wayne, Indiana. We performed  our work from April 2007 to September 2007, in accordance with generally  accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Colleen Phillips (Assistant  Director), Carol Cha, Neil Doherty, Nancy Glover, Mustafa Hassan, Karen  Richey, and Teresa Smith made key contributions to this report.