Geostationary Weather Satellites: Progress Made, but Weaknesses in Scheduling, Contingency Planning, and Communicating with Users Need to Be Addressed
Released: 2013-09-19T00:00:00

Letter

Background

Since the 1970s, geostationary satellites have been used by the United States to provide meteorological data for weather observation, research, and forecasting. NOAA’s National Environmental Satellite, Data, and Information Service is responsible for managing the civilian operational geostationary satellite system, called GOES. Geostationary satellites can maintain a constant view of the earth from a high orbit of about 22,300 miles in space.

NOAA operates GOES as a two-satellite system that is primarily focused on the United States (see fig. 1). These satellites provide timely environmental data about the earth’s atmosphere, surface, cloud cover, and the space environment to meteorologists and their audiences. They also observe the development of hazardous weather, such as hurricanes and severe thunderstorms, and track their movement and intensity to reduce or avoid major losses of property and life. The ability of the satellites to provide broad, continuously updated coverage of atmospheric conditions over land and oceans is important to NOAA’s weather forecasting operations.

To provide continuous satellite coverage, NOAA acquires several satellites at a time as part of a series and launches new satellites every few years (see table 1). NOAA’s policy is to have two operational satellites and one backup satellite in orbit at all times.

Four GOES satellites–GOES-12, GOES-13, GOES-14, and GOES-15– are currently in orbit. Both GOES-13 and GOES-15 are operational satellites with GOES-13 covering the eastern United States and GOES- 15 in the western United States (see fig. 1). GOES-14 is currently in an on-orbit storage mode and available as a backup for the other two satellites should they experience any degradation in service. GOES-12 is at the end of its service life, but it is being used to provide limited coverage of South America. The GOES-R series is the next generation of satellites that NOAA is planning. The first two satellites in the series (called GOES-R and GOES-S) are planned for launch in October 2015 and February 2017, respectively.

Each of the operational geostationary satellites continuously transmits raw environmental data to NOAA ground stations. The data are processed at these ground stations and transmitted back to the satellite for broadcast to primary weather services and the global research community in the United States and abroad. Raw and processed data are also distributed to users via ground stations through other communication channels, such as dedicated private communication lines and the Internet. Figure 2 depicts a generic data relay pattern from a geostationary satellite to the ground stations and commercial terminals.

NOAA established the GOES-R program to develop and launch the next series of geostationary satellites and to ensure the continuity of geostationary satellite observations. Since its inception, the GOES-R program has undergone several changes in cost and scope. As originally envisioned, GOES-R was to encompass four satellites hosting a variety of advanced technology instruments and providing 81 environmental products. The first two satellites in the series were expected to launch in September 2012 and April 2014. However, in September 2006, NOAA decided to reduce the scope and technical complexity of the GOES-R program because of expectations that total costs, which were originally estimated to be $6.2 billion, could reach $11.4 billion. Specifically, NOAA reduced the minimum number of satellites from four to two, cancelled plans for developing an advanced instrument (which reduced the number of planned satellite products from 81 to 68), and divided another instrument into two separate acquisitions. The agency estimated that the revised program would cost $7 billion and kept the planned launch dates unchanged.

Subsequently, NOAA made several other important decisions about the cost and scope of the GOES-R program. In May 2007, NOAA had an independent cost estimate completed for the GOES-R program. After reconciling the program office’s cost estimate of $7 billion with the independent cost estimate of about $9 billion, the agency established a new program cost estimate of $7.67 billion. This was an increase of $670 million from the previous estimate. The program also moved the launch dates for the first two satellites to December 2014 and April 2016. Further, in November 2007, to mitigate the risk that costs would rise, program officials decided to remove selected program requirements from the baseline program and treat them as contract options that could be exercised if funds allowed. These requirements included the number of products to be distributed, the time to deliver the remaining products (product latency), and how often these products would be updated with new satellite data (refresh rate). For example, program officials eliminated the requirement to develop and distribute 34 of the 68 envisioned products, including low cloud and fog, sulfur dioxide detection, and cloud liquid water. Program officials included the restoration of the requirements for the products, latency times, and refresh rates as options in the ground system contract that could be acquired at a later time. Program officials later reduced the number of products that could be restored as a contract option (called option 2) from 34 to 31 because they determined that two products were no longer feasible and two others could be combined into a single product.

In late 2009, NOAA changed the launch dates for the first two satellites to October 2015 and February 2017, in part due to a bid protest related to award of the spacecraft contract. More recently, NOAA restored two satellites to the program’s baseline, making GOES-R a four-satellite program once again. In February 2011, as part of its fiscal year 2012 budget request, NOAA requested funding to begin development for two additional satellites in the GOES-R series—GOES-T and GOES-U. The program estimates that the development for all four satellites in the GOES-R series—GOES-R, GOES-S, GOES-T, and GOES-U—is to cost $10.9 billion through 2036, an increase of $3.2 billion over its prior life cycle cost estimate of $7.67 billion for the two-satellite program. See table 2 for an overview of key changes to the GOES-R program.

The GOES-R program is divided into flight and ground projects that have separate areas of responsibility and oversee different sets of contracts. The flight project, which is managed by NASA, includes instruments, spacecraft, launch services, satellite integration, and on-orbit satellite initialization. Table 3 summarizes the GOES-R instruments and their planned capabilities.

The ground project is directed by NOAA and is made up of three main components: the core ground system, an infrastructure of antennas, and a product access subsystem. In turn, the core ground system comprises four functional modules supporting operations, product generation, product distribution, and configuration control. Key components of the ground project are described in table 4.

NOAA is responsible for GOES-R program funding and overall mission success. The NOAA Program Management Council, which is chaired by NOAA’s Deputy Undersecretary, is the oversight body for the GOES-R program. However, since it relies on NASA’s acquisition experience and technical expertise to help ensure the success of its programs, NOAA implemented an integrated program management structure with NASA for the GOES-R program (see fig. 3). NOAA also located the program office at NASA’s Goddard Space Flight Center.

In recent years, we issued a series of reports aimed at addressing weaknesses in the GOES-R program. Key areas of focus included (1) improving communications with external data users, (2) developing contingency plans, and (3) addressing key cost and schedule risks.

Improving communications with external users. In September 2010, we reported that while NOAA had identified GOES data users and involved internal NOAA users in developing and prioritizing GOES-R requirements, it had not adequately involved other federal users who rely on GOES data by documenting their input and communicating major changes to the program. We recommended that the program establish processes for satellite data requirements definition and prioritization that include documented input from external users, as well as processes to notify these non-NOAA agencies of GOES-R program status and changes. In February 2012, the GOES-R program developed a communications plan that described how external stakeholders would be notified of GOES-R progress, status changes, and other relevant activities. However, NOAA has not yet fully implemented the plan, as demonstrated by the communication shortfalls discussed later in this report.

Developing contingency plans. In September 2010, we reported that while there was a potential gap in backup coverage due to satellite launch delays, NOAA had not established adequate continuity plans for its geostationary satellites. We recommended that the program’s plan include implementation procedures, resources, staff roles, and timetables needed to transition to a single satellite, a foreign satellite, or other solution. In December 2012, NOAA finalized a contingency plan that generally included these elements. However, more work remains to ensure that the plan is viable.

More recently, in February 2013, we added the potential gaps in weather satellite data to our biennial High-Risk list. In that report, we noted that NOAA had established a contingency plan for a potential gap in the GOES program, but it needed to demonstrate its progress in coordinating with the user community to determine their most critical requirements, conducting training and simulations for contingency operations scenarios, evaluating the status of viable foreign satellites, and working with the user community to account for differences in product coverage under contingency operations scenarios. We also stated that NOAA should update its contingency plan to provide more details on its contingency scenarios, associated time frames, and any preventative actions it is taking to minimize the possibility of a gap.

Addressing key cost and schedule risks. In June 2012, we reported that the GOES-R program might not be able to ensure that it had adequate resources to cover unexpected problems in remaining development, and that unresolved schedule deficiencies existed in its integrated master schedule and contractor schedules. We also reported that the program estimated a 48 percent chance that the planned GOES-R launch date of October 2015 would be reached. We recommended that the program assess and report on the reserves needed for completing remaining development for each satellite in the series, and address shortfalls in the schedule management practices we identified such as eliminating unnecessary constraints and creating a realistic allocation of resources, in order to minimize the likelihood of a potential gap. The agency agreed with these recommendations and took steps to address them by identifying needed reserve levels and refining program schedules.

NOAA Has Made Progress in Developing GOES-R, but Continues to Face Challenges that Could Increase the Risk of a Satellite Data Gap

NOAA has completed its design of the GOES-R program, and has made progress in building components of the flight and ground segments. Program officials also report that the program is operating within its estimated budget of $10.9 billion. However, key information on reserves has not been reported to management. Further, both the flight and ground segments have experienced delays in achieving major milestones due to technical challenges, and weaknesses in the development of master schedules could cause further delays. Program officials stated that they have made improvements on how they manage cost reserves and schedules, but acknowledged that there will always be opportunities for improvement because the reserves and schedules are so dynamic on a big program like GOES-R. These challenges have the potential to impact the expected launch date of the first GOES-R satellite, which would delay the availability of an on-orbit backup and increase the potential for a gap in GOES satellite coverage should either of the two operational satellites fail prematurely.

Program Has Completed Design and Begun Building Components of the First Satellite

NASA and NOAA are following NASA’s standard space system life cycle on the GOES-R program. This life cycle includes distinct phases, including concept and technology development; preliminary design and technology completion; final design and fabrication; system assembly, integration and testing, launch and checkout; and operations and sustainment. There are key program reviews throughout each of the phases, including preliminary design review, critical design review, and system integration review. NOAA and NASA jointly conduct key reviews on the flight and ground segments individually as well as for the program as a whole, and then make decisions on whether to proceed to the next phase. Figure 4 provides an overview of the life cycle phases, key program reviews, and associated decision milestones. In addition, the key reviews are described in table 5.

The GOES-R program has completed final design and begun building components of the flight and ground systems. Specifically, the program completed critical design reviews for the flight and ground projects and for the overall program between April and November 2012. In its evaluation of the program as part of the critical design review, an independent review board complimented the program on several recent achievements, stating that the program was beyond the level of maturity expected at that phase, and that the program’s planning was a major factor in the launch date of the first satellite remaining October 2015.

As the spacecraft and instruments are developed, NASA conducts several interim reviews and tests before proceeding to the next major program-level review, the system integration review. These include a pre- environmental review, which represents the conclusion of an initial round of testing before exposing the instrument to testing under adverse environmental conditions; environmental testing of key functions under adverse conditions; and a pre-shipment review, which is conducted on each instrument to ensure it is ready to be shipped for integration and testing on the spacecraft.

The GOES-R flight components are in various stages leading up to the system integration review. Of the six GOES-R instruments, one has completed environmental testing and its pre-shipment review; four instruments are in the midst of these reviews and tests; and one instrument has not yet passed its pre-environmental review. In addition, the program began building the spacecraft in February 2013. Table 6 provides more information on progress made on the key flight project components.

Similar to the flight project, major ground system milestones are focused on building and testing components and the program has made progress in this area. Specifically, on the core ground system, a prototype for the operations module was delivered in late 2012 and used for initial testing and training. In July 2013, the ground project delivered the iteration of the operations module that will be used to support the first satellite. In addition, the program has installed antenna dishes at NOAA’s primary satellite communications site, and completed two key reviews of antennas at the GOES remote backup site. The Product Distribution and Access System recently completed a review that will allow testing to begin on its first release. An integration review for ground components is also expected to take place in January 2014. More detail on the progress of the ground project can be seen in table 7.

The program’s next major milestone is a programwide system integration review, which is scheduled for March 2014. Based on the results of that review, NOAA and NASA will decide whether to move the program to the next phase: the system assembly, integration and test, and launch and checkout phase.

Contingency Reserves Are Generally in Line with Goals for Overall Program Development; Reporting on Reserve Values Remains Limited

The GOES-R program is estimated to cost $10.9 billion. As of February 2013, the program estimated that this amount was divided into four categories, with $6.0 billion for the flight project, $1.7 billion for the ground project, $2.0 billion for other program costs (including, among other things, program/project support and formulation) and $1.2 billion for operations and support. Program officials reported that the program is currently operating without cost overruns on any of its main components, but noted that the program life cycle costs may increase by $150 to $300 million if full funding in the current fiscal year is not received.

A portion of the amounts planned for the flight project and ground project are allocated to contingency reserves (also called management reserve). The program also keeps a programwide contingency allocation separate from those of the flight and ground projects. A contingency reserve provides program managers ready access to funding in order to resolve problems as they occur and may be necessary to cover increased costs resulting from unexpected design complexity, incomplete requirements, or other uncertainties. NASA’s Goddard Space Flight Center requires its flight projects, including GOES-R, to maintain contingency reserves during system development, at a level of 25 percent of development costs. The GOES-R program requires its flight and ground projects to maintain 20 percent of planned remaining development costs as reserve funding. The program office also maintains contingency reserves equal to 10 percent of planned remaining development costs to cover program support costs and to supplement the flight and ground projects’ reserves if necessary. According to a NOAA program official, the GOES-R program is able to meet NASA’s requirement through the combination of the 20 percent flight and ground project requirements and the supplemental 10 percent program-level reserve. An official also stated that the method of keeping separate reserves at the program and project levels was chosen as it was successful on past projects.

The GOES-R flight project, ground project, and program office are at or above the amount of reserves they are required to carry. Specifically, as of March 2013, the overall contingency reserve percentages for the flight and ground projects were at 20 and 28 percent, respectively, which are at or above the required level of 20 percent. The program reserves were at 11 percent, slightly above the required level of 10 percent. Reserve values and percentages are provided in table 8.

We previously reported that, in order to oversee GOES-R contingency funding, senior managers should have insight into the amount of reserves set aside for each satellite in the program and detailed information on how reserves are being used on both the flight and ground components. While the GOES-R program continues to regularly report on contingency funds, it does not report key information on the status of its reserve funding to senior level management.

At monthly program management council meetings, the program reports summary information, such as the total value of contingency reserves and reserve percentage held for each fiscal year. Reserve totals are given for the flight and ground projects as well as for the program overall. However, the program does not report on the reserves needed for completing remaining development for each satellite in the series or provide detailed information on how reserves are being used. Thus, for example, if the later satellites in the series have a high level of reserves, they could mask low reserve values for earlier satellites in the series. Further, in its monthly presentations to senior managers and NOAA executives, the program does not include information on the cause of any changes in reserve values from the prior month or the assumptions it makes when calculating reserves. For example, the flight reserve value recently went up by 2 percentage points because the program decided to include reserve funding for the GOES-T satellite in 2018, and the ground reserve values went down by 10 percentage points because the program shifted reserve funding from the ground to the flight projects. Neither of these changes was identified or explained in the monthly presentations. The lack of insight on how the reserves are calculated and modified could lead executives to misinterpret the size of the remaining reserves. Program officials noted that they took steps after our previous report to clarify what they report about reserves, but noted that the amount of information needed to fully explain reserve calculations and changes could be too much information for an executive-level briefing. Without regularly providing sufficiently detailed budget information, it may be more difficult for program management to have the information they need to make the best decisions possible regarding the program’s future funding.

Recent and Potential Milestone Delays and Continued Weaknesses in Scheduling Practices Increase the Potential for a Delayed Launch

The GOES-R program established programwide milestones, including the mission operations review and flight operations review, to determine the program’s ability to proceed to system integration and to complete mission operations, respectively. It also established five end-to-end system tests to validate compatibility between the space and ground segments before the launch of the first satellite.

However, over the past year, the program delayed many of these key milestones and tests. Delays in the mission operations review means that the large-scale integration of flight and ground components will not occur until 21 months prior to launch. Similarly, delaying end-to-end tests until 17 months prior to launch will allow the program less time to respond to any problems that occur. Table 9 highlights key milestones and the extent of recent delays.

The GOES-R program is also experiencing technical issues on the flight and ground projects that could cause further schedule delays.

The original supplier for a key component on the spacecraft moved to a different facility, introducing risk due to the loss of experienced personnel and the impact on schedule. This led the program to find an alternative supplier. While a design review was performed to confirm resolution of the issue in April 2013, this change may lead to a delay of up to 6 months in integrating the component on the spacecraft. Program officials noted that this delay is not expected to impact the program’s critical path or major milestones.

The Geostationary Lightning Mapper’s electronics unit experienced problems during testing, which led the program office to delay the tests. The program is considering several options to address this issue, including using the electronics unit being developed for a later GOES satellite to allow key components to proceed with testing. If the issue cannot be resolved, it would affect the instrument’s performance. As a result, the program is also considering excluding the Geostationary Lightning Mapper from the first GOES satellite. The program plans to make its decision on whether or not to include the instrument in late 2013. The removal of this instrument would cause a significant reduction in the satellite’s functionality. Key GOES users have stated that they would prefer that NOAA delay launching the GOES-R satellite rather than launch it without the Geostationary Lightning Mapper.

The program delayed the start of work on the ground system at the NOAA satellite operations facility by three months, from a planned date of October 2012 to January 2013, following a bid protest of the award of a contract to upgrade the facility. This delay compressed an already tight schedule for testing the ground system.

Testing for a number of ground system requirements has been postponed until future releases and builds, potentially causing modification to the schedule for these future products.

Power amplifiers for the antenna systems experienced higher than expected failure rates, which could lead to schedule delays and decreases in operational availability.

Given that fewer than 3 years remain before GOES-R’s expected launch in October 2015, continued delays in key milestones and reviews decrease the likelihood that the launch date will be met. Program officials recently acknowledged that the GOES-R launch date may be delayed by about 6 months, and attributed the cause of the delay to a shortfall of $54 million in anticipated funding in fiscal year 2013.

Delays in the program’s remaining schedule are also at risk of further growth due to weaknesses in the program’s scheduling methods. Program schedules not only provide a road map for systematic program execution, but also provide the means by which to gauge progress, identify and address potential problems, and promote accountability. Achieving success in managing large-scale programs depends in part on having an integrated and reliable schedule that defines, among other things, when work activities and milestone events will occur, how long they will take, and how they are related to one another. Without such a reliable schedule, program milestones may slip.

In June 2012, we reported on weaknesses in program schedules that comprised portions of the program’s Integrated Master Schedule, including subordinate schedules for the spacecraft and core ground system. At that time, our work identified nine best practices associated with developing and maintaining a reliable schedule. These are (1) capturing all activities, (2) sequencing all activities, (3) assigning resources to all activities, (4) establishing the duration of all activities, (5) integrating schedule activities horizontally and vertically, (6) establishing the critical path for all activities, (7) identifying reasonable float time between activities, (8) conducting a schedule risk analysis, and (9) updating the schedule using logic and durations. See table 10 for a description of each of these best practices.

In a previous report, we observed that important schedule components in GOES-R related schedules had not been included or completed, and recommended that these shortfalls be addressed. NOAA has since improved selected practices on its spacecraft and core ground schedules, but other practices stayed the same or worsened. Specifically, for the spacecraft, 2 practices were improved, 5 stayed the same, and 2 became weaker. For the core ground system, 4 practices were improved, 3 stayed the same, and 2 became weaker. Table 11 compares our assessments of the spacecraft and core ground system schedules in July 2011 and November 2012.

NOAA has improved elements of the schedules for both components. Specifically, the spacecraft schedule has eliminated level of effort activities and has assigned resources for a greater percentage of activities. The core ground schedule now has an automated process by which all subcontractor records are combined to create an integrated schedule. It has a series of connected activities that lead to what contractor officials consider its main milestone delivery, and has implemented a detailed schedule risk analysis for a key upcoming release.

However, scheduling issues remain on the schedules for both components. For example, both schedules have issues with sequencing remaining activities and integration between activities. Regarding the spacecraft schedule, there is a small subset of activities with incomplete links between activities, and more than 20 percent of remaining detail activities have lags, or a set number of days between an activity and its successor. In the core ground schedule, a number of activities are missing either predecessor or successor activities, and there are several activities representing the end of the project on or about the same date. Without the right linkages, activities that slip early in the schedule do not transmit delays to activities that should depend on them. When this happens, the schedule will not provide a sufficient basis for understanding the program as a whole, and users of the schedule will lack confidence in the dates and the critical path.

Both schedules also have a very high average of total float time for detailed activities. Specifically, total float time is greater than two months for nearly two-thirds of remaining detailed activities in the spacecraft schedule, and at least a year for more than 10 percent of remaining detail activities in the core ground schedule. In the case of spacecraft, officials stated that high levels of float time were often due to activities that had been completed at one time for several satellites, only one of which was immediately needed. Officials also provided detailed information on the activities with the highest amount of float. In the case of the core ground schedule, officials stated that many activities occurring after the main milestone date, which occurs nearly five years prior to the end of the schedule, do not have a true successor, and therefore are calculated only to the end of the contract. Officials also stated that values and trends in float time are monitored regularly for both schedules. Such high values of total float time can falsely depict true project status, making it difficult to determine which activities drive key milestone dates. Without reasonable values of total float time, it cannot be used to identify activities that could be permitted to slip and thus release and reallocate resources to activities that require more resources to be completed on time.

In addition, the project’s critical path does not match up with activities that make up the driving path on the core ground schedule. Contractors monitor a driving path monthly to both major and minor milestone deliveries. However, until the schedule can produce a true critical path, it will be more difficult for the program office to provide reliable time line estimates or identify when problems or changes may occur and their effect on downstream work. Also, without a valid critical path to the end of the schedule, management cannot focus on activities that will have a detrimental effect on the key project milestones and deliveries if they slip.

Further, neither schedule file has fully integrated resources with schedule activities. As of November 2012, contractor officials stated that the ground system schedule was not feasible given available resources and that they were in the process of revising their immediate schedules to make them feasible. The spacecraft schedule contains major resource categories that correspond to contractor sites and work phases. However, thresholds for overruns of resource allocations are functionally disabled within the schedules through the setting of an arbitrarily high value for maximum resources per category. In response, contractor officials stated that account managers are responsible for monitoring resource levels and that weekly meetings are held to ensure that resource issues are discussed. Information on resource needs and availability in each work period assists the program office in forecasting the likelihood that activities will be completed as scheduled. If the current schedule does not allow insight into current or project allocation of resources, then the risk of delays in the program’s schedule is significantly increased.

Deficiencies in scheduling practices such as the ones outlined here could increase the likelihood of launch date delays, because decision making would be based on data that does not accurately depict current status, thus impeding management’s ability to conduct meaningful oversight on the program’s schedules. Program officials noted that they have made improvements in scheduling practices, but explained that because the schedules are so dynamic there are always areas for improvement. Lack of the proper understanding of current program status due to schedules that are not fully reliable undercuts the ability of the program office to manage a high-risk program like GOES-R.

Delays in the GOES-R Launch Date Could Increase the Risk of a Satellite Data Gap

Potential delays in the launch date of the first GOES-R satellite would increase the risk of a gap in GOES satellite coverage. NOAA’s policy is to have two operational satellites and one backup satellite in orbit at all times. This policy proved useful in December 2008 and again in September 2012, when the agency experienced problems with one of its operational satellites, but was able to move its backup satellite into place until the problems had been resolved.

NOAA is facing a period of at least a year when it will not have a backup satellite in orbit. Specifically, in April 2015, NOAA expects to retire one of its operational satellites (GOES-13) and move its backup satellite (GOES- 14) into operation. Thus, the agency will have only two operational satellites in orbit—and no backup satellite—until GOES-R is launched and completes an estimated 6-month post-launch test period. If GOES-R is launched in October 2015, the soonest it could be available for operational use would be April 2016. Any delay to the GOES-R launch would extend the time without a backup to more than one year. Figure 5 shows anticipated operational and test periods for the two most recent series of GOES satellites.

In addition to the year or more during which no back-up satellite would be available, there is a chance that NOAA would have to operate with a single operational satellite. In December 2012, an independent review board estimated that there is a 36 percent chance that the GOES constellation would have only one operational satellite at the expected date of GOES-R’s launch. Thus, if NOAA were to experience a problem with either of its operational satellites before GOES-R is in orbit and operational, it would need to rely on older satellites that are beyond their expected operational lives and may not be fully functional. Without a full complement of operational GOES satellites, the nation’s ability to maintain the continuity of data required for effective weather forecasting could be compromised. This, in turn, could put the public, property, and the economy at risk.

NOAA Has a Process for Managing Changes in GOES-R Requirements, but Changes Could Affect Some Users

System requirements describe the functionality needed to meet user needs and perform as intended in an operational environment. According to leading industry, academic, and government entities, a disciplined process for developing and managing requirements can help reduce the risks of developing or acquiring a system. One key aspect of effective requirements management involves managing changes to requirements through a standardized process. Table 12 outlines best practices of a sound change management process and key questions for evaluating the process.

The GOES-R program has a change management process that satisfied three practices, partially satisfied two practices, and did not satisfy one practice. Specifically, GOES-R has established a change management process that tracks and documents changes in requirements, documents the rationale for the changes as well as the potential impact of the change on cost and schedule, and ensures that changes are reviewed and approved by a change control board. In addition, the program has evaluated the impact of key changes on selected users and communicated with those users. However, as we first reported in 2010, the program is still weak in evaluating the impact of changes on external users who rely on GOES data products and in effectively communicating changes to those satellite data users. Specifically, table 13 outlines how the GOES-R program performed on each of the best practices for managing changes in requirements, and is followed by a more detailed discussion of key shortfalls.

While the program generally communicates requirements changes to key users within NOAA’s National Weather Service community, it does not communicate as well with satellite data users external to NOAA. Many such users are dependent on GOES satellite data for their respective missions. Officials responsible for working with satellite weather observations at three agencies were unaware of selected changes in GOES-R requirements. For example, the Federal Aviation Administration uses the satellites’ data and images to manage air traffic across the country, and the Navy uses the data for oceanic weather forecasting, as well as tactical ocean analysis of regions of interest. They stated that NOAA had not reached out to them to alert them to these changes or ask if the changes would impact them. Similarly, Forest Service officials were concerned that potential changes in spectrum allocations could affect their ability to obtain data from their own ground-based weather observation systems because they currently rely on GOES-R communication channels to obtain this data.

GOES-R program officials noted that they provide regular briefings to the Office of the Federal Coordinator for Meteorology, an interagency council with membership from fifteen federal departments and agencies involved in meteorological activities (including the Departments of Agriculture, Defense, and Transportation) and that the Air Force represents the Department of Defense community on the GOES-R Series Independent Advisory Committee. However, they acknowledged that they cannot ensure that the information they provide is disseminated within the agencies. Further, GOES officials explained that one reason for the distinction between the internal and external users is that the internal users belong to formal working groups and receive regular updates from the GOES-R program, while the other users generally have more informal or indirect connections with the program. Instead of direct communications such as e-mails, the other users may receive information about GOES-R requirements changes from publically available information or through other meteorological partnerships with NOAA. Without consistent and direct communication, users may be unaware of changes to program requirements that are critical to their respective missions. Because GOES-R users across the country have missions that preserve and protect property and life, it is critical that these organizations are made aware of any changes as soon as they are made, so that they can assess and mitigate any significant impacts.

Since 2007, NOAA has changed multiple system requirements on the GOES-R program. These changes involved strengthening or relaxing specifications on selected products, finalizing a decision not to develop 31 products, and modifying programmatic requirements not tied to any individual product. For example, NOAA strengthened specifications for the geographic coverage, image resolution, and refresh rate on a product depicting total precipitable water, and strengthened accuracy specifications for a product depicting cloud layers and heights. NOAA also relaxed specifications to provide less measurement accuracy on a product depicting hurricane intensity, less geographic coverage on a product depicting sea surface temperatures, less resolution on a product that tracks the motion of clouds through the atmosphere, and less timely updates on a product depicting lightning detection. The GOES-R program also documented NOAA’s earlier decision not to develop or provide 31 products that it labeled as optional, noting that the products will only be developed if funding becomes available. In addition, programmatic changes include the elimination of 97 percent mission availability as a measure of minimum performance success and the decision not to transmit raw satellite data to users. Table 14 provides an overview of key changes in product and program requirements since 2007.

While NOAA officials stated that they believe that only one of the changes that were made since 2007 was significant, internal and external users noted that they found many of the changes to be significant. In addition, selected satellite data users noted concern at the loss of 17 of the optional products that are no longer being developed. The changes that users found significant, along with user reasoning for why these changes are significant, are listed in table 15. GOES-R program officials acknowledged that the National Weather Service and other users will have impacts from the loss or degradation of products, but that it is not always accurate to assume that GOES-R could have met the original requirements. In 2011, an algorithm development executive board reported that several original requirements could not have been met because, among other reasons, they relied on a hyperspectral instrument that was removed from the program, the requirements were poorly stated and it only became evident later that GOES could not support them, and there were scientific limitations on the development of the products that only became evident after development had started. Program officials stated that they have identified alternative methods for obtaining certain products (some outside the scope of GOES-R) and that they are proactively trying to develop alternative products in coordination with users and other development organizations.

In addition to the changes that have already been implemented on the GOES-R program, there are other potential changes that could occur. For example, by the end of 2013, the program plans to decide whether or not to include the Geostationary Lightning Mapper on the GOES-R satellite. Also, there could be changes in the spectrum allocated to weather satellite data. Officials from the National Weather Service and Forest Service raised concerns that these potential changes could also affect their operations. Because these changes have the potential to impact satellite data user operations, it is critical that the GOES-R program communicates program changes to the extended user community. By doing so, satellite data users can establish plans to mitigate any shortfalls in data and minimize the impact of the changes on their operations.

NOAA Developed GOES-R Contingency Plans, but Weaknesses Increase the Impact of a Potential Coverage Gap

GOES satellite data are considered a mission-essential function because of their criticality to weather observations and forecasts. These forecasts—such as those for severe storms, hurricanes, and tornadoes— can have a substantial impact on our nation’s people, infrastructure, and economy. Consequently, NOAA policy requires that there must be two in- orbit GOES satellites and one on-orbit spare in operation at all times. If one of the operational satellites were to fail, the on-orbit spare could be moved into position to take the place of the failed satellite. However, if there are delays in the launch of the GOES-R satellite or if either of the two satellites currently in operation were to fail, NOAA would not have an on-orbit spare to fill the gap.

Government and industry best practices call for the development of contingency plans to maintain an organization’s essential functions in the case of an adverse event. These practices include key elements such as defining failure scenarios, identifying and selecting strategies to address failure scenarios, developing procedures to implement the selected strategies, identifying any actions needed to implement the strategies, testing the plans, and involving affected stakeholders. These elements can be grouped into categories, including (1) identifying failure scenarios and impacts, (2) developing contingency plans, and (3) validating and implementing contingency plans (see table 16).

NOAA has established contingency plans for both its GOES satellites and its associated ground systems. In September 2010, we recommended that NOAA develop and document continuity plans for the operation of geostationary satellites that include the implementation procedures, resources, staff roles, and timetables needed to transition to a single satellite, a foreign satellite, or other solution. In September 2011, the GOES-R program provided a draft plan documenting a strategy for conducting operations if there were only a single operational satellite. In December 2012, the program provided us with a final version of this plan. It included scenarios for three, two, and one operational satellites. In addition to this satellite contingency plan, NOAA has another contingency-related plan with activation procedures for its satellites.

Furthermore, the NOAA office responsible for ground-based satellite operations and products has created plans for contingency operations at the GOES ground system facility, the Satellite Operations Control Center. Specifically, NOAA’s plans describe the transfer of critical functions to a backup facility during an emergency. The continuity plan contains, among other things, descriptions of the alternate locations for resources, and the performance of key functions and implementation procedures.

When compared to best practices, NOAA’s satellite and ground system contingency plans had many strengths and a few weaknesses. Specifically, the satellite contingency plan fully implemented seven elements, partially implemented nine elements, and did not implement one element. The ground system contingency plan fully implemented ten elements, partially implemented six elements, and one element was not applicable. Table 17 shows the extent to which the satellite and ground system contingency plans fully implemented, partially implemented, or did not implement key contingency planning elements.

NOAA has implemented most of the best practices on both the GOES satellite and ground contingency plans. Specifically, NOAA identified failure scenarios, recovery priorities, and minimum levels of acceptable performance. NOAA also established contingency plans that identify solutions and high-level activities and triggers to implement the solutions. Further, the agency has tested its contingency plans, trained staff on how to implement the contingency plans, and updated the plans when warranted. The agency also successfully implemented its contingency plans when it experienced problems with one of its operational satellites. Specifically, when GOES-13 experienced problems in September and October 2012, NOAA activated its contingency plans to move its back-up satellite into position to provide observations until GOES-13 was once again operational. While the agency has not needed to address the loss of a back-up satellite in recent years, contingency plans cover this situation by determining if older GOES satellites could provide coverage, moving the single satellite into a central position over the country, and seeking data from foreign satellites.

However, both satellite and ground contingency plans contain areas that fall short of best practices. For example, NOAA has not demonstrated that the contingency strategies for both its satellite and ground system are based on an assessment of costs, benefits, and impact on users. Further, the satellite plan does not specify procedures for working with the user community to account for potential reductions in capability under contingency operations. For example, officials from the Federal Aviation Administration noted that NOAA’s contingency plans do not define the compatibility, security, and standard protocol language they should use if a foreign satellite were to be utilized. Also, while selected users reported that, in the past, they have been well informed by NOAA when changes in service occur, including the problems with GOES-13, others were either not informed or received information on outages through a third party. Moreover, selected users stated that certain contingency operations could have a significant impact on their operations. For example, Federal Aviation Administration officials stated that flight approaches in Alaska that were enabled using the Global Positioning System were affected by the GOES-13 outage in late 2012. As another example, Forest Service officials explained that if GOES were to experience an outage and not have a backup satellite available, it was their understanding that NOAA would either move a single satellite into a central position over the country or obtain observations from a foreign satellite. Under both of these scenarios, they could lose views of wildland fires and their ability to obtain data from ground-based observation networks. Nearly all users stated that the effects of a switch to a single satellite or foreign satellite configuration would be significant.

In addition, while NOAA’s failure scenarios for its satellite system are based on the number of available satellites—and the loss of a backup satellite caused by a delayed GOES-R launch would fit into these scenarios—the agency did not identify alternative solutions or time lines for preventing a GOES-R launch delay. According to NOAA officials, a gap caused by a delayed launch would trigger the same contingency actions as a failure on launch or the loss of a currently on-orbit satellite. However, this does not take into account potential actions that NOAA could undertake to prevent a delayed launch, such as removing selected functionality or compressing test schedules.

NOAA officials stated that their focus on primary users and on the number of available satellites is appropriate for their contingency plans. Given the potential for a delay in the launch of the GOES-R satellite and the expectation that there will be at least a year with no backup satellite in orbit, it is important that NOAA consider ways to prevent a delay in the GOES-R launch, and ensure its contingency plans are fully documented, tested, and communicated to affected stakeholders. Further, it is critical that NOAA and users are aware of how contingency scenarios will affect user operations. Until comprehensive plans are developed, it is less certain that NOAA can provide a consistent level of service and capabilities in the event of an early failure or late launch. This in turn could have a devastating effect on the ability of meteorologists to observe and report on severe weather conditions.

Conclusions

The GOES-R program is well on its way toward developing the first satellite in the series, but it continues to face risks that could delay the first satellite’s launch. Among these risks are issues we have previously raised on how the program manages reserve funds and implements sound scheduling practices. Specifically, the agency does not provide important details on its contingency reserve funds to senior executives, including the reserves allocated for each of the four satellites or key assumptions made in calculating reserves. Without this information, program officials could misinterpret the size of the remaining reserves and make poor decisions regarding the program’s future funding. The agency has improved selected scheduling practices, but others remain weak—in part, according to agency officials, due to the dynamic nature of scheduling a program as complex as the GOES-R satellite program. As the agency closes in on its expected launch date, technical issues in developing the space and ground segments and scheduling problems could make it more difficult to launch on schedule, and program officials now acknowledge that the launch date may be delayed by 6 months. Any delay in the anticipated launch date would expand a potential one-year gap in the availability of an on-orbit backup GOES satellite, and raise the risk of a gap in geostationary satellite data should one of the two operational satellites experience a problem.

While the agency has made multiple changes to GOES-R requirements in recent years, it has not effectively involved satellite data users in those changes. Specifically, internal NOAA and external satellite data users were not fully informed about changes in GOES-R requirements and did not have a chance to communicate their concerns about the impact these changes could have on their ability to perform their missions. Many of these users expressed concerns about the effect these changes could have on their ability to fulfill their missions, including facilitating air traffic, conducting military operations, and fighting wildland fires. Until NOAA improves its outreach and communication with external satellite data users, its changes in requirements could cause unexpected impacts on critical user operations.

Given the possibility of a gap in geostationary satellite coverage, NOAA has established contingency plans for both its GOES satellites and ground systems; these plans include the likely scenario in which there will not be an on-orbit backup. While these plans include many elements called for in industry best practices, the satellite contingency plan did not assess the potential impacts of a failure on users, or specify actions for working with the user community to address these potential reductions in capability under contingency operations. They also did not identify alternative solutions or time lines for preventing a delay in the GOES-R launch date. The absence of a fully-tested and complete set of GOES-R- related contingency plans and procedures could have a major impact on levels of service provided in the event of a satellite or ground system failure.

Recommendations for Executive Action

To address risks in the GOES-R program development and to help ensure that the satellite is launched on time, we are making the following four recommendations to the Secretary of Commerce. Specifically, we recommend that the Secretary of Commerce direct the NOAA Administrator to:

Direct program officials to include information on the amount of reserve funding for each of the four satellites in the program as well as information on the calculation and use of reserves in regular briefings to NOAA senior executives, so that executives are fully informed about changes in reserve levels.

Given the likely gap in availability of an on-orbit GOES backup satellite in 2015 and 2016, address the weaknesses identified in this report on the core ground system and the spacecraft schedules. These weaknesses include, but are not limited to, sequencing all activities, ensuring there are adequate resources for the activities, and conducting a schedule risk analysis.

Improve communications with internal and external satellite data users on changes in GOES-R requirements by (a) assessing the impact of changes on user’s critical operations; (b) seeking information from users on any concerns they might have about past or potential changes; and (c) disseminating information on past and potential changes in requirements to satellite data users.

Revise the satellite and ground system contingency plans to address weaknesses identified in this report, including providing more information on the potential impact of a satellite failure, identifying alternative solutions for preventing a delay in GOES-R launch as well as time lines for implementing those solutions, and coordinating with key external stakeholders on contingency strategies.

Agency Comments and Our Evaluation

We sought comments on a draft of our report from the Department of Commerce and NASA. We received written comments on a draft of this report from Commerce transmitting NOAA’s comments. NOAA concurred with all four of our recommendations and identified steps that it is taking to implement them. It also provided technical comments, which we have incorporated into our report, as appropriate. NOAA’s comments are reprinted in appendix II.

While NOAA concurred with our recommendation to include information on reserve funding for each of the four satellites in the program and information on the calculation and use of reserves in regular briefings to senior executives, and suggested that its current processes fulfill this recommendation, we do not believe they do. Specifically, NOAA stated that the GOES-R program currently reports on reserve funding at two major monthly management meetings, which alerts management if reserves fall below designated thresholds for the remaining work on all four satellites. The agency also stated that its reporting of the percent of “unliened” contingency funding—the amount of contingency funding not allocated to a potential risk or issue—for the remaining work addresses our concern regarding whether there are sufficient reserves to complete the GOES-R series.

However, the GOES-R program does not currently identify the reserve funding needed for each individual satellite or provide details on how reserves are being calculated and used at the monthly management meetings. By not providing reserve information on the individual satellites, the program is not alerting management about potential near-term funding shortfalls. For example, maintaining a high level of reserves on the later satellites could mask a low level of reserves in the near-term for GOES-R and S. Such a scenario could affect the satellites’ development schedules and launch dates. Further, by not obtaining details on the assumptions made when calculating reserves and the causes of changes in reserve values, management is unable to determine if changes in reserve levels are due to the addition, subtraction, or use of funds, or to changes in the assumptions used in the calculations. Given the importance of reserve funds in ensuring the satellite development remains on track, management should be aware of reserve funding levels for each individual satellite and of the underlying reasons for changes in reserve levels. Therefore, we continue to believe that additional action is needed by NOAA to respond to our recommendation.

After we received agency comments and while our report was in final processing, NOAA notified us that the launch dates of the first and second GOES-R series satellites would be delayed. Given the late receipt of this information, our report reflects the previous launch date.

NASA did not provide comments on the report’s findings or recommendations, but noted that it would provide any input it might have to NOAA for inclusion in that agency’s comments.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to interested congressional committees, the Secretary of Commerce, the Administrator of NASA, the Director of the Office of Management and Budget, and other interested parties. The report also will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) assess the National Oceanic and Atmospheric Administration’s (NOAA) progress in developing the Geostationary Operational Environmental Satellite-R series (GOES-R) program and in addressing key cost and schedule risks that we identified in a prior report, (2) evaluate the program’s efforts to manage changes in requirements and whether any significant changes have recently occurred, and (3) evaluate the adequacy of GOES-R contingency plans.

To assess NOAA’s progress in developing the GOES-R satellite program, we compared the program’s planned completion dates for key milestones identified in its management control plan and system review plan against actual and currently estimated completion dates. We analyzed monthly program status briefings to identify the current status and recent development challenges of flight and ground project components and instruments. To assess NOAA’s efforts to address key cost risks, we compared program-reported data on development costs and reserves to best practices in reserve funding as identified by the program’s management control plan, which, in turn, reflects National Aeronautics and Space Administration requirements. We calculated reserve percentages using program office data on development costs and reserves, and compared these calculations to the reserve percentages reported by the program to management. To assess NOAA’s efforts to address key schedule risks, we compared schedules for two key GOES-R components to best practices in schedule development as identified in our Cost Estimating and Assessment Guide. Similar to our previous report, we used a five-part rating system. We then compared our previous assessment to our current assessment to identify practices that were improved, stayed the same, or became weaker over time. We conducted interviews with GOES-R program staff to better understand milestone time frames, to discuss current status and recent development challenges for work currently being performed on GOES-R, and to understand how the program reports costs and reserve totals. We also examined the reliability of data on cost reserves and program schedules. Regarding cost reserves, we examined reliability by recalculating reserve percentages based on supporting data over a period of one year, and compared the results to those presented by the program to management. Regarding schedules, we created a template that examined each schedule in areas such as missing logic, tasks completed out of sequence, and completed tasks with start or finish dates in the future. As a result, we found both the reserve information and the schedules to be reliable for the purposes of conducting our analyses.

To evaluate the program’s efforts to manage changes in requirements, we compared GOES-R practices for managing requirements changes against best practices, which we drew from several leading industry sources including the Software Engineering Institute’s Capability Maturity Model®–Integration, the Program Management Institute’s Program Manager’s Body of Knowledge, the Federal Information Security Controls Audit Manual and the Information Technology Governance Institute’s Control Objectives for Information and related Technology governance framework. We assessed GOES-R practices as having satisfied, partially satisfied, or not satisfied each best practice. We analyzed changes from 2007 to the present in the program’s Level I Requirements Document to determine the extent of the changes. We also identified concerns about these changes from a subset of satellite data users. We selected users from both inside and outside NOAA’s National Weather Service, the main GOES satellite user, based on several factors: the importance of GOES data to the organization’s core mission, the user’s reliance on GOES products that have changed or may change, and—for agencies outside of NOAA—the percentage of spending devoted to meteorological operations. The user organizations outside of NOAA included in our review were: the US Department of Agriculture, the Department of Transportation’s Federal Aviation Administration, and the Department of Defense’s Navy and Air Force. User organizations inside of NOAA’s National Weather Service included the Aviation Weather Center, Space Weather Prediction Center, Storm Prediction Center, Environmental Modeling Center, and a Weather Forecast Office.

To evaluate the adequacy of GOES-R contingency plans, we compared contingency plans and procedures for both GOES satellites and the GOES ground system against best practices developed from leading industry sources such as the National Institute of Standards and Technology, the Software Engineering Institute’s Capability Maturity Model®–Integration, and our prior work. We analyzed the contingency plans to identify strategies for various failure scenarios and determined whether the satellite and ground system contingency plans fully implemented, partially implemented, or did not implement each of the practices. We also interviewed selected satellite data users to better determine the impact of a GOES failure scenario on their operations, and the level of communication they have had with NOAA satellite offices on current contingency plans.

We performed our work at NOAA, National Aeronautics and Space Administration, and US Department of Agriculture offices in the Washington, D.C., area and at National Weather Service offices in Kansas City, Missouri; Norman, Oklahoma; and Sterling, Virginia. We conducted this performance audit from October 2012 to September 2013, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Commerce

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, individuals making contributions to this report included Colleen Phillips (assistant director), Paula Moore (assistant director), Shaun Byrnes, Kathleen Feild, Nancy Glover, Franklin Jackson, Kaelin Kuhn, Jason Lee, Scott Pettis, Meredith Raymond, Maria Stattel, and Jessica Waselkow.