Environmental Satellites: NOAA Needs to Ensure Its Timelines Are Accurate, Clear, and Fully Documented
Released: 2016-09-08T00:00:00

Letter

NOAA Has a Process for Updating Flyout Charts, but It Is Not Established in Policy

NOAA has a process for updating its flyout charts, but this process is not established in policy. NOAA’s process for updating its flyout charts involves obtaining updated information on the health of operational satellites from internal specialists and program based studies, such as satellite availability assessments; obtaining updated information on the launch schedules for new satellites; having relevant individuals and entities review the updated charts; and obtaining approval from the Assistant Administrator of the National Environmental Satellite, Data, and Information Service (NESDIS) to publish the new chart on its public facing website. This process is triggered at least once a year in preparation for the budget process or more often when important changes occur, such as a loss of use of a satellite or when a budget decision affects a launch date. Officials estimated that it can take several weeks to prepare, review, and obtain consensus to publish the new charts. This process is partially documented in a draft policy from June 2011.

NESDIS officials stated that the draft policy serves as documentation of the agency’s process; however, the policy was never finalized and is currently out-of-date because several details have changed since 2011. For example, the outdated draft does not include the name of the office responsible for updating the flyout charts, or define the use of fuel limited life to estimate the lifespan of operational satellites. According to the NESDIS Assistant Administrator, the agency recently appointed a new director with responsibility for updating and finalizing the policy. However, NESDIS officials have not yet established a schedule for releasing the updated policy. Without a revised and finalized policy in place to govern the flyout chart process, NOAA runs an increased risk that its practices will be inconsistent and unclear.

NOAA Has Made Several Recent Changes to Its Flyout Charts

Using the process outlined in NOAA’s draft policy, program officials have updated the geostationary and polar-orbiting satellite flyout charts three times since March 2014. Key changes included adding newly planned satellites; removing a satellite that reached the end of its life; and adjusting planned dates for when satellites would launch, begin operations, and reach the end of their lives. For example, in one set of changes between April 2015 and January 2016, NOAA (1) delayed the GOES-R launch date from March to October 2016, resulting in a corresponding move in its estimated end-of-life, and (2) added 6-month on-orbit checkout periods to the next three satellites in the series, called GOES-S, T, and U satellites. Figure 1 shows the changes for NOAA’s geostationary satellites between April 2015 and January 2016.

NOAA Has Not Consistently and Accurately Updated Its Flyout Charts

While NOAA regularly updates its flyout charts and most of the data on specific satellites were aligned with supporting program documents, the agency has not consistently ensured that its charts were supported by stringent analysis, accurate, clearly communicated, and fully documented.

As specified by relevant guidance to agencies for facilitating congressional decision-making and enforcing government internal controls, agencies should ensure that the information presented to Congress is accurate, clear, and supported by documentation. Our review of changes to the flyout charts since March 2014 found that most of the updates to the flyout charts accurately reflected relevant information from satellite program offices at the time they were published; and were subject to internal review before they were finalized as evidenced by summary packages which can include e-mails, documents tracking the changes, and the official approval of the charts by the Assistant Administrator.

However, in its efforts to provide updated flyout charts to Congress, the agency has not consistently ensured that the data were (1) supported by stringent analyses of the satellites’ health and availability; (2) accurate and consistent with supporting program data; (3) clear in how a satellite’s extended life is portrayed; and (4) fully documented. More specifically:

Stringent analysis: NOAA does not require regular satellite availability assessments for any of its environmental satellite programs. Satellite managing agencies often perform technical assessments of health and future availability of operational satellites to aid in planning and budgeting. For example, the Air Force requires satellite programs to complete an independent assessment of satellite and constellation health each year as part of its budget preparations. While NOAA conducts an annual assessment for its JPSS program, it does not conduct such assessments for its GOES-R program. Without requiring regular availability assessments for all satellites, NOAA risks not having timely information on the probability of continued success of its operational satellites for program budgeting purposes.

Accuracy: NOAA’s flyout chart updates are not always accurate and consistent with agency documentation including program schedules for future satellites and polar availability assessments. Out of 26 instances where we compared flyout chart data to underlying program data for a particular satellite, we identified 2 instances where the flyout charts did not accurately depict the underlying program schedules. For example, the flyout charts showed launch dates for two satellites as 4 months earlier and 3 months later than program data. Both of these issues were later corrected when the next chart was updated 6- 12 months later. However, they were inaccurate at the time they were provided to Congress.

In addition, NOAA’s updates were at times inconsistent with the polar satellite availability assessment data. For example, NOAA’s January 2016 flyout chart depicts JPSS-1 lasting through March 2024 while a 2015 availability assessment shows only a 55 percent probability that the satellite will be fully functional in 2024. JPSS program officials explained that polar-orbiting availability assessments are used to only show degrading health over time, while the flyout charts portray expected satellite lifespans. However, we believe it is not accurate to show a satellite as functioning on the flyout chart when underlying analyses show that the satellite is unlikely to be fully functioning. A part of the reason for this lack of consistency is that NOAA does not have a policy in place that requires taking steps to ensure the accuracy of its charts. Until NOAA ensures its flyout charts correctly represent the best available knowledge on the health and availability of the satellite, the agency runs an increased risk that its charts will not be useful or trusted to inform the budget and appropriations processes and provide program updates.

Clarity: NOAA does not clearly and consistently depict how long a satellite might last once it is beyond its design life. For example, NOAA received a contractor study in 2005 showing that its geostationary satellites were likely to last a total of 10 years after launch, which was beyond the initial 7-year design life. Although the study was conducted in 2005, the agency did not update the satellites’ expected lives on the flyout charts until 2015. Similarly, in its 2015 and 2016 charts, NOAA shows its expectation that the NOAA-18 and 19 satellites would last 1 more year by extending the expected operation of the satellites by 1 year, even though the polar availability assessments show that they will likely last longer. In addition, in 2015 and 2016, NOAA adjusted its flyout charts to show extended life on three GOES satellites and the Suomi National Polar-orbiting Partnership (S-NPP) satellite using an extension labeled as “fuel limited life” in 2015. The agency later explained that this term is intended to show the maximum possible life assuming all instruments and the spacecraft continue to function, and not the satellite’s expected life. However, the agency did not clearly define this term on its charts, thereby allowing readers to assume that the agency expects the satellites to last through the end of the fuel-limited life period. Part of the reason that NOAA does not consistently describe how long a satellite is expected to last is that the agency does not have a policy in place requiring a standard approach or nomenclature. Until the agency establishes a consistent approach to describing a satellite’s extended life, it is at risk that its charts will be misconstrued, including by those making budget and appropriations decisions.

Documentation: While standard internal controls and NOAA’s draft policy calls for documenting the reasons for changes to the flyout charts and the executive approval for those changes, NOAA does not consistently document the justification for its updates. For example, of the six geostationary and polar summary packages we received by NOAA, three included justification for at least one key change and three did not include key program documentation for the changes to the flyout charts. Furthermore, based on the 27 key changes we noted on the flyout charts between March 2014 and January 2016, 9 were justified in NOAA documentation and 18 were not.

Program officials explained that documentation supporting each change exists and is widely circulated and vetted; however, we were unable to find this documentation in the packages provided by NOAA. Part of the reason for the inconsistencies is that NOAA does not have a policy in place requiring the creation and approval of standard justification packages. Until the agency documents and maintains a standard justification and approval package for each update, it risks not having all of the information it needs to justify a change to its flyout charts.

Conclusions

While NOAA has a process in place for updating its flyout charts and it regularly updates them, the agency’s process has multiple shortcomings and is not established in policy. Between March 2014 and January 2016, agency officials revised the flyout charts three times to add newly planned satellites; remove a satellite that ceased operations; and change the expected dates for launch, beginning operations, and end-of-life.

In its efforts to update its flyout charts, NOAA provides regular updates that are mostly consistent with supporting documentation. However, the agency does not require its satellite programs to conduct regular assessments of satellite availability, which could aid in determining how long its satellites will likely last. Moreover, the information in the flyout charts is not always consistent with supporting agency documentation; is not always consistent in how it presents a satellite’s extended life; and is not always supported by a complete justification package. Part of the reason for these issues is that NOAA has not established a policy that includes these steps. Until NOAA addresses the shortfalls in its practices and updates its policy to help ensure the flyout charts are accurate, consistent, and well-documented, it runs an increased risk that its flyout charts will be misleading to Congress and may lead to less-than-optimal decisions.

Recommendations for Executive Action

Given the importance of providing accurate and clear information to facilitate congressional decision making and inform the public, we are making the following five recommendations to the Secretary of Commerce. Specially, we recommend that the Secretary direct NOAA’s Assistant Administrator for Satellite and Information Services to take the following actions for its geostationary and polar-orbiting satellite programs:

Require satellite programs to perform regular availability assessments and use these analyses to inform the flyout charts and support its budget requests.

Ensure that flyout chart updates are consistent with supporting data from the program and from satellite availability assessments.

Establish and implement a consistent approach to depicting satellites that are expected to last beyond their design lives.

For each flyout chart update, maintain a complete package of documentation on the reasons for any changes and executive approval of the changes.

Revise and finalize the draft policy governing how flyout charts are to be updated to address the shortfalls with analysis, accuracy, consistency, and documentation noted in the above recommendations.

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of Commerce for review and comment. We received NOAA’s written comments from the Department of Commerce, which are reproduced in appendix II. NOAA concurred with all five of our recommendations and noted that it plans to implement a more consistent approach in updating its flyout charts. The agency added that our review provided valuable feedback concerning how we and Congress use the charts, and underscored the importance of ensuring that viewers understand that complex operational and acquisition decisions cannot be depicted in a single graphic. NOAA also provided technical comments, which we have incorporated into our report and the briefing slides in appendix I, as appropriate.

In response to our first recommendation to require satellite programs to perform regular availability assessments and use them to inform the flyout charts, NOAA concurred and noted that operations personnel perform regular health and status monitoring for satellites under their command and control, make predictions of fuel-limited life, and post status updates to the operational satellite web pages. NOAA also noted that upcoming JPSS and GOES-R satellites will transmit more health data which will enable more complete availability assessments in the future. We acknowledge that performing health and status monitoring of satellites in orbit is important. Further, conducting availability assessments for all satellites should help the agency understand the potential instrument failure before the end of fuel-limited life and enable timely and accurate information for program and budget planning.

In response to our second recommendation to ensure that flyout chart updates are consistent with supporting data, NOAA concurred while acknowledging the risk of a reader reaching inaccurate conclusions from its flyout charts. NOAA explained that its flyout charts are not meant to be a replacement for more detailed charts and documentation, which are made available to Congress. We believe that this risk would be reduced if the charts were checked to ensure they accurately reflect the underlying data.

NOAA also concurred with our third recommendation to establish and implement a consistent approach to depicting satellites expected to last beyond their design lives. Prior to providing this report, we obtained comments from NOAA on the recommendations in our briefing provided to subcommittee staff on May 31, 2016. While NOAA initially partially concurred with this recommendation in the briefing, NOAA subsequently concurred and acknowledged the need to establish a consistent approach across satellites.

The agency concurred with our fourth recommendation to maintain a complete package of documentation on the reasons for any changes to the flyout charts and the approval of those changes. NOAA stated that it updated its draft policy governing its flyout chart process to include a requirement to maintain documentation for flyout chart changes. The agency stated that while it has not maintained more detailed information, it will now do so.

NOAA also concurred with the fifth recommendation to finalize its draft policy governing how flyout charts are to be updated. The agency noted that the new policy is in internal coordination and should be formally approved very soon. We believe that addressing our recommendations to improve processes and policies will help ensure that the flyout charts NOAA uses to inform Congress and other stakeholders are supported by strong analyses, accurately reported, and clearly communicated.

We are sending copies of this report to interested congressional committees, the Secretary of Commerce, the Director of the Office of Management and Budget, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Briefing for Staff Members of a Congressional Committee

Appendix II: Comments from the Department of Commerce

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Colleen Phillips (Assistant Director), Torrey Hardee (Analyst in Charge), Chris Businsky, Shaun Byrnes, Rebecca Eyler, Franklin Jackson, and Umesh Thakkar made key contributions to this report.