Environmental Satellites: NOAA Needs to Ensure Its Timelines Are Accurate, Clear, and Fully Documented
Published: 2016-09-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: NOAA manages two weather satellite programs that provide critical environmental data used in weather forecasts and warnings: a geostationary and a polar-orbiting satellite program. The agency is acquiring the next generation of satellites to replace existing satellites that are approaching the end of their expected lives. NOAA regularly publishes timelines, called flyout charts, depicting its expectations for how long its operational satellites will last and when it plans to launch new satellites. These charts are used to support budget requests, provide status reports, facilitate appropriations discussions with congressional committees, and inform the public.
GAO was asked to review NOAA's recent flyout charts. GAO's objectives were to (1) describe NOAA's process for updating its satellite flyout charts; (2) identify changes NOAA has made to its flyout charts in recent years and the justification for those changes; and (3) assess NOAA's recent efforts to update its flyout charts. To do so, GAO reviewed agency policies and procedures for updating its charts; analyzed changes made to the charts since March 2014; and compared NOAA's approach to Air Force practices, internal control standards, and program documentation.
section_title: What GAO Found
paragraphs: The National Oceanic and Atmospheric Administration's (NOAA) process for updating its flyout charts involves obtaining updated information on the health of operational satellites and schedules for new satellites, having relevant individuals review the updated charts, and obtaining approval from a senior NOAA official to publish the charts. This process is partially documented in a 2011 draft policy.
NOAA updated the geostationary and polar-orbiting flyout charts three times between March 2014 and January 2016. Key changes included adding newly planned satellites; removing a satellite that reached the end of its life; and adjusting planned dates for when satellites would launch, begin operations, and reach the end of their lives. For example, in one set of changes between April 2015 and January 2016, NOAA extended the life of older polar orbiting satellites by 1 year, added a new fuel limited life period to its most recently launched satellite (called S-NPP), and changed the launch date and the end-of-life date for another satellite (called JPSS-2), as shown below.
While NOAA has regularly updated its flyout charts and most of the data on specific satellites were aligned with supporting program documents, it has not consistently ensured that the data were supported by stringent analysis, consistent with underlying program data, clearly communicated, and fully documented. For example, unlike the Air Force, NOAA does not require regular availability assessments for its satellite programs. Also, NOAA's flyout chart updates are not always accurate and consistent with program schedules and polar availability assessments. Further, NOAA does not fully document its changes to the charts. For example, GAO's assessment of 27 key changes between March 2014 and January 2016 showed that 9 were justified in NOAA documentation and 18 were not. Part of the reason for these issues is that NOAA has not established a clear policy to standardize its approach. Until NOAA addresses the shortfalls in its practices and revises and finalizes its draft policy to help ensure the charts are accurate, consistent, and well documented, it runs an increased risk that its flyout charts will be misleading to Congress and may lead to less-than-optimal decisions.
section_title: What GAO Recommends
paragraphs: GAO recommends that NOAA take steps to improve the accuracy and consistency of its flyout charts, and to revise and finalize the draft policy for updating its flyout charts to address the shortfalls GAO noted. NOAA agreed with GAO's recommendations and identified plans to implement them.

section_title: Letter
paragraphs: The National Oceanic and Atmospheric Administration (NOAA) manages  two major environmental weather satellite programs that provide critical  data used in observing and forecasting weather. The geostationary and  the polar-orbiting programs are essential to the United States’ ability to  maintain the continuity of data required for weather forecasting and  providing watches and warnings of severe weather events.
NOAA is currently planning and executing major satellite acquisition  programs to replace aging geostationary and polar satellites that are  approaching the end of their expected lives. These include the $10.9  billion Geostationary Operational Environmental Satellite–R series  (GOES-R) program and the $11.3 billion Joint Polar Satellite System  (JPSS) program. As we have previously reported, these programs have  troubled legacies of cost increases, missed milestones, and technical  problems that have resulted in reduced functionality and major delays to  planned launch dates over time. Because of the criticality of satellite data  to weather forecasting, the possibility of a satellite data gap, and the  potential impact of a gap on the health and safety of the U.S. population  and economy, we added this issue to our high risk list in 2013 and it  remained on the list in 2015.
NOAA regularly publishes “flyout charts” for the geostationary and polar- orbiting satellite programs on its website. These charts depict timelines  for the launch, on-orbit storage, and operational life of the satellites.  NOAA uses these flyout charts to support budget requests, provide status  reports, and facilitate appropriations discussions with congressional  committees; alert satellite data users on when they will need to upgrade  their systems to accommodate new satellite technologies; and inform the  public of plans for maintaining continuity of satellite data.
Given your interest in the expected lifespans of NOAA’s two largest  satellite programs, you asked us to review NOAA’s recent flyout charts.  Our objectives were to: (1) describe NOAA’s process for updating,  approving, and publishing its satellite flyout charts; (2) identify changes  NOAA has made to its flyout charts in recent years and the justification for  those changes; and (3) assess NOAA’s recent efforts to update its flyout  charts.
On May 31, 2016, we briefed subcommittee staff on the results of our  study. An updated version of that briefing is provided in appendix I. This  report officially transmits our results to the committee and our  recommendations to the agency. To perform this work, we reviewed  agency policies and procedures for updating and approving flyout charts;  analyzed all changes NOAA made to its flyout charts since March 2014;  and compared NOAA’s approach against practices used by the Air Force,  internal controls, and relevant program assessments. The briefing slides  included in appendix I provide greater detail on our scope and  methodology.
We conducted this performance audit from February 2016 to September  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: NOAA Has a Process for Updating Flyout Charts, but It Is Not Established in Policy
paragraphs: NOAA has a process for updating its flyout charts, but this process is not  established in policy. NOAA’s process for updating its flyout charts  involves obtaining updated information on the health of operational  satellites from internal specialists and program based studies, such as  satellite availability assessments; obtaining updated information on the  launch schedules for new satellites; having relevant individuals and  entities review the updated charts; and obtaining approval from the  Assistant Administrator of the National Environmental Satellite, Data, and  Information Service (NESDIS) to publish the new chart on its public facing  website. This process is triggered at least once a year in preparation for  the budget process or more often when important changes occur, such as  a loss of use of a satellite or when a budget decision affects a launch  date. Officials estimated that it can take several weeks to prepare, review,  and obtain consensus to publish the new charts. This process is partially  documented in a draft policy from June 2011.
NESDIS officials stated that the draft policy serves as documentation of  the agency’s process; however, the policy was never finalized and is  currently out-of-date because several details have changed since 2011.  For example, the outdated draft does not include the name of the office  responsible for updating the flyout charts, or define the use of fuel limited  life to estimate the lifespan of operational satellites. According to the  NESDIS Assistant Administrator, the agency recently appointed a new  director with responsibility for updating and finalizing the policy. However,  NESDIS officials have not yet established a schedule for releasing the  updated policy. Without a revised and finalized policy in place to govern  the flyout chart process, NOAA runs an increased risk that its practices  will be inconsistent and unclear.
section_title: NOAA Has Made Several Recent Changes to Its Flyout Charts
paragraphs: Using the process outlined in NOAA’s draft policy, program officials have  updated the geostationary and polar-orbiting satellite flyout charts three  times since March 2014. Key changes included adding newly planned  satellites; removing a satellite that reached the end of its life; and  adjusting planned dates for when satellites would launch, begin  operations, and reach the end of their lives. For example, in one set of  changes between April 2015 and January 2016, NOAA (1) delayed the  GOES-R launch date from March to October 2016, resulting in a  corresponding move in its estimated end-of-life, and (2) added 6-month  on-orbit checkout periods to the next three satellites in the series, called  GOES-S, T, and U satellites. Figure 1 shows the changes for NOAA’s  geostationary satellites between April 2015 and January 2016.
section_title: NOAA Has Not Consistently and Accurately Updated Its Flyout Charts
paragraphs: While NOAA regularly updates its flyout charts and most of the data on  specific satellites were aligned with supporting program documents, the  agency has not consistently ensured that its charts were supported by  stringent analysis, accurate, clearly communicated, and fully documented.
As specified by relevant guidance to agencies for facilitating  congressional decision-making and enforcing government internal  controls, agencies should ensure that the information presented to  Congress is accurate, clear, and supported by documentation. Our  review of changes to the flyout charts since March 2014 found that most  of the updates to the flyout charts accurately reflected relevant  information from satellite program offices at the time they were published;  and were subject to internal review before they were finalized as  evidenced by summary packages which can include e-mails, documents  tracking the changes, and the official approval of the charts by the  Assistant Administrator.
However, in its efforts to provide updated flyout charts to Congress, the  agency has not consistently ensured that the data were (1) supported by  stringent analyses of the satellites’ health and availability; (2) accurate  and consistent with supporting program data; (3) clear in how a satellite’s  extended life is portrayed; and (4) fully documented. More specifically:
Stringent analysis: NOAA does not require regular satellite  availability assessments for any of its environmental satellite  programs. Satellite managing agencies often perform technical  assessments of health and future availability of operational satellites  to aid in planning and budgeting. For example, the Air Force requires  satellite programs to complete an independent assessment of satellite  and constellation health each year as part of its budget preparations.  While NOAA conducts an annual assessment for its JPSS program, it  does not conduct such assessments for its GOES-R program. Without  requiring regular availability assessments for all satellites, NOAA risks  not having timely information on the probability of continued success  of its operational satellites for program budgeting purposes.
Accuracy: NOAA’s flyout chart updates are not always accurate and  consistent with agency documentation including program schedules  for future satellites and polar availability assessments. Out of 26  instances where we compared flyout chart data to underlying program  data for a particular satellite, we identified 2 instances where the flyout  charts did not accurately depict the underlying program schedules.  For example, the flyout charts showed launch dates for two satellites  as 4 months earlier and 3 months later than program data. Both of  these issues were later corrected when the next chart was updated 6- 12 months later. However, they were inaccurate at the time they were  provided to Congress.
In addition, NOAA’s updates were at times inconsistent with the polar  satellite availability assessment data. For example, NOAA’s January  2016 flyout chart depicts JPSS-1 lasting through March 2024 while a  2015 availability assessment shows only a 55 percent probability that  the satellite will be fully functional in 2024. JPSS program officials  explained that polar-orbiting availability assessments are used to only  show degrading health over time, while the flyout charts portray  expected satellite lifespans. However, we believe it is not accurate to  show a satellite as functioning on the flyout chart when underlying  analyses show that the satellite is unlikely to be fully functioning. A  part of the reason for this lack of consistency is that NOAA does not  have a policy in place that requires taking steps to ensure the  accuracy of its charts. Until NOAA ensures its flyout charts correctly  represent the best available knowledge on the health and availability  of the satellite, the agency runs an increased risk that its charts will  not be useful or trusted to inform the budget and appropriations  processes and provide program updates.
Clarity: NOAA does not clearly and consistently depict how long a  satellite might last once it is beyond its design life. For example,  NOAA received a contractor study in 2005 showing that its  geostationary satellites were likely to last a total of 10 years after  launch, which was beyond the initial 7-year design life. Although the  study was conducted in 2005, the agency did not update the satellites’  expected lives on the flyout charts until 2015. Similarly, in its 2015 and  2016 charts, NOAA shows its expectation that the NOAA-18 and 19  satellites would last 1 more year by extending the expected operation  of the satellites by 1 year, even though the polar availability  assessments show that they will likely last longer. In addition, in 2015  and 2016, NOAA adjusted its flyout charts to show extended life on  three GOES satellites and the Suomi National Polar-orbiting  Partnership (S-NPP) satellite using an extension labeled as “fuel  limited life” in 2015. The agency later explained that this term is  intended to show the maximum possible life assuming all instruments  and the spacecraft continue to function, and not the satellite’s  expected life. However, the agency did not clearly define this term on  its charts, thereby allowing readers to assume that the agency  expects the satellites to last through the end of the fuel-limited life  period. Part of the reason that NOAA does not consistently describe  how long a satellite is expected to last is that the agency does not  have a policy in place requiring a standard approach or nomenclature.  Until the agency establishes a consistent approach to describing a  satellite’s extended life, it is at risk that its charts will be misconstrued,  including by those making budget and appropriations decisions.
Documentation: While standard internal controls and NOAA’s draft  policy calls for documenting the reasons for changes to the flyout  charts and the executive approval for those changes, NOAA does not  consistently document the justification for its updates. For example, of  the six geostationary and polar summary packages we received by  NOAA, three included justification for at least one key change and  three did not include key program documentation for the changes to  the flyout charts. Furthermore, based on the 27 key changes we noted  on the flyout charts between March 2014 and January 2016, 9 were  justified in NOAA documentation and 18 were not.
Program officials explained that documentation supporting each  change exists and is widely circulated and vetted; however, we were  unable to find this documentation in the packages provided by NOAA.  Part of the reason for the inconsistencies is that NOAA does not have  a policy in place requiring the creation and approval of standard  justification packages. Until the agency documents and maintains a  standard justification and approval package for each update, it risks  not having all of the information it needs to justify a change to its flyout  charts.
section_title: Conclusions
paragraphs: While NOAA has a process in place for updating its flyout charts and it  regularly updates them, the agency’s process has multiple shortcomings  and is not established in policy. Between March 2014 and January 2016,  agency officials revised the flyout charts three times to add newly planned  satellites; remove a satellite that ceased operations; and change the  expected dates for launch, beginning operations, and end-of-life.
In its efforts to update its flyout charts, NOAA provides regular updates  that are mostly consistent with supporting documentation. However, the  agency does not require its satellite programs to conduct regular  assessments of satellite availability, which could aid in determining how  long its satellites will likely last. Moreover, the information in the flyout  charts is not always consistent with supporting agency documentation; is  not always consistent in how it presents a satellite’s extended life; and is  not always supported by a complete justification package. Part of the  reason for these issues is that NOAA has not established a policy that  includes these steps. Until NOAA addresses the shortfalls in its practices  and updates its policy to help ensure the flyout charts are accurate,  consistent, and well-documented, it runs an increased risk that its flyout  charts will be misleading to Congress and may lead to less-than-optimal  decisions.
section_title: Recommendations for Executive Action
paragraphs: Given the importance of providing accurate and clear information to  facilitate congressional decision making and inform the public, we are  making the following five recommendations to the Secretary of  Commerce. Specially, we recommend that the Secretary direct NOAA’s  Assistant Administrator for Satellite and Information Services to take the  following actions for its geostationary and polar-orbiting satellite  programs:
Require satellite programs to perform regular availability assessments  and use these analyses to inform the flyout charts and support its  budget requests.
Ensure that flyout chart updates are consistent with supporting data  from the program and from satellite availability assessments.
Establish and implement a consistent approach to depicting satellites  that are expected to last beyond their design lives.
For each flyout chart update, maintain a complete package of  documentation on the reasons for any changes and executive  approval of the changes.
Revise and finalize the draft policy governing how flyout charts are to  be updated to address the shortfalls with analysis, accuracy,  consistency, and documentation noted in the above  recommendations.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of Commerce for  review and comment. We received NOAA’s written comments from the  Department of Commerce, which are reproduced in appendix II. NOAA  concurred with all five of our recommendations and noted that it plans to  implement a more consistent approach in updating its flyout charts. The  agency added that our review provided valuable feedback concerning  how we and Congress use the charts, and underscored the importance of  ensuring that viewers understand that complex operational and  acquisition decisions cannot be depicted in a single graphic. NOAA also  provided technical comments, which we have incorporated into our report  and the briefing slides in appendix I, as appropriate.
In response to our first recommendation to require satellite programs to  perform regular availability assessments and use them to inform the flyout  charts, NOAA concurred and noted that operations personnel perform  regular health and status monitoring for satellites under their command  and control, make predictions of fuel-limited life, and post status updates  to the operational satellite web pages. NOAA also noted that upcoming  JPSS and GOES-R satellites will transmit more health data which will  enable more complete availability assessments in the future. We  acknowledge that performing health and status monitoring of satellites in  orbit is important. Further, conducting availability assessments for all  satellites should help the agency understand the potential instrument  failure before the end of fuel-limited life and enable timely and accurate  information for program and budget planning.
In response to our second recommendation to ensure that flyout chart  updates are consistent with supporting data, NOAA concurred while  acknowledging the risk of a reader reaching inaccurate conclusions from  its flyout charts. NOAA explained that its flyout charts are not meant to be  a replacement for more detailed charts and documentation, which are  made available to Congress. We believe that this risk would be reduced if  the charts were checked to ensure they accurately reflect the underlying  data.
NOAA also concurred with our third recommendation to establish and  implement a consistent approach to depicting satellites expected to last  beyond their design lives. Prior to providing this report, we obtained  comments from NOAA on the recommendations in our briefing provided  to subcommittee staff on May 31, 2016. While NOAA initially partially  concurred with this recommendation in the briefing, NOAA subsequently  concurred and acknowledged the need to establish a consistent approach  across satellites.
The agency concurred with our fourth recommendation to maintain a  complete package of documentation on the reasons for any changes to  the flyout charts and the approval of those changes. NOAA stated that it  updated its draft policy governing its flyout chart process to include a  requirement to maintain documentation for flyout chart changes. The  agency stated that while it has not maintained more detailed information,  it will now do so.
NOAA also concurred with the fifth recommendation to finalize its draft  policy governing how flyout charts are to be updated. The agency noted  that the new policy is in internal coordination and should be formally  approved very soon. We believe that addressing our recommendations to  improve processes and policies will help ensure that the flyout charts  NOAA uses to inform Congress and other stakeholders are supported by  strong analyses, accurately reported, and clearly communicated.
We are sending copies of this report to interested congressional  committees, the Secretary of Commerce, the Director of the Office of  Management and Budget, and other interested parties. In addition, the  report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or at pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix III.
section_title: Appendix I: Briefing for Staff Members of a Congressional Committee
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Colleen Phillips (Assistant  Director), Torrey Hardee (Analyst in Charge), Chris Businsky, Shaun  Byrnes, Rebecca Eyler, Franklin Jackson, and Umesh Thakkar made key  contributions to this report.