Polar-Orbiting Operational Environmental Satellites: Technical Problems, Cost Increases, and Schedule Delays Trigger Need for Difficult Trade-off Decisions
Released: 2005-11-16T00:00:00

Letter

Background

Since the 1960s, the United States has operated two separate operational polar-orbiting meteorological satellite systems: the Polar-orbiting Operational Environmental Satellites (POES), managed by the National Oceanic and Atmospheric Administration (NOAA) and the Defense Meteorological Satellite Program (DMSP), managed by the Department of Defense (DOD). The satellites obtain environmental data that are processed to provide graphical weather images and specialized weather products and are the predominant input to numerical weather prediction models. These images, products, and models are all used by weather forecasters, the military, and the public. Polar satellites also provide data used to monitor environmental phenomena, such as ozone depletion and drought conditions, as well as data sets that are used by researchers for a variety of studies, such as climate monitoring.

Unlike geostationary satellites, which maintain a fixed position above the earth, polar-orbiting satellites constantly circle the earth in an almost north-south orbit, providing global coverage of conditions that affect the weather and climate. Each satellite makes about 14 orbits a day. As the earth rotates beneath it, each satellite views the entire earth’s surface twice a day. Currently, there are two operational POES satellites and two operational DMSP satellites that are positioned so that they can observe the earth in early morning, mid morning, and early afternoon polar orbits. Together, they ensure that, for any region of the earth, the data provided to users are generally no more than 6 hours old. Figure 1 illustrates the current operational polar satellite configuration. Besides the four operational satellites, six older satellites are in orbit that still collect some data and are available to provide some limited backup to the operational satellites should they degrade or fail. In the future, both NOAA and DOD plan to continue to launch additional POES and DMSP satellites every few years, with final launches scheduled for 2007 and 2011, respectively.

Each of the polar satellites carries a suite of sensors designed to detect environmental data that are either reflected or emitted from the earth, the atmosphere, and space. The satellites store these data and then transmit them to NOAA and Air Force ground stations when the satellites pass overhead. The ground stations then relay the data via communications satellites to the appropriate meteorological centers for processing. The satellites also broadcast a subset of these data in real time to tactical receivers all over the world.

Under a shared processing agreement among four satellite data processing centers—NOAA’s National Environmental Satellite Data and Information Service (NESDIS), the Air Force Weather Agency, the Navy’s Fleet Numerical Meteorology and Oceanography Center, and the Naval Oceanographic Office—different centers are responsible for producing and distributing, via a shared network, different environmental data sets, specialized weather and oceanographic products, and weather prediction model outputs. Each of the four processing centers is also responsible for distributing the data to its respective users. For the DOD centers, the users include regional meteorology and oceanography centers, as well as meteorology and oceanography staff on military bases. NESDIS forwards the data to NOAA’s National Weather Service for distribution and use by government and commercial forecasters. The processing centers also use the Internet to distribute data to the general public. NESDIS is responsible for the long-term archiving of data and derived products from POES and DMSP.

In addition to the infrastructure supporting satellite data processing noted above, properly equipped field terminals that are within a direct line of sight of the satellites can receive real-time data directly from the polar- orbiting satellites. There are an estimated 150 such field terminals operated by U.S. and foreign governments and academia. Field terminals can be taken into areas with little or no data communications infrastructure—such as on a battlefield or a ship—and enable the receipt of weather data directly from the polar-orbiting satellites. These terminals have their own software and processing capability to decode and display a subset of the satellite data to the user. Figure 2 depicts a generic data relay pattern from the polar-orbiting satellites to the data processing centers and field terminals.

NPOESS Overview

Given the expectation that combining the POES and DMSP programs would reduce duplication and result in sizable cost savings, a May 1994 Presidential Decision Directive required NOAA and DOD to converge the two satellite programs into a single satellite program capable of satisfying both civilian and military requirements. The converged program, NPOESS, is considered critical to the United States’ ability to maintain the continuity of data required for weather forecasting and global climate monitoring through the year 2020. To manage this program, DOD, NOAA, and the National Aeronautics and Space Administration (NASA) formed a tri-agency Integrated Program Office, located within NOAA.

Within the program office, each agency has the lead on certain activities. NOAA has overall program management responsibility for the converged system and for satellite operations; DOD has the lead on the acquisition; and NASA has primary responsibility for facilitating the development and incorporation of new technologies into the converged system. NOAA and DOD share the costs of funding NPOESS, while NASA funds specific technology projects and studies. Figure 3 depicts the organizations comprising the Integrated Program Office and lists their responsibilities.

Program acquisition plans call for the procurement and launch of six NPOESS satellites over the life of the program, as well as the integration of 13 instruments, consisting of 10 environmental sensors and 3 subsystems. Together, the sensors are to receive and transmit data on atmospheric, cloud cover, environmental, climate, oceanographic, and solar-geophysical observations. The subsystems are to support nonenvironmental search and rescue efforts, sensor survivability, and environmental data collection activities. According to the program office, 7 of the 13 planned NPOESS instruments involve new technology development, whereas 6 others are based on existing technologies. In addition, the program office considers 4 of the sensors involving new technologies critical, because they provide data for key weather products; these sensors are shown in bold in table 1, which lists the planned instruments and the state of technology on each.

In addition to the sensors and subsystems listed above, in August 2004, the President directed NASA and the Departments of Defense, the Interior, and Commerce to place a LANDSAT-like imagery capability on the NPOESS platform. This new capability is to collect imagery data of the earth’s surface similar to the current LANDSAT series of satellites, which are managed by the Department of Interior’s U.S. Geological Survey and are reaching the end of their respective lifespans. One instrument was launched in 1984 and is now long past its 3-year design life; the newer satellite is not fully operational. LANDSAT is an important tool in environmental monitoring efforts, including land cover change, vegetation mapping, and wildfire effects. The decision to add a LANDSAT-like sensor to the NPOESS platform is currently being revisited by the President’s Office of Science and Technology Policy and the Office of Management and Budget.

In addition, the NPOESS Preparatory Project (NPP), which is being developed as a major risk reduction and climate data continuity initiative, is a planned demonstration satellite to be launched several years before the first NPOESS satellite is to be launched. It is planned to host three of the four critical NPOESS sensors (the visible/infrared imager radiometer suite, the cross-track infrared sounder, and the advanced technology microwave sounder), as well as a noncritical sensor (the ozone mapper/profiler suite). NPP will provide the program office and the processing centers an early opportunity to work with the sensors, ground control, and data processing systems. Specifically, this satellite is expected to demonstrate the validity of about half of the NPOESS environmental data records and about 93 percent of its data processing load.

NPOESS Acquisition Strategy

NPOESS is a major system acquisition that consists of three key phases: the concept and technology development phase, which lasted from roughly 1995 to early 1997; the program definition and risk reduction phase which began in early 1997 and ended in August 2002; and the engineering and manufacturing development and production phase, which began with the award of the development and production contract in August 2002 and will continue through the end of the program. Before the contract was awarded in 2002, the life cycle cost estimate for the program was estimated to be $6.5 billion over the 24-year period from the inception of the program in 1995 through 2018. Shortly after the contract was awarded, the life cycle cost estimate grew to $7 billion.

When the NPOESS development contract was awarded, program officials identified an anticipated schedule and funding stream for the program. The schedule for launching the satellites was driven by a requirement that the satellites be available to back up the final POES and DMSP satellites should anything go wrong during the planned launches of these satellites. In general, program officials anticipate that roughly 1 out of every 10 satellites will fail either during launch or during early operations after launch.

Early program milestones included (1) launching NPP by May 2006, (2) having the first NPOESS satellite available to back up the final POES satellite launch in March 2008, and (3) having the second NPOESS satellite available to back up the final DMSP satellite launch in October 2009. If the NPOESS satellites were not needed to back up the final predecessor satellites, their anticipated launch dates would have been April 2009 and June 2011, respectively.

In 2003, we reported that these schedules were subsequently changed as a result of changes in the NPOESS funding stream. A DOD program official reported that between 2001 and 2002 the agency experienced delays in launching a DMSP satellite, causing delays in the expected launch dates of another satellite. In late 2002, DOD shifted the expected launch date for the final satellite from 2009 to 2010. As a result, the department reduced funding for NPOESS by about $65 million between fiscal years 2004 and 2007. According to program officials, because NOAA is required to provide the same level of funding that DOD provides, this change triggered a corresponding reduction in funding by NOAA for those years. As a result of the reduced funding, program officials were forced to make difficult decisions about what to focus on first. The program office decided to keep NPP as close to its original schedule as possible, because of its importance to the eventual NPOESS development, and to shift some of the NPOESS deliverables to later years. This shift affected the NPOESS deployment schedule. To plan for this shift, the program office developed a new program cost and schedule baseline.

After this new baseline was completed in 2004, we reported that the program office increased the NPOESS cost estimate from about $7 billion to $8.1 billion, and delayed key milestones, including the planned launch of the first NPOESS satellite—which was delayed by 7 months. The cost increases reflected changes to the NPOESS contract as well as increased program management funds. According to the program office, contract changes included extension of the development schedule, increased sensor costs, and additional funds needed for mitigating risks. Increased program management funds were added for non-contract costs and management reserves.

We also noted that other factors could further affect the revised cost and schedule estimates. Specifically, the contractor was not meeting expected cost and schedule targets of the new baseline because of technical issues in the development of key sensors. Based on its performance through May 2004, we estimated that the contractor would most likely overrun its contract at completion in September 2011 by $500 million. In addition, we reported that risks associated with the development of the critical sensors, integrated data processing system, and algorithms, among other things, could contribute to further cost increases and schedule slips.

NPOESS Schedules, Costs, and Trends Continue to Worsen

Over the past year, NPOESS cost increases and schedule delays have demonstrated worsening trends. NPOESS has continued to experience problems in the development of a key sensor, resulting in schedule delays and anticipated cost increases. Further, contractor data show that costs and schedules are likely to continue to increase in the future. Our trend analysis shows that the contractor will most likely overrun costs by $1.4 billion, resulting in a life cycle cost of about $9.7 billion, unless critical changes are made. Program risks, particularly with the development of critical sensors, could further increase NPOESS costs and delay schedules. Management problems at multiple levels—subcontractor, contractor, program office, and executive leadership—have contributed to these cost and schedule issues.

NPOESS Sensor Problems Triggered Schedule Delays and Cost Increases

NPOESS has continued to experience problems in the development of a key sensor, resulting in schedule delays and anticipated cost increases. In early 2005, the program office learned that a subcontractor could not meet cost and schedule due to significant technical issues on the visible/infrared imager radiometer suite (VIIRS) sensor—including problems with the cryoradiator, excessive vibration of sensor parts, and errors in the sensor’s solar calibration. These technical problems were further complicated by inadequate process engineering and management oversight by the VIIRS subcontractor. To address these issues, the program office provided additional funds for VIIRS, capped development funding for the conical-scanned microwave imager/sounder (CMIS) and the ozone mapper/profiler suite sensors, and revised its schedule in order to keep the program moving forward.

By the summer of 2005, the program office reported that significant technical issues had been resolved—but they had a significant impact on the overall NPOESS program. Regarding NPOESS schedule, the program office anticipated at least a 10-month delay in the launch of the first satellite (totaling at least a 17-month delay from the time the contract was awarded) and a 6-month delay in the launch of the second satellite. A summary of recent schedule changes is shown in table 2. The effect of these delays is evident in the widening gap between when the last POES satellite is expected to launch and when the first NPOESS satellite could be available if needed as a backup. This is significant because if the last POES satellite fails on launch, it will be at least 3 years before the first NPOESS satellite could be launched. During that time, critical weather and environmental observations would be unavailable—and military and civilian weather products and forecasts would be significantly degraded.

As for NPOESS costs, program officials reported that the VIIRS development problems caused the program to overrun its budget, and that they need to reassess options for funding the program. They did not provide an updated cost estimate, noting that new cost estimates are under development. A summary of recent program cost growth is shown in table 3.

Trends in Contractor Data Show Continued Cost and Schedule Overruns; Overall Costs Projected to Grow

In addition to the overall program office cost and schedule estimates, it is valuable to assess contractor data to monitor the contractor’s progress in meeting deliverables since contractor costs comprise a substantial portion of the overall program costs. NPOESS contractor data show a pattern of cost and schedule overruns—and a most likely contract cost growth of about $1.4 billion.

One method project managers use to track contractor progress on deliverables is earned value management. This method, used by DOD for several decades, compares the value of work accomplished during a given period with that of the work expected in that period. Differences from expectations are measured in both cost and schedule variances. Cost variances compare the earned value of the completed work with the actual cost of the work performed. For example, if a contractor completed $5 million worth of work and the work actually cost $6.7 million, there would be a –$1.7 million cost variance. Schedule variances are also measured in dollars, but they compare the earned value of the work completed to the value of work that was expected to be completed. For example, if a contractor completed $5 million worth of work at the end of the month, but was budgeted to complete $10 million worth of work, there would be a –$5 million schedule variance. Positive variances indicate that activities are costing less or are completed ahead of schedule. Negative variances indicate that activities are costing more or are falling behind schedule. These cost and schedule variances can then be used in estimating the cost and time needed to complete the program.

Using contractor-provided data, our analysis indicates that NPOESS cost performance continues to experience negative variances. Figure 4 shows the 6-month cumulative cost variance for the NPOESS contract. From March 2005 to September 2005, the contractor exceeded its cost target by $103.7 million, which is about 9 percent of the contractor’s budget for that time period. The contractor has incurred a total cost overrun of $253.8 million with NPOESS development only about 36 percent complete. This information is useful because trends often tend to continue and can be difficult to reverse unless management attention is focused on key risk areas and risk mitigation actions are aggressively pursued. Studies have shown that, once programs are 15 percent complete, the performance indicators are indicative of the final outcome.

Based on contractor performance from March 2005 to September 2005, we estimate that the current NPOESS contract will overrun its budget—worth approximately $3.4 billion—by between $788 million and $2 billion. Our projection of the most likely cost overrun is about $1.4 billion. The contractor, in contrast, estimates about a $371 million overrun at completion of the NPOESS contract. Adding our projected $1.4 billion overrun to the prior $8.1 billion life cycle cost estimate and the project office’s estimated need for $225 million in additional management costs brings the total life cycle cost of the program to about $9.7 billion.

Our analysis also indicates that the contract is showing a negative schedule variance. Figure 5 shows the 6-month cumulative schedule variance of NPOESS. From March 2005 to September 2005, the contractor was unable to complete $27.8 million worth of scheduled work. In September, the contractor was able to improve its overall schedule performance because of an unexpectedly large amount of work being completed on the spacecraft (as opposed to the sensors). It was not a reflection of an improvement in the contractor’s ability to complete work on the critical sensors. Specifically, performance on the development of critical sensors over the past 6 months continued to be poor, which indicates that schedule performance will likely remain poor in the future. This is of concern because an inability to meet contract schedule performance could be a predictor of future rising costs, as more spending is often necessary to resolve schedule overruns.

Risks Could Further Affect NPOESS Cost and Schedules

Risk management is a leading management practice that is widely recognized as a key component of a sound system development approach. An effective risk management approach typically includes identifying, prioritizing, resolving, and monitoring project risks.

Program officials reported that they recognize several risks with the overall program and critical sensors that, if not mitigated, could further increase costs and delay the schedule. In accordance with leading management practices, the program office developed a NPOESS risk management program that requires assigning a severity rating to risks that bear particular attention, placing these risks in a database, planning response strategies for each risk in the database, and reviewing and evaluating risks in the database during monthly program risk management board meetings.

The program office identifies risks in two categories: program risks, which affect the whole NPOESS program and are managed at the program office level, and segment risks, which affect only individual segments and are managed at the integrated product team level. The program office has identified 17 program risks, including 10 medium to medium-high risks. Some of these risks include the delivery of four sensors (VIIRS, CMIS, the cross-track infrared sounder and the ozone mapper/profiler suite) and the integrated data processing system; and the uncertainty that algorithms will meet system performance requirements. Figure 6 identifies the 17 program risks and their assigned levels of risk.

Managing the risks associated with the development of VIIRS, the ozone mapper/profiler suite, the cross-track infrared sounder, the integrated data processing system, and algorithm performance is of particular importance because these are to be demonstrated on the NPP satellite that is currently scheduled for launch in April 2008. The risks with the development of CMIS are also important because CMIS is one of the four critical sensors providing data for key weather products.

At present, the program office considers two critical sensors—VIIRS and CMIS—to present key program risks because of technical challenges that each is facing. In addition to the previously reported VIIRS problems, the sensor continues to experience significant problems dealing with the technical complexity of the ground support equipment. The testing of optical and solar diffuser components has also been more challenging than expected and is taking longer than planned to complete. In addition, the delivery of components for integration onto the sensor, including the electronics material from two subcontractors, has been behind schedule due to technical challenges. Until the current technical issues are resolved, delays in the VIIRS delivery and integration onto the NPP satellite remain a potential threat to the expected launch date of the NPP.

The CMIS sensor is experiencing schedule overruns that may threaten its expected delivery date. Based on the prime contractor’s analysis, late deliveries of major CMIS subsystems will occur unless the current schedule is extended. For example, the simulator hardware is already expected to be delivered late, based on the current contractual requirement of December 2006. CMIS also continues to experience technical challenges in the design of the radio frequency receivers, the structure, and the antenna. In addition, extensive effort has been expended to resolve system reliability and thermal issues, among other things. To the program office’s credit, it is aware of these risks and is using its risk management plans to help mitigate them.

Current Program Issues Due, In Part, to Problems at Multiple Management Levels

Problems involving multiple levels of management—including subcontractor, contractor, program office, and executive leadership—have played a role in bringing the NPOESS program to its current state. As noted earlier, VIIRS sensor development issues were attributed, in part, to the subcontractor’s inadequate project management. Specifically, after a series of technical problems, internal review teams sent by the prime contractor and the program office found that the VIIRS subcontractor had deviated from a number of contract, management, and policy directives set out by the main office and that both management and process engineering were inadequate. Neither the contractor nor the program office recognized the underlying problems in time to fix them. After these issues were identified, the subcontractor’s management team was replaced. Further, in January 2005, the NPOESS Executive Committee (Excom) called for an independent review of the VIIRS problems. This independent review, delivered in August 2005, reported that the program management office did not have the technical system engineering support it needed to effectively manage the contractor, among other things. Additionally, the involvement of NPOESS executive leadership has wavered from frequent heavy involvement to occasional meetings with few resulting decisions. Specifically, the Excom has met five times over the last 2 years. Most of these meetings did not result in major decisions, but rather triggered further analysis and review. For instance, program officials and the program’s Tri-agency Steering Commitee identified five options to present at the executive committee meeting in mid-August 2005 and expected to receive direction on how to proceed with the project. The Excom did not select an option. Instead, it requested further analysis of the options by another independent review team, and an independent cost estimate by DOD’s Cost Analysis Improvement Group.

Sound management is critical to program success. In our reviews of major acquisitions throughout the government, we have reported that sound program management, contractor oversight, risk identification and escalation, and effective and timely executive level oversight are key factors determining a project’s ability to be delivered on time, within budget, and with promised functionality. Given the history of large cost increases and the factors that could further affect NPOESS costs and schedules, continued oversight, strong leadership, and timely decision making are more critical than ever.

Options for Moving Forward Are under Consideration, but Cost, Schedule, and Impact on Users Are Not Fully Understood

In August 2005, the program office briefed its Executive Committee on the program’s cost, schedule, and risks. The program office noted that the budget for the program was no longer executable and offered multiple alternatives for reconfiguring the program. Specifically, the program office and contractor developed 26 options during the March to August 2005 timeframe. Of these options, the Tri-agency Steering Committee selected five options, shown in table 4. All of these options alter the costs, schedules, and deliverables for the program. While the options’ preliminary life cycle cost estimates range from $8.8 billion to $9.2 billion, they all involve reductions in functionality and limited probabilities for meeting schedules within the cited budgets. None of the options presented discussed the potential for adding funding in the short term to hold off longer-term life cycle cost increases.

Project officials anticipated that at its August meeting, the Excom would decide on an option and provide directions for keeping the project moving. However, Excom officials requested further analysis and detailed cost estimates, and they deferred a decision among alternatives until December 2005.

New Options Under Consideration Would Affect Cost, Schedule, and System Users; Full Extent Unknown

Last week, we learned that in addition to the five options presented in August 2005, program executives are considering nine new options. While we were not provided any details about the nine new options, program officials informed us that they too will affect NPOESS costs, schedule, and promised functionality for system users—although their full impact is not yet clear. Program officials expect the Excom to decide on a limited number of options on November 22, 2005, and to obtain independent cost estimates of those options and make a decision to implement one of the options in December 2005. After a decision is made, the prime contractor will need time to develop more precise cost estimates and the program office with need to renegotiate the contract. Until a decision is made, the program remains without a plan for moving forward. Further, there are opportunity costs in not making a decision—that is, some options may no longer be viable, contractors are not working towards a chosen solution, and other potential options become more difficult to implement Clearly, timely decisions are needed to allow the program to move forward and for satellite data users to start planning for any data shortfalls they may experience. Until a decision is made on how the program is to proceed, the contractor and program office cannot start to implement the chosen solution and some decisions, such as the ability to hold schedule slips to a minimum, become much more difficult.

In summary, NPOESS is a program in crisis. Over the last few years, it has been troubled by technical problems, cost increases, and schedule delays. Looking forward, technical challenges persist; costs are likely to grow; and schedule delays could lead to gaps in satellite coverage. Program officials and executives are considering various options for dropping functionality in order to handle cost and schedule increases, but the full impact of these options is not clear. Moving forward, continued oversight, strong leadership, and informed and timely decision making are more critical than ever.

This concludes my statement. I would be pleased to respond to any questions that you or other members of the Committee may have at this time.

Contact and Acknowledgements

If you have any questions regarding this testimony, please contact David Powner at (202) 512-9286 or by email at pownerd@gao.gov. Individuals making contributions to this testimony include Carol Cha, Neil Doherty, Joanne Fiorino, Kathleen S. Lovett, Colleen Phillips, and Karen Richey.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) discuss the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program’s schedule, cost, trends, and risks and (2) describe plans and implications for moving the program forward. To accomplish these objectives, we focused our review on the Integrated Program Office, the organization responsible for the overall NPOESS program. We also met with officials from the Department of Defense, the National Aeronautics and Space Administration, and NOAA’s National Weather Service and National Environmental Satellite Data and Information Service to discuss user needs for the program.

To identify schedule and cost changes, we reviewed program office contract data, the Executive Committee minutes and briefings, and an independent review team study, and we interviewed program officials. We compared changes in NPOESS cost and schedule estimates to prior cost and schedule estimates as reported in our July 2002 and July 2003 testimonies and in our September 2004 report.

To identify trends that could affect the program baseline in the future, we assessed the prime contractor’s cost and schedule performance. To make these assessments, we applied earned value analysis techniques to data from contractor cost performance reports. We compared the cost of work completed with the budgeted costs for scheduled work for a 6-month period, from March to September 2005, to show trends in cost and schedule performance. We also used data from the reports to estimate the likely costs at the completion of the prime contract through established earned value formulas. This resulted in three different values, with the middle value being the most likely. We used the base contract without options for our earned value assessments.

To identify risks, we reviewed program risk management documents and interviewed program officials. Further, we evaluated earned value cost reports to determine the key risks that negatively affect NPOESS’s ability to maintain the current schedule and cost estimates.

To assess options and implications for moving the program forward, we reviewed the five options presented at the Executive Committee briefing and met with representatives of the National Weather Service and National Environmental Satellite Data and Information Service to obtain their views on user’s needs and priorities for satellite data.

NOAA officials generally agreed with the facts presented in this statement and provided some technical corrections, which we have incorporated. We performed our work at the Integrated Program Office, DOD, NASA, and NOAA in the Washington, D.C., metropolitan area, between June 2005 and November 2005, in accordance with generally accepted government auditing standards.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.