Polar-Orbiting Operational Environmental Satellites: Restructuring Is Under Way, but Technical Challenges and Risks Remain
Released: 2007-06-07T00:00:00

Letter

Background

Since the 1960s, the United States has operated two separate operational polar-orbiting meteorological satellite systems: the Polar-orbiting Operational Environmental Satellite (POES) series—managed by NOAA, and the Defense Meteorological Satellite Program (DMSP)—managed by the Air Force. These satellites obtain environmental data that are processed to provide graphical weather images and specialized weather products. These satellite data are also the predominant input to numerical weather prediction models, which are a primary tool for forecasting weather 3 or more days in advance—including forecasting the path and intensity of hurricanes. The weather products and models are used to predict the potential impact of severe weather so that communities and emergency managers can help prevent and mitigate their effects. Polar satellites also provide data used to monitor environmental phenomena, such as ozone depletion and drought conditions, as well as data sets that are used by researchers for a variety of studies such as climate monitoring.

Unlike geostationary satellites, which maintain a fixed position relative to the earth, polar-orbiting satellites constantly circle the earth in an almost north-south orbit, providing global coverage of conditions that affect the weather and climate. Each satellite makes about 14 orbits a day. As the earth rotates beneath it, each satellite views the entire earth’s surface twice a day. Currently, there are two operational POES satellites and two operational DMSP satellites that are positioned so that they can observe the earth in early morning, midmorning, and early afternoon polar orbits. Together, they ensure that, for any region of the earth, the data provided to users are generally no more than 6 hours old. Figure 1 illustrates the current operational polar satellite configuration. Besides the four operational satellites, six older satellites are in orbit that still collect some data and are available to provide limited backup to the operational satellites should they degrade or fail. In the future, both NOAA and the Air Force plan to continue to launch additional POES and DMSP satellites every few years, with final launches scheduled for 2009 and 2012, respectively.

Each of the polar satellites carries a suite of sensors designed to detect environmental data that are either reflected or emitted from the earth, the atmosphere, and space. The satellites broadcast a subset of these data in real time to properly equipped field terminals that are within a direct line of sight; these field terminals are located at universities, on battlefields, or on ships. Additionally, the polar satellites store the observed environmental data and then transmit them to NOAA and Air Force ground stations when the satellites pass overhead. The ground stations then relay the data via communications satellites to the appropriate meteorological centers for processing.

Under a shared processing agreement among four satellite data processing centers—NOAA’s National Environmental Satellite Data and Information Service (NESDIS), the Air Force Weather Agency, the Navy’s Fleet Numerical Meteorology and Oceanography Center, and the Naval Oceanographic Office—different centers are responsible for producing and distributing, via a shared network, different environmental data sets, specialized weather and oceanographic products, and weather prediction model outputs.

Each of the four processing centers is also responsible for distributing the data to its respective users. For the DOD centers, the users include regional meteorology and oceanography centers, as well as meteorology and oceanography staff on military bases, the Naval Fleet, and mobile field sites. NESDIS forwards the data to NOAA’s National Weather Service for distribution and use by government and commercial forecasters. The processing centers also use the Internet to distribute data to the general public. NESDIS is responsible for the long-term archiving of data and derived products from POES and DMSP. Figure 2 depicts a generic data relay pattern from the polar-orbiting satellites to the data processing centers and field terminals.

Polar Satellite Data and Products

Polar satellites gather a broad range of data that are transformed into a variety of products. Satellite sensors observe different bands of radiation wavelengths, called channels, which are used for remotely determining information about the earth’s atmosphere, land surface, oceans, and the space environment. When first received, satellite data are considered raw data. To make them usable, the processing centers format the data so that they are time-sequenced and include earth location and calibration information. After formatting, these data are called raw data records. The centers further process these raw data records into channel-specific data sets, called sensor data records and temperature data records. These data records are then used to derive weather and climate products called environmental data records (EDR). EDRs include a wide range of atmospheric products detailing cloud coverage, temperature, humidity, and ozone distribution; land surface products showing snow cover, vegetation, and land use; ocean products depicting sea surface temperatures, sea ice, and wave height; and characterizations of the space environment. Combinations of these data records (raw, sensor, temperature, and environmental data records) are also used to derive more sophisticated products, including outputs from numerical weather models and assessments of climate trends. Figure 3 is a simplified depiction of the various stages of satellite data processing, and figures 4 and 5 depict examples of EDR weather products.

NPOESS Overview

With the expectation that combining the POES and DMSP programs would reduce duplication and result in sizable cost savings, a May 1994 Presidential Decision Directive required NOAA and DOD to converge the two satellite programs into a single satellite program capable of satisfying both civilian and military requirements. The converged program, NPOESS, is considered critical to the United States’ ability to maintain the continuity of data required for weather forecasting and global climate monitoring through the year 2026. To manage this program, DOD, NOAA, and NASA formed the tri-agency Integrated Program Office, located within NOAA.

Within the program office, each agency has the lead on certain activities: NOAA has overall program management responsibility for the converged system and for satellite operations; DOD has the lead on the acquisition; and NASA has primary responsibility for facilitating the development and incorporation of new technologies into the converged system. NOAA and DOD share the costs of funding NPOESS, while NASA funds specific technology projects and studies. Figure 6 depicts the organizations that make up the NPOESS program office and lists their responsibilities.

The NPOESS program office is overseen by an Executive Committee, which is made up of the Administrators of NOAA and NASA and the Undersecretary of the Air Force.

NPOESS Acquisition Strategy

NPOESS is a major system acquisition that was originally estimated to cost about $6.5 billion over the 24-year life of the program from its inception in 1995 through 2018. The program is to provide satellite development, satellite launch and operation, and ground-based satellite data processing. These deliverables are grouped into four main categories: (1) the space segment, which includes the satellites and sensors; (2) the integrated data processing segment, which is the system for transforming raw data into EDRs and is to be located at the four processing centers; (3) the command, control, and communications segment, which includes the equipment and services needed to support satellite operations; and (4) the launch segment, which includes the launch vehicle services.

When the NPOESS engineering, manufacturing, and development contract was awarded in August 2002, the cost estimate was adjusted to $7 billion. Acquisition plans called for the procurement and launch of six satellites over the life of the program, as well as the integration of 13 instruments— consisting of 10 environmental sensors and three subsystems. Together, the sensors were to receive and transmit data on atmospheric, cloud cover, environmental, climatic, oceanographic, and solar-geophysical observations. The subsystems were to support nonenvironmental search and rescue efforts, sensor survivability, and environmental data collection activities. The program office considered 4 of the sensors to be critical because they provide data for key weather products; these sensors are in bold in table 1, which describes each of the expected NPOESS instruments.

In addition, NPP was planned as a demonstration satellite to be launched several years before the first NPOESS satellite in order to reduce the risk associated with launching new sensor technologies and to ensure continuity of climate data with NASA’s Earth Observing System satellites.

NPP is to host three of the four critical NPOESS sensors (VIIRS, CrIS, and ATMS), as well as one other noncritical sensor (OMPS). NPP is to provide the program office and the processing centers an early opportunity to work with the sensors, ground control, and data processing systems.

When the NPOESS development contract was awarded, the schedule for launching the satellites was driven by a requirement that the satellites be available to back up the final POES and DMSP satellites should anything go wrong during the planned launches of these satellites. In general, satellite experts anticipate that roughly 1 out of every 10 satellites will fail either during launch or during early operations after launch.

Early program milestones included (1) launching NPP by May 2006, (2) having the first NPOESS satellite available to back up the final POES satellite launch in March 2008, and (3) having the second NPOESS satellite available to back up the final DMSP satellite launch in October 2009. If the NPOESS satellites were not needed to back up the final predecessor satellites, their anticipated launch dates would have been April 2009 and June 2011, respectively.

NPOESS Experienced Cost Increases, Schedule Delays, and Technical Problems over Several Years

Over the last few years, NPOESS has experienced continued cost increases and schedule delays, requiring difficult decisions to be made about the program’s direction and capabilities. In 2003, we reported that changes in the NPOESS funding stream caused a delay in the program’s schedule. Specifically, in late 2002, DOD shifted the expected launch date for its final DMSP satellite from 2009 to 2010. As a result, the department reduced funding for NPOESS by about $65 million between fiscal years 2004 and 2007. According to program officials, because NOAA was required to provide the same level of funding that DOD provides, this change triggered a corresponding reduction in funding by NOAA for those years. As a result of the reduced funding, program officials were forced to make difficult decisions about what to focus on first. The program office decided to keep NPP as close to its original schedule as possible because of its importance to the eventual NPOESS development and to shift some of the program’s deliverables to later years. This shift affected the NPOESS deployment schedule. To plan for this shift, the program office developed a new program cost and schedule baseline.

After this new baseline was completed in 2004, we reported that the program office increased the NPOESS cost estimate from about $7 billion to $8.1 billion; delayed key milestones, including the planned launch of the first NPOESS satellite—which was delayed by 7 months; and extended the life of the program from 2018 to 2020. The cost increases reflected changes to the NPOESS contract, as well as increased program management funds. According to the program office, contract changes included extension of the development schedule, increased sensor costs, and additional funds needed for mitigating risks. Increased program management funds were added for noncontract costs and management reserves.

At that time, we also noted that other factors could further affect the revised cost and schedule estimates. Specifically, the contractor was not meeting expected cost and schedule targets on the new baseline because of technical issues in the development of key sensors, including the critical VIIRS sensor. Based on its performance through May 2004, we estimated that the contractor would most likely overrun its contract at completion in September 2011 by $500 million—thereby increasing the projected life cycle cost to $8.6 billion. In addition, we reported that risks associated with the development of the critical sensors, integrated data processing system, and algorithms, among other things, could contribute to further cost increases and schedule slips—and we noted that continued oversight was critical. The program office’s baseline cost estimate was subsequently adjusted to $8.4 billion.

In mid-November 2005, we reported that NPOESS continued to experience problems in the development of a key sensor, resulting in schedule delays and anticipated cost increases. At that time, we projected that the program’s cost estimate had grown to about $10 billion based on contractor cost and schedule data. We reported that the program’s issues were due, in part, to problems at multiple levels of management— including subcontractor, contractor, program office, and executive leadership. Recognizing that the budget for the program was no longer executable, the NPOESS Executive Committee planned to make a decision in December 2005 on the future direction of the program—what would be delivered, at what cost, and by when. This involved deciding among options involving increased costs, delayed schedules, and reduced functionality. We noted that continued oversight, strong leadership, and timely decision making were more critical than ever and we urged the committee to make a decision quickly so that the program could proceed.

However, we subsequently reported that, in late November 2005, NPOESS cost growth exceeded a legislatively mandated threshold that requires DOD to certify the program to Congress. This placed any decision about the future direction of the program on hold until the certification took place in June 2006. In the meantime, the program office implemented an interim program plan for fiscal year 2006 to continue work on key sensors and other program elements using fiscal year 2006 funding.

Nunn-McCurdy Process Led to a Decision to Restructure the NPOESS Program

The Nunn-McCurdy law requires DOD to take specific actions when a major defense acquisition program exceeds certain cost thresholds. In November 2005, key provisions of the act required the Secretary of Defense to notify Congress when a major defense acquisition was expected to overrun its project baseline by 15 percent or more and to certify the program to Congress when it was expected to overrun its baseline by 25 percent or more. At that time, NPOESS exceeded the 25 percent threshold, and DOD was required to certify the program. Certifying a program entailed providing a determination that (1) the program is essential to national security, (2) there are no alternatives to the program that will provide equal or greater military capability at less cost, (3) the new estimates of the program’s cost are reasonable, and (4) the management structure for the program is adequate to manage and control costs. DOD established tri-agency teams—made up of DOD, NOAA, and NASA experts—to work on each of the four elements of the certification process.

In June 2006, DOD (with the agreement of both of its partner agencies) certified a restructured NPOESS program, estimated to cost $12.5 billion through 2026. This decision approved a cost increase of $4 billion over the prior approved baseline cost and delayed the launch of NPP and the first two satellites by roughly 3 to 5 years. The new program also entailed establishing a stronger program management structure, reducing the number of satellites to be produced and launched from 6 to 4, and reducing the number of instruments on the satellites from 13 to 9— consisting of 7 environmental sensors and 2 subsystems. It also entailed using NPOESS satellites in the early morning and afternoon orbits and relying on European satellites for midmorning orbit data. Table 2 summarizes the major program changes made under the Nunn-McCurdy certification decision.

The Nunn-McCurdy certification decision established new milestones for the delivery of key program elements, including launching NPP by January 2010, launching the first NPOESS satellite (called C1) by January 2013, and launching the second NPOESS satellite (called C2) by January 2016. These revised milestones deviated from prior plans to have the first NPOESS satellite available to back up the final POES satellite should anything go wrong during that launch.

Delaying the launch of the first NPOESS satellite means that if the final POES satellite fails on launch, satellite data users would need to rely on the existing constellation of environmental satellites until NPP data becomes available—almost 2 years later. Although NPP was not intended to be an operational asset, NASA agreed to move NPP to a different orbit so that its data would be available in the event of a premature failure of the final POES satellite. However, NPP will not provide all of the operational capability planned for the NPOESS spacecraft. If the health of the existing constellation of satellites diminishes—or if NPP data is not available, timely, and reliable—then there could be a gap in environmental satellite data. Table 3 summarizes changes in key program milestones over time.

In order to reduce program complexity, the Nunn-McCurdy certification decision decreased the number of NPOESS sensors from 13 to 9 and reduced the functionality of 4 sensors. Specifically, of the 13 original sensors, 5 sensors remain unchanged, 3 were replaced with less capable sensors, 1 was modified to provide less functionality, and 4 were cancelled. Table 4 shows the changes to NPOESS sensors, including the 4 identified in bold as critical sensors.

The changes in NPOESS sensors affected the number and quality of the resulting weather and environmental products, called EDRs. In selecting sensors for the restructured program, the Nunn-McCurdy process placed the highest priority on continuing current operational weather capabilities and a lower priority on obtaining selected environmental and climate measuring capabilities. As a result, the revised NPOESS system has significantly less capability for providing global climate measures than was originally planned. Specifically, the number of EDRs was decreased from 55 to 39, of which 6 are of a reduced quality. The 39 EDRs that remain include cloud base height, land surface temperature, precipitation type and rate, and sea surface winds. The 16 EDRs that were removed include cloud particle size and distribution, sea surface height, net solar radiation at the top of the atmosphere, and products to depict the electric fields in the space environment. The 6 EDRs that are of a reduced quality include ozone profile, soil moisture, and multiple products depicting energy in the space environment.

Given the changes in planned sensors, program officials established a planned configuration for NPP and the four satellites of the NPOESS program, called C1, C2, C3, and C4 (see table 5). Program officials acknowledged that this configuration could change if other parties decided to develop the sensors that were cancelled. However, they noted that the planned configuration of the first satellite cannot change without increasing the risk that the launch will be delayed.

Earned Value Management Techniques Provide Insight on Program Cost and Schedule

To be effective, project managers need current information on a contractor’s progress in meeting contract deliverables. One method that can help project managers track this progress is earned value management. This method, used by DOD for several decades, compares the value of work accomplished during a given period with that of the work expected in that period.

Differences from expectations are measured in both cost and schedule variances. Cost variances compare the earned value of the completed work with the actual cost of the work performed. For example, if a contractor completed $5 million worth of work and the work actually cost $6.7 million, there would be a –$1.7 million cost variance. Schedule variances are also measured in dollars, but they compare the earned value of the work completed with the value of work that was expected to be completed. For example, if a contractor completed $5 million worth of work at the end of the month but was budgeted to complete $10 million worth of work, there would be a –$5 million schedule variance. Positive variances indicate that activities are costing less or are completed ahead of schedule. Negative variances indicate activities are costing more or are falling behind schedule. These cost and schedule variances can then be used in estimating the cost and time needed to complete the program.

NPOESS Acquisition Restructuring Is Well Under Way, but Key Steps Remain To Be Completed

Since the June 2006 decision to revise the scope, cost, and schedule of the NPOESS program, the program office has made progress in restructuring the satellite acquisition; however, important tasks leading up to revising and finalizing contract changes remain to be completed. Restructuring a major acquisition program like NPOESS is a process that involves identifying time critical and high priority work and keeping this work moving forward, while reassessing development priorities, interdependencies, deliverables, risks, and costs. It also involves revising important acquisition documents including the memorandum of agreement on the roles and responsibilities of the three agencies, the acquisition strategy, the system engineering plan, the test and evaluation master plan, the integrated master schedule defining what needs to happen by when, and the acquisition program baseline. The Nunn- McCurdy certification decision required the Secretaries of Defense and Commerce and the Administrator of NASA to sign a revised memorandum of agreement by August 6, 2006. It also required that the program office, Program Executive Officer, and the Executive Committee revise and approve key acquisition documents including the acquisition strategy and system engineering plan by September 1, 2006, in order to proceed with the restructuring. Once these are completed, the program office can proceed to negotiate with its prime contractor on a new program baseline defining what will be delivered, by when, and at what cost.

The NPOESS program office has made progress in restructuring the acquisition. Specifically, the program office has established interim program plans guiding the contractor’s work activities in 2006 and 2007 and has made progress in implementing these plans. Specifically, the program office reported that it had completed 156 of 166 key milestones during fiscal year 2006—including completing ambient and thermal vacuum testing of the VIIRS engineering unit. Of the 10 remaining milestones resulting from unanticipated problems in the development of VIIRS and CrIS, 5 have since been completed, and 5 are still pending. The program office plans to complete 222 milestones in fiscal year 2007— including completing performance tests on the OMPS (nadir) sensor—and notes that they are slightly ahead of plans in that they have completed 62 milestones through January 20, 2007, which is 2 more than had been planned. Figures 7 and 8 depict the program office’s progress against key milestones in fiscal year 2006 and to date in fiscal year 2007.

The program office has also made progress in revising key acquisition documents. It revised the system engineering plan, the test and evaluation master plan, and the acquisition strategy plan, and obtained approval of these documents by the Program Executive Officer. The program office and contractor also developed an integrated master schedule for the remainder of the program—beyond fiscal year 2007. This integrated master schedule details the steps leading up to launching NPP by September 2009, launching the first NPOESS satellite in January 2013, and launching the second NPOESS satellite in January 2016. Near-term steps include completing and testing the VIIRS, CrIS, and OMPS sensors; integrating these sensors with the NPP spacecraft and completing integration testing; completing the data processing system and integrating it with the command, control, and communications segment; and performing advanced acceptance testing of the overall system of systems for NPP.

However, key steps remain for the acquisition restructuring to be completed. These steps include obtaining the approval of the Secretaries of Commerce and Defense and the Administrator of NASA on the memorandum of agreement among the three agencies, and obtaining the approval of the NPOESS Executive Committee on key acquisition documents, including the system engineering plan, the test and evaluation master plan, and the acquisition strategy. These approvals are currently over 6 months past due. Agency officials noted that the September 1, 2006, due date for the key acquisition documents was not realistic given the complexity of coordinating documents among three different agencies, but did not provide a new estimate for when these documents would be approved.

Finalizing these documents is critical to ensuring interagency agreements and will allow the program office to move forward in completing other activities related to restructuring the program. These activities include conducting an integrated baseline review with the contractor to reach agreement on the schedule and work activities, and finalizing changes to the NPOESS development and production contract—thereby allowing the program office to lock down a new acquisition baseline cost and schedule. The program office expects to conduct an integrated baseline review by May 2007 and to finalize the contract changes by July 2007. Until key acquisition documents are finalized and approved, the program faces increased risk that it will not be able to complete important restructuring activities in time to move forward in fiscal year 2008 with a new program baseline in place. This places the NPOESS program at risk of continued delays and future cost increases.

Progress Has Been Made in Establishing an Effective NPOESS Management Structure, but Executive Turnover Will Increase Risks, and Staffing Problems Remain

The NPOESS program has made progress in establishing an effective management structure, but—almost a year after this structure was endorsed during the Nunn-McCurdy certification process—the Integrated Program Office still faces staffing problems. Over the past few years, we and others have raised concerns about management problems at all levels of the NPOESS program, including subcontractor and contractor management, program office management, and executive-level management. Two independent review teams also noted a shortage of skilled program staff, including budget analysts and system engineers. Since that time, the NPOESS program has made progress in establishing an effective management structure—including establishing a new organizational framework with increased oversight by program executives, instituting more frequent subcontractor, contractor, and program reviews, and effectively managing risks and performance. However, DOD’s plans for reassigning the Program Executive Officer in Summer 2007 increase the program’s risks. Additionally, the program lacks a staffing process that clearly identifies staffing needs, gaps, and plans for filling those gaps. As a result, the program office has experienced delays in getting core management activities under way and lacks the staff it needs to execute day-to-day management activities.

NPOESS Program Has Made Progress in Establishing an Effective Management Structure and Increasing Oversight Activities, but Executive Turnover Will Increase Program Risks

The NPOESS program has made progress in establishing an effective management structure and increasing the frequency and intensity of its oversight activities. Over the past few years, we and others have raised concerns about management problems at all levels of management on the NPOESS program, including subcontractor and contractor management, program office management, and executive-level management. In response to recommendations made by two different independent review teams, the program office began exploring options in late 2005 and early 2006 for revising its management structure.

In November 2005, the Executive Committee established and filled a Program Executive Officer position, senior to the NPOESS Program Director, to streamline decision making and to provide oversight to the program. This Program Executive Officer reports directly to the Executive Committee. Subsequently, the Program Executive Officer and the Program Director proposed a revised organizational framework that realigned division managers within the Integrated Program Office responsible for overseeing key elements of the acquisition and increased staffing in key areas. In June 2006, the Nunn-McCurdy certification decision approved this new management structure and the Integrated Program Office implemented it. Figure 9 provides an overview of the relationships among the Integrated Program Office, the Program Executive Office, and the Executive Committee, as well as key divisions within the program office.

Operating under this new management structure, the program office implemented more rigorous and frequent subcontractor, contractor, and program reviews, improved visibility into risk management and mitigation activities, and institutionalized the use of earned value management techniques to monitor contractor performance. Specifically, program officials and the prime contractor now review the subcontractors’ cost and schedule performance on a weekly basis. The information from these meetings feeds into monthly government meetings with the prime contractor to review progress against milestones, issues, and risks. Further, the Program Director conducts monthly reviews with each technical division lead to review the divisions’ achievements, risks, and plans. Program officials note that these frequent reviews allow information on risks to be quickly escalated from subcontractors to contractors, to the program component level, and to the Program Director—and they allow program officials to better manage efforts to reduce risks. The program office also reported that all division leads were trained in earned value management techniques and were effectively using these techniques both to monitor subcontractor and contractor performance on a weekly basis and to identify potential problems as soon as possible.

In addition to these program office activities, the Program Executive Officer implemented monthly program reviews and increased the frequency of contacts with the Executive Committee. Specifically, the Program Executive Officer holds monthly program management reviews where the Program Director and program division leads (for example, those in charge of systems engineering or ground systems) provide briefings on the program’s earned value, progress, risks, and concerns. We observed that these briefings allow the Program Executive Officer to have direct insight into the challenges and workings of the Integrated Program Office and allow risks to be appropriately escalated and addressed. These meetings also provide an open forum for managers to raise concerns and ask questions about operational challenges. For example, when NASA officials expressed concerns that vibration levels used during testing were higher than necessary and were causing damage to key sensor components, the Program Director and Program Executive Officer immediately established a forum to discuss and mitigate this issue. The Program Executive Officer briefs the Executive Committee in monthly letters, apprising committee members of the program’s status, progress, risks, and earned value and the Executive Committee now meets on a quarterly basis—whereas in the recent past, we reported that the Executive Committee had met only five times in 2 years.

While the NPOESS program has made progress in establishing an effective management structure, this progress is currently at risk. We recently reported that DOD space acquisitions are at increased risk due in part to frequent turnover in leadership positions, and we suggested that addressing this will require DOD to consider matching officials’ tenure with the development or delivery of a product. In March 2007, NPOESS program officials stated that DOD is planning to reassign the recently appointed Program Executive Officer in Summer 2007 as part of this executive’s natural career progression. As of March 2007, the Program Executive Officer has held this position for 16 months. Given that the program is currently still being restructured, and that there are significant challenges in being able to meet critical deadlines to ensure satellite data continuity, such a move adds unnecessary risk to an already risky program.

NPOESS Program Has Filled Key Vacancies but Lacks A Programwide Staffing Process

The NPOESS program office has filled key vacancies in recent months but lacks a staffing process that identifies programwide staffing requirements and plans for filling those needed positions. Sound human capital management calls for establishing a process or plan for determining staffing requirements, identifying any gaps in staffing, and planning to fill critical staffing gaps. Program office staffing is especially important for NPOESS, given the acknowledgment by multiple independent review teams that staffing shortfalls contributed to past problems. Specifically, these review teams noted shortages in the number of system engineers needed to provide adequate oversight of subcontractor and contractor engineering activities and in the number of budget and cost analysts needed to assess contractor cost and earned value reports. To rectify this situation, the June 2006 certification decision directed the Program Director to take immediate actions to fill vacant positions at the program office with the approval of the Program Executive Officer.

Since the June 2006 decision to revise NPOESS management structure, the program office has filled multiple critical positions, including a budget officer, a chief system engineer, an algorithm division chief, and a contracts director. In addition, on an ad hoc basis, individual division managers have assessed their needs and initiated plans to hire individuals for key positions. However, almost a year after the certification, the program office still lacks a programwide process for identifying and filling all needed positions. As a result, division managers often wait months for critical positions to be filled. For example, in February 2006, the NPOESS program estimated that it needed to hire up to 10 new budget analysts. As of September 2006, none of these positions had been filled. Today, program officials estimate that they only needed to fill 7 budget analyst positions, of which 2 positions have been filled, and 5 remain vacant. Additionally, even though the certification decision directed immediate action to fill critical vacancies, the program still has vacancies in 5 systems engineering positions and 10 technical manager positions. The majority of the vacancies—4 of the 5 budget positions, 4 of the 5 systems engineering positions, and 8 of the 10 technical manager positions—are to be provided by NOAA. NOAA officials noted that each of these positions is in some stage of being filled—that is, recruitment packages are being developed or reviewed, vacancies are being advertised, or candidates are being interviewed, selected, and approved.

The program office attributes its staffing delays to not having the right personnel in place to facilitate this process—and did not even begin to develop a staffing process—until November 2006. Program officials noted that the tri-agency nature of the program adds unusual layers of complexity to the hiring and administrative functions because each agency has its own hiring and performance management rules. In November 2006, the program office brought in an administrative officer who took the lead in pulling together the division managers’ individual assessments of needed staff—currently estimated to be 25 vacant positions—and has been working with the division managers to refine this list. This new administrative officer plans to train division managers in how to assess their needs and to hire needed staff and to develop a process by which evolving needs are identified and positions are filled. However, there is as yet no date set for establishing this basic programwide staffing process.

As a result of the lack of a programwide staffing process, there has been an extended delay in determining what staff are needed and in bringing those staff on board—which has resulted in delays in performing core management activities such as establishing the program office’s cost estimate and bringing in needed contracting expertise. Additionally, until a programwide staffing process is in place, the program office risks not having the staff it needs to execute day-to-day management activities.

Methodology Supporting the June 2006 Cost and Schedule Estimate Was Reliable, but Recent Events Could Increase Program Costs

In June 2006, DOD certified a restructured NPOESS program that was estimated to cost $11.5 billion for the acquisition portion of the program and scheduled to launch the first satellite in 2013. The Office of the Secretary of Defense’s Cost Analysis Improvement Group (cost analysis group)—the independent cost estimators charged with developing the estimate for the acquisition portion of the program—used an acceptable methodology to develop this estimate. When combined with an estimated $1 billion for operations and support after launch, this brings the program life cycle cost to $12.5 billion. Recent events, however, could further increase program costs or delay schedules. Specifically, the program continues to experience technical problems on key sensors, and costs and schedules will be adjusted during negotiations on contract changes. The NPOESS program office is developing its own cost estimate to refine the one developed in June 2006 that it will use to negotiate contract changes. A new baseline cost will be established once the contract is finalized.

Certified Program Estimates Were Developed Using an Acceptable Methodology

The cost and schedule estimate for the restructured NPOESS program was developed by DOD’s cost analysis group using an acceptable methodology. Cost-estimating organizations throughout the federal government and industry use certain key practices—related to planning, conducting, and reporting the estimate—to ensure a sound estimate. Table 6 lists the elements of a sound cost estimating methodology. In addition, to ensure the validity of the data assumptions that go into the estimate, leading organizations use actual historical costs and seek an independent validation of critical cost drivers.

DOD’s cost analysis group used an acceptable methodology in developing the NPOESS cost estimate in that they planned, conducted, and reported the estimate consistent with leading practices. The cost analysis group’s cost estimating approach was largely driven by the program’s principal “ground rule” to maintain the continuity of weather data without a gap. Specifically, the cost analysis group assessed two risks: (1) the uncertainty of the health of the current polar-satellite constellation and (2) the uncertainty of when the new satellite system could be delivered (including the time needed to evaluate new satellites once in orbit). The resulting analysis showed that the restructured NPOESS system could be delivered and the first satellite launched by 2013 with a high level of confidence in maintaining satellite data continuity.

To determine specific costs, the group used the existing work breakdown structure employed by the program office as the basis for performing its work. This work breakdown structure consists of seven major elements, including ground systems; spacecraft; sensors; assembly, integration and test; system engineering/contractor program management; government program management; and launch.

The cost analysis group also took steps to ensure the validity of the data that went into the estimate. For each element, the cost analysis group visited all major contractor sites to collect program data including schedule (including the original, rebaselined, and current schedules, and risks affecting the current schedule); current staffing profile by month; the history of staffing used; the qualifications of people charging the program; the program’s technical approaches; the contractor’s program legacy (a justification that the contractor has worked on similar projects in the past and that the contractor should be able to adapt that knowledge to the current work).

The cost analysis group also compared this data with contractor labor rates from the Defense Contract Management Agency and obtained NASA’s validation of the costs associated with the most significant cost driver, the VIIRS sensor.

Since schedule was the primary uncertainty factor in the cost analysis, it also was the driver of overall costs. Specifically, the cost analysis group took its risk-adjusted schedule durations for the major cost elements and adjusted the contractor-submitted manning profiles accordingly. They then used NPOESS historical data on labor rates and materials to calculate the cost of these elements.

Consistent with DOD practice, the cost analysis group established its cost estimate at a 50 percent confidence level. However, cost analysts could not provide an upper limit for potential cost growth, explaining that the program contains “failsafe” measures to use alternative technologies (such as using legacy systems) if schedules are delayed and costs increase. As a result, cost analysts reported that they have a high level of confidence that acquisition costs will not exceed $11.5 billion—but a lower level of confidence that the configuration of sensors will remain unchanged.

Recent Events Could Lead to Increased Program Costs or Delayed Schedules

While the June 2006 cost estimate for the acquisition portion of the program was reasonable at the time it was made, several recent events could cause program life cycle costs to grow or schedules to be delayed. Specifically, the program continues to experience technical problems on key sensors. The CrIS sensor being developed for the NPP satellite suffered a major structural failure in October 2006. A failure review board is currently working to resolve the root causes of the failure. While program officials note that they should be able to cover costs related to investigating the problem, the full cost and schedule to fix the sensor is not yet known. Also, VIIRS development, which has been the program’s primary cost driver, is not yet complete and continues to be a high-risk development. This too, could lead to higher final program costs or delayed schedules.

Program costs are also likely to be adjusted during upcoming negotiations on contract changes. The NPOESS program office is developing its own cost estimate to refine the one developed in June 2006. Program officials plan to use this revised cost estimate to negotiate contract changes. A new baseline cost will be established once the contract is finalized—an event that the Program Director expects to occur by July 2007.

Major Program Segments Are Under Development, but Significant Risks Remain

Major segments of the NPOESS program—the space segment, the ground systems segment, and the launch segment—are under development; however, significant problems have occurred and risks remain. The program office is aware of these risks and is working to mitigate them, but continued problems could affect the program’s overall cost and schedule. Given the tight time frames for completing key sensors, integrating them on the NPP spacecraft, and getting the ground-based data processing systems developed, tested, and deployed, it will be important for the NPOESS Integrated Program Office, the Program Executive Office, and the Executive Committee to continue to provide close oversight of milestones and risks.

Space Segment—Progress Made, but Key Sensors Continue to Face Major Risks

The space segment includes the sensors and the spacecraft. Four sensors are of critical importance—VIIRS, CrIS, OMPS, and ATMS—because they are to be launched on the NPP satellite. Initiating work on another sensor, the Microwave imager/sounder, is also important because this new sensor—replacing the cancelled CMIS sensor—will need to be developed in time for the second NPOESS satellite launch. Over the past year, the program made progress on each of the sensors and the spacecraft. However, two sensors, VIIRS and CrIS, have experienced major problems. The status of each of the components of the space segment is described in table 7.

Earned value management tools are used to compare the value of work accomplished with the work expected during a given time period, and any differences are measured in cost and schedule variances. The NPOESS space segment experienced negative cost and schedule variances between January 2006 and January 2007 (see fig. 10).

From January 2006 to January 2007, the contractor exceeded cost targets for the space segment by $17 million—which is 4 percent of the space segment budget for that time period. Similarly, the contractor was unable to complete $14.6 million worth of work in the space segment. The main factors in the cost and schedule variances were due to underestimation of the scope of work, pulling resources from lower priority tasks to higher priority items, and unforeseen design issues on key sensors. For example, VIIRS continued to experience negative cost variance trends due to unplanned efforts, which included refurbishing and recertifying the VIIRS calibration chamber, completing the testing of the engineering design unit, and resolving a problem with the testing equipment needed to adjust VIIRS’ temperature during a key test.

Unplanned efforts for CrIS that attributed to the negative cost and schedule variances included additional time required for testing and material management. The schedule variances for VIIRS and CrIS were mainly due to resources being pulled from other areas to support higher priority tasks, extended testing and testing delays, management changes, and improper material handling. Further, there is a high likelihood that CrIS will continue to experience cost and schedule variances against the fiscal year 2007 interim program plan until the issues that caused its structural failure are addressed.

Program Office Is Monitoring Sensor Risks and Evaluating Options

Program officials regularly track risks associated with various NPOESS components and work to mitigate them. Having identified both VIIRS and CrIS as high risk, OMPS as a moderate risk, and the other components as low risk, the program office is working closely with the contractors and subcontractors to resolve sensor problems.

Program officials have identified work-arounds that will allow them to move forward in testing the VIIRS engineering unit and have approved the flight unit to proceed to a technical readiness review milestone in May 2007. Regarding CrIS, as of March 2007, a failure review board identified root causes of its structural failure, identified plans for resolving them, and initiated inspections of sensor modules and subsystems for damage. An agency official reported that there is sufficient funding in the fiscal year 2007 program office’s and contractor’s management reserve funds to allow for troubleshooting both VIIRS and CrIS problems. However, until the CrIS failure review board fully determines the amount of rework that is necessary to fix the problems, it is unknown if additional funds will be needed or if the time frame for CrIS’ delivery will be delayed. According to agency officials, CrIS is not on the program schedule’s critical path, and there is sufficient schedule margin to absorb the time it will take to conduct a thorough failure review process.

Managing the risks associated with the development of VIIRS and CrIS are of particular importance because these are to be demonstrated on the NPP satellite currently scheduled for launch in September 2009. Additionally, any delay in the NPP launch date could affect the overall NPOESS program because the success of the program depends on the lessons learned in data processing and system integration from the NPP satellite.

Ground Segment— Progress Has Been Made, but Work Remains

Development of the ground segment—which includes the interface data processing system, the ground stations that are to receive satellite data, and the ground-based command, control, and communications system—is under way and on track. However, important work pertaining to developing the algorithms that translate satellite data into weather products within the integrated data processing segment remains to be completed. Table 8 describes each of the components of the ground segment and identifies the status of each. Additionally, appendix II provides an overview of satellite data processing algorithms.

Ground Segment Cost and Schedule Are on Track; Work and Risks Remain

Using contractor-provided data, our analysis indicates cost and schedule performance on key elements of the NPOESS ground segment were generally on track or positive against the fiscal year 2006 and 2007 interim program plans. For the IDPS component, the contractor completed slightly less work than planned and finished slightly under budget. This caused cost and schedule variances of less than 1 percent off of expectations. (see fig. 11). For the command, control, and communications component, the contractor was able to outperform its planned targets by finishing under budget by $3 million (6.2 percent of the budget for this time period) and by completing $31,000 (less than 1 percent) worth of work beyond what was planned (see fig. 12).

The NPOESS program office plans to continue to address risks facing IDPS development. Specifically, the IDPS team is working to reduce data processing delays by seeking to limit the number of data calls, improve the efficiency of the data management system, increase the efficiency of the algorithms, and increase the number of processors. The program office also developed a resource center consisting of a logical technical library, a data archive, and a set of analytical tools to coordinate, communicate, and facilitate the work of algorithm subject matter experts on algorithm development and calibration/validation preparations. Managing the risks associated with the development of the IDPS system is of particular importance because this system will be needed to process NPP data.

Launch Segment—NPP Launch Preparation Has Begun, while NPOESS Launch Planning Remains a Future Event

Different agencies are responsible for launching NPP and NPOESS. NASA is responsible for the NPP launch and began procuring the launch vehicle for NPP in August 2006. Program officials expect to have it delivered by July 2009, less than 2 months before the scheduled NPP launch in September 2009.

The NPOESS Integrated Program Office is responsible for launching the NPOESS satellites. According to program officials, the Air Force is to procure launch services for the program through DOD’s Evolved Expendable Launch Vehicle contract. These services are to be procured by January 2011, 2 years before the first scheduled launch.

Conclusions

NPOESS restructuring is well under way, and the program has made progress in establishing an effective management structure. However, key steps remain in restructuring the acquisition, including completing important acquisition documents such as the system engineering plan, the acquisition program baseline, and the memorandum of agreement documenting the three agencies’ roles and responsibilities. Until these key documents are finalized, the program is unable to finalize plans for restructuring the program. Additionally, the program office continues to have difficulty filling key positions and lacks a programwide staffing process. Until the program establishes an effective and repeatable staffing process, it will have difficulties in identifying and filling its staffing needs in a timely manner. Having insufficient staff in key positions impedes the program office’s ability to conduct important management and oversight activities, including revising cost and schedule estimates, monitoring progress, and managing technical risks. The program faces even further challenges if DOD proceeds with plans to reassign the Program Executive Officer this summer. Such a move would add unnecessary risk to an already risky program.

In addition, the likelihood exists that there will be further cost increases and schedule delays because of technical problems on key sensors and pending contract negotiations. Major program segments—including the space and ground segments—are making progress in their development and testing. However, two critical sensors have experienced problems and are considered high risk, and risks remain in developing and implementing the ground-based data processing system. Given the tight time frames for completing key sensors, integrating them, and getting the ground-based data processing systems developed, tested, and deployed, continued close oversight of milestones and risks is essential to minimize potential cost increases and schedule delays.

Recommendations for Executive Action

Because of the importance of effectively managing the NPOESS program to ensure that there are no gaps in the continuity of critical weather and environmental observations, we are making recommendations to the Secretaries of Defense and Commerce and to the Administrator of NASA to ensure that the responsible executives within their respective organizations approve key acquisition documents, including the memorandum of agreement among the three agencies, the system engineering plan, the test and evaluation master plan, and the acquisition strategy, as quickly as possible but no later than April 30, 2007.

We are also recommending that the Secretary of Defense direct the Air Force to delay reassigning the recently appointed Program Executive Officer until all sensors have been delivered to the NPOESS Preparatory Program; these deliveries are currently scheduled to occur by July 2008.

We are also making two additional recommendations to the Secretary of Commerce. We recommend that the Secretary direct the Undersecretary of Commerce for Oceans and Atmosphere to ensure that NPOESS program authorities develop and implement a written process for identifying and addressing human capital needs and for streamlining how the program handles the three different agencies’ administrative procedures, and establish a plan for immediately filling needed positions.

Agency Comments and Our Evaluation

We received written comments on a draft of this report from the Deputy Secretary of the Department of Commerce (see app. III), the Deputy Assistant Secretary for Networks and Information Integration of the Department of Defense (see app. IV), and the Deputy Administrator of the National Aeronautics and Space Administration (see app. V). All three agencies agreed that it was important to finalize key acquisition documents in a timely manner, and DOD proposed extending the due dates for the documents to July 2, 2007. Because the NPOESS program office intends to complete contract negotiations by July 4, 2007, we remain concerned that any further delays in approving the documents could delay contract negotiations and thus increase the risk to the program.

In addition, the Department of Commerce agreed with our recommendation to develop and implement a written process for identifying and addressing human capital needs and to streamline how the program handles the three different agencies’ administrative procedures. The department also agreed with our recommendation to plan to immediately fill open positions at the NPOESS program office. Commerce noted that NOAA identified the skill sets needed for the program and has implemented an accelerated hiring model and schedule to fill all NOAA positions in the NPOESS program. The department also stated that the Program Director will begin presenting the detailed staffing information at monthly program management reviews, including identifying any barriers and recommended corrective actions. Commerce also noted that NOAA has made NPOESS hiring a high priority and has documented a strategy— including milestones—to ensure that all 20 needed positions are filled by June 2007.

DOD did not concur with our recommendation to delay reassigning the Program Executive Officer, noting that the NPOESS System Program Director responsible for executing the acquisition program would remain in place for 4 years. The Department of Commerce also noted that the Program Executive Officer position is planned to rotate between the Air Force and NOAA. Commerce also stated that a selection would be made prior to the departure of the current Program Executive Officer to provide an overlap period to allow for knowledge transfer and ensure continuity.

However, over the last few years, we and others (including an independent review team and the Commerce Inspector General) have reported that ineffective executive-level oversight helped foster the NPOESS program’s cost and schedule overruns. We remain concerned that reassigning the Program Executive at a time when NPOESS is still facing critical cost, schedule, and technical challenges will place the program at further risk.

While it is important that the System Program Director remain in place to ensure continuity in executing the acquisition, this position does not ensure continuity in the functions of the Program Executive Officer. The current Program Executive Officer is experienced in providing oversight of the progress, issues, and challenges facing NPOESS and coordinating with Executive Committee members, as well as DOD authorities responsible for executing Nunn-McCurdy requirements. Additionally, while the Program Executive Officer position is planned to rotate between agencies, the memorandum of agreement documenting this arrangement is still in draft and should be flexible enough to allow the current Program Executive Officer to remain until critical risks have been addressed.

Further, while Commerce plans to allow a period of overlap between the selection of a new Program Executive Officer and the departure of the current one, time is running out. The current Program Executive Officer is expected to depart in early July 2007 and, as of mid-April 2007, a successor has not yet been named. NPOESS is an extremely complex acquisition, involving three agencies, multiple contractors, and advanced technologies.

There is not sufficient time to transfer knowledge and develop the sound professional working relationships that the new Program Executive Officer will need to succeed in that role. Thus, we remain convinced that given NPOESS’s current challenges, reassigning the current Program Executive Officer at this time would not be appropriate.

All three agencies also provided technical comments, which we have incorporated in this report as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to interested congressional committees, the Secretary of Commerce, the Secretary of Defense, the Administrator of NASA, the Director of the Office of Management and Budget, and other interested parties. In addition, this report will be available at no charge on our Web site at http://www.gao.gov.

If you have any questions on matters discussed in this report, please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) evaluate the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program office’s progress in restructuring the acquisition; (2) evaluate the program office’s progress in establishing an effective management structure; (3) assess the reliability of the new life cycle cost estimate and proposed schedule; and (4) identify the status and key risks facing the program’s major segments (the launch, space, data processing, and ground control segments) and evaluate the adequacy of the program’s efforts to mitigate these risks.

To evaluate the NPOESS program office’s progress in restructuring the acquisition program, we reviewed the program’s Nunn-McCurdy certification decision memo and program documentation including status briefings and milestone progress reports. We also interviewed program office officials and attended conferences and senior-level management program review meetings to obtain information on the program’s acquisition restructuring.

To evaluate the program office’s progress in establishing an effective management structure, we reviewed the Nunn-McCurdy decision memo for the program, as well as program documentation and briefings. We assessed the status of efforts to implement recommendations regarding the program’s management structure, including the work of the team responsible for reviewing the management structure under the Nunn- McCurdy review. We also analyzed the program office’s organizational charts and position vacancies. Finally, we interviewed officials responsible for reviewing the management structure of the program under Nunn- McCurdy, attended senior-level management review meetings to obtain information related to the program’s progress in establishing and staffing the new management structure, and interviewed program office officials responsible for human capital issues to obtain clarification on plans and goals for the new management structure.

To assess the reliability of the new life cycle cost estimate and proposed schedule, we analyzed the Office of the Secretary of Defense’s Cost Analysis Improvement Group’s (cost analysis group) cost estimating methodology and the assumptions used to develop its independent cost estimate. Specifically, we assessed the cost estimating group’s methodology against 12 best practices recognized by cost-estimating organizations within the federal government and industry for the development of reliable cost estimates. These best practices are also contained in a draft version of our cost guide, which is currently being developed by GAO cost experts. We also assessed cost- and schedule- related data, including the work breakdown structure and detailed schedule risk analyses to determine the reasonableness of the cost analysis group’s assumptions. We also interviewed cost analysis group officials to obtain clarification on cost and schedule estimates and their underlying assumptions. Further, we interviewed program officials to identify any assumptions that may have changed.

To identify the status and key risks facing the program’s major segments (the launch, space, data processing, and ground control segments) and to evaluate the adequacy of the program’s efforts to mitigate these risks, we reviewed the program’s Nunn-McCurdy certification decision memo and other program documentation. We analyzed briefings and monthly program management documents to determine the status and risks of the key program segments. We also analyzed earned value management data obtained from the contractor to assess the contractor’s performance to cost and schedule. We reviewed cost reports and program risk management documents and interviewed program officials to determine the program segments’ risks that could negatively affect the program’s ability to maintain the current schedule and cost estimates. We also interviewed agency officials from the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA), the Department of Defense (DOD), and the NPOESS program office to determine the status and risks of the key program segments. Finally, we observed senior-level management review meetings and attended conferences to obtain information on the status of the NPOESS program.

We performed our work at the NPOESS Integrated Program Office and at DOD, NASA, and NOAA offices in the Washington, D.C., metropolitan area between July 2006 and April 2007 in accordance with generally accepted government auditing standards.

Appendix II: Overview of Satellite Data Processing Algorithms and the Calibration and Validation Process

Algorithms are sets of instructions, expressed mathematically, that translate satellite sensor measurements into usable information. In the NPOESS program, government contractors are responsible for algorithm development; the program office is responsible for independently validating the algorithms. Scientists develop these algorithms, which are then written as computer code to be incorporated into the interface data processing system (IDPS) operational system. The NPOESS ground system uses three primary types of algorithms: Algorithms to develop raw data records “unpack” the digital packets received by the antennas/IDPS (the ones and zeros) and sent from the satellite, associate the data with the information about the satellite’s location and, finally, translate it back into the data it was when it started at the sensor.

Algorithms used to develop sensor and temperature data records allow the on-ground users to understand what the sensor saw. It translates the information from the sensor into a measure of the various forms of energy (e.g., brightness, temperature, radiance).

Algorithms used to produce the weather products called environmental data records (EDR) are crosscutting. They combine various data records, as well as other data, in order to produce measures useful to scientists. Additionally, EDRs can be “chained”—that is, the output of one EDR algorithm will become an input into the next EDR algorithm. To illustrate, cloud detection/mask is an important “base” EDR because many EDRs, like sea surface temperature, are only calculated when clouds are not present. Figure 13 shows the flow of the data and algorithms.

A corollary to algorithm development is the calibration and validation process. According to a senior algorithm scientist, in this process, once the satellite has been launched, scientists verify that the sensors accurately report what ground conditions are. For example, one EDR from the visible/infrared imager radiometer suite (VIIRS) is “ocean color.” Once the sensor is in orbit, scientists can compare the results that the VIIRS sensor reports on ocean color with the known results from sensors on ocean buoys that also measure ocean color in select locations. Then, if the sensors do not accurately report the ground conditions, scientists can calibrate, or “tweak,” the algorithms used to develop sensor, temperature, and environmental data records to report on ground conditions more accurately.

According to an agency official, fully calibrating a simple sensor once it has been launched can take approximately a year. A more complicated sensor can take 18 months to 2 years (see fig. 14).

Appendix III: Comments from the Department of Commerce

Appendix IV: Comments from the Department of Defense

Appendix V: Comments from the National Aeronautics and Space Administration

Appendix VI: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact named above, Colleen Phillips, Assistant Director; Carol Cha; Neil Doherty; Nancy Glover; Kathleen S. Lovett; Karen Richey; and Teresa Smith made key contributions to this report.