Polar-Orbiting Operational Environmental Satellites: Restructuring Is Under Way, but Technical Challenges and Risks Remain
Published: 2007-04-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Polar-orbiting Operational Environmental Satellite System (NPOESS) is a tri-agency acquisition--managed by the Departments of Commerce and Defense and the National Aeronautics and Space Administration--which experienced escalating costs, schedule delays, and technical difficulties. These factors led to a June 2006 decision to restructure the program thereby decreasing the program's complexity, increasing its estimated cost to $12.5 billion, and delaying the first two satellites by 3 to 5 years. GAO was asked to (1) assess progress in restructuring the acquisition, (2) evaluate progress in establishing an effective management structure, (3) assess the reliability of the cost and schedule estimate, and (4) identify the status and key risks facing the program's major segments. To do so, GAO analyzed program and contractor data, attended program reviews, and interviewed program officials
section_title: What GAO Found
paragraphs: The NPOESS program office has made progress in restructuring the acquisition by establishing and implementing interim program plans guiding the contractors' work activities in 2006 and 2007; however, important tasks leading up to finalizing contract changes remain to be completed. Executive approvals of key acquisition documents are about 6 months late--due in part to the complexity of navigating three agencies' approval processes. Delays in finalizing these documents could hinder plans to complete contract negotiations by July 2007 and could keep the program from moving forward in fiscal year 2008 with a new program baseline. The program office has also made progress in establishing an effective management structure by adopting a new organizational framework with increased oversight from program executives and by instituting more frequent and rigorous program reviews; however, plans to reassign the recently appointed Program Executive Officer will likely increase the program's risks. Additionally, the program lacks a process and plan for identifying and filling staffing shortages, which has led to delays in key activities such as cost estimating and contract revisions. Until this process is in place the NPOESS program faces increased risk of further delays. The methodology supporting a June 2006 independent cost estimate with the expectation of initial satellite launch in January 2013 was reliable, but recent events could increase program costs and delay schedules. Specifically, the program continues to experience technical problems on key sensors and program costs will likely be adjusted during upcoming negotiations on contract changes. A new baseline cost and schedule reflecting these factors is expected by July 2007. Development and testing of major NPOESS segments--including key sensors and ground systems--are under way, but significant risks remain. For example, while work continues on key sensors, two of them experienced significant problems and are considered high risk. Additionally, while progress has been made in reducing delays in the data processing system, work remains in refining the algorithms needed to translate sensor observations into useable weather products. Given the tight time frames for completing this work, it will be important for program officials and executives to continue to provide close oversight of milestones and risks.

section_title: Letter
paragraphs: The planned National Polar-orbiting Operational Environmental Satellite  The planned National Polar-orbiting Operational Environmental Satellite  System (NPOESS) program is expected to be a state-of-the-art,  System (NPOESS) program is expected to be a state-of-the-art,  environment-monitoring satellite system that will replace two existing  environment-monitoring satellite system that will replace two existing  polar-orbiting environmental satellite systems. Polar-orbiting satellites  polar-orbiting environmental satellite systems. Polar-orbiting satellites  provide data and imagery that are used by weather forecasters,  provide data and imagery that are used by weather forecasters,  climatologists, and the military to map and monitor changes in weather,  climatologists, and the military to map and monitor changes in weather,  climate, the oceans, and the environment. The NPOESS program is  climate, the oceans, and the environment. The NPOESS program is  considered critical to the United States’ ability to maintain the continuity  considered critical to the United States’ ability to maintain the continuity  of data required for weather forecasting (including severe weather events  of data required for weather forecasting (including severe weather events  such as hurricanes) and global climate monitoring through the year 2026.  such as hurricanes) and global climate monitoring through the year 2026.
Three agencies share responsibility for the NPOESS acquisition: the  Three agencies share responsibility for the NPOESS acquisition: the  Department of Commerce’s National Oceanic and Atmospheric  Department of Commerce’s National Oceanic and Atmospheric  Administration (NOAA), the Department of Defense (DOD)/United States  Administration (NOAA), the Department of Defense (DOD)/United States  Air Force, and the National Aeronautics and Space Administration  Air Force, and the National Aeronautics and Space Administration  (NASA). To manage the NPOESS program, these agencies established a tri- (NASA). To manage the NPOESS program, these agencies established a tri- agency integrated program office. In recent years, this program has  agency integrated program office. In recent years, this program has  experienced escalating costs, schedule delays, and technical difficulties,  experienced escalating costs, schedule delays, and technical difficulties,  leading to a June 2006 decision to restructure the program. This decision  leading to a June 2006 decision to restructure the program. This decision  decreased the complexity of the program by reducing the number of  decreased the complexity of the program by reducing the number of  satellites and sensors, increased the estimated cost of the program to $12.5  satellites and sensors, increased the estimated cost of the program to $12.5  billion, and delayed the launches of the first two satellites by 3 to 5 years.  billion, and delayed the launches of the first two satellites by 3 to 5 years.
This report responds to your request that we (1) evaluate the NPOESS  This report responds to your request that we (1) evaluate the NPOESS  program office’s progress in restructuring the acquisition, (2) evaluate the  program office’s progress in restructuring the acquisition, (2) evaluate the  program office’s progress in establishing an effective management  structure, (3) assess the reliability of the life cycle cost estimate and  proposed schedule, and (4) identify the status and key risks facing the  program’s major segments and evaluate the adequacy of the program’s  efforts to mitigate these risks.
To evaluate the program office’s progress in restructuring the acquisition,  we assessed program documentation, attended management status  briefings, and interviewed program officials. To determine progress in  establishing a new management structure, we assessed the status of  efforts to implement past recommendations regarding the management  structure and staffing, attended senior-level management review meetings,  reviewed program documents, and interviewed program officials. To  assess the cost estimate, we evaluated the methodology and assumptions  used to develop the estimate and interviewed program officials to identify  any assumptions that may have changed. To determine the status, risk, and  risk mitigation efforts for the program, we analyzed monthly program  management documents and interviewed NOAA, NASA, and DOD officials  to determine concerns with these mitigation efforts. In addition, this  report builds on other work we have done on environmental satellite  programs over the last several years.
We conducted our work at the NPOESS Integrated Program Office  headquarters and at DOD, NOAA, and NASA facilities in the Washington,  D.C., metropolitan area. We performed our work from July 2006 to April  2007 in accordance with generally accepted government auditing  standards. Appendix I contains additional details on our objectives, scope,  and methodology.
subsections: section_title: Background
paragraphs: Since the 1960s, the United States has operated two separate operational  polar-orbiting meteorological satellite systems: the Polar-orbiting  Operational Environmental Satellite (POES) series—managed by NOAA,  and the Defense Meteorological Satellite Program (DMSP)—managed by  the Air Force. These satellites obtain environmental data that are  processed to provide graphical weather images and specialized weather  products. These satellite data are also the predominant input to numerical  weather prediction models, which are a primary tool for forecasting  weather 3 or more days in advance—including forecasting the path and  intensity of hurricanes. The weather products and models are used to  predict the potential impact of severe weather so that communities and  emergency managers can help prevent and mitigate their effects. Polar  satellites also provide data used to monitor environmental phenomena,  such as ozone depletion and drought conditions, as well as data sets that  are used by researchers for a variety of studies such as climate monitoring.
Unlike geostationary satellites, which maintain a fixed position relative to  the earth, polar-orbiting satellites constantly circle the earth in an almost  north-south orbit, providing global coverage of conditions that affect the  weather and climate. Each satellite makes about 14 orbits a day. As the  earth rotates beneath it, each satellite views the entire earth’s surface  twice a day. Currently, there are two operational POES satellites and two  operational DMSP satellites that are positioned so that they can observe  the earth in early morning, midmorning, and early afternoon polar orbits.  Together, they ensure that, for any region of the earth, the data provided  to users are generally no more than 6 hours old. Figure 1 illustrates the  current operational polar satellite configuration. Besides the four  operational satellites, six older satellites are in orbit that still collect some  data and are available to provide limited backup to the operational  satellites should they degrade or fail. In the future, both NOAA and the Air  Force plan to continue to launch additional POES and DMSP satellites  every few years, with final launches scheduled for 2009 and 2012,  respectively.
Each of the polar satellites carries a suite of sensors designed to detect  environmental data that are either reflected or emitted from the earth, the  atmosphere, and space. The satellites broadcast a subset of these data in  real time to properly equipped field terminals that are within a direct line  of sight; these field terminals are located at universities, on battlefields, or  on ships. Additionally, the polar satellites store the observed  environmental data and then transmit them to NOAA and Air Force ground  stations when the satellites pass overhead. The ground stations then relay  the data via communications satellites to the appropriate meteorological  centers for processing.
Under a shared processing agreement among four satellite data processing  centers—NOAA’s National Environmental Satellite Data and Information  Service (NESDIS), the Air Force Weather Agency, the Navy’s Fleet  Numerical Meteorology and Oceanography Center, and the Naval  Oceanographic Office—different centers are responsible for producing  and distributing, via a shared network, different environmental data sets,  specialized weather and oceanographic products, and weather prediction  model outputs.
Each of the four processing centers is also responsible for distributing the  data to its respective users. For the DOD centers, the users include  regional meteorology and oceanography centers, as well as meteorology  and oceanography staff on military bases, the Naval Fleet, and mobile field  sites. NESDIS forwards the data to NOAA’s National Weather Service for  distribution and use by government and commercial forecasters. The  processing centers also use the Internet to distribute data to the general  public. NESDIS is responsible for the long-term archiving of data and  derived products from POES and DMSP. Figure 2 depicts a generic data  relay pattern from the polar-orbiting satellites to the data processing  centers and field terminals.
subsections: section_title: Polar Satellite Data and Products
paragraphs: Polar satellites gather a broad range of data that are transformed into a  variety of products. Satellite sensors observe different bands of radiation  wavelengths, called channels, which are used for remotely determining  information about the earth’s atmosphere, land surface, oceans, and the  space environment. When first received, satellite data are considered raw  data. To make them usable, the processing centers format the data so that  they are time-sequenced and include earth location and calibration  information. After formatting, these data are called raw data records. The  centers further process these raw data records into channel-specific data  sets, called sensor data records and temperature data records. These data  records are then used to derive weather and climate products called  environmental data records (EDR). EDRs include a wide range of  atmospheric products detailing cloud coverage, temperature, humidity,  and ozone distribution; land surface products showing snow cover,  vegetation, and land use; ocean products depicting sea surface  temperatures, sea ice, and wave height; and characterizations of the space  environment. Combinations of these data records (raw, sensor,  temperature, and environmental data records) are also used to derive  more sophisticated products, including outputs from numerical weather  models and assessments of climate trends. Figure 3 is a simplified  depiction of the various stages of satellite data processing, and figures 4  and 5 depict examples of EDR weather products.
section_title: NPOESS Overview
paragraphs: With the expectation that combining the POES and DMSP programs would  reduce duplication and result in sizable cost savings, a May 1994  Presidential Decision Directive required NOAA and DOD to converge the  two satellite programs into a single satellite program capable of satisfying  both civilian and military requirements. The converged program,  NPOESS, is considered critical to the United States’ ability to maintain the  continuity of data required for weather forecasting and global climate  monitoring through the year 2026. To manage this program, DOD, NOAA,  and NASA formed the tri-agency Integrated Program Office, located within  NOAA.
Within the program office, each agency has the lead on certain activities:  NOAA has overall program management responsibility for the converged  system and for satellite operations; DOD has the lead on the acquisition;  and NASA has primary responsibility for facilitating the development and  incorporation of new technologies into the converged system. NOAA and  DOD share the costs of funding NPOESS, while NASA funds specific  technology projects and studies. Figure 6 depicts the organizations that  make up the NPOESS program office and lists their responsibilities.
The NPOESS program office is overseen by an Executive Committee,  which is made up of the Administrators of NOAA and NASA and the  Undersecretary of the Air Force.
section_title: NPOESS Acquisition Strategy
paragraphs: NPOESS is a major system acquisition that was originally estimated to  cost about $6.5 billion over the 24-year life of the program from its  inception in 1995 through 2018. The program is to provide satellite  development, satellite launch and operation, and ground-based satellite  data processing. These deliverables are grouped into four main categories:  (1) the space segment, which includes the satellites and sensors; (2) the  integrated data processing segment, which is the system for transforming  raw data into EDRs and is to be located at the four processing centers; (3)  the command, control, and communications segment, which includes the  equipment and services needed to support satellite operations; and (4) the  launch segment, which includes the launch vehicle services.
When the NPOESS engineering, manufacturing, and development contract  was awarded in August 2002, the cost estimate was adjusted to $7 billion.  Acquisition plans called for the procurement and launch of six satellites  over the life of the program, as well as the integration of 13 instruments— consisting of 10 environmental sensors and three subsystems. Together,  the sensors were to receive and transmit data on atmospheric, cloud  cover, environmental, climatic, oceanographic, and solar-geophysical  observations. The subsystems were to support nonenvironmental search  and rescue efforts, sensor survivability, and environmental data collection  activities. The program office considered 4 of the sensors to be critical  because they provide data for key weather products; these sensors are in  bold in table 1, which describes each of the expected NPOESS  instruments.
In addition, NPP was planned as a demonstration satellite to be launched  several years before the first NPOESS satellite in order to reduce the risk  associated with launching new sensor technologies and to ensure  continuity of climate data with NASA’s Earth Observing System satellites.
NPP is to host three of the four critical NPOESS sensors (VIIRS, CrIS, and  ATMS), as well as one other noncritical sensor (OMPS). NPP is to provide  the program office and the processing centers an early opportunity to  work with the sensors, ground control, and data processing systems.
When the NPOESS development contract was awarded, the schedule for  launching the satellites was driven by a requirement that the satellites be  available to back up the final POES and DMSP satellites should anything  go wrong during the planned launches of these satellites. In general,  satellite experts anticipate that roughly 1 out of every 10 satellites will fail  either during launch or during early operations after launch.
Early program milestones included (1) launching NPP by May 2006, (2)  having the first NPOESS satellite available to back up the final POES  satellite launch in March 2008, and (3) having the second NPOESS satellite  available to back up the final DMSP satellite launch in October 2009. If the  NPOESS satellites were not needed to back up the final predecessor  satellites, their anticipated launch dates would have been April 2009 and  June 2011, respectively.
section_title: NPOESS Experienced Cost Increases, Schedule Delays, and Technical Problems over Several Years
paragraphs: Over the last few years, NPOESS has experienced continued cost  increases and schedule delays, requiring difficult decisions to be made  about the program’s direction and capabilities. In 2003, we reported that  changes in the NPOESS funding stream caused a delay in the program’s  schedule. Specifically, in late 2002, DOD shifted the expected launch date  for its final DMSP satellite from 2009 to 2010. As a result, the department  reduced funding for NPOESS by about $65 million between fiscal years  2004 and 2007. According to program officials, because NOAA was  required to provide the same level of funding that DOD provides, this  change triggered a corresponding reduction in funding by NOAA for those  years. As a result of the reduced funding, program officials were forced to  make difficult decisions about what to focus on first. The program office  decided to keep NPP as close to its original schedule as possible because  of its importance to the eventual NPOESS development and to shift some  of the program’s deliverables to later years. This shift affected the  NPOESS deployment schedule. To plan for this shift, the program office  developed a new program cost and schedule baseline.
After this new baseline was completed in 2004, we reported that the  program office increased the NPOESS cost estimate from about $7 billion  to $8.1 billion; delayed key milestones, including the planned launch of the  first NPOESS satellite—which was delayed by 7 months; and extended the  life of the program from 2018 to 2020. The cost increases reflected  changes to the NPOESS contract, as well as increased program  management funds. According to the program office, contract changes  included extension of the development schedule, increased sensor costs,  and additional funds needed for mitigating risks. Increased program  management funds were added for noncontract costs and management  reserves.
At that time, we also noted that other factors could further affect the  revised cost and schedule estimates. Specifically, the contractor was not  meeting expected cost and schedule targets on the new baseline because  of technical issues in the development of key sensors, including the  critical VIIRS sensor. Based on its performance through May 2004, we  estimated that the contractor would most likely overrun its contract at  completion in September 2011 by $500 million—thereby increasing the  projected life cycle cost to $8.6 billion. In addition, we reported that risks  associated with the development of the critical sensors, integrated data  processing system, and algorithms, among other things, could contribute  to further cost increases and schedule slips—and we noted that continued  oversight was critical. The program office’s baseline cost estimate was  subsequently adjusted to $8.4 billion.
In mid-November 2005, we reported that NPOESS continued to experience  problems in the development of a key sensor, resulting in schedule delays  and anticipated cost increases. At that time, we projected that the  program’s cost estimate had grown to about $10 billion based on  contractor cost and schedule data. We reported that the program’s issues  were due, in part, to problems at multiple levels of management— including subcontractor, contractor, program office, and executive  leadership. Recognizing that the budget for the program was no longer  executable, the NPOESS Executive Committee planned to make a decision  in December 2005 on the future direction of the program—what would be  delivered, at what cost, and by when. This involved deciding among  options involving increased costs, delayed schedules, and reduced  functionality. We noted that continued oversight, strong leadership, and  timely decision making were more critical than ever and we urged the  committee to make a decision quickly so that the program could proceed.
However, we subsequently reported that, in late November 2005, NPOESS  cost growth exceeded a legislatively mandated threshold that requires  DOD to certify the program to Congress. This placed any decision about  the future direction of the program on hold until the certification took  place in June 2006. In the meantime, the program office implemented an  interim program plan for fiscal year 2006 to continue work on key sensors  and other program elements using fiscal year 2006 funding.
section_title: Nunn-McCurdy Process Led to a Decision to Restructure the NPOESS Program
paragraphs: The Nunn-McCurdy law requires DOD to take specific actions when a  major defense acquisition program exceeds certain cost thresholds. In  November 2005, key provisions of the act required the Secretary of  Defense to notify Congress when a major defense acquisition was  expected to overrun its project baseline by 15 percent or more and to  certify the program to Congress when it was expected to overrun its  baseline by 25 percent or more. At that time, NPOESS exceeded the 25  percent threshold, and DOD was required to certify the program.  Certifying a program entailed providing a determination that (1) the  program is essential to national security, (2) there are no alternatives to  the program that will provide equal or greater military capability at less  cost, (3) the new estimates of the program’s cost are reasonable, and (4)  the management structure for the program is adequate to manage and  control costs. DOD established tri-agency teams—made up of DOD,  NOAA, and NASA experts—to work on each of the four elements of the  certification process.
In June 2006, DOD (with the agreement of both of its partner agencies)  certified a restructured NPOESS program, estimated to cost $12.5 billion  through 2026. This decision approved a cost increase of $4 billion over the  prior approved baseline cost and delayed the launch of NPP and the first  two satellites by roughly 3 to 5 years. The new program also entailed  establishing a stronger program management structure, reducing the  number of satellites to be produced and launched from 6 to 4, and  reducing the number of instruments on the satellites from 13 to 9— consisting of 7 environmental sensors and 2 subsystems. It also entailed  using NPOESS satellites in the early morning and afternoon orbits and  relying on European satellites for midmorning orbit data. Table 2  summarizes the major program changes made under the Nunn-McCurdy  certification decision.
The Nunn-McCurdy certification decision established new milestones for  the delivery of key program elements, including launching NPP by January  2010, launching the first NPOESS satellite (called C1) by January 2013,  and launching the second NPOESS satellite (called C2) by January 2016.  These revised milestones deviated from prior plans to have the first  NPOESS satellite available to back up the final POES satellite should  anything go wrong during that launch.
Delaying the launch of the first NPOESS satellite means that if the final  POES satellite fails on launch, satellite data users would need to rely on  the existing constellation of environmental satellites until NPP data  becomes available—almost 2 years later. Although NPP was not intended  to be an operational asset, NASA agreed to move NPP to a different orbit  so that its data would be available in the event of a premature failure of  the final POES satellite. However, NPP will not provide all of the  operational capability planned for the NPOESS spacecraft. If the health of  the existing constellation of satellites diminishes—or if NPP data is not  available, timely, and reliable—then there could be a gap in environmental  satellite data. Table 3 summarizes changes in key program milestones over  time.
In order to reduce program complexity, the Nunn-McCurdy certification  decision decreased the number of NPOESS sensors from 13 to 9 and  reduced the functionality of 4 sensors. Specifically, of the 13 original  sensors, 5 sensors remain unchanged, 3 were replaced with less capable  sensors, 1 was modified to provide less functionality, and 4 were  cancelled. Table 4 shows the changes to NPOESS sensors, including the 4  identified in bold as critical sensors.
The changes in NPOESS sensors affected the number and quality of the  resulting weather and environmental products, called EDRs. In selecting  sensors for the restructured program, the Nunn-McCurdy process placed  the highest priority on continuing current operational weather capabilities  and a lower priority on obtaining selected environmental and climate  measuring capabilities. As a result, the revised NPOESS system has  significantly less capability for providing global climate measures than was  originally planned. Specifically, the number of EDRs was decreased from  55 to 39, of which 6 are of a reduced quality. The 39 EDRs that remain  include cloud base height, land surface temperature, precipitation type  and rate, and sea surface winds. The 16 EDRs that were removed include  cloud particle size and distribution, sea surface height, net solar radiation  at the top of the atmosphere, and products to depict the electric fields in  the space environment. The 6 EDRs that are of a reduced quality include  ozone profile, soil moisture, and multiple products depicting energy in the  space environment.
Given the changes in planned sensors, program officials established a  planned configuration for NPP and the four satellites of the NPOESS  program, called C1, C2, C3, and C4 (see table 5). Program officials  acknowledged that this configuration could change if other parties  decided to develop the sensors that were cancelled. However, they noted  that the planned configuration of the first satellite cannot change without  increasing the risk that the launch will be delayed.
section_title: Earned Value Management Techniques Provide Insight on Program Cost and Schedule
paragraphs: To be effective, project managers need current information on a  contractor’s progress in meeting contract deliverables. One method that  can help project managers track this progress is earned value  management. This method, used by DOD for several decades, compares  the value of work accomplished during a given period with that of the  work expected in that period.
Differences from expectations are measured in both cost and schedule  variances. Cost variances compare the earned value of the completed  work with the actual cost of the work performed. For example, if a  contractor completed $5 million worth of work and the work actually cost  $6.7 million, there would be a –$1.7 million cost variance. Schedule  variances are also measured in dollars, but they compare the earned value  of the work completed with the value of work that was expected to be  completed. For example, if a contractor completed $5 million worth of  work at the end of the month but was budgeted to complete $10 million  worth of work, there would be a –$5 million schedule variance. Positive  variances indicate that activities are costing less or are completed ahead  of schedule. Negative variances indicate activities are costing more or are  falling behind schedule. These cost and schedule variances can then be  used in estimating the cost and time needed to complete the program.
section_title: NPOESS Acquisition Restructuring Is Well Under Way, but Key Steps Remain To Be Completed
paragraphs: Since the June 2006 decision to revise the scope, cost, and schedule of the  NPOESS program, the program office has made progress in restructuring  the satellite acquisition; however, important tasks leading up to revising  and finalizing contract changes remain to be completed. Restructuring a  major acquisition program like NPOESS is a process that involves  identifying time critical and high priority work and keeping this work  moving forward, while reassessing development priorities,  interdependencies, deliverables, risks, and costs. It also involves revising  important acquisition documents including the memorandum of  agreement on the roles and responsibilities of the three agencies, the  acquisition strategy, the system engineering plan, the test and evaluation  master plan, the integrated master schedule defining what needs to  happen by when, and the acquisition program baseline. The Nunn- McCurdy certification decision required the Secretaries of Defense and  Commerce and the Administrator of NASA to sign a revised memorandum  of agreement by August 6, 2006. It also required that the program office,  Program Executive Officer, and the Executive Committee revise and  approve key acquisition documents including the acquisition strategy and  system engineering plan by September 1, 2006, in order to proceed with  the restructuring. Once these are completed, the program office can  proceed to negotiate with its prime contractor on a new program baseline  defining what will be delivered, by when, and at what cost.
The NPOESS program office has made progress in restructuring the  acquisition. Specifically, the program office has established interim  program plans guiding the contractor’s work activities in 2006 and 2007  and has made progress in implementing these plans. Specifically, the  program office reported that it had completed 156 of 166 key milestones  during fiscal year 2006—including completing ambient and thermal  vacuum testing of the VIIRS engineering unit. Of the 10 remaining  milestones resulting from unanticipated problems in the development of  VIIRS and CrIS, 5 have since been completed, and 5 are still pending. The  program office plans to complete 222 milestones in fiscal year 2007— including completing performance tests on the OMPS (nadir) sensor—and  notes that they are slightly ahead of plans in that they have completed 62  milestones through January 20, 2007, which is 2 more than had been  planned. Figures 7 and 8 depict the program office’s progress against key  milestones in fiscal year 2006 and to date in fiscal year 2007.
The program office has also made progress in revising key acquisition  documents. It revised the system engineering plan, the test and evaluation  master plan, and the acquisition strategy plan, and obtained approval of  these documents by the Program Executive Officer. The program office  and contractor also developed an integrated master schedule for the  remainder of the program—beyond fiscal year 2007. This integrated  master schedule details the steps leading up to launching NPP by  September 2009, launching the first NPOESS satellite in January 2013, and  launching the second NPOESS satellite in January 2016. Near-term steps  include completing and testing the VIIRS, CrIS, and OMPS sensors;  integrating these sensors with the NPP spacecraft and completing  integration testing; completing the data processing system and integrating  it with the command, control, and communications segment; and  performing advanced acceptance testing of the overall system of systems  for NPP.
However, key steps remain for the acquisition restructuring to be  completed. These steps include obtaining the approval of the Secretaries  of Commerce and Defense and the Administrator of NASA on the  memorandum of agreement among the three agencies, and obtaining the  approval of the NPOESS Executive Committee on key acquisition  documents, including the system engineering plan, the test and evaluation  master plan, and the acquisition strategy. These approvals are currently  over 6 months past due. Agency officials noted that the September 1, 2006,  due date for the key acquisition documents was not realistic given the  complexity of coordinating documents among three different agencies, but  did not provide a new estimate for when these documents would be  approved.
Finalizing these documents is critical to ensuring interagency agreements  and will allow the program office to move forward in completing other  activities related to restructuring the program. These activities include  conducting an integrated baseline review with the contractor to reach  agreement on the schedule and work activities, and finalizing changes to  the NPOESS development and production contract—thereby allowing the  program office to lock down a new acquisition baseline cost and schedule.  The program office expects to conduct an integrated baseline review by  May 2007 and to finalize the contract changes by July 2007. Until key  acquisition documents are finalized and approved, the program faces  increased risk that it will not be able to complete important restructuring  activities in time to move forward in fiscal year 2008 with a new program  baseline in place. This places the NPOESS program at risk of continued  delays and future cost increases.
section_title: Progress Has Been Made in Establishing an Effective NPOESS Management Structure, but Executive Turnover Will Increase Risks, and Staffing Problems Remain
paragraphs: The NPOESS program has made progress in establishing an effective  management structure, but—almost a year after this structure was  endorsed during the Nunn-McCurdy certification process—the Integrated  Program Office still faces staffing problems. Over the past few years, we  and others have raised concerns about management problems at all levels  of the NPOESS program, including subcontractor and contractor  management, program office management, and executive-level  management. Two independent review teams also noted a shortage of  skilled program staff, including budget analysts and system engineers.  Since that time, the NPOESS program has made progress in establishing  an effective management structure—including establishing a new  organizational framework with increased oversight by program  executives, instituting more frequent subcontractor, contractor, and  program reviews, and effectively managing risks and performance.  However, DOD’s plans for reassigning the Program Executive Officer in  Summer 2007 increase the program’s risks. Additionally, the program lacks  a staffing process that clearly identifies staffing needs, gaps, and plans for  filling those gaps. As a result, the program office has experienced delays in  getting core management activities under way and lacks the staff it needs  to execute day-to-day management activities.
subsections: section_title: NPOESS Program Has Made Progress in Establishing an Effective Management Structure and Increasing Oversight Activities, but Executive Turnover Will Increase Program Risks
paragraphs: The NPOESS program has made progress in establishing an effective  management structure and increasing the frequency and intensity of its  oversight activities. Over the past few years, we and others have raised  concerns about management problems at all levels of management on the  NPOESS program, including subcontractor and contractor management,  program office management, and executive-level management. In response  to recommendations made by two different independent review teams, the  program office began exploring options in late 2005 and early 2006 for  revising its management structure.
In November 2005, the Executive Committee established and filled a  Program Executive Officer position, senior to the NPOESS Program  Director, to streamline decision making and to provide oversight to the  program. This Program Executive Officer reports directly to the Executive  Committee. Subsequently, the Program Executive Officer and the Program  Director proposed a revised organizational framework that realigned  division managers within the Integrated Program Office responsible for  overseeing key elements of the acquisition and increased staffing in key  areas. In June 2006, the Nunn-McCurdy certification decision approved  this new management structure and the Integrated Program Office  implemented it. Figure 9 provides an overview of the relationships among  the Integrated Program Office, the Program Executive Office, and the  Executive Committee, as well as key divisions within the program office.
Operating under this new management structure, the program office  implemented more rigorous and frequent subcontractor, contractor, and  program reviews, improved visibility into risk management and mitigation  activities, and institutionalized the use of earned value management  techniques to monitor contractor performance. Specifically, program  officials and the prime contractor now review the subcontractors’ cost and  schedule performance on a weekly basis. The information from these  meetings feeds into monthly government meetings with the prime  contractor to review progress against milestones, issues, and risks.  Further, the Program Director conducts monthly reviews with each  technical division lead to review the divisions’ achievements, risks, and  plans. Program officials note that these frequent reviews allow information  on risks to be quickly escalated from subcontractors to contractors, to the  program component level, and to the Program Director—and they allow  program officials to better manage efforts to reduce risks. The program  office also reported that all division leads were trained in earned value  management techniques and were effectively using these techniques both  to monitor subcontractor and contractor performance on a weekly basis  and to identify potential problems as soon as possible.
In addition to these program office activities, the Program Executive  Officer implemented monthly program reviews and increased the  frequency of contacts with the Executive Committee. Specifically, the  Program Executive Officer holds monthly program management reviews  where the Program Director and program division leads (for example,  those in charge of systems engineering or ground systems) provide  briefings on the program’s earned value, progress, risks, and concerns. We  observed that these briefings allow the Program Executive Officer to have  direct insight into the challenges and workings of the Integrated Program  Office and allow risks to be appropriately escalated and addressed. These  meetings also provide an open forum for managers to raise concerns and  ask questions about operational challenges. For example, when NASA  officials expressed concerns that vibration levels used during testing were  higher than necessary and were causing damage to key sensor  components, the Program Director and Program Executive Officer  immediately established a forum to discuss and mitigate this issue. The  Program Executive Officer briefs the Executive Committee in monthly  letters, apprising committee members of the program’s status, progress,  risks, and earned value and the Executive Committee now meets on a  quarterly basis—whereas in the recent past, we reported that the  Executive Committee had met only five times in 2 years.
While the NPOESS program has made progress in establishing an effective  management structure, this progress is currently at risk. We recently  reported that DOD space acquisitions are at increased risk due in part to  frequent turnover in leadership positions, and we suggested that  addressing this will require DOD to consider matching officials’ tenure  with the development or delivery of a product. In March 2007, NPOESS  program officials stated that DOD is planning to reassign the recently  appointed Program Executive Officer in Summer 2007 as part of this  executive’s natural career progression. As of March 2007, the Program  Executive Officer has held this position for 16 months. Given that the  program is currently still being restructured, and that there are significant  challenges in being able to meet critical deadlines to ensure satellite data  continuity, such a move adds unnecessary risk to an already risky  program.
section_title: NPOESS Program Has Filled Key Vacancies but Lacks A Programwide Staffing Process
paragraphs: The NPOESS program office has filled key vacancies in recent months but  lacks a staffing process that identifies programwide staffing requirements  and plans for filling those needed positions. Sound human capital  management calls for establishing a process or plan for determining  staffing requirements, identifying any gaps in staffing, and planning to fill  critical staffing gaps. Program office staffing is especially important for  NPOESS, given the acknowledgment by multiple independent review  teams that staffing shortfalls contributed to past problems. Specifically,  these review teams noted shortages in the number of system engineers  needed to provide adequate oversight of subcontractor and contractor  engineering activities and in the number of budget and cost analysts  needed to assess contractor cost and earned value reports. To rectify this  situation, the June 2006 certification decision directed the Program  Director to take immediate actions to fill vacant positions at the program  office with the approval of the Program Executive Officer.
Since the June 2006 decision to revise NPOESS management structure, the  program office has filled multiple critical positions, including a budget  officer, a chief system engineer, an algorithm division chief, and a  contracts director. In addition, on an ad hoc basis, individual division  managers have assessed their needs and initiated plans to hire individuals  for key positions. However, almost a year after the certification, the  program office still lacks a programwide process for identifying and filling  all needed positions. As a result, division managers often wait months for  critical positions to be filled. For example, in February 2006, the NPOESS  program estimated that it needed to hire up to 10 new budget analysts. As  of September 2006, none of these positions had been filled. Today,  program officials estimate that they only needed to fill 7 budget analyst  positions, of which 2 positions have been filled, and 5 remain vacant.  Additionally, even though the certification decision directed immediate  action to fill critical vacancies, the program still has vacancies in 5 systems  engineering positions and 10 technical manager positions. The majority of  the vacancies—4 of the 5 budget positions, 4 of the 5 systems engineering  positions, and 8 of the 10 technical manager positions—are to be provided  by NOAA. NOAA officials noted that each of these positions is in some  stage of being filled—that is, recruitment packages are being developed or  reviewed, vacancies are being advertised, or candidates are being  interviewed, selected, and approved.
The program office attributes its staffing delays to not having the right  personnel in place to facilitate this process—and did not even begin to  develop a staffing process—until November 2006. Program officials noted  that the tri-agency nature of the program adds unusual layers of  complexity to the hiring and administrative functions because each agency  has its own hiring and performance management rules. In November 2006,  the program office brought in an administrative officer who took the lead  in pulling together the division managers’ individual assessments of  needed staff—currently estimated to be 25 vacant positions—and has been  working with the division managers to refine this list. This new  administrative officer plans to train division managers in how to assess  their needs and to hire needed staff and to develop a process by which  evolving needs are identified and positions are filled. However, there is as  yet no date set for establishing this basic programwide staffing process.
As a result of the lack of a programwide staffing process, there has been  an extended delay in determining what staff are needed and in bringing  those staff on board—which has resulted in delays in performing core  management activities such as establishing the program office’s cost  estimate and bringing in needed contracting expertise. Additionally, until a  programwide staffing process is in place, the program office risks not  having the staff it needs to execute day-to-day management activities.
section_title: Methodology Supporting the June 2006 Cost and Schedule Estimate Was Reliable, but Recent Events Could Increase Program Costs
paragraphs: In June 2006, DOD certified a restructured NPOESS program that was  estimated to cost $11.5 billion for the acquisition portion of the program  and scheduled to launch the first satellite in 2013. The Office of the  Secretary of Defense’s Cost Analysis Improvement Group (cost analysis  group)—the independent cost estimators charged with developing the  estimate for the acquisition portion of the program—used an acceptable  methodology to develop this estimate. When combined with an estimated  $1 billion for operations and support after launch, this brings the program  life cycle cost to $12.5 billion. Recent events, however, could further  increase program costs or delay schedules. Specifically, the program  continues to experience technical problems on key sensors, and costs and  schedules will be adjusted during negotiations on contract changes. The  NPOESS program office is developing its own cost estimate to refine the  one developed in June 2006 that it will use to negotiate contract changes.  A new baseline cost will be established once the contract is finalized.
subsections: section_title: Certified Program Estimates Were Developed Using an Acceptable Methodology
paragraphs: The cost and schedule estimate for the restructured NPOESS program was  developed by DOD’s cost analysis group using an acceptable methodology.  Cost-estimating organizations throughout the federal government and  industry use certain key practices—related to planning, conducting, and  reporting the estimate—to ensure a sound estimate. Table 6 lists the  elements of a sound cost estimating methodology. In addition, to ensure  the validity of the data assumptions that go into the estimate, leading  organizations use actual historical costs and seek an independent  validation of critical cost drivers.
DOD’s cost analysis group used an acceptable methodology in developing  the NPOESS cost estimate in that they planned, conducted, and reported  the estimate consistent with leading practices. The cost analysis group’s  cost estimating approach was largely driven by the program’s principal  “ground rule” to maintain the continuity of weather data without a gap.  Specifically, the cost analysis group assessed two risks: (1) the uncertainty  of the health of the current polar-satellite constellation and (2) the  uncertainty of when the new satellite system could be delivered (including  the time needed to evaluate new satellites once in orbit). The resulting  analysis showed that the restructured NPOESS system could be delivered  and the first satellite launched by 2013 with a high level of confidence in  maintaining satellite data continuity.
To determine specific costs, the group used the existing work breakdown  structure employed by the program office as the basis for performing its  work. This work breakdown structure consists of seven major elements,  including ground systems; spacecraft; sensors; assembly, integration and  test; system engineering/contractor program management; government  program management; and launch.
The cost analysis group also took steps to ensure the validity of the data  that went into the estimate. For each element, the cost analysis group  visited all major contractor sites to collect program data including    schedule (including the original, rebaselined, and current schedules, and  risks affecting the current schedule);   current staffing profile by month;  the history of staffing used;  the qualifications of people charging the program; the program’s technical approaches; the contractor’s program legacy (a justification that the contractor has  worked on similar projects in the past and that the contractor should be  able to adapt that knowledge to the current work).
The cost analysis group also compared this data with contractor labor  rates from the Defense Contract Management Agency and obtained  NASA’s validation of the costs associated with the most significant cost  driver, the VIIRS sensor.
Since schedule was the primary uncertainty factor in the cost analysis, it  also was the driver of overall costs. Specifically, the cost analysis group  took its risk-adjusted schedule durations for the major cost elements and  adjusted the contractor-submitted manning profiles accordingly. They  then used NPOESS historical data on labor rates and materials to calculate  the cost of these elements.
Consistent with DOD practice, the cost analysis group established its cost  estimate at a 50 percent confidence level. However, cost analysts could  not provide an upper limit for potential cost growth, explaining that the  program contains “failsafe” measures to use alternative technologies (such  as using legacy systems) if schedules are delayed and costs increase. As a  result, cost analysts reported that they have a high level of confidence that  acquisition costs will not exceed $11.5 billion—but a lower level of  confidence that the configuration of sensors will remain unchanged.
section_title: Recent Events Could Lead to Increased Program Costs or Delayed Schedules
paragraphs: While the June 2006 cost estimate for the acquisition portion of the  program was reasonable at the time it was made, several recent events  could cause program life cycle costs to grow or schedules to be delayed.  Specifically, the program continues to experience technical problems on  key sensors. The CrIS sensor being developed for the NPP satellite  suffered a major structural failure in October 2006. A failure review board  is currently working to resolve the root causes of the failure. While  program officials note that they should be able to cover costs related to  investigating the problem, the full cost and schedule to fix the sensor is  not yet known. Also, VIIRS development, which has been the program’s  primary cost driver, is not yet complete and continues to be a high-risk  development. This too, could lead to higher final program costs or delayed  schedules.
Program costs are also likely to be adjusted during upcoming negotiations  on contract changes. The NPOESS program office is developing its own  cost estimate to refine the one developed in June 2006. Program officials  plan to use this revised cost estimate to negotiate contract changes. A new  baseline cost will be established once the contract is finalized—an event  that the Program Director expects to occur by July 2007.
section_title: Major Program Segments Are Under Development, but Significant Risks Remain
paragraphs: Major segments of the NPOESS program—the space segment, the ground  systems segment, and the launch segment—are under development;  however, significant problems have occurred and risks remain. The  program office is aware of these risks and is working to mitigate them, but  continued problems could affect the program’s overall cost and schedule.  Given the tight time frames for completing key sensors, integrating them  on the NPP spacecraft, and getting the ground-based data processing  systems developed, tested, and deployed, it will be important for the  NPOESS Integrated Program Office, the Program Executive Office, and  the Executive Committee to continue to provide close oversight of  milestones and risks.
subsections: section_title: Space Segment—Progress Made, but Key Sensors Continue to Face Major Risks
paragraphs: The space segment includes the sensors and the spacecraft. Four sensors  are of critical importance—VIIRS, CrIS, OMPS, and ATMS—because they  are to be launched on the NPP satellite. Initiating work on another sensor,  the Microwave imager/sounder, is also important because this new  sensor—replacing the cancelled CMIS sensor—will need to be developed  in time for the second NPOESS satellite launch. Over the past year, the  program made progress on each of the sensors and the spacecraft.  However, two sensors, VIIRS and CrIS, have experienced major problems.  The status of each of the components of the space segment is described in  table 7.
Earned value management tools are used to compare the value of work  accomplished with the work expected during a given time period, and any  differences are measured in cost and schedule variances. The NPOESS  space segment experienced negative cost and schedule variances between  January 2006 and January 2007 (see fig. 10).
From January 2006 to January 2007, the contractor exceeded cost targets  for the space segment by $17 million—which is 4 percent of the space  segment budget for that time period. Similarly, the contractor was unable  to complete $14.6 million worth of work in the space segment. The main  factors in the cost and schedule variances were due to underestimation of  the scope of work, pulling resources from lower priority tasks to higher  priority items, and unforeseen design issues on key sensors. For example,  VIIRS continued to experience negative cost variance trends due to  unplanned efforts, which included refurbishing and recertifying the VIIRS  calibration chamber, completing the testing of the engineering design unit,  and resolving a problem with the testing equipment needed to adjust  VIIRS’ temperature during a key test.
Unplanned efforts for CrIS that attributed to the negative cost and  schedule variances included additional time required for testing and  material management. The schedule variances for VIIRS and CrIS were  mainly due to resources being pulled from other areas to support higher  priority tasks, extended testing and testing delays, management changes,  and improper material handling. Further, there is a high likelihood that  CrIS will continue to experience cost and schedule variances against the  fiscal year 2007 interim program plan until the issues that caused its  structural failure are addressed.
subsections: section_title: Program Office Is Monitoring Sensor Risks and Evaluating Options
paragraphs: Program officials regularly track risks associated with various NPOESS  components and work to mitigate them. Having identified both VIIRS and  CrIS as high risk, OMPS as a moderate risk, and the other components as  low risk, the program office is working closely with the contractors and  subcontractors to resolve sensor problems.
Program officials have identified work-arounds that will allow them to  move forward in testing the VIIRS engineering unit and have approved the  flight unit to proceed to a technical readiness review milestone in May  2007. Regarding CrIS, as of March 2007, a failure review board identified  root causes of its structural failure, identified plans for resolving them, and  initiated inspections of sensor modules and subsystems for damage. An  agency official reported that there is sufficient funding in the fiscal year  2007 program office’s and contractor’s management reserve funds to allow  for troubleshooting both VIIRS and CrIS problems. However, until the CrIS  failure review board fully determines the amount of rework that is  necessary to fix the problems, it is unknown if additional funds will be  needed or if the time frame for CrIS’ delivery will be delayed. According to  agency officials, CrIS is not on the program schedule’s critical path, and  there is sufficient schedule margin to absorb the time it will take to  conduct a thorough failure review process.
Managing the risks associated with the development of VIIRS and CrIS are  of particular importance because these are to be demonstrated on the NPP  satellite currently scheduled for launch in September 2009. Additionally,  any delay in the NPP launch date could affect the overall NPOESS  program because the success of the program depends on the lessons  learned in data processing and system integration from the NPP satellite.
section_title: Ground Segment— Progress Has Been Made, but Work Remains
paragraphs: Development of the ground segment—which includes the interface data  processing system, the ground stations that are to receive satellite data,  and the ground-based command, control, and communications system—is  under way and on track. However, important work pertaining to  developing the algorithms that translate satellite data into weather  products within the integrated data processing segment remains to be  completed. Table 8 describes each of the components of the ground  segment and identifies the status of each. Additionally, appendix II  provides an overview of satellite data processing algorithms.
section_title: Ground Segment Cost and Schedule Are on Track; Work and Risks Remain
paragraphs: Using contractor-provided data, our analysis indicates cost and schedule  performance on key elements of the NPOESS ground segment were  generally on track or positive against the fiscal year 2006 and 2007 interim  program plans. For the IDPS component, the contractor completed slightly  less work than planned and finished slightly under budget. This caused  cost and schedule variances of less than 1 percent off of expectations. (see  fig. 11). For the command, control, and communications component, the  contractor was able to outperform its planned targets by finishing under  budget by $3 million (6.2 percent of the budget for this time period) and by  completing $31,000 (less than 1 percent) worth of work beyond what was  planned (see fig. 12).
The NPOESS program office plans to continue to address risks facing  IDPS development. Specifically, the IDPS team is working to reduce data  processing delays by seeking to limit the number of data calls, improve the  efficiency of the data management system, increase the efficiency of the  algorithms, and increase the number of processors. The program office  also developed a resource center consisting of a logical technical library, a  data archive, and a set of analytical tools to coordinate, communicate, and  facilitate the work of algorithm subject matter experts on algorithm  development and calibration/validation preparations. Managing the risks  associated with the development of the IDPS system is of particular  importance because this system will be needed to process NPP data.
section_title: Launch Segment—NPP Launch Preparation Has Begun, while NPOESS Launch Planning Remains a Future Event
paragraphs: Different agencies are responsible for launching NPP and NPOESS. NASA  is responsible for the NPP launch and began procuring the launch vehicle  for NPP in August 2006. Program officials expect to have it delivered by  July 2009, less than 2 months before the scheduled NPP launch in  September 2009.
The NPOESS Integrated Program Office is responsible for launching the  NPOESS satellites. According to program officials, the Air Force is to  procure launch services for the program through DOD’s Evolved  Expendable Launch Vehicle contract. These services are to be procured by  January 2011, 2 years before the first scheduled launch.
section_title: Conclusions
paragraphs: NPOESS restructuring is well under way, and the program has made  progress in establishing an effective management structure. However, key  steps remain in restructuring the acquisition, including completing  important acquisition documents such as the system engineering plan, the  acquisition program baseline, and the memorandum of agreement  documenting the three agencies’ roles and responsibilities. Until these key  documents are finalized, the program is unable to finalize plans for  restructuring the program. Additionally, the program office continues to  have difficulty filling key positions and lacks a programwide staffing  process. Until the program establishes an effective and repeatable staffing  process, it will have difficulties in identifying and filling its staffing needs  in a timely manner. Having insufficient staff in key positions impedes the  program office’s ability to conduct important management and oversight  activities, including revising cost and schedule estimates, monitoring  progress, and managing technical risks. The program faces even further  challenges if DOD proceeds with plans to reassign the Program Executive  Officer this summer. Such a move would add unnecessary risk to an  already risky program.
In addition, the likelihood exists that there will be further cost increases  and schedule delays because of technical problems on key sensors and  pending contract negotiations. Major program segments—including the  space and ground segments—are making progress in their development  and testing. However, two critical sensors have experienced problems and  are considered high risk, and risks remain in developing and implementing  the ground-based data processing system. Given the tight time frames for  completing key sensors, integrating them, and getting the ground-based  data processing systems developed, tested, and deployed, continued close  oversight of milestones and risks is essential to minimize potential cost  increases and schedule delays.
section_title: Recommendations for Executive Action
paragraphs: Because of the importance of effectively managing the NPOESS program  to ensure that there are no gaps in the continuity of critical weather and  environmental observations, we are making recommendations to the  Secretaries of Defense and Commerce and to the Administrator of NASA  to ensure that the responsible executives within their respective  organizations approve key acquisition documents, including the  memorandum of agreement among the three agencies, the system  engineering plan, the test and evaluation master plan, and the acquisition  strategy, as quickly as possible but no later than April 30, 2007.
We are also recommending that the Secretary of Defense direct the Air  Force to delay reassigning the recently appointed Program Executive  Officer until all sensors have been delivered to the NPOESS Preparatory  Program; these deliveries are currently scheduled to occur by July 2008.
We are also making two additional recommendations to the Secretary of  Commerce. We recommend that the Secretary direct the Undersecretary  of Commerce for Oceans and Atmosphere to ensure that NPOESS program  authorities develop and implement a written process for identifying and  addressing human capital needs and for streamlining how the program  handles the three different agencies’ administrative procedures, and  establish a plan for immediately filling needed positions.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the Deputy  Secretary of the Department of Commerce (see app. III), the Deputy  Assistant Secretary for Networks and Information Integration of the  Department of Defense (see app. IV), and the Deputy Administrator of the  National Aeronautics and Space Administration (see app. V). All three  agencies agreed that it was important to finalize key acquisition  documents in a timely manner, and DOD proposed extending the due  dates for the documents to July 2, 2007. Because the NPOESS program  office intends to complete contract negotiations by July 4, 2007, we remain  concerned that any further delays in approving the documents could delay  contract negotiations and thus increase the risk to the program.
In addition, the Department of Commerce agreed with our  recommendation to develop and implement a written process for  identifying and addressing human capital needs and to streamline how the  program handles the three different agencies’ administrative procedures.  The department also agreed with our recommendation to plan to  immediately fill open positions at the NPOESS program office. Commerce  noted that NOAA identified the skill sets needed for the program and has  implemented an accelerated hiring model and schedule to fill all NOAA  positions in the NPOESS program. The department also stated that the  Program Director will begin presenting the detailed staffing information at  monthly program management reviews, including identifying any barriers  and recommended corrective actions. Commerce also noted that NOAA  has made NPOESS hiring a high priority and has documented a strategy— including milestones—to ensure that all 20 needed positions are filled by  June 2007.
DOD did not concur with our recommendation to delay reassigning the  Program Executive Officer, noting that the NPOESS System Program  Director responsible for executing the acquisition program would remain  in place for 4 years. The Department of Commerce also noted that the  Program Executive Officer position is planned to rotate between the Air  Force and NOAA. Commerce also stated that a selection would be made  prior to the departure of the current Program Executive Officer to provide  an overlap period to allow for knowledge transfer and ensure continuity.
However, over the last few years, we and others (including an independent  review team and the Commerce Inspector General) have reported that  ineffective executive-level oversight helped foster the NPOESS program’s  cost and schedule overruns. We remain concerned that reassigning the  Program Executive at a time when NPOESS is still facing critical cost,  schedule, and technical challenges will place the program at further risk.
While it is important that the System Program Director remain in place to  ensure continuity in executing the acquisition, this position does not  ensure continuity in the functions of the Program Executive Officer. The  current Program Executive Officer is experienced in providing oversight  of the progress, issues, and challenges facing NPOESS and coordinating  with Executive Committee members, as well as DOD authorities  responsible for executing Nunn-McCurdy requirements. Additionally,  while the Program Executive Officer position is planned to rotate between  agencies, the memorandum of agreement documenting this arrangement is  still in draft and should be flexible enough to allow the current Program  Executive Officer to remain until critical risks have been addressed.
Further, while Commerce plans to allow a period of overlap between the  selection of a new Program Executive Officer and the departure of the  current one, time is running out. The current Program Executive Officer is  expected to depart in early July 2007 and, as of mid-April 2007, a successor  has not yet been named. NPOESS is an extremely complex acquisition,  involving three agencies, multiple contractors, and advanced technologies.
There is not sufficient time to transfer knowledge and develop the sound  professional working relationships that the new Program Executive  Officer will need to succeed in that role. Thus, we remain convinced that  given NPOESS’s current challenges, reassigning the current Program  Executive Officer at this time would not be appropriate.
All three agencies also provided technical comments, which we have  incorporated in this report as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to interested  congressional committees, the Secretary of Commerce, the Secretary of  Defense, the Administrator of NASA, the Director of the Office of  Management and Budget, and other interested parties. In addition, this  report will be available at no charge on our Web site at  http://www.gao.gov.
If you have any questions on matters discussed in this report, please  contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) evaluate the National Polar-orbiting Operational  Environmental Satellite System (NPOESS) program office’s progress in  restructuring the acquisition; (2) evaluate the program office’s progress in  establishing an effective management structure; (3) assess the reliability of  the new life cycle cost estimate and proposed schedule; and (4) identify  the status and key risks facing the program’s major segments (the launch,  space, data processing, and ground control segments) and evaluate the  adequacy of the program’s efforts to mitigate these risks.
To evaluate the NPOESS program office’s progress in restructuring the  acquisition program, we reviewed the program’s Nunn-McCurdy  certification decision memo and program documentation including status  briefings and milestone progress reports. We also interviewed program  office officials and attended conferences and senior-level management  program review meetings to obtain information on the program’s  acquisition restructuring.
To evaluate the program office’s progress in establishing an effective  management structure, we reviewed the Nunn-McCurdy decision memo  for the program, as well as program documentation and briefings. We  assessed the status of efforts to implement recommendations regarding  the program’s management structure, including the work of the team  responsible for reviewing the management structure under the Nunn- McCurdy review. We also analyzed the program office’s organizational  charts and position vacancies. Finally, we interviewed officials responsible  for reviewing the management structure of the program under Nunn- McCurdy, attended senior-level management review meetings to obtain  information related to the program’s progress in establishing and staffing  the new management structure, and interviewed program office officials  responsible for human capital issues to obtain clarification on plans and  goals for the new management structure.
To assess the reliability of the new life cycle cost estimate and proposed  schedule, we analyzed the Office of the Secretary of Defense’s Cost  Analysis Improvement Group’s (cost analysis group) cost estimating  methodology and the assumptions used to develop its independent cost  estimate. Specifically, we assessed the cost estimating group’s  methodology against 12 best practices recognized by cost-estimating  organizations within the federal government and industry for the  development of reliable cost estimates. These best practices are also  contained in a draft version of our cost guide, which is currently being  developed by GAO cost experts. We also assessed cost- and schedule- related data, including the work breakdown structure and detailed  schedule risk analyses to determine the reasonableness of the cost  analysis group’s assumptions. We also interviewed cost analysis group  officials to obtain clarification on cost and schedule estimates and their  underlying assumptions. Further, we interviewed program officials to  identify any assumptions that may have changed.
To identify the status and key risks facing the program’s major segments  (the launch, space, data processing, and ground control segments) and to  evaluate the adequacy of the program’s efforts to mitigate these risks, we  reviewed the program’s Nunn-McCurdy certification decision memo and  other program documentation. We analyzed briefings and monthly  program management documents to determine the status and risks of the  key program segments. We also analyzed earned value management data  obtained from the contractor to assess the contractor’s performance to  cost and schedule. We reviewed cost reports and program risk  management documents and interviewed program officials to determine  the program segments’ risks that could negatively affect the program’s  ability to maintain the current schedule and cost estimates. We also  interviewed agency officials from the National Aeronautics and Space  Administration (NASA), the National Oceanic and Atmospheric  Administration (NOAA), the Department of Defense (DOD), and the  NPOESS program office to determine the status and risks of the key  program segments. Finally, we observed senior-level management review  meetings and attended conferences to obtain information on the status of  the NPOESS program.
We performed our work at the NPOESS Integrated Program Office and at  DOD, NASA, and NOAA offices in the Washington, D.C., metropolitan area  between July 2006 and April 2007 in accordance with generally accepted  government auditing standards.
section_title: Appendix II: Overview of Satellite Data Processing Algorithms and the Calibration and Validation Process
paragraphs: Algorithms are sets of instructions, expressed mathematically, that  translate satellite sensor measurements into usable information. In the  NPOESS program, government contractors are responsible for algorithm  development; the program office is responsible for independently  validating the algorithms. Scientists develop these algorithms, which are  then written as computer code to be incorporated into the interface data  processing system (IDPS) operational system. The NPOESS ground  system uses three primary types of algorithms:    Algorithms to develop raw data records “unpack” the digital packets  received by the antennas/IDPS (the ones and zeros) and sent from the  satellite, associate the data with the information about the satellite’s  location and, finally, translate it back into the data it was when it started at  the sensor.
Algorithms used to develop sensor and temperature data records allow the  on-ground users to understand what the sensor saw. It translates the  information from the sensor into a measure of the various forms of energy  (e.g., brightness, temperature, radiance).
Algorithms used to produce the weather products called environmental  data records (EDR) are crosscutting. They combine various data records,  as well as other data, in order to produce measures useful to scientists.  Additionally, EDRs can be “chained”—that is, the output of one EDR  algorithm will become an input into the next EDR algorithm. To illustrate,  cloud detection/mask is an important “base” EDR because many EDRs,  like sea surface temperature, are only calculated when clouds are not  present. Figure 13 shows the flow of the data and algorithms.
A corollary to algorithm development is the calibration and validation  process. According to a senior algorithm scientist, in this process, once the  satellite has been launched, scientists verify that the sensors accurately  report what ground conditions are. For example, one EDR from the  visible/infrared imager radiometer suite (VIIRS) is “ocean color.” Once the  sensor is in orbit, scientists can compare the results that the VIIRS sensor  reports on ocean color with the known results from sensors on ocean  buoys that also measure ocean color in select locations. Then, if the  sensors do not accurately report the ground conditions, scientists can  calibrate, or “tweak,” the algorithms used to develop sensor, temperature,  and environmental data records to report on ground conditions more  accurately.
According to an agency official, fully calibrating a simple sensor once it  has been launched can take approximately a year. A more complicated  sensor can take 18 months to 2 years (see fig. 14).
section_title: Appendix III: Comments from the Department of Commerce
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: Comments from the National Aeronautics and Space Administration
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Colleen Phillips, Assistant  Director; Carol Cha; Neil Doherty; Nancy Glover; Kathleen S. Lovett; Karen  Richey; and Teresa Smith made key contributions to this report.