Space Station: Cost Control Difficulties Continue
Released: 1996-07-24T00:00:00

Letter

Introduction

In late 1997, the National Aeronautics and Space Administration (NASA) is scheduled to begin assembling the International Space Station (ISS). The facility is scheduled to be completely assembled in orbit by June 2002, and NASA is planning a 10-year operational life following its assembly. Its current configuration (see fig 1.1) is the result of the program’s last redesign, conducted in 1993 to compensate for additional funding cuts and to bring Russia into the program as a full partner along with Japan, Canada, the European Space Agency, and the United States.

NASA’s cost estimate for ISS development and operation is $17.4 billion from October 1993 through the planned completion of assembly in space in June 2002. NASA’s $17.4 billion program includes a prime contract effort to develop the U.S. portion of ISS, nonprime efforts involving a large number of smaller contracts to develop the ground-based and on-orbit capability to use and operate it, and NASA headquarters-managed contracts to develop on-orbit research facilities and conduct research.

In its fiscal year 1996 budget, NASA estimated the price of the prime contractor’s activities at approximately $6.3 billion; the nonprime efforts, $5.6 billion; research capability effort, $2.6 billion; and about $3.1 billion in financial reserves for the remaining 7 years of development. Figure 1.2 shows the percentage distribution of the ISS development budget. In mid-1993, a $2.1-billion annual funding limitation was imposed by the administration on the program to prevent it from consuming increasingly larger portions of NASA’s research and development budget.

In November 1993, NASA signed a letter contract with the Boeing Company for the prime development effort. The prime effort encompassed the work previously under separate contracts with the former Space Station Freedom work package contractors. Under this prime contract, the former work package contractors—McDonnell Douglas, Rocketdyne, and Boeing-Huntsville—became major product group subcontractors to Boeing. Product group subcontracts account for over 80 percent of the total estimated cost of the prime contract.

Concerns About Cost Reporting at the Johnson Space Center

In 1994, the Defense Contract Audit Agency identified a number of cost reporting weaknesses by NASA contractors at the Johnson Space Center, including some contractors that had worked on the Freedom version of the space station. These weaknesses, which led to misleading reporting of the program’s true cost status, including underreporting of potential overruns, were the focus of a July 1994 hearing before the Subcommittee on Oversight and Investigations of the House Committee on Energy and Commerce. In responding to these audit findings, NASA required its major space station contractors and product group subcontractors to establish detailed contract budgets and baselines, and to report periodic cost and schedule performance status using validated performance measurement systems. NASA also stipulated that (1) the performance measurement baseline would be changed only by formal modifications to the contract for scope of work changes, (2) cost growth on the contract would be proposed by the contractor and authorized by NASA before the additional costs were incurred, (3) an estimate to complete work would be evaluated on a monthly basis, and (4) the contract would include an incentive fee feature to encourage and motivate contractor cost performance.

Objectives, Scope, and Methodology

The Chairman of the Subcommittee on Investigations and Oversight, House Commerce Committee, asked us to review the program’s cost and schedule status and NASA’s actions to improve cost reporting. To accomplish these objectives, we interviewed ISS program office and contractor personnel and reviewed pertinent documents, including the prime contract between NASA and Boeing, contractor performance measurement system reports, the prime contractor’s management system plan, and surveillance reports prepared by the Department of Defense’s (DOD) Defense Plant Representative Office personnel at product group subcontractor locations.

We also interviewed NASA officials at the Johnson Space Center and the Marshall Space Flight Center, observed reviews conducted by NASA and Boeing personnel to verify that the prime and the major product group subcontractors had implemented valid cost and schedule control systems, and identified the program’s cost and schedule trends over time using performance measurement data. We did not review the portion of the estimated development budget representing the $2.6 billion for research capability.

We performed our review from August 1995 to April 1996 in accordance with generally accepted government auditing standards.

The ISS Program Still Faces Substantial Cost and Schedule Threats

The ISS program has been producing flight hardware since 1993. However, it continues to face cost and schedule issues that threaten the already limited financial reserves available to complete the station within its $17.4-billion total and $2.1-billion annual budget, including (1) the large number of authorized unpriced changes to the prime development contract, (2) unfavorable cost and schedule trends, and (3) potentially understated cost estimates at completion. Further, NASA’s ability to complete the station program on schedule and within budget is potentially threatened by the risk of the Russian Space Agency (RSA) not totally or substantially meeting its commitments.

Financial Reserves Continue to Be Limited in the Near Term

In our June 1995 report, we noted that the station’s financial reserves for fiscal years 1996 and 1997 were low at $102 million and $182 million, respectively. During fiscal year 1995, program managers identified cost savings and deferrals that increased the fiscal year 1996 reserves. If the cost of the currently known threats to financial reserves are realized and station managers are unable to find ways to offset the added cost, the financial reserves will continue to be low over the next several years, as shown in table 2.1.

The two largest threats—a crew rescue vehicle ($586 million) and a control module ($250 million)—account for $836 million, or about 59 percent, of the $1.42 billion. Another threat is related to an accounting issue. As initially raised by the prime contractor, NASA would pay an additional $76 million in overhead costs. NASA does not believe the additional costs should be allowed, but has agreed to abide by the decision of the DOD administrative contracting officer. A decision by DOD is pending.

Inadequate reserves would hinder program managers’ ability to cope with unanticipated technical problems. If a problem’s solution could not be funded by available reserves, program managers could be faced with either exceeding the annual funding cap or deferring or rephasing other activities, thus possibly delaying the development schedule and likely increasing overall funding requirements. However, program managers believe that it is unlikely that all known contingencies will require funding. They also said that they are continuing their efforts to identify future cost savings and develop other strategies that could be used to offset the potential cost growth threats. Recently, they were able to increase the reserves by negotiating with the Office of Life and Microgravity Sciences to fund some items, such as laboratory support equipment, that were to have been funded by the development budget. Also, ISS managers have challenged the managers of the nonprime efforts to reduce their future costs by 10 percent. If the nonprime managers are successful, reserves would increase.

In commenting on a draft of this report, NASA said that, as of April 1996, the financial reserves for fiscal year 1996 totaled $176 million, with $108 million in threats. NASA anticipates that there will be an unused balance of fiscal year 1996 reserves to carryover and augment the expected fiscal year 1997 reserve level.

Further Potential Use of Reserves

NASA’s current list of potential cost increases does not include some items that could further erode financial reserves. First, contractor estimates of the cost of authorized changes are higher than NASA’s estimates. The higher contractor estimates, if realized, would create a greater use of reserves than NASA anticipates. Second, the prime contractor’s performance measurement status reports show that cost and schedule performance is worsening. Third, the prime contractor’s performance measurement status reports do not appear to present realistic estimates of completion costs on some station elements, thus potentially masking additional overruns.

Cost Baseline Could Increase When Contract Changes Are Negotiated

Authorized unpriced changes to the prime contract baseline since January 1995 may increase prime contract costs by over $723 million, or about 14 percent of the original contract amount, according to contractor estimates. About $300 million of this amount is related to the Russian functional energy block, which was added to the prime contract when Boeing agreed to manage the acquisition, and to the accounting change noted above that Boeing wants but NASA is disputing. As of January 1996, the contractor-estimated value of the remaining changes was, therefore, just over $400 million. In April 1996, NASA budgeted funds to accommodate most, but not all, of this amount because ISS program managers expect to negotiate these changes for less than the contractors’ estimates. If program managers are unable to negotiate them at the lower amounts they expect, financial reserves will be further reduced.

NASA’s goal for negotiating prices on contract changes, as stated in the NASA Supplement to the Federal Acquisition Regulations, is 180 days. However, as of March 1996, 39 unpriced contract actions with an estimated value of $507.6 million exceeded NASA’s target date for completing negotiations. A major problem has been obtaining initial cost proposals from the prime contractor and product group subcontractors for starting price negotiations. As of early March 1996, contractors had provided proposals covering less than 20 percent of the value of all changes.

Program managers recognize the unpriced change orders as a major problem and have implemented a task team to expedite their final pricing. NASA’s strategy includes (1) negotiating older changes for which cost proposals have been received as a lump settlement; (2) requesting immediate contractor preparation and submittal of cost proposals for the top 20 changes, which represent approximately 71 percent of the estimated cost of all changes, and immediately negotiating them; (3) establishing time frames for contractor preparation and submittal of cost proposals and negotiations for the remaining critical changes; and (4) suspending and, perhaps, canceling 10 changes that were determined to be less critical, even though the contractor has already begun work and incurred costs. Table 2.2 shows the status of the prime contract changes in March 1996.

NASA officials told us that they have now received proposals covering many of these changes and that they plan to have them all negotiated by July 31, 1996.

Unfavorable Cost and Schedule Trends

Data from the prime contractor’s performance measurement status reports show the prime development program is behind its planned cost and schedule, with deteriorating cost and schedule trends. As illustrated in figure 2.1, the cost variance went from a positive $26.5 million in January 1995 to a negative $88.6 million in April 1996. Similarly, the schedule variance worsened over the same time period, going from a negative $43.2 million in January 1995 to a negative $87.9 million.

Technical difficulties have caused the unfavorable cost and schedule conditions. Problems include the development of the node, which is scheduled to be launched on the first U.S. launch in December 1997, and the U.S. Laboratory section. Most of the adverse cost and schedule variances are due to development problems being experienced by lower tier subcontractors. Data from the Performance Measurement Status Report show that 67 percent of the over cost and 52 percent of the behind schedule conditions at the end of 1995 existed at lower tier subcontractors where performance measurement oversight is limited. Program managers have identified lower tier subcontract performance as a top program risk. The program manager for the largest product group subcontractor also told us that subcontractor performance is one of his major concerns.

These unfavorable trends indicate developing cost and schedule problems that may be difficult to overcome. Contractor analyses in early 1996 showed that the unfavorable cost variance could exceed $114 million by July 1996 if the trends continued and recovery actions were unsuccessful. Analyses of cost performance data at that time predicted estimates of project completion costs ranging from $5.93 billion to $6.29 billion. As previously mentioned, the prime contractor’s January 1996 cost estimate, including all authorized but unpriced changes, was $5.87 billion. Against this baseline, estimated overruns at project completion ranged from about $60 million to over $400 million.

Understated Completion Cost Estimates

The monthly performance measurement status reports include instances where the contractor’s estimated total completion costs did not recognize over budget conditions. The accompanying narrative did not explain how the over budget conditions would be corrected and included indications that overrun conditions could worsen. To the extent that contractors’ estimates of completion costs are based on overly optimistic recovery plans, reported estimated completion costs would be understated and would potentially obfuscate the funding requirements for completing the program. Understated costs will ultimately further strain the limited financial reserves.

Accurate information and analyses of cost and schedule status and estimates of total completion costs are important because they provide cost visibility of the actual funding requirements for the approved scope of work. At a minimum, estimates of completion costs should consider actual performance and costs to date, projections of future performance, and estimates of the cost of work remaining. Research has shown that once a project is about 15-percent complete, it becomes increasingly unlikely that unfavorable cost or schedule trends will be reversed and more likely that the percent overrun at completion will be greater than the percent overrun to date. Such circumstances heighten the importance of contractors detailing their recovery plans when they claim that there will be no or little cost overrun at completion. NASA’s and Boeing’s surveillance audits of contractor cost and schedule control system compliance, which are discussed in chapter 3, also identified the lack of realistic completion estimates as a problem area.

The following examples illustrate potentially understated completion cost estimates from the January 1996 performance measurement status reports.

The structures and mechanism line item, with 40 percent of the work completed, showed a $11.3-million, or 11-percent, overrun. The estimate at completion forecast a $5-million overrun. The report narrative said that a lower tier subcontractor forecast a $16-million overrun at project completion for its portion of the work. This subcontractor had developed a recovery plan that would take 2 years to 3 years to implement. The product group subcontractor had reviewed its subcontractor’s cost estimate and had established a lower provisional estimate.

The U.S. Node 1 line item, with 64 percent of work completed, showed a $17.4-million, or 15-percent, overrun and a 7-percent behind schedule condition, valued at $8.3 million. However, the contractor forecasts about a $0.3-million underrun at completion. The report narrative said that plans to regain schedule include using three shifts working 7 days a week, which indicates that additional cost overruns can be expected. The narrative also said that this line item will experience a cost overrun at completion but efforts are being made to limit its extent.

With 62 percent of work completed, the U.S. Laboratory line item is $19.2 million, or 5 percent, over budget and $20.1 million behind schedule.

The report, however, predicted only about a $0.6-million underrun at completion. Reasons for the current overruns included increased spending to recover from schedule delays, purchase substitute parts, and pay premiums for accelerated deliveries; testing taking longer than planned; and poor vendor performance. The narrative said that the prime contractor is working on recovery plans to reduce these unfavorable variances, but that the majority of the cost variance is unrecoverable. It also predicted that several U.S. Laboratory cost elements would not recover from their current overruns.

The communications and tracking line item, with 55 percent of work completed, showed a $5.3-million, or 6-percent, overrun but forecast completing the effort with no overrun. The narrative said that some elements had experienced technical development problems and testing failures and that they will experience cost overruns at completion. However, the narrative also said that the unfavorable cost variances will be offset by good performance elsewhere within this cost account and that a subcontractor will cover other unanticipated costs with management reserve funding. The narrative did not say how the current overrun would be corrected, but stated that the primary tier II subcontractor was managing to reduce budgets, which, if successful, would produce savings to offset the overrun.

With 46 percent of work completed, the thermal control line item shows a $3.9-million, or 7-percent, overrun, and the estimate at completion predicts a $1.8-million overrun. The narrative said that a subcontractor, who is primarily responsible for much of the current overrun, has prepared an estimate that projects a $2.7-million overrun at completion. While the narrative said that recovery plans are in place to reduce the overrun by about $0.4 million, the narrative did not say how the remaining $2.4-million overrun would be reduced to $1.8 million.

A product group subcontractor reported a $2.6-million, or 9-percent, overrun of the radiator orbital replacement unit line item at the 49-percent completion point. However, this subcontractor reported completing the project with no overrun, although a lower tier subcontractor performing the work forecast a $2.6-million overrun at completion for its portion of the work. The narrative explained that because the lower tier subcontractor had started cost recovery actions to potentially reduce some of the current overrun and was evaluating other actions, the product group subcontractor chose not to recognize any variance at completion until additional studies were completed.

The performance measurement status reports also do not show the most recent bottom-up project completion cost estimates of the largest product group subcontractor, McDonnell Douglas, which was conducted prior to the July 1995 negotiations with the prime contractor. The estimate totaled $1.99 billion, $210 million more than the amount negotiated with Boeing and currently reported in the performance measurement system. During negotiations, the subcontractor agreed on a lower estimated cost of $1.78 billion, provided that it would be exempted from sharing the cost of overruns up to $150 million.

The difference between the estimate used in the performance measurement system and McDonnell Douglas’ most recent bottom-up estimate is an additional cost risk. McDonnell Douglas did not plan to do its next bottom-up estimate of completion costs until the fall of 1996, although it is considering completing a new estimate by this summer. By the time a new estimate is done, over half of the total development effort is scheduled to be completed. If the new estimate projects a cost overrun, recovery will be difficult, if not impossible.

Additional Program Cost Risk of Decreased RSA Participation

In mid-December 1995, RSA proposed major changes in its involvement in the development effort. RSA proposed the change because it claimed that it could not sustain the number of launches needed to support both MIR and ISS. The proposal added to concerns about RSA’s ability to fulfill its obligations to the program. If RSA does not meet its commitments, NASA’s cost to develop and operate ISS will increase. Some of this potential cost is included in the potential threats to the financial reserves previously discussed in this chapter. However, a total and sudden RSA withdrawal would likely prevent the ISS program from achieving the completion of assembly within its current $17.4-billion estimate. NASA officials stated that a complete and sudden withdrawal by RSA could result in a renegotiation of the annual $2.1-billion funding profile and the overall $17.4-billion assembly completion estimate.

Initially, RSA committed to provide a Service Module for habitation and for guidance, navigation, and control; Soyuz vehicles to provide for emergency return through the completion of station assembly; and Progress vehicles to resupply dry cargo (food, clothing, etc.) and the propellant needed to maneuver and reboost ISS to maintain orbit. RSA’s December 1995 proposed revision to that commitment included two options. Both essentially called for ISS elements to be attached to the MIR Space Station. One would have allowed RSA to delay development of the Service Module and the upgrading of Progress vehicles. Under the other, RSA would deliver the Service Module on time, but all other modules would be deferred past 2002. The most significant change for both options was the deletion of the Zenit launch vehicle, thereby eliminating RSA’s use of upgraded Progress vehicles with larger payload capacity and its ability to launch the Science Power Platform.

NASA rejected RSA’s proposal because of unacceptable safety, cost, and operational impacts. However, it countered with an offer to ease RSA’s launch requirements by using the space shuttle to fly supplies to MIR, and to launch the Science Power Platform. RSA accepted and, in return, committed to, among other things, delivering the Service Module on time, increasing the capacity of the Progress vehicle, developing a new vehicle for resupplying propellant and dry cargo, and modifying the Soyuz vehicles to accommodate a larger percentage of the U.S. astronaut corps.

Despite RSA’s recommitment, NASA officials continue to be concerned about RSA’s ability to meet its obligations. If RSA is totally or substantially unable to meet its obligations, NASA would have to make up the shortfall. Some of the cost of this additional work is included in the list of potential threats to the program’s financial reserves. For example, NASA has designated $250 million for developing a Control Module as a potential substitute for the Service Module. However, other additional costs are not currently identified as threats to reserves, including, for example, the cost of modifying shuttle orbiters to enable them to resupply propellant to the station.

NASA headquarters officials said that the cost impact could not be entirely met by using reserves, if RSA is totally or substantially unable to meet its commitments. Early estimates of the cost and schedule impacts are currently being analyzed. The specific impacts are not yet known since they depend on the timing and extent of RSA’s withdrawal. However, ISS managers told us that, in general, the major impact would be the slip in the assembly schedule. A delay in completing assembly would likely increase ISS’ cost because requirements would increase and development and operations would be rephased over a longer period of time.

Performance Measurement Systems Being Implemented, but Problems Remain

NASA has made significant progress in implementing detailed and comprehensive performance measurement systems at its major contractors and product group subcontractors working on the prime development program. However, NASA’s ability to routinely monitor cost and schedule status using such systems is hampered because (1) the baseline against which cost and schedule progress is measured is not yet completely established and (2) some inconsistencies and deficiencies in implementing the systems impact the accuracy of the cost and schedule information reported.

NASA has made slower progress in obtaining performance measurement data from nonprime contractors. Nonprime activity is spread across several NASA centers and many contracts do not require performance measurement data. Also, NASA managers responsible for all the nonprime work have not yet agreed with ISS managers on the type and detail of performance data to be provided; and some nonprime contractors providing performance data have not had their performance measurement systems reviewed.

Performance Measurement of Prime Development Effort

The contractors working on the prime development effort have made significant progress in implementing validated performance measurement systems to monitor cost and schedule status. Contractors have established detailed time-phased budgets and schedules against which progress is being measured and reported. The contractors are submitting monthly reports that include: detailed cost and schedule performance information by work breakdown structure to the 5th level, variance analyses, and corrective action plans to mitigate significant cost and schedule problems.

However, some obstacles to an effective performance measurement system remain. The baseline against which progress is measured is not yet completely agreed to. Also, surveillance audits conducted by NASA and the prime contractor disclosed problems that need to be corrected to ensure accurate, consistent, and comprehensive reporting.

Performance Measurement Baseline Not Yet Completely Established

The prime contractor’s negotiations with the other major product group subcontractor was completed to NASA’s satisfaction.

Douglas to overrun this amount by up to $150 million without invoking the incentive fee clause requiring contractors to share in any overruns.

Obstacles to Effective Performance Measurement Remain

All the contractors passed their initial surveillance audits performed by NASA or the prime contractor to assess the implementation of their performance measurement systems. However, the reviews identified implementation deficiencies, including the following:

The method used by the prime contractor to record “earned value” was not consistent with its corporate system description and may have resulted in subjective and inaccurate measurement of development progress. Also, the prime contractor’s criteria for completing established milestones were not well defined and some managers were taking credit for milestones not fully completed. A program official told us that the contractor is revamping its performance measurement system to correct these problems. The target period for reporting data using the new system is the summer of 1996.

The prime contractor’s budgets contain excessive level-of-effort work. ISS program officials tasked the prime contractor to determine the extent of level-of-effort work within both prime and product group subcontractor budgets and identify ways to reduce that amount.

One product group subcontractor did not have good processes for overseeing its lower tier subcontractors’ performance measurement status. Program officials tasked the prime contractor to provide more in-depth reporting on actions planned to mitigate the subcontractors’ cost and schedule problems.

One product group subcontractor was reporting its estimate at completion at the negotiated amount instead of its most likely estimate of costs to complete contracted work. Also, the latest revised completion cost estimates in other product group subcontractors’ monthly performance measurement status reports were thought to be unrealistic. A program official told us that this issue was addressed in the prime contractor’s award fee evaluation for the period ending March 1996.

Two product group subcontractors had not completed comprehensive re-evaluations of estimates to complete contracted work in accordance with their corporate system descriptions. These subcontractors’ corporate system descriptions require that such re-evaluations be done at least annually. One of these product group subcontractors expected to report the revised estimates in the April 1996 performance measurement status report. The other product group subcontractor was considering completing a re-evaluation by June 1996.

Until corrected, these deficiencies could impact the accuracy of reported cost and schedule progress. However, NASA has made progress in addressing the problems noted during the surveillance audits, including those previously mentioned. In addition, NASA plans to perform periodic follow-up reviews and has delegated surveillance responsibilities to DOD contract administration organizations where they are co-located at contractor plants.

Slower Progress in Implementing Performance Measurement on Nonprime Effort

ISS program managers do not get the same level of detailed performance data on nonprime work that they get on the prime development effort. There are several reasons for this. First, the work is widely dispersed, with many contracts spread out across various NASA centers and managed by different NASA organizations, with no single entity responsible for the entire effort. Second, many contracts were awarded before NASA issued its draft policy on reporting performance measurement information and thus do not require performance status reporting; in other cases, some of the contracts’ dollar values are too low to require detailed reporting. Third, ISS program managers have not agreed with all contract managers at the centers on what performance measurement information will be reported. Finally, some contractors who were providing performance measurement data either did not have recently reviewed systems or were providing only summary data with little detail or analyses.

NASA officials said that the recent designation of Johnson as lead center for the ISS program is intended to provide the needed consolidated management and oversight in these areas. In addition, in commenting on a draft of this report, NASA said that the program recently made significant progress in obtaining performance measures on the nonprime effort, noting the new quarterly earned value reporting (discussed in the following section) and the coordination of earned value determination techniques between the nonprime organizations and the ISS program office.

Nonprime Contracts Dispersed and Not Conducive to Performance Measurement Reporting

Nonprime work, which is related primarily to ISS operation and use, including the construction of the control center, operational training facilities, and space shuttle modifications to support ISS, accounts for $5.6 billion of the total $17.4 billion development budget. According to program office statistics, 5 NASA field centers had more than 65 contractors working on nonprime tasks in fiscal year 1995.

Although generally dispersed, the nonprime effort is somewhat concentrated at the Johnson Space Center, where various directorates are responsible for more than 27 contractors working on over 40 tasks. Except for a small number of contracts awarded for specific ISS-related requirements, virtually all of the nonprime work was added to existing center support contracts. For example, ISS operations and training tasks were added to existing contracts under the Johnson Space Center Mission Operations Directorate, and engineering development work was added to the Engineering Directorate’s support contract. The tasks added to existing contracts usually have dollar values too low to require performance measurement reporting.

These support contracts have historically been level-of-effort contracts on which progress is difficult to determine because work is typically measured in terms of resources consumed instead of accomplishments against established milestones. Also, some support contracts were awarded before NASA revised its policy on using performance measurement in 1995. Consequently, current performance measurement reporting requirements were not included in the contracts. For example, the major contracts supporting ISS-related tasks for the Space and Life Sciences Directorate at Johnson Space Center were awarded in 1991, 1993, and 1995. The contract awarded in 1995 is a completion form contract, but its value of about $20 million, is too low to require performance measurement reporting. The contract that provides most engineering support was awarded in 1994, also before NASA revised its policy.

Managers at the Johnson Space Center are attempting to convert support service contracts from level of effort to completion form contracts, with established product delivery milestones to improve their performance measurement oversight. In commenting on a draft of this report, NASA said that agreements have recently been reached for quarterly earned value reporting on nonprime tasks estimated to cost $2 million annually or $5 million in total.

Agreements on Performance Measurement Reporting Not Yet Reached With All Centers

Issues involved in reaching agreement with contract managers to provide performance management assessments for their ISS-related tasks include who will prepare the assessments, the scope of coverage, and the detail that will be provided. This task is made more difficult because the performance measurement systems specified by the centers’ contracts do not prescribe the same level of disciplined planning and contractor conformity to validated performance measurement systems as required of the ISS prime development contractor, and nonprime contract managers are not as familiar with the newer performance measurement concepts as the prime contract managers are.

In October 1995, ISS program managers queried the major nonprime contract managers to obtain a performance progress assessment. The assessment reported most of the nonprime effort to be behind schedule and within budget. However, in reviewing that data, ISS program managers agreed that the reported information was not an accurate assessment of performance progress because performance was not measured against budgeted milestones and, in most cases, only reflected spending status; that is, actual costs incurred compared to budgeted costs.

Extent of Review of Contractors’ Performance Measurement Systems Varies

While most nonprime contracts involving ISS-related tasks do not require performance measurement assessments, a few do. Examples include the Mission Systems and Training Systems contracts and portions of the space shuttle and space suit contracts managed by the Johnson Space Center, and the utilization contract managed by the Marshall Space Flight Center.

The Mission Systems and Training Systems contracts were awarded in 1989 and the Johnson Space Center’s Mission Operations Directorate, which oversees these contracts, began pursuing performance measurement reporting for these two contracts in late 1992. As a result, these two contractors have implemented performance measurement systems. NASA has reviewed and approved both contractors’ systems. The contractors provide NASA with monthly performance measurement status reports using DOD-specified formats.

The space shuttle and space suit contractors are also providing performance measurement assessments for some ISS-related tasks, even though their contracts do not require a validated performance measurement system or specify the type of performance data to be provided. However, NASA has not verified that these contractors are properly using performance measurement systems to track and report progress. The reports provided by these contractors do not include variance analyses or corrective action plans.

NASA last conducted a review of the utilization contract at the Marshall Space Flight Center in 1992, when that contract was a part of the Space Station Freedom Work Package 1 contract. However, with the inception of ISS, the Freedom contract was novated, resulting in the current utilization contract. The contractor’s system has not been reviewed since the novation. NASA officials at the Marshall Space Flight Center told us that they plan to conduct a review as soon as the contractor establishes a new baseline, most likely sometime in the summer of 1996. Changes in the content of the contract, budget changes, and NASA’s efforts to reduce its funding requirements have led to the current instability in the contract’s baseline. NASA officials at the Marshall Space Flight Center said that the instability had prevented them from using the performance measurement system to its fullest advantage.

What Is a Performance Measurement System and What Is It Supposed to Do?

Performance measurement is a management tool for planning, monitoring, and controlling all aspects of program and project management—cost, schedule, and technical requirements. Performance measurement has its origins in the Department of Defense (DOD) programs of the 1960s.Interest and application of the performance measurement concept spread to other government agencies in the 1970s and 1980s. Today, it is being applied to major programs of DOD, the National Security Agency, the Department of Energy, the Federal Aviation Administration, and the National Aeronautics and Space Administration (NASA).

NASA’s ability to track the development schedule for the International Space Station (ISS) is critical as the total program is spending over $5 million a day. To improve cost control and management of space station development contracts, NASA committed its contractors to implementing validated performance measurement systems. Also, in July 1995, NASA issued a draft policy directive and procedures to adopt DOD’s Cost/Schedule Management Guide on all NASA contracts. The new directive established a set of performance measurement system criteria for NASA that were identical to those used by DOD. The intent of standardizing NASA and DOD requirements was to enable NASA contractors to use DOD processes and management reports and save the cost of creating and operating separate systems.

When fully implemented, a performance measurement system has a detailed baseline program from which to measure whether tasks are ahead of or behind schedule, and/or under or over budget. The system is intended to provide early identification of potential cost and schedule problems and accurate information to help program managers take corrective actions. Trends can be extrapolated from the data to produce a range of cost and schedule estimates at completion for part or all of a project or its major segments.

Effective performance management requires a stable budget baseline that is consistent with an integrated master program schedule. A properly integrated time-phased budget baseline and program schedule identifies detailed work requirements or milestones against which ongoing contract performance can be measured.

The performance measurement monthly status reports compare planned budgets and schedules with actual work performed. The amount of work accomplished against the work schedule is called the “earned value.” Knowledge of the earned value is essential to monitoring progress because it provides insights into the cost and schedule status of tasks. The performance measurement status reports also highlight significant variances against the baseline cost and schedule. Variances that exceed established thresholds require analyses of the nature and cause of the problem. These analyses are required to describe the impact of the variance on the program and corrective actions taken or planned. Trend analyses of the cumulative costs and of the schedule for completing the project can be used to further assess project progress and to project the total cost and schedule to complete the project called the “estimate at completion.”

Realistic estimates at completion permit program managers to compare estimated final costs to the budgeted and contracted amounts and provide cost visibility with regard to what the actual funding requirements might be for the approved scope of work. To be meaningful, an estimate at completion should include actual costs and accomplishments to date, knowledgeable projections of what remains to be accomplished, and realistic estimates of how much the remaining work will cost. Such estimates should be examined on a monthly basis and updated when warranted.

Comments From the National Aeronautics and Space Administration

The following are GAO’s Comments on NASA’s letter dated May 16, 1996.

GAO Comments

1. NASA’s general observations are included throughout the report text to the extent appropriate. 2. NASA’s views on the adequacy of the remaining fiscal year 1996 financial reserves and the carryover of unused fiscal year 1996 reserves to fiscal year 1997 have been added to the report. 3. NASA pointed out that outstanding issues related to fees were settled with a contract modification on May 4, 1996, and stated that the cost baseline is not an unresolved issue and that costs are agreed to between the prime contractor and the product group subcontractors. The report has been revised to recognize the updated information provided by NASA. However, the annual phasing of funds is still not settled. Also, as discussed in the report, MacDonnell Douglas still professes a higher likely cost baseline than the one they are being measured against in the performance management status report. When these matters are settled, the cost baseline will be complete. 4. The updated information on earned value reporting and coordination has been added to the body of the report. 5. We added information showing a range of cost overrun estimates, from “best” to “worst” case. 6. Information regarding cost growth sharing through reductions to contractor’s fee has been added to the report.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.