In January 2004, the President announced a new “Vision for Space Exploration” calling for human and robotic missions to the Moon, Mars, and beyond. Over the next two decades, NASA plans to spend over $100 billion to develop a number of new capabilities, supporting technologies, and facilities that are critical to enabling space exploration missions. Development of the critical capabilities and technologies will be largely dependent on NASA contractors, who constitute more than two-thirds of NASA’s workforce. According to NASA officials, 87 percent of NASA’s $16.6 billion budget for fiscal year 2006 was spent on work performed by its contractors. Since 1990, we have designated NASA’s contract management as a high- risk area. This is based primarily on NASA’s lack of a modern integrated financial management system that can provide reliable information on contract spending and performance as well as NASA’s lack of emphasis on end results, product performance, and cost control. For example, our most recent high-risk report stated that while NASA has taken actions to improve its contract management function, it continues to face considerable challenges in implementing its contracts effectively. NASA is organized under four mission directorates—Aeronautics Research, Exploration, Science, and Space Operations—each of which covers a major area of the agency’s research and development efforts. The agency is composed of NASA headquarters, 10 field centers, and the contractor-operated Jet Propulsion Laboratory. NASA and other federal agencies can choose among numerous contract types for acquiring goods and services that can differ in part according to the nature of the fee that agencies offer to the contractor for achieving or exceeding specified objectives or goals. According to the FAR, a CPAF contract is appropriate to use when it is difficult to measure key elements of performance. It is widely used to procure nonroutine services such as the development of new systems. Typically, award-fee contracts emphasize several aspects of contractor performance, such as schedule performance, technical performance, and cost control. Because development and administration of award-fee contracts involve substantially more effort over the life of a contract than other types of contracts, the FAR and NASA’s Award Fee Contracting Guide specify that the expected benefits of using an award-fee contract must exceed the additional administrative effort and cost involved. The theory behind CPAF contracts is that although the government assumes most of the cost risk, it retains control over most or all of the contractor’s potential profit as leverage. On CPAF contracts, the award fee is often the only source of potential fee for the contractor. According to the NASA FAR Supplement and NASA’s Award Fee Contracting Guide, these contracts can include a base fee of anywhere from 0 to 3 percent of the estimated value of a nonservice contract. However, NASA’s regulations and guide do not allow the use of a base fee on service contracts. Table 1 shows the percentage of award fee available on the contracts we examined. (See app. II for a description of these contracts.) NASA relies heavily on CPAF contracts. This contract type accounted for 48 percent of obligated contract dollars and 7.7 percent of contract actions from fiscal years 2002 through 2004. By comparison, between fiscal years 1999 and 2003, award-fee contracts accounted for 13 percent of the contract dollars and 3.4 percent of contract actions at the Department of Defense (DOD). A CPAF contract includes an estimate of the total cost of what is being contracted for, may include a fee with a possible base amount fixed at the inception of the contract, and includes an award amount that is intended to motivate excellence in contract performance. The award fee is paid based upon the government’s periodic judgmental evaluations of contractor performance. When developing evaluation plans, NASA’s award-fee guide indicates that evaluation plans may include outcomes, outputs, inputs, or a combination of these elements. NASA’s guide expresses a preference for outcome factors. It notes that while it is sometimes valuable to consider input and output factors when evaluating contractor performance, outcome factors are better indicators of success relative to the desired result. An outcome factor is an assessment of the results of an activity compared to its intended purpose. Outcome-based factors are the least administratively burdensome type of performance evaluation factor, and should provide the best indicator of overall success. Outcome- based factors should therefore be the first type of evaluation factor considered for use, and are often ideal for nonroutine efforts. An output factor is the tabulation, calculation, or recording of activity or effort and can be expressed in a quantitative or qualitative manner. Output factors may be more desirable for routine efforts, but are administratively more burdensome than outcome factors due to the tabulation, calculation, or recording requirements. When output factors are used, care should be taken to ensure that there is a logical connection between the reported measures and the program’s mission, goals, and objectives. Input factors refer to intermediate processes, procedures, actions, or techniques that are key elements influencing successful contract performance. These may include testing and other engineering processes and techniques; quality assurance and maintenance procedures; subcontracting plans; purchasing department management; and inventory, work assignment, and budgetary controls. For CPAF contracts, NASA personnel conduct periodic, typically semiannual evaluations of contractor’s performance against the criteria specified in a performance evaluation plan. During the course of the evaluation period, performance monitors track contractor performance, and once the period is over they assess the performance and report to the performance evaluation board (PEB). The PEB considers the reports as well as any other pertinent information and prepares a report for the fee determination official (FDO) with findings and recommendations. The contractor is given an opportunity to provide a self-assessment of its performance during the evaluation period, which is often a written report. The FDO may meet with the PEB to discuss the report, after which a final determination is made in writing as to the amount of fee to be paid. The FDO provides the determination to the contracting officer and a copy of the related document to the contractor. When discussing award-fee contracts, it is important to acknowledge the acquisition environment in which they are used. Award fees are intended to motivate excellent contractor performance, which should result in excellent program outcomes. However, award fees should not be used to make up for factors internal or external to the acquisition environment that hinder the success of acquisition outcomes. These factors may include inadequate resources and financial management systems, lack of knowledge prior to starting the acquisition, or unsound acquisition practices. We have reported that in some cases, NASA’s failure to define requirements adequately and develop realistic cost estimates resulted in projects costing more, taking longer, and achieving less than originally planned. The persistence of these problems in NASA contract management is not only indicative of undisciplined processes or practices such as these, but may also reflect the fact that the design, development, and production of major space systems are extremely complex technical processes that must operate within complex budget and political processes. Even properly run programs can experience problems that may arise from unknowns, such as technical obstacles and changes in circumstances. Only a few things need to go wrong to cause major problems, and many things must go right for a program to be successful. The NASA FAR Supplement and NASA’s Award Fee Contracting Guide address many of the issues and problems identified by NASA on the use of award-fee contracts and provide criteria for appropriately using such contracts. Much of the guidance on award-fee contracting was issued in response to weaknesses in CPAF contracting practices identified by NASA internal reviews and NASA’s Office of Inspector General in the early 1990s. Those weaknesses included the awarding of excessive fees with limited emphasis on acquisition outcomes (end results, product performance, and cost control); rollover of unearned fee; use of base fee; and the failure to use both positive and negative incentives. NASA updated its award-fee guide in 1994, 1997, and 2001 to explain and elaborate on its award-fee policy. The 2001 revision also reflects the FAR’s additional emphasis on using performance-based contracts. NASA’s award-fee guide emphasizes tying fees to outcome factors. The guide states that outcome-based factors are the least administratively burdensome type of evaluation factor and should provide the best indicator of overall success. The award-fee guide warns against micromanaging performance and diluting the emphasis of criteria by spreading the potential award fee over a large number of performance evaluation factors. Instead, the guide recommends selecting broad performance evaluation factors, such as technical factors, project management, and cost control supplemented by a limited number of subfactors under these factors. Cost control is required to be a key performance evaluation factor in award-fee performance evaluation plans, largely because of past performance issues in which contractors were paid millions of dollars in fees on contracts that were experiencing hundreds of millions of dollars in cost overruns. The NASA FAR Supplement states that cost control shall be no less than 25 percent of the total weighted evaluation factors when explicit evaluation factor weightings are used. The NASA FAR Supplement states that emphasis on cost control should be balanced against other performance requirement objectives, and the contractor should not be incentivized to pursue cost control to the point that overall performance is significantly degraded. NASA’s regulations prohibit rolling over unearned fee to subsequent evaluation periods for service contracts. For such contracts, each interim evaluation and the last evaluation are final. Another key element of the current award-fee regulations is an increased emphasis on overall contractor performance and the end product, rather than on incremental progress. NASA requires conducting interim evaluations on end item contracts until final product delivery to monitor performance prior to contract completion and establish the basis for making interim payments. At the end of the contract, a final evaluation is conducted and the contractor’s total performance is evaluated against the award-fee plan to determine total earned award fee. For example, the contractor may be evaluated and paid an interim fee once every 6 months until the product is delivered. During the final evaluation, the contractor’s performance is evaluated to determine total earned award fee. The final evaluation may result in the contractor retaining the fee previously awarded or receiving additional or less fee than previously awarded and thus refunding a portion of the fee to the government. The final evaluation provides NASA the opportunity to make an award-fee decision based on actual quality, total cost, and ability to meet the contract schedule at the point the final product is delivered. Further, under the award-fee policy in effect prior to the 1994 and subsequent revisions to the guidance, base fee was allowed on all CPAF contracts. NASA’s current regulations prohibit the use of base fee on service contracts and restrict the use of base fee on end item contracts, such as for hardware. When base fee is used, it is not to exceed 3 percent of estimated contract costs and it should only be paid if the final award-fee evaluation is satisfactory or better. We note that base fee, which was paid on two of the three end item contracts we reviewed, did not exceed 3 percent, and none of the seven service contracts included base fee. Another issue addressed by NASA’s regulations is the use of both positive and negative performance incentives in its CPAF contracts. The NASA FAR Supplement provides that award-fee contracts with primary deliverables of hardware and with a total estimated cost and fee of greater than $25 million require both kinds of incentives based on measurements of hardware performance against objective criteria. Performance incentives are separate and distinct from award fee and measure contractor performance up to delivery and acceptance. Performance incentives are designed to reward contractors when performance of delivered hardware is above minimum contract requirements. For example, if the government establishes a specified level of objective performance for a product that the contractor exceeds, the contractor can be paid a performance incentive in addition to the award fee already paid. If the contractor just meets this measure, it cannot receive an additional performance incentive and keeps the award fee already paid. If the contractor fails to meet the measure, however, it must pay a negative performance incentive fee that reduces or eliminates the entire award fee. To address inconsistencies among NASA centers in how they evaluate contractor performance, the current award-fee regulations also provide a uniform rating system to be used for all NASA award-fee contracts. It includes adjectival ratings as well as a numerical scoring system of 0-100. Scores of 61-70 percent are considered satisfactory, and the regulations specify that contractors receiving a rating of less than 61 percent will not receive any fee. A contractor is not to be paid any base fee or award fee for less than satisfactory overall performance. NASA’s award-fee guide encourages the use of performance-based contracts for the procurement of services and supplies. The guide states that constructing performance-based contracts that clearly define performance requirements, include easily understood performance standards, and have an objective incentive mechanism will result in contractors having a clearer understanding of the government’s expectations and will ultimately facilitate enhanced contractor performance. Finally, because of the cost and administrative burden associated with administering award-fee contracts, the FAR and NASA’s award-fee guide specify consideration of the costs and benefits of using a CPAF contract before committing to this contract type. Through an evaluation of the administrative costs versus the expected benefits, the contracting officer should be able to assess whether the benefits the government gains through a CPAF contract will outweigh the additional costs of overseeing and administering the contract. The award-fee guide provides an example of how to calculate the administrative cost and states that benefits could be measured in dollars saved through cost control or enhanced technical capability. Although the revisions in NASA’s regulations and guidance on award-fee contracts address many weaknesses previously identified, the contracts that we reviewed did not always demonstrate use of award fees by the centers in the way that NASA prefers as outlined in its guidance. Some performance evaluation plans or reports included input evaluation factors, which are not the best indicators of success relative to the desired result, although they are allowed by the guidance. Other contracts included numerous subcategories for evaluating the contractor that can lessen the importance of any particular subcategory and reduce the leverage of the award fee on any particular criterion. Also, although the FAR and NASA’s award-fee guide calls for a consideration of the costs and benefits of using cost-plus-award-fee contracts because of the cost and administrative burden involved, we found no examples of a documented analysis of costs and benefits. Finally, NASA officials expressed satisfaction with the results of the contracts based on their evaluations of contractor performance against criteria established in the award-fee plan. Those evaluations would indicate generally good performance. However, that performance did not always translate into desired program outcomes. NASA paid a majority of the available award fee on all of the contracts we reviewed, including those end item contracts that did not deliver a capability within initial cost, schedule, and performance parameters. That disconnect raises questions as to the extent NASA is achieving the effectiveness it sought through the establishment of guidance on the use of award fees. Further, NASA has not evaluated the overall effectiveness of award fees in promoting program outcomes and does not have metrics in place for measuring their effectiveness in achieving program outcomes. Some performance evaluation subfactors included in performance evaluation plans or reports were not outcome oriented. NASA’s award-fee guide states that while it is sometimes valuable to consider input and output factors when evaluating contractor performance, it is NASA’s preference when feasible to tie fees to evaluation factors that are based on outcomes because outcome-based factors provide the best indicator of overall success. The award-fee guide recommends selecting broad performance evaluation factors, such as technical factors, project management, and cost control, and cautions that factors related to intermediate processes, procedures, and actions may cause the contractor to divert its attention from the overall desired outcome. The guide states that those types of factors, while allowed, are not always true indicators of the contractor’s performance and should be relied on with caution. Further, with service contracts, input factors may be of little or no value as a basis for evaluation. While the contracts we reviewed generally used outcome factors as part of the evaluation of performance, some supporting subfactors that formed the basis of the ratings for performance measured compliance with process or input factors that may not provide the best indicators of success relative to the desired results. For example, a part of the award fee on the Mechanical System Engineering Services (MSES) contract was to be awarded for program and business management performance. There were five subfactors under this primary performance factor. Two of these subfactors, program planning and organizational management and business management were input subfactors. These two input subfactors measure contractor processes or inputs, but do not focus on final results. Subfactors in the Landsat-7 contract included input subfactors such as responsiveness of the contractor’s corporate management, quality and effectiveness of the contractor’s scheduling system, and prudent utilization of manpower and timely removal of manpower upon completion of tasks. The NASA award-fee guide cautions that spreading the potential award fee over a large number of performance evaluation factors dilutes emphasis on any particular performance evaluation criterion, increases the prospect of any one item being too small and thus overlooked, and increases the administrative burden. It encourages broad performance evaluation factors such as technical factors, project management, and cost control, which should be supplemented by only a limited number of subfactors describing significant evaluation elements over which the contractor has effective management control. Our analysis showed that a large number of subfactors were used to evaluate contractor performance for some contracts. For example, the Jet Propulsion Laboratory (JPL) contract, which includes both service and product deliverables defined in task orders under the contract, uses three primary performance evaluation factors for measuring contractor performance—programmatic, scientific, and engineering; institutional management; and support to outreach initiative programs. Although the JPL performance evaluation plan characterizes award-fee subfactors as representing major areas of emphasis during the performance period, the award-fee subfactors used to support the broad performance evaluation factors were numerous—96 subfactors were used to evaluate the contractor’s performance in fiscal year 2004, and 108 subfactors were used in fiscal year 2005. The Engineering and Technical Support for Life Sciences contract used three broad performance evaluation factors also—technical performance, schedule performance and contract management, and cost control—but evaluated the contractor on numerous supporting subfactors identified as tasks or subtasks in the contractor performance evaluation reports. For example, on one task order under this contract, performance evaluation reports for various evaluation periods showed as many as 50 different subtasks used to evaluate the contractor’s performance for the primary evaluation criteria: (1) technical performance and (2) schedule performance and contract management. The Landsat-7 contract also included a number of subfactors. Contractor performance under this contract was evaluated in several different areas each time the performance evaluation board met. Technical performance and program management were grouped together in one primary performance evaluation factor, and business management and cost performance were grouped together in the other primary performance evaluation factor. There were 9 subfactors under technical performance and 12 subfactors under program management, including quality and effectiveness of the contractor’s scheduling system. Under business management and cost performance, 17 evaluation subfactors and elements were to be considered, including compliance with general contract provisions and clauses and weekly scheduling of teleconferences to determine schedule status. In addition to the number of subfactors that fell under the two primary performance evaluation factors, there were nine additional evaluation criteria, including resourcefulness, communication, and responsiveness. Although the FAR and NASA’s award-fee guide require consideration of the costs and benefits of using a CPAF contract before committing to this contract type to determine whether the benefits outweigh the additional cost and administrative burden of managing the contract, we found no instances where a documented cost-benefit analysis had been done for any of the contracts under our review. According to the guidance, since award- fee contracts require additional administrative effort, they should be used only when the contract values, performance period, and expected benefits are sufficient to warrant that additional management effort. Careful selection of the most appropriate contract type and careful tailoring should prevent a situation in which the burden of administering the award fee is out of proportion to the improvements expected in the quality of the contractor’s performance and in overall management. In addition, CPAF contracts can be particularly costly and burdensome for NASA to administer because of contract reporting and review requirements. Major cost drivers include the number of award-fee periods, performance monitors, and performance evaluation board members necessary for implementing the award-fee process. For example, according to NASA’s Award Fee Contracting Guide’s conservative estimate, it would cost about $387,000 to administer the award-fee process over the life of a 5-year contract. The guide notes that the estimate does not represent all associated administrative cost that might arise. Although NASA procurement officials acknowledged that formal cost-benefit analyses were not prepared, some officials referred to determination and findings statements or acquisition strategy meeting documents associated with specific contracts as providing some evidence of consideration given to whether or not CPAF contracts should be used. While NASA officials expressed satisfaction with the results of the contracts, in some cases there appeared to be a disconnect between the fee paid and program results. NASA paid most of the available fee on all of the contracts we reviewed—including on projects that showed cost increases, schedule delays, and technical problems. The total estimated value of the 10 contracts we reviewed was more than $31 billion. NASA paid between 80 and 99 percent of the maximum award fee possible on these contracts. The average was 90 percent, which equated to almost a billion dollars in total award fees paid under the 10 contracts. Table 2 shows the percentage of award fee paid for each of the 10 contracts we reviewed. NASA officials expressed satisfaction with contract results, which was further evidenced by its evaluations of contractor performance against criteria established in the award-fee plan. While NASA’s evaluations would indicate generally good performance, such performance did not always translate into desired program outcomes. That disconnect raises questions as to the extent NASA is achieving the effectiveness it sought through the establishment of guidance on the use of award-fees. On the end item contracts we reviewed, although there were some periods in which NASA paid a lesser percentage of the available fee, NASA ultimately paid more than 90 percent of the available fee based on its evaluation of contractor performance against criteria in the award-fee plan even when those contracts did not deliver capability within initial cost, schedule, and performance parameters. For example: The prime contractor for the International Space Station (ISS) has received 92 percent of the total award fee available—$425.3 million— although the cost increased by 131 percent, from $5.6 billion to $13 billion, in part due to increased contract scope and delays caused by the Columbia accident, but also contractor cost overruns. In addition, the contractor estimates that it will incur an additional $76 million in overruns by the time the contract is completed. Further, the completion date for space station assembly under the prime contract was delayed by 8 years. In some cases these delays were caused by actions not within the control of the contractor, such as problems with the shuttle program and actions by the international partners. The contractor for the Earth Observing System Data and Information System (EOSDIS) Core System (ECS) was paid 97 percent of the available award fee—$103.2 million—despite a delay in the completion of the contract by more than 2 years and an increase in the cost of the contract from $766 million to $1.2 billion. Technical problems, schedule delays, and cost control problems led to a major restructuring of the contract. The Landsat-7 contractor was paid 99 percent of the available award fee or more than $17 million. The original contract was managed by the Air Force but was subsequently transferred to NASA and rebaselined. The cost of the contract when transferred to NASA and rebaselined was $342.7 million. The Landsat-7 launch was delayed by 9 months and although the original scope of the work under the contract was significantly reduced, the cost of the contract increased. By the time the contract was complete, costs had risen 20 percent to $409.6 million. While some NASA officials pointed out that problems encountered on these contracts were at times outside the control of the contractor, difficulties such as these with achieving program results have resulted in NASA contract management being considered a high-risk area by GAO. We did not review these contracts to determine responsibility for undesirable results and therefore make no conclusion as to whether the fee paid was appropriate on each particular contract. However, the high fees paid on contracts where programs experienced disappointing results raise questions as to the effectiveness of award fees as a tool for obtaining desired program outcomes. For the service contracts we reviewed, NASA officials reported that they were satisfied with the results and quality of services provided. While we could not assess these contracts against cost, schedule, and performance outcomes as we could with the end item contracts, we did assess the award-fee criteria used in these contracts against NASA guidance. Here we found instances of process and input-oriented subfactors and the inclusion of numerous subfactors in evaluating performance. Further, we found no evidence that a cost-benefit analysis had been performed prior to choosing the contract type. Taken together, this is not the preferred approach according to NASA guidance, which raises questions as to the degree to which performance outcomes—getting the quality of service desired—was actually the basis for judging contractor performance and awarding fee. NASA views CPAF contracts as a viable and often preferred mechanism for acquiring the types of goods and services that the agency procures. NASA’s satisfaction with the results of these contracts is evidenced by the level of fee paid on all of the contracts we reviewed and is based on NASA’s evaluation of compliance with criteria contained in its award-fee plans. However, the agency has not evaluated the overall effectiveness of award fees in promoting desired outcomes. As noted, NASA developed its new policies on award-fee contracts because the agency and its Office of Inspector General found that it was paying excessive fees with limited emphasis on acquisition outcomes. However, according to NASA officials, the agency has not completed any assessments of the effectiveness of award fees since the award-fee policy was restructured in the 1990s, nor has it developed metrics or performance measures to conduct such evaluations. Further, NASA lacks an agencywide system with the capability of compiling and aggregating award-fee information and for identifying trends and outcomes. According to NASA officials, even NASA’s modern Integrated Enterprise Management Program (IEMP) will not provide this capability. Thus, NASA cannot meaningfully judge how well award fees are improving or can improve contractor performance and program outcomes. NASA could better link its award fees to desired results by making greater use of outcome factors, its preferred criteria for evaluating award fee contracts. While NASA has established policies and guidance that provide an appropriate framework for their use, the agency has not always used award fees as preferred by its guidance. To the extent that NASA uses input evaluation factors and numerous subfactors for evaluating performance, NASA may be diluting the leverage of award fees in achieving desired results. Our review raises questions as to the extent NASA is achieving the effectiveness it sought through the establishment of guidance on the use of award fees. However, NASA has not evaluated the overall effectiveness of its implementation of award fees. We are making the following three recommendations to increase the likelihood that the award fees NASA pays incentivize high performance from its suppliers. We recommend that the NASA Administrator reemphasize to the NASA centers the importance of tying award-fee criteria to desired outcomes and limiting the number of subfactors used in evaluations. To ensure that cost-plus-award-fee contracts are used only when their benefits outweigh the costs, we recommend that the NASA Administrator direct the centers to consider costs and benefits in choosing this contract type by requiring documentation explaining how the perceived benefits will offset the additional cost associated with its administration as required by the FAR. Finally, we recommend that the NASA Administrator require the development of metrics for measuring the effectiveness of award fees, establish a system for collecting data on the use of award-fee contracts, and regularly examine the effectiveness of award fees in achieving desired acquisition outcomes. In commenting on a draft of this report, NASA concurred with our recommendations and indicated that it would reemphasize its current guidance as recommended, address the issues raised by the report in training, and cover those issues in its internal reviews of procurement operations at the individual Space Centers. In terms of our recommendation to develop metrics for measuring the effectiveness of award fees and establish a system for collecting data on the use of award- fee contracts, NASA concurred and indicated it would explore the best way to develop and use metrics for evaluating the effectiveness of award fees and set up a system for collecting data on award-fee contracts. NASA said it planned to contact the Department of Defense to obtain information on its process, since DOD is also developing such a data collection system and metrics for measuring the effectiveness of award fees. NASA also provided technical comments on the draft, which have been incorporated as appropriate. As agreed with your office, unless you announce its contents earlier, we will not distribute this report further until 30 days from its date. At that time, we will send copies to interested congressional committees and the NASA Administrator. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or calvaresibarra@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are acknowledged in appendix IV. Our objectives were to determine (1) the extent the National Aeronautics and Space Administration’s (NASA) guidance addresses the problems previously identified with the use of award-fee contracts and (2) whether NASA follows its guidance in using award fees to achieve desired outcomes. We selected 10 NASA cost-plus-award-fee (CPAF) contracts to review. Our selection was based on contract data from the Federal Procurement Data System. We extracted information on all NASA contracts active between fiscal years 2002 and 2004 that were coded as CPAF. To ensure the validity of the database from which we drew our contracts, we confirmed the contract type of each of the 10 contracts we selected through NASA contracting officers and contract documentation. The contracts we selected were the top 10 dollar value contracts active from fiscal years 2002 through 2004. These contracts account for about $7.6 billion, or 44 percent, of obligated cost-plus-award-fee-dollars for the 3-year period. To determine the extent NASA’s guidance addresses the problems previously identified with the use of award-fee contracts and whether NASA follows its guidance in using award fees to achieve desired outcomes, we interviewed responsible program and procurement officials at NASA headquarters and six NASA centers. We also reviewed the Federal Acquisition Regulation (FAR), the NASA FAR Supplement, and NASA’s Award Fee Contracting Guide. We conducted a literature review and examined previous reports, studies, and analyses done by GAO, NASA, the NASA Inspector General, or others that included information related to NASA’s use of award fees and other relevant issues. Additionally, we reviewed contract files, obtained information from program and contracting officials through the use of a structured questionnaire, and discussed the application of award-fee criteria with NASA officials involved in the award-fee process. The contract documents we reviewed contained information related to the development and implementation of the award fee. This information included the basic contract and statement of work, acquisition planning documents, award- fee modifications, performance evaluation plan documentation describing fee criteria for specific evaluation periods, contractor self-assessments, performance evaluation board reports, and fee determination documents. We used this information to corroborate and supplement the information provided by NASA officials in response to structured questionnaires we prepared and interviews we conducted. We e-mailed the questionnaires and received written responses for all 10 of the contracts. We conducted structured interviews with contracting and program officials concerning the development, implementation, and effectiveness of the award-fee structure for some of the contracts. To accomplish our work, we visited NASA headquarters in Washington, D.C. We also visited and held teleconferences with Goddard Space Flight Center in Greenbelt, Maryland, responsible for managing 3 of the contracts we reviewed; Johnson Space Center in Houston, Texas, responsible for managing 3 of the contracts; and Marshall Space Flight Center in Huntsville, Alabama, responsible for managing 1 of the contracts. We held teleconferences with officials at the Jet Propulsion Laboratory in Pasadena, California; Kennedy Space Center in Cape Canaveral, Florida; and Ames Research Center in Moffett Field, California, responsible for managing 1 contract each under our review. Our work was conducted from August 2005 through October 2006 in accordance with generally accepted government auditing standards. NAS5-60000 was an end item hardware cost-plus-award-fee contract between NASA and Hughes Applied Information Systems Incorporation. Raytheon Information Systems Company acquired Hughes in December 1999 and became the prime contractor. The contract, currently closed, was managed by Goddard Space Flight Center. The 10-year research and development contract was awarded in March 1993 for the development and operation of the Earth Observing System Data and Information System Core System. The period of performance on the contract actually ended in April 2005, and the contract has since been closed. According to Goddard Space Flight Center procurement officials, the desired program outcome or objective of the contract was to develop a technically capable system to process data from NASA’s satellites at a reasonable cost. Procurement officials stated that the Earth Observing System Data and Information System Core System, a state-of-the-art data-processing system, is currently dedicated to the processing and dissemination of NASA Earth Science satellite data. NAS15-10000 is an end item hardware cost-plus-award-fee contract between NASA and the Boeing Company. The contract, currently active, is managed by the Johnson Space Center. A letter contract was awarded in November 1993 and was definitized in January 1995 as a cost-plus- incentive-fee award-fee contract. In October 1999, during a restructuring of the contract, the cost-plus-incentive-fee award-fee contract was converted to a cost-plus-award-fee contract. The contract was extended in December 2003, partially because of the Columbia accident. This planned 10-year contract is for the design, development, manufacture, and on-orbit assembly of the U.S. on-orbit segment of the International Space Station. The contract also included provisions for a level of effort that included (1) sustaining engineering, (2) multi-element integrated testing, (3) logistics and maintenance–post production support, (4) technical definition of contract changes, and (5) other engineering support. According to Johnson Space Center procurement officials, the desired program outcomes or objectives of this contract are (1) completion of the U. S. on- orbit segment, delivery, and on-orbit acceptance of the space station; (2) sustaining engineering of the U.S. on-orbit segment hardware and software and common hardware and software provided to international partners/participants and payloads; (3) post-production support of the U.S. on-orbit segment hardware and common hardware provided to the international partners/participants; and (4) space station end-to-end subsystems management for the majority of the subsystems and specialty engineering disciplines. NAS5-32633 was an end item hardware cost-plus-award-fee contract between NASA and Lockheed Martin Missiles and Space. The contract, currently closed, was managed by Goddard Space Flight Center. The research and development contract was initially awarded by the Air Force in October 1992 and transferred to NASA in May 1994. The contract was for the design, development, fabrication, integration, test, and pre- and post-launch support of the Landsat-7 spacecraft. Landsat-7 was launched in April 1999; the contract was completed in 2005. The purpose of the Landsat-7 satellite is to obtain continuous remotely sensed, high-resolution imagery of the earth’s surface for environmental monitoring, disaster assessment, land use and regional planning, cartography, range management, and oil and mineral exploration. According to Goddard Space Flight Center procurement officials, the desired program outcome or objective of the contract was to develop an operational satellite that met the science requirements of users and the laws requiring the data be obtained at a reasonable cost. NAS8-60000 was a cost-plus-award-fee service contract between NASA and Computer Sciences Corporation. The contract, managed by the Marshall Space Flight Center, was in the process of being closed as of June 2006. It was awarded in May 1994, and covered a 2-year period of performance, but included options to extend the period of performance for an additional 6 years—through April 30, 2002. The contract was extended three times, with the period of performance ending on March 30, 2004. The primary purpose of the contract was to provide services in the area of program information system mission services. The contractor’s responsibilities were to manage, be responsible for, and provide information services to meet requirements of the Information Systems Services Office and its customers. According to Marshall Space Flight Center procurement officials, the desired program outcome or objective of the contract was to provide services including operating and maintaining existing equipment and software; gathering, analyzing, defining, and documenting systems requirements; and planning, designing, developing, acquiring, integrating, testing, and implementing new systems or enhancements to existing systems. NAS2-14263 was a cost-plus-award-fee service contract between NASA and Lockheed Martin Engineering and Science Company, defined under task orders. The contract, managed by Ames Research Center, was in the process of being closed as of June 2006. Its period of performance ended in September 2003. The 5-year research and development contract was awarded in May 1995 for the provision of engineering and technical support services for Ames Research Center life sciences. The work to be performed included engineering and technical support for life sciences projects, including space shuttle life sciences payloads, other life science payloads, the Space Station Biological Research Project, ground-based life sciences research, and advanced life support technology development. According to Ames Research Center procurement officials, the desired program outcome or objective of the contract was to achieve support for space life science projects, life sciences research, and related technology. NAS9-19100 was a cost-plus-award-fee service contract between NASA and Lockheed Martin with indefinite delivery, indefinite quantity task orders; performance-based; and level-of-effort provisions. Following the merger of Lockheed and Martin in 1995, NASA consolidated two existing contracts to form NAS9-19100 with an effective date of October 1, 1996. The contract, managed by Johnson Space Center, was in the process of being closed as of June 2006. The period of performance ended in January 2005. The contract included requirements related to hardware, government- furnished crew equipment, facilities, laboratory maintenance, life sciences, flight hardware, and support for the science and engineering requirements of the Space Shuttle Program and the International Space Station Program. According to Johnson Space Center procurement officials, the desired program outcomes or objectives of the contract were to provide engineering and science support to all engineering directorates at Johnson Space Center as well as support both the science and engineering requirements of the shuttle and space station programs. NAS9-98100 was a cost-plus-award-fee service contract between NASA and Lockheed Martin Space Operations Company, with task orders and level- of-effort provisions. The contract, which was in the process of being closed as of June 2006, was managed by the Johnson Space Center. It was awarded on September 25, 1998, with a basic 5-year period of performance and an option for an additional 5-year period. NASA chose not to exercise the option for the second 5-year period of performance. The contract required (1) developing an integrated operations approach to spacecraft design, operations, and data processing that minimized cost and the support infrastructure required to conduct space operations; (2) obtaining a highly capable and accountable contractor that would be responsible for providing space operations mission and data services; and (3) providing a contract and management structure that would enable outsourcing, commercialization, or privatization of some or all service under the contract. According to Johnson Space Center procurement officials, the desired program outcomes or objectives of the contract were to (1) provide excellent quality and reliable mission and data services at a significantly reduced cost; (2) move end-to-end mission and service responsibility and accountability to industry; (3) implement an integrated architecture that reduces overlap, eliminates unnecessary duplication, and reduces life cycle costs; (4) define streamlined processes that minimize intermediaries required to define requirements and deliver services; and (5) adopt private sector commercial practices and services. NAS10-99001 is a cost-plus-award-fee service contract between NASA and Space Gateway Support. The contract, currently active, is managed by Kennedy Space Center. The contract was awarded on October 1, 1998, for a basic 5-year period of performance and included an option for an additional 5 years. NASA exercised that option on October 1, 2003. The purpose was to provide for base operations support at NASA’s Kennedy Space Center and the Air Force’s Cape Canaveral Air Force Station, as well as specific requirements at Patrick Air Force Base and Florida Annexes into one consolidated contract. In addition to NASA and the Air Force, other primary customers include the Navy, Department of Interior, Spaceport Florida, and commercial customers such as Boeing, Lockheed Martin, Orbital Science, and Astrotech. According to Kennedy Space Center procurement officials, the desired program outcomes or objectives of the contract are to (1) enhance safety for the public and on-site workforce; (2) provide protection of human, national, and environmental resources; (3) provide high-quality and responsive service to customers; (4) reduce the cost of doing business for NASA and the Air Force; (5) provide flexibility to respond to new requirements and unplanned events; (6) improve supportability and reliability through innovative methodologies and concepts; (7) provide common support practices and systems; and (8) increase small business subcontracting goals. NAS5-01090 is a cost-plus-award-fee service contract between NASA and Swales and Associates, with a line item for indefinite delivery, indefinite quantity task orders. The contract, currently active, is managed by Goddard Space Flight Center. NAS5-01090 was awarded in January 2001 with a period of performance of 5 years and 30 days. According to Goddard Space Flight Center procurement officials, the period of performance was extended and was currently scheduled to end on August 15, 2006. The purpose of the contract is to provide engineering services for the study, design, development, fabrication, integration, testing, verification, and operations of space flight and ground system hardware and software, including development and validation of new technologies to enable future science missions. According to Goddard Space Flight Center procurement officials, the desired program outcomes or objectives of the contract were to obtain high-quality performance, desired results, and output. NAS7-03001 is a cost-plus-award-fee contract between NASA and the California Institute of Technology, a private nonprofit educational institution, which establishes the relationship for the operation of the Jet Propulsion Laboratory (JPL) federally funded research and development center. The contract, currently active, is a 5-year research and development contract awarded in November 2002 for the operation and management of JPL. The contract allows for extension or decrease to the initial period of performance in 3- or 9-month increments. JPL is a NASA- owned facility as well as an operating division of Caltech. Caltech has operated JPL as a federally funded research and development center since 1959 to meet certain government research and development needs, which, according to the contract, could not be met as effectively by existing government resources or normal contractor relationships. The contract includes both service and product deliverables, which are defined in task orders issued under the contract. The contract encompasses a large number of discrete programs and projects—approximately 500 active task orders. According to NASA procurement officials, the desired program outcomes or objectives of the contract are specific performance requirements defined in task orders issued under the contract. The contract encompasses support of exploration of the solar system, including earth-based investigations, investigations and studies to support NASA missions in the fields of earth science and astrophysics and astrobiology, as well as development of supporting fundamental technologies. In addition to the individual named above, Thomas Denomme, Assistant Director; James Beard; Shirley Johnson; Julia Kennon; Heather Barker Miller; Kenneth Patton; Sylvia Schatz; and Robert Swierczek made key contributions to this report.

Summary: Cost-plus-award-fee contracts accounted for almost half of the National Aeronautic and Space Administration's (NASA) obligated contract dollars for fiscal years 2002-2004. Since 1990, we have identified NASA's contract management as a high-risk area--in part because of a lack of emphasis on end results. Congress asked us to examine (1) the extent NASA's guidance on award fees addresses problems previously identified with the use of award-fee contracts and (2) whether NASA follows its guidance in using award fees to achieve desired outcomes. We reviewed the top 10 dollar value award-fee contracts active from fiscal years 2002 through 2004. NASA guidance on the use of cost-plus-award-fee (CPAF) contracts provides criteria to improve the effectiveness of award fees. For example, the guidance emphasizes outcome factors that are good indicators of success in achieving desired results, cautions against using numerous evaluation factors, prohibits rollover of unearned fee, and encourages evaluating the costs and benefits of such contracts before using this contract type. However, NASA does notalways follow the preferred approach laid out in its guidance. For example, some evaluation criteria contained input or process factors, such as program planning and organizational management. Moreover, some contracts included numerous supporting subfactors that may dilute emphasis on any specific criteria. Although the Federal Acquisition Regulation and NASA guidance require considering the costs and benefits of choosing a CPAF contract, NASA did not perform such analyses. In some cases there appears to be a significant disconnect between program results and fees paid. For example, NASA paid the contractor for the Earth Observing System Data and Information System Core System 97 percent of the available award fee despite a delay in the completion of the contract by over 2 years and an increase in the cost of the contract of more than 50 percent. NASA officials expressed satisfaction with the results of the contracts we reviewed, and this was further evidenced by the extent of fee paid. NASA's satisfaction was based on its evaluations of contractor performance against criteria established in the award-fee plan. While NASA's evaluations would indicate generally good contractor performance, that performance did not always translate into desired program outcomes. That disconnect raises questions as to the extent NASA is achieving the effectiveness it sought through the establishment of guidance on the use of award fees. NASA has not evaluated the overall effectiveness of award fees and does not have metrics in place for conducting such evaluations.