SEPTEMBER 25, 2009

AUDIT REPORT

REPORT NO. IG-09-022-Redacted (ASSIGNMENT NO. A-08-016-00)

OFFICE OF AUDITS

NASA SHOULD RECONSIDER THE AWARD 

EVALUATION PROCESS AND CONTRACT TYPE FOR THE 

OPERATION OF THE JET PROPULSION LABORATORY 

OFFICE OF INSPECTOR GENERAL

National Aeronautics and 

Space Administration

 
REPORT NO. IG-09-022-R 

Note: This redacted version of the report omits predecisional information that is exempt 

from public release under the Freedom of Information Act, exemption (b)(5). 

Final report released by:

 signed

Debra D. Pettitt

Acting Assistant Inspector General for Auditing 

Acronyms 

CPAF Cost-Plus-Award-Fee

CPFF Cost-Plus-Fixed-Fee

CPM Contract Performance Monitor 

ESMD Exploration Systems Mission Directorate 

FAR Federal Acquisition Regulation

FDO Fee Determination Official

FFRDC Federally Funded Research and Development Center

FY Fiscal Year

GAO Government Accountability Office 

JET JPL Evaluation Tool 

JPL Jet Propulsion Laboratory 

MEP Mars Exploration Program 

MRO Mars Reconnaissance Orbiter

MSL Mars Science Laboratory

NFS NASA FAR Supplement

NMO NASA Management Office

OIG Office of Inspector General

PEB Performance Evaluation Board

PPE Primary Performance Evaluator

SMD Science Mission Directorate

SOMD Space Operations Mission Directorate 
SEPTEMBER 25, 2009

REPORT NO. IG-09-022-R 

OVERVIEW

NASA SHOULD RECONSIDER THE AWARD EVALUATION 

PROCESS AND CONTRACT TYPE FOR THE OPERATION OF THE 

JET PROPULSION LABORATORY

The Issue

The Jet Propulsion Laboratory (JPL) is a NASA federally funded research and 

development center (FFRDC) operated under contract by the California Institute of 

Technology (Caltech), a private nonprofit educational institution. The NASA 

Management Office (NMO) at JPL oversees the contract. Since 1993, NASA has 

awarded three cost-plus-award-fee (CPAF) contracts1

The current 5-year CPAF contract, awarded to Caltech in November 2002, is valued at 

approximately $7.5 billion

 to Caltech for the operation of JPL. 

Work under these contracts has included accomplishments in astrophysics, earth sciences, 

solar system exploration, and technology. Examples since 2003 include “Spirit” and 

“Opportunity,” the Mars Exploration Rovers; the Spitzer Space Telescope; the 

Microwave Limb Sounder; and the Tropospheric Emission Spectrometer. 

2

We initiated this audit to determine whether NASA appropriately calculated and justified 

the award fees and term extensions earned by Caltech for the operation of JPL. (See 

Appendix A for details of the audit’s scope and methodology.) 

—$1.5 billion per year, including available award fees of 

$22 million. The current contract also includes an award term incentive, allowing

Caltech to earn term extensions on the contract in increments of 3 or 9 months for up to 

an additional 5 years. The potential total value of these incentives is $7.5 billion. Award 

fee and award term determinations for the JPL contract are made by the NASA Associate 

Administrator based on annual evaluations of the contractor’s performance against

criteria established in the NASA “Award Fee Contracting Guide,” June 27, 2001. 

 

1 The Federal Acquisition Regulation, Subpart 16.305, “Cost-Plus-Award-Fee Contracts,” defines a CPAF 

contract as a cost-reimbursement contract that provides for a fee consisting of (a) a base amount (which 

may be zero) fixed at inception of the contract, and (b) an award amount, based on a judgmental 

evaluation by the Government, sufficient to provide motivation for excellence in contract performance.

2 The contract states that the estimated cost of the contract will be “the sum of the estimated costs set forth 

in task orders issued.” 
OVERVIEW

ii REPORT NO. IG-09-022-R 

Results 

NASA can improve its management of the JPL contract:

• NASA’s overall assessment of contractor performance may have been overstated 

because the Agency’s performance evaluations for fiscal year (FY) 2007 were 

incomplete or did not otherwise comply with its guidance. 

• The contractor’s poor performance on a large, significant project was offset in 

NASA’s assessment by higher performance on smaller projects because the 

Agency did not use proportional weighting in its evaluations.

• NASA’s award of $16 million in fees and 27 months of contract term extensions, 

valued at $3.375 billion, were unsupported because the Agency’s performance 

evaluation factors did not include an assessment of required cost control 

measures.

• NASA does not have assurance that the existing contract still meets its needs or 

provides the best value for the taxpayer because the Agency did not fully comply 

with Federal Acquisition Regulation (FAR) requirements for a 5-year 

comprehensive review of the use and need for the FFRDC. 

• NASA’s use of a single CPAF contract for all aspects of the FFRDC creates a 

significant administrative and management burden for the Agency that is 

unnecessary given that there is a basis for the contracting officer to establish fair 

and reasonable prices for routine operations and maintenance of the facility. 

The Assistant Administrator for Procurement and the NMO concurred with our 

recommendations to address these issues and agreed that there are potential monetary 

benefits associated with their implementation.

We found that performance evaluations and the method used to calculate the award fee 

score for Caltech’s FY 2007 performance may not accurately reflect the overall 

performance of the contractor (Finding A). For the period of performance ending 

September 30, 2007, this resulted in potentially higher ratings, which could have led to 

NASA inappropriately paying award fees and awarding a term extension. Although the 

criteria used for evaluating the contractor’s performance were sufficiently specific and 

measurable, NASA evaluators provided incomplete assessments or assigned 

inappropriate ratings. For example, the evaluation for one criterion stated that the 

contractor did not meet the Satisfactory or Excellent standards due to cost; however, the 

evaluator provided a rating of Good,3

 

3 NASA guidance requires adjectival ratings (Excellent, Very Good, Good, Satisfactory, 

Poor/Unsatisfactory) to be used for award-fee contracts.

 stating that he did not want to penalize the 
OVERVIEW

REPORT NO. IG-09-022-R iii

contractor’s technical performance for poorer cost performance. In addition, there is no 

documentation to indicate that criteria were proportionately weighted by a project’s 

overall importance to NASA or by project cost, which allowed exceptional contractor 

performance on smaller projects to conceal poorer performance on a larger, more 

significant project. 

In addition, the performance evaluation factors did not include cost control measures 

weighted at no less than 25 percent of the total weighted evaluation factors, as required by 

NASA guidance. We calculated that cost control measures were weighted between 

5 percent and 14 percent of the total (Finding B). The low percentage weighting 

deemphasized the importance of controlling cost, minimized the effectiveness of cost 

control, and gave the contractor minimal incentive to control costs. As a result, a portion 

of Caltech’s award fees were unsupported costs and an ineffective use of Government 

funds. Specifically, we consider $16 million of the $97 million paid in award fees for 

Caltech’s performance under the 2003 contract to be unsupported. In addition, we 

question the 27 months in term extensions, valued at approximately $3.375 billion, 

awarded to Caltech. 

We also found that NASA did not adequately consider alternative sources for the 

operation of JPL as an FFRDC and could not provide support to justify its decision to 

continue procuring the services of Caltech (Finding C). The FAR requires a 

comprehensive review of the use and need for FFRDCs every 5 years and provides five 

criteria for the review, which is designed to consider alternative sources to meet the 

sponsor’s needs and to ensure that the contractor is providing the best value for the 

taxpayer. However, the review NASA conducted in 2002 prior to awarding the most 

recent contract for operating its FFRDC, in 2003, was less than exhaustive in its search 

for viable competitors. A review conducted in 2008 addressed four of the five FAR 

criteria, but did not include consideration of alternative sources. NMO Procurement 

Office personnel decided to postpone the search for alternative sources until the current 

contract term, including extensions, was due to expire. Without a comprehensive review 

or assessment of viable competitors that could provide services for the operation of JPL, 

NASA cannot have assurance that it is obtaining the best value for the taxpayer. To be 

compliant with the FAR, NASA should conduct a comprehensive review when the 

contract is extended by an award term, which is, potentially, every year. However, NASA 

may be able to eliminate the completion of successive FFRDC reviews by amending the 

Performance Evaluation Plan to, in effect, escrow all award terms earned or lost during 

the base years of the contract and conducting a comprehensive use and need review at the 

end of the base term. 

Lastly, we determined that a single CPAF contract is not the best contract vehicle to 

procure services from Caltech (Finding D). JPL had been managed using multiple costplus-fixed-fee (CPFF) contacts for more than 35 years, until NASA consolidated the two 

CPFF contracts to a single CPAF contract in 1993. We concluded that using a single 

CPAF contract for the operation of the entire FFRDC is difficult for NASA to effectively 
OVERVIEW

iv REPORT NO. IG-09-022-R 

manage because the contractor is responsible for multiple, complex, and often unrelated 

deliverables (for example, products crucial to NASA’s missions in the areas of earth 

sciences and solar system exploration, as well as normal facility functions such as public 

affairs and maintenance of the grounds). In addition, a CPAF contract requires 

significant oversight and documentation to evaluate contractor performance, and NASA 

did not perform a cost-benefit analysis to ensure that the benefits of a CPAF contract 

adequately offset the additional costs associated with contract administration. NASA also 

did not adequately evaluate alternative contract vehicles to support JPL operations. As a 

result, NASA may not be getting the best value for the taxpayer. 

Management Action

To ensure that evaluations appropriately reflect contractor performance, we recommended

that NASA provide detailed, explicit direction to contract performance evaluators 

concerning their responsibilities, monitor the documentation provided by the evaluators 

for accuracy and completeness, and apply a weighting factor to each criterion or project 

based on significance and cost. In addition, the NMO Procurement Officer should 

(1) modify the “Performance Evaluation Plan for Management of the Jet Propulsion 

Laboratory” to clearly include cost control measures weighted at no less than 25 percent 

of the performance evaluation factors; (2) perform a comprehensive use and need review 

of the FFRDC, to include a proactive search for alternative sources; (3) consider 

amending the Performance Evaluation Plan to “escrow” earned award terms; 

(4) reevaluate the use of award terms for any future FFRDC contract; and (5) perform a 

cost-benefit analysis and evaluate alternative contract vehicles for any FFRDC follow-on 

contracts.

Comments from the Assistant Administrator for Procurement; the NMO Director, JPL;

and the NMO Procurement Officer in response to a draft of this report concurred with our 

recommendations; in subsequent correspondence, the NMO agreed that there are potential 

monetary benefits to implementing the recommendations. The Assistant Administrator 

for Procurement, who signed the comments, stated that the NMO will propose additional 

language to clarify the performance evaluation process; ensure evaluation submissions are 

complete; request that annual criteria is weighted; capture cost control at no less than 

25 percent of the award fee evaluation score; proactively consider alternative sources; 

reconsider the use of award term incentives in future JPL or FFRDC contracts; conduct a 

cost-benefit analysis if a CPAF contract is used again; and consider alternative contract 

vehicles for the next FFRDC contract. See Appendix D for the full text of management’s 

comments, including an appendix of “Technical Comments” providing clarifying details 

on certain issues. 

We consider management’s proposed actions to be responsive for all recommendations. 

The recommendations are resolved and will be closed upon completion and verification 

of management’s corrective actions. 
SEPTEMBER 25, 2009

REPORT NO. IG-09-022-R 

CONTENTS

INTRODUCTION

Background _________________________________________ 1 

Objectives __________________________________________ 3 

RESULTS

Finding A: Contractor Performance Evaluations and Method 

Used to Calculate Ratings May Not Reflect Overall 

Performance_______________________________ 4 

Finding B: Cost Control Evaluation Factor Did Not Meet the 

Required 25 Percent________________________ 13

Finding C: Continuation of JPL as an FFRDC Was Not Adequately 

Justified _________________________________ 16

Finding D: A Single CPAF Contract Is Not the Best Vehicle 

for Managing NASA’s FFRDC__________________ 21

APPENDIX A 

Scope and Methodology_______________________________ 27

Review of Internal Controls ____________________________ 28

Prior Coverage ______________________________________ 29

APPENDIX B 

Award Fee Process___________________________________ 30

APPENDIX C 

FY 2007 Evaluation Criteria ____________________________ 31

APPENDIX D 

Management Comments ______________________________ 33

APPENDIX E 

Report Distribution___________________________________ 38

SEPTEMBER 25, 2009

REPORT NO. IG-09-022-R 1 

INTRODUCTION

Background

The Jet Propulsion Laboratory (JPL) is a NASA federally funded research and 

development center (FFRDC) operated under contract by the California Institute of 

Technology (Caltech), a private nonprofit research university located in Pasadena, 

California. In November 1943, JPL received funding from the U.S. Army Air Corps 

(Army); the Army formally transferred JPL to NASA, although it remained under the 

management of Caltech, in December 1958. 

From 1958 to 1993, NASA procured services from Caltech for the operation and 

management of the FFRDC using cost-plus-fixed-fee (CPFF) contracts. A CPFF contract 

is a cost-reimbursement contract that provides for payment to the contractor of a 

negotiated fee that is fixed at the contract’s inception. The fixed fee does not vary with 

actual cost, but may be adjusted as a result of changes in the work to be performed under 

the contract. Starting in the 1970s, there were two CPFF contracts—one for research and 

development activities and the other for operation of the facility. In 1993, the two CPFF 

contracts were consolidated into a single cost-plus-award-fee (CPAF) contract. A CPAF 

contract is a cost-reimbursement contract that provides for a fee consisting of (1) a base 

amount fixed at inception of the contract, and (2) an award amount that the contractor 

may earn in whole or in part during the performance period. Since 1993, NASA has 

awarded three CPAF contracts to Caltech for the operation of JPL: NAS7-1260 in 1993; 

NAS7-1407 in 1998; and NAS7-03001 in 2003 (the current contract). The 2003 contract 

included, for the first time, an award term incentive. This incentive is used to reward the 

contractor by extending the contract period of performance without a new competition or 

to penalize the contractor by shortening the performance period for poor execution of 

contract requirements. 

The 2003 contract afforded Caltech the opportunity to earn both award fees and award 

terms. Award fees are intended to encourage and reward the contractor for excellence in 

such areas as quality, timeliness, technical ingenuity, and cost-effective management. 

The available award fee pool was $22 million for each annual performance period, or 

$110 million over the 5 years of the contract’s initial period of performance. By the end 

of fiscal year (FY) 2008, Caltech had earned $97.02 million in award fees.

Contract Clause H-54, “Award Term,” allows NASA to increase or decrease the 

contract’s initial 5-year performance period for up to an additional 5 years, in increments 

of 3 or 9 months, based on the performance of the contractor. The total period of 

performance for the contract cannot exceed 10 years. The contract’s initial period of 

performance was from October 1, 2003, through September 30, 2008, and by the end of 
INTRODUCTION

2 REPORT NO. IG-09-022-R 

FY 2008, Caltech had earned additional award terms totaling 27 months, valued at 

approximately $3.375 billion, which extended the contract’s period of performance to 

December 31, 2010. The value of the extended, 71⁄4-year contract is estimated at 

$10.875 billion.

Award fee and award term determinations are made by the Fee Determination Official 

(FDO), who is the NASA Associate Administrator, based on the proposed adjectival

ratings and numerical scores for the evaluation factors provided by the Performance 

Evaluation Board (PEB). The NASA “Award Fee Contracting Guide,” June 27, 2001, 

requires the following adjectival ratings and numerical scores to be used on all award-fee 

contracts: 

• Excellent, 100-91; 

• Very Good, 90-81; 

• Good, 80-71; 

• Satisfactory, 70-61; and 

• Poor/Unsatisfactory, less than 61. 

The Guide states that the award fee earned is calculated by applying the total numerical 

score (the award fee score) to the award fee pool. For example, an award fee score of 85 

yields an award fee of 85 percent of the award fee pool for that performance period. 

The FDO sets the award fee score based on information provided by the PEB. The PEB 

receives recommended adjectival ratings and numerical scores for a contractor’s 

performance evaluation factors from the Primary Performance Evaluators (PPEs). To 

prepare the primary performance evaluation reports provided to the PEB, the PPEs review 

the criteria and performance evaluations from their organizations’ Contract Performance 

Monitors (CPMs). The CPMs monitor, assess, and evaluate the contractor’s performance 

in assigned areas, documenting their assessments and adjectival ratings in Performance 

Monitor Reports. The ratings assigned by the CPMs are based on their observations and 

knowledge of the quality of the contractor’s work and the contractor’s adherence to the 

established criteria. (See Appendix B for a synopsis of the award fee process.) 
INTRODUCTION

REPORT NO. IG-09-022-R 3 

Table 1 shows the recommended and awarded adjectival ratings and numerical scores, the 

award fees earned, the award terms earned, and the earned award terms’ estimated values 

for FY 2004 through FY 2008. 

 

Table 1. FY 2004 through FY 2008 Evaluations and 

Award Fees and Award Terms Earned 

Fiscal 

Year 

PEB 

Recommendation 

for Rating/Score 

FDOAwarded

Rating/Score

Award Fee

Earned (in 

millions)

Award Term 

Earned 

Estimated 

Value (in 

millions) of 

Award Term

2004 Excellent/96 $21.12 plus 9 months $1,125 

2005 Very Good/85 18.70 plus 3 months 375 

2006 Excellent/91 20.02 plus 9 months 1,125 

2007 Very Good/88 19.36 plus 3 months 375 

2008 Very Good/81 17.82 plus 3 months 375 

Total $97.02 27 months $3,375 

Objectives 

The audit’s overall objective was to determine whether the award fees and award terms 

earned by Caltech for the operation of JPL were appropriately calculated and justified. 

Specifically, we determined whether

• the contractor’s performance ratings were completed in accordance with the 

established criteria;

• evaluators’ recommendations on earned award fees and award terms were based 

on valid collection methods and supportable data; and 

• the award fees and award term extensions earned by Caltech accurately reflected 

the contractor’s overall performance, cost, and schedule. 

We also reviewed internal controls as they related to the overall objective. See 

Appendix A for details of the audit’s scope and methodology, our review of internal 

controls, and a list of prior coverage. 
RESULTS

4 REPORT NO. IG-09-022-R 

FINDING A: CONTRACTOR 

PERFORMANCE EVALUATIONS AND 

METHOD USED TO CALCULATE 

RATINGS MAY NOT REFLECT 

OVERALL PERFORMANCE

For FY 2007, performance evaluations for Caltech and the method NASA used to 

calculate the ratings may not reflect the overall performance of the contractor. The 

NASA “Performance Evaluation Plan, General Guidance for Writing Annual Award 

Fee Criteria,” August 9, 2006, states that “[q]uantitative measures should be used 

whenever the effectiveness or efficiency of the given performance can be 

unequivocally measured.” In addition, NASA’s Award Fee Contracting Guide states 

that contractor performance evaluation factors should be explicit, tied to desired 

outcomes, and based on characteristics of an individual procurement. We found that 

the criteria used for evaluating Caltech’s performance were sufficiently specific and 

measurable and could be used to make an objective assessment of the contractor’s

performance. However, NASA evaluators provided incomplete assessments or 

assigned inappropriate ratings for 12 of the 45 criteria (27 percent) when evaluating 

the contractor’s performance (as shown in Appendix C). In addition, criteria for 

different projects did not appear to be weighted within the evaluation factors by the 

projects’ overall importance to NASA or their costs. This resulted in potentially 

higher ratings than warranted and the possibility that Caltech inappropriately 

received award fees and term extensions. 

Quantifiable and Measurable Award Fee Criteria Are Required

NASA’s General Guidance for Writing Annual Award Fee Criteria states:

• Annual award fee criteria should be as specific and measurable as possible. 

Specific and measurable criteria will assist the CPMs in evaluating contractor 

performance and assist the contractor in understanding NASA’s expectations and 

the manner in which its performance will be evaluated. 

• Where feasible, quantitative or objective measures are preferred over qualitative 

or subjective ones. Quantitative measures should be used whenever the 

effectiveness or efficiency of the given performance can be unequivocally 

measured. 

• Award fee criteria should be crafted to make it as easy as possible to make an 

objective assessment of contractor performance and to minimize the subjectivity 

of the process. 
RESULTS

REPORT NO. IG-09-022-R 5 

NASA’s Award Fee Contracting Guide, Section 3.7.4, “Documentation,” states: 

• The reporting formats used by monitors should be structured to ensure clarity and 

conciseness. The narrative comments provide detailed, pertinent information. 

For example, they cover the circumstances under which reported performance 

levels were achieved. These comments also discuss the contractor’s efficiency in 

managing assigned personnel and other resources. Enough detail should be 

included in reports to the PEB to ensure their findings and recommendations are 

accurate and fair and can be supported to the FDO. 

• Appropriate documentation is vital to support the recommendations of PEBs, 

particularly where these recommendations differ from the conclusions reported by 

cognizant monitors. Since the evaluation is a judgment, based on all pertinent 

information, that information needs to be identified, discussed, and substantiated 

in the documentation. The FDO will want to review the documentation to satisfy 

any concerns regarding contractor performance before deciding whether to accept 

the recommended award fee or some higher or lower amount. 

Performance Evaluations Could More Accurately Reflect 

Contractor Performance

We determined that the FY 2007 award fee criteria were specific and measurable and 

could be used to make an objective assessment of the contractor’s performance. 

However, NASA CPMs provided either incomplete assessments or inappropriate ratings 

when evaluating the contractor’s performance for 12 of the 45 (27 percent) FY 2007 

criteria. Higher ratings may have occurred in part because the FY 2007 Performance 

Evaluation Plan did not require the rating official (CPM or PPE) to address all elements, 

or metrics, of the criteria in the evaluation. The online form only included comment 

boxes for “Degree to which contractor performance met established performance 

standards,” “Specific Strengths,” “Weaknesses,” and “Mitigating Factors,” which allowed

the evaluator to decide what specific criteria metrics to address. In addition, although the 

Performance Evaluation Plan contained both general criteria and annual criteria, none of 

the CPMs and PPEs that we interviewed indicated that the general criteria was addressed. 

The Performance Monitor Reports only addressed the annual criteria. 

For example, the criterion “Office of Safety and Mission Assurance-2” has five metrics, 

which were not completely addressed in the assessment. Following is the standard for an 

Excellent rating:

In addition to meeting the satisfactory performance standard, [Caltech must] achieve 

continual im provement in th e e ffective im plementation o f J PLs q uality a ssurance 

program, to include the following: [1] Implement improvements in the material review 

board process for procured JPL Critical Items, and in the process of identification and 

documentation o f i nspection cr iteria. Update applicable p rocedures an d t raining. 
RESULTS

6 REPORT NO. IG-09-022-R 

[2] Finalize a nd implement a s upplier r ating s ystem th at in corporates a ppropriate 

rating factors, captures all necessary data, and produces necessary reports for users to 

better assess and select suppliers. [3] Finalize and implement a standard set of defect 

codes to identify non-conformances. [4] Develop a Q uality A ssurance E ngineering 

handbook to promote consistency in the performance of hardware quality assurance. 

[5] Define, co llect, an d an alyze a s tandard s et o f quality metrics to monitor the 

execution of quality assurance activities.

The Performance Monitor Report for this criterion for Caltech’s FY 2007 performance 

states that 

 

 

 

 

 . 

 

 

 

 

 

 

 

 

 .

4

 

 . The 

Satisfactory standards for the criterion state:

Start E nvironmental T ests o n s chedule in mid-October 2007. Complete pr e-ship 

review and ship on schedule (mid-May ’07) with no critical issues identified. Launch 

on schedule (August ’07), completing all tests in the incompressible test list. Total 

mission cost equals the mission’s Confirmed cost cap ($383M), plus augmentation of 

no more than $29M ($410M total) per December 7th Cost-to-Go Review, and with 

minimum work to be done post launch. Ship and launch risk rated medium-to-low.

The Satisfactory standard stipulates that the contractor could not request augmentation of 

more than $29 million. 

 

 

 

 

4 Management’s comments (see Appendix D, page 39) specifically address this example, noting that rating 

officials are not required to document all elements of the rating criteria and that the Office of Safety and 

Mission Assurance considered the assessment “to have properly evaluated each rating element and to have 

fulfilled the documentation instructions.” The comments state that future evaluations will fully document 

each rating element.
RESULTS

REPORT NO. IG-09-022-R 7 

 

 

 

 

 

 

 . 

A Documented Tendency toward High Ratings and Awards

In 2007, a Government Accountability Office (GAO) report, “NASA Procurement: Use 

of Award Fees for Achieving Program Outcomes Should Be Improved” (GAO-07-58, 

January 17, 2007), found that NASA paid most of the available award fee on all of the 

contracts that GAO reviewed, including contracts for projects that showed cost increases, 

schedule delays, and technical problems. The report stated that NASA paid more than 

90 percent of the available fee based on its evaluation of the contractor’s performance 

against criteria even when those contractors did not deliver capability within initial cost, 

schedule, and performance parameters. The 2003 JPL contract included $110 million in 

award fees for the initial 5-year performance period; by the end of FY 2008, NASA had 

paid Caltech $97.02 million (88 percent) in award fees.

In interviews with Office of Inspector General (OIG) auditors, the NMO Procurement 

Office staff stated that high performance ratings are critical to the JPL contractor and that 

a satisfactory rating is perceived as a negative. High performance ratings correlate to 

significant award fees and additional award terms for the contractor. Our review of the 

evaluations for the award fee criteria indicated that Caltech received a rating above 

satisfactory more than 95 percent of the time. Specifically, the FY 2007 ratings for the 

45 criteria (listed in Appendix C) were 31 Excellent, 8 Very Good, 4 Good, 

1 Satisfactory, and 1 Poor/Unsatisfactory. 

The potential for inflated evaluations as the result of long-term contracting relationships 

was noted by Vernon Edwards, a consultant in Government contracting who has written 

several articles on the use of award terms. In an October 2000 article, “Award Term: The 

Newest Incentive,”5

A potential disadvantage of a long-term relationship is the possibility that the agents 

of the contracting parties will begin to conduct business on a personal basis instead of

a p roper p rofessional b asis. C ontractual r elationships ar e, af ter all, human 

relationships. P eople who have come to know and like one another in the course of 

time may relax their standards and overlook performance deficiencies for the sake of 

their personal relations; they may become reluctant to criticize or to take an action 

that could hurt the other person. On the other hand, a change in personnel may bring 

conflict as the parties try to adapt to new personalities and changes in long-standing 

he states:

 

5 Available online at http://www.wifcon.com/anal/analaterm.htm (accessed September 11, 2009).
RESULTS

8 REPORT NO. IG-09-022-R 

ways of seeing, understanding, and doing. These sorts of developments are probably 

a natural and unavoidable byproduct of long-term business relationships.

Lack of Proportionate Weighting May Lead to Inflated 

Performance Scores

NASA did not appear to use proportionate weighting of criteria and, therefore, the 

process used to arrive at Caltech’s award fee score for FY 2007 may not have reflected

the overall performance of the contractor. For example, the programmatic performance 

evaluation factor comprises 19 criteria for six programs6

 

6 The SMD programs are Astrophysics, Earth Science, Heliophysics, Mars Exploration, Planetary Science, 

and Management and Policy.

 under the Science Mission 

Directorate (SMD), a criterion under the Exploration Systems Mission Directorate 

(ESMD), and one under the Space Operations Mission Directorate (SOMD), as shown in 

the following figure and listed in Appendix C. However, CPMs do not assign numerical 

scores to the criteria; PPEs assign a numerical score to each program, but do not appear to 

consider a criterion’s significance in doing so. SMD’s overall score is the average of the 

six programs’ numerical scores.
RESULTS

REPORT NO. IG-09-022-R 9 

In addition, the numerical score of for the programmatic performance evaluation 

factor was not calculated, either as a straight average or using proportional weighting, but 

was a consensus score subjectively determined by the PEB. 

Without establishing weighted criteria to reflect significance or cost, NASA had no 

assurance that it was accurately evaluating Caltech’s performance. For example, SMD’s 

Mars Exploration Program (MEP) was valued at approximately $477 million in FY 2007, 

accounting for approximately 32 percent of the overall estimated $1.5 billion JPL contract 

for that year. The program includes three projects: the Mars Science Laboratory (MSL), 

Phoenix, and the Mars Reconnaissance Orbiter (MRO), each with a criterion (Mars-1, 

Mars-2, and Mars-3, respectively). The MSL project, valued at $359 million (75 percent 

of the MEP budget), with a cost overrun of $35 million, earned a . 

The Phoenix project, valued at $75 million (16 percent), was . The 

MRO project, valued at $42 million (9 percent), . Although 

the PPE may have considered the projects’ significance when determining the MEP’s 

adjectival rating ( ), there is no documentation of the methodology used for 

that determination or of the methodology used to calculate the numerical score. The 

numerical score assigned by the PPE for the MEP, , was then averaged with the five 

E: ....................Excellent

VG: .............Very Good

G: .........................Good

S: ................Satisfactory

P: ...Poor/Unsatisfactory

Programmatic Evaluation Factor

FY 2007 Criteria and Ratings, by Mission Directorate
RESULTS

10 REPORT NO. IG-09-022-R 

other SMD programs to provide an overall SMD score. This process and calculation 

method essentially conceals poor performance because a small project with a high rating, 

such as the MRO project, carries the same weight as a more significant project that costs 

more, such as the MSL project, which . 

We reevaluated the MEP using a weighted calculation that was based on the projects’ 

percent of the FY 2007 MEP budget. Because the projects were not given numerical 

scores, we used the score at the top of the range for the adjectival rating assigned by the 

CPMs (see Table 2). Using this method, we determined that the MEP 

 

 —which would have also affected the subsequent SMD rating and score. 

Table 2. Weighted Evaluation of the Mars Exploration Program by Cost 

Project

Dollars 

Spent

(millions)

Percent 

of Total 

Adjectival 

Rating

OIG Calculation

Numerical 

Score*

Weighted 

Score

Adjectival 

Rating

MSL $359.479 75.43 45.26 

Phoenix 75.462 15.83 12.67 

MRO 41.637 8.74 8.74 

MEP $476.578 100.00 66.66 Satisfactory 

* High end of adjectival rating’s range. Multiplying these scores by the project’s percent of the MEP budget yields 

the weighted numerical score.

Although the methodology was not documented, NASA personnel stated that project 

significance was considered when determining ratings. Using the significance order 

provided by the PPE ( ), we considered 

significance alone to determine a weighted score for the MEP using arbitrary percentages 

based on the projects’ order of significance (see Table 3). We then averaged that score 

(76) with the score we determined using cost alone (66.7), which results in an overall 

grade of Good (72).7

 

7 Management’s comments (see Appendix D, page

 However, without documenting its methodology, NASA cannot be 

assured of the accuracy of its evaluation process.

39) specifically addressed this example, noting that it “does not readily suggest a shortcoming on the part of the Mars Program PPE’s evaluation.” For our 

calculations, we assigned arbitrary percentages because CPMs only assign adjectival ratings, not 

percentages. To obtain numerical scores for our calculations, we used the numerical score at the high end 

of the given adjectival rating. Therefore, while our calculation resulted in a grade of 72 for the Mars 

Exploration Program, it does not mean to imply that the number is accurate.
RESULTS

REPORT NO. IG-09-022-R 11

Table 3. Weighted Evaluation of the Mars Exploration Program by Significance

Project

Significance

Rankinga 

(percent) 

Adjectival 

Rating

OIG Calculation

Numerical 

Scoreb Weighted Score

Adjectival

Rating

MSL 

Phoenix 

MRO 

MEP 76 Good 

a The OIG-assigned, arbitrary percentages to reflect the projects’ order of significance, as stated by the PPE.

b High end of adjectival rating’s range. Multiplying these scores by the project’s significance ranking 

percentage within the MEP yields the weighted numerical score.

Despite the importance of the projects that Caltech manages under the JPL contract, 

NASA has not established weighted criteria relative to their significance or cost. Large 

projects appear to be equally evaluated in the award fee process and have minimal impact 

on the award fee scores, even if the standard of excellence is not met. MSL, which was 

worth approximately 24 percent of the total $1.5 billion FY 2007 contract and 

 , appeared to be weighted the same as any of the other 

criteria in the programmatic factor. Averaging these ratings rather than weighting 

projects according to their relative importance to NASA’s mission or their budgets does 

not ensure an accurate assessment of overall contractor performance. 

NASA’s failure to objectively support the performance ratings and proportionately weight 

the evaluation factors’ criteria relative to their significance resulted in potentially higher 

ratings than warranted and the possibility that Caltech inappropriately received award fees 

and term extensions. Since 2003, Caltech has earned more than $97.02 million in award 

fees and 27 months in award term extensions, valued at approximately $3.375 billion. 

The additional award terms also resulted in the contract being extended without 

competition.

Recommendations, Management’s Response, and Evaluation of 

Management’s Response

Recommendation 1. The NASA Management Office Procurement Officer should provide 

in the Performance Evaluation Plan, or other applicable documents, specific, explicit 

direction to the Contract Performance Monitors to evaluate and document the contractor’s 

annual performance for all of the criteria’s metrics under the evaluation factors.

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will propose clarifying language for consideration by the PEB. The 
RESULTS

12 REPORT NO. IG-09-022-R 

Performance Evaluation Plan will be updated with the clarifying language and submitted 

to the FDO by August 31, 2009, and explicit direction will be given in the e-mail texts for 

the mid-year and final award fee evaluation calls.

Evaluation of Management’s Response. Management’s proposed actions are

responsive. The recommendation is resolved and will be closed upon completion and 

verification of corrective actions to be taken during the mid-year (March 2010) and final 

(September 2010) award fee evaluation calls. 

Recommendation 2. The NASA Management Office Procurement Officer should monitor 

Contract Performance Monitors’ input for accuracy and completeness. 

Management’s Response. The Assistant Administrator for Procurement concurred, 

agreeing with the intent of the recommendation. The NMO will work with those 

directorates and offices providing evaluation input to ensure that the input is complete. In 

September 2009, the NMO will convene a meeting of CPMs and PPEs to address our 

report recommendations. Further, explicit direction will be given in the e-mail texts for 

the mid-year and final award fee evaluation calls. As stated in the “Technical 

Comments” appendix of management’s comments: 

The NMO concurs with the need for a renewed emphasis on the documentation of the 

evaluation process at all levels, but particularly at the CPM and PPE stages. This is 

particularly im portant in th is s ituation w here b oth s ubjective ( programmatic a nd 

strategic influence factors) and objective (quantitative measures or metrics) elements 

must be integrated in order to obtain a rational performance evaluation.

Evaluation of Management’s Response. Management’s proposed actions are 

responsive. The recommendation is resolved and will be closed upon completion and 

verification of corrective actions to be taken at the September 2009 meeting and during 

the mid-year (March 2010) and final (September 2010) award fee evaluation calls. 

Recommendation 3. The NASA Management Office Procurement Officer should include 

in the Performance Evaluation Plan a weighting factor for each criterion or project based on 

significance and cost, as determined by the Performance Evaluation Board. 

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will submit the weighting request to the JPL-sponsoring 

Headquarters offices and will then implement the requirements approved by the PEB

relative to weighting individual annual criteria. The estimated completion date is 

September 30, 2010. 

Evaluation of Management’s Response. Management’s proposed actions are

responsive. The recommendation is resolved and will be closed upon completion and 

verification of management’s corrective actions. 
RESULTS

REPORT NO. IG-09-022-R 13

FINDING B: COST CONTROL 

EVALUATION FACTOR DID 

NOT MEET THE REQUIRED 

25 PERCENT

NASA did not effectively employ cost control as a measure for monitoring JPL’s 

contract performance because “cost control” was not established as a separate 

performance evaluation factor in the CPAF contract. The NASA supplement to the 

Federal Acquisition Regulation (FAR), NASA FAR Supplement (NFS)

Version 04.0, November 1, 2004, Subpart 1816.4, “Incentive Contracts,” requires 

that CPAF contracts include a separate cost control evaluation factor worth no less 

than 25 percent of the total weighted evaluation factors. The NASA Management 

Office (NMO) Procurement Officer stated that cost control was included within the 

criteria of the programmatic evaluation factor. However, we determined that while 

the performance evaluation factors included some cost control measures, the 

combined weight of those measures was only 5 percent to 14 percent of the total 

weighted factors each year, far below the 25 percent requirement. In addition, NASA 

evaluators did not reference the cost control measures sufficiently in their evaluations 

to meet the intent of the NFS with regard to monitoring and controlling cost. As a 

result, we calculated that $16 million of the $97 million paid in award fees for 

Caltech’s performance under the 2003 contract were not in compliance with NFS 

requirements. Therefore, we consider that amount to be unsupported and an 

ineffective use of Government funds. We also question the 27 months in term 

extensions, valued at approximately $3.375 billion, awarded to Caltech. 

NASA FAR Supplement Requires Cost Control Evaluation Factor

FAR 16.402-1, “Cost Incentives,” states that most incentive-type contracts, including 

CPAF contracts, are required to contain a cost incentive or constraint. 

NFS 1816.405-274, “Award Fee Evaluation Factors,” requires a cost control evaluation 

factor in all award-fee contracts and that the cost control factor be no less than 25 percent 

of the total weighted evaluation factors. NFS 1816.405-274 further states that the 

predominant consideration when evaluating cost control should be an objective 

measurement of the contractor’s performance against the estimated cost of the contract, 

including the cost of undefinitized contract actions, when appropriate. NASA’s Award 

Fee Contracting Guide, Section 3.4.3, “Weighting of Evaluation Factors,” reflects this 

requirement, stating that “cost control should always be a substantial [evaluation] factor. 

When percentage weights are used, the cost control factor will be at least 25 percent of 

the total award fee. . . . This ensures that the factors are balanced and, when making tradeoffs, the contractor assigns the proper importance to all factors.” 
RESULTS

14 REPORT NO. IG-09-022-R 

NASA did not effectively employ cost control as a measure for monitoring Caltech’s

performance. The Performance Evaluation Plan did not include cost control as a separate 

performance evaluation factor. For example, although 12 of the 45 criteria for the 

performance evaluation factors in the FY 2007 Plan included some cost control measures, 

those cost control measures were not weighted to be at least 25 percent of the total 

weighted evaluation factors, as required by the NFS and NASA’s Award Fee Contracting 

Guide. The NMO Procurement staff stated that the required 25 percent cost factor was

included in the programmatic factor, but could not support that statement. 

The Performance Evaluation Plan distinguished three evaluation factors—programmatic, 

institutional, and outreach—but did not include cost control as a separate factor. We 

found that cost control was included as one of three elements under the programmatic

evaluation factor and that the institutional evaluation factor included a cost control 

criterion. However, the Performance Evaluation Plan did not specify the weight assigned 

to the cost control measures included in the evaluation factors. The July 2005 

Procurement Management Survey Report by the NASA Office of Procurement, noted that 

“although there [was] not a specific breakout of the weightings of the sub-factors, it [was] 

clear that cost control [was] something less than 25 percent.” 

Cost control measures included in the performance evaluation factors were not sufficient 

to meet the intent of the NFS. We reviewed the FY 2007 award fee criteria and 

determined that 12 criteria within the programmatic and institutional factors contained 

some measure of cost control. We calculated the weight of those cost control measures to 

be 12 percent, which is well below the required 25 percent. (See Appendix A for a 

discussion of our calculation.) 

Insufficient cost control measures gave minimum incentive for the contractor to control 

cost because a poor cost control performance rating could easily be offset by non-cost 

factors. In addition, the low percentage weighting deemphasized the importance of 

controlling cost and minimized the effectiveness of cost control. Because the evaluation 

factors did not include weighted cost control measures of at least 25 percent, as required 

by the NFS, but 12 percent, we determined that 13 percent of the award fees paid to 

Caltech for the FY 2007 performance period were not in compliance with NFS 

requirements. Similar low percentages, ranging from 5 percent to 14 percent, of cost 

control were evident in the other performance periods of the contract. Therefore, we 

consider $16,033,600 (17 percent) of the $97,020,000 award fees to be unsupported costs 

and an ineffective use of Government funds. Table 4 shows how we determined the 

portion of award fees that were not in compliance with NFS requirements. 
RESULTS

REPORT NO. IG-09-022-R 15

Table 4. Summary of Paid Award Fees 

FY 2004 through FY 2008 

Award 

Fee

 Period 

Available

 Award Fee 

Award Fee

 Score 

Paid

Award Fees 

Unmet Weight

of Cost Factor

Portion of Award Fee

Not NFS-Compliant 

FY 2004 $22,000,000 96 percent $21,120,000 19 percent $4,012,800 

FY 2005 22,000,000 85 percent 18,700,000 20 percent 3,740,000 

FY 2006 22,000,000 91 percent 20,020,000 19 percent 3,803,800 

FY 2007 22,000,000 88 percent 19,360,000 13 percent 2,516,800 

FY 2008 22,000,000 81 percent 17,820,000 11 percent 1,960,200 

 Total $110,000,000 $97,020,000 $16,033,600 

Because NASA did not effectively employ cost control as a performance measure for the 

JPL contract, NASA’s evaluation of Caltech’s performance could have resulted in 

inappropriately awarded fees and term extensions. Further, if management does not 

change the Performance Evaluation Plan criteria to meet the 25 percent cost control 

requirement, future award fees

8

Recommendation, Management’s Response, and Evaluation of 

Management’s Response

 could be awarded without appropriate emphasis on cost 

control. 

Recommendation 4. The NASA Management Office Procurement Officer should modify 

the Performance Evaluation Plan to clearly indicate a cost control factor weighted at no less 

than 25 percent of the award fee evaluation.

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will coordinate an implementation strategy that will consider 

developing a process to capture the cost factor. The estimated implementation date is 

September 30, 2010. 

Evaluation of Management’s Response. Management’s action is responsive. The 

recommendation is resolved but will remain open pending corrective actions to be taken 

by September 30, 2010. 

 

8

 For the FY 2009 and FY 2010 evaluation periods, $44 million in award fees will be available to Caltech

($22 million each year).
RESULTS

16 REPORT NO. IG-09-022-R 

FINDING C: CONTINUATION OF 

JPL AS AN FFRDC WAS NOT 

ADEQUATELY JUSTIFIED

NASA did not adequately consider alternative sources and could not provide support 

to justify its decision to continue procuring services from Caltech for the operation of 

JPL as an FFRDC. The FAR requires a comprehensive review of the use and need 

for an FFRDC every 5 years. FAR 35.017-4, “Reviewing FFRDCs,” provides five 

criteria for the review, stating that “the sponsor, prior to extending the contract or 

agreement with an FFRDC, shall conduct a comprehensive review of the use and 

need for the FFRDC” and that the review should include consideration of alternative 

sources to meet the sponsor’s needs concerning operation of the FFRDC. However, 

the review NASA conducted in 2002 prior to awarding the most recent contract, in 

2003, was less than exhaustive in its search for viable competitors. A review 

conducted in 2008 addressed four of the five FAR criteria, but did not include 

consideration of alternative sources. NMO Procurement Office personnel decided to 

postpone the review until the current contract term, including extensions, was due to 

expire. Without a comprehensive review or assessment of viable competitors that 

could provide services for the operation of JPL, NASA cannot have assurance that it 

is obtaining best value for the taxpayer. 

FAR Specifies a 5-Year Review Requirement for Evaluating an 

FFRDC’s Use and Need

FAR 35.017, “Federally Funded Research and Development Centers,” specifies the policy 

for the “establishment, use, review, and termination of Federally Funded Research and 

Development Centers (FFRDCs) and related sponsoring agreements.” It states that the 

FFRDC agreement will not exceed 5 years, but can be renewed every 5 years as long as a 

comprehensive review of the use and need for the FFRDC is conducted. Approval to 

continue or terminate the sponsorship shall be based on the results of the comprehensive 

review and rest with the head of the sponsoring agency. The FAR states that the review 

should include the following: 

(1) an examination of the sponsor’s special technical needs and mission requirements 

that are performed b y t he F FRDC t o d etermine whether and at w hat l evel t hey 

continue to exist; (2) consideration of alternative sources to meet the sponsor’s needs; 

(3) an as sessment o f t he ef ficiency and ef fectiveness o f t he F FRDC in meeting the 

sponsor’s n eeds, in cluding th e F FRDC’s a bility to maintain its objectivity, 

independence, q uick r esponse cap ability, cu rrency i n i ts f ield(s) of ex pertise, an d 

familiarity w ith t he n eeds of i ts s ponsor; ( 4) an as sessment o f t he ad equacy of the 

FFRDC management in ensuring a cost-effective operation; and (5) a determination 

that th e c riteria f or e stablishing th e F FRDC c ontinue to be satisfied and t hat t he 
RESULTS

REPORT NO. IG-09-022-R 17

sponsoring agreement i s i n c ompliance w ith FAR 35.017-1, “ Sponsoring 

Agreements.” 

NASA Chose to Continue Procuring the Services of Caltech for the 

Operation of JPL Without Adequate Support

The FFRDC use and need determination review completed in 2002 by NMO personnel 

did address the five FAR criteria. However, the review was not proactive in the search 

for alternative sources because, as stated in the review, NASA concluded that the JPL 

FFRDC was a unique resource. The review did not assess other contractors’ abilities, but 

rather focused on obstacles to procuring services from those contractors, which prevented 

NASA from determining whether alternative sources could perform the work. For 

example, the review indicated obstacles to not retaining Caltech, such as closeout liability 

and procurement of NASA’s physical records from Caltech. The backup support for the 

review indicated two potential competitors but stated “a one-to-one comparison [was] not 

possible.” NMO personnel confirmed that the acquisition team assembled for this review 

did not contact the two potential alternative sources to determine whether they could 

perform the work of Caltech. 

The most recent review performed in 2008 addressed four of the five FAR criteria, but 

did not include a consideration of alternative sources because earning additional years 

postponed the acquisition planning process. NMO personnel stated that NASA normally 

conducts a search for alternate sources during the acquisition planning process. Because 

the contract was extended, NMO Procurement personnel postponed the search for 

alternate sources until planning for the competition or renewal of the JPL contract. As 

stated in the 2008 review, “Review and Findings” (Section 2.d.): 

[R]enewal o f t he co ntract with C altech is n ot n ecessary a t th is tim e. A s described 

above, the period of performance will run until at least September 30, 2010, but could 

possibly e xtend u ntil D ecember 31, 2012. A de tailed, comprehensive acquisition 

strategy will be performed to explore all alternatives prior to competing or negotiating 

a follow-on contract to the present agreement. This activity is expected to commence 

no later than October 2009.

This consideration of alternative sources will be more than 7 years after the 2002 review 

was completed and may be delayed further with the award of additional term extensions. 

Guidance Encourages Competition

A March 4, 2009, Presidential Memorandum9

 

9

Available online at 

states that the Government must move 

away from cost-reimbursement contracts toward fixed-price contracts and must also 

http://www.whitehouse.gov/the press office/Memorandum-for-the-Heads-ofExecutive-Departments-and-Agencies-Subject-Government/ (accessed September 11, 2009).
RESULTS

18 REPORT NO. IG-09-022-R 

move away from sole-source awards toward full and open competition. The 

Memorandum also states that “it is the policy of the Federal Government that executive 

agencies shall not engage in non-competitive contracts except in those circumstances 

where their use can be fully justified and where appropriate safeguards have been put in 

place to protect the taxpayer.”

NASA has used a CPAF contract for its FFRDC since 1993 and introduced award term 

incentives in the 2003 contract. Since 2003, NASA has awarded term extensions to 

Caltech without competition for each of the 5 periods of performance. As of April 2009, 

NASA had awarded Caltech 27 months in term extensions, worth approximately 

$3.3 billion.

10

Guidelines for when or how to effectively use award terms do not exist in any Federal or 

NASA guidance. However, the FAR explicitly requires the FFRDC sponsor conduct a 

comprehensive review, to include “consideration of alternative sources to meet the 

sponsor’s needs,” prior to extending the contract. Therefore, NASA should have 

completed a comprehensive review with each term extension awarded beyond the FARimposed 5-year limit for FFRDC review. Completion and documentation of such a 

review would provide assurance that Caltech is providing NASA the best value for 

taxpayer money. 

 In addition, the structure of the 2003 contract allows Caltech to earn 

additional award fees and term extensions for performance periods added to the contract 

as a result of previously awarded term extensions, which could further increase the time 

between competitive evaluations.

An Alternative Approach to Award Term Extensions Could 

Effectively and Efficiently Meet FAR Requirements 

Because the FAR lacks specifics on both the practical use of award terms and issuance of 

contract modifications for non-negotiated changes, NASA is challenged to effectively and 

efficiently integrate the FAR requirements with award term extensions. NASA 

Procurement Information Circular 06-02, “Use of Award Term Incentive,” January 25, 

2006, acknowledges this, stating that “with the lack of regulatory guidance relating to 

award term, the structure of the incentive and the evaluation methodologies employed 

have grown very diverse.” 

The Performance Evaluation Plan states that “the earned award fee and award term will 

be incorporated into the contract through a unilateral contract modification issued by the 

contracting officer within 60 days (goal) of the FDO’s determination for the period.” 

This is in line with FAR 43.2, “Change Orders,” Section 204(b)(1), which states that 

“contracting officers shall negotiate equitable adjustments that resulted from change 

orders in the shortest practicable time.” Although the FAR does not address when or how 

 

10Each additional year that Caltech earns in award term extensions is worth approximately $1.5 billion. 
RESULTS

REPORT NO. IG-09-022-R 19

to effectively use award terms, it does state that a comprehensive review of the use and 

need of the FFRDC must be completed every 5 years or prior to extending the contract. 

Therefore, to be compliant with the FAR, NASA should conduct a comprehensive review 

when the contract is extended by an award term, which is, potentially, every year. 

However, NASA may be able to eliminate the completion of successive FFRDC reviews

by amending the Performance Evaluation Plan to, in effect, escrow all award terms 

earned or lost during the base years of the contract and conducting a comprehensive use 

and need review at the end of the base term. The FFRDC review could then be costeffectively completed, as required, at both contract extension and every 5 years. This 

approach would provide for a more effective and efficient use of award terms, allowing 

NASA to be FAR-compliant, save administrative effort, and avoid unnecessarily 

expending taxpayer dollars by completing reviews in successive years. In addition, the 

contract modification covering the escrowed award terms could exclude the provision for 

award terms, thus eliminating the current practice of allowing additional award terms to 

be earned during the extended contract performance periods. NASA is unique in 

allowing these additional award terms; other Federal agencies operating FFRDCs allow 

award terms to be earned only during the FFRDC contract’s base years. 

Recommendations, Management’s Response, and Evaluation of 

Management’s Response

Recommendation 5. The NASA Management Office Procurement Officer should perform 

a comprehensive review in accordance with FAR, to include a proactive search for 

alternative sources, as soon as practicable.

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will draft a plan to assemble updated NASA requirements for JPL 

that will be used in a proactive search for alternative sources. The estimated completion

date is unknown, but a proactive search will commence by fall 2009 and continue through 

the acquisition phase of the new contract. 

Evaluation of Management’s Response. Management’s proposed actions are 

responsive. The recommendation is resolved and will be closed upon completion and 

verification of corrective actions to be taken during the acquisition phase of the new 

contract. 

Recommendation 6. The NASA Management Office Procurement Officer should 

reevaluate the use of award term for any future FFRDC contract. 

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will consult with the Office of Procurement and the NASA 

Associate Administrator’s office on JPL contract and incentive alternatives. They do not 

foresee using an award term for future JPL contracts, and the current contract does not 
RESULTS

20 REPORT NO. IG-09-022-R 

allow any additional award terms beyond those already awarded. The estimated 

completion date for considering alternative contract vehicles is February 2010, during the 

acquisition planning phase of the new contract. 

Evaluation of Management’s Response. Management’s proposed action is responsive. 

The recommendation is resolved and will be closed upon completion and verification of 

management’s corrective action. 
RESULTS

REPORT NO. IG-09-022-R 21

FINDING D: A SINGLE CPAF

CONTRACT IS NOT THE BEST 

VEHICLE FOR MANAGING 

NASA’S FFRDC

We determined that a single CPAF contract is not the best contract vehicle to procure 

services from Caltech to manage and operate JPL as an FFRDC. JPL had been 

managed using multiple CPFF contacts for more than 35 years. In 1993, in part to 

respond to a GAO recommendation,11

Using a single CPAF contract for the operation of the entire FFRDC is difficult for 

NASA to effectively manage because the contractor is responsible for multiple, 

complex, and often unrelated deliverables. In addition, a CPAF contract requires 

significant oversight and documentation to evaluate contractor performance and, by 

its nature, does not always motivate cost control. The FAR suggests the use of a 

fixed-price contract when the costs can be established reasonably and the risk is low, 

as is the case with a number of the services provided by Caltech under the single 

CPAF contract. NASA did not perform a cost-benefit analysis to ensure that the 

benefits of a CPAF contract adequately offset the additional costs associated with 

contract administration. NASA also did not adequately evaluate alternative contract 

vehicles to support JPL operations. As a result, NASA may not be getting the best 

value for the taxpayer.

 the two CPFF contracts, one for research and 

development activities and the other for facility operations, were consolidated into a 

single CPAF contract.

FAR and NASA’s Award Fee Contracting Guide Describe the 

Best Use of Various Contract Types

FAR 16.103b, “Negotiating Contract Type,” states that a “firm-fixed-price contract, 

which best utilizes the basic profit motive of business enterprise, shall be used when the 

risk involved is minimal or can be predicted with an acceptable degree of certainty.” 

Similarly, FAR 16.202-2, “Firm-Fixed-Price Contracts, Application,” states that such a 

contract is suitable for acquiring commercial items or for acquiring other supplies or 

services on the basis of reasonably definite functional or detailed specifications when the 

contracting officer can establish fair and reasonable prices at the outset. The FAR 

addresses time-and-materials contracts in FAR 16.601(b), “Time-and-Materials 

Contracts, Application,” stating that such a contract may be used only when it is not 

possible at the time of placing the contract to estimate accurately the extent or duration of 

 

11GAO. “NASA Procurement: Proposed Changes to the Jet Propulsion Laboratory Contract” 

(GAO/NSIAD-93-178, July 1993).
RESULTS

22 REPORT NO. IG-09-022-R 

the work or to anticipate costs with any reasonable degree of confidence. Managing the 

facility operations portion of JPL could potentially be best served by either type of 

contract vehicle.

Conversely, a CPAF contract, or a number of CPAF contracts, may be the most 

appropriate vehicle for the research and development activities performed at JPL. 

FAR 16.301-2, “Cost-Reimbursement Contracts, Application,” states that costreimbursement contracts are suitable for use only when uncertainties involved in contract 

performance do not permit costs to be estimated with sufficient accuracy to use any type 

of fixed-price contract. Also, FAR 16.405 2(b)(iii) states that a CPAF contract is suitable 

for use when any additional administrative effort and cost required to monitor and 

evaluate performance are justified by the expected benefits. In addition, FAR 16.405-2(a) 

states that a CPAF contract provides for an award that is sufficient to motivate excellence 

in such areas as quality, timeliness, technical ingenuity, and cost-effective management.

NASA’s Award Fee Contracting Guide, Section 1.3 states a CPAF contract is appropriate 

to use when key elements of performance cannot be objectively measured. CPAF 

contracts can also be used to procure design, development, and initial fabrication of stateof-the-art hardware, such as a new type of planetary probe or an experimental aircraft. 

The Guide further states that since award fee contracts require additional administrative 

effort, they should only be used when the contract values, performance period, and 

expected results warrant that additional management effort. As noted in NASA’s Award 

Fee Contracting Guide, evaluation of performance under a CPAF contract requires much 

greater effort than a CPFF contract. 

NASA Did Not Analyze the Cost or Benefit of Using a Single 

CPAF Contract

During GAO’s fieldwork on NASA’s use of award fees for a report issued in 

January 2007,12

Because GAO found no instances where a documented cost-benefit analysis had been 

done for any of the NASA contracts that it reviewed for its 2007 report, GAO 

 GAO questioned whether NASA had conducted a cost-benefit analysis to 

justify the use of a single CPAF contract for procuring services from Caltech for the 

operation of JPL. Specifically, GAO asked whether (1) a cost-benefit analysis was 

conducted to justify the use of a CPAF contract; (2) what costs were included, and 

(3) what benefits were considered in the analysis. The NMO Procurement Officer at the 

time responded that a cost-benefit analysis was completed, but the analysis was not 

conducted with enough detail to include a specific delineation of costs, and that the use of 

a CPAF contract was deemed appropriate during acquisition strategy planning sessions. 

 

12GAO. “NASA Procurement: Use of Award Fees for Achieving Program Outcomes Should be Improved” 

(GAO-07-58, January 2007)
RESULTS

REPORT NO. IG-09-022-R 23

recommended that the NASA Administrator direct the Centers to consider costs and 

benefits in choosing a CPAF contract by requiring, in accordance with the FAR, 

documentation explaining how the perceived benefits would offset the additional costs 

associated with contract administration. However, NASA has not had an opportunity to 

complete a thorough cost-benefit analysis because a follow-on contract has not been 

awarded. 

For a CPAF contract to be cost-effective, NASA needs to ensure objective metrics are 

developed and a supportable mechanism is established for evaluating the effectiveness of 

performance against desired results. Without a detailed cost-benefit analysis, NASA is 

unnecessarily increasing the risk of not getting the best value for the taxpayer. 

Trends in the Use of Cost-Reimbursement Contracts

In 1993, NASA consolidated the two CPFF contracts into a single CPAF contract for the 

operation and management of JPL. According to the NMO, the motivation behind 

moving to a cost-reimbursement contract was to inspire the contractor to perform as 

efficiently as possible while succeeding at its multiple research and development 

activities. The NMO Procurement staff also stated that the move served to respond to a 

GAO recommendation for NASA to use a cost-reimbursement contract for its FFRDC. 

However, the recommendation did not specify that NASA should use a single CPAF 

contract to procure services from Caltech for the management of JPL.

GAO’s 1993 report13

While a CPAF contract was in line with Government contracting trends in the 1990s 

that continue today, cost-reimbursement contracts are no longer the preferred vehicles 

for the private sector. According to the 14th Annual Government Contractor Survey, 

conducted in 2008 by Grant Thornton,

recommended that NASA “authorize a deviation from NASA 

policy to pay a fee only if its purpose and amount have been adequately justified in 

writing and, if a fee is authorized, apply NASA’s agencywide initiative for contract 

excellence to the JPL contract and base the fee on management performance.” GAO 

made this recommendation because NASA was paying Caltech an annual fee based on 

estimates of the volume of work to be conducted at JPL. This was despite a NASA 

policy prohibiting the payment of fees to educational institutions and to which no waiver 

had been obtained. 

14

 

13GAO. “NASA Procurement: Proposed Changes to the Jet Propulsion Laboratory Contract” 

(GAO/NSIAD-93-178, July 1993).

 Government agencies increased their use of 

cost-reimbursement contracts over the last 4 years, noting that the use of costreimbursement contracts within the Federal Government went from 28 percent in 

2005 to 45 percent in 2008. The Survey also notes that contracting in the private sector 

14Grant Thornton is one of the six global accounting, tax, and advisory organizations.
RESULTS

24 REPORT NO. IG-09-022-R 

“normally uses fixed-priced or time-and-materials contracts while the government 

continues to maximize the use of cost-reimbursable contracts.”

The Survey results indicate that private industry is moving in the direction called for by 

the Presidential Memorandum of March 4, 2009, which states that the Government must 

move away from cost-reimbursement contracts. The Presidential Memorandum also 

states that Federal Government executive agencies should not engage in non-competitive 

contracts unless their use can be fully justified and urges a preference for fixed-price type 

contracts. 

Use of Multiple Contracts May Save Time and Money

We concluded that using a single CPAF contract for the operation of the entire FFRDC is 

difficult for NASA to effectively manage because the contractor is responsible for 

multiple, complex, and often unrelated deliverables. For example, the contractor is 

delivering products that are crucial to NASA’s missions in the areas of astrophysics, earth 

sciences, solar system exploration, and technology. However, the JPL contractor is also 

responsible for such things as diversity and equal opportunity, media releases, 

information assurance, safety and mission assurance, and outreach. At other NASA 

Centers, contracts generally are awarded by project and evaluations of the contractors’ 

performance do not include the operation of normal center functions (for example, safety, 

public affairs, maintenance of the grounds). Assessing contractor performance in these 

multiple, diverse areas creates a significant burden on NASA resources and requires 

several different levels of management and staff, including scientists, Deputy Directors, 

Directors, and Associate Administrators at NASA Headquarters. 

For routine facility operations at the FFRDC, we believe that either a firm-fixed-price or a 

time-and-materials contract would be a more appropriate procurement vehicle. A fixedprice contract provides for a firm price or, in appropriate cases, an adjustable price. It can 

also provide for an adjustable price that includes a ceiling price, a target price, or both. A 

time-and-materials contract has the contractor acquire services on the basis of direct labor 

hours at specific fixed hourly rates and acquire supplies at cost. Either type of contract 

could be used effectively for JPL’s facility operations. This would place maximum risk 

and full responsibility for all contract costs on the contractor, incentivizing the contractor 

to control costs and perform effectively and efficiently. 

For research and development activities, we believe that a CPFF contract would be the 

most appropriate contract vehicle. A CPFF contract would allow the contractor’s costs to 

be covered and a set fee paid each contract year. While a CPAF contract is also a viable 

contract alternative, we believe that potential benefits of a CPAF contract are outweighed 

by the time and effort required to appropriately evaluate contractor performance for such 

a large and diverse contract. NASA could potentially save time and money by using a 

CPFF contract. 
RESULTS

REPORT NO. IG-09-022-R 25

Recommendations, Management’s Response, and Evaluation of 

Management’s Response

Recommendation 7. The NASA Management Office Procurement Officer should ensure 

implementation of the 2007 GAO recommendation to conduct a cost-benefit analysis to 

justify the use of CPAF contracts.

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that the NMO will coordinate with the Office of Procurement to ensure that a costbenefit analysis is conducted if a CPAF contract is used for the new contract. The action 

will be completed in conjunction with corrective actions taken in response to 

Recommendation 8. 

Evaluation of Management’s Response. Management’s proposed action is responsive. 

The recommendation is resolved and will be closed upon completion and verification of 

management’s corrective action.

Recommendation 8. NASA should consider alternative contract vehicles, or a mix of 

contract vehicles, for any FFRDC follow-on contracts. 

Management’s Response. The Assistant Administrator for Procurement concurred, 

stating that an alternative contract vehicle or a mix of contract vehicles will be considered

during the acquisition planning process for the new contract. The estimated completion 

date is fall 2009, when NMO will provide NASA Headquarters a set of contracting 

options. 

Evaluation of Management’s Response. Management’s proposed action is responsive. 

The recommendation is resolved and will be closed upon completion and verification of 

management’s corrective action.

APPENDIXES

REPORT NO. IG-09-022-R 27

APPENDIX A 

Scope and Methodology

We performed this audit from August 2008 through July 2009 in accordance with 

generally accepted government auditing standards. Those standards require that we plan 

and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable 

basis for our findings and conclusions based on our audit objectives. We believe that the 

evidence obtained provides a reasonable basis for our findings and conclusions based on 

our audit objectives. 

We concentrated our review on FY 2007 because, at the start of our audit, it was the most 

current complete contract period of performance available. We reviewed the 

Performance Evaluation Plan, the criteria used to evaluate the contractor, the PPEs’ 

primary performance evaluation reports, and the CPMs’ Performance Monitor Reports for 

FY 2007. We interviewed PPEs and CPMs to gain an understanding of how they 

evaluated the contractor and how they determined adjectival ratings and numerical scores. 

We used FY 2007 documentation in our discussions with the PPEs and CPMs, and we are 

confident that the answers we obtained were not from the “FY-2008 or FY-2009 

perspective,” as posited in management’s comments (see Appendix D, page 36).

We performed work at the NMO and the Defense Contract Audit Agency office located at 

JPL to assess the contract value and the values related to the work performed under the 

contract. We interviewed the NMO Procurement Officer, contracting officers, and

Defense Contract Audit Agency representatives. To determine whether the requirements 

to operate JPL as an FFRDC were met for FYs 2004–2008 and 2009–2013, we reviewed 

the use and need determination documents. 

Computer-Processed Data. We used computer-processed data to perform this audit. 

The primary performance evaluation reports and Performance Monitor Reports are 

entered into the JPL Evaluation Tool (JET) system, a system set up by the NMO in 2007 

to track and archive the documents. The NASA Headquarters Information Technology 

Support Services contractor designed the JET system, but the contract was not renewed 

and a new contractor took responsibility for implementing the JET system. However, 

software problems arose with the transition and some data entered into the system for 

FY 2007 was lost. Therefore, we do not have reasonable assurance that we received all of

the documents used to brief the PEB and the FDO. However, the NMO provided us data 

that was last modified on March 14, 2007, and was thought to be the most accurate and 

complete available. We believe that we received the majority of the documentation and, 

therefore, our audit work, findings, and conclusions were not affected.
APPENDIX A 

28 REPORT NO. IG-09-022-R 

OIG Calculation of the Weight of Cost Control Measures. Although the following 

calculation is specific to the FY 2007 award fee criteria, similar calculations were made 

for award years FY 2004 through FY 2006 and FY 2008. Criteria in both the 

programmatic and institutional performance evaluation factors included some measure of 

cost control. As shown in Appendix C, 12 of the 45 criteria for FY 2007 had some 

measure of cost control. We calculated the weight of those cost control measures to be 

12 percent of the total weighted evaluation factors (11.24 percent under the programmatic 

factor and 1.19 percent under the institutional factor; rounded from 12.43 percent to 

12 percent), which is well below the required 25 percent. Following are details of our 

calculations.

Programmatic Performance Evaluation Factor. The Performance Evaluation Plan 

weighted the programmatic factor at 65 percent. The factor comprised 21 criteria 

addressing three elements: technical, schedule, and cost. The Performance Evaluation 

Plan did not specify a weight for each element; therefore, we considered each to be 

equally important and assigned a weight of one-third (0.33) of the programmatic 

factor’s weight. 

Weight for each of the three elements (technical, schedule, and cost): 

0.33 x 65 percent = 21.45 percent 

Of the programmatic factor’s 21 criteria, 11 (52 percent) included a cost control 

measure. We calculated the weight for cost control measures using the weight for the 

cost element (21.45 percent) and the percentage of the factor’s criteria that included 

cost control. 

Weight of cost control measures in the programmatic factor: 

52 percent x 21.45 percent = 11.15 percent

Institutional Performance Evaluation Factor. The Performance Evaluation Plan 

weighted the institutional factor at 25 percent. Of the factor’s 21 criteria, 

1 (4.76 percent) addressed cost. We applied the institutional factor’s full weight 

(25 percent) to the percentage represented by the criterion addressing cost. 

Weight of cost control measures in the institutional factor: 

4.76 percent x 25 percent = 1.19 percent

Review of Internal Controls 

We reviewed and evaluated the internal controls associated with documenting the 

evaluation of performance, assigning adjectival ratings and numerical scores, and 

determining the final award fee score. We found deficiencies in all three areas, as 

discussed in this report. Our recommendations, if implemented, should correct the 

weaknesses we identified. 
APPENDIX A 

REPORT NO. IG-09-022-R 29

Prior Coverage

During the last 5 years, GAO issued a report of particular relevance to the subject of this 

report. A 2004 NASA report was also relevant. Unrestricted reports can be accessed 

over the internet at http://www.gao.gov (GAO) and 

http://oig.nasa.gov/audits/reports/FY04 (NASA). 

“Federal Contracting: Guidance on Award Fees Has Led to Better Practices but Is Not 

Consistently Applied” (GAO-09-630, May 2009) 

Government Accountability Office

“NASA Procurement: Use of Award Fees for Achieving Program Outcomes Should Be 

Improved” (GAO-07-58, January 17, 2007) 

“Audit of Incentive/Award Fee Structure Under the Space Flight Operations Contract” 

(IG-04-014, March 23, 2004) 

National Aeronautics and Space Administration
APPENDIX B 

30 REPORT NO. IG-09-022-R 

AWARD FEE PROCESS

NASA’s Award Fee Contracting Guide states that the award fee earned is calculated by 

applying the total numerical score (the award fee score) to the award fee pool. The Fee 

Determination Official (FDO) sets the award fee score, based on information provided by 

the Performance Evaluation Board (PEB). That information is an accumulation of 

assessments by NASA evaluators: Primary Performance Evaluators (PPEs) and Contract 

Performance Monitors (CPMs). 

At the time of our audit, NASA had 18 PPEs for the JPL contract: SMD had six, one for 

each program, and other Directorates and Offices each had one. CPMs monitor and 

assess the contractor’s performance, providing biannual Performance Monitor Reports 

and recommending an adjectival rating to the appropriate PPE. 

The PPEs use the Performance Monitor Reports to assign numerical scores and to prepare 

a primary performance evaluation report that summarizes the CPMs’ assessments of the 

contractor’s performance. The PPEs then prepare summary charts and make 

presentations to the PEB as to the strengths and weaknesses of the contractor and 

recommending an overall numerical score. PEB members discuss the evaluations and 

provide the FDO with their recommendation. The FDO decides on the final award fee 

score, which is used to make award fee and award term determinations.
APPENDIX C 

REPORT NO. IG-09-022-R 31

FY 2007 EVALUATION 

CRITERIA

The following table shows the 45 criteria of the three peformance evaluation factors 

(programmatic, institutional, and outreach), the NASA evaluators’ FY 2007 

recommended rating, and whether the criterion addressed cost control. The 

 . 

No. Directorate/Office Criteria

Performance 

Evaluation 

Factor 

Recommended

Rating

Cost 

Control 

1 SMD/Astrophysics Astro-1 Programmatic 

2 SMD/Astrophysics Astro-2 Programmatic 

3 SMD/Astrophysics Astro-3 Programmatic 

4 SMD/Astrophysics Astro-4 Programmatic 

5 SMD/Earth Science ES-1 Programmatic 

6 SMD/Earth Science ES-2 Programmatic 

7 SMD/Earth Science ES-3 Programmatic 

8 SMD/Earth Science ES-4 Programmatic 

9 Exploration Systems Mission ESMD-1 Programmatic 

10 SMD/Heliophysics Helio-1 Programmatic 

11 SMD/Heliophysics Helio-4 Programmatic 

12 SMD/Management and Policy 

Division MPD-2 Programmatic 

13 SMD/Management and Policy 

Division MPD-3 Programmatic 

14 SMD/Mars Exploration 

Program Mars-1 Programmatic 

15 SMD/Mars Exploration 

Program Mars-2 Programmatic 

16 SMD/Mars Exploration 

Program Mars-3 Programmatic 

17 SMD/Planetary Science PS-1 Programmatic 

18 SMD/Planetary Science PS-2 Programmatic 

19 SMD/Planetary Science PS-3 Programmatic 

20 SMD/Planetary Science PS-4 Programmatic 

21 Space Operations Mission SOMD-1 Programmatic 
APPENDIX C 

32 REPORT NO. IG-09-022-R 

No. Directorate/Office Criteria

Performance 

Evaluation 

Factor 

Recommended

Rating

Cost 

Control 

22 Chief Engineer CEO-1 Institutional 

23 Chief Engineer CEO-2 Institutional 

24 Chief Engineer CEO-3 Institutional 

25 Chief Information Officer CIO-1 Institutional 

26 Chief Information Officer CIO-2 Institutional 

27 Chief Information Officer CIO-3 Institutional 

28 Diversity and Equal 

Opportunity D&EO-1 Institutional 

29 Diversity and Equal 

Opportunity D&EO-2 Institutional 

30 Infrastructure and 

Administration OIA-1 Institutional 

31 Infrastructure and 

Administration OIA-2 Institutional 

32 Infrastructure and 

Administration OIA-3 Institutional 

33 Infrastructure and 

Administration OIA-4 Institutional 

34 Infrastructure and 

Administration OIA-5 Institutional 

35 Infrastructure and 

Administration OIA-6 Institutional 

36 Infrastructure and 

Administration OIA-7 Institutional 

37 NASA Management Office NMO-1 Institutional 

38 NASA Management Office NMO-2 Institutional 

39 NASA Management Office NMO-3 Institutional 

40 Safety and Mission Assurance OSMA-1 Institutional 

41 Safety and Mission Assurance OSMA-2 Institutional 

42 Safety and Mission Assurance OSMA-7 Institutional 

43 Small and Disadvantaged 

Business Utilization SDBU-1 Outreach 

44 Strategic Communications CSC PA-1 Outreach 

45 Strategic Communications CSC PA-2 Outreach 
APPENDIX D 

REPORT NO. IG-09-022-R 33

MANAGEMENT COMMENTS
APPENDIX D 

34 REPORT NO. IG-09-022-R 
APPENDIX D 

REPORT NO. IG-09-022-R 35
APPENDIX D 

36 REPORT NO. IG-09-022-R 
APPENDIX D 

REPORT NO. IG-09-022-R 37
APPENDIX E 

38 REPORT NO. IG-09-022-R 

REPORT DISTRIBUTION

National Aeronautics and Space Administration

Administrator

Deputy Administrator

Assistant Administrator for Procurement

NASA Management Office Director, JPL

NASA Management Office Procurement Officer 

Note: The redacted version of this report was distributed to non-NASA organizations 

and individuals and to members of Congress. 

Non-NASA Organizations and Individuals

Office of Management and Budget

Deputy Associate Director, Energy and Science Division

Branch Chief, Science and Space Programs Branch

Government Accountability Office 

Director, Defense, State, and NASA Financial Management, Office of Financial 

Management and Assurance

Director, NASA Issues, Office of Acquisition and Sourcing Management 

Congressional Committees and Subcommittees, Chairman and 

Ranking Member

Senate Committee on Appropriations 

Subcommittee on Commerce, Justice, Science, and Related Agencies

Senate Committee on Commerce, Science, and Transportation

Subcommittee on Science and Space

Senate Committee on Homeland Security and Governmental Affairs

House Committee on Appropriations 

Subcommittee on Commerce, Justice, Science, and Related Agencies

House Committee on Oversight and Government Reform 

Subcommittee on Government Management, Organization, and Procurement 

House Committee on Science and Technology 

Subcommittee on Investigations and Oversight 

Subcommittee on Space and Aeronautics 
REPORT NO. IG-09-022-R 39

Major Contributors to the Report: 

Ray Tolomeo, Director, Science and Aeronautics Research Directorate

Vincent Scott (former Director, OIG Procurement Directorate) 

Diane Choma, Project Manager

Stephanie Chapman, Team Lead (former OIG management analyst) 

William Falter, Auditor 

Liz Shifflett, Report Process Manager
SEPTEMBER 25, 2009

REPORT No. IG-09-022-R 

OFFICE OF AUDITS

OFFICE OF INSPECTOR GENERAL

ADDITIONAL COPIES

Visit http://oig.nasa.gov/audits/reports/FY09 to obtain additional copies of this report, or contact the 

Assistant Inspector General for Auditing at 202-358-1232. 

COMMENTS ON THIS REPORT

In order to help us improve the quality of our products, if you wish to comment on the quality or 

usefulness of this report, please contact Mr. Joel Rodriguez, Quality Assurance, at 

Joel.Rodriguez@nasa.gov or 202-358-4440. 

SUGGESTIONS FOR FUTURE AUDITS

To suggest ideas for or to request future audits, contact the Assistant Inspector General for Auditing. 

Ideas and requests can also be mailed to:

Assistant Inspector General for Auditing

NASA Headquarters

Washington, DC 20546-0001 

NASA HOTLINE

To report fraud, waste, abuse, or mismanagement, contact the NASA OIG Hotline at 800-424-9183 or 

800-535-8134 (TDD). You may also write to the NASA Inspector General, P.O. Box 23089, L’Enfant 

Plaza Station, Washington, DC 20026, or use http://oig.nasa.gov/hotline.html#form. The identity of 

each writer and caller can be kept confidential, upon request, to the extent permitted by law.