INTERIM REPORT ON AWARD-FEE CRITERIA 

FOR THE TRANSPORTATION INFORMATION 

PROJECT SUPPORT CONTRACT 

Volpe Center 

Research and Innovative Technology Administration 

Report Number: FI-2008-070 

Date Issued: August 14, 2008 
 

U.S. Department of 

Memorandum

Transportation 

Office of the Secretary 

of Transportation 

Office of Inspector General 

Subject: ACTION: Interim Report on Award-Fee Criteria 

 for the Transportation Information Project 

Support Contract 

Volpe Center, Research and Innovative 

 Technology Administration 

Report Number: FI-2008-070 

Date: August 14, 2008 

From: Mark H. Zabarsky 

Assistant Inspector General for 

 Acquisition and Procurement Audits 

Reply to 

Attn. of: JA-60 

To: Acting Director, Volpe Center 

As part of our ongoing audit of the Use of Cost-Plus-Award-Fee (CPAF) contracts 

within the Department, we are issuing this interim report regarding the award of 

the Volpe National Transportation Systems Center (Volpe)1

 Transportation 

Information Project Support (TRIPS) contract for information technology support. 

The objectives of our audit were to determine whether: (1) award-fee plans 

established adequate criteria for evaluating contractor performance, and (2) the 

amount of award fees paid to contractors was adequately supported.2

On February 7, 2006, Volpe awarded a 5-year, hybrid-structured contract to 

Computer Sciences Corporation (CSC) for a broad range of information 

technology support services. This contract includes both CPAF and fixed-price 

line items for approximately $178 million. Volpe established an award-fee pool 

totaling about $8.9 million. The contractor is currently in the fifth award-fee 

performance period.3

 Approximately $4.4 million in award-fees remain available 

for the sixth and subsequent performance periods. The TRIPS contract states that 

Volpe may unilaterally change the performance evaluation plan provided the 

contractor receives notice of the changes at least 45 days prior to the beginning of 

the evaluation period to which the changes apply. 

 

1

 Volpe is part of the Research and Innovative Technology Administration (RITA) of the Department of 

Transportation. 2

 This interim report does not address the second objective, which will be addressed in our Department-wide audit of 

award fees. 3

 A performance period is a 6-month timeframe. 
 2

We performed this audit in accordance with generally accepted government 

auditing standards as prescribed by the Comptroller General of the United States. 

The details of our scope and methodology are presented in Exhibit A. The 

information in this interim report will be included in a later report addressing 

Department-wide CPAF contracting issues. 

FINDINGS 

We found that Volpe did not structure the performance evaluation plan in a way 

that effectively motivated the contractor to improve performance and achieve 

acquisition outcomes. Volpe officials stated that they used the National 

Aeronautics and Space Administration’s (NASA) Award-Fee Contracting Guide 

as a reference in preparing the TRIPS performance evaluation plan. NASA 

developed this guidance to improve the effectiveness of using award-fee contracts. 

For example, the guidance emphasizes tying fees to outcome-based4

 criteria and 

evaluating the costs and benefits of such contracts before using this contract type. 

We found that Volpe did not always follow the preferred approach laid out in 

NASA’s guidance. For example, the criteria in the performance evaluation plan 

were vague and did not include measurable criteria needed to adequately evaluate 

contractor performance. Further, the descriptions defining adjectival ratings were 

vague and inconsistent and did not clearly define the basis for assigning such a 

rating. This resulted in performance monitors arbitrarily determining which 

ratings they believed best reflected how well the contractor performed. The effect 

of having evaluation criteria without clearly defined metrics, and vague and 

conflicting adjectival ratings, could result in inflated contractor performance 

evaluations and, consequently, inappropriately approved award fees. 

Also, although the Federal Acquisition Regulation (FAR) and NASA’s guidance 

require considering the costs and benefits before choosing a CPAF contract 

because of the cost and administrative burden associated with these contracts, 

Volpe did not perform such analysis. Through an evaluation of the administrative 

costs versus the expected benefits, the contracting officer should be able to assess 

whether the benefits the Government gains through a CPAF contract will 

outweigh the additional costs of overseeing and administering the contract. 

Without such an evaluation, Volpe had no assurance that a CPAF-type contract 

was appropriate. As a result of our audit, senior Volpe officials agreed to take 

action to address issues cited in this report. 

 

4

 Outcome-based criteria are measurable, quantifiable factors. They assess the results of an activity compared to its 

intended purpose. An example of an outcome criterion is to ensure that 99.5 percent of payment vouchers are paid 

within 30 days of receipt. 
 3

Performance Evaluation Plan Award-fee Criteria Were Too Vague 

The performance evaluation plan did not include clear and measurable award-fee 

criteria needed to adequately evaluate contractor performance. For example, the 

Quality of Deliverables criteria used undefined terms such as “achieves high 

quality deliverables” as the standard of review. The Timeliness of Deliverables 

criteria used general terms such as “Contractor provides timely notification of 

potential schedule slippages” and the Project Staffing and Planning factor used 

vague wording that the “Contractor effectively staffs, plans and manages job 

orders.” 

NASA’s guidance states that outcome-based criteria are the least administratively 

burdensome type of performance evaluation criteria, and should provide the best 

indicator of overall success. Guidance from other Federal Government agencies, 

such as the Departments of the Air Force, Navy, and Army, also prefer using 

measurable criteria to evaluate contractor performance. Additionally, guidance 

from these agencies, including NASA, states that using evaluation criteria that are 

too broad can result in evaluators not being able to provide meaningful comments 

to support ratings. 

Establishing specific criteria based on performance objectives would reduce the 

risk of unwarranted or subjective performance evaluations and ratings. 

Performance monitors, i.e., Volpe staff who track and assess contractor 

performance daily, cannot provide meaningful comments and evaluations using 

too-broadly-defined criteria. The effect of having evaluation criteria without 

establishing clear and measurable metrics for assessing performance could result 

in inflated contractor performance evaluations and, consequently, inappropriately 

approved award fees. For the first four award-fee periods of the TRIPS contract, 

Volpe paid the contractor approximately $3.4 million (93 percent of the available 

award fee) without assurance of whether the acquisition outcomes fell short of, 

met, or exceeded expectations. Without indicating areas of emphasis or desired 

outcomes, Volpe does not have assurances that contract objectives are being met, 

nor does the contractor have motivation to perform the best possible job in those 

areas deemed critical. 

Additionally, Volpe did not provide performance monitors with a performance 

evaluation plan containing the award-fee criteria. Instead, they provided the 

monitors a questionnaire to fill out at the end of each performance period. In our 

comparison of the award-fee criteria in the performance evaluation plan with the 

questionnaire, we found that three criteria were inadvertently missing from the 

questionnaire—under the category of Financial and Business Management. 

Having thorough and complete evaluation criteria is critical as it indicates to the 

contractor and performance monitors which aspects of contractor performance are 

most significant during the evaluation period. Missing criteria could result in 
 4

missed opportunities to motivate the contractor and help to mitigate significant 

Government risks. 

Adjectival Rating Factors Were Too Vague and Inconsistent for 

Rating Performance 

The conversion table (Exhibit B) used to compute the amount of award fee 

contains a payment structure that associates a range of award fees (between 0 and 

100) with four adjectival ratings—Excellent, Very Good, Satisfactory, and Poor. 

The descriptions defining the adjectival ratings, however, did not clearly define 

the basis for assigning such a rating. For example, a rating of very good is defined 

as: 

“Contractor’s performance of most contract tasks is consistently above 

standard and provides highly effective results; several strengths observed. 

The few, if any, weaknesses have minimal adverse effect on overall 

performance; there are no deficiencies. Although some areas may require 

improvement, these areas are minor and are more than offset by better 

performance in other areas. The few, if any, recurring problems have been 

noted. Contractor identifies problems in timely manner and solutions are 

implemented with no adverse impact on quality, cost, schedule, or 

timeliness.” 

Terms, such as “above standard,” “highly effective results,” and “timely manner” 

are undefined. Performance monitors used personal criteria, applied arbitrarily, 

that possibly resulted in the Government overpaying award fees with funds that 

could have been put to better use. Discussions with eight performance monitors 

revealed that each had a different interpretation on how to apply the adjectival 

ratings to contractor performance. For example, to rate the quality of the work 

performed by the contractor on a project, one monitor stated that an “excellent” 

rating equated to having one error in a report, while another monitor stated three. 

In another example, one monitor said that to assign a rating of “very good,” the 

contractor must perform only minor rework, while another said that the 

contractor’s performance was “above and beyond meeting the requirements 

without raving comments” from the sponsor. 

Additionally, Volpe’s performance evaluation plan for its TRIPS contract contains 

inconsistent descriptions of adjectival rating factors. For example, a 

“satisfactory” rating is defined, in part, as: 

“...Contractor’s performance of most contract tasks is effective and 

provides adequate results; some strengths and weaknesses observed; any 

reportable deficiencies have a limited impact on overall performance.” 
 5

We compared the above definition of a satisfactory rating to the plan’s description 

of the terms “weaknesses” and “deficiencies,” and found they do not support a 

satisfactory rating. 

 Table 1. Definitions of Performance Weakness and Deficiency 

Performance Weakness Performance Deficiency 

“Performance that has resulted in 

serious disruption in quality, effort, 

cost, schedule, or impact. Performance 

that has negatively impacted the 

project and increased the risk of 

unsuccessful delivery of high quality 

products or services on time and/or 

within budget.” 

“Performance that has created 

unacceptable results and/or failed to 

address, and/or meet contract, task 

order or job order level requirements or 

performance evaluation criteria.” 

Source: Volpe’s performance evaluation plan for TRIPS contract 

Volpe must clearly describe its adjectival ratings so there will be a basis for 

performance monitors and award-fee board to use in assessing contractor 

performance. 

Volpe Has No Assurance That Contract Type Is Appropriate For the 

TRIPS Contract 

Volpe contracting officials did not justify the cost effectiveness of selecting a 

CPAF-type contract. Performance evaluation on an award-fee contract requires 

greater effort and more resources than other types of contracts because oversight is 

required to monitor and document contractor performance. FAR and award-fee 

guides used by other Federal agencies require agencies to consider the costs and 

benefits before choosing a CPAF-type contract. For example, NASA guidance 

states that before a CPAF contract is selected, a contracting officer should perform 

a cost-benefit analysis of the expected benefits versus the added administrative 

costs. 
 6

RECOMMENDATIONS 

We recommend that the Acting Director, Volpe Center, direct the Chief of the 

Contracting Office, Volpe, to: 

1. Develop measurable award-fee criteria for evaluating contractor performance. 

2. Revise the questionnaire to include all award-fee criteria contained in the 

performance evaluation plan. 

3. Describe adjectival ratings clearly so there will be a defined basis for assessing 

performance and ratings consistent with the conversion table. 

4. Reevaluate the contract type for future TRIPS procurement contracts and 

justify the use of an award-fee contract by performing a cost/benefit analysis. 

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL’S 

RESPONSE 

We discussed the issues cited in this report with senior Volpe officials on February 

8, 2008. As a result, senior Volpe officials agreed to take the following actions to 

address the deficiencies cited in this report:5

• Update the current performance evaluation plan to add more measurable 

award-fee criteria. 

• Re-examine the adjectival rating definitions and improve them for the 

award-fee board to use in assessing contractor performance. 

• Conduct a training session with contracting officer technical representatives 

and job order initiators to ensure consistent performance monitoring and 

evaluation. 

• Reevaluate the contract type for future TRIPS procurement contracts and 

justify the use of an award-fee contract. 

As Volpe’s management actions are ongoing at the time of this report, we cannot 

evaluate the efficiency of these actions; however, we believe the results of these 

actions will correct problems identified in this report. Implementing the planned 

corrective actions will put approximately $4.4 million in expected award-fees to 

 

5

 Volpe’s planned corrective actions were provided in separate memorandum, dated June 12, 2008. 
 7

better use by revising the performance evaluation plan and ensuring Volpe’s 

acquisition objectives are being met. 

ACTIONS REQUIRED 

In accordance with Department of Transportation Order 8000.1C, we would 

appreciate receiving your written comments within 30 calendar days. If you 

concur with the recommendations, please indicate the specific action taken or 

planned and provide the target date for completion. If you do not concur with the 

findings or recommendations, please provide your rationale. You may provide 

alternative courses of action that you believe would resolve the issues presented in 

this report. Also, please comment whether you agree that the estimated $4.4 

million in award-fees for the TRIPS contract could be put to better use by revising 

the performance evaluation plan. 

We appreciate the courtesies and cooperation of Volpe representatives during this 

audit. If you have any questions concerning this report, please call me at (202) 

366-5225 or Terrence Letko, the Program Director, at (202) 366-9917. 

# 

cc: Research and Innovative Technology Administrator 

 Volpe National Transportation Systems Center Director 

Senior Procurement Executive 

Director, Office of Acquisition Services, RITA 

Martin Gertel, M-1 

Timothy Klein, RTG-30 

Dilcy Garro, Volpe Audit Liaison 
 8

EXHIBIT A. SCOPE AND METHODOLOGY 

This report is associated with our Department-wide audit of the Use of 

CPAF Contracts, Project Number 07F3011F000. We conducted this 

performance audit in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient, appropriate evidence to provide a reasonable basis 

for our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. 

To accomplish our objectives, we reviewed the: 

3⁄4 Performance evaluation plan; 

 

3⁄4 Statements of work and deliverables for the contracts and selected 

task orders; 

3⁄4 Performance monitors’ rating results; 

3⁄4 Federal Acquisition Regulation; 

3⁄4 Transportation Acquisition Regulation; 

3⁄4 Best Practices for award-fee contracts (Departments of the Army, Navy, 

and Air Force, National Aeronautics and Space Administration, and 

Environmental Protection Agency Award-Fee Guides) and; 

3⁄4 Grading methodologies of performance monitors. 

We also interviewed: 

3⁄4 Eight performance monitors or job order initiators to determine how they 

rated the contractors’ performance; and 

3⁄4 Volpe acquisition and program officials regarding the performance 

evaluation plan and the award-fee process. 

Exhibit A. Scope and Methodology 
 9

EXHIBIT B. TRANSPORTATION INFORMATION PROJECT 

SUPPORT CONTRACT AWARD-FEE CONVERSION CHART 

This chart is used to determine the percentage of award fees earned from the 

available award-fee pool:

Adjectival Rating Percentage of Fee 

Excellent 90-100 

Very Good 80-89 

Satisfactory 65-79 

Poor 0 

Exhibit B. Transportation Information Project Support Contract 

Award-Fee Conversion Chart 
 10

EXHIBIT C. MAJOR CONTRIBUTORS TO THIS REPORT 

Name Title 

Terrence Letko Program Director 

Dormayne Dillard-Christian Project Manager 

Christopher Minovich Senior Auditor 

Jelilat Ojodu Auditor 

Jean Diaz Writer/Editor 

Exhibit C. Major Contributors to This Report 
 

The following pages contain textual versions of the charts found in this document. 

These pages were not in the original document but have been added here to 

accommodate assistive technology. 
 

Interim Report on Award-Fee Criteria for the Transportation 

Information Project Support Contract 

Section 508 Compliant Presentation 

Table 1. Definitions of Performance Weakness and Deficiency 

Performance Weakness is described as “Performance that has resulted in serious 

disruption in quality, effort, cost, schedule, or impact. Performance that has 

negatively impacted the project and increased the risk of unsuccessful delivery of 

high quality products or services on time and/or within budget.” 

Performance Deficiency is described as “Performance that has created 

unacceptable results and/or failed to address, and/or meet contract, task order or 

job order level requirements or performance evaluation criteria.” 

Source is Volpe’s performance evaluation plan for TRIPS contract. 

Exhibit B. Transportation Information Project Support Contract Award-Fee 

Conversion Chart 

An adjectival rating of excellent earns a 90-100 percent award fee from the 

available award-fee pool. 

An adjectival rating of very good earns an 80-89 percent award fee from the 

available award-fee pool. 

An adjectival rating of satisfactory earns a 65-79 percent award fee from the 

available award-fee pool. 

An adjectival rating of poor does not earn a percent of the award fee from the 

available award-fee pool.