Defense Contracting: Enhanced Training Could Strengthen DOD's Best Value Tradeoff Decisions
Released: 2010-10-28T00:00:00

Letter

Background

The Federal Acquisition Regulation (FAR) Part 15 allows the use of several best value competitive source selection techniques to meet agency needs. Within the best value continuum, DOD may choose an approach that it considers the most advantageous to the government, including the lowest price technically acceptable (LPTA) process and the tradeoff process.

DOD may elect to use the LPTA process in acquisitions where the requirement is clearly definable and the risk of unsuccessful contract performance is minimal. In such cases, DOD may determine that cost or price should play a dominant role in source selection. When using the LPTA process, DOD specifies its minimal technical requirements in the solicitation. Once DOD determines that the contractors meet or exceed the technical requirements, no tradeoffs between cost or price and non- cost factors are permitted and the award is made based on the lowest price offered to the government.

By contrast, DOD may elect to use a tradeoff process in acquisitions where the requirement is less definitive, more development work is required, or the acquisition has greater performance risk. In these instances, non-cost evaluation factors, such as technical capabilities or past performance, may play a dominant role in the source selection and tradeoffs among price and non-cost factors allow DOD to accept other than the lowest priced proposal. This report focuses on DOD’s use of the tradeoff process, and specifically, in which non-cost factors, when combined, were considered more important than cost or price.

When using a tradeoff process, the FAR requires that evaluation factors and significant subfactors that affect contract award and their relative importance be clearly stated in the solicitation; and the solicitation must provide whether all evaluation factors other than cost or price, when combined, are significantly more important than, approximately equal to, or significantly less important than cost or price. Additionally, the FAR requires that each factor represent key areas of importance and emphasis to be considered in the source selection decision and that support meaningful comparison and discrimination between and among competing proposals. The FAR also requires the source selection authority document the perceived benefits of the higher priced proposal and the rationale for tradeoffs in the contract file. The resulting source select decision should be based on a comparative assessment of proposals against all source selection criteria in the solicitation. The decision must also include the rationale for any business judgments and tradeoffs made or relied on by the source selection official, including benefits asso ciated with additional costs. Although the rationale for the source selection decision must be documented, the documentation need not quantify the tradeoffs that led to the d ecision.

In fiscal year 2009, DOD obligated about $380 billion on contracts for goods and services. Our analysis of data reported by DOD to FPDS-NG indicates that $69.9 billion or 18 percent of DOD’s obligations were made on new contracts competitively awarded in fiscal year 2009 (see figure 1). By contrast, about $176 billion were modifications to or orders issued under contracts that were awarded prior to fiscal year 2009 and $133 billion were awarded non-competitively, which in combination totaled to nearly 82 percent of DOD’s reported contract obligations in fiscal year 2009.

Properly managing the acquisition of goods and services requires an acquisition workforce with the right skills and capabilities. In March 2009, however, we reported that DOD lacked complete information on the skill sets of the current acquisition workforce and whether these skill sets were sufficient to accomplish its missions. In April 2009, the Secretary of Defense announced his intent to grow the acquisition workforce by 15 percent by fiscal year 2015. As part of this strategy, DOD indicated that it intends to grow its contracting career field by more than 6,400 personnel, an increase of more than 28 percent from fiscal year 2008 staffing levels.

DOD Relies Heavily on Best Value Processes to Evaluate Contractor Offers

DOD relies heavily on the use of the best value process to evaluate offers from potential contractors. DOD chose a best value process for approximately 95 percent of its new, competitively awarded contracts on which it had obligated $25 million or more in fiscal year 2009. Almost half of DOD’s contracts—47 percent—were awarded using a tradeoff process in which non-cost evaluation factors, when combined, were more more important than price. Figure 2 shows how often DOD used the different important than price. Figure 2 shows how often DOD used the different best value processes and other source selection approaches. best value processes and other source selection approaches.

Bet ve process (95%)

Bet ve trdeoff process (69%)

In 69 percent of the contract awards, DOD used the best value tradeoff process. When doing so, it acquired services approximately four times as often as it acquired products. Over half of these procurements were for building or civil engineering construction services, including projects for troop housing, administrative facilities, and hurricane protection systems. Other services procured using the tradeoff process were equipment maintenance and professional management services. For example, in fiscal year 2009, the Army Corps of Engineers awarded a contract worth more than $963 million to construct one of the largest pumping stations in the world, along with floodgates and floodwalls for hurricane protection. Similarly, the Air Force awarded the Contract Field Team program multiple-award contract, with an estimated base value of $2.6 billion for modification, maintenance and repair of systems including aircraft and missile defense for the departments of the Army, Navy, Air Force and several federal agencies. Small arms and electronic countermeasure equipment were among the products most frequently procured using a tradeoff process, including the contracts for the Squad Automatic Weapons—lightweight, automatic rifles issued to each Army and Marine rifle squad—and an Army contract to procure devices that counteract radio-controlled improvised explosives. Our analysis of selected characteristics of contracts awarded using a best value tradeoff process in fiscal year 2009 is shown in figure 3.

Percentages may not sum to 100 percent due to rounding.

Combination refers to contracts that allow for orders to be placed using more than one pricing arrangement.

As part of our work, we reviewed 10 IDIQ contracts that had been awarded using a best value tradeoff process and 23 task and delivery orders under these contracts which had obligations ranging from $11 million to over $319 million. In most cases, DOD did not issue the task or delivery orders we reviewed using a tradeoff process. For example, an Air Force official explained that the initial task orders for the Contract Field Team program, including 13 orders in our sample, were issued under an LPTA process because data needed to assess contractor performance and timeliness were not yet available to use given the short time between award of the base contract and issuance of the first orders. An Air Force official indicated that once they had obtained sufficient performance data, they intended to issue task orders using a tradeoff process when possible. DOD officials issued six other task orders on the basis of negotiating with a contractor who had been awarded a single award IDIQ contract. The four remaining orders were awarded using a tradeoff process. For example, the Army wanted infrared vision enhancement equipment for nighttime and battlefield use in Iraq and Afghanistan to be delivered as quickly as possible. Consequently, the Army used a contractor’s ability to meet delivery requirements as the principal evaluation factor in selecting the contractor for delivery order award.

Some DOD officials noted that the use of various source selection evaluation methods can change over time. For example: Defense Logistics Agency (DLA) officials noted that they have recently transitioned from principally using the tradeoff process to using the LPTA process for most fuel purchases because the majority of their procurements were for a commercial product in relatively stable domestic and international markets. They noted, however, that they still use the tradeoff process in less stable areas, such as Iraq and Afghanistan, where they require more information about vendors’ past performance and technical capability when operating in war zones.

Conversely, Army Corps officials in New Orleans reported that they have been using the tradeoff process more frequently since the increase in civil works construction projects following Hurricane Katrina. While they typically used sealed bids in the past, they told us that use of the tradeoff process enabled them to better assess contractors’ ability to meet safety and schedule requirements.

DOD Used a Best Value Tradeoff Process to Address Complex or Time Sensitive Needs, but Paid Relatively Few Price Differentials

DOD officials tended to use a best value tradeoff process with non-cost factors weighted more important than price when they were willing to accept a higher price if a contractor could demonstrate certain advantages, such as meet a deadline, demonstrate that it understood complex technical issues, or propose an innovative approach. DOD often indicated in tradeoff solicitations that non-cost factors would be significantly more important than price in making award decisions, but our analysis indicated that DOD selected a lower priced proposal among those offerors remaining in the final competition almost as often as it selected a higher technically rated, but more costly, proposal. Overall, DOD paid a price differential—the difference in the price of the offeror awarded the contract and the price of the offeror next in line for award—in 21 of the 68 contracts in which a price differential was considered. Most differentials were less than 5 percent. While DOD officials told us that the tradeoff process provides an essential tool to obtain desired capabilities, they rely on the case-by-case judgment of contracting and program officials to determine the best acquisition approach suited to program requirements and do not specifically track whether use of the tradeoff process is in DOD’s interest.

Contract Requirements Drove DOD’s Selection of Technical and Past Performance Evaluation Factors

The FAR and DOD guidance generally provide acquisition staff flexibility to develop evaluation factors that meet their procurement needs and does not indicate which evaluation factors should be most important. The FAR requires that DOD officials consider, among other things, past performance on all negotiated competitive acquisitions exceeding $100,000, but DOD officials have broad discretion in selecting other non- cost factors and their relative importance. The factors are intended to provide meaningful discriminators to evaluate proposals. Army, Navy, and Air Force officials told us that they formed interdisciplinary teams that developed evaluation factors and the factors’ relative importance by consensus.

We found that 88 of the 129 contracts we reviewed used a best value tradeoff process. Our analysis shows that DOD considered past performance and technical evaluation factors as the most important among the non-cost factors. Figure 4 shows the five most frequently used non-cost evaluation factors for the 88 contracts in our review in which a tradeoff was conducted and how often the technical and past performance factors were most important among the non-cost factors.

Air Force

DOD officials told us that the selection of these evaluation factors and their relative importance was based on specific acquisition requirements, such as the ability to meet production deadlines, ensure compatibility with existing ship and aircraft systems, or provide needed security for delivery of goods in war zones. Our analysis found that DOD considered non-cost factors more important than price in 60 of the 88 contracts awarded using a tradeoff process. The following illustrate instances where DOD’s acquisition needs led them to make non-cost factors the principle criteria for source selection.

Army officials had to quickly meet surge requirements based on a Joint Urgent Operational Needs Statement for roadside bomb detectors as well as services to provide training and support the system once fielded, and accordingly, made technical capability the most important evaluation factor. The acquisition plan specified that deliveries of the critical technology and support services needed to be made within 6 months of contract award.

Army officials sought contractors with innovative approaches and a superior understanding on how to counter the threat of roadside bombs in awarding a professional services contract for a range of training programs to be used within the Military Service Combat Training Centers. According to the Army, the selected contractor provided a proposal that was superior in nearly all the technical categories sought by the Army.

The Navy considered contractors’ proposed technical approach the most important evaluation factor for a helicopter upgrade kit procurement because the design had to be compatible with existing helicopters. In this case, the timing of fleet deployment was also critical and the Navy sought a contractor that could meet their schedule.

DLA used a tradeoff process primarily for commercial fuel contracts in dangerous areas, such as Iraq and Afghanistan, due to the heightened need for contractor reliability in these war zones. In these situations, DLA officials explained the tradeoff process allowed them to emphasize security and past performance in their evaluations to mitigate acquisition risks, especially since they do not know the vendors well.

Army Corps of Engineers officials that needed to procure construction services for barracks for wounded soldiers made the technical and performance capability factors most important because they needed to be responsive to new schedule and price targets. These officials used the tradeoff process to incentivize timeliness and price reductions, and they were also able to obtain better features, such as more durable materials.

DOD officials also told us they used these non-cost factors to encourage contractors to provide innovative solutions to meet DOD’s needs. For example, Army officials expressed a need for technological innovation in a solicitation for equipment, field support services, and associated maintenance needed to intercept enemy communications. The Army encouraged the contractors to develop a system that would enable them to upgrade the equipment frequently over the life of the contract. The statement of work clarified that these upgrades would be essential to maintain relevancy in the battlefield and keep pace with technology advancements. Similarly, Marine Corps officials we spoke to about an urban warfare training system told us that they used the tradeoff process to seek innovative designs when awarding a 5-year, $1 billion dollar contract. Marine Corps officials indicated that they had a system that worked, but wanted to push industry to come up with a solution that allowed the Marines to reconfigure building structures more quickly and to provide more realistic and current combat scenarios prior to deployment. In the winning design, the offeror proposed using modular building sets that Marines could assemble more quickly to maximize the training opportunities available in the field.

In contrast, our analysis found that the 28 cases in which DOD officials considered non-cost factors as equal to or less important than price were nearly all related to construction projects. For example, in 15 cases we reviewed, the Army Corps of Engineers considered non-cost factors such as management and technical, experience, and past performance as equal to price to address less complex project requirements such as building a new runway for aircraft and constructing a maintenance facility. In these instances, the contracting and program officials were able to request and review information from potential contractors and conduct a tradeoff process that would not be available through an LPTA approach, but still considered price of equal importance to non-cost factors in the award decision.

DOD Paid Relatively Few Price Differentials Despite Best Value Tradeoff Solicitations That Emphasized Non-Cost Factors

For the 88 contracts awarded using a best value tradeoff process, DOD considered whether to pay a price differential in 68 contracts. Our analysis indicated that DOD selected the lower priced option nearly as often as it selected the highest rated, but more costly, proposal. In the 18 cases in which DOD officials decided not to pay a price differential, they determined that the lower price outweighed the advantages of the offeror with the higher technical rating. In doing so, DOD officials decided not to pay over $800 million in price differentials. In 29 other cases, DOD awarded contracts to the offerors that had both the lowest price and the highest non-cost factor rating.

DOD accepted a higher price in 21 of the 68 contracts in which a price differential was considered, for a combined difference of more than $230 million. Most differentials paid were less than 5 percent above the price submitted by the offeror next in line for award. The largest price differential from the contracts in our sample was 48 percent higher, or roughly $13.6 million more, than the next in line offeror’s price. In this case, Marine Corps officials determined that the product—burn resistant clothing for use by soldiers in Iraq—was worth the price difference because it provided substantially greater 2nd and 3rd degree burn protection than the product proposed by the other offeror. Figure 5 shows the frequency with which DOD elected to pay or not pay a price differential for the 68 contracts in which a price differential was considered, as well as the value of the price differentials either paid or not paid.

DOD Relies on the Judgment of Contracting and Program Officials to Determine Whether Use of Best Value Tradeoffs Meet DOD’s Needs

DOD contracting and program officials believed that the use of best value tradeoffs provide DOD an essential tool, which allows them to obtain better insights into the contractors’ capabilities and their understanding of the government’s needs, and the reasonableness of the contractor’s approach. DOD and military department officials stated that they do not specifically track use of the tradeoff process to determine if DOD’s interests are met. Instead, they rely on the judgment of contracting and program officials to select the best acquisition approach suited to program requirements on a case by case basis. Further, DOD officials stated that they do not track whether the solicitation approach used correlated with whether the contractor successfully met the terms of the contract and noted that many factors ultimately contribute to the success or failure of an individual acquisition that may not have been foreseeable when awarding the contract. For example, DOD officials noted that DOD would often use a best value tradeoff process to award a contract to develop a major weapons system. As our work has found, DOD often encounters cost increases, schedule delays, and performance shortfalls on its major systems.

DOD Faces Several Challenges in Using the Best Value Tradeoff Process

DOD officials acknowledged several challenges in using the best value tradeoff process such as the difficulties in developing meaningful evaluation factors, the additional time investment needed to conduct best value procurements, and the business judgment required of acquisition staff when compared to other acquisition approaches. DOD officials also noted that the complexity of the tradeoff process increases the risk of bid protests. To help address source selection challenges, DOD is drafting a source selection guide to improve consistency and standardize source selection procedures for competitively awarded negotiated procurements.

DOD officials told us that developing non-cost factors that meaningfully discriminate between offers is a challenging part of the tradeoff process. They noted that as the complexity of the acquisition increases, so does the need for individuals with the expertise to help develop the evaluation factors. For example, Army Corps of Engineers officials told us that the contract for one of the world’s largest flood pump stations required experts with experience in issues ranging from water flow management to real estate to develop evaluation factors. Further, Navy officials explained that while they often use past performance as a non-cost discriminator, it can be difficult to identify differences between contractor proposals because contractors often provide their best performance examples and the government often lacks data to evaluate additional contractor projects. Our past work has also identified governmentwide challenges in obtaining needed past performance information to support contract award decisions. Further, the absence of meaningful non-cost discriminators can result in offerors receiving equal scores on the factors that were identified as being significantly more important than price. As such, the decision may default simply to a consideration of price alone. For example, Air Force officials noted that they are considering updating factors used to award task orders under the Contract Field Team contract because contractors tend to receive the highest ratings for each non-cost factor reviewed, so price is typically the only discriminator.

DOD officials also noted that using the best value tradeoff process is often far more time-consuming than other approaches. Navy officials told us that the tradeoff process is administratively burdensome and requires a large time investment from program staff, which can make it challenging to keep the same acquisition team together for an entire procurement. During our site visits, many contract and program staff told us that the tradeoff process often takes between 18 and 24 months. In addition, in Afghanistan and Iraq, the challenges of conducting a tradeoff process have contributed to decisions by the CENTCOM Joint Theater Support Contracting Command and Army Corps of Engineers to discourage its use. For example, recent Army Corps of Engineer projects in Afghanistan have emphasized using simpler, less complex designs or requirements that are more suitable for the use of a lowest price technically acceptable approach.

The complex nature of the best value tradeoff process, including decisions on whether to pay a price differential, requires much greater business judgment when compared to other acquisition approaches. DOD officials stated that making tradeoff decisions, particularly when to pay a price differential, is among the most difficult aspects of the tradeoff process, which will become more challenging with less experienced staff coming into the acquisition workforce. DOD officials indicated that DOD intends to increase the size of its contracting career field by more than 6,400 personnel through fiscal year 2015. With the influx of new staff, many of the contracting officers we met with noted challenges in preparing staff to conduct the tradeoff process. For example, a Navy contracting officer told us that guidance and training only go so far to prepare acquisition staff to conduct best value tradeoff procurements. Instead, acquisition staff need to be involved in a number of best value tradeoff procurements to develop the business judgment necessary to conduct a successful acquisition.

DOD officials stated that the complexity of the tradeoff process also increases the risk of bid protests. Of the 88 contracts we reviewed that used a tradeoff process, 15 were the subject of a bid protest to GAO. While most of the protests were denied, DOD took corrective actions in 5 cases, including 4 cases in which DOD terminated the contract or made a new source selection decision when it determined that it failed to adhere to the solicitations’ requirements.

Some of the services have developed initiatives to address these challenges. For example, the Air Force set up an Acquisition Center of Excellence (ACE) at Tinker Air Force Base, which provides pre-award source selection assistance to contract and program staff. Air Force officials stated that ACE reviews the evaluation factors within individual source selection plans, serves in an advisory capacity on source selection teams and holds workshops for contracting officers. Similarly, Army officials at Ft. Monmouth’s Communications—Electronics Command have developed an online business tool—the ASSIST tool—that shepherds contracting officers through the solicitation process. For example, the tool provided a list of steps that must be completed for best value tradeoff procurements and automatically routes documents through source selection evaluation boards and other participating officials for review, as required.

DOD is also drafting a departmentwide source selection guide to improve consistency and standardize source selection procedures for competitively awarded negotiated procurements. Given the influx of new acquisition staff, DOD officials stated they wanted to develop a more prescriptive guide for best value procurements. While the DOD draft source selection guidance contains information on various aspects of the best value process, such as the source selection decision document, DOD officials told us it does not address price differentials. Numerous DOD officials underscored the importance of training in the use of the best value process, particularly training that addresses the tradeoff decision that acquisition staff must make. For example, one Army Corps of Engineers official told us that source selection officials would benefit by training that contains real life lessons on how other officials have made price differential decisions during the tradeoff process. Similarly, Marine Corps and Army officials told us that while decisions are made on a case-by-case basis, informal rules of thumb regarding price differentials can come into play and indicated that additional guidance or training, especially case studies or scenarios, would be helpful.

The Defense Acquisition University (DAU) is responsible for providing training to the DOD acquisition workforce. According to DAU officials, they offer more than 10 courses that contain elements of the best value tradeoff process, but none of the current courses provide case studies or scenarios that focus on reaching price differential decisions during source selection. They noted that once the new source selection guidance is implemented, which is anticipated for January 2011, they plan to augment existing contracting courses to reflect the new guidance.

Conclusions

The best value tradeoff process underlies the vast majority of DOD competitively awarded contracts, and effective use of this process hinges on making sound tradeoffs between price and non-cost factors. By focusing on non-cost factors, DOD anticipates that it will obtain technical solutions that are innovative and address complex and time-sensitive program requirements. Applying a tradeoff process, however, does not guarantee successful acquisitions, nor is it without other challenges. In particular, using a tradeoff process can be more complex and take more time than other source selection methods, and requires that acquisition staff have proper guidance, needed skills, and sound business judgment. With the anticipated influx of more than 6,400 DOD contracting personnel over the next few years, providing a firm foundation for use of the tradeoff process is essential. While DOD and the military departments have taken steps to improve source selection procedures, acquisition personnel noted a lack of training to assist them in deciding whether or not a price differential is warranted when making tradeoff decisions. For example, while DOD’s new source selection guide provides insights on the source selection process, it is silent on how to reach decisions on when to pay a price differential, as is DOD’s current training curriculum. DOD has an opportunity as it updates its training curriculum to provide acquisition staff with better insights using real life examples on reaching tradeoff decisions. Taking this step can help DOD minimize the risk of paying a price differential when not warranted or losing the benefit of a technically superior solution.

Recommendation for Executive Action

To help DOD effectively employ the best value tradeoff process, we recommend that the Secretary of Defense direct the Director of Defense Procurement and Acquisition Policy to work with the Defense Acquisition University to develop training elements, such as case studies or scenarios that focus on reaching tradeoff decisions, including consideration of price differentials, as it updates the source selection curriculum.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. DOD concurred with our recommendation and intends to request the Panel on Contracting Integrity—comprised of senior DOD leaders tasked, in part, to help improve DOD's performance—to assist the Defense Acquisition University in developing training case studies and scenarios that focus on reaching tradeoff decisions. DOD’s letter is reprinted in appendix II.

We are sending copies of this report to interested congressional committees and the Secretary of Defense. In addition, this report will be available at no charge on GAO’s Web site at http://www.gao.gov.

Should you or your staff have any questions on the matters covered in this report, please contact me at (202) 512-4841 or huttonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

Section 845 of the National Defense Authorization Act for Fiscal Year 2010 directed GAO to report on the Department of Defense’s (DOD) use of the best value tradeoff process, and specifically for cases in which DOD evaluated contractors’ proposals on factors other than cost or price, if these non-cost factors, when combined, were considered more important than cost or price. To respond to the mandate, we determined (1) how often and for what types of contracts DOD used the best value tradeoff process; (2) why and how DOD used the best value tradeoff process; and (3) what challenges, if any, DOD faces in using the best value tradeoff process.

To determine how often and for what types of contracts DOD used the best value tradeoff process, we used data from the Federal Procurement Data System-Next Generation (FPDS-NG) as of January 2010 to identify a population of contracts based on the following criteria: (1) newly awarded by DOD in fiscal year 2009; (2) competitively awarded, and (3) had obligations of $25 million or more in fiscal year 2009.

We established the $25 million threshold because the Defense Federal Acquisition Regulation Supplement (DFARS) requires contracts with total estimated costs of $25 million or more in any fiscal year to prepare written acquisition plans, which contain information about the source selection approach. This analysis identified 363 contracts. From this population, we selected a probability sample of 160 contracts, including 60 indefinite delivery contracts, and reviewed associated solicitations, source selection decision documents, and other contract documents to determine the solicitation approach DOD used.

We verified the obligations and contract award fields in FPDS-NG with contract data to ensure that the contracts within our sample were within scope. Thirty-one contracts from our initial sample of 160 contracts were outside the scope of our review because they were incorrectly coded in key parameters, such as being coded as competitively awarded when they were not or had misreported the amount of obligations made on the contract or task order. We excluded these contracts from our sample and determined that FPDS-NG was sufficiently reliable for the purposes of our review after adjusting for these errors.

Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval (e.g., plus or minus 8 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. Unless otherwise noted, percentage estimates based on our sample have 95 percent confidence intervals that are within plus or minus 8 percentage points of the estimate itself. Confidence intervals for other numeric estimates are reported along with the estimate itself. Table 1 summarizes the estimated percentage of contracts of various source selection approaches reviewed.

Based on our analysis of the remaining 129 sample contracts, we estimate that the total number of best value tradeoff, lowest price technically acceptable, or sealed bid award decisions (in-scope contracts) in the full population of interest was about 293. For contracts that utilized a best value tradeoff process, we categorized them based on the relative importance placed on price. In addition, we determined contract type, the type of procurement (product versus services), and the type of product or service for our sample contracts using FPDS-NG data and verified this information with the contract documents.

To determine why and how DOD used the best value tradeoff process and, in particular, when non-cost factors were considered more important than price, we obtained and reviewed DOD and service level acquisition guidance related to source selection policies and procedures that describe how and when the tradeoff process may be used, including those used for issuing task orders. In addition, for each of the contracts within our sample, we obtained contract documentation including the acquisition plan, solicitation, and source selection decision memorandum and reviewed them in preparation for interviews with DOD officials. In several cases, the solicitation was unclear as to which type of tradeoff process was used. In these cases, we relied on the source selection decision document to categorize the tradeoff process used.

We judgmentally selected buying activities to visit based on factors including the number of contracts awarded on a best value tradeoff basis, contract type, and goods or services procured. Buying activities included at least one command from each military department as well as a defense agency. We reviewed 27 contracts and 23 task orders through our site visits. Specifically, we judgmentally selected 23 of 48 task orders for review by compiling all task orders issued on indefinite delivery/indefinite quantity contracts obligating over $10 million that were administered by the officials at the sites we visited. We chose this dollar threshold to exceed the FAR requirement to provide fair opportunity notices for task orders valued at $5 million or more. Results from these selected contracts or task orders cannot be generalized beyond the specific contract contracts or task orders selected.

During the course of our review, we interviewed officials from the following commands: Department of the Army, U.S. Army Corps of Engineers, New Orleans District Office, Louisiana, and Afghanistan Engineering District, Kabul and Kandahar, Afghanistan; Department of the Army, Armament Research, Development and Engineering Center, Picatinny Arsenal, New Jersey; Department of the Army, Communications–Electronics Command, Fort Monmouth, New Jersey; Department of the Navy, Marine Corps Combat Development Command, Department of the Navy, Naval Air Systems Command, Patuxent River, Maryland and Lakehurst, New Jersey; Department of the Air Force, Air Force Materiel Command, Tinker Air Force Base, Oklahoma; Defense Logistics Agency Energy, Ft. Belvoir, Virginia; and Joint Theater Support Contracting Command, U.S. Central Command, Kabul and Kandahar, Afghanistan, and Baghdad, Iraq.

We interviewed DOD acquisition and contracting officials to identify their rationale for the selected source selection approach (e.g., the thought process behind why a best value approach was chosen over other approaches). For award decisions that used a best value tradeoff process, we discussed why the evaluation factors were chosen and how their relative weights were assigned. We also interviewed officials about the process used and the underlying rationale when issuing selected task orders. We also interviewed officials to determine what the expected outcomes were from using the best value tradeoff process.

We reviewed applicable DOD source selection decision documents and related memoranda to determine how often DOD paid a price differential, the amount of the price differential, and the reasons that were given underlying the decision to pay a higher price. We defined a price differential as a positive difference in price between the offeror who received the award and the offeror next in line for award.

To determine what challenges if any, DOD faces in using the best value tradeoff process, we reviewed DOD guidance and interviewed officials from Defense Procurement and Acquisition Policy, the military departments and defense agencies.

We conducted this performance audit from March 2010 through October 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Timothy DiNapoli, Assistant Director; William Russell, Katheryn Hubbell, Paige Muegenburg, Jodi Munson, Anna Russell, Sylvia Schatz, Roxanna Sun, and Bob Swierczek made key contributions to this report.