Source: http://dau.dodlive.mil/2015/06/25/past-performance-as-a-indicator-of-future-performance-selecting-an-industry-partner-to-maximize-the-probability-of-program-success/
Timestamp: 2018-06-23 20:04:17
Document Index: 473390215

Matched Legal Cases: ['art 15', 'art 36', 'art 15', 'art 8', 'art 36', 'art 15', 'art 15', 'art 42', 'art 15', 'art 13']

Past Performance as a Indicator of Future Performance: Selecting an Industry Partner to Maximize the Probability of Program Success | Defense Acquisition University
By James Bradshaw and Su Chang
Selecting the right contractor is perhaps the most critical factor in achieving future program success. This research identifies past performance strategies to maximize favorable program outcomes.
Selecting the right contractor is perhaps the most critical factor in achieving future program success. This research identifies processes and proposes strategies that could strengthen how the government, specifically the Department of Defense (DoD), selects an industry partner to maximize the probability of program success.
Ideally, the relationship between the government and a contractor should be a genuine business partnership. When awarding a contract for a major weapon system or program, especially one that requires significant development, the DoD must expect the relationship to be a long-term commitment. The contractor and government program management office (PMO) each play numerous critical roles throughout the program planning, executing, and reporting cycles. In a successful program, a contractor organization and the PMO work in harmony to strive toward program success. They understand and leverage each other’s strengths; they communicate frequently; they collectively manage the vast web of program stakeholders; and they collectively establish the reputation and credibility of the program.
The government must therefore strive to enter into business relationships that provide high confidence of success. Selecting an experienced and qualified business partner is a critical step in achieving this objective. While evaluation of the contractor’s proposal is important, this review can provide only a prediction of what may happen and a description of how the contractor plans to perform. By contrast, the contractor’s experience and past performance offers an objective measure of what the firm has actually accomplished in the past.
The paper identifies three key problem areas:
DoD evaluates the contractor’s proposed solution rather than the contractor’s record of actual performance. During the source selection process DoD evaluates a contractor’s proposal, which consists of cost, technical, and past performance volumes. In most cases, however, past performance has little or no impact on the final selection of a contractor. Instead, this aspect is often minimized as a superficial pass/fail test rather than being viewed as an effective tool to predict how well a contractor will perform in the future. Without objective information on past performance and processes to evaluate contractor qualifications, DoD is reluctant to discriminate among proposals based on past performance, especially given the risk of protest.
DoD lacks consistent and thorough processes to evaluate key personnel. The personnel on the contractor program team make vital contributions to program success, and as such, their qualifications should be an important evaluation factor. Contractors generally identify key personnel, and include their resumes as part of the proposal package. However, most source selections limit evaluation of these key personnel to a simple determination of whether the proposed team members meet the minimum experience criteria.
DoD lacks adequate tools to collect, analyze, and report past performance information consistently across contracts. The ability to evaluate past performance effectively depends heavily on the availability of tools and the quality of data content. The government is currently improving and consolidating its repositories of past performance data; however, the DoD lacks capabilities for information collection and has not consistently enforced data entry requirements.
These problem areas represent three different deficiencies in the evaluation of a contractor’s past performance. Evaluation is hampered by the lack of adequate tools to collect, analyze, and report relevant information. The lack of processes to assess the capabilities and experience of key personnel puts at risk the government’s ability to predict future contract performance. In addition, the ability to collect, analyze, and report information on past performance is impaired by the lack of policies and processes to incentivize better reporting. Together, these shortcomings affect the overall contracting process, and contribute to a cycle of inadequacies that has resulted in the numerous contracting challenges we face today. The DoD needs to take a comprehensive, coherent approach to resolve these problems in order to achieve overall past performance improvements.
With the passage of the Federal Acquisition Streamlining Act of 1994, Congress acknowledged that it is appropriate and relevant for the government to consider a contractor’s past performance in evaluating whether that contractor should receive future work (U.S. Department of Defense, 2008).1 In response, Federal departments initiated procedures and systems to record and use information on past contractor performance during source selection. Although the government recognizes that systematically documenting the contractor’s performance becomes a powerful motivator for a contractor to sustain high-quality outcomes, a consistent process for recording performance has been difficult to instantiate and maintain.2
In May 2010, the Office of Federal Procurement Policy (OFPP) directed government agencies to integrate past performance data systems to provide consistency and ensure reliability of data across the Federal government. The Acquisition Committee for eGovernment (ACE) determined that the entire Federal government will use the DoD Contractor Performance Assessment Reporting System (CPARS) as the single system for collecting and transmitting performance evaluations to the Past Performance Information Retrieval System (PPIRS).3 The Navy created CPARS in 1998 to meet information requirements established by the Federal Acquisition Regulation (FAR). Due to inconsistent CPARS data collection and reporting, DoD published a best practices guide titled “A Guide to Collection and Use of Past Performance Information” in May 2003. In June 2007, DoD issued another CPARS Policy Guide that required all new contracts within a certain dollar threshold to register in CPARS within 30 days of contract award.4
On January 21, 2011, the former OFPP Administrator, Daniel Gordon, sent a memorandum to Chief Acquisition Officers and Senior Procurement Executives on enhancing assessments of contractor past performance, which included steps and strategies for improving the collection of information. The memorandum emphasized that while compliance with reporting requirements is important, the quality of the reports submitted is far more crucial. It also acknowledged various challenges that contribute to the low number and quality of these assessments, which include staff shortages and the transition to the new Federal-wide system that integrates the PPIRS, CPARS, and the Federal Awardee Performance and Integrity Information System (FAPPIS).5
Despite efforts by DoD and other government organizations, the policies and mandates lack enforcement mechanisms to incentivize better behavior and improve past performance practices. This paper recommends solutions that will strengthen the data, processes, and tools associated with evaluating past performance and provide the government with a strategy to benefit from the implementation of these resources.
DoD Evaluates the Contractor’s Proposed Solution vs. Evaluating the Contractor’s Record of Actual Performance.
FAR Part 15.3 covers the selection of a contractor in competitive negotiations. According to the FAR, the government is required to evaluate three areas for acquisitions: the quality of the proposed product or service, the price or cost to the government, and past performance (for acquisitions that exceed the simplified acquisition threshold). The government often requests the contractor to submit volumes covering technical, cost, and past performance as part of its proposal. The technical proposal conveys the contractor’s proposed technical solution or response to a requirement; the cost proposal identifies the proposed costs of delivering the proposed technical solution; and the past performance volume shows how the contractor performed on previous similar or related efforts as an early indicator of potential future performance.
Unfortunately, the DoD currently lacks both adequate data on past performance and effective processes to evaluate a company’s qualifications, including key personnel. Thus, evaluation of past performance is not used as effectively as it could be to predict future performance on a contract. These conclusions are borne out by a 2009 study by the U.S. Government Accountability Office. The study examined past performance practices by analyzing 62 procurements in five agencies that perform major acquisitions: the DoD, the Department of Energy, the General Services Administration (GSA), the Department of Homeland Security, and the National Aeronautics and Space Administration. The study found that 82 percent of the past performance evaluations did not contain narratives sufficiently detailed to establish that the resulting ratings were credible or justifiable.6
In many of these acquisitions, the government cited a company’s technical approach as the most important non-cost factor. More than 60 percent of the contracting officers “stated that past performance is rarely or never a deciding factor in selecting a contractor.”7 However, the acquisitions that placed emphasis on past performance noted that this encouraged companies to perform better.8 The report succinctly concluded: “Regardless of the source used, contracting officials agreed that for past performance information to be meaningful in contract award decisions, it must be documented, relevant, and reliable.”9
Without such information, the government evaluation must rely heavily on factors (primarily technical and cost) in the contractor’s proposal that describe a hypothetical situation: how the contractor plans to perform on a contract. In essence, this means that evaluations are based on the quality of a document produced by a professional proposal writing team during the 45– to 90-day solicitation timeframe. These teams specialize in showcasing their company’s capabilities; they are not necessarily experts in devising solutions within the company’s technical capabilities, and that the company can execute with acceptable risk. Proposals are often written with little or no input from the staff members who will execute the day-to-day contract requirements. As a result, proposals submitted for competitive source selections often present “optimistic” solutions that carry significant inherent risk. This risk often materializes during contract execution, which directly contributes to program failures, cost overruns, and schedule delays, which are endemic to DoD acquisitions. The heavy scrutiny of technical and cost proposal evaluations can prove to be a wasted exercise when contractors continually overrun budgets or require major engineering change proposals throughout the life of the acquisition.
Given the shortcomings described above, the evaluation of past performance often becomes a superficial pass/fail test. The majority of best value source selections require the government to evaluate past performance on the basis of a sliding scale that ranges from high-confidence to no-confidence. The DoD uses this scale to assess its level of confidence in a contractor’s ability to perform based on previous related experience. Without adequate information about all offerors, the DoD is reluctant to place strong emphasis on past performance, especially due to the high risk of protest. As a result, the DoD usually assigns all offerors a similar confidence rating in order to “level the playing field,” and uses other evaluation factors to discriminate among proposals. Therefore, not surprisingly, the government often fails to select a business partner with an established history of high performance and relevant experience.
The Office of Federal Procurement Policy (OFPP) conducted a pilot test with 30 contracts that used past performance as a significant evaluation factor. Those contracts identified a 20 percent increase in average customer satisfaction, confirming the assertion that the increased use of past performance data supports positive program results.10 To improve how the DoD leverages past performance in its source selections, the remainder of this section reviews proposed solutions that incorporate well-founded best practices to mitigate the risks associated with inadequate past performance evaluations.
2.1: Proposed Solutions
2.1.1: Alpha Contracting Negotiations with Pre-Qualified Vendor Candidates
Alpha contracting negotiations with pre-qualified vendors leverage the best practices of the Qualifications-Based Selection (QBS) strategy authorized for Architecture and Engineering (A/E) contracts under FAR Part 36. The QBS strategy was originally developed because certain creative professional services cannot be fairly priced before the creative process has taken place. The QBS process improves innovation, quality, and creativity by allowing the government to negotiate the contract requirements and pricing jointly with the contractor against an independent cost estimate. The Brooks Act of 1972 delineates the framework for QBS using the following phased approach:
The government determines the desired qualifications, including the experience and demonstrated competence of interested parties. The information provided by the offerors should emphasize technical ability (cost is not considered as part of the initial evaluation).
The government creates a short list, ranking the bidders by their qualifications.
The government conducts interviews with the firms on the short list and then re-ranks the firms.
Finally, the government negotiates a statement of work (SOW) and fair price with the most qualified firm.
Unfortunately, the FAR only allows use of QBS for contracts involving the construction or building trades, including transportation systems. However, the DoD could leverage some of the benefits offered by the A/E procurement strategy by combining some QBS attributes with other contracting best practices and strategies. For example, DoD can use a prequalification process to narrow the pool of qualified vendors, similar to step 1 in the QBS process. This can be done by using an advisory multi-step process described in FAR Part 15.202, or by establishing a competitive range based entirely on past performance criteria. Narrowing the field of vendor candidates would ensure that the DoD optimizes its time and resources by conducting Request for Proposal (RFP) activities only with the vendors that have the highest likelihood of being selected for contract award. This approach also benefits potential contractors, since they do not have to expend valuable proposal development funds for solicitations they have little chance of winning.
Processes to narrow down the pool of qualified vendors on the basis of past performance and experience can require upfront time and investment from the government. However, these strategies can save significant time during the “back-end” of the proposal evaluation process. Ranking the vendors also helps keep competitive pressures in play during negotiations. The government mitigates the disadvantages associated with a sole-source negotiation environment because it maintains the flexibility to move to the next-ranked vendor at any point in the negotiations.
Additionally, DoD can use a question and answer session as part of an oral presentation strategy, similar to the QBS interview process described in step 3 above, and in Section 3.1.1 of this paper. Open communication with vendors to verify experience and qualifications is an important, but often overlooked activity. Usually the government accepts the qualifications and experience stated in a company’s proposal at face value. However, greater insight regarding the contractor’s related experience and the qualifications of the proposed key personnel would increase the accuracy of the government’s assessment.
Lastly, DoD can apply the same SOW and contract price negotiation process described in Step 4 of the QBS through the use of alpha contracting: an innovative strategy that allows the government to perform many activities jointly with the contractor. Together, the government and contractor develop the SOW and proposal in a streamlined fashion. This process offers a number of advantages and performance improvements, such as enhancing communication, refining and clarifying requirements, and ensuring the technical solution is bounded by the capabilities of the contractor. The alpha contracting process also has the potential to yield significant savings in time and cost. For example, the Army PM-Tank Main Armament Systems M830A1 procurement utilized the alpha contracting process and reduced the lead-time for procurement administration by 55 percent, thus saving $1 million, which was subsequently used to buy additional units.11
It may be possible to combine all of the above techniques and use a GSA Blanket Purchase Agreement (BPA) to model the QBS strategy from Step 1 through 4. Under this arrangement, the DoD could select a set of contractors from a GSA schedule and use a prequalification process to narrow the field of vendors. Next, the DoD would use a Q&A process to interview candidates and rank order the potential vendors. The DoD could then utilize the alpha contracting process to negotiate a BPA with the highest ranked firm. If negotiations with the top-ranked vendor resulted in unfair pricing, or if the DoD and the vendor were unable to agree on a SOW, the DoD would have the right to conduct negotiations with the second-ranked firm. When negotiations are complete, the DoD would issue a BPA using a GSA schedule contract authorized under FAR Part 8.405-3.
This GSA BPA approach has certain restrictions that may limit applicability across a broad range of acquisitions (e.g., GSA BPAs are limited to commercial products and services and do not allow cost-type contracts).12 Furthermore, the QBS approach is currently limited to A/E contracts authorized under FAR Part 36. However, the need for creativity and innovation is not limited to A/E procurements. Agile software development, for example, values flexibility, innovation, and collaboration and can greatly benefit from a QBS-like acquisition approach.
As a result, the DoD should pursue an amendment to FAR Part 15 to allow a QBS-like process that can be leveraged for many different types of acquisitions. The DoD can achieve maximum benefit from this approach if the amendment allowed a prequalification process to narrow the field of potential vendors. An interview or Q&A process can facilitate the vendor ranking process, and alpha contracting would allow the DoD to negotiate a SOW and pricing based on the ranking of firms. All of these individual strategies are permitted under different areas of the FAR; however, a FAR Part 15 amendment would allow the DoD to combine them into a single process to obtain maximum benefit.
2.1.2: Solution Demonstrations as a Source Selection Technique
The limited availability of past performance information makes it difficult to verify the relevance of previous experience. As an alternative strategy, DoD can use demonstrations as part of the source selection process to validate previous experience. A demonstration could include any material representation of past experience (e.g., prototypes, software demonstrations). Using this strategy, the DoD would require vendors to submit a functioning prototype, or a previously developed relevant product, and to demonstrate its capabilities as part of the proposal process.
The DoD could apply this strategy to both hardware and software procurements; for example, screen mock-ups and software demonstrations from previous efforts can be used to evaluate certain software procurements. The demonstration could be submitted prior to the formal proposal process, or could be used to supplement a written technical proposal. The proposal instructions would specify the desired functionality of the demonstrated product, and thoroughly describe how the DoD would evaluate the demonstration.
This source selection strategy will not only help to narrow the pool of qualified vendors, but will also offer a way for vendors to establish recent and relevant experience. The demonstration becomes a tangible piece of evidence that the contractor has verified and proven its experience with the technology and/or solution proposed.
DoD Lacks Consistent and Thorough Processes to Evaluate Key Personnel.
The overall experience and business processes of the contractor are central to evaluation of past performance. However, the qualifications of the personnel proposed for key roles on the contract have even greater importance for the future success of the program. The capabilities and attributes of the key personnel proposed should be directly relevant to the successful completion of the program’s requirements and supported by the strength of the company’s experience.
DoD processes should aspire not only to select the strongest contractor, but also to ensure that the contractor assigns the “A Team” to the program. When companies know their key personnel will be closely evaluated, they have an incentive to offer their best performers, rather than simply personnel who meet minimal qualification requirements. Top performing personnel with a proven track record will also be inclined to ensure the technical solution proposed can be executed with acceptable risk, potentially providing internal checks and balances with the proposal writing team.
Today, contractors generally identify key personnel, and include their resumes as part of the proposal package, but most source selections limit evaluation of these key personnel to a simple check of whether the proposed team members meet experience criteria. The DoD should have consistent and thorough processes for the evaluation of key personnel that go beyond a checklist of experience criteria. This paper recommends that the DoD increase the use of oral presentations and Q&A sessions to verify the qualifications of key personnel.
3.1: Proposed Solution
3.1.1: Evaluate Key Personnel Using Oral Presentations
Throughout the government, oral presentations are often used in conjunction with written proposals to clarify or support all or some aspects of the contractor’s technical and/or management proposal.13 The U.S. Air Force Source Selection Procedures Guide states that “Discussions” may be either oral or written, and specifically requires such discussions for those areas of the proposal, such as past performance, that are significant enough to affect the source selection decision. These presentations can greatly enhance the evaluation process. They give the government an opportunity to evaluate the effectiveness of the contractor’s communication style, clarify the contractor’s proposal, and assess important attributes and group dynamics of the contractor’s key personnel team. In addition, oral presentations remove the gloss created by professional proposal writers and give the government an opportunity to interact directly with the contractor team that will perform the day-to-day tasks required in the contract.
The DoD should take advantage of this opportunity to interact directly with key vendor personnel. To further enhance the benefits of these interactions, the DoD should mandate that proposed key personnel present the oral proposals. This prevents contractors from using their best marketing representatives to make presentations, and gives the government an opportunity to evaluate the overall strength of the contractor team that will perform the actual program tasks.
In addition, the DoD should use the Q&A portion of the oral presentation to verify the qualifications and experience of key personnel. The Air Force recommends maximum use of oral presentations to complement or replace written technical volumes where they best fit the acquisition.14 The DoD can review the resumes of the proposed personnel ahead of time, and prepare targeted questions that will help the government understand the breadth and depth of the team member’s actual experience relative to the program’s requirements. The government should structure questions according to behavioral interviewing techniques: asking questions that require the candidates to describe a past situation in which they exhibited a specific capability, such as problem solving, teamwork, or planning and organizing.
The premise behind behavioral interviewing is that the most accurate predictor of future performance is past performance in similar situations. This strategy is used by more than 70 percent of Fortune 500 companies, which credit it with being 55 percent predictive of future on-the-job-behavior, while traditional interviewing has been determined to be only 10% predictive.15 This method of interviewing provides two types of information that improves insight regarding proposed key personnel: 1) examples of how they actually exhibited the desired behavior versus a hypothetical discussion, and 2) evidence of the success or failure of the solution.
DoD’s evaluation of key personnel should go beyond verifying a checklist of experience criteria. Key considerations must include formal assessments of the contractor’s trustworthiness with respect to foreign influence and control, the rigor of corporate security protocols, and the integrity of the contractor’s global supply chain relationships.
Lastly, the contract should include key personnel clauses that protect the government from “bait and switch” tactics. In some instances, contractors have replaced the team originally proposed with potentially less qualified candidates shortly after contract award. To mitigate this risk, DoD contracts should require that the key personnel originally proposed on the contract remain with the program for a minimum time period (e.g., 12–18 months), unless that individual leaves the company. This becomes especially important if the government uses key personnel as an important evaluation factor in the source selection process.
DoD Lacks the Adequate Tools and Information to Collect, Analyze, and Report Past Performance Information Consistently Across Contracts.
A high-quality data repository greatly strengthens the government’s ability to evaluate past performance as part of the source selection process. Well-documented information, easily accessible in such repositories, enables the government to defend its position during a contract protest. Thus, a data-driven process to collect, analyze, and report past performance information is critical in allowing the government to incorporate past performance in a source selection decision.
As noted in Section 1, two important past performance tools used today are CPARS and PPIRS. CPARS is a suite of web-enabled applications that document contractors’ past performance in accordance with the FAR. It uses electronic workflow to automate contracting officers’ evaluations of contractor performance. It also allows contractors to submit comments electronically in response to a government assessment and either concur or non-concur. The DoD-developed PPIRS is a web-enabled government enterprise application supporting source selections. Since the FAR requires all Federal agencies to post contractor performance evaluations in PPIRS, it now acts as the government-wide warehouse for information on contractor past performance.16 As a warehouse, it allows query-based, read-only retrieval and review of contractor data by qualified government acquisition personnel. Contractors can view their own data by using a central contractor registration to gain access.
Although the FAR requires the government to evaluate past performance, the collection and reporting of past performance data has been inconsistent and often untimely. The lack of consistent, available, and reliable data hinders the DoD’s ability to make past performance an effective part of the proposal evaluation process. The government has recently made improvements to many of the past performance tools, including CPARS and PPIRS, to provide commonality of data entry format and increase the availability of data for retrieval. Despite these improvements, current data repositories still perform their functions inadequately, or are used inaccurately across DoD. The problems with the government’s current data repositories have plagued DoD for years. Poor data quality, incompatible data formats, inconsistent reporting timeliness, lack of data entry, diffused accountability, and subjective evaluations are among the issues associated with the way DoD has collected, analyzed, and reported past performance information.
4.1: Proposed Solutions
4.1.1: Implement and Enforce DoD Policies for Past Performance
The OFPP and the Defense Procurement and Acquisition Policy (DPAP) Directorate in the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) have recently issued guidance and policy that emphasize the importance of information on past performance. Current policies, however, lack appropriate enforcement mechanisms to make this policy effective and actionable. AT&L/DPAP should therefore issue a DoD-wide mandate to populate past performance databases in a timely and consistent manner. Per FAR Part 42.15, the responsibility for reporting past performance resides with the contracting officer, but it is important to make this a shared responsibility between the PMO and the contracting office. The program team (e.g., program manager, contracting officer’s technical representative) should evaluate how well the contractor is performing against the SOW requirements. The contracting office should collect this information from the PMO, and also provide input on the contractor’s performance against the contract requirements (e.g., reporting requirements, terms and conditions). The contracting office should have ultimate responsibility for collecting this information and populating the past performance database in a timely and consistent manner.
DoD should increase enforcement of this policy through a variety of mechanisms. Agencies should be required to report metrics on how well they are complying with this mandate. AT&L/DPAP should collect these metrics on a monthly basis and publish the results on DPAP’s website. The head of each Combatant Command, Service, or Agency (C/S/A) should be held accountable for C/S/A-wide compliance with this policy.
Additionally, DoD should require that past performance reviews become a mandatory part of acquisition program reviews by the Program Executive Office. This will ensure that program offices are held accountable for meeting this policy requirement. As a further incentive, DoD should prohibit programs from exercising option awards, award terms, or award fee payouts until they have populated past performance information into a database.
Lastly, DoD should include an evaluation of compliance with this process as part of each contracting officer’s annual pay and performance review. Metrics that report compliance with this mandate at the individual contracting officer level will further enhance the enforcement of this policy.
4.1.2: Identify Quantifiable Metrics for Agency-Wide Reporting
Past performance evaluations today tend to be based on subjective judgments by the evaluator. This makes it difficult to compare performance across programs and contractors. Additionally, observations show that program managers and contracting officers are often reluctant to report negatively on past performance because this can reflect poorly on their own ability to manage the program or contract. Furthermore, to avoid conflict with the contractor, the government may refrain from documenting performance deficiencies in official databases. As a result, the past performance write-up does not always reflect a contractor’s performance accurately.
The DoD should agree on a set of quantifiable metrics to capture objective data within standardized past performance databases. The use of such quantifiable and simple yes/no metrics can provide a consistent and reliable way to compare contractor performance across contracts. For example, earned-value management data can be used to report cost and schedule performance on complex acquisitions. Award fee determinations can also provide a metric, since the award fee represents a bilateral agreement and contractors often use this metric to measure their own performance. Simple yes/no metrics can be used for non-Earned Value Management (EVM) contracts. Such questions could include: Did the contractor perform within proposed cost? Did the contractor deliver on time? and Did the program encounter a Nunn-McCurdy breach? Each of the metrics and questions can be followed by a data field that allows the government to explain the evaluations provided in greater detail.
4.1.3: Include Validated and Approved Contract Deliverables in Past Performance Databases
The DoD can increase the quantity and quality of the data in past performance databases by incorporating information on validated and approved contract deliverables. For example, summary narratives on award fees provide a wealth of information on how the contractor performed during the award fee period. As noted above, award fees reflect consensus between the government and contractor on the factors that contributed to the final fee. This type of information can be very useful during evaluations of past performance, since it gives the DoD a more complete picture of how well the contractor performed over the life of a contract.
Lastly, past performance databases should include a mechanism for uploading documents directly into CPARS to share with other government counterparts. This would greatly enrich the quality and quantity of past performance data.
The lack of adequate past performance data, tools, and processes hinders the government from effectively evaluating the qualifications of companies and key personnel. The DoD tends to deemphasize past performance evaluations during the competitive source selection process, largely because data that would allow it to discriminate among proposals is unavailable.
Current government tools have proven inadequate to collect, analyze, and report information on past performance. The absence of timely, reliable, documented data impedes the entire cycle of collecting, reporting, and evaluating past performance during source selection (see Figure 1). The inability to use past performance as a predictor of future contract success places the DoD at risk of repeating program mistakes.
Evaluating a company’s performance record allows the government to assess the effectiveness of a contractor’s management processes, tools, and resources. Because effective performance depends on qualified and experienced personnel who know how to replicate success, the ability to evaluate proposed key personnel also becomes an important aspect of estimating a company’s ability to meet the contract requirements. Yet the DoD is not utilizing the necessary processes to fully assess the capabilities of the key personnel proposed on a contract.
The DoD currently lacks a comprehensive approach to tackle the issues identified in this paper. Since the problems are interdependent, they require a holistic approach that addresses the tools, processes, and policies surrounding the past performance cycle pictured above. The processes will not improve without adequate tools to provide the data. They will not be effective without the proper policies in place to enforce them, and they cannot be implemented without the proper processes and tools to support them.
The recommendations summarized below have been brought forward throughout this paper as important steps toward improving the DoD’s probability of achieving successful future program outcomes:
Use alpha contracting negotiations with pre-qualified vendor candidates. Amend FAR Part 15 to allow for a QBS-like approach across a broader range of acquisitions.
Use solution demonstrations as part of source selection to increase accuracy in evaluations of the contractor’s performance record.
Include oral presentations in the evaluation process; improve the consistency and depth of personnel evaluations by using the behavioral Q&A process to verify the relevant qualifications and experience cited in proposed key personnel resumes.
Issue a policy-wide mandate that enforces past performance data entry standards. Use an online tool to track and report compliance to this policy requirement.
Agree on a set of quantifiable metrics to capture objective data within standardized past performance databases.
Increase the quantity and quality of data in past performance databases by incorporating relevant contract performance documentation from validated and approved contract deliverables.
Gordon, D. I. (2011, January 21). Improving contractor past performance assessments: Summary of the Office of Federal Procurement’s Policy’s review, and strategies for improvement. Washington, DC: Office of Federal Procurement Policy, Office of Management and Budget, Executive Office of the President. Retrieved July 20, 2012, from http://www.whitehouse.gov/sites/default/files/omb/procurement/contract_perf/PastPerformanceMemo-21-Jan-2011.pdf
Hansen, K., Quintcareers.com, “Behavioral Job Interviewing Strategies for Job-Seekers”.
Jones, A. (n.d). PROCNET, PM-TMAS’s M830A1 Alpha contracting process. Retrieved July 20, 2012, from http://procnet.pica.army.mil/Reference/articles/Archives/120mm.htm
Office of Federal Procurement Policy, Office of Management and Budget, Executive Office of the President. (2000, May). Best practices for collecting and using current and past performance information. Washington, DC: Office of Management and Budget. Retrieved July 20, 2012, from http://www.whitehouse.gov/omb/best_practice_re_past_perf
Past Performance Information Retrieval System. Retrieved March 7, 2012, from http://www.ppirs.gov
Sade, M. (2009). Seven steps to performance based acquisition, seven steps project. Retrieved July 20, 2012, from https://www.acquisition.gov/comp/seven_steps/index.html
U.S. Air Force. (2000, March). Source Selection Procedures Guide. Washington, DC: Office of the Secretary of the Air Force. Retrieved July 20, 2012, from https://acc.dau.mil/CommunityBrowser.aspx?id=56983
U.S. Department of Defense. (2008, February 29). Contractor past performance information (Inspector General Report No. D-2008-057). Washington, DC: U.S. Department of Defense.
U.S. General Services Administration, U.S. Department of Defense, and National Aeronautics and Space Administration. (2010, January). Federal Acquisition Regulation (FAR). Washington, DC: Acquisition Central. Retrieved July 202, 2012, from https://www.acquisition.gov/far/
U.S. General Services Administration, Strategic IT Analysis Division. (1997, December). Past performance (White Paper). Washington, DC: Office of Management and Budget. Retrieved July 20, 2012, from https://www.acquisition.gov/sevensteps/library/ASIwp-past-perform.pdf
U.S. Government Accountability Office. (2009, April). Better performance information needed to support agency contract award decisions (GAO-09-374). Washington, DC: Government Accountability Office.
1U.S. Department of Defense. (February 29, 2008). Contractor Past Performance Information (Inspector General Report No. D-2008-057). Washington, DC: U.S. Department of Defense, p. i.
2Office of Federal Procurement Policy, Office of Management and Budget, Executive Office of the President. (May 2000). Best Practices for Collecting and Using Current and Past Performance Information. Washington, DC: Office of Management and Budget., p. 1.
3Past Performance Information Retrieval System. Retrieved March 7, 2012, from http://www.ppirs.gov http://www.ppirs.gov/
4Contractor Past Performance Information, p. 5.
5Gordon, D. I. (January 21, 2011). Improving Contractor Past Performance Assessments: Summary of the Office of Federal Procurement’s Policy’s Review, and Strategies for Improvement. Washington, DC: Office of Federal Procurement Policy, Office of Management and Budget, Executive Office of the President.
6U.S. Government Accountability Office. (April 2009). Better Performance Information Needed to Support Agency Contract Award Decisions (GAO-09-374). Washington, DC: Government Accountability Office, pp.8–9.
7Ibid, pp. 8–9.
8Ibid, pp. 2–3.
9Ibid, pp. 8–9.
10U.S. General Services Administration, Strategic IT Analysis Division. (December 1997). Past Performance (White Paper). Washington, DC: Office of Management and Budget., p. 2.
11Jones, A. (n.d.) PROCNET, PM-TMAS’s M830A1 Alpha Contracting Process.
12U.S. General Services Administration, U.S. Department of Defense, and National Aeronautics and Space Administration. (January 2010). Federal Acquisition Regulation (FAR). Washington, DC: Acquisition Central. Blanket Purchase Agreements, Subpart 13.303.
13Sade, M. “Seven Steps to Performance Based Acquisitions,” Seven Steps Project, www.acquisition.gov.
14U.S. Air Force. (March 2000). Source Selection Procedures Guide. Washington, DC: Office of the Secretary of the Air Force.
15Hansen, K. Quintcareers.com, “Behavioral Interviewing Strategies for Job Seekers.”
16Information found at http://www.ppirs.gov However, it should be noted that while all federal acquisitions are required to use the PPIRS, not all government organizations have complied.
James Bradshaw has more than three decades of information technology acquisition experience in the areas of proposal development, project management and purchasing management. He has implemented improved source selection and project delivery processes supporting government and commercial programs. Mr. Bradshaw is currently a principal economic business analyst and group leader with the MITRE Corporation, and holds professional certifications in project management (PMP), and purchasing management (C.P.M.)
Su Chang has 10 years of experience and has worked for both DoD and Civilian Government agencies. She is a principal economic business analyst and group leader with the MITRE Corporation where she specializes in defense contracting and acquisition reform initiatives. Ms. Chang has been certified as DAWIA Level III in Contracting, and is a former 1102 civilian contract negotiator.
This entry was posted in Limits of Competition and tagged James Bradshaw, Limits of Competition, Performance, Su Chang on June 25, 2015 by abrownie.
← Continuous Competition as an Approach to Maximum Performance Limits of Competition for Depot Maintenance Contracting →
DoDLive > Defense Acquisition University > Limits of Competition > Past Performance as a Indicator of Future Performance: Selecting an Industry Partner to Maximize the Probability of Program Success