# Presentation: 110453

## Source Selection for the Source Evaluation Board

**U.S. Department of Energy**
**Office of Procurement and Assistance Management**
**September 29, 2005 **

## Preface

- 	**The information contained in these briefing materials is intended to provide a general orientation to Source Evaluation Boards (SEB) for the conduct of major acquisitions conducted pursuant to the Federal Acquisition Regulations (FAR) Part 15. This information consists of (1) certain regulatory information from the FAR and the Department of Energy Acquisition Regulation (DEAR), (2) other prescribed requirements, and (3) information derived from common practices and lessons learned from other acquisitions. This orientation material is not a substitute for the SEB’s reading and understanding of the applicable regulations and appropriately applying the regulations and good business judgment to the particular circumstances of the acquisition.   In addition these briefing materials are not intended to cover every aspect of the acquisition process.  While the focus of these materials is toward major acquisitions, its principles may be applied, as appropriate, for other competitive acquisitions as well.**
- 	**This information is internal DOE briefing material.  The information contained in these materials does not form and should not be construed to convey any rights to third parties.**

## Table of Contents

**What is Source Selection? – Page 5**
**Typical Flow of Source Selection – Page 6**
**Source Evaluation Board -  Page 7**
**Procurement Integrity – Page 11**
**Schedule for the Acquisition – Page 14**
**RFP Development – Page 15**
**Draft RFPs – Page 17**
**Statement of Work – Page 18**
**Evaluation Factors for Selection – Page 20**
**Experience Evaluation Criterion – Page 24**
**Past Performance Evaluation Criterion – Page 26**
**Key Personnel Evaluation Criterion – Page 31**
**Evaluation of Risk – Page 32**
**RFP Instructions – Page 33**

## Table of Contents

**Cost Proposal Instructions – Page 35**
**Oral Presentations – Page 37**
**Cost Realism & Probable Cost Determination – Page 41**
**Award Without Discussions – Page 45**
**Competitive Range – Page 48**
**Discussion/Negotiations – Page 49**
**Proposal Revisions/Final Proposal Revisions – Page 53**
**SEB Report – Page 55**
**SEB Briefing to SSO – Page 58**
**Debriefing of Offerors – Page 59**
**Protest Issues – Page 63**
**Rating Plans – Page 64**
**SEB’s Preparation Prior to Receipt of Proposals – Page 68**
**HQ Business Clearance – Page 69**

## What is Source Selection?

**“****Source selection” is the **_**selection process**_** used in competitive, negotiated contracting to select the proposal that offers the best value to the Government**
**The source selection process is governed by statute and regulations**
**“****Best value” means the expected outcome of an acquisition that, in the Government’s estimation, provides the greatest overall benefit to the Government in response to the Government’s requirement**
**Key Tenets of Best Value Source Selection**
- Full and open competition
- Integrity of the participants and the process
- Effective communication with offerors
- Clarity of solicitation
- Fairness to offerors
- Checks and balances in the process
**The Government official responsible for selecting the winning offeror in a negotiated procurement is the Source Selection Official (SSO)**
**The Source Evaluation Board (SEB) and the Contracting Officer are charged with the responsibility of conducting the acquisition in a manner that allows the SSO to make the best value decision **
- The SEB works for the SSO

## Source Evaluation Board 

**The SSO appoints the SEB Chairman and SEB voting members **
**The purpose of the SEB is to solicit and evaluate proposals and report its evaluation results to the SSO**
- Evaluation must be consistent with the RFP and the Rating Plan
**Composition of the SEB **[FAR 15.303(b)(1)]
- Membership should be tailored for the particular acquisition to assure a comprehensive evaluation of proposals
- Includes a balanced team of experts in various fields based on what is being acquired
- The voting and non-voting members include, as appropriate, personnel in the areas of contracting, legal, technical, finance, human resources, etc.
- Members, as a collective group, should have sufficient experience in RFP development and proposal evaluation
- Responsibilities of individual team members are defined based on their knowledge and  specialty/expertise
- Specialty evaluation teams may be needed to support the SEB, e.g., technical, cost, business, safety, etc.

## Source Evaluation Board

**SSO**

***SEB **
**Chairman**
***SEB **
**Members**

**Cost **
**Team**

**Specialty **
**Advisors**

**Business Mgmt.**
**Team**

**Technical**
**Team**

**Counsel**

***Contracting**
**Officer**

**Ex-Officio**
**Advisors**

**Contract**
**Specialist**

- *The Contracting Officer may also serve as the SEB Chairman or an SEB member.

## Source Evaluation Board

**Contractor personnel may be non-voting members or advisors if special expertise is required**
- Certain DOE internal approvals and nondisclosure procedures are required [DEAR 915.207-70(f)(2) and (3); FAR 37.203(d) and 204]
- Written determination that available Federal employees do not have the requisite expertise
- Approved by the SSO
- Concurrence by the HCA
- RFP must advise offerors that non-Federal personnel may be used in the evaluation [DEAR 915.207-70(f)(4)]

## Source Evaluation Board

**Training**
- SEB, as a group, should consider attending a training course on source selection
- Provides common guidance to entire SEB during solicitation and evaluation process
- Provides a common basis for establishing a cohesive team
- Bringing course to site location is cost effective for the entire SEB and advisors
- Course vendors will tailor course to the SEB’s needs
**Co-location of SEB members**
- Substantial benefit to enhance effectiveness among team
- Separation from normal work site insulates SEB from other work
- Enhances focus on the acquisition
- Facilitates interactions between members to achieve consensus

## Procurement Integrity Act(FAR 3.104; DOE AG, Ch. 3) 

**The purpose of the Procurement Integrity Act is to preserve the integrity of the Federal procurement process and assure fair treatment of bidders, offerors and contractors.**
**The Act covers -**
- Current Federal Employees
- Certain Former Employees
- Bidders and Offerors
- Other personnel involved in agency procurements and contracts
**Disclosure of proposal information is prohibited including:**
- Any contents of proposals
- Number or names of offerors
- Names of offerors have been publicly disclosed by DOE when permission was obtained from all of the offerors

## Procurement Integrity Act (FAR 3.104; DOE AG, Ch. 3) 

**Disclosure of source selection information is prohibited**
- Information prepared by the Government for the purpose of evaluating a proposal
- Source selection information includes:
- Source selection or evaluation plans
- Reports of evaluation panels, boards or advisors
- Competitive range report or SEB report
- Draft RFPs are not to be disclosed unless released for public comment
**Discussing prospective employment with a prospective offeror is prohibited**
- Prohibition is on both the Federal employee and the offeror
- If an offeror makes contact for this purpose, report it
**Certain Federal employees involved in the evaluation of proposals may not accept employment from the selected contractor for a period of 1 year after the procurement is over **
- Applies to Contracting Officer, SSO, members of the SEB and certain others involved in the evaluation of proposals

## Procurement Integrity Act (FAR 3.104; DOE AG, Ch. 3) 

**Consider your normal job responsibilities and how they may affect your SEB responsibilities, e.g., do you interact with an existing contractor that may be a prospective offeror on your procurement**
**You may continue to perform your current job, but be careful in your interactions to assure they are not in any way related to or affect your SEB responsibilities and objectivity**
**Avoid social interactions with offerors or potential offerors**
**Disclose anything that may be questionable to the Contracting Officer**
- Don’t dismiss a connection as de minimis
- Disclosure when the event occurs documents the matter in case a question arises later
- Disclose any social or private financial relationship with anyone who works for an offeror or potential offeror
- Disclose any family member’s (spouse/children) association with an offeror during the procurement process that may give rise to an actual or appearance of a conflict of interest
- Consider your stock and investment interests
**A specific briefing on the Procurement Integrity Act by counsel is recommended**

## Schedule for the Acquisition

**Make it realistic**
**If a schedule is published, either internal to DOE or publicly, it will become the expectation**
**Allow sufficient time for HQ business clearance review process, and appropriate and necessary senior management involvement**
- RFP
- SEB Reports
- Source selection documents
- HQ briefings
**Allow sufficient time to hold discussions and to resolve protests**
- Even if award without discussions is planned, discussions may become necessary depending on the proposals
- While the schedule may not always be able to build in protest time, assure there is a contingency plan for continuing work, as necessary

## RFP Development

**Order of development**
- 1st - Statement of Work (SOW) (Section C)
- Defines requirement
- Can’t define what to evaluate (Section M) until the requirement is defined
- 2nd - Evaluation Criteria (Section M)
- Defines what portion of the SOW or capability is to be evaluated
- Can’t write RFP instructions (Section L) until you define what is to be evaluated
- 3rd - RFP Instructions (Section L)
- Defines instructions for proposal submission as they relate to the defined SOW (Section C) and evaluation criteria (Section M)
-  4th - Other RFP/Contract requirements
**While certain portions of the RFP can be developed concurrently, this order of development needs to be the guiding principle**

## RFP Development

**Tailor the evaluation criteria and instructions to the specific acquisition**
- Don’t copy evaluation criteria/instructions from another RFP without assuring the criteria/instructions fit the SOW and are what the SSO/SEB desires to consider in its evaluation
**The SEB needs to think through what should be evaluated and how it will be evaluated**
**It is critical that the SOW (Section C), the evaluation criteria (Section M), and the instructions (Section L) be consistent**
- If different people write each part, requires close coordination and review
- Inconsistencies can result in changes in the RFP, delays in the acquisition, and less advantageous offers
**Each SEB member needs to be involved in and understand the what, why, and how of the evaluation criteria, the SOW, and the proposal instructions**
**Ensure the individual or team who will be involved in the cost evaluation is involved in preparing the cost instructions**

## Draft RFPs(FAR 15.201; 5.101(b); DOE AG  Ch. 15.1, Part I)

**Publishing a draft RFP for public comment is a technique for obtaining public input prior to issuing the final RFP**
- Not a required step in the process
**Gets early involvement of interested parties**
- Helps stimulate competition interest
- Allows teaming arrangements to be formed
- Gets feedback on RFP from prospective offerors
**Beneficial when award without discussions is contemplated**
**New or complex SOW requirements**
**Unique contract requirements **
- Cost sharing
- Technology issues
**Does not require full RFP; include key portions, e.g., SOW, evaluation criteria, unique provisions**

## Statement of WorkSection C 

**The Statement of work (SOW) falls into one of two categories – **
- (1) Functional/performance requirements (Performance-based)
- (2) Design requirements
**Functional/performance SOW describes the function or performance desired (the what) and allows the offerors to propose the process (the how) to accomplish the objectives**
- Functional/performance requirements should be used to the maximum extent possible
- This method should produce better technical solutions at better prices as a result of offeror innovation
- Contractor gets paid based upon the fulfillment of predetermined contractual results, i.e., payment based on degree of performance
- Performance-based SOW includes:
- Objectives, performance standards, and measures
- Positive and/or negative incentives
- Quality assurance/surveillance plan
- References: FAR 37.6, Acquisition Guide, Chapter 37, and Performance-Based Service Contracting Toolkit on Professionals Web Page

## Statement of Work Section C

**Design requirements describe both the objective (the what) and the process (the how) to be followed to accomplish the objective**
- May result in less than the optimal technical solution
**When defined objectives are not known, such as environmental remediation actions in which the final environmental impact statement (EIS) and record of decision (ROD) has not been completed, the RFP should define the end point criteria or end state upon which the proposal and contract is to be based**
**Don’t include duplicate contract terms in the SOW that are in other sections of the contract or include RFP-like provisions from section L or M in the SOW**
**The content and nature of the SOW will drive other important aspects of the RFP and contract**
- Contract type: Cost-plus-award-fee, cost-plus-incentive-fee, or fixed price?
- Evaluation criteria: What are the most important aspects of the SOW to evaluate?
- Performance requirements: Are performance requirements adequately defined to allow effective contract administration?

## Evaluation Factors For Selection Section M (FAR 15.304)

**Represent key areas of importance to be considered in selection decision**
- Not necessary to evaluate all areas of the statement of work
- Limit the evaluation criteria to those critical few
- Too many evaluation criteria add unnecessarily to the burden, complexity, and cost to both the offerors and the government
**Identify meaningful discriminators among proposals**
- Define what will distinguish offerors
- Reveal measurable differences or risk levels among proposals

## Evaluation Factors For Selection Section M(FAR 15.304)

**Required areas of evaluation **[FAR 15.304(c)]
- Price or cost
- Quality of product or service such as –
- Compliance with solicitation requirements
- Technical excellence
- Management capability
- Personnel qualifications
- Experience (what has been performed)
- Past performance (what has been the quality of the performance)
- Extent of small business participation in acquisitions, not set-aside for small business

## Evaluation Factors For SelectionSection M(FAR 15.304)

**RFP must specify the evaluation factors and any significant sub-factors and their relative importance**
**RFP must specify the relative importance of the individual **_**non-cost/price**_** evaluation factors to one another, e.g.,**
- Descending order of importance
- Significantly more important
- Twice as important
- Actual numerical weighting may be specified in the RFP.  Typical numerical weighting ranges -
- Technical 30-45%
- Business Management 15-20%
- Key Personnel 15-30%
- Project Management 15-25%
- ES&H 10-15%
- Experience 5-15%
- Past Performance 5-10%

## Evaluation Factors For Selection Section M(FAR 15.304)

**RFP must specify the relative importance of all (as a whole) of the **_**non-cost/price**_** evaluation factors to the **_**cost/price**__ _**factors**
- 	[FAR 15.304(e)]
- Significantly more important than cost or price
- Approximately equal to cost or price
- Significantly less important than cost or price
**Offerors need to understand how their proposals will be evaluated so they can better prepare their proposals**

## Experience Evaluation CriterionSection M

**Request experience information that is similar in type, scope, and complexity to that of the RFP SOW**
**Assure no overlap between experience and past performance**
- Experience is _what _offeror has performed and past performance is _how well_ they have performed
**Specify time period of experience, e.g., last 5 years, and whether more recent experience will be given more consideration in the evaluation**
- Consider using the same time period for both experience and past performance

## Experience Evaluation CriterionSection M

**Specify whether, in addition to the offeror, the experience of predecessor companies, proposed subcontractors, parent companies, LLC members, joint venture members, etc. will be evaluated**
**Evaluation of experience for individual entities needs to be considered in the context of what each entity is proposing to perform under the SOW**
**Provide specific instructions in section L as to which entities are to provide experience information**

## Past Performance Evaluation Criterion Section M(FAR 15.305(a)(2); DOE AG Ch.15.1, Part V)

**Currency and relevancy of the information**
- Past performance information is usually requested for a specified period of time, e.g., last three years.
- Request past performance information that is similar in type, scope, and complexity to that of the RFP SOW
**Source of the information**
- Advise offerors in the RFP that DOE may contact sources other than those references identified by the offeror for past performance information
- Ensure that an offerors past performance on work at DOE facilities is considered
- GAO holds agencies to a higher standard for considering the performance of its own contractors, because that information is so close at hand

## Past Performance Evaluation Criterion Section M(FAR 15.305(a)(2); DOE AG Ch. 15.1, Part V)

**Context of the past performance information**
- How difficult or risky was the work to perform
- Similarity and recentness of the work
**General trends in performance**
- Ensure there is adequate information upon which to base a trend in performance
- Limited data point(s) is not necessarily an indicator of overall performance, i.e., one client reference is not enough, request at least 2-3 references for each proposed entity
**Consider offerors’ compliance with small business and small disadvantaged business subcontracting  plan goals, if applicable**

## Past Performance Evaluation Criterion Section M(FAR 15.305(a)(2); DOE AG Ch. 15.1, Part V)

**Specify whether, in addition to the offeror, the past performance of predecessor companies, proposed subcontractors, parent companies, LLC members, joint venture members, etc. will be evaluated**
- Provide specific instructions in section L as to which entities are to provide past performance information
**Consider requesting past performance for those activities which are listed under experience**
- If a different set of experience is given than under past performance, the SEB may have to obtain more information under past performance related to the experience provided, i.e., need an apples to apples comparison of experience to performance

## Past Performance Evaluation Criterion Section M(FAR 15.305(a)(2); DOE AG Ch. 15.1, Part V)

**Evaluation of past performance for individual entities needs to considered in the context of what each entity is proposed to perform under the SOW**
**If no relevant performance record**
- Offeror must be evaluated neither favorably or unfavorably as stated in RFP
- Examples for evaluation include:
- Pre-determined score, e.g., midpoint of scoring range
- Pre-determined adjectival rating as a “neutral”

## Past Performance Evaluation Criterion Section M(FAR 15.305(a)(2); DOE AG Ch. 15.1, Part V)

**Methods of collecting data**
- Standard questionnaire in the RFP provides consistent information
- Tailor the questionnaire to the RFP; don’t use generic questions
- Require offerors to send out questionnaire to references who submit directly to DOE; require offerors to follow-up with references to assure questionnaire submitted
- If references do not respond to questionnaire, DOE must use reasonable effort to attempt to acquire information from the references identified or from other sources
**Receipt of adverse past performance information **[FAR 15.306(b)(1)(i)]
- If adverse past performance information is the determining factor preventing an offeror from being placed in the competitive range, the offeror must be given an opportunity to address adverse past performance information for which it has not had a prior opportunity to respond
- If award to be made without discussions, clarifications may be held with an offeror having adverse past performance [FAR 15.306(a)(2)]
- If discussions are held, adverse past performance information would be an evaluated weakness which must be disclosed to the offeror during discussions
- Names of individuals providing reference information cannot be disclosed to the offeror [FAR 15.306(e)(4)]

## Key Personnel Evaluation CriterionSection M

**Normal evaluation is on experience, education, leadership, demonstrated performance (reference checks), and in some cases letter of commitment and oral presentations**
- A standard resume form should be specified in the RFP
- A standard reference form should be included in the RFP that the offeror provides to key personnel references for submission directly to DOE (similar approach to past performance)
- If reference checks are not returned to DOE, SEB must use a reasonable amount of effort to obtain reference checks
- Don’t rely solely on the reference form; talking directly with a reference can provide valuable insight
- If key personnel involve incumbent contractor employees, don’t rely solely on SEB members’ knowledge.  Obtain “3rd party” reference input
**Evaluation of key personnel is often evaluated via a documented “interview” or a “test problem”**
- Oral interview with the SEB with predetermined questions
- Test problem with a predetermined problem scenario; key personnel team is given limited amount of time during the oral presentation to review the problem and present a group solution
- Orals must be documented, e.g., video, audio, written record (transcription or notes), briefing slides** **[FAR 15.102(e)]

## Evaluation of RiskSection M

**Evaluation of proposals must identify and document risk **[FAR 15.305(a)]
**Risk may be assessed in different manners**
- Inherently through the relative strengths and weaknesses of each evaluation criterion
- Through a separate risk evaluation criterion
**Offerors may also be evaluated on their ability to identify risks of the work to be performed and proposed means they plan to use to mitigate those risks**

## RFP Instructions Section L  

**Instructions on the preparation and submission of proposals must:**
- Be clearly and precisely stated
- Be keyed to the evaluation factors and subfactors
- Describe the type, scope, content, and format of the information to be submitted
- Describe the order in which proposal responses and materials are to appear
- Be limited to the information needed for the evaluation – don’t ask for information the SEB doesn’t intend to evaluate
**Think through what the SEB expects to see in the proposals based on the evaluation criteria and then write the instructions**
- Don’t just restate the evaluation criteria and ask for the offeror’s approach to performing that particular aspect of the SOW
- The instructions need to be meaningful and helpful to the offeror

## RFP Instructions Section L

**There needs to be a balance between defining information to include in the proposal and allowing the offeror the discretion, i.e., allowing the SEB to see what the offeror knows  **
**  **
**Be careful not to put evaluation criterion (Section M) in the instructions (Section L)**
**Be careful not to put proposal instructions in the contract provisions, e.g., SOW, Section H **

## Cost Proposal Instructions

**Cost proposal instructions need to be comprehensive, distinctly clear, and informative**
- The objective is to obtain sufficient information in the cost proposal to adequately understand the amount and basis for the estimated cost in comparison to the technical proposal
- This is especially important if award without discussions is anticipated
**Specify level of detail required in proposal**
- Require costs to be proposed by cost element – labor, fringe benefits, materials, subcontracts, other direct costs, overhead, and G&A
- Requires costs to be proposed by year
- Labor hours and rates should be proposed by labor category
- Require assumptions and the basis of estimate to be provided
- Prescribe a format or structure
- Request information that supports the indirect rates proposed
- This is particularly important for newly formed offerors, e.g. LLC, when historical information is not readily available from a cognizant audit entity, e.g., DCAA
- Prescribe a WBS to be followed
- WBS should be consistent with the ICE to allow comparative analysis
- Define progressive roll-ups of cost’ let the offerors do the math

## Cost Proposal Instructions

**Emphasize the necessity for consistency between the cost proposal and the technical proposal **
**When transition costs are included in the cost proposal, require any costs for incumbent personnel to be included**
**If the RFP provisions require the offeror to meet certain wage and benefit requirements, request information that demonstrates these requirements are being met**
**For specific and unusual costs that may be applicable to a specific location/site, provide guidance or at least draw attention to such costs, e.g., applicability or non-applicability of certain taxes on some DOE sites**

## Oral Presentations (FAR 15.102)

**Oral presentations **
- May be used as a substitute for written information
- May be used to augment written information
- Can be effective in streamlining the source selection process
**Oral presentations may encompass**
- Entire proposal
- Except the offer (Vol. I), representations and certifications, and certain other information must be submitted in writing
- Portion of proposal, e.g.,
- Business management approach
- Test problem
- A hypothetical problem given to Key Personnel at an oral presentation to assess understanding of issues and ability to work together
- Key Personnel Interviews

## Oral Presentations (FAR 15.102)

**Oral presentations **
- _Are_ the delivery of the proposal or a portion thereof
- _Are not_ “oral discussions” which are negotiations leading to the opportunity for the offeror to revise its proposal in accordance with FAR 15.306(d)
- If award is to be made without discussions, care must be taken in the oral presentations so as not to unintentionally engage in discussions and allow the offeror to revise its proposal
- If the Government inadvertently enters into discussions they must be conducted in accordance with FAR 15.306, Exchanges with offerors after receipt of proposals and FAR15.307, Proposal revisions
**RFP provides specific instructions on method, timing, and content of oral presentation**
- Slides to present at the oral presentation, if applicable, are provided to the SEB at a common time, usually when proposals are due
- Strict time limitations for the oral presentation must be specified in the RFP
- Offerors may want to see the location (room) of the oral presentations
- RFP must specify what the Government will/will not provide, e.g., overhead projectors, marker boards, tables/chairs, sound equipment, etc.
- Questions may be asked of the offeror to facilitate the SEB’s understanding of the oral presentation
- Offeror is not allowed to modify its proposal via the oral presentation
- Offerors will spend considerable time in rehearsing for oral presentations depending on the extent of the content; both advance prepared presentations and spontaneous situations like interviews or test problems

## Oral Presentations (FAR 15.102)

**All SEB members must be present at all oral presentations**
**SSO may choose to attend oral presentations**
- SSO and SEB need to have well defined roles during the oral presentation
- SEB should conduct the oral presentation; SSO is present to listen
- SSO must attend “all or none” of the offerors’ presentations
**Oral presentations are well suited for evaluation of **
- M&O contracts
- Support services
- Business management/project management
- Key personnel capabilities through “interview” or “test problem”

## Oral Presentations (FAR 15.102)

**Oral presentations are not well suited for **
**evaluation of**
- Experience/past performance
- Technically complex approaches
- Design/construction
- Production
- Large number of proposals, depending on the extent of the oral presentation material
**Helpful guides**
- DOE Acquisition Guide, Chapter 15.2, Part VI
- “Guidelines for the Use of Oral Presentations, DOE Professionals Web Page”

## Cost Realism & Probable Cost (FAR 15.404-1; DOE AG Ch. 15.1, Part VIII)

**Cost realism analysis is the process of independently reviewing and evaluating specific elements of each offeror’s proposed cost estimate to determine whether the estimated proposed cost elements**** **
- Are realistic for the work to be performed
- Reflect a clear understanding of the work requirements
- Are consistent with the unique methods of performance and materials described in the offeror’s technical approach
**Cost realism analysis is performed on cost-reimbursement contracts to determine the probable cost of performance for each offeror**
- Probable cost should reflect the Government’s best estimate of the cost of any contract that is most likely to result from the offeror’s proposal

## Cost Realism & Probable Cost (FAR 15.404-1; DOE AG Ch. 15.1, Part VIII)

**Probable cost is determined by adjusting each offeror’s proposed cost and fee, when appropriate, to reflect any additions or reductions in cost elements to realistic levels based on the results of the cost realism analysis**
- Probable cost may differ (higher or lower) from the proposed cost
- Significant differences between proposed cost and probable costs may signal increased performance risks
**The probable cost, including fee, is used for purposes of evaluation to determine the best value to the Government**

## Cost Realism & Probable Cost (FAR 15.404-1; DOE AG Ch. 15.1, Part VIII)

**Cost realism analysis and probable cost determination must be supported by **
- Technical analysis
- Requires individuals with specialized knowledge and experience to evaluate feasibility of technical approaches, processes, facilities, labor, materials, etc.
- Field Pricing  (FAR 15.404-2)
- Requires individuals with specialized knowledge and experience to evaluate labor rates, indirect rates, escalation, etc.
- May require input or audit from the cognizant audit agency, e.g., Defense Contract Audit Agency (DCAA)
- Independent Government Cost Estimate (IGCE)
- IGCE is developed prior to receipt of proposals
- IGCE should be developed in a manner that will facilitate its use in the evaluation of proposals, e.g. same WBS as required by RFP
- After receipt of proposals, the IGCE may have to be adjusted to reflect technical approaches not contemplated in the original IGCE

## Cost Realism & Probable Cost (FAR 15.407-2; DOE AG Ch. 15.1, Part V)

**Cost realism analysis documentation**
- Sufficient documentation should be developed to support the conclusions reached for the probable cost
- Analysis and documentation must support not only those costs that are not reasonable (resulting in an adjustment to the proposed cost) but the basis for the reasonableness of costs not questioned
- Cost analysis must be definitive as to reasonableness or unreasonableness
- If the SEB doesn’t agree with the recommendations of the technical and cost analysis inputs, documentation should reflect the SEB’s ultimate judgment on these issues
- Don’t leave conflicting analyses without reconciling
- This issue is subject to protest

## Award Without Discussions (FAR 15.306(a); DOE AG, Ch. 15.2, Part XI)

**Awards may be made without determining a competitive range and conducting discussions with offerors in the competitive range**
- RFP must state Government’s intent to award without discussions
- Proposals are complete and have no deficiencies that require discussions
- “Deficiencies” are defined in FAR 15.001 as “a material failure of a proposal to meet a Government requirement or a combination of significant weaknesses in a proposal that increases the risk of unsuccessful contract performance to an unacceptable level.”
**Acquisition time can be shortened from solicitation to award if discussions are not needed**
**Reduces cost to the offeror, i.e., time is money**

## Award Without Discussions (FAR 15.306(a); DOE AG, Ch. 15.2, Part XI)

**Certain complex actions may not be conducive to award without discussions**
** **
**Consider on a case-by-case basis**
**Actions not always appropriate for award without discussions**
- Large CPIF contracts with complex cost proposals
- Complex statements of work where specific technical approaches are to be evaluated
- Features of the contract that may require an advance understanding or agreement with the contractor
- Advance understanding on cost
- Incorporation into the contract of technical or other aspects from the proposal
- Negotiation/incorporation of unique contract provisions, e.g., intellectual property

## Award Without Discussions (FAR 15.306(a); DOE AG, Ch. 15.2, Part XI)

**Certain circumstances encountered during proposal evaluation may prevent award without discussions, even to the highest rated technical proposal**
- Exception to the terms of the contract
- Insufficient information in the cost proposal to make a probable cost adjustment
- Performance guarantee improperly executed
- Fee proposed exceeds the limit in the RFP
**Discussions may be warranted**
- To gain a better understanding of the proposals
- To obtain a better deal for the Government
**Clarifications and award without discussions **
- Offerors may be given the opportunity to clarify certain aspects of proposals, if award is to be made without discussions
- Only examples in FAR are: in reference to past performance information and to resolve minor or clerical error
- Contracting Officer will determine whether an offeror will be given the opportunity to clarify its proposal

## Competitive Range[FAR 15.306(c)]

**If discussions are to be conducted, a competitive range of offerors must first be established**
- Determined by the Contracting Officer
- Concurrence by the SSO
- Competitive range report prepared; content similar to SEB report
**The competitive range consists of the “most highly rated proposals”**
- Consider non-cost _and_ cost factors
- May be reduced to a number at which an efficient competition can be conducted
- Must have a reasonable chance of being selected for award
- Consider the extent to which weaknesses can be corrected
- Where there is doubt as to whether or not an offeror should be included, include that offeror

## Discussions/Negotiations[FAR 15.306(d),(e)]

**Discussions/negotiations are exchanges (written or oral) with all offerors in the competitive range that are undertaken with the intent of –**
- Allowing offerors to revise proposals
- Enhance the Government’s ability to obtain the best value from each offeror
**Terms “discussions” and “negotiations” are synonymous in this context**

## Discussions/Negotiations[FAR 15.306(d),(e)]

**Negotiations may include “bargaining,” which includes**
- Persuasion, alternation of assumptions and positions, and give-and-take
- Negotiations may apply to price, schedule, technical requirements, type of contract, and other terms of the proposed contract
**Negotiations **_**must**_** disclose to the offeror deficiencies, significant weaknesses, and adverse past performance information to which the offeror has not yet had an opportunity to respond.  **
- The Contracting Officer is encouraged to discuss other aspects of the proposal that could be altered or explained to enhance the offeror’s potential for award
- The Contracting Officer is not required to discuss every area where the  proposal could be improved
- While FAR 15.306(d) only requires “significant weaknesses” to be disclosed, “weaknesses” if so identified by the SEB should also be disclosed

## Discussions/Negotiations[FAR 15.306(d),(e)]

**Negotiations may not**
- Favor one offeror over another
- Reveal an offeror’s technical solution/approach to another offeror
- Reveal an offeror’s price to another offeror without that offeror’s permission
- Reveal the names of individuals providing reference information about an offeror’s past performance
- Reveal source selection information, e.g., rating plan
**Negotiations must be “meaningful” and tailored to each offeror’s proposal**
- Aspects of the proposal that are not understood must be addressed during negotiations in order for the SEB to perform a comprehensive evaluation
- Seek to get the offeror to revise its proposal to increase its value to the Government
- Provide the SSO the ability to select from the best of the best

## Discussions/Negotiations[FAR 15.306(d),(e)]

**The weaknesses and deficiencies from the competitive range report should be provided to offerors prior to discussions**
- Often this is done verbatim from the competitive range report
**Request additional information on the cost proposal where offeror has not adequately supported its cost proposal and there is not a clear basis to determine a probable cost**
**Offerors may be told that its price is too high or too low and reveal the results of the analysis supporting that conclusion**
**The Government may disclose to all offerors the cost or price that the Government’s price analysis, market research, or other reviews have identified as reasonable**
**Identify other matters for discussion that may not be weaknesses or deficiencies but will improve the Government’s understanding of the proposal**

## Proposal Revisions Final Proposal Revisions (FAR 15.307)

**Purpose: Clarify or document understandings reached during discussions**
**Proposal revisions may be obtained at any time during discussions**
**Don’t request final revised proposals until the SEB has sufficient information and understanding to complete its evaluation**
- Ask for revised proposals when appropriate
- Delay requesting final revised proposals until all appropriate information has been discussed
**Final proposal revisions required at the conclusion of discussions from **_**all **_**offerors still in the competitive range; common cut-off date for receipt**

## Proposal Revisions Final Proposal Revisions (FAR 15.307)

**For final proposal revisions, advise offerors the Government intends to make award without obtaining further revisions**
- If negotiations are still needed after receiving final proposal revisions, offerors may be given the opportunity to submit a second, final proposal revision
**Clearly instruct offerors as to how proposal revisions/final proposal revisions are to be submitted**
- Summary document or revised/changed pages
- Specify whether any page limitations apply
- Clarify whether any aspects of the proposal are to be incorporated into the contract and be specific
- Require a new, signed SF 33, Solicitation, Offer, and Award

## SEB Report(FAR 15.305(a); 15.308; DOE AG Ch. 15.1, Part XII)

**The relative strengths, deficiencies, significant weaknesses, cost/fee evaluation, and risks must be documented in the SEB report**
- Acquisition Guide contains an outline of topics to be included in the SEB report
**Report must reflect the deliberations of the SEB and be consistent with the RFP and the rating plan**
- If advisory reports reflect significantly different evaluation from SEB conclusions, this should be reconciled in the SEB report

## SEB Report (FAR 15.305(a); 15.308; DOE AG Ch. 15.1, Part XII)

**Evaluations and ratings must be substantiated by specific strengths, weaknesses, deficiencies and risks to be considered credible and justifiable**
- Numerical scores, adjectival ratings, or other techniques are not conclusive upon which to make source selection decisions
- General assessments such as weak, poor, or excellent must be supported with specific reasons as to why the cited aspect of the *proposal is weak, poor, or excellent
**Does the report include a recommendation for selection?**
- If SSO requires, yes
- Normally a selection recommendation is not made

## SEB Report (FAR 15.305(a); 15.308; DOE AG Ch. 15.1, Part XII)

**If a competitive range was determined, weaknesses in the competitive range report should be addressed**
**What was proposed in final revised proposal**
**What is SEB’s evaluation of the final revised proposal**
**Does the final revised proposal eliminate the weakness **
**When drafting the strengths and weaknesses for the SEB report,  consider this to be the specific documentation that will be given to offerors for debriefs**
- Structure strengths and weakness to identify what was proposed, what is good or bad, what is the effect, and how it relates to the evaluation criteria
- Make the strengths and weaknesses understandable as stand-alone statements as much as possible
- Make sure all strengths and weaknesses are within the evaluation criteria
**Consider the SEB report as the first line of defense in a protest, i.e., make it clear, fair, consistent with the RFP, and definitive**

## SEB Briefing to SSO

**Normally the SEB formally briefs the SSO**
**Briefing consists of the key points from the SEB report**
**All SEB members and selected advisors, as appropriate, should attend or otherwise be available to answer questions**
**Advise SSO of various viewpoints considered to arrive at SEB consensus**
**Make available as the SSO desires: SEB report, sub-team reports, proposals, etc.**

## Debriefing of Offerors (FAR 15.505; 15.506; DOE AG Ch. 15.1, Part XVI)

**Offerors may request a debriefing**
**Pre-award** – within 3 days of notification of exclusion from the competitive range; or may request delay until post award
**Post-award **– within 3 days of notification of award
- In the government’s notification letter, advise offerors that a debriefing is available and provide the date (4-8 days after notification)
- Allow offerors to accept or reject the offer of a debriefing; this may start the “protest clock”
**Debriefing may be in writing or orally (phone or meeting)**
**Purpose of a debriefing is for the offeror to understand the evaluation of its proposal and for the SEB to answer reasonable questions about the process**
**Contracting Officer or SEB chair should conduct the debriefing, primarily supported by SEB members, as appropriate, and SEB counsel; if not conducting the debriefing the Contracting Officer must attend**

## Debriefing of Offerors (FAR 15.505; 15.506; DOE AG Ch. 15.1, Part XVI)

| Debriefing information includes |  |
| --- | --- |
| Pre-award | Post-award |
| Significant weaknesses or deficiencies in the offeror’s proposal
- Provide strengths and weaknesses from the SEB report
 | Significant weaknesses or deficiencies in the offeror’s proposal
- Provide strengths and weaknesses from  the SEB report
 |
| Past performance information on the debriefed offeror
- May not include the names of individuals providing information | Past performance information on the debriefed offeror
-May not include the names of individuals providing information
 |
| Summary of the rationale for eliminating the offeror from the competition | Summary of the rationale for award
- Probably a very general statement that does not disclose any information from the successful offeror’s proposal |
| Reasonable questions about the source selection process and applicable regulations | Reasonable questions about the source selection process and applicable regulations |
| Evaluated cost/price of the debriefed offeror
- Do not disclose any information, e.g., nature of adjustment, that would disclose SEB’s approach to other offerors’ evaluation | Evaluated cost/price and technical rating of the successful offeror and the debriefed offeror |

## Debriefing of Offerors (FAR 15.505; 15.506; DOE AG Ch. 15.1, Part XVI)

**SEB should be prepared**
- Assure the debriefing chair (SEB Chairman or CO) and other participants know the debriefing is to be conducted
- Prepare in advance reviewing the SEB report and RFP
- Know what to say and not to say
- Request debriefed offeror to provide questions in advance
**Lay out ground rules at the beginning of the debriefing with the offeror**
- Why we are here – provide additional information about the evaluation of your proposal
- What we can’t divulge – information on other offeror’s proposals
- Note the information provided to the offeror in advance, e.g., strengths, weaknesses, evaluated cost/price
- Prepared to answer questions submitted and others as appropriate
- Focus on the substance of the strengths/weaknesses, not the number or the ratings/score
**Debriefing Chair (SEB Chairman or CO) controls the meeting **
- Direct the offeror to information you can provide
- Direct offeror questions to appropriate SEB personnel
- Don’t argue about ratings (scores)
- Tone of the debriefing set by the debriefing chair can avoid protests
- Don’t argue
- Explain what can be said and explain what can’t be said and why
- Be helpful and provide as much information as possible within regulatory limits

## Debriefing of Offerors (FAR 15.505; 15.506; DOE AG Ch. 15.1, Part XVI)

**Stick to the “script”**
- RFP
- Procurement process defined in the RFP and the rating plan
- Selection based on the defined process
- Evaluation provided to the offeror
- Anything added to the strengths and weaknesses already provided opens up the potential that the record is not complete
- Nothing much that can/should be added to the strengths/weaknesses provided
- Do not provide information about other offerors’ proposals
**Bring the debriefing to a close**
- Don’t carryover questions/answers to future days
- Protest clock may not start until the debriefing is officially over
**Debriefings  properly conducted can help avoid a protest**
**Debriefings improperly conducted can prompt a protest**

## Protest Issues(FAR 15.507; FAR 33)

**Unstated evaluation criterion**
- Evaluating a proposal based on a factor not included in the evaluation criteria in section M
**Unreasonable technical evaluation**
- Technical evaluation is not well founded, not based on logical reasoning, or not well documented
**Unreasonable cost evaluation**
- Evaluation did not adequately analyze the proposed costs
- Probable cost adjustments not based on logical reasoning and calculations
- The cost evaluation is not well documented
**Poor technical-cost trade-off**
- Not consistent with the solicitation
- Not based on logical reasoning
**Failure to follow the evaluation factors**
- Some evaluation factors not properly considered

## Rating Plans(FAR 15.305(a); DOE AG Ch. 15.1, Part III)

**The purpose of a rating (scoring/ evaluation) plan is to help evaluators assess a proposal’s merit in relation to the evaluation criteria stated in the RFP**
- Describes the methodology to be used in evaluating proposals
- Allows proposals to be evaluated consistently against a uniform objective standard rather than proposal against proposal
- Developed by the SEB and approved by the SSO
**Rating plan consists of**
- Evaluation factors and sub-factors (as stated in the RFP)
- Rating system, e.g., adjectival, color coding, numerical, ordinal, or a combination thereof
- Evaluation standards or descriptions which explain the basis for assignment of the various rating system grades/scores

## Rating PlansFAR 15.305(a); DOE AG  Ch.15.1,Part III

**Guidelines for developing standards for non-cost evaluation factors**
- Define the standard by a narrative description that specifies a target performance level that the proposal must achieve in order to meet the factor or subfactor
- Uses a scale of words, colors, numbers, or other indicators to denote the degree to which proposals meet the standards for each factor or subfactor
- Describe guidelines for higher or lower ratings as compared to the standard
- Avoid overly general standards
- Makes consensus among evaluators more difficult
- May obscure the differences between proposals
**Common rating systems**
- Adjectival: Outstanding, good, satisfactory, marginal, unsatisfactory
- Color: Blue, green, yellow, amber, red
- Numerical: 0 -10 or 0 -100

## Rating PlansFAR 15.305(a); DOE AG Ch. 15.1,Part III

**The description of the basis for the assignment of the various rating systems is the most important aspect to define for all systems, i.e., outstanding vs. good, green vs. yellow, 80 vs. 90**
**Color and adjectival are similar**
- May force evaluator’s emphasis on strengths and weaknesses that equate to color or adjective rather than relying on a numbering system
**Numerical **
- Still requires a definition for assigning number as with color or adjectival
- Use staggered numeric system, e.g., 1,3, 5,  or 80, 90, 100, etc. to help separate proposal scores
- May cause “public school” mentality for scoring and less reliance on actual strengths and weaknesses
- Even when numerical systems are utilized, substantive strengths and weaknesses must still be identified to support the numerical score

##  Sample Rating Plans(FAR 15.305(a); DOE AG Ch.15.1,Part III)

| Numerical | Adjectival | Color | Definition/Standard |
| --- | --- | --- | --- |
| 10 | Excellent | Blue | Proposal demonstrates excellent understanding of requirements and approach that significantly exceeds performance or capability standards.  Has exceptional strengths that will significantly benefit the Government. |
| 8 | Good | Green | Proposal demonstrates good understanding of requirements and approach that exceeds performance or capability standards.  Has one or more strengths that will benefit the Government. |
| 5 | Satisfactory | Yellow | Proposal demonstrates acceptable understanding of requirements and approach that meets performance or capability standards. Acceptable solution.  Few or no strengths.  |
| 3 | Marginal | Amber | Proposal demonstrates shallow understanding of requirements and approach that only marginally meets performance or capability standards necessary for minimal but acceptable performance. |
| 0 | Unsatisfactory | Red | Fails to meet performance or capability standards.  Requirements can only be met by major changes to the proposal. |

## SEB Preparation Prior to Receipt of Proposals

**Ensure all SEB members and all advisors understand the process and expectations **
- Provide written information/guidance as appropriate
- Hold meetings prior to receipt of proposals
- Make sure everyone is ready to perform his/her job when proposals are received
- Make sure everyone generally understands others’ functions and how they fit into the process
**Plan the details**
- Steps and schedule for the evaluation process
- Review the RFP evaluation criteria and rating plan
- Receipt and safeguarding of proposals; limitations on where proposals can be taken
- Review of conflict of interest certifications after receipt of proposals, i.e., an unexpected offeror can affect previous certifications of advisors and SEB members
- Review the confidentiality of proposals particularly related to normal management chain
- Content and format of advisory reports, e.g., identify strengths/weakness and no point scores; advisory reports are part of the official record
- Participation in any oral presentations is “all or none” for specific individuals, e.g., SEB members, certain advisors

## HQ Business Clearance(DEAR 971; DOE AG Ch. 71)

**Annual call identifies actions selected for review**
**Documents normally reviewed for major procurements**
- SSO and SEB designation
- Site Utilization and Management Plan
- Acquisition Plan
- Draft (public) and final RFP
- Expect several revisions
- Draft Rating Plan
- Draft SEB report
- Expect several revisions
- Draft source selection statement
- Expect several revisions
- Communications plan for award
- Draft press release announcing award
**Engage your HQ “buddy” early and throughout the acquisition process**