# Presentation: 886345

## Procurement Forum March 12, 2008Agenda

- Welcome
- Performance Appraisal Rating System and Guidelines
- 210.S Overview
**Contractor Performance Evaluations (NF 1680)**
- Closeout Best Practices
- Wrap-Up

## Procurement Training Forum

- Contractor Performance Information
- (NF 1680)

## OUTLINE

- Policy and Procedures
- Evaluation Form
- NF 1680
- Rating Definitions
- Best Practices
- Samples

## POLICY

- FAR 42.15, NFS 1842.15, and PIC  01-12
- Purpose
- Communications with Contractor
- Source Selection Information
- Evaluations used to determine award fee or incentive fee payments may be used to satisfy FAR requirements
- Update to PPDB each award fee period for award and incentive fee contracts

**Notes:**
Past Performance Information includes, for example, 

the contractor’s record of conforming to contract requirements and to standards of good workmanship; 
the contractor’s record of forecasting and controlling costs; 
the contractor’s adherence to contract schedules, including the administrative aspects of performance; 
the contractor’s history of reasonable and cooperative behavior and commitment to customer satisfaction; 
and generally, the contractor’s business-like concern for the interest of the customer.

Fees, however, should reflect contractor performance, and past performance evaluations should closely parallel fee determinations

## PROCEDURES

- Evaluations required for:
- Contracts exceeding simplified acquisition threshold
- Contracts exceeding 1 year period of performance
- Purchase Orders and BOA orders over $100K
- Evaluation _not_ required for Interagency Agreements

**Notes:**
No evaluations for award under FAR Subpart 8.6, Federal Prison Industries, Inc.
No evaluations for awards with non-profit agencies employing the blind or severely disabled (FAR Subpart 8.7)
Evaluations for construction and A&E awards per FAR 36.201 and FAR 36.604
Evaluations for construction and A&E via SF 1420 and SF 1421 (not entered into PPBD)

## PROCEDURES

- Interim evaluations, annually
- Within 60 days of anniversary
- Optional on SBIR/STTR Phase II’s
- Final evaluations at completion
- Final, summarizing all performance, must be done
- Minimum of 30 days for contractor comments
- Review of timely disagreement (upon contractor request) at one level above Contracting Officer and must be within 30 days
- Copy of all finalized evaluations must be provided to contractor

**Notes:**
For entity, division or unit that performs contract
Content and format specified by agency procedures
Tailored to size, content, and complexity of contract requirements
When Small Business Subcontract Plan required:
Assessment of performance against, and efforts to achieve, goals in plan

## PROCEDURES

- Copies of full evaluation (with contractor comments, etc.) must be retained and marked “Source Selection Information”
- Disclosure restricted to Government personnel and the contractor during period it’s used as SSI
- Retain for purposes of source selection for 3 years after completion of contract performance
- Retain in contract file as provided in FAR 4.8
- Create and save NF 1680 in PPDB

**Notes:**

   Agency procedures must provide for review one level above C.O. for disagreements
   Ultimate conclusion is agency decision
Agencies shall share past performance information when requested, by interview or copy of evaluation
Past performance systems should ensure only authorized personnel have access to data

The NF1680 generated by the PPDB is not marked as “Source Selection Information” from the system.  (Claudia)

## NF 1680

- NASA Form 1680, entitled, "Evaluation of Performance," shall be used to document evaluations.
- Contractor performance rated for the following:
- Attributes
- Quality
- Timeliness
- Price/Cost
- Other (as specified)
- Ratings
- Excellent
- Very Good
- Good
- Satisfactory
- Poor/Unsatisfactory

- Attributes should
- be defined as part
- of the evaluation

- Rating definitions at NFS 1816.405-275, Award Fee Evaluation Scoring

## RATING DEFINITIONS

_**Excellent**_:  Of exceptional merit; exemplary performance in a timely, efficient, and economical manner; very minor (if any) deficiencies with no adverse effect on overall performance.
_**Very good**_:  Very effective performance, fully responsive to contract requirements; contract requirements  accomplished in a timely, efficient, and economical manner for the most part; only minor deficiencies.
_**Good**_:  Effective performance; fully responsive to contract requirements; reportable deficiencies, but with little identifiable effect on overall performance.
_**Satisfactory**_:  Meets or slightly exceeds minimum acceptable standards; adequate results; reportable deficiencies with identifiable, but not substantial, effects on overall performance.
_**Poor/Unsatisfactory**_:  Does not meet minimum acceptable standards in one or more areas; remedial action required in one or more areas; deficiencies in one or more areas which adversely affect overall performance.

## BEST PRACTICES

- Start early with COTR request
- Minimum of 30 days before anniversary date
- Use MM or Outlook schedule reminders
- Meet with COTR in-person, if possible
- Give purpose for evaluation
- Provide rating definitions and samples
- Work with COTR and Other Stakeholders (COTR Management, Resources, QA, End Users, etc.)
- Frequent follow-ups may be necessary
- Use detailed and thoughtful narratives, that:
- Define attributes
- Assign ratings in context of rating definition
- Use facts (as opposed to opinions)
- Maintain consistency with assigned rating definition

- See Samples

**Notes:**
Large dollar complex contracts can take up to 90 days to get evaluation ready to release to contractor for comment.

Agency procedures provide for input by technical office and end user, where appropriate.
The Contract Specialist/Contracting Officer is a stakeholder too.  CO/CS should understand
and agree with ratings and narratives, particularly under Cost/Price and Other attributes. 

## BEST PRACTICES

- Prepare and edit narratives in MS Word and cut and paste into PPDB
- Provide contractor advance notice it will receive evaluation
- Explain purpose of evaluation
- With evaluation, provide contractor:
- Rating definitions
- Notice that evaluation will be placed in database after 30 days
- Invitation to discuss evaluation, if appropriate
- Invite COTR to evaluation discussion
- Prepare COTR before discussion

**Notes:**
Some configuration control may be needed on evaluations with many stakeholders.

## 		BEST PRACTICES

- Contractors may ask why they did not receive a higher rating
- Discussion is _not_ a negotiation
- Ratings should not change simply because contractor disagrees
- Narratives should not change, unless factually incorrect
- Review rating definitions with contractor
- Consistency between narrative and rating definition helps
- Remind contractor that written comments on evaluation are welcome
- Contractor’s comments should be added verbatim

## BEST PRACTICES

- Give COTR (and other stakeholders) a due date for responses to contractor comments
- COTR will likely respond quickly, but in some cases follow-ups may be necessary
- If COTR needs to coordinate with his/her manager, you may want to go to the manager (e.g., Project Manager)
- NASA review should --
- Address contractor’s disagreements
- Document discussions, if any, held after comments received
- Summarize final findings

## BEST PRACTICES

- Past Performance Database (PPDB)
- _[https://prod.nais.nasa.gov/cgi-bin/ppdb/home.cgi](https://prod.nais.nasa.gov/cgi-bin/ppdb/home.cgi)_
- Web Data Entry
- Contract Number (NAS 5 XXXXX, S XXXXXY, NNG0XXXXXC) – Must be active contract
- New eval or edit in-process eval
- Verify contract data (changes to PSST)
- Enter evaluation
- Must have cage code or DUNS number
- Enter CO signature date and save when evaluation complete (this triggers input into database)
- Reports

## Backup

- Samples

## EXCELLENT

-        **Other:**
-       This attribute covers the contractor's performance in the area of the contract/business management function (including staffing and resources).  NASA GSFC is very pleased with the level of commitment the contractor has in connection with the contract/business management function.  The contractor is a Small Disadvantaged Business that exhibits a high level of commitment to meeting contract level requirements.  As this is the contractor’s first contract with NASA, the many contract/business requirements (especially those associated with cost-reimbursement contracting) have the potential to overwhelm the small company, but the contractor displays an eagerness to learn how to do business with NASA and outstanding professionalism.  It is a pleasure to work with the contractor’s business and financial team.  In particular, NASA is pleased with the level of commitment that the contractor’s business management staff has exhibited in the area of reporting via the Contractor Cost Report (NF533 Monthly and Quarterly). NASA recognizes the contractor’s inexperience in this area and is pleased with the can do attitude that the business team exhibits.  It is very important to NASA that it receive timely submission of the cost report data.  This is essential as its contents are relied upon by senior management.  The contractor has made significant improvements in this area and we are looking forward to their continued timely submissions.

- Source Selection Information—See FAR 3.104

## VERY GOOD

-    **Timeliness:**
-     This attribute covers contractor performance in the area of compliance with delivery dates for contract level deliverables.  In general NASA GSFC finds the contractor's performance to be fully responsive to contract requirements.  One minor deficiency is noted, but it does not effect the overall performance.  The Quarterly Status Reviews are slightly behind schedule, due to unforeseen and unavoidable events.  Other contract level deliverables have been received in a timely manner.

- Source Selection Information—See FAR 3.104

## GOOD

-    **Cost/Price:**
**  ** This attribute covers contractor performance in the area of price or cost control.  In general, NASA GSFC finds the contractor's performance to be effective in this area.

- Source Selection Information—See FAR 3.104

## SATISFACTORY

-     **Technical:**
-      This attribute covers contractor performance in the areas of compliance with contract requirements related to the spacecraft’s technical performance, as well as, quality assurance; and efficiency and effectiveness of the contractor’s program management function (including, management of staffing and resources).  In general, NASA GSFC finds the contractor’s performance meets minimum acceptable standards.  The Observatory has been integrated, and currently is in performance testing.  Compliance with a majority of the contract’s technical requirements is yet to be verified, and the technical performance of the spacecraft is ultimately established after successful completion of the Observatory environmental test program.  Several areas of concern are discussed below: ...

- Source Selection Information—See FAR 3.104

## POOR

-        **Timeliness:**
-        This attribute covers contractor performance in the areas of progress towards and compliance with contract delivery dates, as well as, the efficiency and effectiveness of the schedule management function.  A poor/unsatisfactory rating is given as the contractor’s performance in these areas does not meet minimum acceptable standards.
-        The contractor senior management failed to support its own program management team with adequate staffing, which led to significant delays in the C&DH (2 years), as well as most other activities across the program.  Delay in completion of the C&DH resulted in a significant delay in completing Spacecraft Integration and Test (I&T), a major milestone event under the delivery order.  The spacecraft bus and I&T teams were not prepared when the instruments arrived in the Summer of 2006.  Spacecraft bus I&T started only 2 weeks prior to the arrival of the science instrument in July 2006.  The original I&T plan had spacecraft bus I&T completion five months before delivery of the instruments.
-        Many program activities are not well planned including Observatory comprehensive performance testing, fault management testing, Observatory power testing, and on-orbit scenario testing, ultimately leading to delays in completion of these activities.  Launch site planning and preparation activities are at risk of falling behind.
-        The common underlying issue is not the skills of the contractor engineers or management supporting the program, but inadequate staffing and resources in general.

- Source Selection Information—See FAR 3.104