Number range CHAPTEROMB 0990-0115

**PART I - THE SCHEDULE** **Request for Proposal No. AHRQ-06-000002**

**SECTION A - SOLICITATION FORM**

Date Issued: **May 22, 2006**

Date Questions Due: **June 7, 2006, 4PM**

Date Notice of Intent Due: **June 7, 2006**

Date Proposals Due: **July 25, 2006 12 Noon EST**

You are invited to submit a proposal to the Agency for Healthcare
Research and Quality (AHRQ) for Request for Proposal (RFP) No.
AHRQ-06-00002, entitled "Innovations Clearinghouse." Your proposal must
be developed and submitted in accordance with the requirements and
instructions of this RFP.

A cost reimbursement, performance based cost reimbursable contract is
contemplated for a period of three years with one two-year option.

For this acquisition, the AHRQ recommended goal (as a percentage of
total planned subcontracting dollars for the base period) is 30% for
Small Businesses, which shall include at least 5% (as a percentage of
total contract value for the base period) for Small Disadvantaged
Businesses, at least 5% (as a percentage of total planned subcontract
dollars for the base period) for Women-Owned Small Businesses, and at
least 3% (as a percentage of total planned subcontract dollars for the
base period) for HUBZone Small Businesses and at least 3% (as a
percentage of total planned subcontract dollars for the base period) for
Veteran-Owned Small Businesses. These goals represent AHRQ's expectation
of the minimum level for subcontracting. The North American Industry
Classification System (NAICS) code that best describes the requirement
is 541990. The small business size standard is \$6.5 million.

Offerors shall submit the following:

A. Technical Proposal (See Section L.9) (Original, 11 copies, 1
electronic)

B. Past Performance Information (See Section L.10) (Original and 3
copies)

C. Small Disadvantaged Business Participation Plan (See Section L.11)
(Original and 2 copies)

D. Business Proposal (See Section L.12) (Original and 5 copies)

The Small Business Subcontracting Plan must be submitted as a separate
section of the Business Proposal. (This does not apply to small business
concerns).

Your technical proposal must be concisely written and should be limited
to **150 typewritten pages** (double-spaced), exclusive of personnel
qualifications (i.e., CV or biosketch, see Section L.10 for additional
details). This limitation is for administrative purposes only and
exceeding the limitation shall not, of itself, be considered a basis for
rejection of your proposal. Each electronic copy must be on an
individual CD, the format compatible with Microsoft Word. See L.10 for
instructions on Appendices.

Your proposal must provide the full name of your company, the address,
including county, Tax Identification Number (TIN), DUN and Bradstreet
No., and if different, the address to which payment should be mailed. A
point of contact must be easily identified

**YOUR ATTENTION IS CALLED TO THE LATE PROPOSAL PROVISIONS PROVIDED IN
SECTION L.3 OF THIS RFP. YOUR ATTENTION IS ALSO DIRECTED TO THE
TECHNICAL PROPOSAL INSTRUCTIONS PROVIDED IN SECTION L.10 OF THE
SOLICITATION.**

If you intend to submit a proposal in response to this solicitation,
please inform the Contract Officer of your intent by completing the
Proposal Intent Response Form (***Attachment 3*** to this solicitation)
and submit the form no later than the date specified. You may send it to
the address below or fax it to 301-427-1740, Attention: Mary Haines,
Contracting Officer.

Questions regarding this solicitation shall be received in this office
no later than the date specified. (See Section L.7). It is preferred
that all questions be submitted electronically by e-mail to Mary Haines,
Contracting Officer at the following email address:
<mary.haines@ahrq.hhs.gov> . Otherwise, please address your written
questions to Mary Haines, Contracting Officer, Agency for Healthcare
Research and Quality, 540 Gaither Road, Rockville, Maryland 20850 and
the envelope should be marked "Proposal Questions RFP No.
AHRQ-06-00002."

Answers to questions will be provided in the form of an Amendment to
this solicitation and will be posted on AHRQ's web page:
[[www.ahrq.gov]{.underline}](http://www.ahrq.gov/) under "Funding
Opportunities," "Contracts" and the Federal Business Opportunities web
page: [www.fedbizopps.gov](http://www.fedbizopps.gov/). It is your
responsibility to monitor the web sites where the RFP will be posted to
learn about any amendments to the solicitation

**[Discussions with any other individual outside the Division of
Contracts Management, may result in rejection of the potential offeror's
proposal]{.underline}.**

The proposal shall be signed by an authorized official to bind your
organization and must be received in our Contracts Office no later than
**12 noon**, **EST**, on **July 25, 2006**. Your proposal must be mailed
to the following address:

Agency for Healthcare Research and Quality

Division of Contracts Management

540 Gaither Road, Room 4319

Rockville, Maryland 20850

Hand carried proposals may be dropped off at the above location.
However, please allow ample time as proposals cannot be accepted until
they have gone through security. We will not be held responsible for any
delays that may be incurred getting your proposal through security.

NOTE: The U.S. Postal Service's "Express Mail" [does not]{.underline}
deliver to our Rockville, Maryland address. Packages delivered via this
service will be held at a local post office for pick-up. [The Government
will not be responsible for picking up any mail at a local post
office]{.underline}. If a proposal is not received at the place, date,
and time specified herein, it will be considered a "late proposal."

The RFP does not commit the Government to pay any cost for the
preparation and submission of a proposal. It is also brought to your
attention that the Contracting Officer is the only individual who can
legally commit the Government to the expenditure of public funds in
connection with the proposed acquisition.

In accordance with Federal Acquisition Circular (FAC) 2001-16, all
contractors must be registered in the central contractor registration
(CCR) database in order to conduct business with the government \[See
Section I -  FAR clause 52.204-7 Central Contractor Registration (OCT
2003), Alternate 1 (Oct 2003)\] .  As stated in paragraph (h) of this
clause, additional information can be obtained at
[http://www.ccr.gov](http://www.ccr.gov/) or by calling 1-888-227-2423,
or 269-961-5757.

Requests for any information concerning this RFP should be referred to
Mary Haines at: <mary.haines@ahrq.hhs.gov> Please note e-mail requests
**must** state subject as **RFP AHRQ 06-00002**.

Sincerely,

Mary Haines

> Contracting Officer
>
> Agency for Healthcare Research and Quality

**[TABLE OF CONTENTS]{.underline}**

**[PART I]{.underline} [Pages]{.underline}**

Section A Solicitation 1-3

Table of Contents 4

Section B Supplies or Services & Prices/Costs 5-8

Section C Description/Specification/Work Statement 8

Section D Packaging and Marking 8

Section E Inspection and Acceptance 8-12

Section F Deliveries or Performance 12-15

Section G Contract Administration Data 15-20

Section H Special Contract Requirements 21-25

**[PART II]{.underline}**

Section I Contract Clauses 26-32

**[PART III]{.underline}**

Section J List of Attachments 32

**[PART IV]{.underline}**

Section K Representations and Instructions 33-39

Section L Instructions, Conditions & Notices to Offerors 40-64

Section M Evaluation Factors for Award 65-69

**[Attachments]{.underline}**

1\. Statement of Work 70 - 107

with Appendices

A. Glossary of Key Terms Used in the Innovations Clearinghouse RFP 2 pgs

B. Innovations Clearinghouse Expert Panel Meeting 20 pgs

C. AHRQ Web Product and web Site Development Guidelines 6 pgs

D Guidelines for Developing AHRQ Web-based Tools 6 pgs

E DRAFT AHRQ Technical Reference Model, Version 0.1

January 24, 2006 19 pgs

2\. Past Performance Questionnaire and Contractor Performance Form 5 pgs

3\. Proposal Intent Response Sheet 1 pg

4\. Breakdown of Proposed Estimated Cost and Labor Hours 2 pg

**[Exhibit 1]{.underline}** Performance Requirements Summary

(to be provided as amendment to the RFP)

**[SECTION B-SUPPLIES OR SERVICES AND PRICES/COSTS]{.underline}**

**B.1 BRIEF DESCRIPTION OF SUPPLIES OR SERVICES**

> "Innovations Clearinghouse" See Section C (Attachment 1) for a
> complete description.

**B.2. ESTIMATED COST**

**Note:** The Government estimates the cost of this procurement at
approximately \$3.3 Million for Year 1, \$3.2 Million for Year 2, \$3.4
Million for Year 3 and \$6.6 Million for the two-year option period.

> a\. The estimated cost (exclusive of fees) for performance of the work
> under this three (3) year contract, including direct and indirect
> costs is \$ (TO BE NEGOTIATED)

b.  The fixed fee for this contract is \$ (TO BE NEGOTIATED). The fixed
    fee shall be paid in installments based on the percentage of
    completion of work, as determined by the Contracting Officer.
    Payment shall be subject to the withholding provisions of the Clause
    ALLOWABLE COST AND PAYMENT and FIXED FEE incorporated herein.

c.  The maximum amount of award fee that may be earned for this contract
    is \$ (TO BE NEGOTIATED). Award fee earned shall be based upon an
    evaluation and determination by the Government as to the
    Contractor's level of performance in accordance with the following
    procedures:

    1.  The Contractor's performance shall be evaluated on a
        seimi-annual basis, during the performance of the contract. The
        award periods and maximum amounts for each are listed in Section
        H, Special Contract Requirements, H.1 Performance Evaluation and
        Award Fee.

    2.  The criteria set forth in the Performance Requirements Summary,
        Attachment 1, shall be used to evaluate the Contractor's
        performance.

    3.  The Contractor further agrees that the final determination as to
        the amount of Award Fee earned will be made by the Contracting
        Officer, taking into consideration an analysis and evaluation of
        the Contractor's performance made by the Evaluation Group
        described in Section H.1, and shall not be subject to the terms
        of the "Disputes" clause of this contract. The Contractor shall
        be advised in writing of the decision setting forth reasons why
        the Award Fee was earned or why it was not earned, in order that
        the Contractor may improve its performance during the next
        six (6) month period, if the latter is applicable.

    4.  Notwithstanding any other provisions of this contract, the fee
        for performing this contract shall not exceed the statutory
        limitations prescribed in the first sentence of Section 304(b)
        of the Federal Property and Administrative Services Act (41 USC
        254(b)) for services other than research, development or
        experimental work.

    5.  Authorization to claim and be reimbursed for award fee under
        this contract will be accomplished through correspondence
        initiated by the Contracting Officer. When the award fee is
        determined, the Contractor will be notified. At that time the
        Contractor may submit a public voucher for payment of the total
        award fee earned for the period evaluated. Payment of the award
        fee shall be subject to the withholding provision of the clause
        entitled "Fixed Fee."

d\. The Government's maximum obligation, represented by the sum of the
estimated cost plus the fixed fee and award fee obtainable for the
contract period is as follows:

(TO BE NEGOTIATED)

+---------------+-------------+----------+-------------+-------------+
| Period of     | Estimated   | Fixed    | Maximum     | Total       |
|               |             | Fee      |             | Estimated   |
| Performance   | Cost        |          | Award Fee   |             |
|               |             |          |             | Cost Plus   |
|               |             |          |             | All         |
|               |             |          |             |             |
|               |             |          |             | Fees        |
+---------------+-------------+----------+-------------+-------------+
| Year 1        |             |          |             |             |
|               |             |          |             |             |
| 09/27/06 --   |             |          |             |             |
| 09/26/07      |             |          |             |             |
+---------------+-------------+----------+-------------+-------------+
| Year 2        |             |          |             |             |
|               |             |          |             |             |
| 09/27/07 -    |             |          |             |             |
| 09/26/08      |             |          |             |             |
+---------------+-------------+----------+-------------+-------------+
| Year 3        |             |          |             |             |
|               |             |          |             |             |
| 09/27/08 --   |             |          |             |             |
|               |             |          |             |             |
| 09/26/09      |             |          |             |             |
+---------------+-------------+----------+-------------+-------------+
| TOTAL         |             |          |             |             |
+---------------+-------------+----------+-------------+-------------+

e\. Total funds currently available for payment and allotted to this
contract are \$(TO BE NEGOTIATED) of which \$ (TO BE NEGOTIATED)
represents the estimated cost, and of which \$(TO BE NEGOTIATED)
represents the fixed fee and \$(TO BE NEGOTIATED) represents the award
fee pool.

f\. It is estimated that the amount currently allotted will cover
performance of the contract through (TO BE NEGOTIATED) .

g\. The Contracting Officer may allot additional funds to the contract
without the concurrence of the Contractor. For further provisions on
funding, see the LIMITATION OF COST/LIMITATION OF FUNDS and the
ALLOWABLE COST AND PAYMENT (AND FIXED FEE) clauses incorporated herein.

h\. COST AND PAYMENT (AND FIXED FEE) clauses incorporated into this
contract.

**B.3 OPTION PERIOD**

In the event that the option period is exercised, the total estimated
cost, fixed fee and award fee will be increased by the following
amounts:

(TO BE NEGOTIATED)

+--------------+------------+------------+-------------+-------------+
| Period of    | Estimated  | Fixed      | Maximum     | Total       |
| Performance  |            |            |             | Estimated   |
|              | Cost       | Fee        | Award Fee   | Cost Plus   |
|              |            |            |             | All Fees    |
+--------------+------------+------------+-------------+-------------+
| Option       |            |            |             |             |
|              |            |            |             |             |
| 09/30/08 --  |            |            |             |             |
| 09/29/09     |            |            |             |             |
+--------------+------------+------------+-------------+-------------+
| TOTAL        |            |            |             |             |
+--------------+------------+------------+-------------+-------------+

**B.4 PROVISIONS APPLICABLE TO DIRECT COSTS**

a\. Items Unallowable Unless Otherwise Provided Notwithstanding the
clauses, ALLOWABLE COST AND PAYMENT, and FIXED FEE, incorporated into
this contract, unless authorized in writing by the Contracting Officer,
the costs of the following items or activities shall be unallowable as
direct costs:

\(1\) Acquisition, by purchase or lease, of any interest in real
property;

\(2\) Rearrangement or alteration of facilities;

\(3\) Purchase or lease of any item of general purpose-office furniture
or office equipment regardless of dollar value. (General purpose
equipment is defined as any items of personal property which are usable
for purposes other than research, such as office equipment and
furnishings, pocket calculators, etc.);

\(4\) Accountable Government property (defined as both real and personal
property with an acquisition cost of \$1,000 or more, with a life
expectancy of more than two years) and \"sensitive items\" (defined and
listed in the Contractor\'s Guide for Control of Government Property,
1990, regardless of acquisition value;

\(5\) Travel to attend general scientific meetings;

\(6\) Foreign Travel;

\(7\) Any costs incurred prior to the contract\'s effective date;

\(8\) Rental of meeting rooms not otherwise expressly paid for by the
contract;

\(9\) Any formal subcontract arrangements not otherwise expressly
provided for in the contract

> \(10\) Consultant fees in excess of \$1000/day; and

\(11\) Information Technology hardware or software.

b.  This contract is subject to the provisions of Public Law (P.L.)
    99-234 which amends the Office of Federal Procurement Policy Act to
    provide that contractor costs for travel, including lodging, other
    subsistence, and incidental expenses, shall be allowable only to the
    extent that they do not exceed the amount allowed for Federal
    employees. The Contractor, therefore, shall invoice and be
    reimbursed for all travel costs in accordance with Federal
    Acquisition Regulations (FAR) 31.205-46.

**[SECTION C/ STATEMENT OF WORK]{.underline}**

**DESCRIPTION/SPECIFICATION/WORK STATEMENT**

**The Statement of Work is located at Attachment 1.**

Number range CHAPTER **[SECTION D - PACKAGING AND MARKING]{.underline}**

Not Applicable

**[SECTION E - INSPECTION AND ACCEPTANCE]{.underline}**

**E.1 INSPECTION AND ACCEPTANCE**

a\. The contracting officer or the duly authorized representative will
perform inspection and acceptance of materials and services to be
provided.

b\. For the purpose of this SECTION the Government Project Officer is
the authorized technical representative of the contracting officer.

c\. Inspection and acceptance will be performed at:

Agency for Healthcare Research and Quality

540 Gaither Road

Rockville, Maryland 20850

**E.2 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)**

> This contract incorporates the following clause by reference, with the
> same force and effect as if it were given in full text. Upon request,
> the Contracting Officer will make its full text available.

**FAR Clause No. Title and Date**

52.246-5 Inspection of Services-Cost Reimbursement (April 1984)

**E.3 PERFORMANCE EVALUATION AND AWARD FEE**

The contract will be awarded and managed as a Performance-Based Service
Contract (PBSC), using the Award Fee mechanism.

**1. [Performance Evaluation Factors]{.underline}**

The Government will evaluate the following factors of the Contractor's
performance, weights will be provided in an amendment to the RFP:

  ---------------------------------------------- ----------- ---------- -----------
  **Performance Factor**                         **Related   **Weight   **Weight
                                                 SOW Tasks** (Year 1)** (Year 2 and
                                                                        Beyond)**

  1\. Recruit, Manage, and Support an Expert     Task 1                 
  Panel.                                                                

  2\. Establish, Test and Apply Inclusion        Task 2                 
  Criteria.                                                             

  3\. Design, Develop, Implement, and Maintain a Task 3                 
  System to Obtain, Publish, Update and Archive                         
  Innovations.                                                          

  4\. Design, Develop, Implement, and Maintain a Task 4                 
  System for Indexing and Abstracting and                               
  Validating the Indexing and Abstracting of                            
  Innovations.                                                          

  5\. Design, Develop, Test, Deploy, Maintain    Task 5                 
  and Operate the Innovations Clearinghouse                             
  Technical Infrastructure and Web Site.                                

  6\. Maintain and Operate AHRQ's QualityTools   Task 6                 
  Database and Web Site.                                                

  7\. Provide Technical Assistance to            Task 7                 
  QualityTools and Innovations Clearinghouse and                        
  Users.                                                                

  8\. Design, Develop, Implement, and Maintain   Task 8      \-\--      
  Opportunities for Learning and Networking                             
  Among Innovations Clearinghouse Users.                                

  9\. Assist the Agency in Evaluation of the     Task 9      \-\--      
  Innovations Clearinghouse Web Site.                                   

  10\. Project Management.                       Task 10                
  ---------------------------------------------- ----------- ---------- -----------

**2. [Performance Requirements Summary]{.underline}**

> Exhibit 1 summarizes the performance standards and Government
> surveillance methods for each of the above performance factors.
>
> **3. [Award Fee Plan]{.underline}**
>
> ([Note to Offerors]{.underline}: The Government anticipates a
> Cost-Plus-Award Fee (CPAF) contract to result from this solicitation.
> In this type of contract, the Contractor will receive a small base
> fee. In addition to the base fee, award fee will be tied to the
> evaluation of specific products and services in accordance with
> Exhibit 1- Performance Requirements Summary.
>
> The Agency's decision to pay or not to pay Award Fee in no way alters
> the Contractor's responsibilities to perform any services or produce
> any deliverables required by this contract. The Agency's decision to
> pay or not to pay Award Fee in no way alters the Agency's obligation
> to pay the Contractor for satisfactory deliverables in accordance with
> this contract.

Award Fee is available for services and products identified below.

> **Annual Amounts Available for Award Fee (to be evaluated
> semi-annually):**

**Contract Year**

+-------------+------+------------+----------+-----------+-----------+
| **          | **%  | **Award    | **Award  | **Award   | **Award   |
| Performance | of   | for        | for      | for**     | for**     |
| Evaluation  | A    | Evaluation | Eval     |           |           |
| Factor**    | ward | Unac       | uation** | **Eva     | **E       |
|             | Fee  | ceptable** |          | luation** | valuation |
|             | Po   |            | **Satisf |           | Outs      |
|             | ol** | Rating     | actory** | **Exceeds | tanding** |
|             |      | score of   |          | Expec     |           |
|             | **   | below 60   | 60-79    | tations** | 90-100    |
|             | Year | reduces    | score    |           | score     |
|             | 1**  | Base Fee   |          | 80-89     |           |
|             |      | by 50% for | 40%      | score     | 100%      |
|             | **\_ | rating     |          |           |           |
|             | \_\_ | period.    |          | 80%       |           |
|             | \_\_ |            |          |           |           |
|             | \_** |            |          |           |           |
|             |      |            |          |           |           |
|             | **Y  |            |          |           |           |
|             | ears |            |          |           |           |
|             | 2    |            |          |           |           |
|             | -5** |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Recruit,    | TBD% |            |          |           |           |
| Manage, and | \_\_ |            |          |           |           |
| Support an  | \_\_ |            |          |           |           |
| Expert      | \_\_ |            |          |           |           |
| Panel.      |      |            |          |           |           |
|             | TBD% |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Establish,  | TBD% |            |          |           |           |
| Test and    | \_\_ |            |          |           |           |
| Apply       | \_\_ |            |          |           |           |
| Inclusion   | \_\_ |            |          |           |           |
| Criteria.   |      |            |          |           |           |
|             | TBD% |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Design,     | TBD% |            |          |           |           |
| Develop,    | \_\_ |            |          |           |           |
| Implement,  | \_\_ |            |          |           |           |
| and         | \_\_ |            |          |           |           |
| Maintain a  |      |            |          |           |           |
| System to   | TBD% |            |          |           |           |
| Obtain,     |      |            |          |           |           |
| Publish,    |      |            |          |           |           |
| Update and  |      |            |          |           |           |
| Archive     |      |            |          |           |           |
| I           |      |            |          |           |           |
| nnovations. |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Design,     | TBD% |            |          |           |           |
| Develop,    | \_\_ |            |          |           |           |
| Implement,  | \_\_ |            |          |           |           |
| and         | \_\_ |            |          |           |           |
| Maintain a  |      |            |          |           |           |
| System for  | TBD% |            |          |           |           |
| Indexing    |      |            |          |           |           |
| and         |      |            |          |           |           |
| Abstracting |      |            |          |           |           |
| I           |      |            |          |           |           |
| nnovations. |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Design,     | TBD% |            |          |           |           |
| Develop,    | \_\_ |            |          |           |           |
| Test,       | \_\_ |            |          |           |           |
| Deploy,     | \_\_ |            |          |           |           |
| Maintain    |      |            |          |           |           |
| and Operate | TBD% |            |          |           |           |
| the         |      |            |          |           |           |
| Innovations |      |            |          |           |           |
| Cl          |      |            |          |           |           |
| earinghouse |      |            |          |           |           |
| Technical   |      |            |          |           |           |
| Inf         |      |            |          |           |           |
| rastructure |      |            |          |           |           |
| and Web     |      |            |          |           |           |
| Site.       |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Maintain    | TBD% |            |          |           |           |
| and Operate | \_\_ |            |          |           |           |
| AHRQ's      | \_\_ |            |          |           |           |
| Q           | \_\_ |            |          |           |           |
| ualityTools |      |            |          |           |           |
| Database    | TBD% |            |          |           |           |
| and Web     |      |            |          |           |           |
| Site.       |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Provide     | TBD% |            |          |           |           |
| Technical   | \_\_ |            |          |           |           |
| Assistance  | \_\_ |            |          |           |           |
| to          | \_\_ |            |          |           |           |
| Q           |      |            |          |           |           |
| ualityTools | TBD% |            |          |           |           |
| and         |      |            |          |           |           |
| Innovations |      |            |          |           |           |
| Cl          |      |            |          |           |           |
| earinghouse |      |            |          |           |           |
| Users.      |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Design,     | \    |            |          |           |           |
| Develop,    | -\-- |            |          |           |           |
| Implement,  |      |            |          |           |           |
| and         | \_\_ |            |          |           |           |
| Maintain    | \_\_ |            |          |           |           |
| Op          | \_\_ |            |          |           |           |
| portunities |      |            |          |           |           |
| for         | TBD% |            |          |           |           |
| Learning    |      |            |          |           |           |
| and         |      |            |          |           |           |
| Networking  |      |            |          |           |           |
| of          |      |            |          |           |           |
| Cl          |      |            |          |           |           |
| earinghouse |      |            |          |           |           |
| Users.      |      |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Assist the  | \    |            |          |           |           |
| Agency in   | -\-- |            |          |           |           |
| Evaluation  |      |            |          |           |           |
| of the      | \_\_ |            |          |           |           |
| Innovations | \_\_ |            |          |           |           |
| Cl          | \_\_ |            |          |           |           |
| earinghouse |      |            |          |           |           |
| Web Site.   | TBD% |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+
| Project     | TBD% |            |          |           |           |
| Management. | \_\_ |            |          |           |           |
|             | \_\_ |            |          |           |           |
|             | \_\_ |            |          |           |           |
|             |      |            |          |           |           |
|             | TBD% |            |          |           |           |
+-------------+------+------------+----------+-----------+-----------+

On a semi-annual basis, the Contractor's products and services will be
evaluated in terms of the above performance factors by an Award Fee
Evaluation Group (AFEG). The AFEG will consist of the Project Officer,
the Contracting Officer or his/her designee, and, as appropriate, other
Government officials selected by the Project Officer (depending on
specific expertise) and approved by the Contracting Officer.

Each member of the Award Fee Evaluation Group will evaluate the
Contractor's performance against the performance standards of quality
and timeliness listed in Exhibit 1.

A numerical rating scale of 0 to 100 will be used. The scale is defined
as follows:

  ------------------------------------ -------------- ------------- -----------
  **Definition of Rating**             **Adjective    **Numerical   **Fee %**
                                       Rating**       Rating**      

  **[Outstanding]{.underline}**-       Outstanding    90 - 100      100%
  Contractor's performance exceeds                                  
  standards by substantial margin; the                              
  performance monitor can cite few                                  
  areas for improvement, all of which                               
  are minor.                                                        

  **[Exceeds                           Exceeds        80 -- 89      80%
  Expectations]{.underline}**-         Expectations                 
  Contractor's performance exceeds                                  
  standards, and although there may be                              
  several areas for improvement, these                              
  are more than offset by better                                    
  performance in other areas.                                       

  **[Satisfactory]{.underline}**-      Satisfactory   60 -- 79      40%
  Contractor's performance is                                       
  generally satisfactory, and areas                                 
  for improvement are approximately                                 
  offset by better performance in                                   
  other areas.                                                      

  **[Unacceptable]{.underline}**-      Unacceptable   Below 60      Base Fee
  Contractor's performance is less                                  Reduced by
  than standards by a substantial                                   50%
  margin, and the performance monitor                               
  can cite many areas for improvement                               
  which are not offset by better                                    
  performance in other areas.                                       
  ------------------------------------ -------------- ------------- -----------

Each member of the AFEG will give each performance factor a numerical
rating, and those ratings will be averaged. An average score of less
than 60 (Unacceptable) will result in a reduction in the base fee of 50%
for the performance factor for the rating period. An average score of
60-79 (Satisfactory) will result in award of 40% of the Award Fee for
the performance factor. An average score of 80-89 (Exceeds Expectations)
will result in award of 80% of the Award Fee, and an average of 90-100
(Outstanding) will result in award of 100% of the Award Fee for the
performance factor. **The Award Fee determinations are not subject to
the disputes clause.**

> **[SECTION F - PERIOD OF PERFORMANCE AND DELIVERY
> SCHEDULE]{.underline}**

**F.1 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)**

> This contract incorporates the following clause by reference, with the
> same force and effect as if they were given in full text. Upon
> request, the Contracting Officer will make their full text available.

**FEDERAL ACQUISITION REGULATION (FAR) (48 CFR CHAPTER 1) CLAUSES**

**FAR Clause No. Title and Date**

52.242-15 Stop Work Order (AUG 1989)

Alternate I (APRIL 1984)

**F.2 PERIOD OF PERFORMANCE**

> The Government anticipates the period of performance shall begin on or
> about September 27, 2006 and run through September 26, 2009 with one
> two-year option (if exercised) from September 27, 2009 through
> September 26, 2011.

**F.3 DELIVERY SCHEDULE**

> **Schedule of Deliverables and/or Reporting Requirements**
>
> The items specified for delivery below are subject to the review and
> approval of the Project Officer before final acceptance. The
> Contractor shall be required to make revisions deemed necessary by the
> Project Officer.
>
> The Contractor shall produce the following scheduled
> reports/deliverables in the amount, and within the time frame
> indicated. Deliverables shall be submitted to the Project Officer,
> Agency for Healthcare Research and Quality, 540 Gaither Road, Room
> 6361, Rockville, Maryland 20850.
>
> The Contractor shall submit the following items in accordance with the
> stated delivery schedule as noted below:

+------+---------------------------+-----------------+-----------------+
| **Ta | **Description**           | **Quantity**    | **Due Date**    |
| sk** |                           |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Kick-off Meeting          | 1               | Within 1 week   |
| 10.2 |                           |                 | of the          |
|      |                           |                 | effective date  |
|      |                           |                 | of the contract |
|      |                           |                 | (EDOC)          |
+------+---------------------------+-----------------+-----------------+
| 3    | Final Vision/Project      | 1 electronic    | Within 2 weeks  |
| .5.7 | Initiation Document       | copy to PO      | of EDOC         |
+------+---------------------------+-----------------+-----------------+
| 3.   | Project Work Plan,        | 1 electronic    | Draft: Within 3 |
| 5.8, | including                 | copy to PO      | weeks of EDOC   |
| 3.   | deliverables-oriented     |                 |                 |
| 5.9, | (systems and content)     |                 | Final: within 1 |
|      | work breakdown structure, |                 | week after      |
| 3.1  | an IMS and Critical Path  |                 | receipt of PO   |
| 0.6, |                           |                 | approval        |
| 3.   |                           |                 |                 |
| 10.7 |                           |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Name of Web site/Project  | Per             | Within 1 month  |
| 5.15 |                           | requirements of | EDOC            |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Finalize Project Expert   | 1 electronic    | Within 2 months |
| .1.2 | Panel and Editorial Board | copy to PO      | EDOC            |
|      | Nomination                |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Participate in activities | Per             | Ongoing between |
| .6.1 | oriented to transitioning | requirements of | EDOC and        |
| --   | into maintaining the      | SOW             | December 1,     |
| 3    | QualityTools database and |                 | 2006            |
| .6.5 | Web site                  |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Maintain QualityTools     | Per             | December 1,     |
| .6.6 | legacy database and Web   | requirements of | 2006 and        |
| --   | site: systems and content | SOW             | ongoing         |
| 3.   |                           |                 | thereafter      |
| 6.19 |                           |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Publish on a weekly basis | Per             | December 1,     |
| 6.15 | for a total of at least   | requirements of | 2006 until 18   |
|      | 150 new or updated tools  | SOW             | months EDOC     |
|      | between December 1, 2006  |                 |                 |
|      | and launch of the fully   |                 |                 |
|      | operational Innovations   |                 |                 |
|      | Clearinghouse Web site    |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Finalize Project Expert   | 1 electronic    | Within 3 months |
| 1.3, | Panel Membership and      | copy to PO      | EDOC            |
| 3    | Editorial Board           |                 |                 |
| .1.4 |                           |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Project Expert Panel      | Draft: 1        | Draft to PO, 21 |
| .1.6 | Meeting Agenda and        | electronic copy | calendar days   |
|      | Materials                 | to PO           | prior to        |
|      |                           |                 | meeting         |
|      |                           | Final: 1 copy   |                 |
|      |                           | to each meeting | Final to        |
|      |                           | participant     | participants 14 |
|      |                           |                 | calendar days   |
|      |                           |                 | prior to        |
|      |                           |                 | meeting         |
+------+---------------------------+-----------------+-----------------+
| 3.   | Expert Panel Meeting      | Per             | At least once   |
| 1.5, |                           | requirements of | every year of   |
| 3    |                           | SOW             | the contract    |
| .1.7 |                           |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Expert Panel Meeting      | Draft: 1        | Draft: 14 days  |
| .1.8 | Summary                   | electronic copy | after Meeting   |
|      |                           |                 |                 |
|      |                           | Final: 1        | Final: 7 days   |
|      |                           | electronic and  | after receiving |
|      |                           | three print     | PO comments     |
|      |                           | copies to PO    |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Inclusion Criteria        | 1 electronic    | Within 4 months |
| 2.1, |                           | copy to PO; 1   | EDOC            |
| 3    |                           | copy for each   |                 |
| .2.4 |                           | Expert Panel    |                 |
|      |                           | Member          |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | Electronic Log of Actions | 1 electronic    | Monthly with    |
| .2.8 | and Summary of Reasons    | copy to PO      | Progress Report |
|      | for Exclusion             |                 | beginning 6     |
|      |                           |                 | months EDOC     |
+------+---------------------------+-----------------+-----------------+
| 3    | Incentive and Reward      | 1 electronic    | Draft: within 2 |
| .3.1 | System                    | copy to PO      | months EDOC     |
|      |                           |                 |                 |
|      |                           |                 | Implemented     |
|      |                           |                 | within 3 months |
|      |                           |                 | EDOC            |
+------+---------------------------+-----------------+-----------------+
| 3    | *Federal Register*        | Per             | Within 1 month  |
| .3.2 | mailing                   | requirements of | of the posting  |
|      |                           | SOW             | in the *Federal |
|      |                           |                 | Register*       |
+------+---------------------------+-----------------+-----------------+
| 3    | Database of Innovators    | Per             | Implemented     |
| .3.5 | and Organizations That    | requirements of | within 4 months |
|      | Support Innovation        | SOW             | EDOC and then   |
|      |                           |                 | ongoing         |
+------+---------------------------+-----------------+-----------------+
| 3.4. | Template of Attributes    | 1 electronic    | Draft within 4  |
| 2.1- | and Definitions           | copy to PO      | months EDOC;    |
|      |                           |                 | final within 2  |
| 3.4  |                           |                 | weeks of        |
| .2.6 |                           |                 | receiving       |
|      |                           |                 | comments        |
+------+---------------------------+-----------------+-----------------+
| 3.   | On-line submission        | Per             | Within 1 month  |
| 3.6, | package                   | requirements of | of finalizing   |
| 3    |                           | SOW             | the inclusion   |
| .3.7 |                           |                 | criteria and    |
|      |                           |                 | database record |
|      |                           |                 | template        |
+------+---------------------------+-----------------+-----------------+
| 3    | Submit Information        | Per             | Within 1 week   |
| .3.8 | Collection Package for    | requirements of | after receiving |
|      | OMB Clearance             | SOW             | approval from   |
|      |                           |                 | PO of online    |
|      |                           |                 | submission form |
+------+---------------------------+-----------------+-----------------+
| 3    | Content Prioritization    | 1 electronic    | Within 6 months |
| .4.5 | Strategy                  | copy to PO      | of EDOC         |
+------+---------------------------+-----------------+-----------------+
| 3.   | Contractor-moderated      | Per             | Draft within 13 |
| 5.18 | mechanism for users to    | requirements of | months EDOC;    |
|      | post comments             | SOW             | implemented     |
|      |                           |                 | within 18       |
|      |                           |                 | months EDOC     |
+------+---------------------------+-----------------+-----------------+
| 3.5. | Stage I of Innovations    | Per             | Within 9 months |
| 16.1 | Web site deployment       | requirements of | EDOC            |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Usability testing         | Per             | Within 15       |
| 5.22 | completed                 | requirements of | months of EDOC  |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Usability testing results | Per             | Within 10 days  |
| 5.22 |                           | requirements of | of completing   |
|      |                           | SOW             | testing         |
+------+---------------------------+-----------------+-----------------+
| 3.5. | Stage II of Innovations   | Per             | Within 18       |
| 16.2 | Web site deployment       | requirements of | months EDOC     |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3    | 200 Innovations in        | Per             | Within 18       |
| .4.4 | database with associated  | requirements of | months EDOC,    |
|      | Lessons Learned for Web   | SOW             | then on-going   |
|      | site launch and then      |                 |                 |
|      | 250-300 every year        |                 |                 |
|      | thereafter                |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.   | Publish tool content on a | Per             | 19 months of    |
| 6.15 | weekly basis for a total  | requirements of | EDOC and yearly |
|      | of 250-300 tools per year | SOW             | thereafter      |
|      | in Innovations Web site   |                 |                 |
+------+---------------------------+-----------------+-----------------+
| 3.5. | Stage III of Innovations  | Per             | Within 26       |
| 16.3 | Web site deployment       | requirements of | months EDOC     |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3.7  | Technical Assistance      | Per             | December 1,     |
|      |                           | requirements of | 2006 for        |
|      |                           | SOW             | QualityTools;   |
|      |                           |                 | within 9 months |
|      |                           |                 | EDOC for        |
|      |                           |                 | Innovations     |
|      |                           |                 | Clearinghouse   |
|      |                           |                 | (stage I        |
|      |                           |                 | deployment)     |
+------+---------------------------+-----------------+-----------------+
| 3    | Electronic Mail Box for   | Per             | QualityTools:   |
| .7.6 | Web Site: Set up,         | requirements of | beginning       |
|      | maintain, timely          | SOW             | December 1,     |
|      | responses                 |                 | 2006;           |
|      |                           |                 | Innovations:    |
|      |                           |                 | On-going after  |
|      |                           |                 | Site activation |
+------+---------------------------+-----------------+-----------------+
| 3.8  | Learning and Networking   | Plan: 1         | Draft Plan:     |
|      | Opportunities             | electronic copy | within 13       |
|      |                           | to PO           | months of EDOC  |
|      |                           |                 |                 |
|      |                           | Implementation: | Final Plan:     |
|      |                           | per             | within 15       |
|      |                           | requirements of | months EDOC     |
|      |                           | SOW             |                 |
|      |                           |                 | Plan            |
|      |                           |                 | Implementation: |
|      |                           |                 | within 18       |
|      |                           |                 | months EDOC     |
|      |                           |                 | (coincident     |
|      |                           |                 | with stage II   |
|      |                           |                 | Web site        |
|      |                           |                 | deployment)     |
+------+---------------------------+-----------------+-----------------+
| 3.6  | Evaluating Web sites      | Per             | QualityTools:   |
| .19; |                           | requirements of | quarterly after |
| 3.9  |                           | SOW             | December 1,     |
|      |                           |                 | 2006            |
|      |                           |                 |                 |
|      |                           |                 | Innovations:    |
|      |                           |                 | quarterly after |
|      |                           |                 | code deployment |
|      |                           |                 | to live site    |
+------+---------------------------+-----------------+-----------------+
| 3    | Innovations Clearinghouse | Per             | Mock-up within  |
| .7.7 | Tutorial: Online and      | requirements of | 24 months EDOC; |
|      | CD-ROM                    | SOW             | final within 26 |
|      |                           |                 | months of EDOC  |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Systems Documentation     | Per             | Ongoing through |
| 0.12 |                           | requirements of | life of the     |
|      |                           | SOW             | contract in     |
|      |                           |                 | accordance with |
|      |                           |                 | SDLC,           |
|      |                           |                 | Configuration   |
|      |                           |                 | Management Plan |
|      |                           |                 | and Project     |
|      |                           |                 | Work Plan       |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Web Site Archive          | Per             | Weekly after    |
| 0.13 |                           | requirements of | initiation of   |
|      |                           | SOW             | Web Site        |
+------+---------------------------+-----------------+-----------------+
| 3.10 | Contract Progress         | 1 electronic    | 10 calendar     |
| .14, | Reports\*                 | copy to PO and  | days after the  |
| 3.1  |                           | 1 to CO         | end of the      |
| 0.15 |                           |                 | month beginning |
|      |                           |                 | 2^nd^ month of  |
|      |                           |                 | EDOC and every  |
|      |                           |                 | month           |
|      |                           |                 | thereafter      |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Management Meetings       | Per             | Monthly         |
| 0.16 |                           | Requirements of |                 |
|      |                           | SOW             |                 |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Management Meeting Action | 1 electronic    | Within 3        |
| 0.16 | Items                     | copy to PO      | business days   |
|      |                           |                 | after monthly   |
|      |                           |                 | meeting         |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Annual Report\*           | Outline and     | Outline: every  |
| 0.17 |                           | Draft: 1        | 11^th^ month of |
| thr  |                           | electronic copy | the yearly      |
| ough |                           | to PO           | cycle; Draft:   |
| 3.1  |                           |                 | every 12^th^    |
| 0.20 |                           | Final: 1        | month of the    |
|      |                           | electronic and  | yearly cycle;   |
|      |                           | 3 print copies  | Final: 2 weeks  |
|      |                           | to PO; 1        | after PO        |
|      |                           | electronic copy | approval of     |
|      |                           | and 1 print     | draft           |
|      |                           | copy to CO      |                 |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Close-out/Transition Plan | Draft: 1        | Draft: 4 months |
| 0.22 |                           | electronic copy | before contract |
|      |                           | to PO           | expires         |
|      |                           |                 |                 |
|      |                           | Final:          | Final: in Final |
|      |                           | integrated into | Report          |
|      |                           | and delivered   |                 |
|      |                           | via Final       |                 |
|      |                           | Report          |                 |
+------+---------------------------+-----------------+-----------------+
| 3.1  | Final Report\*            | Draft: 1        | Draft: within 2 |
| 0.21 |                           | electronic copy | months of       |
|      |                           | to PO           | contract        |
|      |                           |                 | expiration;     |
|      |                           | Final: 1        |                 |
|      |                           | electronic and  | Final: at the   |
|      |                           | 4 print copies  | end of the      |
|      |                           | to PO; 1        | final month of  |
|      |                           | electronic copy | the contract    |
|      |                           | and 1 print     |                 |
|      |                           | copy to CO      |                 |
+------+---------------------------+-----------------+-----------------+
| Sec  | Subcontracting Report for | Submission via  | April 30        |
| tion | Individual Conracts       | eSRS (DHHS)     | (annually)      |
|      | (SF-294)                  |                 |                 |
| L.13 |                           |                 | October 30      |
|      |                           |                 | (annually)      |
+------+---------------------------+-----------------+-----------------+
| Sec  | Summry Subconactor Report | Submission via  | April 30(       |
| tion | (SF 295)                  | eSrRS (DHHS)    | annually)       |
|      |                           |                 |                 |
| L.13 |                           |                 | June 30         |
|      |                           |                 | (annually)      |
+------+---------------------------+-----------------+-----------------+
| Sec  | Small Disadvantaged       | 1 hard copy     | Contract        |
| tion | Business Participation    |                 | Completion      |
|      | Report                    | 1 submission    |                 |
| L.12 |                           | via eSRS        |                 |
|      |                           | (DHHHS)         |                 |
+------+---------------------------+-----------------+-----------------+

\*In addition, one copy of the final work plan, the monthly progress
reports and annual final reports shall be submitted to the Contracting
Officer at the following address:

Agency for Healthcare Research and Quality

ATTN: Contracting Officer

Division of Contracts Management

540 Gaither Road

Rockville, Maryland 20850

**[SECTION G - CONTRACT ADMINISTRATION DATA]{.underline}**

**G.1 KEY PERSONNEL**

> Pursuant to the Key Personnel clause incorporated in Section I of this
> contract, the following individual(s) is/are considered to be
> essential to the work being performed hereunder:

[NAME]{.underline} [TITLE]{.underline}

**(TO BE COMPLETED AT TIME OF CONTRACT AWARD)**

> The clause cited above contains a requirement for review and approval
> by the Contracting Officer of written requests for a change of Key
> Personnel reasonably in advance of diverting any of these individuals
> from this contract. Receipt of written requests at least 30 days prior
> to a proposed change is considered reasonable.

**G.2 PROJECT OFFICER & ALTERNATE PROJECT OFFICER**

> Number range CHAPTERThe following Project Officer(s) and Task Order
> Officer(s) will represent the Government for the purpose of this
> contract:

**(TO BE COMPLETED AT TIME OF CONTRACT AWARD)**

> The Project Officer and Task Order Officer is/are responsible for: (1)
> monitoring the contractor\'s technical progress, including the
> surveillance and assessment of performance and recommending to the
> contracting officer changes in requirements; (2) interpreting the
> statement of work and any other technical performance requirements;
> (3) performing technical evaluation as required; (4) performing
> technical inspections and acceptances required by this contract; and
> (5) assisting in the resolution of technical problems encountered
> during performance.
>
> The Contracting Officer is the only person with authority to act as an
> agent of the Government under this contract. Only the Contracting
> Officer has authority to: (1) direct or negotiate any changes in the
> statement of work; (2) modify or extend the period of performance; (3)
> change the delivery schedule; (4) authorize reimbursement to the
> contractor of any costs incurred during the performance of this
> contract; or (5) otherwise change any terms and conditions of this
> contract.
>
> The Government may unilaterally change its Project Officer or Task
> Order Officer designation

**G.3 INVOICE SUBMISSION**

a\. INVOICE SUBMISSION

> Billing Instructions are attached and made part of this contract.
> Instructions and the following directions for the submission of
> invoices must be followed to meet the requirements of a \"proper\"
> payment request pursuant to FAR 32.9, and must be in accordance with
> the General Provisions clause 52.232-25 Prompt Payment (OCT 2003).

Invoices/financing requests shall be submitted in an original and three
copies to:

Contracting Officer

Agency for Healthcare Research and Quality

Division of Contracts Management

540 Gaither Road

Rockville, Maryland 20850

**G.4 INFORMATION ON VOUCHERS**

\(1\) The Contractor IS REQUIRED to include the following minimum
information on vouchers:

\(a\) Contractor's name and invoice date;

\(b\) Contract Number;

\(c\) Description and price of services actually rendered;

\(d\) Other substantiating documentation or information as required by
the contract;

\(e\) Name (where practicable), title, phone number, and complete
mailing address or responsible official to whom payment is to be sent;
and

\(f\) The Internal Revenue Service Taxpayer Identification Number.

\(2\) The Contractor shall furnish the following [minimum]{.underline}
information in support of costs submitted:

\(a\) [Direct Labor]{.underline} - include all persons, listing the
person's name, title, number of hours or days worked, hourly rate
(unburdened) the total cost per person and a total amount of this
category;

\(b\) [Fringe Costs]{.underline} - show rate, base and total amount as
well as verification/allowability or rate changes (when applicable);

\(c\) [Overhead or Indirect Costs]{.underline} - show rate, base and
total amount as well as verification/allowability or rate changes (when
applicable);

\(d\) [Consultants]{.underline} - include the name, number of days or
hours worked, a total amount per consultant and a total amount for this
category;

\(e\) [Travel]{.underline} - include for each airplane or train trip
taken the name of the traveler, date of travel, destination, the
transportation costs including ground transportation, shown separately,
and per diem costs. Other travel costs shall also be listed. A total
amount for this category shall be provided;

\(f\) [Subcontractors]{.underline} - include for each subcontractor, the
same data and level of detail that is being provided for the prime
contractor. A total number for this category shall be provided.

\(g\) [Data Processing]{.underline} - include all non-labor costs, i.e.,
computer time, equipment purchase, lease or rental, data tapes, etc. A
total amount for this category shall be provided.

\(h\) [Other]{.underline} - include a listing of all other direct
charges to the contract, i.e., office supplies, telephone, equipment
rental, duplication, etc.

\(i\) [Equipment Cost]{.underline} - itemize and identify separately
from material costs including reference to approval in all cases;

\(j\) [G&A]{.underline} - show rate, base and total as well as
verification/allowability of rate changes (when applicable); and

(k) [Fee]{.underline} - show rate, base and total;

(l) [Current amount billed]{.underline} by individual cost element and
    total dollar amount; and

(m) [Cumulative amount billed]{.underline} by individual cost element
    and total dollar amount.

\(3\) Payment shall be made by:

PSC Finance

Parklawn Building, Room 16-23

5600 Fishers Lane

Rockville, Maryland 20857

Telephone Number (301) 443-6766

**G.5 INDIRECT COST RATES and FEE**

> In accordance with Federal Acquisition Regulation (FAR) (48 CFR
> Chapter 1) Clause 52.216-7, [Allowable Cost and Payment]{.underline},
> incorporated by reference in this contract, in Part II, Section I, the
> primary contact point responsible for negotiating provisional and/or
> final indirect cost rates is the cognizant contracting official as set
> forth in FAR Subpart 42.7 - Indirect Cost Rates.
>
> Reimbursement will be limited to the rates and time periods covered by
> the negotiated agreements. The rates, if negotiated, are hereby
> incorporated without further action of the contracting officer.

**G.6 ELECTRONIC FUNDS TRANSFER**

> Pursuant to FAR 52.232-33, Payment by Electronic Funds Transfer -
> Central Contractor Registration (OCT 2003), the Contractor shall
> designate a financial institution for receipt of electronic funds
> transfer payments. This designation shall be submitted, in writing, to
> the finance office designated in the contract.

**SECTION H - SPECIAL CONTRACT REQUIREMENTS**

**H.1 RELEASE AND USE AND COPYRIGHT OF DATA FIRST PRODUCED FROM WORK
PERFORMED UNDER THIS CONTRACT**

> \(a\) *Release and Use -- Data* *first produced in the performance of
> the Contract.* As permitted in FAR 52.227-17, the provisions of this
> Section H.1 shall apply to any release or use of data first produced
> in the performance of the Contract and any analysis, tools,
> methodologies, or recorded product based on such data.
>
> \(b\) *Release and Use -- Requirements related to confidentiality and
> quality.* To ensure public trust in the confidentiality protections
> afforded participants in Agency for Healthcare Research and Quality
> (AHRQ)-supported research, AHRQ requires and monitors compliance by
> its contractors with section 934(c) of the Public Health Service Act
> (PHS Act) (42 U.S.C. 299c-3(c)), which states in part that
>
> No information, if the establishment or person supplying the
> information or described in it is identifiable, obtained in the course
> of activities undertaken or supported under this title, may be used
> for any purpose other than the purpose for which it was supplied
> unless such establishment or person has consented\...to its use for
> such other purpose. Such information may not be published or released
> in other form if the person who supplied the information or who is
> described in it is identifiable unless such person has consented\...to
> its publication or release in other form.
>
> In addition to this requirement, section 933(b)(1) of the PHS Act (42
> U.S.C. 299c-2(b)(1)) requires AHRQ to assure that statistics and
> analyses developed with Agency support are of high quality,
> comprehensive, timely, and adequately analyzed. Accordingly \--
>
> \(1\) prior to the release or use of data based upon work performed
> under this Contract, the Contractor agrees to consult with the Project
> and Contract Officers regarding the proposed release or use. The
> Contractor will in good faith consider, discuss, and respond to any
> comments or suggested modifications that are provided by AHRQ within
> two months of receiving the proposed release or use.
>
> The purpose of such consultation is to assure that:
>
> \(A\) identifiable information is being used exclusively for the
> purpose(s) for which it was supplied or appropriate consents have been
> obtained;
>
> \(B\) the confidentiality promised to individuals and establishments
> supplying identifiable information or described in it is not violated;
> and
>
> \(C\) the quality of statistical and analytical work meets the
> statutory standards cited above.
>
> \(2\) The Contractor must satisfy conditions (1)(A) and (1)(B). At the
> conclusion of any consultation required by paragraph (b)(1) above, if
> AHRQ and the Contractor cannot agree that a proposed use or release
> satisfies condition (1)(C) above:

\(A\) the research professional at the Contractor responsible for the
quality of the Contract work will, in advance of any release or use of
such data, certify in a letter to the Contracting Officer what
differences of opinion cannot be resolved regarding the statutory
standards referenced in condition (1)(C) and the basis for Contractor
assertions that these standards have been met; and

\(B\) the Contractor must print prominently on the release or other
product, or on any portion that is released, or state prior to any oral
presentation or release of such material, the following disclaimer:

> THIS presentation/ publication/or other product is derived from work
> supported under a contract with THE AGENCY FOR HEALTHCARE RESEARCH AND
> QUALITY (AHRQ) (# ). However, this presentation/ publication/or other
> product HAS NOT BEEN APPROVED by the agency.
>
> \(c\) *Required Statement Regarding Protected Information*. On all
> written material or other recorded products, or preceding any
> presentation or other oral disclosure, release or use of material
> based on identifiable information obtained in the course of work
> performed under this contract, the Contractor shall make the following
> statement:
>
> IDENTIFIABLE INFORMATION ON WHICH THIS REPORT, PRESENTATION, OR OTHER
> FORM OF DISCLOSURE IS BASED IS PROTECTED BY FEDERAL LAW, SECTION
> 934(c) OF THE PUBLIC HEALTH SERVICE ACT, 42 U.S.C. 299c-3(c). NO
> IDENTIFIABLE INFORMATION ABOUT ANY INDIVIDUALS OR ENTITIES SUPPLYING
> THE INFORMATION OR DESCRIBED IN IT MAY BE KNOWINGLY USED EXCEPT IN
> ACCORDANCE WITH THEIR PRIOR CONSENT. ANY CONFIDENTIAL IDENTIFIABLE
> INFORMATION IN THIS REPORT OR PRESENTATION THAT IS KNOWINGLY DISCLOSED
> IS DISCLOSED SOLELY FOR THE PURPOSE FOR WHICH IT WAS PROVIDED.
>
> \(d\) *Copyright -- Data first produced in the performance of the
> Contract.* Subject to the terms of this Section regarding release and
> use of data, AHRQ, through its Contracting Officer, will grant
> permission under FAR 52.227-17(c)(1)(i) to the Contractor to establish
> claim to copyright subsisting in scientific and technical articles
> based on or containing data first produced in the performance of this
> contract that are submitted for publication in academic, technical or
> professional journals, symposia proceedings or similar works. When
> claim to copyright is made, the Contractor shall affix the applicable
> copyright notice of 17 U.S.C. 401 or 402 and acknowledgment of
> Government sponsorship (including contract number) to the data when
> such data are delivered to the Government, as well as when the data
> are published or deposited for registration as a published work in the
> U.S. Copyright Office. In such circumstances, the Contractor hereby
> agrees to grant to AHRQ, and others acting on its behalf, a paid-up,
> nonexclusive, irrevocable, worldwide license for all such data to
> reproduce, prepare derivative works, distribute copies to the public,
> and perform publicly and display publicly, by or on behalf of AHRQ. A
> description of this license will be incorporated into the copyright
> notices required above.
>
> \(e\) *Subcontracts.* Whenever data, analyses, or other recorded
> products are to be developed by a subcontractor under this Contract,
> the Contractor must include the terms of H.1 in the subcontract,
> without substantive alteration, with a provision that the
> subcontractor may not further assign to another party any of its
> obligations to the Contractor. No clause may be included to diminish
> the Government's stated requirements or rights regarding release or
> use of products or materials based on data derived from work performed
> under this contract.

**H.2 LACK OF COMPLIANCE WITH REQUIREMENTS FOR RELEASE OR USE**

> Failure to submit materials for statutorily mandated confidentiality
> and statistical and analytic quality reviews as required by Section
> H.1 of this contract will be viewed as a material violation and breach
> of the terms of this contract, as the requirements of this provision
> are necessary for AHRQ to carry out its statutory obligations and
> responsibilities. Such violations, as well as other violations, of the
> contract terms, which are deemed serious, could result in the
> initiation of debarment proceedings in accordance with the Federal
> Acquisition Regulations and the Department of Health and Human
> Services implementing regulations. Records of the Contractor\'s
> performance, including the Contractor\'s performance pertaining to
> this Contract, will be maintained in AHRQ\'s Contracts Management
> Office and will be considered as an element of past performance which
> is part of all subsequent competitive contract proposal reviews.

**H.3 SUBCONTRACTS**

> Award of any subcontract is subject to the prior written approval of
> the Contracting Officer upon review of the supporting documentation.
> Failure to obtain prior written approval of the Contracting Officer
> may result in disallowance of use of Federal funds to cover services
> under the subcontract. The contractor must include in any subcontracts
> executed or used to provide the support specified in this contract the
> terms of requirements H.1 and H.2. These requirements are to be
> included without substantive alteration, and no clause may be included
> to diminish these requirements. If approved, a copy of the signed
> subcontract shall be provided to the Contracting Officer.

**H.4 LATE PAYMENTS TO THE GOVERNMENT**

> Late payment of debts owed the Government by the Contractor, arising
> from whatever cause, under this contract/order shall bear interest at
> a rate or rates to be established in accordance with the Treasury
> Fiscal Requirements Manual. For purposes of this provision, late
> payments are defined as payments received by the Government more than
> 30 days after the Contractor has been notified in writing by the
> Contracting Officer of:

a\. The basis of indebtedness.

b\. The amount due.

c\. The fact that interest will be applied if payment is not received
within 30 days from the date of mailing of the notice.

d\. The approximate interest rate that will be charged.

**H.5 PRIVACY ACT**

> The Privacy Act clauses cited in Section I (FAR 52.224-1 and 52.224-2)
> are applicable to the consultant records kept by the Contractor for
> the Agency for Healthcare Research and Quality.
>
> You are hereby notified that the Contractor and its employees are
> subject to criminal penalties for violations of the Act (5 U.S.C.
> 552a(i)) to the same extent as employees of the Department. The
> Contractor shall assure that each Contractor employee is aware that
> he/she can be subjected to criminal penalties for violations of the
> Act. Disposition instructions: Records are to be destroyed after
> contract closeout is completed and final payment is made and in
> accordance with IRS regulations.

**H.6 PRO-CHILDREN ACT of 1994**

> The Pro-Children Act of 1994, P.L. 103-227, imposes restrictions on
> smoking where certain federally funded children's' services are
> provided. P.L. 103-227 states in pertinent part:
>
> PHS strongly encourages all grant and contract recipients to provide a
> smoke-free workplace and to promote the non-use of all tobacco
> products. In addition, P.L. 103-227, the Pro-Children Act of 1994,
> prohibits smoking in certain facilities (or in some cases, any portion
> of a facility) in which regular or routine education, library, day
> care, health care or early childhood development services are provided
> to children."

**H.7** **SALARY CAP GUIDE NOTICE**

> Number range CHAPTERPursuant to P.L. 109-149, no Fiscal Year 2006
> (October 1, 2005 - September 30, 2006) funds may be used to pay the
> direct salary of an individual through this contract at a rate in
> excess of the direct salary rate for Executive Level I of the Federal
> Executive Pay Scale. That rate is \$183,500 per year for the period of
> January 1, 2006 through December 31, 2006. Direct salary is exclusive
> of overhead, fringe benefits, and general and administrative expenses.
> The salary limit also applies to individuals proposed under
> subcontracts. If this is a multi-year contract, it may be subject to
> unilateral modifications by the Government if any salary rate ceilings
> are established in future DHHS appropriation acts. P.L. 109-149 states
> in pertinent part:
>
> None of the funds appropriated in this Act for the National Institutes
> of Health, the Agency for Healthcare Research and Quality, and the
> Substance Abuse and Mental Health Services Administration shall be
> used to pay the salary of an individual, through a grant or other
> extramural mechanism, at a rate in excess of Executive Level I.
>
> Contractors shall absorb that portion of an employee's salary (plus
> the dollar amount for fringe benefits and indirect costs associated
> with the excess) that exceeds a rate in excess of Executive Level I

**H.8** **PERSONNEL SECURITY REQUIREMENTS (Information only, not
required for contractor employees not working in DHHS owned or leased
space)**

Number range CHAPTER**BACKGROUND**

> The Office of Assistant Secretary for Management and Budget,
> Department of Health and Human Services (DHHS), requires that all DHHS
> employees and contractor employees (including subcontractors) who will
> be working in a DHHS-owned or leased space and/or who will have access
> to DHHS equipment, and non-public privileged, proprietary, or trade
> secret information, undergo a background investigation.

**GENERAL**

> Notwithstanding other submission requirements stated elsewhere in this
> contract, the contractor shall appoint and identify a Contractor
> Security Representative and submit the following information for each
> employee to the Contracting Officer within thirty (30) calendar days
> after contract award.

SF-85 Questionnaire for Non-Sensitive Positions

HHS Credit Release

OF-306 Declaration for Federal Employment Current resume

*Note: Forms are available at:
[[http://www.gsa.gov/Portal/formslibrary.jsp]{.underline}](http://www.gsa.gov/Portal/formslibrary.jsp)*

> Within thirty (30) days after contract award each employee will be
> required to have electronic fingerprinting performed ----
> Fingerprinting services are available by appointment only through the
> Program Support Staff (PSC) and will be arranged by AHRQ.

**Number range CHAPTER(05/06-DCM)**

**(FAC 2005-09)**

**PART II - CONTRACT CLAUSES**

**SECTION I**

**CONTRACT CLAUSES**

**GENERAL CLAUSES FOR A COST-PLUS-A-FIXED-FEE CONTRACT**

CLAUSES INCORPORATED BY REFERENCE (FEBRUARY 1998)

This contract incorporates the following clauses by reference, with the
same force and effect as if they were given in full text. Upon request,
the Contracting Officer will make their full

text available. Also, the full text of a clause may be accessed
electronically at this address: <http://www.arnet.gov/far/> .

I. FEDERAL ACQUISITION REGULATION (FAR) (48 CFR CHAPTER 1)

CLAUSES

FAR Clause No. Title and Date

52.203-3 Gratuities (APR 1984)

52.203-5 Covenant Against Contingent Fee (APR 1984)

52.203-6 Restrictions on Subcontractor Sales to the Government

> (Jul 1995)

52.203-7 Anti-Kickback Procedures (JUL 1995)

52.203-8 Cancellation, Rescission, and Recovery of Funds for Illegal

or Improper Activity (JAN 1997)

52.203-10 Price or Fee Adjustment for Illegal or Improper Activity (JAN
1997)

52.203-12 Limitation on Payments to Influence Certain Federal
Transactions (SEP 2005)

52.204-4 Printing or Copying Double-Sided on Recycled Paper (AUG 2000)

52.204-7 Central Contractor Registration. (OCT 2003)

52.209-6 Protecting the Government\'s Interest When Subcontracting With
Contractors Debarred, Suspended, or Proposed for Debarment (JAN 2005)

52.215-14 Integrity of Unit Prices (OCT 1997) Alternate I

> (OCT 1997) (when contracting without full and open competition)

52.215-17 Wavier of Facilities Capital Cost of Money (OCT 1997)

52.216-18 Ordering (OCT 1995)

52.216-19 Ordering Limitations (OCT 1995)

52.216-20 Definite Quantity (OCT 1995)

52.216-21 Requirements (OCT 1995)

52.216-22 Indefinite Quantity (OCT 1995)

52.215-2 Audit and Records - Negotiation (JUN 1999)

52.215-8 Order of Precedence-Uniform Contract Format (Oct 1997)

52.215-10 Price Reduction for Defective Cost or Pricing Data (OCT 1997)

> (applicable to contract actions over \$550,000)

52.215-12 Subcontractor Cost or Pricing Data (OCT 1997)

> (applicable to contract actions over \$550,000)

52.215-15 Pension Adjustments and Asset Reversions (OCT 2004)

52.215-18 Reversion or Adjustment of Plans for Postretirement Benefits
(PRB) Other Than Pensions (JUL 2005)

52.215-19 Notification of Ownership Changes (OCT 1997)

52.216-7 Allowable Cost and Payment (DEC 2002)

52.216-8 Fixed Fee (MAR 1997)

52.217-8 Option to Extend Services (NOV 1999)

52.217-9 Option to Extend the Term of the Contract (MAR 2000)

52.219-4 Notice of Price Evaluation Preference for HUBZone Small
Business Concerns (JUL 2005)

52.219-8 Utilization of Small Business Concerns (MAY 2004)

52.219-9 Small Business Subcontracting Plan (JULY 2005)

> (Applicable to contracts over \$500,000)

52.219-16 Liquidated Damages - Subcontracting Plan (JAN 1999)

52.219-25 Small Disadvantaged Business Participation Program -
Disadvantaged Status and Reporting ( OCT 1999)

52.222-2 Payment for Overtime Premiums (JUL 1990). The amount in

> paragraph (a) is \"zero\" unless different amount is separately
>
> stated elsewhere in contract.

52.222-3 Convict Labor (JUNE 2003)

52.222-26 Equal Opportunity (APR 2002)

52.222-35 Equal Opportunity for Special Disabled Veterans, Veterans of
the Vietnam Era, and Other Eligible Veterans. (DEC 2001)

52.222-36 Affirmative Action for Workers With Disabilities (JUNE 1998)

52.222-37 Employment Reports on Special Disabled Veterans, Veterans of
the Vietnam Era, and Other Eligible Veterans. (DEC 2001)

52.222-39 Notification of Employee Rights Concerning Payment of Union
Dues or Fees (DEC 2004)

52.223-6 Drug Free Workplace (MAY 2001)

52.223-14 Toxic Chemical Release Reporting (AUG 2003)

52.224-1 Privacy Act Notification (APR 1984)

52.224-2 Privacy Act (APR 1984)

52.225-1 Buy American Act - Supplies (JUNE 2003)

52.225-13 Restrictions on Certain Foreign Purchases (FEB 2006)

52.227-1 Authorization and Consent (JULY 1995)

52.227-2 Notice and Assistance Regarding Patent and Copy-

> Right Infringement (AUG 1996)

52.227-3 Patent Indemnity (APRIL 1984)

52.227-17 Rights in Data -- Special Works (JUNE 1987)

52.228-7 Insurance-Liability to Third Persons (MAR 1996)

52.230-2 Cost Accounting Standards (APR 1998)

52.230-3 Disclosure and Consistency of Cost Accounting Practices (APR
1998)

52.230-6 Administration of Cost Accounting Standards (APR 2005)

52.232-9 Limitation on Withholding of Payments (APRIL 1984)

52.232-17 Interest (JUNE 1996)

52.232-18 Availability of Funds (APR 1984)

52.232-20 Limitation of Cost (APR 1984)

52.232-22 Limitation of Funds (APR 1984)

52.232-23 Assignment of Claims (JAN 1986)

52.232-25 Prompt Payment (OCT 2003)

52.232-33 Payment by Electronic Funds Transfer Central Contractor
Registration (Oct 2003)

52.233-1 Disputes (JULY 2002)

52.233-3 Protest After Award (AUG 1996) Alternate I (JUNE 1985)

52.233-4 Applicable Law for Breach of Contract Claim (OCT 2004)

52.237-10 Identification of Uncompensated Overtime (Oct 1997)

52.239-1 Privacy or Security Safeguards (AUG 1996)

52.242-1 Notice of Intent to Disallow Costs (APRIL 1984)

52.242-3 Penalties for Unallowable Costs (MAY 2001)

52.242-4 Certification of Final Indirect Costs (Jan 1997)

52.242-13 Bankruptcy (JULY 1995)

52.243-2 Changes - Cost Reimbursement (AUG 1987) - Alternate II

> (APRIL 1984)

52.244-2 Subcontracts (AUGUST 1998)

52.244-5 Competition in Subcontracting (DEC 1996)

52.245-5 Government Property (Cost Reimbursement, Time-and-

> Material, or Labor-Hour Contract (MAY 2004)

52.246-5 Inspection of Services-Cost Reimbursement (APRIL 1984)

52.246-23 Limitation of Liability-(FEB 1997)

52.248-1 Value Engineering (FEB 2000)

52.249-6 Termination (Cost-Reimbursement) (MAY 2004)

52.249-14 Excusable Delays (APRIL 1984)

52.251-1 Government Supply Sources (APRIL 1984)

52.253-1 Computer Generated Forms (JAN 1991)

**II. DEPARTMENT OF HEALTH AND HUMAN SERVICES ACQUISITION**

**REGULATION (HHSAR) (48 CFR CHAPTER 3) CLAUSES**

HHSAR

[Clause No.]{.underline} [Title and Date]{.underline}

352.202-1 Definitions (JAN 2001)

> Alternate h

352.223-70 Safety and Health (JAN 2001)

352.232-9 Withholding of Contract Payments (APRIL 1984)

352.233-70 Litigation and Claims (APR 1984)

352.242-71 Final Decisions on Audit Findings (APRIL 1984)

352.270-1 Accessibility of Meetings, Conferences, and Seminars to

> Persons With Disabilities (JAN 2001)

352.270-5 Key Personnel (APRIL 1984)

352.270-6 Publication and Publicity (JUL 1991)

352.270-7 Paperwork Reduction Act (JAN 2001)

352.270-8 Protection of Human Subjects (JAN 2001)

**[Number range CHAPTERThe following clauses are applicable to this
contract and are provided in full text:]{.underline}**

.

**OPTION TO EXTEND THE TERM OF THE CONTRACT (MAR 2000) (FAR 52.217-9)**

\(a\) The Government may extend the term of this contract by written
notice to the Contractor within 30 days of the expiration date of the
contract; provided that the Government gives the Contractor a
preliminary written notice of its intent to extend at least 60 days
before the contract expires. The preliminary notice does not commit the
Government to an extension.

\(b\) If the Government exercises this option, the extended contract
shall be considered to include this option clause.

\(c\) The total duration of this contract, including the exercise of any
options under this clause, shall not exceed 60 months.

(End of clause)

**KEY PERSONNEL (APR 1984) (HHSAR 352.270-5)**

The personnel specified in this contract are considered to be essential
to the work being performed hereunder. Prior to diverting any of the
specified individuals to other programs, the Contractor shall notify the
Contracting Officer reasonably in advance and shall submit justification
(including proposed substitutions) in sufficient detail to permit
evaluation of the impact on the program. No diversion shall be made by
the Contractor without the written consent of the Contracting Officer;
provided, that the Contracting Officer may ratify in writing such
diversion and such ratification shall constitute the consent of the
Contracting Officer required by this clause. The contract may be amended
from time to time during the course of the contract to either add or
delete personnel, as appropriate.

(End of clause)

**PART III- LIST OF DOCUMENTS, EXHIBITS AND ATTACHMENTS**

**SECTION J - LIST OF ATTACHMENTS**

**[Attachments]{.underline}**

1\. Statement of Work 70 - 107

with Appendices

A. Glossary of Key Terms Used in the Innovations Clearinghouse RFP 2 pgs

B. Innovations Clearinghouse Expert Panel Meeting 20 pgs

C. AHRQ Web Product and web Site Development Guidelines 6 pgs

D Guidelines for Developing AHRQ Web-based Tools 6 pgs

E DRAFT AHRQ Technical Reference Model, Version 0.1

January 24, 2006 19 pgs

2\. Past Performance Questionnaire and Contractor Performance Form 5 pgs

3\. Proposal Intent Response Sheet 1 pg

4\. Breakdown of Proposed Estimated Cost and Labor Hours 1 pg

**[Exhibit 1]{.underline}** Performance Requirements Summary (to be
provided as amendment to the RFP)

**NOTE: ALL ATTACHMENTS and THE EXHIBIT ARE LOCATED AT THE END OF THIS
REQUEST FOR PROPOSAL**.

**Number range CHAPTERNumber range CHAPTER (FAC 2005-09)**

**PART IV. REPRESENTATIONS AND INSTRUCTIONS**

**SECTION K**

**REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS**

K.1 HHSAR 315.204-5 Representations and Instructions

K.2. FAR 52.204-8 Annual Representations and Certifications (JAN 2005)

K.3. FAR 52.222-21 Prohibition of Segregated Facilities (FEB 1999)

K.4. FAR 52.230-1 Cost Accounting Standards Notices and Certification
(JUNE 2000)

K.5. FAR 15.406-2 Certificate of Current Cost and Pricing Data

K.6. P.L. 103-227 Certification Regarding Environmental

> Tobacco Smoke

K.l REPRESENTATIONS AND INSTRUCTIONS

\(a\) Section K, Representations, certifications, and other statements
of offerors.

\(1\) This section shall begin with the following and continue with the
applicable representations and certifications:

TO BE COMPLETED BY THE OFFEROR: (The Representations and Certifications
must be executed by an individual authorized to bind the Offeror.) The
Offeror makes the following Representations and Certifications as part
of its proposal. (Check or complete all appropriate boxes or blanks on
the following pages.)

(Name of Offeror) (RFP No.)

(Signature of Authorized Individual) (Date)

(Typed Name of Authorized Individual)

NOTE: The penalty for making false statements in offers is prescribed in
18 U.S.C. 1001.

K.2. ANNUAL REPRESENTATIONS AND CERTIFICATIONS (JAN 2005) (FAR 52.204-8)

(a)(1) If the clause at 52.204-7, Central Contractor Registration, is
included in this solicitation, paragraph (b) of this provision applies.

\(2\) If the clause at 52.204-7 is not included in this solicitation,
and the offeror is currently registered in CCR, and has completed the
ORCA electronically, the offeror may choose to use paragraph (b) instead
of completing the corresponding individual representations and
certifications in the solicitation. The offeror shall indicate which
option applies by checking one of the following boxes:

\[ \] (i) Paragraph (b) applies

> \[ \] (ii) Paragraph (b) does not apply and the offeror has completed
> the individual representations and certification in the solicitation.

\(b\) The offeror has completed the annual representations and
certifications electronically via the Online Representations and
Certifications Application (ORCA) website at
[[http://orca/bpn.gov.]{.underline}](http://orca/bpn.gov.) After
reviewing the ORCA database information, the offeror verifies by
submission of the offer that the representations and certifications
currently posted electronically have been entered or updated within the
last 12 months, are current, accurate, complete, and applicable to this
solicitation (including the business size standard applicable to the
NAICS code referenced for this solicitation), as of the date of this
offer and are incorporated in this offer by reference (see FAR 4.1201);
except for the changes identified below (offeror to insert changes,
identifying change by clause number, title, date). These amended
representation(s) and/or certification(s) are also incorporated in this
offer and are current, accurate, and complete as of the date of this
offer.

FAR Clause# Title Date Change

Any changes provided by the offeror are applicable to this solicitation
only, and do not result in an update to the representations and
certifications posted on ORCA.

(End of provision)

K.3. PROHIBITION OF SEGREGATED FACILITIES

(FEB 1999) (FAR 52.222-21)

\(a\) \"Segregated facilities,\" as used in this clause, means any
waiting rooms, work areas, rest rooms and wash rooms, restaurants and
other eating areas, time clocks, locker rooms and other storage or
dressing areas, parking lots, drinking fountains, recreation or
entertainment areas, transportation, and housing facilities provided for
employees, that are segregated by explicit directive or are in fact
segregated on the basis of race, color, religion, or national origin
because of written or oral policies or employee custom. The term does
not include separate or single-user rest rooms or necessary dressing or
sleeping areas provided to assure privacy between the sexes.

\(b\) The Contractor agrees that it does not and will not maintain or
provide for its employees any segregated facilities at any of its
establishments, and that it does not and will not permit its employees
to perform their services at any location under its control where
segregated facilities are maintained. The Contractor agrees that a
breach of this clause is a violation of the Equal Opportunity clause in
this contract.

\(c\) The Contractor shall include this clause in every subcontract and
purchase order that is subject to the Equal Opportunity clause of this
contract.

(End of Clause)

K.4. COST ACCOUNTING STANDARDS NOTICES AND

CERTIFICATION

(FAR 52.230-1) (JUNE 2000)

NOTE: This notice does not apply to small businesses or foreign
governments. This notice is in three parts, identified by Roman numerals
I through III.

> Offerors shall examine each part and provide the requested information
> in order to determine Cost Accounting Standards (CAS) requirements
> applicable to any resultant contract.
>
> If the offeror is an educational institution, Part II does not apply
> unless the contemplated contract will be subject to full or modified
> CAS-coverage pursuant to 48CFR 9903.201-2(c)(5) or
> 9903.201-2(c)(6),respectively.

I. Disclosure Statement - Cost Accounting Practices and Certification

\(a\) Any contract in excess of \$500,000 resulting from this
solicitation, will be subject to the requirements of the Cost Accounting
Standards Board (48 CFR, Chapter 99), except for those contracts which
are exempt as specified in 48 CFR 9903.201-1.

\(b\) Any offeror submitting a proposal which, if accepted, will result
in a contract subject to the requirements of 48 CFR Chapter 99 must, as
a condition of contracting, submit a Disclosure Statement as required by
48 CFR 9903.202. When required, the Disclosure Statement must be
submitted as a part of the offeror\'s proposal under this solicitation
unless the offeror has already submitted a Disclosure Statement
disclosing the practices used in connection with the pricing of this
proposal. If an applicable Disclosure Statement has already been
submitted, the offeror may satisfy the requirement for submission by
providing the information requested in paragraph (c) of Part I of this
provision. Caution: In the absence of specific regulations or agreement,
a practice disclosed in a Disclosure Statement shall not, by virtue of
such disclosure, be deemed to be a proper, approved, or agreed-to
practice for pricing proposals or accumulating and reporting contract
performance cost data.

\(c\) Check the appropriate box below:

\[ \] (1) Certificate of Concurrent Submission of Disclosure Statement.

> The offeror hereby certifies that, as a part of the offer, copies of
> the Disclosure Statement have been submitted as follows: (i) original
> and one copy to the cognizant Administrative Contracting Officer (ACO)
> or cognizant Federal agency official authorized to act in that
> capacity, as applicable, and (ii) one copy to the cognizant Federal
> auditor.

(Disclosure must be on Form No. CASB DS-1 or CASB

> DS-2, as applicable. Forms may be obtained from the cognizant ACO or
> Federal official and/or from the loose-leaf version of the Federal
> Acquisition Regulation.)

Date of Disclosure
Statement:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Name and Address of Cognizant

ACO or Federal official where filed:

> The offeror further certifies that practices used in estimating costs
> in pricing this proposal are consistent with the cost accounting
> practices disclosed in the Disclosure Statement.

\[ \] (2) Certificate of Previously Submitted Disclosure Statement.

> The offeror hereby certifies that the required Disclosure Statement
> was filed as follows:
>
> Date of Disclosure
> Statement:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Name and Address of Cognizant

ACO or Federal official where filed:

> The offeror further certifies that the practices used in estimating
> costs in pricing this proposal are consistent with the cost accounting
> practices disclosed in the applicable Disclosure Statement.
>
> \[ \] (3) Certificate of Monetary Exemption.
>
> The offeror hereby certifies that the offeror together with all
> divisions, subsidiaries, and affiliates under common control, did not
> receive net awards of negotiated prime contracts and subcontracts
> subject to CAS totaling more than \$25 million in the cost accounting
> period immediately preceding the period in which this proposal was
> submitted. The offeror further certifies that if such status changes
> before an award resulting from this proposal, the offeror will advise
> the Contracting Officer immediately.
>
> \[ \] (4) Certificate of Interim Exemption.
>
> The offeror hereby certifies that (i) the offeror first exceeded the
> monetary exemption for disclosure, as defined in (3) of this
> subsection, in the cost accounting period immediately preceding the
> period in which this offer was submitted and (ii) in accordance with
> 48 CFR, Subpart 9903.202-1, the offeror is not yet required to submit
> a Disclosure Statement. The offeror further certifies that if an award
> resulting from this proposal has not been made within 90 days after
> the end of that period, the offeror will immediately submit a review
> certificate to the Contracting Officer, in the form specified under
> subparagraph (c)(1) or (c)(2) of Part I of this provision, as
> appropriate, to verify submission of a completed Disclosure Statement.
>
> Caution: Offerors currently required to disclose because they were
> awarded a CAS-covered prime contract or subcontract of \$25 million or
> more in the current cost accounting period may not claim this
> exemption (4). Further, the exemption applies only in connection with
> proposals submitted before expiration of the 90-day period following
> the cost accounting period in which the monetary exemption was
> exceeded.

II\. Cost Accounting Standards - Eligibility for Modified Contract
Coverage

> If the offeror is eligible to use the modified provisions of 48 CFR,
> Subpart 9903.201-2(b) and elects to do so, the offeror shall indicate
> by checking the box below. Checking the box below shall mean that the
> resultant contract is subject to the Disclosure and Consistency of
> Cost Accounting Practices clause in lieu of the Cost Accounting
> Standards clause.
>
> \[ \] The offeror hereby claims an exemption from the Cost Accounting
> Standards clause under the provisions of 48 CFR, Subpart 9903.201-2(b)
> and certifies that the offeror is eligible for use of the Disclosure
> and Consistency of Cost Accounting Practices clause because during the
> cost accounting period immediately preceding the period in which this
> proposal was submitted, the offeror received less than \$25 million in
> awards of CAS-covered prime contracts and subcontracts or the offeror
> did not receive a single CAS-covered award exceeding \$1 million. The
> offeror further certifies that if such status changes before an award
> resulting from this proposal, the offeror will advise the Contracting
> Officer immediately.

Caution: An offeror may not claim the above eligibility for modified
contract coverage if this proposal is expected to result in the award of
a CAS-covered contract of \$25 million or more or if, during its current
cost accounting period, the offeror has been awarded a single
CAS-covered prime contract or subcontract of \$25 million or more.

III\. Additional Cost Accounting Standards Applicable to Existing
Contracts

> The offeror shall indicate below whether award of the contemplated
> contract would, in accordance with subparagraph (a)(3) of the Cost
> Accounting Standards clause, require a change in established cost
> accounting practices affecting existing contracts and subcontracts.

\[ \] Yes \[ \] No

(End of Provision)

ALTERNATE I (APR 1996)

\[ \] (5) Certificate of Disclosure Statement Due Date by Educational
Institution.

> If the offeror is an educational institution that, under the
> transition provisions of 48 CFR 9903.202-1(f), is or will be required
> to submit a Disclosure Statement after receipt of this award, the
> offeror hereby certifies that (check one and complete):

\[\] (a) A Disclosure Statement filing Due Date of

> has been established with the cognizant Federal agency.
>
> \[\] (b) The Disclosure Statement will be submitted within the six
> month period ending months after receipt of this award.
>
> Name and Address of cognizant ACO or Federal Official where Disclosure
> Statement is to be filed:

**(END OF ALTERNATE I)**

K.5. CERTIFICATE OF CURRENT COST OR PRICING DATA

(FAR 15.406-2)

CERTIFICATE OF CURRENT COST OR PRICING DATA

When cost or pricing data are required, the contracting officer shall
require the contractor to execute a Certificate of Current Cost or
Pricing Data using the format in this paragraph, and shall include the
executed certificate in the contract file.

This is to certify that, to the best of my knowledge and belief, the
cost or pricing data (as defined in Section 15.401 of the Federal
Acquisition Regulation(FAR) and required under FAR subsection 15.403-4)
submitted, either actually or by specific identification, in writing, to
the contracting officer or the contracting officer\'s representative in
support of \* are accurate, complete, and current as of \*\*.

This certification includes the cost or pricing data supporting any
advance agreements and forward pricing rate agreements between the
offeror and the Government that are part of the proposal.

FIRM

NAME Signature

TITLE

DATE OF EXECUTION\*\*\*

\* Identify the proposal, request for price adjustment, or other
submission involved, giving the appropriate identifying number (e.g.,
Request for Proposal number).

\*\* Insert the day, month, and year when price negotiations were
concluded and price agreement was reached or, if applicable, an earlier
date agreed upon between the parties that is as close as practicable to
the date of agreement on price.

\*\*\* Insert the day, month, and year of signing, which should be as
close as practicable to the date when the price negotiations were
concluded and the contract price agreed to.

End of Certificate

K.6. ENVIRONMENTAL TOBACCO SMOKE

> The Public Health Service strongly encourages all grant and contract
> recipients to provide a smoke-free workplace and to promote the nonuse
> of all tobacco products. In addition, Public Law 103-227, the
> Pro-Children Act of 1994, prohibits smoking in certain facilities (or
> in some cases, any portion of a facility) in which regular or routine
> education, library, day care, health care or early childhood
> development services are provided to children.

[CERTIFICATION REGARDING ENVIRONMENTAL TOBACCO SMOKE]{.underline}

Public Law 103-227, also known as the Pro-Children Act of 1994 (Act),
requires that smoking not be permitted in any portion of any indoor
facility owned or leased or contracted for by an entity and used
routinely or regularly for the provision of health, day care, early
childhood development services, education or library services to
children under the age of 18, if the services are funded by Federal
programs either directly or through State or local governments, by
Federal grant, contract, loan, or loan guarantee. The law also applies
to children\'s services that are provided in indoor facilities that are
constructed, operated, or maintained with such federal funds. The law
does not apply to children\'s services provided in private residences;
portions of facilities used for inpatient drug or alcohol treatment;
service providers whose sole source of applicable Federal funds is
Medicare or Medicaid; or facilities where WIC coupons are redeemed.
Failure to comply with the provisions of the law may result in the
imposition of a civil monetary penalty of up to \$1000 for each
violation and/or the imposition of an administrative compliance order on
the responsible entity.

By signing this certification, the offeror/contractor certifies that the
submitted organization will comply with the requirements of the Act and
will not allow smoking within any portion of any indoor facility used
for the provision of services for children as defined by the Act.

The submitting organization agrees that it will require that the
language of this certification be included in any subawards which
contain provisions for children\'s services and that all subrecipients
shall certify accordingly.

Organization:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Signature\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Title\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Date\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**SECTION L - INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS**

**L.1 SOLICITATION PROVISIONS INCORPORATED BY REFERENCE (FEB 1998) (FAR
52.252-1)**

> This solicitation incorporates the following solicitation provisions
> by reference, with the same force and effect as if they were given in
> full text. Upon request, the contracting officer will make the full
> text available. Also, the full text of a clause may be assessed
> electronically at this address: http://www.arnet.gov/far/

a\. Federal Acquisition Regulation (FAR) (48 CFR Chapter 1) Solicitation
Provisions

\(1\) 52.215-16 Facilities Capital Cost of Money (OCT 1997)

> \(2\) 52.215-20 Requirements for Cost or Pricing Data or Information
> Other Than Cost or Pricing Data (OCT 1997)

**L.2 DATA UNIVERSAL NUMBERING (DUNS) (OCT 2003) (FAR 52.204-6)**

\(a\) The offeror shall enter, in the block with its name and address on
the cover page of its offer, the annotation "DUNS" or "DUNS+4" followed
by the DUNS number or "DUNS+4" that identifies the offeror's name and
address exactly as stated in the offer. The DUNS number is a nine-digit
number assigned by Dun and Bradstreet Information Services. The DUNS+4
is the DUNS number plus a 4-character suffix that may be assigned at the
discretion of the offeror to establish additional CCR records for
identifying alternative Electronic Funds Transfer (EFT) accounts (see
Subpart 32.11) for the same parent concern.

\(b\) If the offeror does not have a DUNS number, it should contact Dun
and Bradstreet directly to obtain one.

\(1\) An offeror may obtain a DUNSnumber---

\(i\) If located within the United States, by calling Dun and Bradstreet
at 1-866-705-5711 or via the iInternet at
[*http://www.dnb.*com](http://www.dnb.com/); or

\(ii\) If located outside the United States, by contacting the local Dun
and Bradstreet office.

\(2\) The offeror should be prepared to provide the following
information:

\(i\) Company legal business name.

\(ii\) Tradestyle, doing business, or other name by which your entity is
commonly recognized.

\(iii\) Company physical street address, city, state and Zip Code.

\(iv\) Company mailing address, sity, state and Zip Code (if separate
from physical).

\(v\) Company telephone number.

\(vi\) Date the company was started.

\(vii\) Number of employees at your location.

\(viii\) Chief executive officer/ key manager.

\(ix\) Line of business (industry)

\(X\) Company Headquarters name and address (reporting relationship
within your entity).

(End of provision)

**L.3 INSTRUCTIONS TO OFFERORS - COMPETITIVE ACQUISITION (MAY 2001)**

**ALTERNATE I (JAN 2004)(FAR 52.215-1)**

\(a\) Definitions. As used in this provision --

> "Discussions" are negotiations that occur after establishment of the
> competitive range that may, at the Contracting Officer's discretion,
> result in the offeror being allowed to revise its proposal.
>
> "In writing," "writing," or "written" means any worded or numbered
> expression that can be read, reproduced, and later communicated, and
> includes electronically transmitted and stored information.
>
> "Proposal modification" is a change made to a proposal before the
> solicitation's closing date and time, or made in response to an
> amendment, or made to correct a mistake at any time before award.
>
> "Proposal revision" is a change to a proposal made after the
> solicitation closing date, at the request of or as allowed by a
> Contracting Officer as the result of negotiations.
>
> "Time," if stated as a number of days, is calculated using calendar
> days, unless otherwise specified, and will include Saturdays, Sundays,
> and legal holidays. However, if the last day falls on a Saturday,
> Sunday or legal holiday, then the period shall include the next
> working day.

\(b\) Amendments to solicitations. If this solicitation is amended, all
terms and conditions that are not amended remain unchanged. Offerors
shall acknowledge receipt of any amendment to this solicitation by the
date and time specified in the amendment(s).

\(c\) Submission, modification, revision, and withdrawal of proposals.

\(1\) Unless other methods (e.g., electronic commerce or facsimile) are
permitted in the solicitation, proposals and modifications to proposals
shall be submitted in paper media in sealed envelopes or packages (i)
addressed to the office specified in the solicitation, and (ii) showing
the time and date specified for receipt, the solicitation number, and
the name and address of the offeror. Offerors using commercial carriers
should ensure that the proposal is marked on the outermost wrapper with
the information in paragraphs (c)(1)(i) and (c)(1)(ii) of this
provision.

\(2\) The first page of the proposal must show---

\(i\) The solicitation number;

\(ii\) The name, address, and telephone and facsimile numbers of the
offeror (and electronic address if available);

\(iii\) A statement specifying the extent of agreement with all terms,
conditions, and provisions included in the solicitation and agreement to
furnish any or all items upon which prices are offered at the price set
opposite each item;

\(iv\) Names, titles, and telephone and facsimile numbers (and
electronic addresses if available) of persons authorized to negotiate on
the offeror's behalf with the Government in connection with this
solicitation; and

\(v\) Name, title, and signature of person authorized to sign the
proposal. Proposals signed by an agent shall be accompanied by evidence
of that agent's authority, unless that evidence has been previously
furnished to the issuing office.

\(3\) Submissions, modification, revision, and withdrawal of proposals.

\(i\) Offerors are responsible for submitting proposals, and any
modification or revisions, so as to reach the Government office
designated in the solicitation by the time specified in the
solicitation. If no time is specified in the solicitation, the time for
receipt is 4:30 p.m., local time, for the designated Government office
on the date that proposal or revision is due.

\(ii\) (A) Any proposal, modification, or revision received at the
Government office designated in the solicitation after the exact time
specified for receipt of offers is "late" and will not be considered
unless it is received before award is made, the Contracting Officer
determines that accepting the late offer would not unduly delay the
acquisition; and -

\(1\) If it was transmitted through an electronic commerce method
authorized by the solicitation, it was received at the initial point of
entry to the Government infrastructure not later than 5:00 p.m. one
working day prior to the date specified for receipt of proposals; or

\(2\) There is acceptable evidence to establish that it was received at
the Government installation designated for receipt of offers and was
under the Government's control prior to the time set for receipt of
offers; or

\(3\) It is the only proposal received.

\(B\) However, a late modification of an otherwise successful proposal
that makes its terms more favorable to the Government, will be
considered at any time it is received and may be accepted.

\(iii\) Acceptable evidence to establish the time of receipt at the
Government installation includes the time/date stamp of that
installation on the proposal wrapper, other documentary evidence of
receipt maintained by the installation, or oral testimony or statements
of Government personnel.

\(iv\) If an emergency or unanticipated event interrupts normal
Government processes so that proposals cannot be received at the office
designated for receipt of proposals by the exact time specified in the
solicitation, and urgent Government requirements preclude amendment of
the solicitation, the time specified for receipt of proposals will be
deemed to be extended to the same time of day specified in the
solicitation on the first work day on which normal Government processes
resume.

\(v\) Proposals may be withdrawn by written notice received at any time
before award. Oral proposals in response to oral solicitations may be
withdrawn orally. If the solicitation authorizes facsimile proposals,
proposals may be withdrawn via facsimile received at any time before
award, subject to the conditions specified in the provision at 52.215-5,
"Facsimile Proposals." Proposals may be withdrawn in person by an
offeror or an authorized representative, if the representative's
identity is made known and the representative signs a receipt for the
proposal before award.

\(4\) Unless otherwise specified in the solicitation, the offeror may
propose to provide any item or combination of items.

\(5\) Offerors shall submit proposals submitted in response to this
solicitation in English, unless otherwise permitted by the solicitation,
and in U.S. dollars, unless the provision at FAR 52.225-17, Evaluation
of Foreign Currency Offers, is included in the solicitation.

\(6\) Offerors may submit modifications to their proposals at any time
before the solicitation closing date and time, and may submit
modifications in response to an amendment, or to correct a mistake at
any time before award.

\(7\) Offers may submit revised proposals only if requested or allowed
by the Contracting Officer.

\(8\) Proposals may be withdrawn at any time before award. Withdrawals
are effective upon receipt of notice by the Contracting Officer.

> \(d\) Offer expiration date. Proposals in response to this
> solicitation will be valid for the number of days specified on the
> solicitation cover sheet (unless a different period is proposed by the
> offeror).

\(e\) Restriction on disclosure and use of data. Offerors that include
in their proposals data that they do not want disclosed to the public
for any purpose, or used by the Government except for evaluation
purposes, shall ---

\(1\) Mark the title page with the following legend:

> "This proposal includes data that shall not be disclosed outside the
> Government and shall not be duplicated, used, or disclosed--in whole
> or in part--for any purpose other than to evaluate this proposal." If,
> however, a contract is awarded to this offeror as a result of--or in
> connection with-- the submission of this data, the Government shall
> have the right to duplicate, use, or disclose the data to the extent
> provided in the resulting contract. This restriction does not limit
> the Government's right to use information contained in this data if it
> is obtained from another source without restriction. The data subject
> to this restriction are contained in sheets \[insert numbers or other
> identification of sheets\]; and

\(2\) Mark each sheet of data it wishes to restrict with the following
legend:

> "Use or disclosure of data contained on this sheet is subject to the
> restriction on the title page of this proposal."

\(f\) Contract award.

\(1\) The Government intends to award a contract or contracts resulting
from this solicitation to the responsible offeror(s) whose proposal(s)
represents the best value after evaluation in accordance with the
factors and subfactors in the solicitation.

\(2\) The Government may reject any or all proposals if such action is
in the Government's interest.

\(3\) The Government may waive informalities and minor irregularities in
proposals received.

\(4\) The Government intends to evaluate proposals and award a contract
after conducting discussions with offerors whose proposals have been
determined to be within the competitive range. If the Contracting
Officer determines that the number of proposals that would otherwise be
in the competitive range exceeds the number at which an efficient
competition can be conducted, the Contracting Officer may limit the
number of proposals in the competitive range to the greatest number that
will permit an efficient competition among the most highly rated
proposals. Therefore, the offeror's initial proposal should contain the
offeror's best terms from a price and technical standpoint.

\(5\) The Government reserves the right to make an award on any item for
a quantity less than the quantity offered, at the unit cost or prices
offered, unless the offeror specifies otherwise in the proposal.

\(6\) The Government reserves the right to make multiple awards if,
after considering the additional administrative costs, it is in the
Government's best interest to do so.

\(7\) Exchanges with offerors after receipt of a proposal do not
constitute a rejection or counteroffer by the Government.

\(8\) The Government may determine that a proposal is unacceptable if
the prices proposed are materially unbalanced between line items or
subline items. Unbalanced pricing exists when, despite an acceptable
total evaluated price, the price of one or more contract line items is
significantly overstated or understated as indicated by the application
of cost or price analysis techniques. A proposal may be rejected if the
Contracting Officer determines that the lack of balance poses an
unacceptable risk to the Government.

\(9\) If a cost realism analysis is performed, cost realism may be
considered by the source selection authority in evaluating performance
or schedule risk.

\(10\) A written award or acceptance of proposal mailed or otherwise
furnished to the successful offeror within the time specified in the
proposal shall result in a binding contract without further action by
either party.

\(11\) If a post-award debriefing is given to requesting offerors, the
Government shall disclose the following information, if applicable:

\(i\) The agency's evaluation of the significant weak or deficient
factors in the debriefed offeror's offer.

\(ii\) The overall evaluated cost or price and technical rating of the
successful and the debriefed offeror and past performance information on
the debriefed offeror.

\(iii\) The overall ranking of all offerors, when any ranking was
developed by the agency during source selection

(iv) A summary of the rationale for award

(v) For acquisitions of commercial items, the make and model of the item
    to be delivered by the successful offeror.

(vi) Reasonable responses to relevant questions posed by the debriefed
     offerors as to whether source-selection procedures set forth in the
     solicitation, applicable regulations, and other applicable
     authorities were followed by the agency.

(End of provision)

**L.4 TYPE OF CONTRACT (APRIL 1984) (FAR 52.216-1)**

> Number range CHAPTERThe Government contemplates award of a cost
> reimbursement, completion type, contract resulting from this
> solicitation.
>
> It is anticipated that one contract award will be made from this
> solicitation and that the award is estimated to be made effective
> September, 27, 2006.

**L.5 SERVICE OF PROTEST (AUG 1996)(FAR 52.233-2)**

\(a\) Protests, as defined in Section 33.101 of the Federal Acquisition
Regulation, that are filed directly with an agency, and copies of any
protests that are filed with the General Accounting Office (GAO) shall
be served on the Contracting Officer (addressed as follows) by obtaining
written and dated acknowledgment of receipt from:

Director, Division of Contracts Management

Agency for Healthcare Research and Quality

540 Gaither Road

Rockville, Maryland 20850

\(b\) The copy of any protest shall be received in the office designated
above within one day of filing a protest with the GAO.

**L.6 POINT OF CONTACT FOR TECHNICAL INQUIRIES**

> The technical contact for additional information and answering
> inquiries is the Contracting Officer. All questions regarding this
> solicitation shall be in writing and received by the Contracting
> Officer no later than 4pm EST **June 7, 2006**. Number range CHAPTERIt
> is preferred that all questions be e-mailed to Mary Haines at
> <mhaines@ahrq.gov> or <mary.haines@ahrq.hhs.gov> . Number range
> CHAPTERHard copies can be sent to:

Agency for Healthcare Research and Quality

OPART/Contracts Management

540 Gaither Road

Rockville, MD 20850

Attention: Mary Haines, Contracting Officer

> Fax: (301) 427-1740

Number range CHAPTER**L.7 REFERENCE MATERIALS**

> Attachment 1, Statement of Work provides numerous web sites to access
> information about AHRQ's on-going programs supporting AHRQ which are
> necessary for complete understanding of this requirment.

**L.8 GENERAL INSTRUCTIONS**

[Introduction]{.underline}

> The following instructions will establish the acceptable minimum
> requirements for the format and contents of proposals. Special
> attention is directed to the requirements for technical and business
> proposals to be submitted in accordance with these instructions:

a\. [Contract Type and General Provisions]{.underline}: Number range
CHAPTERIt is contemplated that a cost-reimbursement, task order type
contract will be awarded. In addition to the special provisions of this
request for proposal (RFP), any resultant contract shall include the
general clauses applicable to the selected offeror\'s organization and
type of contract awarded. Any additional clauses required by Public Law,
Executive Order, or procurement regulations, in effect at the time of
execution of the proposed contract, will be included.

b\. [Authorized Official and Submission of Proposal]{.underline}: The
proposal shall be signed by an official authorized to bind your (the
offeror\'s) organization. Your proposal shall be submitted in the number
of copies, to the address, and marked as indicated in the cover letter
of this solicitation. Proposals will be typewritten, reproduced on
letter sized paper and will be legible in all required copies. To
expedite the proposal evaluation, all documents required for responding
to the RFP should be placed in the following order:

I. [TECHNICAL PROPOSAL]{.underline}: See Technical Proposal Instructions
for recommended format (L.10). Please mark as original or copy.

II. [PAST PERFORMANCE INFORMATION]{.underline}: See Past Performance
    Information Instructions for format (L.11)

III. Number range CHAPTER[SMALL DISADVANTAGED BUSINESS PARTICIPATION
     PLAN:]{.underline} See Small Disadvantaged Business Plan
     Instructions for format (L.12)

IV. [BUSINESS PROPOSAL]{.underline}: See Business Proposal Instructions
    for recommended format (L.13).

c\. [Separation of Technical, Past Performance Information, Small
Disadvantaged Business Participation Plan and Business
Proposal]{.underline}: The proposal shall be in 4 parts:

> \(1\) Technical Proposal; (2) Past Performance Information; (3) Small
> Disadvantaged Business Participation Plan and (4) Business Proposal
> (including the Small Business Subcontracting Plan). Each of the parts
> shall be separate and complete in itself so that evaluation of one may
> be accomplished independently of, and concurrently with, evaluation of
> the other. [The technical proposal shall not contain reference to
> cost]{.underline}; however resources information, such as data
> concerning labor hours and categories, materials, subcontracts, etc.,
> shall be contained in the technical proposal so that your
> understanding of the Statement of Work (SOW) may be evaluated. It must
> disclose your technical approach in as much detail as possible,
> including, but not limited to, the requirements of the technical
> proposal instructions.

d\. [Evaluation of Proposals]{.underline}: The Government will evaluate
technical proposals in accordance with the criteria set forth in Section
M, [Evaluation/Award Criteria]{.underline}.

e\. [Rejection of Proposals]{.underline}: The Government reserves the
right to reject any or all proposals received. It is understood that
your proposal will become part of the official contract file.

f\. [Unnecessarily Elaborate Proposals]{.underline}: Unnecessarily
elaborate brochures or other presentations beyond those sufficient to
present a complete and effective proposal are not desired and may be
construed as an indication of the offeror\'s lack of cost consciousness.
Elaborate art work, expensive visual and other presentation aids are
neither necessary nor wanted.

g\. [Privacy Act]{.underline}: The Privacy Act of 1974 (Public Law
(P.L.) 93-579) requires that a Federal agency advise each individual
whom it asks to supply information: 1) the authority which authorized
the solicitation; 2) whether disclosure is voluntary or mandatory; (3)
the principal purpose or purposes for which the information is intended
to be used; (4) the uses outside the agency which may be made of the
information; and 4) the effects on the individual, if any, of not
providing all or any part of the requested information.

Therefore:

> \(1\) The Government is requesting the information called for in this
> RFP pursuant to the authority provided by Section 301(g) of the Public
> Health Service Act, as amended, and P.L. 92-218, as amended.

\(2\) Provisions of the information requested are entirely voluntary.

\(3\) The collection of this information is for the purpose of
conducting an accurate, fair, and adequate review prior to a discussion
as to whether to award a contract.

\(4\) Failure to provide any or all of the requested information may
result in a less than adequate review.

\(5\) The information provided by you may be routinely disclosed for the
following purposes:

> -to the cognizant audit agency and the General Accounting Officer for
> auditing;

-to the Department of Justice as required for litigation;

-to respond to Congressional inquiries; and

-to qualified experts, not within the definition of Department employees
for opinions as a part of the review process.

In addition, the Privacy Act of 1974 (P.L. 93-579, Section 7) requires
that the following information be provided when individuals are
requested to disclose their social security number.

> Provision of the social security number is voluntary. Social security
> numbers are requested for the purpose of accurate and efficient
> identification, referral, review and management of AHRQ contracting
> programs. Authority for requesting this information is provided by
> Section 305 and Title IV of the Public Health Service Act, as amended.

h\. The RFP does not commit the Government to pay any cost for the
preparation and submission of a proposal. It is also brought to your
attention that the Contracting Officer is the only individual who can
legally commit the Government to the expenditure of public funds in
connection with this or any acquisition action.

> The Government reserves the right to award a contract without
> discussions if the Contracting Officer determines that the initial
> prices are fair and reasonable and that discussions are not necessary.

**L.10 TECHNICAL PROPOSAL INSTRUCTIONS**

> The technical proposal shall contain an original and eleven (11)
> copies. The technical proposal described below shall be limited to
> **125** **pages** not including biographic sketches, with no less than
> a 11 point font, with the majority of the text double-spaced (lists of
> deliverables, person loading charts, and similar materials need not be
> double-spaced, so long as they are legible). Brief biographic sketches
> or CVs (less than ten pages in length) providing the relevant
> qualifications necessary for this effort are only required for key
> personnel. [The technical proposal shall not contain reference to
> cost]{.underline}; however resources information, such as data
> concerning labor hours and categories, labor mix, materials,
> subcontracts, etc., shall be contained in the technical proposal so
> that your understanding of the Statement of Work (SOW) may be
> evaluated. It must disclose your technical approach in as much detail
> as possible, including, but not limited to, the requirements of these
> instructions. Lengthy proposals and voluminous appendices are neither
> needed nor desired as they are difficult to read and evaluate and may
> indicate the offeror's inability to concisely state their proposal.
> Appendices are to be provided electronically in MS Office format, in
> the same quantity as the technical proposal.

a\. [Recommended Technical Proposal Format]{.underline}

> The offeror's proposal should present sufficient information to
> reflect a thorough understanding of the work requirements and a
> detailed plan for achieving the objectives of the scope of work.
> Technical proposals shall not merely paraphrase the requirements of
> the Agency's scope of work or parts thereof, or use of phrases such as
> "will comply" or "standard techniques will be employed." The technical
> proposal must include a detailed description of the techniques and
> procedures to be used in achieving the proposed end results in
> compliance with the requirements of the Agency's scope of work.
>
> \(1\) [Cover Page]{.underline}: The name of the proposing
> organization, author(s) of the technical proposal, the RFP number and
> the title of the RFP should appear on the cover. The cover page must
> also include the DUNS and TIN as well as a point of contact and
> contact information. One (1) manually signed original copy of the
> proposal and the number of copies specified in the RFP cover letter
> are required.

\(2\) [Table of Contents]{.underline}: Provide sufficient detail so that
all important elements of the proposal can be located readily.

(3) [Introduction]{.underline}: This should be a one or two page summary
    outlining the proposed work, your interest in submitting a proposal,
    and the importance of this effort in relation to your overall
    operation.

(4) [Technical Discussion]{.underline}: The offeror shall prepare a
    technical discussion which addresses evaluation criteria 1, 2, 3
    below (including their subcriteria). Responses to evaluation
    criteria 4 is to be prepared in accordance with Sections L.11. The
    offeror shall further state that no deviations or exceptions to the
    Statement of Work (SOW) are taken. The evaluation criteria (and
    their respective subcriteria) are as follows:

> **[Technical proposals submitted in response to this RFP shall address
> each of the items described below, and shall be organized in the same
> manner and within the page limitations specified. Proposals shall be
> prepared in double-spaced format, with numbered pages.]{.underline}**

**L.9 TECHNICAL PROPOSAL INSTRUCTIONS**

> The technical proposal shall contain an original and eleven (11)
> copies. The technical proposal described below shall be limited to
> **150 pages** not including resumes or bibliographies, with no less
> than a 11 point pitch, with the majority of the text double-spaced
> (lists of deliverables, person loading charts, and similar materials
> need not be double-spaced, so long as they are legible). Brief
> biographic sketches or CVs (less than ten pages in length) providing
> the relevant qualifications necessary for this effort are only
> required for key personnel. [The technical proposal shall not contain
> reference to cost;]{.underline} however resources information, such as
> data concerning labor hours and categories, labor mix, materials,
> subcontracts, etc., shall be contained in the technical proposal so
> that your understanding of the Statement of Work (SOW) may be
> evaluated. It must disclose your technical approach in as much detail
> as possible, including, but not limited to, the requirements of these
> instructions. Lengthy proposals and voluminous appendices are neither
> needed nor desired as they are difficult to read and evaluate and may
> indicate the offeror's inability to concisely state their proposal.
> Appendices are to be provided electronically in MS Office format, in
> the same quantity as the technical proposal.

a\. [Recommended Technical Proposal Format]{.underline}

The Offeror's proposal should present sufficient information to reflect
a thorough understanding of the work requirements and a detailed plan
for achieving the objectives of the scope of work. Proposals shall not
merely paraphrase the requirements of the Agency's scope of work or
parts thereof, or use of paraphrases such as "will comply" or "standard
techniques will be employed." The technical proposal must include a
detailed description of the techniques and procedures to be employed in
achieving the proposed end results in compliance with the requirements
of the Agency's scope of work. Offerors are hereby advised that the
Agency, unless otherwise specified, i.e., by law, will acquire the right
to use, duplicate or disclose in any manner and for any purpose
whatsoever, and have or permit others to do so, all subject data
required to be delivered under any contract resulting from this
solicitation. Any reservations to these stated Agency rights should be
enunciated in the proposal and will be resolved during subsequent
negotiations.

> To assist in the expeditious and comprehensive evaluation of your
> proposal, the Government desires that you follow the guidelines and
> format listed below:

\(1\) [Cover Page]{.underline}: The name of the proposing organization,
author(s) of the technical proposal, the RFP number and the title of the
RFP and the title of the RFP should appear on the cover. The cover page
must also include the DUNS and TIN as well as a point of contact and
contact information. One (1) manually signed original copy of the
proposal and the number of copies specifed in the RFP cover letter are
required. The originals of each element of the proposal must be easily
accessed and identified.

\(2\) [Table of Contents]{.underline}: Provide sufficient detail so that
all important elements of the proposal can be located readily.

\(3\) [Introduction]{.underline}: This should be a one or two page
summary outlining the proposed work, your interest in submitting a
proposal, and the importance of this effort in relation to your overall
operation.

\(4\) [Technical Discussion]{.underline}: The offeror shall prepare a
technical discussion which addresses evaluation criteria A, B, C, and D
below. Evaluation criteria E and F are to be prepared in accordance with
Sections L.10 and L.11. The offeror shall further state that no
deviations or exceptions to the SOW are taken. The evaluation criteria
are as follows:

A. Response to the Statement of Work

B. Staffing and Management Plan

C. Corporate/ Organization Experience

> D. Facilities and Equipment

E. Past Performance (See Section L.10)

> F. Small Disadvantaged Business Participation Plan (See Section L.11)
>
> **[Technical proposals submitted in response to this RFP shall address
> each of the items described below, and shall be organized in the same
> manner and within the page limitations specified. Proposals shall be
> prepared in double-spaced format, with numbered pages.]{.underline}**

b\. [Technical Proposal Requirements]{.underline}

**A.** **Response to Statement of Work (SOW)**

\(1\) The Offeror shall submit a narrative describing pertinent work
already published and/ or performed that is relevant to this project,
particularly as it relates to innovations and change management in
health service delivery, implementation of evidence-based practices,
quality improvement or disparities reduction initiatives, clinical
information tools, learning communities, complex database development,
development and implementation of Web-based services, and to the
Offeror's proposed approach. This narrative should support the Offeror's
interpretation of the statement of work.

\(2\) The Offeror shall submit a draft Vision/Project Initiation
Document (PID). The PID shall contain a statement of purpose and scope
for the project. The PID will show the Offeror's overall understanding
of the issues of this project. The PID shall define the stakeholders'
view of the Innovations Clearinghouse in terms of the key needs and
features identified in the Statement of Work and associated documents,
and at a high level, define the core requirements and allocate them to
iterations and releases of the Innovation Clearinghouse Web site and
database (including the incorporation of the QualityTools Web site and
database into the Innovations Clearinghouse). A concise description of
the challenges and problems that might be encountered in meeting the
objectives and in the performance of the tasks along with a description
of how these challenges and problems can be creatively met or resolved
shall also be provided.

\(3\) In this section, the Offeror shall provide the name(s) of the
author(s) of each section of its proposal.

\(4\) The Offeror shall include a narrative that clearly addresses how
it plans to develop, design and implement a process to produce high
quality deliverables and to meet the requirements of the project,
particularly within the time constraints of the project. This can become
the basis for the Project Work Plan (PWP) and the deliverables-based,
resource loaded Work Breakdown Structure (WBS). The Offeror may use as
many paragraphs and subparagraphs, appropriately titled, as necessary to
clearly outline the technical approach. Within the content of the
narrative, the Offeror shall:

\- indicate the rationale for the approach and relate it to comparable
work in progress elsewhere;

\- address plans for identifying, utilizing and monitoring
subcontractors and consultants;

\- describe the production of clear, concise reports on project
findings;

\- specify quality control activities, problem area identification and
resolution strategies;

\- mention phasing of the project, when appropriate;

\- characterize methodological design and possible or likely outcomes of
approaches proposed;

\- delineate anticipated difficulties within the technical approach
together with recommended solutions; and

\- depict the extent to which the proposed approach can be expected to
meet or exceed the requirements of the statement of work.

\(5\) Additionally, in the discussion of its technical approach to the
SOW, the Offeror shall:

\- Clearly demonstrate its understanding of: health service delivery
creativity, innovation and change; health care quality improvement or
disparities reduction implementation and evaluation; learning
communities; complex database development.

\- Describe previous experience working with expert panels representing
diverse interests.

\- Describe previous experience working with an editorial board.

\- Describe previous experience meeting the Software Engineering
Institute (SEI) Capability Maturity Model Integration (CMMI) level 2
(minimum of three years prior experience) and provide three (3)
references who can attest to the Offeror's successful use of using a
structured formal software development methodology.

\- Describe previous experience complying with government regulations,
such as security requirements, 508 requirements, and CPIC/project
management requirements.

\- Describe its approach to and its ability to successfully design and
implement a large Web-based relational database of innovations-related
information and associated features, such as mediating and posting user
comments, and establishing learning communities and networking
opportunities for Innovations Clearinghouse users. The Offeror shall
also clearly demonstrate its knowledge of software, hardware,
programming, logistics, and technical assistance necessary to support
those mechanisms. The offeror shall include a detailed discussion of the
specific hardware, software, network and other technology architecture
components and solutions proposed to design, develop, build, test,
implement, maintain, backup/recover and secure the system. The offeror
shall include its reasons for choosing specific hardware, software,
network and other technology architecture components and solutions when
they differ from the Government's Draft Technical Reference Model
(Appendix E).

\(6\) The Offeror shall address the technical approach proposed for each
task required by the Statement of Work.

**B**. **Staffing and Management Plan**

> The Offeror shall demonstrate its ability to achieve the delivery of
> performance requirements through the proposed use of corporate
> management and other personnel resources as well as demonstrate that
> the Offeror's organizational structure and capabilities will meet the
> project's milestones in a timely manner. In doing so, and at a
> minimum, the Offeror shall:

1.  Provide a fully supported narrative showing the Offeror's
    understanding of the requirements of the Statement of Work from a
    managerial perspective. The narrative should at a minimum address
    the following topics:

\- Provide reasons for labor skill mix determination (why you chose the
skill mix for this project);

\- Provide reasons for personnel selection and assignment (why you chose
an individual person for an individual job);

\- Indicate the percentage of full time core personnel to the percentage
of consultants/ subcontractors proposed. The Offeror shall provide a
detailed explanation of how the proposed staffing plan ensures that the
work is conducted by individuals with a mastery of the technical
requirements of the Statement of Work.

\- Describe monitoring and control of services provided as it relates to
technical quality, responsiveness, cost control, risk management,
effective and efficient resource utilization, and compliance with
technical requirements and contract provisions. Clearly show proposed
system for quality control of work performed including deliverables to
be produced, and proposed system for management control and contract
provision compliance.

\- Describe managerial problems the Offeror expects to encounter and the
methods proposed to solve these problems. Demonstrate ability and
flexibility to rapidly solve the same or similar managerial problems
encountered previously.

\- Describe the project management approach the Offeror will use to
manage the contract activities. Demonstrate knowledge and experience
with government requirements and industry best practices.

2.  Provide a person-level task-loading chart (to include consultants
    and subcontractor effort) and an organizational chart indicating
    clear lines of authority, delineating staff responsibilities and a
    plan for organizational backup. Employees not currently employed by
    the Offeror will be listed with an asterisk (\*) to denote their
    status.

\(3\) For each person named for the proposed effort, with the exception
of general support personnel, the Offeror shall provide a list of
Federal, State and other contracts to which they are currently obligated
and the proportion of time committed to these contracts during the first
year of contract performance for this project. This information shall be
provided for subcontractor employees and consultants as well.

\(4\) Describe the relationship with all subcontractors and consultants
including monitoring of their performance. **[Letters of commitment for
detailed subcontracting arrangements, consulting arrangements, and for
consultants and members shall be included with the
proposal.]{.underline}** For consultants, the letters of commitment
shall specify the number of hours that each will be available throughout
each year of project performance.

\(5\) Provide a PERT chart showing all tasks to include any critical
milestones and all deliverables.

\(6\) Describe the Offeror's ability and flexibility to respond rapidly
to changes in requirements, budget, priorities, and schedule.

\(7\) Specify the project team, including subcontractors and consultants
in the following labor classes:

> [Class I]{.underline}
>
> Senior management personnel, holding an advanced clinical, technical,
> or professional degree, at the Masters, Ph.D. or M.D. level, with a
> minimum of ten (10) years experience in operationalizing innovative
> health service quality improvement initiatives and teaching others how
> to do so; analyzing biomedical, medical effectiveness or outcomes, or
> other scientific literature, research findings and data, with
> significant experience related to implementation science and
> innovation; evaluation and implementation of evidence-based clinical
> information; management of large complex projects including the
> development and implementation of original work; and work with
> organizations, or medical specialty societies. This class of personnel
> shall also have corporate level management experience that reflects an
> ability to command organizational resources and direct staff within
> the broader organization.
>
> [Class II]{.underline}
>
> Associated management or clinical/professional/technical personnel
> holding an advanced degree, at the Masters, Ph.D., M.D., level, with a
> minimum of five (5) years experience in the development,
> implementation, and evaluation of health care quality improvement or
> disparities reduction initiatives; evaluation and implementation of
> evidence-based clinical information; construction of large relational
> databases; and development of clinically-related classification
> schemes; and development of Web sites.
>
> [Class III]{.underline}
>
> Intermediate clinical/technical personnel, holding a B.S. or B.A.
> degree and at least three (3) years experience in technical activities
> of which at least two (2) are related to construction of larger
> relational databases, development of classification schemes, analysis
> of quality improvement, biomedical, medical effectiveness or outcomes,
> and other scientific literature, research findings, and data. The
> individual is capable of carrying out independent assignments with
> minimum supervision or acting as team leader of small projects.
> Includes specialists in computer programming, World Wide Web site
> construction, etc.
>
> [Class IV]{.underline}
>
> Data support, document search and retrieval staff, report editors, at
> a research assistant level.
>
> The Offeror's proposed Project Director shall be clearly identified.
> This individual shall possess strong corporate level management
> experience. The Project Director is responsible for the overall
> management of this contract, including coordination and cooperation
> with the Government Project Officer and members of the Panel of
> Experts; direction and oversight of all activities to be performed
> under this contract; and assuring the high quality and timeliness of
> work performed.
>
> The Offeror shall clearly identify a PMI-certified Project Manager
> responsible for the day to day management of activities to be
> performed under this contract. The Project Manager must be a highly
> experienced and qualified individual with significant leadership and
> communications skills and demonstrated experience in managing complex,
> large scale projects with similar requirements and outcomes. It is
> expected that the Project Manager will have training and experience in
> health services research, development and implementation of quality
> improvement strategies, construction and management of large
> relational databases, and personnel/project management. In addition,
> the Project Manager must have experience in working with diverse
> groups such as innovators, health care providers and purchasers,
> clinical and health services researchers, and health care policy
> makers.

**C**. **Corporate/Organizational Experience**

The Offeror must possess demonstrated experience as an organization in
conducting and managing projects of this magnitude and complexity. It is
essential that the Offeror demonstrate the capability to organize and
manage resources and personnel effectively, and to successfully
undertake and complete highly technical and complex projects.

The Offeror shall describe its relevant organizational qualifications
and experience, especially that relating to innovating in health service
delivery and creating associated tools for implementation of the
innovation; development of clinically-related classifications schemes
and relational databases; work with expert panels and advisory boards;
formulation of user-friendly World Wide Web sites, and provision of
technical assistance. The descriptions shall delineate how these
qualifications and experience are relevant to fulfilling the
requirements of this proposed contract. In addition, the Offeror shall
submit, if available, up to two examples of relevant or similar work
that it has performed in the past.

**D.** **Facilities and Equipment**

The Offeror shall demonstrate that it has adequate facilities and
equipment for successful completion of tasks under this contract. This
includes, but is not limited to, the availability of PC-compatible word
processing equipment with Microsoft Office, specific software and
hardware packages required to complete the tasks required in the
Statement of Work, duplicating capacity, facsimile capabilities, access
to electronic databases necessary to search biomedical and health
services literature, ability to obtain hard and electronic copy of
innovations, and ability to link via electronic mail with
subcontractors, consultants, Project Expert Panel, and Agency staff.

E.  **Past Performance** (See Section L.10)

F.  **Small Disadvantaged Business Utilization Plan** (See Section L.11)

**L.10 PAST PERFORMANCE** **INFORMATION**

> Offerors shall submit the following information (original and 5
> copies) as part of their proposal for both the offeror and proposed
> major subcontractors:

\(1\) A list of the last five (5) contracts and subcontracts completed
(most relevant or most related) during the past three years and all
contracts and subcontracts currently in process. Reference contracts and
subcontracts completed during the past three years and include recently
completed and ongoing work directly related to the requirements of this
acquisition. Contracts listed may include those entered into by the
Federal Government, agencies of State and local governments and
commercial customers. Offerors that are newly formed entities without
prior contracts should list contracts and subcontracts as required for
all key personnel. Include the following information for each contract
and subcontract:

> a: Name of contracting activity
>
> b: Contract number
>
> c: Contract type
>
> d: Total contract value
>
> e: Contract work
>
> f: Contracting Officer and telephone number
>
> g: Program Manager and telephone number
>
> h: Administrative Contracting Officer, if different from item f, and
> telephone number
>
> i: List of major subcontracts

\(2\) The offeror may provide information on problems encountered on the
contracts and subcontracts identified in (1) above and corrective
actions taken to resolve those problems. Offerors should not provide
general information on their performance on the identified contracts.
General performance information will be obtained from the references.

\(3\) The offeror may describe any quality awards or certifications that
may indicate the offeror possesses a high-quality process for developing
and producing the product or service required. Identify what segment of
the company (one division or the entire company) that received the award
or certification. Describe when the award or certification was bestowed.
If the award or certification is over three years old, present evidence
that the qualifications still apply.

\(4\) Each offeror will be evaluated on his/her performance under
existing and prior contracts for similar products or services.
Performance information will be used for both responsibility
determinations and as an evaluation factor against which offeror's
relative rankings will be compared to assure best value to the
Government. The Government will focus on information that demonstrates
quality of performance relative to the size and complexity of the
procurement under consideration. References other than those identified
by the offeror may be contacted by the Government with the information
received used in the evaluation of the offeror's past performance.

> The attached Past Performance Questionnaire and Contractor Performance
> Form (Attachment 1) shall be completed by those contracting
> organizations listed in (1) above. The evaluation forms shall be
> completed and forwarded directly to the following:
>
> Mary Haines
>
> Agency for Healthcare Research and Quality
>
> Division of Contracts Management
>
> 540 Gaither Road
>
> Rockville, Maryland 20850
>
> FAX: 301-427-1740

Number range CHAPTEREvaluation forms must be received by **July 25,
2006** in order to be included in the review process. It is the
responsibility of the offeror to ensure that these documents are
forwarded to the Contracting Officer.

> **L.11 SMALL DISADVANTAGED BUSINESS PARTICIPATION PLAN:**

In accordance with FAR Part 15.304(c)4, the extent of participation of
Small Disadvantaged Business (SDB) concerns in performance of the
contract shall be evaluated in unrestricted acquisitions expected to
exceed a total estimated cost of \$500,000 (\$1,000,000 for
construction) subject to certain limitations (see FAR 19.201 and
19.1202).

A. All offerors, regardless of size, shall submit the following
information in original and 2 copies.

A plan on the extent of participation of Small Disadvantaged Business
concerns in performance of the contract. Participation in performance of
the contract includes the work expected to be performed by SDB
concern(s). This can include SDB (as prime contractor), joint ventures,
teaming arrangements, and subcontracts. Include the following
information in SDB participation plans:

1\. The extent of an offeror's commitment to use SDB concerns.
Commitment should be as specific as possible, i.e., are subcontract
arrangements already in place, letters of commitment, etc. Enforceable
commitments will be weighted more heavily than non-enforceable ones.

2\. Specifically identify the SDB concerns with point of contact and
phone number.

3\. The complexity and variety of the work SDB concerns are to perform.

4\. Realism for the use of SDB in the proposal.

5\. Past performance of the Offeror in complying with subcontracting
plans for SDB concerns.

6\. Targets expressed as dollars and percentage of total contract value
for each participating SDB; which will be incorporated into and become
part of any resulting contract.

7\. The extent of participation of SDB concerns in terms of the total
acquisition.

B.  SDB participation information will be used for both responsibility
    determinations and as an evaluation factor against which offeror's
    relative rankings will be compared to assure the best value to the
    Government. The Government will focus on information that
    demonstrates realistic commitments to use SDB concerns relative to
    the size and complexity of the acquisition under consideration. The
    Government is not required to contact all references provided by the
    offeror. Also, references other than those identified by the offeror
    may be contacted by the Government to obtain additional information
    that will be used in the evaluation of the offeror's commitment to
    SDB participation.

**L.12 BUSINESS PROPOSAL**

> The offeror shall submit as part of the proposal a
> [separate]{.underline} enclosure titled "Business Proposal." The
> Business Proposal shall include the Cost/Price Proposal, the Small
> Business Subcontracting Plan, and Other Administrative Data in
> accordance with the following:

A. [Cost/Price Proposal]{.underline}

Number range CHAPTERA. Cost/Price Proposal

> A cost proposal, in the amount of an original and five (5) copies,
> shall be provided only to the extent that it shall include:

1\. Certified, unloaded, labor rates for individuals expected to work on
a project of this size and nature (Class Levels I through IV , see
Sections B.3 and L.10).

2\. Certified documentation indicating that the offeror has a cost
accounting system in place which allows for the collection, tracking and
reporting of all costs under a cost reimbursement-type contract.

3\. Certified documentation that the offeror has a current indirect cost
rate agreement in place with a federal agency or that is in the process
of obtaining or revising such an agreement. A copy of the indirect cost
rate agreement or the proposed rate agreement shall be provided.

B.  [Small Business Subcontracting Plan:]{.underline}

> All offerors except small businesses are required to submit a
> subcontracting plan in accordance with the Small Business
> Subcontracting Plan, FAR 52.219-9, incorporated in this solicitation.
> A copy of a model subcontracting plan is available at
> <http://www.knownet.hhs.gov/smallbus/sb-subplan-hhs.pdf> . If the
> model plan is not used, all elements outlined must be addressed in the
> offeror's format. **If the offeror is not a small business and fails
> to submit a subcontracting plan with the initial proposal, the offeror
> will be considered nonresponsive and their proposal will be returned
> without further consideration.**
>
> **[This provision does not apply to small business concerns. This
> provision does apply to all other offerors, including large business
> concerns, colleges, universities and non-profit
> organizations.]{.underline}**

The term "subcontract" means any agreement (other than one involving an
employer-employee relationship) entered into by a Federal Government
prime contractor or subcontractor calling for supplies or services
required for the performance of the original contract or subcontract.
This includes, but is not limited to, agreements/ purchase orders for
supplies and services such as equipment purchase, copying services, and
travel services.

The offeror understands that:

a\. No contract will be awarded unless and until an acceptable plan is
negotiated with the Contracting Officer. The plan will be incorporated
in to the contract.

b\. An acceptable plan must, in the determination of the Contracting
officer, provide the maximum practicable opportunity for small business
concerns and small business concerns owned and controlled by socially
and economically disadvantaged persons to participate in the performance
of the contract.

c\. If a subcontracting plan acceptable to the Contracting Officer is
not negotiated within the time limits prescribed by the contracting
activity and such failure arises out of causes within the control and
with the fault or negligence of the offeror, the offeror shall be
ineligible for award. The Contracting Officer shall notify the
Contractor in writing of the reasons for determining a subcontracting
plan unacceptable early enough in the negotiation process to allow the
Contractor to modify the plan within the time limits prescribed.

d\. Prior compliance of the offeror with other such subcontracting plans
under previous contracts will be considered by the Contracting Officer
in determining the responsibility of the offeror for award of the
contract.

e\. It is the offeror's responsibility to develop a satisfactory
subcontracting plan with respect to small business concerns and small
business concerns owned and controlled by socially and economically
disadvantaged individuals, and women-owned small business concerns, and
that each such aspect of the offeror's plan will be judged independent
of the other.

f\. The offeror will submit, as required by the Contracting Officer,
subcontracting reports in accordance with the instructions thereon, and
as further directed by the Contracting Officer. Subcontractors will also
submit these reports to the Government Contracting Officer or as
otherwise directed, with a copy to the prime Contractor's designated
small and disadvantaged business liaison.

g.  For this particular acquisition, the AHRQ recommended goal (as a
    percentage of total contract value for the base period) is 23% for
    Small Businesses, which shall included at least 5% (as a percentage
    of total planned subcontract dollars for the base period) for Small
    Disadvantaged Businesses, at least 5% (as a percentage of total
    planned subcontract dollars total planned subcontract dollars for
    the base period) for Women-Owned Small Businesses, and at least 2%
    (as a percentage of total planned subcontract dollars for the base
    period) for HUBZone Small Businesses and at least 3% (as a
    percentage of total planned subcontract dollars for the base period)
    for Veteran-Owned Small Businesses. These goals represent AHRQ's
    expectations of the minimum level for subcontracting with small
    business at the prime contract level. Any goal stated less than the
    AHRQ recommended goal shall be justified and is subject to
    negotiation.

C. Other Administrative Data

\(1\) [Terms and Conditions]{.underline}: The proposal shall stipulate
that it is predicated upon the terms and conditions of the RFP. In
addition, it shall contain a statement to the effect that it is firm for
a period of at least 120 days from the date of receipt thereof by the
Government.

> Minimum Bid Acceptance Period (April 1984)

\(a\) \"Acceptance period,\" as used in this provision, means the number
of calendar days available to the Government for awarding a contract
from the date specified in this solicitation for receipt of bids.

\(b\) This provision supersedes any language pertaining to the
acceptance period that may appear elsewhere in this solicitation.

\(c\) The Government requires a minimum acceptance period of 120 days.

\(d\) A bid allowing less than the Government\'s minimum acceptance
period may be rejected.

\(e\) The bidder agrees to execute all that it has undertaken to do, in
compliance with its bid, if that bid is accepted in writing within (i)
the acceptance period stated in paragraph (3) above, or (ii) any longer
acceptance period stated in paragraph (4) above.

\(2\) [Authority to Conduct Negotiations]{.underline}: The proposal
shall list the names and telephone numbers of persons authorized to
conduct negotiations and to execute contracts.

> \(3\) [Property]{.underline}:

\(a\) It is HHS policy that contractors will provide all equipment and
facilities necessary for performance of contracts. Exception may be
granted to furnish Government-owned property, or to authorize purchase
with contract funds, only when approved by the contracting officer. If
additional equipment must be acquired, you shall include the
description, estimated cost of each item and whether you will furnish
such items with your own funds.

\(b\) You shall identify Government-owned property in your possession
and/or property acquired from Federal funds to which you have title,
that is proposed to be used in the performance of the prospective
contract.

\(c\) The management and control of any Government property shall be in
accordance with HHS Publication (OS) 74-115 entitled,
[Contractor\'s]{.underline}

> [Guide for Control of Government Property]{.underline}\" 1990, a copy
> of which will be provided upon request.

\(4\) [Royalties]{.underline}: You shall furnish information concerning
royalties which are

> anticipated to be paid in connection with the performance of work
> under the proposed contract.

\(5\) [Commitments]{.underline}: You shall list other commitments with
the Government relating to the specified work or services and indicate
whether these commitments will or will not interfere with the completion
of work and/or services contemplated under this proposal.

\(6\) [Financial Capacity]{.underline}: You shall provide sufficient
data to indicate that you have the necessary financial capacity, working
capital, and other resources to perform the contract without assistance
from any outside source. If not, indicate the amount required and the
anticipated source. (Financial data such as balance sheets, profit and
loss statements, cash forecasts, and financial histories of your
organization\'s affiliated concerns should be utilized.)

\(7\) [Performance Capability]{.underline}: You shall provide acceptable
evidence of your \"ability to obtain\" equipment, facilities, and
personnel necessary to perform the requirements of this project. If
these are not represented in your current operations, they should
normally be supported by commitment or explicit arrangement, which is in
existence at the time the contract is to be awarded, for the rental,
purchase, or other acquisition of such resources, equipment, facilities,
or personnel. In addition, you shall indicate your ability to comply
with the required or proposed delivery or performance schedule taking
into consideration all existing business commitments, commercial as well
as Government.

\(8\) [Representations and Certifications]{.underline}: Section K,
\"Representations and Certifications and Other Statements of Offerors\"
shall be completed and signed by an official authorized to bind your
organization. **This section shall be made a part of the original
business proposal.**

**L.13 SELECTION OF OFFERORS**

a\. The acceptability of the technical portion of each contract proposal
will be evaluated by the technical review committee. The committee will
evaluate each proposal in strict conformity with the evaluation criteria
of the RFP, utilizing point scores and written critiques. The committee
may suggest that the Contracting Officer request clarifying information
from an offeror.

b\. The business portion of each contract proposal will be subjected to
a cost review, management analysis, etc.

c\. Number range CHAPTERPast performance, Small Business Subcontracting
Plan and the Small Disadvantaged Business Participation Plan of the
technically acceptable offerors will be evaluated by AHRQ staff. A
competitive range will be determined. Oral or written discussions will
be conducted with all offerors in the competitive range, if necessary.
All aspects of the proposals are subject to discussions, including cost,
technical approach, past performance, Small Disadvantaged Business
Participation Plan and contractual terms and conditions. Final Proposal
Revisions will be requested with the reservation of the right to conduct
limited negotiations after submission of the Final Proposal Revisions.

d\. A final best-buy analysis will be performed taking into
consideration the results of the technical evaluation, cost analysis,
past performance, and ability to complete the work within the
Government's required schedule. The Government reserves the right to
make an award to the best advantage of the Government, technical merit,
cost, past performance, and other factors considered.

e.  The Government reserves the right to make a single award, multiple
    awards, or no award at all to the RFP.

Number range CHAPTER**L.14 PROPOSAL INTENT/ APPROVAL FOR CONTACT
INFORMATION FOR BIDDERS LIST**

It is requested that if an offeror intends to submit a proposal to this
solicitation that the attached Proposal Intent Form (Attachment 2) be
completed and returned to the address indicated by July 21, 2005. The
submission of the intent form is not binding on an offeror to submit a
proposal, nor does the failure to submit the form prohibit an offeror
from submitting a proposal. The purpose is to provide us with an
estimated number of proposals to assist us in our planning and logistics
for proposal reviews.

We have added a request to include your contact information to a bidders
list. The bidders list will be provided to interested offerors for
subcontracting opportunities, as part of an amendment to the
solicitation. In order for AHRQ to include your contact information on
the bidders list, you must return the Proposal Intent Form and check the
box that grants permission to add your name no later than the date
listed above.

**[SECTION M - EVALUATION FACTORS FOR AWARD]{.underline}**

**[TECHNICAL EVALUATION CRITERIA]{.underline}**

Number range CHAPTERSelection of an offeror for contract award will be
based on an evaluation of proposals against four factors and award will
be made to that responsible offeror whose proposal is most advantageous
to the Government. The four factors are: scientific technical merit,
cost, past performance, and the Small Disadvantaged Business
Participation Plan. The scientific technical merit of the proposals will
receive paramount consideration in the selection of the Contractor(s)
for this acquisition. Offerors that submit technically acceptable
proposals will then be evaluated for past performance and for their
Small Disadvantaged Business Participation Plan. Following these
evaluations a competitive range will be determined.

All evaluation factors, other than cost or price, when combined are
significantly more important than cost or price. However, cost/price may
become a critical factor in source selection in the event that two or
more offerors are determined to be essentially equal following the
evaluation of all factors other than cost or price. In any event, the
Government reserves the right to make an award to that offeror whose
proposal provides the best overall value to the Government. The
Government reserves the right to make a single award, multiple awards,
or no award at all.

**[THE GOVERNMENT RESERVES THE RIGHT TO MAKE AN AWARD WITHOUT
DISCUSSION]{.underline}**

All proposals will be reviewed in accordance with the governing
regulations and AHRQ policies and procedures. The technical proposal,
past performance information and Small Disadvantaged Business
Participation Plan will be evaluated in terms of the offeror's responses
to each of the evaluation factors. Each proposal will be evaluated on
the likelihood of meeting the Government's requirements. The evaluation
will be based on the technical and administrative capabilities in
relation to the needs of the program, anticipated tasks, and the
reasonableness of costs shown in relation to the work to be performed.
The Government reserves the right to make an award to the best advantage
of the Government.

The evaluation factors and assigned weights which will be used in the
overall review of the offeror's proposal are outlined below. The
technical proposal shall consist of the responses to evaluation criteria
A through F (including subcriteria). The offeror should show that the
objectives stated in the proposal are understood and offer a logical
program for their achievement. The following criteria will be used to
evaluate proposals and will be weighted as indicated in establishing a
numerical rating for all proposals submitted. Factors facilitating the
evaluation of each criteria below are referenced in the corresponding
criteria found in Section L of this solicitation:

> **[Evaluation Criteria]{.underline} [Weight]{.underline}**

**A. Response to Statement of Work** **35**

Proposals prepared in response to the Statement of Work (SOW) will be
evaluated on the following:

1.  Completeness of the proposal.

2.  Offeror's demonstrated understanding of the overall goal of the
    Project and specific objectives and individual tasks of the SOW.

3.  Offeror's demonstrated understanding of the challenges of this work
    and their creative response to those challenges.

4.  Proposed technical approach to accomplishing each of the tasks in
    the SOW.

5.  Qualifications and experience of the personnel to be assigned to the
    various tasks, including key personnel, subcontractors, and
    consultants.

6.  Approach to identifying and obtaining input from Project Expert
    Panel members, Editorial Board, technical experts, and other
    relevant organizations and stakeholders.

7.  Extent to which the proposal demonstrates a capability to complete
    the tasks outlined in the SOW within the specified time frames and
    produce high-quality, accurate deliverables and information.

8.  Extent to which the proposal demonstrates an efficient use of staff
    and other resources to accomplish the required tasks.

**B. Staffing and Management Plan** **30**

> Proposals will be evaluated on the extent to which the Offeror
> demonstrates the availability of experienced and qualified personnel
> or staff, and through subcontracting or consulting arrangements, with
> the breadth and depth of innovation, methodological, technical,
> quality improvement/disparities reduction, and clinical expertise as
> specified in the technical proposal instructions.
>
> Proposals will also be evaluated on the effectiveness of the
> procedures proposed for managing tasks outlined in the SOW and meeting
> critical project milestones, including: efforts to ensure that
> qualified personnel will be available for individual tasks outlined in
> the SOW; plans for organizing, managing, and coordinating the
> respective roles and responsibilities of the various personnel;
> quality control processes; procedures to assure timely start-up and
> completion of work; methods to manage relationships with consultants,
> subcontractors; and members of the Project Expert Panel; and methods
> to ensure the efficient use of staff and other resources in conducting
> required tasks.

**C. Corporate/Organizational Experience** **25**

Proposals will be evaluated on the following criteria:

1.  Extent and relevance of prior experience related to innovating in
    health service delivery and implementing evidence-based practices
    and quality improvement initiatives.

2.  Extent and relevance of prior experience in development of health
    care related classifications schemes and relational databases.

3.  Extent and relevance of prior experience working with expert panels.

4.  Extent and relevance of prior experience in working with innovators
    of health service delivery, as well as, clinical organizations,
    including hospitals, health plans, State and local government
    agencies, and medical, nursing, and health professional specialty
    societies.

5.  Extent and relevance of prior experience successfully completing
    highly technical and complex projects.

6.  Experience developing and maintaining large, complex databases.

7.  Extent and relevance of prior experience developing user-friendly
    World Wide Web sites.

8.  Extent and relevance of prior experience in successfully completing
    projects of a similar nature and scale within the required time and
    budgetary constraints.

9.  Quality of submitted examples of prior work of a similar nature.

**D. Facilities and Equipment** **10**

Proposals will be evaluated on the extent to which the offeror
demonstrates adequate availability of the following:

1.  Appropriate software and hardware capabilities to complete the tasks
    required in the SOW.

2.  Duplicating and facsimile capabilities

3.  Capacity to access electronic databases necessary to search
    biomedical and health services literature and to obtain hard and
    electronic copies of measures.

4.  Capacity to communicate via electronic mail using state-of-the-art
    hardware and software.

5.  Capacity to teleconference using state-of-the-art equipment

**TOTAL POINTS BEFORE PAST PERFORMANCE AND**

**SMALL DISADVANTAGE BUSINESS UTILIZATION PLAN 100**

**E. Past Performance (20 points)**

> Offerors will be evaluated on their past performance (since January 1,
> 2002). Completed questionnaires will provide a basis for determining
> past performance evaluation as well as information obtained from the
> references listed in the proposal, other customers known to the
> Government, consumer protection organizations, and others who may have
> useful and relevant information. Information will also be considered
> regarding any significant subcontractors and key personnel records.
>
> The offerors= past performance will be evaluated on the basis of the
> following factors:
>
> a\. Quality
>
> How well has the offeror conformed to the performance standard in
> providing the services or achieving the stated objective(s) of
> contracts or grants? Quality will be evaluated by the personnel
> provided, the level of effort agreed to in the contract statement of
> work or grant, quality of final products (e.g., findings, tools),
> implementation activities, dissemination and activities that promote
> turning research into action (i.e. promoting uptake of innovation).
>
> b\. Timeliness
>
> How well has the offeror adhered to timetables and delivery schedules
> in providing the required services or products? Consideration is given
> to the offeror's efforts to recommend and/or take corrective actions
> to keep work on schedule.
>
> c\. Business Relations/ Customer satisfaction
>
> The offeror will be rated on professional and cooperative behavior
> with the client.

d.  Cost control

> The offeror will be rated on the ability to set reasonable budgets
> within contracting or grant guidelines and adhere to them in
> conducting research.
>
> Assessment of the offeror=s past performance will be one means of
> evaluating the credibility of the offeror=s proposal, and relative
> capability to meet performance requirements.
>
> Evaluation of past performance will often be quite subjective based on
> consideration of all relevant facts and circumstances. It will not be
> based on absolute standards of acceptable performance. The Government
> is seeking to determine whether the offeror has consistently
> demonstrated a commitment to customer satisfaction and timely delivery
> of services and quality products at fair and reasonable prices.
>
> The assessment of the offeror=s past performance will be used as a
> means of evaluating the relative capability of the offeror and the
> other competitors. Thus, an offeror with an exceptional record of past
> performance may receive a more favorable evaluation than another whose
> record is acceptable, even though both may have acceptable technical
> proposals.
>
> By past performance, the Government means the offeror=s record of
> conforming to specifications and to standards of good workmanship; the
> offeror's record of forecasting and controlling costs; the offeror=s
> adherence to contract schedules, including the administrative aspects
> of performance; the offeror=s reputation for reasonable and
> cooperative behavior and commitment to customer satisfaction; and
> generally, the offeror=s business-like concern for the interest of the
> customer.
>
> The Government will consider the number or severity of an offeror=s
> problems, the effectiveness of corrective actions taken, the offeror=s
> overall work record, and the age and relevance of past performance
> information.
>
> If the offeror or the proposed employees for the offeror, do not have
> a past performance history relative to this acquisition, or past
> performance not relative to this acquisition, the offeror will not be
> evaluated favorably or unfavorably on this factor. A neutral rating
> will be determined.
>
> The Government reserves the right to evaluate relevant past
> performance information not specifically provided by the offeror.
>
> In evaluating past performance the Government, will consider the
> offeror's effectiveness in quality of products or services; timeliness
> of performance; cost control; business practices; customer
> satisfaction, and key personnel past performance.
>
> [NOTICE: Past Performance questionnaires are to be provided to the
> Contracts Office NO LATER than the closing date and time for receipt
> of proposals. It is the offeror's responsibility to ensure that these
> documents are forwarded to the Contract Office (FAX
> 301-427-1740)]{.underline}

**F. Number range CHAPTER[Small Disadvantaged Business Participation
Plan]{.underline} 5 points**

> The evaluation will be based on information obtained from the plan
> provided by the offeror, the realism of the proposal, other relevant
> information obtained from named SDB concerns, and any information
> supplied by the offeror concerning problems encountered in SDB
> participation.
>
> Evaluation of the SDB Participation Plan will be a subjective
> assessment based on a consideration of all relevant facts and
> circumstances. It will not be based on absolute standards of
> acceptable performance. The Government is seeking to determine whether
> the offeror has demonstrated a commitment to use SDB concerns for the
> work that it intends to perform as the prime contractor.
>
> The assessment of the offeror's SDB Participation Plan will be used as
> a means of evaluating the relative capability and commitment of the
> offeror and the other competitors. Thus, an offeror with an
> exceptional record of participation with SDB concerns may receive more
> points and a more favorable evaluation than another whose record is
> acceptable, even though both may have acceptable technical proposals.
>
> SDB participation will be scored with offerors receiving points from 0
> to 5, with 5 being the most favorable.

**TOTAL AVAILABLE POINTS [125]{.underline}**

**ATTACHMENT 1 STATEMENT OF WORK WITH APPENDICES**

**INNOVATIONS CLEARINGHOUSE**

1.  **BACKGROUND**

    1.  [Authorizing Legislation]{.underline}

> The Agency for Healthcare Research and Quality (AHRQ) was established
> in 1989, by Public Law 101-239, as the Agency for Health Care Policy
> and Research. Re-authorizing legislation passed in November 1999[^1]
> establishes AHRQ's mission to improve the quality, safety, efficiency,
> and effectiveness of health care for all Americans. In its role as the
> lead Federal agency for health care research and quality, AHRQ has two
> principal responsibilities:

a.  To sponsor and conduct research which develops evidence-based
    information on health care outcomes, quality, safety, cost, use, and
    access that can be used by decision makers in the public and private
    sectors; and

b.  To ensure that information resulting from its research,
    demonstration, and evaluation activities is disseminated rapidly,
    widely, and in a readily usable form.

> The information helps health care decision-makers ---patients and
> clinicians, health system and hospital leaders, purchasers, and
> policymakers --- make more informed decisions and improve the quality
> of health care services.

2.  [AHRQ Activities]{.underline}

> Legislation re-authorizing AHRQ specifically directs it to support
> activities that:

-   **Establish health care improvement research centers and networks.**
    AHRQ is to employ research strategies and mechanisms that will link
    research directly with clinical practice in order to address the
    full continuum of care and outcomes research, to link research to
    practice improvement, and to speed the **dissemination** of research
    findings to community practice settings.

-   **Develop information on health care quality.** AHRQ is to
    coordinate, conduct, and support research, demonstrations, and
    evaluations related to the measurement and improvement of health
    care quality. AHRQ is also to disseminate scientific findings about
    what works best and facilitate public access to information on the
    quality of, and consumer satisfaction with, health care.

-   **Advance the use of information technology for coordinating patient
    care and conducting quality and outcomes research.** AHRQ is to
    promote the use of information systems to develop and disseminate
    quality measures and tools; create effective linkages between health
    information sources to enhance health care delivery and coordination
    of evidence-based health care services; and promote protection of
    individually identifiable patient information used in health
    services research and health care quality improvement.

-   **Promote innovation in evidence-based health care practices.** AHRQ
    is to conduct and support research on the development, diffusion,
    and use of health care technology; develop, evaluate, and
    disseminate methodologies for assessments of health care practices;
    and promote education and training and provide technical assistance
    in the use of health care practice and technology assessment
    methodologies and results.

    1.  [Putting Research Into Practice]{.underline}

> AHRQ has moved from disseminating or transferring information to
> encouraging **stakeholders** to put research findings into practice
> and in policy making. In 1999, in support of increasing the impact of
> its findings, AHRQ initiated the Translating Research Into Practice
> (TRIP) program, which was designed to evaluate strategies for moving
> research findings into clinical practice. Many other programs have
> been funded, such as the 2002 Partnerships for Quality initiative,
> that strive to speed up the process of diffusing effective,
> evidence-based innovations. An **innovation** is an idea, practice, or
> object that is perceived as new by an individual, or other unit of
> **adoption**.[^2]
>
> Driven by its mission, and commitment to advancing excellence in
> health care, AHRQ continues to focus on applied processes of
> **diffusion** and **implementation** of innovations, and the related
> fields of knowledge transfer or knowledge exchange (a two-way
> transfer).

***Note:*** *words in bold are defined in a glossary, included as
**Appendix A**.*

2.  [Promoting Innovation and Change]{.underline}

> AHRQ is seeking to increase the implementation of health care
> improvement and disparities reduction strategies and innovations in
> the United States. Promoting innovation and change in the health care
> system is vital to providing the highest quality health care possible.
> **Innovation in health care delivery** has been defined as "a novel
> set of behaviors, routines and ways of working, which are directed at
> improving health outcomes, administrative efficiency,
> cost-effectiveness, or user experience, and which are implemented by
> means of planned and coordinated action."[^3]
>
> [Examples.]{.underline} Three (3) examples of innovations in health
> care delivery are provided.
>
> \(1\) The inpatient rapid response team. In this innovation, inpatient
> clinicians are charged with responding to the first sign of patient
> decline reported by nursing, paramedical, and/or medical staff. This
> innovation was first described in a preliminary study published in
> 2002 by Buist et al[^4], and then by Bellomo et al[^5], whose study
> found statistically significant decreases in serious post-operative
> adverse events, mortality, and duration of hospital stay. Hospitals
> throughout the U.S. are implementing this innovative service,
> modifying the approach at the local level, and reaping similar results
> as those described in the literature.[^6]
>
> \(2\) Interim Care Program for the Homeless. This innovation involved
> the establishment of a shelter where people that are homeless and need
> care after a hospitalization are allowed to stay 24 hours a day,
> promoting continuity of care. This type of shelter had not been
> available in the California region in which it was implemented. Before
> this program was created, homeless people leaving the hospital
> presented to traditional shelters looking for post-discharge care and
> support or didn't seek it at all. This program came to fruition
> through collaboration among competing hospitals, and involved advocacy
> groups. Financial support was provided by the hospitals and the State
> of California. While the success of the program has not been formally
> measured, goals of the program appear to have been met.[^7]
>
> \(3\) An intensive care unit catheter-related blood stream infection
> elimination strategy. In this innovation, Berenholtz et al, tackled
> catheter-related bloodstream infections by educating the staff;
> creating a catheter insertion cart; asking providers daily whether
> catheters could be removed; implementing a checklist to ensure
> adherence to evidence-based guidelines for preventing catheter-related
> bloodstream infections; and empowering nurses to stop the catheter
> insertion procedure if a violation of the guidelines was observed.
> This innovation helped to ensure adherence with evidence-based
> infection control guidelines and nearly eliminated catheter-related
> bloodstream infections in a surgical intensive care unit.[^8]
>
> For this solicitation, the innovations of interest to AHRQ are those
> related to health service delivery: the novel processes and structural
> ways of providing health services toward improving quality or reducing
> disparities. Innovations in diagnostic and treatment devices,
> procedures, techniques, and pharmaceuticals are not of interest in
> this solicitation.

3.  [Problem]{.underline}

> Given the complexity of the American health care system (e.g.,
> facilities, operations, personnel, budget and financing, information
> technology, regulatory/accreditation) along with its varying stages of
> modernization, implementation of evidence-based practices and quality
> improvement and disparities reduction strategies requires innovation
> and change. The strategies and the changes needed to implement the
> strategies could each be innovative. Tools developed to facilitate
> innovation and implementation could be innovative. Diffusion of these
> innovations is slow. Lack of a standardized approach to document these
> innovations and a central location in which to find them adds to the
> diffusion-delay.

4.  [A Solution]{.underline}

> The Internet is used as a vehicle to hasten the diffusion and spread
> of information, and to connect people world-wide. It's possible, then,
> that the Internet, along with opportunities for personal interaction,
> could be part of a solution to the problem of slow diffusion of health
> care system and practice innovations. It is AHRQ's intention to
> establish a **clearinghouse** of innovations in health service
> delivery (referred to in this solicitation as the Innovations
> Clearinghouse, for ease of communication) that is appropriately
> designed, backed by content-rich databases and a powerful search
> engine, and features communication venues geared toward adult learners
> with hectic schedules, many responsibilities, and few resources. The
> Web-based Innovations Clearinghouse will 1) facilitate collection of
> innovative practices and present them in a standardized format, 2)
> promote their dissemination, replication, adaptation, and use, and 3)
> foster innovation.

5.  [Stakeholder Need for the Innovations Clearinghouse]{.underline}

> In various meetings with Agency stakeholders over the course of the
> past year, AHRQ has learned that those desiring to make changes in
> their institutions or practices toward a goal of improving quality and
> reducing disparities cannot find in one place the detailed information
> needed to assist them. One of those meetings was held specifically to
> explore the concept of an Innovations Clearinghouse. See **Appendix
> B** for the related meeting summary. From these meeting participants,
> AHRQ learned that although there are Web sites focusing on strategies
> proven to improve health care quality or reduce disparities, none
> seemed to combine all the elements that users found desirable. These
> potential users expressed value in 'one-stop shopping' for
> innovations, adding this Clearinghouse should be a portal to others'
> Web sites (many sites have a single focus, e.g., hospital, Medicare,
> etc...). These potential users voiced enthusiasm for a searchable
> database of innovations where they could compare and assess strategies
> for potential applicability to a similar, or a completely different,
> setting and environment. Providing information in sufficient detail
> for replication was important to these users. Also, providing access
> to the tools used to innovate and implement quality improvement
> strategies was expressed by the meeting participants as equally
> important as characterizing the innovation. Having the Clearinghouse
> designed so that users could upload their own innovations for
> inclusion in the Clearinghouse was expressed as important. Online or
> offline networking and communication opportunities, as a method of
> learning about a particular innovation, would be valued features of
> this Clearinghouse. Finally, a Web site capturing health system or
> practice innovations that were used (trialed, piloted, or incorporated
> enterprise-wide) and failed to result in health improvements, with the
> reasons why it failed, would provide educational opportunities not
> available elsewhere. The research community would find this valuable,
> also.

6.  [This Contract]{.underline}

> This contract is for the design, development, deployment, operation,
> maintenance, and evaluation of an Internet repository for the
> collection, classification, and distribution of health care practice
> and health care system innovations and related information and tools.
> The Innovations Clearinghouse (until renamed) will be a searchable
> database of innovative strategies used to improve health care quality
> and reduce health care disparities.
>
> The Innovations Clearinghouse will foster innovation in health service
> delivery by empowering its targeted audience of potential and current
> "**change agents**" and "**positive deviants**" (such as physicians,
> nurses, and other health professionals, health care providers, health
> plans, integrated delivery systems, purchasers and others) with a
> national-level, publicly accessible mechanism for obtaining objective,
> detailed information on healthcare innovations and tools (including
> innovators' contact information). The Innovations Clearinghouse will
> promote the dissemination, replication, adaptation, and use of
> innovations. The information will include standard innovation
> attributes and related items, such as tools and lessons learned. This
> Web site will provide online or offline networking and communication
> opportunities, and it will provide descriptions of "change packages"
> -- sets of innovations implemented simultaneously and associated
> tools. The Clearinghouse database will cover the broad spectrum of
> settings, systems, and providers. Public health priority
> diseases/conditions, **priority populations**, and efforts to reduce
> disparities in quality will be highlighted.
>
> This contract is also for the maintenance and enhancement of AHRQ's
> QualityTools database. Enhancements to the QualityTools database will
> be those that support the Innovations Clearinghouse.
>
> Under this contract, the QualityTools Web site,
> [www.qualitytools.ahrq.gov](http://www.qualitytools.ahrq.gov/), will
> be maintained until the Innovations Clearinghouse Web site is
> launched. Tools will be continued to be added/updated to the
> QualityTools Web site during this timeframe.
>
> In designing the Innovations Clearinghouse Web site, the Contractor
> shall consider feedback received on the QualityTools Web site.
>
> The Innovations Clearinghouse Web site will include display of the
> innovations and tools databases and related information. That is, the
> Innovations Clearinghouse Web site will access **back-end**
> database(s) containing quality tools and innovations. Some innovations
> will have tools, some will not. Some tools will have been used in an
> innovative context, some will not. When relationships exist between
> tools and innovations, these will be explicitly presented at the
> Innovations Clearinghouse Web site. The innovation identified in
> example 3 above (in section 1.4) has tools associated with it (e.g.,
> checklist). The checklist, if included in the tools database, would be
> presented to the user through a link from the innovation. If browsing
> tools, when this tool is selected, the associated innovation would be
> presented to the user. Relationships have been established between
> tools in the QualityTools database and guidelines in AHRQ's National
> Guideline Clearinghouse and these will need to be maintained. The
> existing QualityTools Web-crawler feature will be imported into the
> Innovations Clearinghouse, with a redesigned interface, for use in
> finding Web pages from specific Web sites that describe applied or
> researched innovations and approaches to transformational change
> (e.g., Institute for Healthcare Improvement \[IHI\], Veterans Affairs
> Quality Enhancement Research Initiative \[VA QUERI\], Medicare Quality
> Improvement \[MedQIC\], Substance Abuse and Mental Health Services
> Administration \[SAMHSA\], Health Resources and Services
> Administration \[HRSA\] Disparities Collaboratives). Alternative
> Web-crawler engines can be proposed along with accompanying rationale.

7.  [Online Submissions]{.underline}

> The Innovations Clearinghouse will allow innovators to submit online
> their innovations and lessons learned to the database so that others
> can learn from them. (The U.S. Department of Homeland Security hosts a
> Web site for emergency preparedness and first-response professionals
> of lessons learned with a similar goal, <https://www.llis.dhs.gov/>.)
> Submissions will be sought for innovations that worked (met or
> surpassed goals) and didn't work (goals not met). All submitters will
> be asked to provide valuable lessons learned from the change
> initiative. All submissions to the Innovations Clearinghouse will be
> reviewed before posting; filters or inclusion criteria will be
> established to promote consistency, fairness, and usable, high-quality
> content.

8.  [Successful AHRQ Web Sites as Models]{.underline}

> The Innovations Clearinghouse will build off of the successes of three
> AHRQ databases, Web sites, and opportunities for learning: National
> Guideline Clearinghouse,
> [www.guideline.gov](http://www.guideline.gov/); National Quality
> Measures Clearinghouse,
> [www.qualitymeasures.ahrq.gov](http://www.qualitymeasures.ahrq.gov/);
> and AHRQ WebM&M, [www.webmm.ahrq.gov](http://www.webmm.ahrq.gov/).
>
> However, the design of the Innovations Clearinghouse Web site will not
> be restricted by these successful Web sites.
>
> In addition, AHRQ has Web site and infrastructure compliance
> requirements that are required to be followed; these are captured in
> Task 3.5 and include the AHRQ Web Product Development Guidelines and
> Guidelines for Developing AHRQ Tools, provided as **Appendix C and
> D**, respectively. Refer to Task 3.5 for details.

9.  [National Guideline Clearinghouse™]{.underline}

> Particularly relevant to this solicitation, AHRQ has developed the
> National Guideline Clearinghouse™ (NGC), in partnership with the
> American Medical Association and the American Association of Health
> Plans (now America\'s Health Insurance Plans \[AHIP\]). The NGC is an
> Internet-based repository of clinical practice guidelines allowing
> detailed comparisons across different guidelines. All guidelines
> included in NGC meet explicit criteria. The contents of NGC include
> structured summaries of guidelines using controlled terms and
> descriptive free-text fields. The structured summaries capture
> bibliographic, clinical, evidentiary and methodological information on
> each guideline. They are indexed using UMLS concepts. They are
> available in brief and complete formats, and in XML, downloadable PDA,
> Microsoft Word®, and Adobe® pdf versions. The site is pocket PC
> compatible. Each guideline summary contains a copyright statement
> created by the guideline developer. NGC allows users to create
> comparisons of structured guideline summaries online. Also, in most
> cases, the full-text of each guideline is available via hyperlink to
> an external site or they are hosted on the NGC site. In instances
> where the guideline developer does not make the full-text available,
> ordering information is posted. The NGC database contains information
> on over 1,800 clinical practice guidelines from more than 200
> organizations world-wide; new guidelines are added every week. The
> site has been operational since December 15, 1998 and has seen a
> steady increase in use. In February 2006, the site received
> approximately 600,000 visits (averaging more than 21,000 visits/day).

10. [National Quality Measures Clearinghouse™]{.underline}

> AHRQ's National Quality Measures Clearinghouse™ (NQMC) is an
> Internet-based repository of evidence-based healthcare quality
> measures and measure sets allowing detailed comparisons across
> different measures. All measures included in NQMC meet explicit
> criteria. NQMC is similar in design and functionality to NGC. The
> contents of NQMC include structured summaries of measures using
> controlled terms and descriptive free-text fields. These summaries
> provide an easy-to-use mechanism for obtaining standardized, detailed
> bibliographic, clinical, evidentiary, and methodological information
> on quality measures. They are available in brief and complete formats,
> and in XML, downloadable PDA, Microsoft Word®, and Adobe® pdf
> versions. The site is pocket PC compatible. Each summary contains a
> copyright statement from the measure developer. Formally launched on
> February 19, 2003, the NQMC Web site now includes structured summaries
> of more than 750 measures from more than 30 organizations and
> individuals, and continues to grow each week. Visits to the site
> continue to grow, too. In February 2006, the site received
> approximately 72,000 visits (averaging more than 2500 visits/day).

11. [AHRQ WebM&M (Morbidity and Mortality on the Web)]{.underline}

> AHRQ WebM&M Web site (Morbidity and Mortality on the Web) is the
> Nation's first Web-based online journal and forum on patient safety.
> The site features expert analysis of medical errors reported
> anonymously by readers, interactive learning modules on patient safety
> (\"Spotlight Cases\"), Perspectives on Safety, forums for online
> discussion, and a "Did You Know" section that provides pithy
> information on a specific medical error or patient safety topic. Every
> month, three interesting cases of medical errors and patient safety
> problems are posted. These cases are submitted through the Web site.
> Writers of cases selected for posting receive an honorarium. Cases are
> compelling and raise key issues about patient safety and systems
> improvement. An editorial team, guided by an Editorial Board, assess
> cases by rating them against explicit criteria. Cases are selected
> based on their ratings and areas of interest.
>
> One of the cases each month is expanded into a "Spotlight Case"---an
> interactive learning module that features readers' polls, quizzes, and
> other multimedia elements. A Powerpoint version of the spotlight case
> is available to download for educational use, such as in student
> teaching conferences, residents' reports, and in nursing education.
>
> The remaining cases each month are followed by a commentary, written
> by an expert in the relevant clinical or patient safety field. The
> format marries an evidence-based approach with anecdotal observations
> and qualitative analysis to produce a lively discussion, which also
> takes advantage of the graphical capabilities of the Internet.
>
> The site includes a users' forum, on which readers can post and react
> to comments that relate to the cases. The site also provides links to
> other resources and interesting facts about patient safety, medical
> errors, and healthcare quality.
>
> AHRQ WebM&M was launched in early February 2003 and has published more
> than 150 cases as of February 2006. In February 2006, the site had
> 46,251 unique visitor sessions for the month representing an average
> of 1,652 unique visits daily.

12. [QualityTools]{.underline}

> QualityTools was established as a database and an Internet-based
> repository of practical, ready-to-use tools (e.g., databases, reports,
> guides, workbooks, and other resources) for measuring, promoting, and
> improving the quality of healthcare. The QualityTools mission is to
> provide health care providers, health plans, policymakers, purchasers,
> patients and consumers an accessible mechanism to implement quality
> improvement recommendations, initiatives, or principles and to improve
> the delivery and receipt of care, inform health care decisions, and
> educate individuals regarding their own health care needs. All tools
> included in QualityTools meet explicit criteria. The contents of
> QualityTools include structured summaries of tools using controlled
> terms and descriptive free-text fields. These summaries provide an
> easy-to-use mechanism for obtaining standardized information on tools,
> including hyperlinks to each quality tool hosted on external Web site.
> A feature unique to the QualityTools site is a **Web** **crawler**.
> When performing a search of the QualityTools site, results include
> relevant tools in the database as well as relevant Web pages from the
> tool developer's Web site. Those relevant Web pages are identified
> through the Web crawler.
>
> Web-enabled versions of the National Healthcare Quality Report (NHQR),
> NHQR State Resources, and the National Healthcare Disparities Report
> (NHDR) are currently available through the QualityTools Web site. With
> this contract, the 2005 and archived 2004 and 2003 versions of these
> Reports and Resources will continue to be available through the
> QualityTools Web site. It is expected that a new Contractor will take
> over hosting of the 2005 and archived 2004 and 2003 Reports and
> Resources in 2007, after that Contractor has Web-enabled the 2006
> versions. Therefore, the 2006 and future versions of the NHQR, NHDR,
> and NHQR State Resources will not be available through the
> QualityTools Web site.
>
> QualityTools was launched on December 22, 2003. It contains more than
> 800 tools. In February 2006, there were approximately 78,000 visits to
> the QualityTools site, which includes visits to the NHQR and NHDR
> sub-sites.

13. [AHRQ Programs That Support Innovations in Health Service
    Delivery]{.underline}

> AHRQ is supporting research intended to accelerate change and
> transformation in health service delivery, patient safety, and health
> information technology. Three AHRQ programs will function as integral
> content feeds to the Innovations Clearinghouse: ACTION (Accelerating
> Change and Transformation in Organizations and Networks), AHRQ Patient
> Safety Network (AHRQ PSNet), and AHRQ Health IT. These are described
> below.
>
> An AHRQ initiative that could also serve as a content feed for the
> Innovations Clearinghouse is the National Health Plan Learning
> Collaborative to Reduce Disparities and Improve Quality. This is a
> public-private partnership bringing together nine of the nation's
> largest health insurance plans to go beyond research and actively
> tackle racial and ethnic inequities in health care service delivery.

14. [ACTION]{.underline}

> Accelerating Change and Transformation in Organizations and Networks
> (ACTION) is the successor to the Integrated Delivery System Research
> Network (IDSRN), a 5-year implementation initiative specifically
> developed by AHRQ to capitalize on the research capacity of, and
> research opportunities in, integrated delivery systems. ACTION is a
> model of field-based research that fosters public-private
> collaboration in rapid-cycle, applied research.
>
> ACTION promotes innovation in health care delivery by accelerating the
> development, implementation, diffusion and uptake of demand-driven and
> evidence-based products, tools, strategies and findings. ACTION
> develops and diffuses scientific evidence about what does and does not
> work to improve health care delivery systems. It provides an
> impressive cadre of delivery-affiliated researchers and sites with a
> demonstrated capacity to "turn research to practice" along with the
> means of testing the application and uptake of the research knowledge.
> Details on the ACTION solicitation can be viewed in AHRQ's online
> contract archive at:
> [www.ahrq.gov/fund/contraarch.htm](http://www.ahrq.gov/fund/contraarch.htm).

15. [AHRQ Patient Safety Network (PSNet)]{.underline}

> AHRQ Patient Safety Network (PSNet) is a new national web-based
> resource featuring the latest news and essential resources on patient
> safety. The site offers weekly updates of patient safety literature,
> news, tools, and meetings ("What's New"), and a vast set of carefully
> annotated links to important research and other information on patient
> safety ("The Collection"). Supported by a robust patient safety
> taxonomy and web architecture, AHRQ PSNet provides powerful searching
> and browsing capability, as well as the ability for diverse users to
> customize the site around their interests (My PSNet). It also is
> tightly coupled with AHRQ WebM&M, the popular monthly journal that
> features user-submitted cases of medical errors, expert commentaries,
> and perspectives on patient safety.
>
> Searching the AHRQ PSNet Web site for innovation, identifies a number
> of potential content leads for the Innovations Clearinghouse.
>
> Additional information about AHRQ PSNet is available at the AHRQ PSNet
> Web site, <http://www.psnet.ahrq.gov/about.aspx>.

16. [AHRQ Health IT]{.underline}

> AHRQ\'s initiative on health information technology (health IT) is a
> key element to the nation\'s 10-year strategy to bring health care
> into the 21st century by advancing the use of information technology.
> The AHRQ initiative includes more than \$166 million in grants and
> contracts in 41 states to support and stimulate investment in health
> IT, especially in rural and underserved areas. Through these and other
> projects, AHRQ and its partners will identify challenges to health IT
> adoption and use, solutions and best practices for making health IT
> work, and tools that will help hospitals and clinicians successfully
> incorporate new IT.
>
> As part of the health IT initiative, AHRQ created the AHRQ National
> Resource Center for Health Information Technology (the National
> Resource Center) to help the health care community make the leap into
> the Information Age. In addition to providing technical assistance,
> the National Resource Center shares new knowledge and findings that
> have the potential to transform everyday clinical practice. AHRQ\'s
> National Resource Center is committed to advancing our national goal
> of modernizing health care through the best and most effective use of
> IT.
>
> As new findings and data become available to the health IT community
> from both the AHRQ portfolio of projects and other sources, they will
> be shared on the AHRQ Health IT Web site through future releases of
> the National Resource Center Health IT Knowledge Repository.
>
> The Knowledge Library contains both evidence-based and theoretic
> content gathered by health IT experts. The content is organized into
> two categories: Core Collection and Partner Contributions. The Core
> Collection contains items found by AHRQ National Resource Center
> experts to be of exceptional quality and central to the health IT
> discipline. Partner Contributions include content provided by
> professional societies and non-profit organizations experienced in
> health IT.
>
> Additional information about AHRQ Health IT is available at
> [www.healthit.ahrq.gov](http://www.healthit.ahrq.gov/).
>
> While health IT innovations are not the focus of the Innovations
> Clearinghouse, AHRQ does not want to exclude them either. Health IT
> innovations represented in the Innovations Clearinghouse will be those
> that occurred concomitantly with other health service delivery
> innovations. The AHRQ Health IT Web site, especially the Knowledge
> Library, will serve as resource to identify potential content for the
> Innovations Clearinghouse.

17. [Challenges]{.underline}

> Eight significant challenges exist in the development of an
> Innovations Clearinghouse:

1)  Design a Web site that meets the unique needs of the targeted
    audience of the Innovations Clearinghouse;

2)  Identify health system or health practice innovations and
    innovators, and stimulate, perhaps generate, and collect a
    comprehensive accumulation of innovations highlighting those
    involving public health priority diseases/conditions, priority
    populations, and efforts to reduce disparities in quality;

3)  Develop innovation inclusion and exclusion criteria keeping in mind
    a requirement to include evidence-based (scientifically tested)
    health service innovations while understanding the value in
    including experience-based innovations (i.e., those not subjected to
    scientific study), the outcome of which may or may not have been
    formally evaluated;

4)  Identify and define the attributes of an innovation and
    organizational climate that should be captured in the database and
    find balance in obtaining enough information to assist potential
    implementers in assessing potential applicability and replication in
    their own settings with that of overburdening the submitters with
    time-consuming submission requirements;

5)  Present "change (innovation) packages" and identify relationships,
    if any, between innovations, and between innovations and tools and
    display those relationships on the Web site;

6)  Determine how lessons learned in innovating, implementing, and
    changing can be captured;

7)  Provide dynamic opportunities for networking and education among
    potential implementers and submitting innovators; and

8)  Maintain the existing QualityTools Web site until a new Innovations
    Clearinghouse Web site is established that permits presentation of
    both the tools and the innovations databases and related
    information.

```{=html}
<!-- -->
```
2.  **OBJECTIVES**

    1.  The goal of the Innovations Clearinghouse is to be a national
        repository of health system and health practice innovations,
        associated tools, and lessons learned, promoting change and
        further innovations and tool development in the health care
        system for the purpose of improving health care quality and
        reducing health care disparities for all Americans.

    2.  The contract objectives are to:

-   Maintain AHRQ's QualityTools database, and establish relationships
    between tools in QualityTools and innovative practices detailed in
    the Innovations Clearinghouse while maintaining relationships with
    other databases (such as that between tools in QualityTools and
    guidelines in AHRQ's National Guideline Clearinghouse)

```{=html}
<!-- -->
```
-   Stimulate, perhaps generate, and collect a comprehensive
    accumulation of innovations and lessons learned

-   Establish a project expert panel that will provide feedback and
    guidance, and to establish an editorial board with capacity to
    review submissions and write expert commentary critiquing innovation
    successes and failures, offering insights and addressing the nuances
    of innovation, implementation, and change

```{=html}
<!-- -->
```
-   Design the Innovations Clearinghouse incorporating feedback provided
    on the existing QualityTools Web site, and develop, test, deploy,
    maintain, and evaluate this national Web-based repository of health
    care system and practice innovations, change initiatives, change
    packages, and associated tools and lessons learned; import
    QualityTools' Web crawler feature with a redesigned interface to
    assist users in finding relevant innovations, change initiatives,
    packages and tools at specific, complementary Web sites for 'one
    stop' searching (or suggest an alternative Web crawler accompanied
    by rationale)

-   Emphasize innovations targeting public health priority
    diseases/conditions, priority populations, and efforts to reduce
    disparities in quality

-   Provide appropriate and timely technical assistance to users

-   Provide dynamic opportunities for learning and networking among the
    users of the Web site

3.  **SPECIFIC REQUIREMENTS**

    1.  **Recruit, Manage, and Support a Project Expert Panel**

> **Recruit a Group of Experts**

1.  The Contractor shall work with the Project Officer to assemble a
    multidisciplinary panel of experts, not to exceed twelve (12)
    members, to provide feedback and guidance to the Contractor on broad
    project areas including:

-   the state-of-the-art in health system and practice innovations and
    change

-   the unique needs of Innovations Clearinghouse users

-   methods for obtaining the maximum number of health service delivery
    innovations possible for inclusion into the Innovations
    Clearinghouse

-   mechanisms for reaching the Innovations Clearinghouse's target
    audience

2.  Final expert nominations shall be provided to the AHRQ Project
    Officer for review and comment no later than two (2) months after
    the effective date of the contract (EDOC). The criteria for
    selection shall include factors such as knowledge of issues
    concerning public and private health service delivery improvements;
    current activity and research in the area of health system redesign
    and analysis or change management; ability to represent the needs of
    a significant segment of the Innovations Clearinghouse target
    audience; and ability and interest in fostering public-private
    partnerships.

> Members of the project expert panel shall represent a broad range of
> interests related to health service delivery innovations, health
> informatics, and professional education. Representatives may include
> health system and practice innovators and implementers of quality
> improvement or disparities reduction strategies; change management and
> quality improvement methodologists and researchers; experts in
> knowledge transfer and Web-based exchange; health systems design and
> analysis experts; health informatics experts; experts in
> organizational transformation; accreditation, health professional, and
> insurance organizations; and users such as clinicians, quality
> assurance specialists or organizations, managed care organizations,
> hospitals, employee benefit managers, and State and Federal government
> programs. No more than two (2) of the twelve (12) panel members will
> be Federal experts. The Government will be responsible for identifying
> and recruiting the appropriate Federal experts. The Project Officer
> will serve on the panel in an *ex officio* status.
>
> A single individual may represent expertise in more than one domain.
>
> A member of the ACTION Advisory Panel (being formulated at the time of
> this writing) may also serve on the Innovations Clearinghouse Project
> Expert Panel.

3.  The Contractor shall finalize recruitment of experts by the
    third (3) month of the project.

> **Establish and Maintain an Editorial Board**

4.  The Contractor shall propose, establish with AHRQ approval, and
    maintain an editorial board with capacity to review submissions and
    write expert commentary on select innovations and/or lessons
    learned. Their comments shall critique innovation successes and
    failures as they relate to systems, health care professional, and
    climate issues (cultural, regulatory, legal, etc...), offering
    insights not obvious from the information provided by the submitters
    and educating users on the nuances of innovation, implementation and
    change. Candidates for the editorial board, which can include
    members of the Project Expert Panel, shall be selected based on
    their national recognition and relevant background and
    responsibilities to include, for example, health service delivery
    innovation, health care quality improvement, quality improvement
    implementation research, health care disparities reduction, change
    management, health professions education, social and/or behavorial
    sciences applied to health service delivery, health care
    administration, informatics, human factors engineering, etc. The
    editorial board can be voluntary or paid, as reimbursable costs to
    the Contract. The Contractor shall finalize the editorial board by
    the third (3) month of the project.

> **Hold expert panel meetings**

5.  The Contractor shall hold and provide support (i.e., pre-, post-,
    and interim support) for Project Expert Panel meetings. The Project
    Expert Panel is expected to meet at least one time in each year of
    the contract. It is also anticipated that quarterly conference calls
    of select panel members and periodic electronic and paper
    correspondence will occur between meetings. The purpose of the
    meetings and communications will be to provide feedback to the
    Contractor on issues related to broad project areas including the
    state-of-the-art in health system and practice innovations and
    change; the unique needs of the Innovations Clearinghouse users;
    methods for obtaining the maximum number of innovations possible for
    inclusion into the Innovations Clearinghouse; and mechanisms for
    reaching the Innovations Clearinghouse target audience.

6.  In consultation with the Project Officer, the Contractor shall plan
    all activities in support of all meetings and inter-meeting
    conference calls and mailings. Specifically, the Contractor shall
    plan agendas; assemble, prepare, print, and distribute materials
    needed for meetings; prepare audio-visual materials as required;
    distribute materials to meeting participants; reserve meeting
    facilities and hotel accommodations; notify members and confirm
    participation in the meetings; arrange travel and process expense
    vouchers for non-Federal participants; and prepare detailed written
    summaries of the meetings. All agendas and materials will be
    provided to the Project Officer for review, comment, and approval at
    least 21 calendar days prior to the meeting, and all materials shall
    be sent to the Project Expert Panel and participants at least 14
    calendar days before the meeting.

7.  The meetings will be one day in length and held on a date in which
    at least 90 percent of expert members can attend. Federal experts
    will be responsible for their own travel arrangements and lodging
    costs. The Contractor shall provide hotel reservations at the
    Federal Government lodging per diem for Federal participants. All
    non-Federal expert members will be reimbursed at the rate of \$200
    per day for their participation in the meetings. It is estimated
    that all of the Expert Panel members will require overnight
    accommodations for one (1) evening for each meeting.

8.  The Contractor shall provide a detailed written draft meeting
    summary within two weeks after the expert meeting. The summary
    should include names and titles of participants and observers,
    agenda, and a substantive, detailed summary of the discussions,
    action items, and recommendations. The Contractor shall provide a
    final summary one (1) week after receiving Agency comments.

```{=html}
<!-- -->
```
2.  **Establish, Test and Apply Inclusion Criteria**

    1.  By four (4) months of EDOC, the Contractor shall propose
        inclusion criteria for including innovations and innovative
        implementation strategies and change initiatives in the
        Innovations Clearinghouse. The Contractor shall work with the
        Editorial Board.

    2.  The proposed inclusion and exclusion criteria shall show
        consideration of the following:

-   AHRQ wants innovations in health service delivery that are novel
    (creative and never been done before) and innovations that are
    perceived as new (have been done before but in a different setting,
    time period, environment, or climate).

-   AHRQ wants innovations that have been rigorously tested (i.e.,
    formal research methodology was applied in designing, piloting, or
    implementing a change in service delivery)

-   AHRQ wants innovations that, if not formally researched, have been
    formally evaluated upon implementation so that users of the Web site
    can learn what health outcomes or other gains (customer
    satisfaction, financial, risk management, patient safety, etc...)
    were achieved

-   AHRQ also wants innovations for which evidence of its impact does
    not meet scientific standards (e.g., only anecdotal accounts of
    success or failure are available), with a method for distinguishing
    among the difference classes of evidence

-   AHRQ does not want innovations in health service delivery that
    contradict evidence-based care. For example, a regional health
    system sets up a creative strategy to screen all men for prostate
    cancer using the prostate specific antigen (PSA) laboratory test.
    This creative strategy is not of interest to AHRQ because screening
    all men for prostate cancer using the PSA test is not an
    intervention supported by scientific evidence.

    1.  The Contractor shall pilot test the proposed inclusion criteria
        against a heterogeneous group of no less than 100 innovations,
        including innovative implementation strategies and change
        initiatives, to test the appropriateness, reliability, and ease
        of application.

    2.  The proposed inclusion criteria shall be distributed to the
        entire Project Expert Panel for comment and the Project Officer
        for both comment and approval. The Project Officer will
        coordinate AHRQ review.

    3.  The inclusion criteria shall be displayed on the Innovations
        Clearinghouse Web site in an easily found location.

    4.  The Contractor shall have each innovation submitted for
        potential inclusion in the Innovations Clearinghouse evaluated
        by at least two persons. In the event of a disagreement, the
        Contractor shall have a third person review the innovation. That
        third person shall be from the editorial board. The Contractor
        shall forward questionable innovations along with all three
        reviewers' comments to the Project Officer who will take these
        evaluations under advisement and make a final decision on the
        inclusion of specific innovations.

    5.  The Contractor shall forward to the Project Officer all
        submissions that are determined by the Contractor to have failed
        to meet inclusion criteria, including a detailed summary of
        reasons for exclusion.

    6.  In addition, the Contractor shall keep a detailed electronic log
        of actions related to submitted innovations. The electronic log
        will be submitted to the Project Officer monthly with the
        progress report and shall be maintained on a password protected
        Web page accessible to the project team and Agency staff.

    7.  Innovations might be expanded to include policy innovations
        during the lifetime of the contract. If this determination is
        made, the Contractor in consultation with the PO and editorial
        board will consider what additional criteria is needed.

3.  **Design, Develop, Implement, and Maintain a System to Obtain,
    Publish, Update and Archive Innovations, Associated Tools, and
    Lessons Learned for the Innovations Clearinghouse**

> **Incentive and reward system**

1.  To encourage submission of innovations, associated tools and lessons
    learned, the Contractor shall design an incentive and reward system
    and submit a draft to the Project Officer within two (2) months of
    EDOC. The draft must address the Contractor's plan for
    post-implementation monitoring and reporting of the system. The
    incentive and reward system shall be implemented within three (3)
    months of the EDOC. The incentive and reward system the Contractor
    designs shall be part of its project budget; no advertisement shall
    be permitted on the Innovations Clearinghouse Web site. Examples of
    possible incentives are an official certificate of participation
    accompanied by a letter of gratitude (for professional dossiers),
    and an Innovation of the Year award. See Appendix B (Innovations
    Clearinghouse Expert Meeting Summary) for additional discussion.

> ***Federal Register* notice**

2.  The Contractor shall mail copies of the Project Officer-prepared
    *Federal Register* notice and an AHRQ letter inviting response to
    the notice. This mailing will be to a broad base of innovators,
    researchers, and organizations that support health service delivery
    innovation, health care quality improvement and disparities
    reduction strategies, or health system and practice change
    initiatives. This mailing shall be sent within one month of the
    posting in the *Federal Register.*

> (For clarification purposes only: Within one (1) month after
> finalizing the inclusion criteria, the Project Officer will prepare a
> notice to be published in the *Federal Register* requesting innovators
> and organizations supporting health system and practice innovations to
> submit innovations, associated tools, and lessons learned along with
> the full name and address of the lead person whom Innovations
> Clearinghouse staff can contact. The *Federal Register* notice will
> describe the Innovations Clearinghouse and will request that
> information be submitted directly to the Contractor. The Project
> Officer, working with AHRQ's Office of Communications and Knowledge
> Transfer, will organize a direct mailing of a letter to relevant
> organizations with copies of the *Federal Register* notice encouraging
> them to submit to the Contractor.)
>
> **Other methods of identifying and stimulating submissions of
> innovations, innovation packages, associated tools, and lessons
> learned**

3.  The Contractor shall:

    1.  Search the peer-reviewed literature for published innovations or
        references to innovations under study

    2.  Search AHRQ-commissioned research reports

    3.  Search existing Web sites, databases, compendiums, source books,
        and news sources for reference to innovations and change in
        health care delivery

    4.  Browse AHRQ's QualityTools database and Web site for tools used
        in innovative health service delivery

    5.  Contact known innovators and researchers, including AHRQ-funded
        researchers, to further identify innovation-related documents

    6.  Connect with AHRQ's Office of Communications and Knowledge
        Transfer (and their contractor(s), at AHRQ's discretion) for
        impact case studies, a potential source of innovative
        implementers of AHRQ products

    7.  Subscribe to AHRQ and relevant others' Web sites' e-mail
        newsletters and update services

    8.  Participate on ListServs or other discussion forums used by
        potential submitters

    9.  Enlist the help of the Project Expert Panel in identifying
        candidate innovations

    10. Collect electronic copy, and hardcopy if necessary, of
        identified innovations, including references and accompanying
        supplemental material

4.  For potential submissions that appear to meet inclusion criteria,
    the Contractor will promote submission through one or more of the
    following:

    1.  Education. The Contractor explains the purpose of the
        Clearinghouse and the advantages of submitting

    2.  Assistance. The Contractor assists the submitter in
        understanding submission requirements, discussing whether the
        innovation was implemented in conjunction with (concurrently or
        sequentially) with other complementary innovations, assembling
        the necessary materials, and reviewing completeness and adequacy
        of submissions, providing comments or making revisions as
        necessary

    3.  Development. In some cases, the potential submitter may be
        unable to articulate the elements of the innovation, or possess
        insufficient resources to be able to compile a submission. In
        these cases, if the innovation is deemed by the PO to be of
        sufficient importance, the Contractor will collect the
        information and develop the submission.

> **Database of innovators and organizations that promote innovation**

5.  The Contractor shall develop and maintain an on-going database of
    innovators and organizations that promote health service delivery
    innovation, implementation of health care quality improvement or
    disparities reduction strategies, and change initiatives in health
    care facilities. The database shall be shared with the Project
    Officer. The Contractor shall ensure that this relational database
    includes full name, mailing and electronic mail addresses, telephone
    and facsimile numbers, and name (or names) of contact person(s) for
    innovations, quality improvement strategies, and change initiatives.
    The Contractor shall maintain the database on an on-going basis, in
    both hard and electronic copy and forward updates to the Project
    Officer for dissemination by AHRQ as appropriate. The database shall
    be maintained in a format that can be used for mailing labels and
    word-processing. This database shall be implemented within four (4)
    months of the EDOC and maintained continuously thereafter.

> **Develop online submission mechanism and secure OMB clearance**

6.  The Contractor shall develop a secure Internet-based mechanism for
    the submission of innovations, associated tools, and lessons learned
    to the Innovations Clearinghouse. An online submission form will be
    used by submitting innovators to capture (1) information needed to
    determine if the submission meets inclusion criteria, (2) content
    for the database, and (3) contact information of the submitter. The
    contact information will be used by the Contractor to communicate
    with the submitter for the purposes of authenticating the submission
    and seeking additional information, if needed to determine that the
    submission meets inclusion criteria and/or to clarify entries in
    data fields. The contact information will also be made available to
    potential implementers of an innovation as one mechanism of
    connecting innovators with implementers. The online submission
    mechanism shall use drop down menus, check boxes, and text fields,
    and identify required fields for completion; this will standardize
    data elements and lessen the workload of the project staff and of
    the submitters.

7.  Within one (1) month after finalizing the Innovations Clearinghouse
    inclusion criteria and database record template, the Contractor
    shall submit to the Project Officer an information collection
    package, including the online submission form, for review and
    approval. The information collection package shall address
    information collected online from submitters AND users.

8.  Within one (1) week after receiving Project Officer approval of the
    online submission form(s), the Contractor shall submit it to the
    Office of Management and Budget (OMB) for clearance.[^9] All federal
    public websites must comply with the Paperwork Reduction Act (44
    U.S.C. Chapter 35) to ensure that information collected from the
    public minimizes burden and maximizes public utility.[^10] One of
    the principal requirements of the Paperwork Reduction Act is that
    Federal agencies must have OMB approval before collecting
    information from the public (such as forms, general questionnaires,
    surveys, instructions, and other types of collections), and they
    must display the current OMB control number on the collection form.

> The Innovations Clearinghouse online submission mechanism can be used
> only after securing clearance from OMB. Approval from OMB may take six
> (6) to eight (8) months.
>
> **Inclusion criteria review**

9.  The Contractor shall assess submissions against inclusion criteria
    as specified in Task 3.2.6.

> **Obtain copyright permissions**

10. The Contractor shall obtain written copyright permission as
    appropriate from the innovators to make their innovation(s) and
    associated tool(s) available for Innovations Clearinghouse users
    through downloadable files or hyperlink to the innovator's Web site,
    if applicable. When innovations or associated tools are proprietary
    the Contractor shall obtain copyright clearance for purposes of
    inclusion in the Innovations Clearinghouse and QualityTools
    databases, respectively, for abstracting and posting the abstract on
    the Web site. The Contractor shall include language in the
    Innovations Clearinghouse Web site that will refer users back to the
    developer, using hyperlinks if possible, to obtain or purchase the
    full documentation, specifications, etc.

11. The Contractor shall maintain copyright permissions on file for the
    life of the project and send them to AHRQ at the end of the
    contract.

> **Annual verification**

12. The Contractor shall contact the innovators included in the
    Innovations Clearinghouse on an annual basis regarding the status of
    their innovation and associated tools (i.e., revisions, no longer
    sustained, further implemented, etc.) developing a mechanism that
    can be systematically applied. A secure Internet-based mechanism
    shall be considered. The purpose of the annual verification process
    is to obtain specific information regarding the current state of use
    of the innovation and associated tools, additional evidence, and
    plans for further refinement, wider implementation, or withdrawal.
    The Contractor shall track and maintain copies of the query
    responses and update the innovations tools databases accordingly.

```{=html}
<!-- -->
```
4.  **Design, Develop, Implement, and Maintain a System for Indexing and
    Abstracting and Validating the Indexing and Abstracting of
    Innovations Included Within the Innovations Clearinghouse**

> **Indexing**

1.  Innovations included in the Innovations Clearinghouse shall be
    indexed using standard reference terminologies (e.g. SNOMED RT,
    etc.), whenever possible.

The Contractor shall utilize the most cost-effective mechanism for
tagging and indexing innovations and should consider the utility of the
Unified Medical Language System (UMLS) Metathesaurus over more costly
proprietary solutions whenever possible. If a necessary concept is not
adequately represented in the Metathesaurus, the Contractor shall select
a term to describe the concept. In doing so, care should be taken to
avoid confusion with other recognized terms and to provide consistent
and reliable application. Further, the Contractor shall provide mapping
to outdated or revised terms when defining and maintaining new terms.
The goal should be to utilize already recognized terminologies and to
create new terms only when concepts are not adequately represented. In
some cases concepts are defined in existing terminologies but not
adequately for the purposes of health care innovations. For example,
"climate" in the Metathesaurus refers to the meteorological environment
as opposed to organizational and external (e.g., sociopolitical)
conditions - most often used in descriptions of how innovations are
spread.

> **Abstracting**

2.  Innovations and lessons learned included in the Innovations
    Clearinghouse shall be displayed using a standardized (structured)
    format.

    1.  The Contractor shall develop a template of innovation attributes
        and definitions with enough information to facilitate
        replication yet not overburden submitters. The Contractor shall
        also develop a template for capturing and publishing lessons
        learned, unless the Contractor can document reasons for
        capturing lessons learned as a data field(s) in the innovations
        template as an alternative. The Contractor shall consider
        previous work done by the Centers for Medicare and Medicaid
        Services (Medicare Quality Improvement Community, MedQIC), the
        Veterans Health Administration, the Institute for Healthcare
        Improvement, the Center for Health Systems Change, the Health
        Disparities Collaborative, and other relevant efforts. The
        Contractor shall create an innovation template or structured
        summary that describes *at least* the following attributes:

-   The innovation

-   Problem the innovation addressed

-   The population to which the innovation was applied

-   The disease/conditions for which the innovation was applied, if
    applicable

-   The setting in which the innovation was applied

-   The types (professional categorizations, and clinical specialty, if
    applicable) and number of providers/staff (total and per
    professional type) involved

-   The number of facilities involved, if applicable

-   The clinical (e.g., surgery, Ob/Gyn, pediatrics), administrative
    (e.g., accounting, medical records, human resources), ancillary
    (e.g., laboratory, pharmacy, respiratory care), and support (e.g.,
    information services, environmental services, maintenance)
    departments involved

-   Geographic location (e.g., International, National, State, Local,
    Other -- with each then specified)

-   The total budget allotted, budget spent

-   Time consumed during design, education/communication, pilot, wider
    implementation, and evaluative phases of innovation project

-   Evidence of innovation's impact: data source, findings, tests of
    statistical significance

-   Grant or Contract/Task Order #, if applicable

-   Study design (e.g., cross-sectional, descriptive, evaluation,
    expert/quasi-experimental, guideline development, implementation and
    evaluation, longitudinal, modeling, natural experiment,
    observational, panel, retrospective data analysis), if applicable

-   Data sources (e.g., claims and admin data, clinical data, employer
    survey, focus groups/panels, interviews, observation,
    patient/caregiver survey, provider survey, registry data), if
    applicable

-   Level of analysis (e.g., national, state, regional,
    facility/institution, patient, plan, provider, unit, other)

-   Target audience (e.g., employer/purchase organizations, health
    industry groups, health information organizations, hospitals,
    patients/caregivers, physicians, policy organizations, other)

-   Tools used

> **NOTE:** some of these data fields exist in a database in use by
> ACTION. AHRQ would like to provide XML output of the ACTION database
> to the Contractor for input into the innovations database for
> ACTION-related submissions.
>
> In addition, the Contractor shall consider including the following
> attributes in the template and provide rationale for not including
> them if thought to be inappropriate:

-   System or practice traits influencing innovations\*:

```{=html}
<!-- -->
```
-   structure (size, maturity, formalization, differentiation,
    decentralization, resources),

-   absorptive capacity for new knowledge (pre-existing knowledge or
    skills, ability to find, interpret, recodify, and integrate
    knowledge),

-   receptive context for change (leadership and vision, good managerial
    relations, risk-taking climate, clear goals and priorities,
    high-quality data capture)

```{=html}
<!-- -->
```
-   System or practice readiness for innovation\*: tension for change,
    innovation-system/practice fit, power balances (supporters vs.
    opponents), assessment of implications, dedicated time/resources,
    and monitoring and feedback

-   The innovation along with the following characteristics\*: relative
    advantage, compatibility, low complexity, trialability,
    observability, potential for **reinvention**, feasibility, risk,
    task issues, nature of knowledge (tacit/explicit), divisibility.

-   Adoption influences\*:

```{=html}
<!-- -->
```
-   Adopter: motivation, values and goals, social networks, skills,
    learning style

-   Adoption decision: optional collective, majority, contingent

-   Adoption process: knowledge, persuasion, decision, implementation,
    confirmation

```{=html}
<!-- -->
```
-   Communication influences\*: social networks, peer opinion,
    marketing, expert opinion, champions, boundary spanners, change
    agents

-   Implementation process influences\*: decision-making devolved to
    frontline teams, hands-on approach by leaders and managers, human
    resource issues (especially training), dedicated resources, internal
    communication, external collaboration, reinvention/development,
    feedback on progress

-   External powers\*: sociopolitical climate, incentives and mandates,
    interorganizational norm-setting and networks, environmental
    stability

> \*Source: Greenhalgh T, Robert G, Bate P. A conceptual model for the
> spread and sustainability of innovations in service delivery and
> organization. In: *How to spread good ideas: a systematic review of
> the literature on diffusion, dissemination and sustainability of
> innovations in health service delivery and organization.* Report for
> the National Co-ordingation Centre for NHS Service Delivery and
> Organisation R&D. 2004.
>
> Additional information about each innovation of interest to the Agency
> includes

-   Major factors (up to 3) in success and/or failure

-   What the innovator would do differently next time

-   How finances/reimbursement affected success/failure

-   The changes that had to be/were made to the innovation in the course
    of its implementation, and why

-   Disparities-related: was disparities reduction a focus? Is there pre
    and post data on disparities to share? Was a specific
    disparities-reduction strategy used?

    1.  Modifications of these attributes and inclusion of additional
        attributes and definitions shall be done in consultation with
        the Project Officer and subject to Project Officer approval.
        Development of the lessons learned template shall be done in
        consultation with the Project Officer and subject to Project
        Officer approval.

    2.  The attribute fields shall include controlled terms in addition
        to descriptive text fields. When developing the
        template/structured summary, the Contractor shall take into
        consideration descriptive attributes chosen as important by
        different innovators, innovative organizations and researchers
        of innovations. The template shall contain copyright information
        supplied by the innovator or holder of the innovator's
        copyright.

    3.  In addition, the Contractor shall include a mechanism to
        capture (1) user comments about an innovation and (2) editorial
        expert analysis and commentary on select submissions. The
        Contractor shall establish a secure, Internet-based mechanism to
        capture user comments, requiring users to provide contact
        information. See related OMB clearance tasks 3.3.6-3.3.8.

> **Validating**

4.  The draft template of attributes and definitions should be
    pilot-tested against a heterogeneous sample of at least 100
    innovations, including lessons learned, for ease of use,
    availability of information, and reliability of abstraction and
    indexing.

5.  The Contractor shall develop a draft template of attributes and
    definitions no later than four (4) months after contract award and
    submit it to the Project Officer for review and comment. The
    Contractor shall finalize the template within two (2) weeks after
    receipt of the comments from the Project Officer (i.e.,
    approximately month 5).

```{=html}
<!-- -->
```
1.  The Contractor shall be responsible for the completeness and
    accuracy of content contained in the innovation and lessons learned
    template(s). The Contractor shall contact the innovation submitter
    when more information is needed but it is not expected that every
    attribute will be completed for every innovation. *However, those
    attributes that are correlated with the inclusion criteria for
    innovations in the Innovations Clearinghouse must be completed*.
    This shall be reflected in the required fields for on-line
    submission. In addition, submissions are likely to lack consistency.
    The Contractor may need to revise submissions to create uniformity
    among database entries, appropriately informing the submitter.
    Innovators shall be given three weeks to comment or provide
    revisions to the structured summary. If no comments are received,
    the Contractor shall assume the innovation submitter has no comment
    but designate that the summary has not been verified by the
    innovation submitter. Innovation summaries shall be identified as
    "verified" or "unverified" by the submitter.

2.  The Contractor shall publish a minimum of 200 innovation structured
    abstracts, including lessons learned, to the Web site within the
    first 18 months of the project. Once added to the Web site, the
    Contractor shall be continually responsible for ensuring that they
    are accurate and current. Each year it is estimated that at a
    minimum 250-300 new or updated innovation structured abstracts shall
    be added to the Web site.

3.  The Contractor shall propose a strategy, within six (6) months of
    EDOC, for what types of innovations shall be addressed and added to
    the Innovations Clearinghouse first. In preparing the strategy, the
    Contractor shall consider different alternatives such as priority
    disease/conditions, priority populations, areas highlighted in the
    *National Healthcare Quality Report* and the *National Healthcare
    Disparities Report*, or compelling innovations. When presenting its
    strategy, the Contractor shall provide a rationale for the approach.

4.  The Contractor shall make available for download complete innovation
    documentation, if any, when copyright permission is received from
    the innovation submitter. The innovation documentation, if any,
    shall be made available for download in at least two formats one of
    which must be ASCII text. In the event that the innovation submitter
    does not provide permission for hosting full-text documentation on
    the Innovations Clearinghouse, the Contractor shall provide a
    hyperlink to the submitter's site (if appropriate) along with
    ordering information.

```{=html}
<!-- -->
```
1.  **Design, Develop, Test, Deploy, Maintain, and Operate the
    Innovations Clearinghouse Technical Infrastructure and Web Site**

> **Overview**

1.  This resource shall conform to Federal and Department of Health and
    Human Services (HHS) requirements in laws, policies, and directives
    for information technology (IT), capital planning and investment,
    and Internet information management and the security of Agency
    information and systems. The Contractor shall follow the guidelines
    identified in **[Appendix C -- AHRQ Web Product and Web Site
    Development Guidelines.]{.underline}** The Contractor shall deliver
    the documentation specified in [**Appendix D** - **Guidelines for
    Developing AHRQ Web-based Tools.**]{.underline} Baselined versions
    of all documents, and source code corresponding to production
    releases for all web sites and data bases, will be maintained in the
    AHRQ Rational library repository.

2.  It is the Contractor's responsibility to comply with the guidelines
    identified in subtask 3.5.1 and others identified in task 3.5 and
    3.10 throughout the life cycle of the contract.

3.  The Contractor shall design, develop, test, deploy, maintain, and
    operate the Innovations Clearinghouse Web site. Guidance is
    available from [www.usability.gov](http://www.usability.gov/), the
    Department of Health and Human Services resource for designing
    usable, useful, and accessible Web sites and user interfaces.
    Feedback on the usability of the QualityTools Web site shall be
    considered in the design of the Innovations Clearinghouse Web site.

4.  The Innovations Clearinghouse Web site shall become operational in a
    phased approach: launched within nine (9) months of EDOC to
    stimulate submissions and innovations, and fully operational within
    eighteen (18) months from the contract start date.

5.  The Contractor shall maintain the Web sites presence on the World
    Wide Web (WWW) via the Internet. The Innovations Clearinghouse Web
    site shall be available through a unique Uniform Resource Locator
    (URL). The QualityTools Web site shall retain its unique URL until
    it has been incorporated within the Innovations Clearinghouse. At
    that time, the Contractor shall provide a redirect from the
    QualityTools URL to the Innovations Clearinghouse URL.

6.  The Contractor shall create, implement and maintain all mechanisms
    necessary to support the Innovations Clearinghouse database. The
    Contractor shall incorporate the Web crawler feature of the
    QualityTools Web site, adjust the search algorithm and redesign the
    interface so that Innovations Clearinghouse users have the capacity
    to retrieve search results from other, specific, complementary sites
    (examples identified in subtask 3.4.2.1). An alternative Web crawler
    can be proposed accompanied by rationale.

> **Startup Requirements**

7.  The Contractor shall deliver a final Vision/Project Initiation
    Document (PID) for the Innovations Clearinghouse within two (2)
    weeks of the effective date of the contract (EDOC). It shall contain
    a statement of purpose and scope for the project. The PID shall
    define the stakeholders' view of the Innovation/QualityTools Web
    site in terms of the key needs and features identified in the
    Statement of Work and associated documents and at a high level,
    define the core requirements and allocate them to iterations and
    releases of the Innovation Clearinghouse Web site and database
    (including the incorporation of the QualityTools Web site and
    database into the Innovations Clearinghouse).

8.  The Contractor shall deliver a Project Work Plan (PWP) that details
    the life cycle of the project, including phases, iterations, and key
    milestones including integration points and procurement events,
    scope, cost, schedule, risks, and quality assurance for all systems,
    software, databases and websites after acceptance of the PID by the
    PO. The draft PWP is due three (3) weeks after EDOC. The final PWP
    is due within one (1) week after the Contractor receives Agency
    comments.

9.  The PWP shall contain a deliverables-based Work Breakdown Structure
    (WBS) (including a WBS dictionary) that accurately and completely
    captures all development work specified in the Statement of Work and
    any other documentation provided as a companion to the RFP. The WBS
    shall identify dependencies among the activities as well as
    dependencies with other contract activities. The WBS shall be
    resource-loaded and accompanied by a time-phased budget (budgeted
    cost of work scheduled, BCWS) that will be baselined and subject to
    a semi-annual Integrated Baseline Review (IBR) by a team of
    Contractor and AHRQ personnel. The time phased budget shall be
    subject to Earned Value Reporting as specified in 3.10.4.

> **Technical Requirements**

10. The Contractor shall use an open systems, industry standard,
    object-oriented software design and software technology solutions,
    3-tier architecture, and enterprise reusable object software
    components to ensure cost-effective system development, operations
    and maintenance.

11. The Contractor shall adhere to the **[Agency's Draft Technical
    Reference Model (TRM) in Appendix E]{.underline}**. Specifications
    identified as "Standard" in the Disposition column shall be
    selected, unless the Contractor requests and receives a waiver from
    the Government. The Contractor shall use commercial off-the-shelf
    (COTS) components when possible.

> **Database Requirements**

12. The search engine software solution shall facilitate effective
    searches by using a combination of indexes and XML metadata tags,
    numeric primary keys, and use of controlled vocabularies for
    important concepts. Both manual and algorithmic indexing shall be
    used.

13. The Contractor shall design, develop, and maintain a database
    backbone to house the Innovations Clearinghouse as one module.

14. The Contractor shall establish a set of common data elements that
    will serve to appropriately link the contents of the Innovations
    Clearinghouse and QualityTools. The Innovations
    Clearinghouse/QualityTools data model shall be expandable to include
    other clinical information modules.

> **Web Site Requirements**

15. The Contractor shall propose names for the Web site (and project)
    and a recommendation with rationale and submit those to the Project
    Officer for review within one (1) month of EDOC. The Project Officer
    shall coordinate AHRQ review and approval. The approved name of the
    Web site (and project) shall be used in official communications
    immediately upon receipt of approval from the Project Officer.

16. The Contractor shall deploy the Innovations Clearinghouse Web site
    in three (3) stages.

    1.  The first stage will be deployed within nine (9) months of the
        EDOC showing AHRQ's commitment to fostering innovation in health
        service delivery. At this stage of deployment, Web site content
        will focus on stimulating creativity and encouraging change. The
        features to be made available at this stage are those items (or
        placeholders when approved by the PO) as specified in 3.5.17;
        commentaries from innovation, implementation and change experts;
        announcements of relevant conferences, meetings, and
        publications; e-mail sign up; solicitations for submissions;
        AHRQ-approved incentive and reward system; and other features
        proposed by the Contractor and approved by the PO. At this first
        stage, structured abstracts of innovations won't be made
        available.

    2.  The second stage will be deployed within 18 months of the EDOC
        and will render the Web site fully operational. This stage will
        include presentation of the structured abstracts of the
        innovations, tools and lessons learned; online submission
        mechanism; contractor-moderated mechanism for users to submit
        comments about innovations; and the AHRQ-approved mechanism(s)
        for learning and networking among users.

    3.  The third stage will be deployed within 26 months of EDOC. This
        final stage will be for the purpose of making an online tutorial
        of the Web site available.

17. The Innovations Clearinghouse Web site shall include:

-   Descriptions of the Innovations Clearinghouse and AHRQ

-   Sections that describe how the contents are maintained, including
    inclusion criteria for innovations, and attributes and definitions
    used in the structured abstract

-   A direct link to a privacy statement approved by the Project Officer

-   Information on how innovations may be submitted and how users can
    contact the Innovations Clearinghouse; this shall be displayed in an
    easily accessible manner

-   Resource sections that contain technical information about the site,
    a description of the technical assistance support (e.g., where to
    e-mail for technical assistance when using the Innovations
    Clearinghouse) and educational pieces about innovations and change
    in healthcare.

-   A site map

-   A Frequently Asked Questions (FAQs) section where users can perform
    both keyword searches or scroll for information

-   Search help

-   Home page footer using standard HHS language, with links to relevant
    pages on the Innovations Clearinghouse Web site

-   Incorporation of the HHS and AHRQ logos as well as the FirstGov.gov
    logo, with associated links

-   Copyright indication, updated annually

    1.  The Innovations Clearinghouse Web site shall include a
        Contractor-moderated mechanism for users to post comments about
        innovations. The Contractor shall develop their approach to
        moderating and deciding which comments to post, and submit a
        draft to the Project Officer 13 months from the contract start
        date. The draft proposal must identify the hardware (e.g.,
        dedicated server) and software or service (e.g., HaloScan) that
        will be used. The mechanism shall be fully implemented within 18
        months of the contract start date, when the Innovations
        Clearinghouse Web site becomes fully operational.

    2.  The Innovations Clearinghouse Web site shall be accessed via
        ".ahrq.gov" domains (i.e., a third-level domain name). The
        Government will be responsible for registering the appropriate
        "gov" (top level) domains, but the Contractor shall register
        analogous non-gov domains.

    3.  The Contractor shall use Extensible Markup Language (XML) as the
        main form of presentation. In using XML, the Contractor shall
        take into account the importance of obtaining consistency in the
        use of medical terms and the work of HL7 (Health Level 7) and
        ASTM (American Standard for Testing and Materials) whenever
        possible.

    4.  The Contractor shall provide Web sites that are in full
        compliance with all relevant sections of the Americans with
        Disabilities Act (ADA). AHRQ will provide specific guidance to
        the Contractor upon award. General information on accessibility
        can be found at the following Web resources:

```{=html}
<!-- -->
```
-   [www.access-board.gov/508.htm](http://www.access-board.gov/508.htm)

-   [www.usability.gov](http://www.usability.gov/)

-   [www.section508.gov](http://www.section508.gov/)

    1.  The Contractor shall employ limited usability testing while
        developing the functional features of the Innovations
        Clearinghouse Web site. Testers shall consist of Project Expert
        Panel members, Federal employees, and selected end-users. The
        Contractor shall consider using the HHS Usability Lab
        (<http://www.dhhs.gov/policies/webpolicies/200505.html>). The
        testing shall be completed by month fifteen (15) of the
        contract. The Contractor shall deliver a Usability Test
        Evaluation Summary with the results of the testing to the
        Project Officer within ten (10) days of the completed testing.
        Guidance on usability testing is provided at
        [www.usability.gov](http://www.usability.gov/).

    2.  The Contractor shall track and monitor site usage and maintain
        and analyze search logs.

    3.  The Contractor shall register the Innovations Clearinghouse Web
        site with search engines and Web site portals appropriate for
        the content and provide metatags for searching.

> **Hardware Requirements**

4.  The Contractor shall provide and support all hardware necessary to
    host and connect the Web sites. The Government reserves the option
    to host the databases and Web sites and provide access to the
    Internet.

5.  The Contractor shall provide for a physical location to house the
    Web sites and arrange for appropriate onsite visits by the Project
    Officer and other program representatives. All visits during normal
    working hours shall be permissible with two hours notice, and within
    24 hours notice for all other times.

6.  The Contractor shall provide access to the Internet for the Web
    sites, and that access shall have sufficient bandwidth to meet the
    following usage requirements (for the Innovations Clearinghouse
    consisting of innovations, tools and related information and
    learning opportunities):

> From the beginning, the Contractor shall plan for a load tolerance of
> at least 500-700 simultaneous users and per-day traffic equal to the
> third-year estimate.
>
> In year one (1) of operation, the anticipated number of user sessions
> on the Innovations Clearinghouse Web site is estimated at 2,000 to
> 4,000 per day. It is expected that usage will double in each
> subsequent year. The Contractor shall construct Web sites that have
> the ability to support 130% of the usage listed above.

7.  The Contractor shall prepare contingency plans, in case the traffic
    appears to be exceeding the above estimates, so that it will be able
    to reconstitute the hardware/software platform to tolerate an
    increased load within 50 workdays. No costs shall be budgeted or
    incurred by the contractor in connection with this contingency plan.
    The contingency plan will be negotiated and funded on an as needed
    basis. Costs for implementation will not be included in the initial
    contract award.

8.  The Contractor shall propose and develop mechanisms for 24 x 7
    systems near-fulltime availability to ensure system uptime for user
    operations including updates

-   The Contractor shall provide for system failures such that: any
    hardware, software, or database failure will not result in a loss of
    more than 27 hours of the most current data; any software failure
    will not result in more than 8 hours of downtime after notification
    of the situation; any hardware failure will not result in more than
    26 hours of downtime; and that any Internet connectivity failure
    will not result in more than 48 hours downtime.

```{=html}
<!-- -->
```
-   The contractor shall provide a system that maximizes security and
    data privacy even in the event of a catastrophic failure.

    1.  The Contractor shall test backup/restore process at a minimum of
        once every month to insure that the process is viable, and
        report the results (pass or fail) to the Project Officer.

> **Hardware and software procurement**

2.  The Government reserves the option to procure and deliver as
    Government Furnished Equipment (GFE) all new information technology
    hardware and software required to operate and maintain system(s)
    used in this project.

> **Configuration and Change Management Requirements**

3.  The Contractor shall maintain all code (source code, builds, and
    installation artifacts) and documentation under configuration
    management.

4.  The Contractor shall follow the Agency's Configuration Management
    Plan for defect management and formal Change Control including
    Change Control Board (CCB) participation, approval and
    prioritization of changes.

5.  The Contractor shall use the Agency's Rational ClearCase for
    configuration management of all baselined documentation and source
    code. Baselines shall be maintained for each deployed version of
    software.

> **Requirements for the Technology-Related Work of the Innovations
> Clearinghouse and Associated Administrative Support Systems**
>
> These requirements address the **Development, Modernization, and
> Enhancement (DME)** and **Steady State (SS)** activities to support
> the Information Technology-related work specified in this document.

-   Development, Modernization and Enhancement (DME) refers to the
    initial creation of an IT system (design, develop, test and deploy),
    adding features to an existing IT system, or changing the technology
    of an existing IT system. With respect to websites, adding new pages
    or functionality, such as a new survey or search function, is DME.
    Changing the text or graphics on a web page or adding new tools or
    innovations to a data base is considered content change and does not
    fall under the definition of DME.

-   Steady State (SS) refers to the maintenance and operation costs at
    current capability and performance level including costs of
    personnel, maintenance of existing information systems, corrective
    software maintenance, voice and data communications maintenance, and
    replacement of broken IT equipment.

    1.  The Contractor shall meet, at minimum, the Software Engineering
        Institute (SEI) Capability Maturity Model Integration (CMMI)
        level 2. The Contractor shall fully describe how they will meet
        this requirement and gain approval of the Agency for any
        procedural or library functions that will be used to meet this
        requirement before project startup. The Contractor shall have
        three (3) years prior experience using a structured formal
        software development methodology. The contractor shall provide
        three (3) references who can attest to the contractor's
        successful use of this methodology.

    2.  The Agency shall conduct an IV&V of each document prior to
        acceptance. The IV&V review period shall be no more than
        ten (10) business days for each document and the Contractor
        shall not initiate the next major phase of work following a life
        cycle milestone until written approval is received from the
        Project Officer.

    3.  The Government reserves the option to request additional
        development, enhancement and modernization work beyond that
        specified in the Statement of Work. In these circumstances the
        Contractor shall deliver an updated Software Development Plan,
        performance baseline, WBS, and BCWS that reflects the additional
        scope of work and revised costs and schedules.

> **Reporting Requirements**

4.  The Contractor shall provide monthly reports of the Web site and
    database uptime and shall include an "Anomalies Report" to the
    Project Officer describing any major problems with the system's
    hardware, software, or security.

5.  The Contractor shall maintain for inspection, documentation on
    requirements management, configuration management, cost estimates,
    project reviews, and quality assurance activities -- particularly as
    it pertains to the RUP and SEI requirements.

6.  The Contractor shall monitor sites linking to the Innovations
    Clearinghouse on a monthly basis and include this information along
    with the total number of visits referred in the monthly progress
    report.

7.  The Contractor shall provide monthly reports that characterize Web
    site usage trends, including information such as: utilization;
    frequently visited pages; and other performance metrics. The HHS
    Office of the Secretary deployed Watchfire WebXM throughout HHS to
    manage and monitor all of the public Web resources that the OP/DIVs
    support.

> This is part of the Web Governance activities of HHS in support of
> E-Gov Act requirements and this Web management environment is designed
> to help improve the overall quality and reliability of the Agency's
> online presence and, at the same time, mitigate online risk. This
> includes the quality and integrity of site content and links,
> accessibility and Section 508 requirements, privacy issues and
> protections, and security. 
>
> Contractor access to or AHRQ-provided reports from Watchfire WebXM
> will support this monthly reporting.

**Documentation Requirements**

8.  The Contractor shall provide to the Agency all documentation of all
    proposed hardware, software, security, backup/recovery, and other
    information technology infrastructure and components and solutions
    needed to support this project. See Appendix C. Baselined versions
    of these documents shall be housed in the Agency Rational
    repository.

> **Security and Privacy Requirements**

9.  The Contractor shall be responsible for the security and the privacy
    of the system.  All federal public websites must comply with Section
    207(f)(1)(b)(iv) of the E-Gov Act of 2002, which requires
    organizations to have security protocols to protect information.

10. The Contractor shall comply with all Federal and HHS security
    guidelines that are in effect at the time of the award of this
    contract. U.S. Laws, Office of Management and Budget requirements,
    HHS Policies and Guides, and Federal Government Computer Security
    Policy and Guides are provided via the Internet:
    <http://www.hhs.gov/ocio/security/docs.html>

>  

11. The Contractor shall insure that the Web sites and data are secured
    behind appropriate perimeter defense technologies and that these
    technologies are programmatically monitored for anomalous traffic
    behavior(s). 

12. The Contractor shall immediately report any unauthorized access to
    the Project Officer and or System Owner. 

13. The Contractor shall insure that PII (Personally Identifiable
    Information, defined by FOIA II) data is never allowed on a system
    with public (Internet) access. 

14. The Contractor shall conduct (or cause the completion of) a Federal
    Information Security Management Act (FISMA)-conforming C&A process
    of the System prior to the System being placed into production. Such
    C&A will be compliant to all PL-107-347 requirements, Federal
    Information Processing Standards (FIPS) mandates
    (<http://www.itl.nist.gov/fipspubs/>), and National Institute of
    Standards and Technology (NIST) guidance. This guidance includes,
    but is not limited to NIST 800-18, 800-30, 800-37, 800-53 (with
    appropriate baseline control sets), and 800-60, and is available on
    the Internet: <http://csrc.nist.gov/>.

15. The SSP produced for the C&A will, at a minimum, contain provisions
    for:

    1.  A Tested Continuity of Operations Plan (COOP)

    2.  Computer Incident Response Capability (CIRC)

    3.  Access Controls containing i.) Rules of Behavior, and ii.)
        Appropriate Use Policies

    4.  Annual Security Awareness Training requirement

    5.  PIA (Privacy Impact Analysis)

    6.  CCP (Change Control Procedures)

    7.  Appropriate NIST 800-53 Control Set with an appropriate
        Supplemental Control Set. Control Sets will be appropriate to
        the SC (Security Classification) determined by using FIPS-199
        and NIST 800-60 requirements and guidance.

16. At the conclusion of the C&A process the contractor shall provide an
    out-brief to the System Owner (SO, who will be the Project Officer)
    and Information Owner(s) that will describe in detail the
    requirements of the Continuous Monitoring Phase for the succeeding
    calendar year (from the date of the Accreditation letter from the
    Designated Approval Authority \[DAA\]). Further, the Contractor will
    identify to the SO all known requirements of FISMA compliance to
    include reporting, continuing Risk Analyses, Plan of Action and
    Milestones (P.O.A.&M) completion, and a discussion that imparts a
    clear understanding to the SO of the Risk Profile (including
    Residual Risk) of the System covered in the C&A process.

17. The Contractor shall post on the Innovations Clearinghouse Web site,
    and the QualityTools Web site beginning December 1, 2006 (if
    needed), a clearly written statement about the PII (Personally
    Identifiable Information, defined by FOIA II) policies.

```{=html}
<!-- -->
```
1.  **Maintain and Operate AHRQ's QualityTools Database and Web site**

> The Contractor shall maintain AHRQ's QualityTools database and Web
> site beginning December 1, 2006. Hosting the current and archived
> Web-enabled versions of the NHQR, NHQR State Resources, and NHDR will
> continue until the Innovations Clearinghouse Web site is fully
> operational or sooner at AHRQ's direction.
>
> **Usage Requirements**

1.  As stated in section 1.13 of this solicitation, QualityTools
    contains more than 800 published tool summaries. In February 2006,
    there were approximately 78,000 visits to the QualityTools site,
    which includes visits to the NHQR and NHDR sub-sites. QualityTools
    consumes about 110Kb/s and peaks at about 240Kb/s. The Contractor
    shall provide bandwidth capacity to meet this load and to support
    usage growth that doubles each year.

> **Pre-Transition**

2.  The contractor must become familiar with the QualityTools Web site,
    its content, functionality, unique features, relationships with
    other AHRQ clinical and quality measurement components, and
    technical architecture and design.

3.  Between the contract start date and December 1, 2006, the Contractor
    shall work collaboratively with the QualityTools Contractor.

4.  The Contractor shall review QualityTools project documentation
    forwarded by the Project Officer to prepare to maintain the
    QualityTools database and Web site, and to enhance the QualityTools
    database if changes are needed in light of the Innovations
    Clearinghouse development. Briefly, QualityTools hardware is generic
    commodity and uses the Microsoft® Windows server operating system. A
    number of commercial server software packages are used, such as,
    Microsoft Internet Security and Acceleration \[ISA\] Server
    (firewall software), Microsoft Internet Information Server \[IIS\]
    (web server software), and Veritas Backup Exec (server backup
    software). The QualityTools database platform is Microsoft SQL.
    Custom/non-commercial packages include Microsoft Visual Basic to
    code the system, and a combination Visual Basic and Visual Basic.NET
    content development tool.

5.  The Innovations Clearinghouse Contractor shall participate in
    transition meetings, planned by the QualityTools Contractor in
    coordination with the Project Officer. Participation in the meetings
    may require travel, reimbursable costs to the Contract; at least one
    meeting will be at AHRQ.

> **Transition, Maintenance and Enhancement**

6.  As of December 1, 2006, maintenance and enhancement of the
    QualityTools database and maintenance of the Web site shall be the
    responsibility of the Innovations Clearinghouse Contractor.

7.  The existing QualityTools Web site and database are not compliant
    with the AHRQ Draft Technical Reference Model (TRM). QualityTools is
    grandfathered and exempted from the TRM standards until such time as
    the QualityTools is integrated into the Innovations Clearinghouse
    infrastructure.

8.  The Contractor shall maintain, monitor, and enhance a system that
    regularly identifies, assesses against inclusion criteria,
    abstracts, and posts to the Web on a weekly basis, practical,
    ready-to-use tools for measuring and improving the quality of health
    care.

> IDENTIFY

9.  The Contractor shall identify content for QualityTools by:

-   Tapping into their network of innovators and adapters

-   Routinely performing searches of the Web sites of known tool
    developers

-   Performing searches of the general Internet to identify new
    developers and tools based on the Institute of Medicine priority
    areas and priority populations

-   Monitoring relevant Internet discussion groups and bulletin boards
    for announcements of new tools

-   Subscribing to AHRQ and relevant others' Web sites' e-mail
    newsletters and update services

-   Working collaboratively with AHRQ's Contractors for the National
    Guideline Clearinghouse and National Quality Measures Clearinghouse

-   Other mechanisms as deemed appropriate by the Contractor

> OBTAIN COPYRIGHT PERSMISSION

1.  The Contractor shall obtain copyright permission when necessary, as
    specified in Task 3.3.10.

> ASSESS AGAINST INCLUSION CRITERIA

2.  The Contractor shall determine if tools identified meet the
    QualityTools inclusion criteria. Each tool must be evaluated for
    inclusion by at least two persons. In the event of a disagreement, a
    third person shall review the tool. The Project Officer will take
    these evaluations under advisement and make a final decision on the
    inclusion of specific tools. The Project Officer, and anyone else
    that the Project Officer deems appropriate, must review tools that
    are submitted and determined by the Contractor to have failed to
    meet criteria. The Contractor shall submit a detailed summary of
    reasons for exclusion and keep a detailed electronic log of actions
    related to submitted measures. The electronic log will be submitted
    to the Project Officer monthly with the progress report and shall be
    maintained on a password protected Web page accessible to the
    project team and Agency staff.

> ABSTRACT

3.  Tools included in QualityTools are displayed using a standardized
    (structured) format. The Innovations Clearinghouse Contractor shall
    use the template and follow the abstraction manual acquired from the
    original QualityTools Contractor to develop the structured
    summaries. This will minimize inter-Contractor and inter-abstractor
    variability and promote standardization of approach.

4.  The Contractor shall be responsible for the completeness and
    accuracy of content contained in the tool templates. Quality
    assurance mechanisms shall be developed and used by the Innovations
    Clearinghouse Contractor.

> PUBLISH

5.  The Contractor shall maintain the process of publishing, on a weekly
    basis, production-ready tool summaries to the QualityTools Web site.
    Production-ready tool summaries are those in which the content is
    complete and accurate and in which the (1) html code is correct and
    provides 508-compliance and (2) hyperlinks are correct and
    functional. The Contractor shall establish a mechanism to preview
    the summaries in a Web environment and assure tool summaries are
    production-ready.

6.  The Contractor shall publish at least 150 new or updated tools
    between December 1, 2006 and launch of the Innovations Clearinghouse
    Web site, stage II deployment. The Contractor shall publish at least
    250-300 new or updated tools each year thereafter.

> WEEKLY E-MAIL SERVICE

7.  The Contractor shall maintain weekly e-mail service to QualityTools
    users who have subscribed to this service. The weekly e-mail update
    service will identify the new and updated tools published to the
    site along with those tools that have been withdrawn. Other relevant
    news (such as, site enhancements, upcoming conferences, etc...) can
    also be announced following Project Officer approval.

> WEB SITE FEATURES

8.  The Contractor shall maintain all existing features of the Web site.
    No enhancements or changes to the Web site are expected since it
    will be integrated into the Innovations Clearinghouse.

> TECHNICAL INFRASTRUCTURE

9.  The Contractor shall be responsible for the technical infrastructure
    supporting the QualityTools database and Web site. Refer to the
    specifications in Task 3.6.4.

> EVALUATE

10. The QualityTools Web site currently solicits users to provide
    feedback about the site via the American Customer Satisfaction Index
    (ACSI) survey to meet requirements for a standard methodology. The
    survey instrument was developed by ForeSee Results with input from
    AHRQ, through an interagency agreement between AHRQ and The Federal
    Consulting Group, U.S. Department of the Treasury. Survey code is
    developed by ForeSee Results, and installed, tested in collaboration
    with ForeSee Results, and deployed by the QualityTools Contractor.
    Under this contract, the Contractor shall maintain and update the
    code at the production site and in accordance with instruction from
    AHRQ and ForeSee Results, as necessary. Also, the Contractor shall
    assist the PO in interpreting survey data analysis. The Contractor
    shall consider survey data results when re-designing the Web site
    into the Innovations Clearinghouse Web site. The Contractor shall
    assist the PO in conducting usability studies and evaluations of the
    Web site with targeted users and/or partners to improve the user
    experience and facilitate end-user business processes with Web
    resources.

```{=html}
<!-- -->
```
2.  **Provide Technical Assistance to QualityTools and Innovations
    Clearinghouse Users**

    1.  The Contractor shall provide technical assistance to
        QualityTools users beginning December 1, 2006 and to Innovations
        Clearinghouse users upon launch. Technical assistance will be
        provided via a variety of formats, as described in tasks
        3.7.2--3.7.8. The Contractor is not expected to routinely
        provide telephone technical assistance.

    2.  The Contractor shall develop text describing the Innovations
        Clearinghouse Web site for use by organizations that list or
        link to the Innovations Clearinghouse Web site. The Contractor
        shall do the same for the QualityTools Web site beginning
        December 1, 2006, if not done by the previous contractor of
        QualityTools. (The Project Officer will coordinate AHRQ
        requirements for publishing, electronic dissemination, and
        public affairs with staff from AHRQ's Office of Communication
        and Knowledge Transfer.) The Contractor shall develop and
        display "linking" policies to organizations that want to
        establish links to the QualityTools and then Innovations
        Clearinghouse Web site. The Contractor shall track, on a monthly
        basis, sites that link to the QualityTools and then Innovations
        Clearinghouse site utilizing COTS software.

    3.  The Contractor shall develop detailed help pages for each
        function of the Innovations Clearinghouse Web site in easily
        understood language. The help function shall be accessible from
        every page of the Innovations Clearinghouse Web site.

    4.  The Contractor shall create a list of Frequently Asked Questions
        (FAQs) and answers that are available via the help function. The
        FAQs shall be updated regularly.

    5.  The Contractor shall display on the Innovations Clearinghouse
        Web site technical specifications that will assist users and
        developers in taking full advantage of the Web site.

    6.  The Contractor shall display information on how to contact
        Innovations Clearinghouse by e-mail on every page of the Web
        site. The Contractor shall answer all e-mail within 24 hours
        during the business week and within 48 hours during the weekend.
        The Contractor shall develop standard answers to questions that
        occur frequently. These questions and answers shall be added to
        the FAQs.

    7.  The Contractor shall develop an on-line and a CD-ROM tutorial on
        how to use the Innovations Clearinghouse. The tutorial,
        including the CD-ROM version, must be compliant with
        requirements of the E-government Act of 2002. The Contractor
        will meet requirements for section 508 compliance, using
        AHRQ-supplied guidance at a minimum. The Contractor shall
        prepare a script and mock-up by 24 months from the contract
        start date and complete the final version by month 26. The
        Government will publish the CD-ROM and create packaging for its
        distribution.

    8.  QualityTools' has an electronic mail-based notification system
        for its users who registered to receive regular updates about
        QualityTools content and Web features. The Contractor will
        maintain this, effective December 1, 2006, and will modify it
        for the Innovations Clearinghouse users permitting them to
        select to receive notices about tools, innovations, or both.

3.  **Design, Develop, Implement, and Maintain Opportunities for
    Learning and Networking Among Innovations Clearinghouse Users**

    1.  The Contractor shall propose options for providing learning
        communities and networking opportunities among Innovations
        Clearinghouse users. Through these efforts, AHRQ intends to
        connect potential implementers with submitting innovators for
        the purpose of promoting replication, adaptation and use of
        innovations, and stimulating re-invention and innovative
        thinking. Ideas could include hosting and moderating
        innovation-specific teleconferences (making some invitation-only
        when trying to connect certain individuals or groups and permit
        more focused discussion), web seminars, virtual meetings, or
        electronic mechanisms, such as a ListServ, where users can
        engage in dialogue and share ideas and concerns, and pose
        questions to each other.

    2.  The Contractor shall submit a work plan for creating learning
        communities to the Project Officer within thirteen (13) months
        of the EDOC. The plan must address implementation,
        post-implementation monitoring, and monthly assessment and
        reporting to the PO. The Contractor shall submit a final
        PO-approved plan within fifteen (15) months of EDOC. The
        Contractor shall implement the plan within (eighteen) 18 months
        of the contract start date.

4.  **Assist the Agency in Evaluation of the Innovations Clearinghouse
    Web Site**

    1.  The Contractor shall extend task 3.6.19 (evaluation of
        QualityTools Web site) to the Innovations Clearinghouse Web site
        at the second stage of deployment.

    2.  The Contractor shall help determine sampling frequency based on
        visitor traffic to the site and deploy the AHRQ-supplied
        American Customer Satisfaction Index (ASCI) survey code on the
        Web site at a time approved by the PO.

    3.  The Contractor shall review rating reports and help analyze
        survey results and customer satisfaction feedback on a quarterly
        basis to identify areas for improvement to the Web site.

    4.  The Contractor shall report on the survey results on an annual
        basis in the Annual Report.

    5.  The Contractor shall contribute to developing updated custom
        questions on the survey to address the changing focus of
        performance measurement for the site.

5.  **Management**

    1.  The Contractor shall provide for the effective and efficient
        management of the technical, administrative, logistical, and
        support functions described in this statement of work. The
        Contractor shall carry out the following tasks in implementing
        the management of the project.

> **Meet with the Project Officer and designated Agency staff**

2.  Not later than one (1) week after the effective date of the contract
    award, the Contractor shall meet with the Project Officer and key
    Agency staff in Rockville, Maryland to discuss the draft Project
    Management Plan submitted with the proposal. Also at this meeting,
    the parties shall clarify any procedural issues related to the
    Statement of Work, propose any modifications deemed necessary, and
    provide an opportunity for the Project Officer to provide the
    Contractor with any additional materials and information to be used
    in the performance of the contract requirements.

> **Use Project Management Industry Best Practices and Earned Value
> Management System**

3.  The Contractor shall provide a PMI-certified project manager or
    PMI-certified within one year of contract award to manage the
    contract.

4.  The Contractor shall use an Earned Value Management System (EVMS) in
    the management of this contract that is consistent with
    ANSI/EIA-STD-748 guidelines.

5.  The Contractor shall report earned value monthly based on actual
    data from (or reconciliable with) its accounting systems. The
    Government and the Contractor shall negotiate the level of the WBS
    at which earned value shall be reported. If subcontractors are used,
    the Contractor shall develop and implement a system to ensure that
    its subcontractors comply with this requirement, and each monthly
    progress report must address performance at the prime and
    subcontractor level.

6.  The Contractor shall deliver with the Project Work Plan (see Tasks
    3.5.8 and 3.5.9) a contract-level Integrated Master Schedule (IMS)
    that rolls up all time-phased WBSs for the individual tasks. This
    IMS shall include the dependencies that exist between tasks.

7.  The Contractor shall deliver with the Project Work Plan (see Tasks
    3.5.8 and 3.5.9) a Critical Path at the contract-level.

8.  The Contractor shall report the following EVM data for the entire
    project and specific deliverables/milestones in the monthly report.

    1.  Budget at Completion (BAC) -- total planned cost

    2.  Budgeted Cost of Work Scheduled (BCWS) -- cumulative expected
        planned cost

    3.  Actual Cost of Work Performed (ACWP) -- cumulative amount
        actually

    4.  Budgeted Cost of Work Performed (BCWP) -- cumulative amount of
        work completed

9.  The Contractor shall provide a corrective action plan with the
    monthly earned value report in cases where cost or schedule, as
    measured by Cost Performance Index (CPI) and Schedule Performance
    Index (SPI), exceeds performance thresholds. The Government and the
    Contractor shall negotiate the performance thresholds. The
    corrective action plan shall address the following items.

    1.  Brief description of the issue (identify the nature of the issue
        as cost, schedule, performance or resources and explain why the
        issue came to be)

    2.  Impact on project completion

    3.  Brief description of the corrective action plan strategy

    4.  Corrective tasks/actions that are required to eliminate or
        mitigate the issue, including task/action number, point of
        contact, start and end dates and comments

10. If the corrective action plan requires rebaselining (the performance
    baseline), the corrective action plan also shall address the
    following items.

    1.  The proposed new baseline

    2.  The reason(s) why rebaselining is required

    3.  Documentation of the reasonableness of the proposed new
        baselines and the accuracy of any new cost estimates

11. The Performance Baseline, WBS, BCWS, Integrated Master Schedule, and
    monthly earned value report shall clearly breakout the deliverables
    and activities related to the development and maintenance of
    systems, software, databases, and websites.  It shall also clearly
    separate development, modernization and enhancement (DME)
    deliverables and activities from operations and maintenance
    deliverables and activities.

> **Systems documentation**

12. The following information formatted according to the templates
    implemented in the Agency's Rational Tools Set will be provided for
    all systems, software, databases, and websites developed under this
    contract, subject to review and approval by the Project Officer. The
    information will be contained in the specific documents described in
    Appendix D for deployed version of software.

-   Vision/Project Initiation Document

-   Use Case Model (actors, use cases)

-   User Interface Prototype/Design

-   Requirements Traceability Matrix

-   Software Architecture Document

-   Test Plan & Test Scripts

-   Test Evaluation Summary

-   End-User Support Material (User's Guide, Training Materials)

-   Product (Source Code and Installation Artifacts)

-   Bill of Materials

-   Release Notes

-   Operations Manual

    1.  In addition, the Contractor shall at least weekly archive
        electronic files uploaded onto the QualityTools Web site
        effective December 1, 2006 and then for the Innovations
        Clearinghouse Web site after launch. The Contractor shall also
        at least weekly archive electronic mail received through the
        QualityTools Web site electronic mail box, effective December 1,
        2006, and then through the Innovations Clearinghouse electronic
        mail box.

> **Monthly Progress Report**

2.  Beginning on the tenth calendar day of the second month of contract
    performance and on the tenth calendar day of every month of contract
    performance thereafter, the Contractor shall submit to the AHRQ
    Contracts Office and to the Project Officer a monthly progress
    report in electronic format, via e-mail as attachments or secure ftp
    or extranet site with e-mail notification that reports are available
    for download. At a minimum, the monthly progress report shall:

-   summarize the progress of work performed during the reporting
    period, note milestones, describe past problems (including a concise
    statement of success or lack thereof in solving the problems
    encountered)

-   describe anticipated problems and proposed solutions

-   describe work planned for the next reporting period, compare
    progress and resource expenditures to the original schedule and
    budget and provide explanations for any variances

-   assess whether the current total estimated contract cost is
    sufficient to complete the contract

-   describe significant changes in the Contractor\'s operational
    personnel

-   summarize all additions, modifications, and configuration changes to
    all project software and database designs, specifications, testing,
    quality assurance efforts, maintenance and operational solutions and
    associated work efforts accomplished during the month

-   provide information on the monthly usage of the QualityTools Web
    site and then the Innovations Clearinghouse when available

-   provide the earned value information requested in 3.10.8

-   include the "Anomalies Reports" as requested in task 3.5.38

    1.  The monthly progress report need not exceed ten (10) pages and
        shall also include disclosure of any new work products, computer
        databases or programs, or any other inventions that may be
        subject to intellectual property protection, together with a
        description of how the Contractor intends to manage intellectual
        property rights to these inventions under the terms and
        conditions of the contract.

> **Management meetings**

2.  During the project, at a minimum, the Contractor shall hold a
    monthly management meeting at its office during which the project
    will be discussed. The participants in the management meetings will
    include the Project Officer (in person or via telephone) and other
    Agency staff as appropriate, the Project Director, and relevant
    project staff as selected by the Contractor. Progress will be
    reviewed, milestones discussed, problems examined, solutions
    explored, and short-range and long-range plans made. Action items
    resulting from this meeting shall be prepared by the Contractor and
    submitted to the Project Officer within three days after completion
    of the meeting. If corrections are necessary, a revised replacement
    shall be submitted within two (2) days after receipt of the Project
    Officer's comments.

> **Annual reports**

3.  The Contractor shall prepare an outline, draft and final annual
    report for the Project Officer that compares work performed in the
    current year to the work that was planned for that year. At a
    minimum, the annual reports will include full details on the purpose
    of the contract, methods of performance, activities undertaken to
    accomplish tasks and goals of the contract and whether the tasks
    were successfully completed, findings, conclusions, and
    recommendations resulting from the work performed. An outline (to
    identify substantive content) shall be completed by the 11^th^month
    of the contract, and every 12 months thereafter during the contract
    period. Draft reports shall be completed by the 12^th^ month of the
    contract, and every 12 months thereafter during the contract period.
    Final annual reports shall be completed within two weeks after
    receipt of PO feedback. This same time cycle of submitting outline,
    draft, and final reports will continue if the period of contract
    performance is extended. The annual report shall be organized to
    reflect the structure of the WBS (at a high-level).

4.  Generally, the annual report will include a cover, title page, table
    of contents, text with associated graphics, and addenda (e.g.,
    appendices, glossary, bibliography, indexes). The main text of the
    annual report shall address all the requirements listed and provide
    all information called for in the scope of work of the contract
    unless otherwise specified by the Agency. All reports, including
    text, tables, and graphics shall be provided in hard copy and as an
    electronic file in a format that is acceptable to the Project
    Officer (e.g., Microsoft Word)

5.  The annual report shall also include an executive summary that
    concisely describes the results to a non-technical audience, which
    may include but not be limited to policymakers and program
    administrators. The executive summary shall be complete and able to
    stand alone as a separate document.

6.  The Contractor shall submit the revised/approved report with
    three (3) hard copies to the Project Officer, one (1) copy to the
    Contracting Officer, and one (1) electronic copy to both the Project
    Officer and the Contracting Officer.

> **Final report**

7.  Two months before the contract ends, the Contractor shall submit a
    draft final report. It will summarize the full contract experience,
    such as: (1) accomplishments of contract objectives; (2) technical
    specifications; (3) evaluations of barriers encountered; (4)
    recommendations to the Agency on ways to improve the process and
    products. The Project Officer may suggest revisions or approve the
    draft. At the end of the final month of performance, the Contractor
    shall submit the revised/approved report with four (4) hard copies
    to the Project Officer, one (1) copy to the Contracting Officer, and
    one (1) electronic copy to both the Project Officer and the
    Contracting Officer.

> **Close-out/Transition Plan**

8.  The Contractor shall prepare and submit a draft close-out/transition
    plan four (4) months before the contract expires. The plan will
    include information on those procedures, documents, files, etc.
    necessary, especially non-IT related, to close-out/transition the
    project. The plan shall be implemented following PO approval and the
    final close-out/transition plan shall be included as part of the
    final report (task 3.10.21).

> **Other Agency contractors**

9.  Work with other Agency content/substance-related Contractors related
    to either the innovations or tool modules. Because some content of
    this contract will be driven or influenced by other existing or
    subsequent Agency contracts that produce new or modified
    information, products, or tools, this Contractor shall, as needed,
    work with other Agency Contractors to enhance the Innovations
    Clearinghouse content.

> **Agency stakeholders**

10. The Contractor shall plan to attend, teleconference or
    web-conference meetings with Agency stakeholders, at the direction
    of the Project Officer, two times a year for four hours each
    meeting. Meeting location will be dependent on the needs of the
    project and the interest in keeping travel and meeting expenses
    down. For example, if the project could benefit from input by
    stakeholders in rural areas after the site has gone live, rather
    than travel those stakeholders to the metropolitan Washington, DC
    area, the Contractor can propose a web-conference that permits
    stakeholders, the Contractor, and the Project Officer to participate
    from their own locations, connected to each other by the Internet.

**SOW** **APPENDIX A**

**GLOSSARY OF KEY TERMS**

**Used in the Innovations Clearinghouse RFP**

For the purposes of this procurement the following key terms and
definitions will be used:

+---------------+------------------------------------------------------+
| Adoption      | A decision to make full use of an innovation as the  |
|               | best course of action available.                     |
+---------------+------------------------------------------------------+
| Back-end      | In their most general meanings, the terms **front    |
|               | end** and **back end** refer to the initial and the  |
|               | end stages of a process flow. These terms acquire    |
|               | more special meanings in particular areas. The       |
|               | general idea is that the front-end is responsible    |
|               | for collecting input from the user, which can be in  |
|               | a variety of forms, and processing it in such a way  |
|               | that it conforms to a specification that the         |
|               | back-end can use. The connection of the front-end to |
|               | the back-end is a kind of interface.                 |
+---------------+------------------------------------------------------+
| Certification | A process designed to certify that a system meets    |
| &             | documented security requirements and will continue   |
| Accreditation | to maintain the accredited security posture          |
| (C&A)         | throughout its system life cycle                     |
+---------------+------------------------------------------------------+
| Change agent  | An individual who influences innovation decisions in |
|               | a direction deemed desirable by a change agency      |
|               | (Rogers, 2003).                                      |
+---------------+------------------------------------------------------+
| Clearinghouse | A central agency for the collection, classification, |
|               | and distribution especially of information           |
+---------------+------------------------------------------------------+
| Development,  | The initial creation of an IT system (design,        |
| Modernization | develop, test and deploy), adding features to an     |
| and           | existing IT system (enhancement), or changing the    |
| Enhancement   | technology of an existing IT system (modernization)  |
| (DME)         |                                                      |
+---------------+------------------------------------------------------+
| Diffusion of  | The process of facilitating the spread and           |
|               | acceptance of research-based innovations through     |
| Innovations   | specific channels over time (Rogers, 2003).          |
|               |                                                      |
|               | The broad dissemination of new ideas, procedures,    |
|               | techniques, materials, and devices and the degree to |
|               | which these are accepted and used. (National Library |
|               | of Medicine, 2005 Medical Subject Heading)           |
+---------------+------------------------------------------------------+
| Dissemination | The packaging, distribution, and reception of        |
|               | knowledge or products across settings. An active and |
|               | planned effort to encourage target groups to adopt   |
|               | an innovation.                                       |
+---------------+------------------------------------------------------+
| I             | An active and planned process of putting             |
| mplementation | evidence-based information to practical use in the   |
|               | field. Examples include incorporating a treatment    |
|               | protocol into practice, customizing a policy or      |
|               | procedure for local use, or institutionalizing a     |
|               | policy or procedure for routine use.                 |
+---------------+------------------------------------------------------+
| Independent   | The verification (does the product conform to the    |
| Validation    | specifications?) and validation (does the product do |
| and           | what the user really requires?) of a software        |
| Verification  | product by an organization that is both technically  |
| (IV&V)        | and managerially separate from the organization      |
|               | responsible for developing the product (The Free     |
|               | On-Line Dictionary of Computing (2003)).             |
+---------------+------------------------------------------------------+
| Innovation    | An idea, practice, or object that is perceived as    |
|               | new by an individual, or other unit of adoption      |
|               | (Rogers, 2003).                                      |
+---------------+------------------------------------------------------+
| Innovation    | A novel set of behaviors, routines and ways of       |
| (relating to  | working, which are directed at improving health      |
| health        | outcomes, administrative efficiency,                 |
| service       | cost-effectiveness, or user experience, and which    |
| delivery)     | are implemented by means of planned and coordinated  |
|               | action (Greenhalgh et al, 2004).                     |
+---------------+------------------------------------------------------+
| Positive      | People in an organization or group who are already   |
| deviants      | doing things in a radically better way and can serve |
|               | as indigenous sources of change. They are usually    |
|               | individuals on the periphery of their organizations  |
|               | or societies who are far removed from the            |
|               | orthodoxies of mainstream change endeavors. These    |
|               | innovators\' uncommon practices and behaviors enable |
|               | them to find better solutions to problems than       |
|               | others in their communities (Pascale and Sternin,    |
|               | 2005).                                               |
+---------------+------------------------------------------------------+
| Priority      | Inner-city; rural; low income; minority; women;      |
| Populations   | children; elderly; and those with special health     |
|               | care needs, including those who have disabilities,   |
|               | need chronic care, or need end-of-life health care   |
+---------------+------------------------------------------------------+
| Reinvention   | The degree to which an innovation is changed or      |
|               | modified by a user in the process of its adoption    |
|               | and implementation.                                  |
+---------------+------------------------------------------------------+
| Stakeholders  | Organizations or members of organizations who are    |
|               | directly affected by AHRQ research activities and    |
|               | findings. These include the leadership of and        |
|               | providers within health systems such as hospitals,   |
|               | managed care organizations, and other provider       |
|               | organizations; State and local health policy makers; |
|               | third-party payers; health care service coalitions;  |
|               | professional associations of these groups and their  |
|               | members; and health services researchers.            |
+---------------+------------------------------------------------------+
| Steady State  | The maintenance and operation costs at current       |
| (SS)          | capability and performance level including costs of  |
|               | personnel, maintenance of existing information       |
|               | systems, corrective software maintenance, voice and  |
|               | data communications maintenance, and replacement of  |
|               | broken IT equipment                                  |
+---------------+------------------------------------------------------+
| Strategy      | A careful plan or method devised toward a goal       |
+---------------+------------------------------------------------------+
| Web crawler   | A program that invokes web pages. The program uses a |
|               | list of starting web addresses and invokes the home  |
|               | page of each site in the list. Then the program      |
|               | scans through the home page looking for new Internet |
|               | addresses. Each time the web crawler program invokes |
|               | a new page in the hierarchy of pages on a given      |
|               | site, the program will look for new addresses and    |
|               | add them to the list of pages to invoke. This        |
|               | process continues until there are no more pages to   |
|               | add to the list. As the program invokes each page,   |
|               | it saves the contents of the page to local storage   |
|               | (usually a local database). This local storage of    |
|               | the 'crawled' web pages in called the 'web cache'.   |
|               | Web crawlers use their web cache to power their      |
|               | search engine.                                       |
+---------------+------------------------------------------------------+

BIBLIOGRAPHY

Greenhalgh T, Robert G, Bate P. How to spread good ideas. A systematic
review of the literature on diffusion, dissemination, and sustainability
of innovations in health service delivery and organization. Report for
the National Co-ordinating Centre for NHS Service Delivery and
Organisation R&D (NCCSD). April 2004.
<http://www.sdo.lshtm.ac.uk/changemanagement.htm>

Downloaded and accessed 12/30/2004

HHS, Certification and Accreditation Guide, July 19, 2005

Pascale RT, Sternin J. Your company's change agents. *Harv Bus Rev*. May
2005; 73-81.

Rogers EM. Diffusion of innovations. Fifth Edition. New York, NY: Free
Press. 2003

The Free On-Line Dictionary of Computing (2003).
<http://foldoc.org/foldoc.cgi?Independent+Verification+and+Validation>
Accessed 03/20/2006

Wikipedia. <http://en.wikipedia.org/wiki/Back-end>. Accessed 03/29/2006

**SOW APPENDIX B**

Innovations Clearinghouse Expert Panel Meeting Summary

![](media/image1.png){width="1.8125in" height="0.7777777777777778in"}

**U.S. Department of Health and Human Services**

Innovations Clearinghouse

Expert Panel Meeting

Meeting Summary

March 9, 2005

![](media/image2.png){width="1.1395833333333334in"
height="1.1395833333333334in"}

**Welcome, Introductions, Purpose**

###### Jean Slutsky, P.A., M.S.P.H

*Director, Center for Outcomes and Evidence*

*Agency for Healthcare Research and Quality (AHRQ)*

*Larry Bartlett, Ph.D.*

*Director*

*Health Systems Research, Inc.*

Ms. Slutsky opened the meeting. She thanked participants for attending
and explained that the meeting was an effort for the Agency for
Healthcare Research and Quality (AHRQ) to ensure that an idea being
tested is useful to those in the front lines. She asked for candid
input, both positive and negative.

The concept of an Innovations Clearinghouse was conceived by a broad
intra-agency committee headed by Mary Nix, M.S., M.T., S.B.B., project
officer for AHRQ's Center for Outcomes and Evidence; and included
Christine Williams, M.Ed., director of AHRQ's Office of Communications
and Knowledge Transfer; Cindy Brach, M.P.P., senior health policy
researcher for AHRQ's Center for Delivery, Organization, and Markets;
Beth Collins Sharp, Ph.D., R.N., health scientist administrator for the
Center for Outcomes and Evidence; and Margaret Coopey, M.P.S., M.G.A.,
R.N., senior health policy analyst for the Center for Outcomes and
Evidence. Now, input is being sought from those outside of AHRQ.

Dr. Bartlett facilitated the meeting. He emphasized the importance of a
range of different perspectives and asked participants to introduce
themselves in the context of how the clearinghouse under consideration
would be useful to them.

-   Mary Nix is the project lead for the Innovations Clearinghouse and
    also for the National Guideline Clearinghouse, National Quality
    Measures Clearinghouse, and QualityTools Web site.

-   Liza Greenberg, R.N., M.P.H., of Health Project and Management
    Consulting, is a clearinghouse customer.

-   James Mold, M.D., M.P.H., is a geriatrician involved with chronic
    care and preventive services. He uses best practices research,
    identifies practitioners already using best practices, and uses
    practice enhancement systems. Dr. Mold is a professor at the
    University of Oklahoma Health Sciences Center and director of the
    Research Division of the Department of Family Medicine and
    Preventive Medicine.

-   Robert Rosati, Ph.D., director of outcomes analysis and research for
    the Visiting Nurse Service of New York, indicated his group's
    efforts are focused on quality improvement (QI), including what
    makes home care better, workforce issues, and improving transitions
    between hospital and home care to avoid readmission.

-   St. Anthony Amofah, M.D., M.B.A., is medical director of the Health
    Choice Network in Miami.

-   Brian Mittman, Ph.D., is senior social scientist for the Veterans
    Administration (VA) Center for the Study of Healthcare Provider
    Behavior in Los Angeles. He is directing an effort to launch a new
    journal, which might include a Web site aimed at researchers.

-   Nancy Ridley, M.S., assistant commissioner, Center for Patient
    Safety, Massachusetts Department of Public Health, noted that she is
    the "token State regulator." She is the principal investigator for
    an AHRQ cooperative agreement and oversees bioterrorism cooperative
    agreements with the Health Resources Services Administration (HRSA)
    in Massachusetts.

-   Donald Casey, Jr., M.D., M.P.H., M.B.A., is chief medical officer
    for Catholic Healthcare Partners, a five-State organization
    headquartered in Cincinnati. His work focuses on quality of care. He
    has been involved with AHRQ since the early 1990s, with a focus on
    guidelines. He is currently involved with the National Quality Forum
    and incoming chair of the Ohio Patient Safety Organization.

-   Melanie Bella, M.B.A., is vice president for policy at the Center
    for Health Care Strategies, in Princeton, NJ. She is launching a
    national chronic care innovation.

-   Jon Kerner, Ph.D., deputy director for research dissemination and
    diffusion for the Division of Cancer Control and Population Sciences
    of the National Cancer Institute (NCI), commented that his division
    does much of the health services research in cancer, including a
    number of collaborations with AHRQ around knowledge transfer.

-   Steve Hines, Ph.D., is a senior scientist at the Delmarva
    Foundation, a quality improvement association and potential customer
    for the proposed clearinghouse. He also has close connections with
    the provider community and works with the National Guideline
    Clearinghouse and MedQIC, the quality clearinghouse of the Center
    for Medicaid & Medicare Services (CMS).

-   Susan O'Brien, M.S., R.N., is a clinical nurse from Madison, WI, and
    has been working with the University of Wisconsin to develop
    clinically based quality care guidelines. She has a personal
    interest in transitions.

-   Bette Seamonds, Ph.D., DABCC, is a clinical biochemist and director
    of chemistry and point-of-care services for Mercy Health Laboratory
    near Philadelphia. She has experience in both industry and academia,
    and works on quality improvement for a network of hospitals,
    developing tools to assess services.

-   Laura Line, M.S., is deputy executive director of the Philadelphia
    Health Management Corporation, where she directs household-level
    health services. Her company is working on improving data collection
    and research.

-   Tine Hansen-Turton, M.G.A., executive director, National Nursing
    Centers Consortium, based in Philadelphia; Jill Rogers, M.S.N.,
    section chief for Health Promotion and Disease Prevention for the
    Delaware Division of Public Health; and Karen Nelson, R.N.,
    assistant vice president for quality management, North Shore---LIJ
    Health System, joined the meeting after the introductions.

Dr. Bartlett reviewed the agenda, which he described as straightforward
and transparent. After an overview of the proposed Innovations
Clearinghouse from Ms. Nix, participants will be asked to address a set
of questions about how useful such a clearinghouse would be for them,
how they could use it, what its content should be, how it should be
designed, and what features would draw in customers. The meeting will
end with a discussion of take-home messages. He encouraged participation
from everyone, because each person at the table has a unique perspective
to share.

## Overview of Clearinghouse for Innovations of Change

*Mary Nix, M.S., M.T., S.B.B.*

*Project Officer, Center for Outcomes and Evidence*

*AHRQ*

Ms. Nix began by highlighting AHRQ's mission: to improve the quality,
safety, efficiency, and effectiveness of health care for all
Americans---a big mountain for a small agency. This is why AHRQ looks to
those outside the agency for help in working toward that mission.

For the new initiative, AHRQ seeks to increase the implementation of
health care improvement strategies in the United States, but this will
not be easy. The American health system is complex, with varying degrees
of modernization, and the implementation of improvement strategies
requires innovation and change. Even after an innovation or change has
been implemented, diffusion to others can be slow. Innovation in health
service delivery is multidimensional.

One possible solution is to capture those dimensions and their details
in a systematic and controlled manner that translates innovation,
facilitates replication, permits research, and promotes innovation and
reinvention. Then a forum must be provided to learn about innovation.
AHRQ's concept is a national repository of improvement or change
initiatives and packages. The name is yet to be determined; for now, it
is being called the Innovations Clearinghouse.

Innovation in health service delivery is defined as "a novel set of
behaviors, routines, and ways of working, which are directed at
improving health outcomes, administrative efficiency,
cost-effectiveness, or user experience; and which are implemented by
means of planned and coordinated action." This definition comes from a
British report, which can be found at
www.sdo.lshtm.ac.uk/changemanagement.htm.

AHRQ's concept for an Innovations Clearinghouse is modeled after---but
not limited by---its Guideline and Quality Measures Clearinghouses, and
Web Morbidity and Mortality (M&M). (These can be accessed, respectively,
at www.guideline.gov, www.qualitymeasures.ahrq.gov, and
www.webmm.ahrq.gov.

The Innovations Web site will be characterized by a number of major
components. These include a searchable database with defined attributes
and the capacity for online comparisons. The ability to offer "change
packages" and/or syntheses of experience is also planned, as well as
additional resources. The site will contain dynamic communities,
including a learning forum, opportunities to submit stories and lessons
learned, and a comments field for users to describe additional
experiences, such as that used by Amazon.com. It will include links to
many established Web sites, including CMS' MedQIC, the Health
Disparities site, and the Institute for Healthcare Improvement (IHI)
site. The Lessons Learned Information Sharing site of the Department of
Homeland Security is a Web site that the Agency's internal work group
has looked at as a model for the Innovations Clearinghouse. A final
component is to incorporate, modify, expand, and relate to AHRQ's
Quality Tools Web site (www.qualitytools.ahrq.gov), because tools are
used to help provide innovation in health service delivery.

In summary, Ms. Nix presented a list of what the Innovations
Clearinghouse should contain, based on other successful Web sites and
the experiences of others voiced to the Agency:

-   Provides a national-level, publicly accessible repository where
    improvement strategies and associated tools can be easily and
    quickly found.

-   Is searchable.

-   Has the capacity to compare and assess strategies for potential
    applicability with a similar setting.

-   Spans the full range from clinical to workforce to system design
    innovations.

-   Has the ability to offer change packages, compilations, and
    clusters.

-   Can make apparent the level of evidence supporting or validating a
    change initiative.

-   Supports online or offline networking and communications
    opportunities.

-   Covers many clinical and operational areas, with no content gaps.

-   Presents lessons learned, even when not favorable, so what went
    wrong can be explored.

-   Ensures access to those with disabilities.

This is a great deal, she acknowledged, but she hopes that the input
from today's meeting will help make it a reality.

**Discussion---Part 1: Proposed Innovations Clearinghouse Concept**

Dr. Mold asked for examples of innovations, noting that they can be both
small and large. Ms. Nix indicated AHRQ is not talking about innovations
in technology or drugs or devices, but rather the innovations used in
practice and systems. She added, however, that she does not want to bias
participants with AHRQ's thinking, but wants participants to help define
what innovations they would like to see in a clearinghouse. Dr. Bartlett
emphasized the importance of defining what the clearinghouse will
contain. He explained that AHRQ is also interested in getting a sense of
how the clearinghouse might fit into what participant organizations do
on a daily basis to determine whether or not it would be useful.

Ms. Greenberg asked who would be using the clearinghouse. Ms. Nix
explained that users would be people who find themselves in the same
situation as AHRQ---trying to improve the quality of care they deliver.
Ms. Slutsky added that although it is expected that the Innovations
Clearinghouse user base will be broad, this site will be designed for
those at the interface of making change for quality improvements.

Dr. Bartlett asked participants if any particular aspects of places they
go for information might limit or focus the scope of this initiative.
Dr. Rosati felt that it is a hit-or-miss process. For example, recently
he hit on a Web site of another organization with an innovation similar
to what his group was looking for, and this is typical. He would hope
for a clearinghouse that is not so hit-and-miss and reduces the enormous
effort it now takes to reach out and find things. People need to know
there is a place to submit innovations.

Dr. Amofah commented that in the context of quality management, no one
wants to reinvent the wheel. Most innovation sharing occurs within his
organization, but also through conferences and the Internet, and he
agreed that much is hit-and-miss. A single place to look at innovations
would be beneficial. He hopes the project will begin with basic
innovations, how to support them, the evidence behind them, and examples
of how they have succeeded.

Ms. Bella indicated that information comes primarily through word of
mouth, and there is a need to know about innovations happening across
sectors, including the private sector. She added that innovation is not
always a positive word for legislators, but people will use something if
it is timely, operational, and instructive. Practitioners want to know
something works, and they are reluctant to be the first to use
something.

Dr. Bartlett asked what Medicare is looking for in quality improvement.
Ms. Bella responded that she would want a how-to approach that explains
how innovations could be operationalized and daily activities improved.
Policymakers need to see that the payback is real and something has
worked elsewhere.

Dr. Mittman reported that the VA represents a core and ideal audience,
with 150 hospitals, 800 freestanding clinics, and 3 to 5 million
patients. Each clinic and hospital has a QI manager, and he polled them
about how they now seek and implement innovations. Most cited literature
and Internet searches as taking precedence over personal contacts. Most
people are accustomed to finding things on the Internet, which supports
the approach of the clearinghouse. A Web site would be useful, save
time, and implement the work of others, particularly if it is
evidence-based. The QI staff would be the main audience, because they
are actively seeking this kind of information, which would be compatible
with the needs of VA quality managers.

Ms. Ridley commented that State health agencies do not have the same
challenges as Medicare, but she also likes the proposed initiative. She
was concerned, however, about a comment that it not be a portal, because
of the wide number of existing sites and varying opportunities to find
what purports to be best practices, but no portal to classify the
evidence base of what is on those sites. A State regulator must be
cautious, and often best practices or guidelines tend to be a quick fix
for regulatory agencies, particularly in the case of high-profile events
with media coverage. It is important that clearinghouse guidelines not
be adopted for purposes for which they were not intended. Dr. Bartlett
noted that this relates to the question of level of evidence.

Dr. Kerner felt that one important consideration is whether the proposed
clearinghouse is being designed for managers of today or managers of
tomorrow. He discovered a surprising amount of comparability on issues
of knowledge transfer with a Department of Defense colleague; field
commanders, for example, want instant messaging in the field. A
transformation is taking place---someone earlier mentioned word of
mouth, but there is a new world of virtual word of mouth, and this must
be addressed. Things are changing rapidly, and AHRQ has to keep up with
the changes. Preliminary concepts must be reviewed by potential users
every step of the way, and many ideas will be discarded in what can be a
painful process.

Dr. Kerner added that he agrees with the need for a Web portal---a lens
to focus on the most important things to speed up the search process. He
envisions the site as a mix of unique material and targeted links to
other vetted material. Users, he felt, will want the answers to several
questions:

-   Where do I stand in relation to these models?

-   How do I find these models of change? This can be time consuming and
    expensive, but it is necessary if the clearinghouse is to be useful.

-   How do I know when I am there? Evaluation tools and information
    about previous use are needed. Context is very important.

Dr. Hines reported that he is also thinking about bigger picture issues
and experience with MedQIC. He posed six key questions and issues:

1.  Who is the audience? It is daunting to envision a site that will be
    optimized for everyone from State health care policymakers to nurses
    working for QI. One-size-fits-all may end up not working for anyone.

2.  What is the purpose (which is related to audience)? Some may just be
    looking for ideas to try; others may be looking for proven fixes
    with a higher standard. Networking---getting connected to people
    informed about innovations---is another possible purpose.

3.  What is the scope?

4.  What is the process of putting the site together? IHI and MedQIC
    both have a model that might be applicable with some tweaking. Those
    who develop the innovations are not the same people who will
    disseminate them, which will require a more centralized process of
    developing categories and packaging.

5.  The structure must be driven by users.

6.  The site must be high quality. Another mediocre site is not going to
    do anyone any good.

Ms. Slutsky acknowledged that the Innovations Clearinghouse is the most
applied part of a larger Agency initiative. AHRQ envisions the Web site
as a complex database with controlled content. AHRQ is painfully aware
of the level of effort required but is also aware that quality is
commensurate with effort.

Dr. Kerner asked if, given the complexity and time investment involved,
there are potential partners for AHRQ to work with. NCI faced a big
challenge in reviewing published research and encountered a review of
published research by the Substance Abuse and Mental Health
Administration (SAMHSA), which they were able to use. Ms. Slutsky agreed
on the need to survey other efforts and determine who has done something
well. Many innovations have occurred in sectors other than health care,
she added, and the lessons-learned feature of the Homeland Security site
is "stunning," although it is a secure site not open to the public.

Dr. Seamonds noted that everyone faces the big issue of the dwindling
dollars available to implement QI and fewer people to develop programs.
The availability of a clearinghouse to provide tools, algorithms, and
packages as models is a great concept, she continued, and could truly
make a difference if done well, but it must be widely applicable.

Ms. Hansen-Turton indicated that her organization deals with both policy
issues and practical tools, and State-by-State analyses are useful in
learning how to position the organization. She suggested Lexus/Nexus as
a good, well-organized Web-based tool with useful search mechanisms that
could be modeled. She urged that the Innovations Clearinghouse include a
mechanism for State-by-State searches.

Ms. O'Brien agreed on the need for the clearinghouse to be interactive
and added that planners and implementers should also ensure that the
innovations are meaningful and include suggestions for small changes
that can be tried immediately. As the health care system becomes
increasingly cumbersome with sicker and sicker patients, bringing in
small changes could be helpful. And while an evidence base is needed, an
experience base is also critically important. She encouraged
experimentation and reporting results.

Ms. Line noted that not all centers have a staff member dedicated to QI,
and informal reporting at monthly meetings is often how information is
disseminated. If someone is doing something innovative, they will often
be invited to join meetings by conference call. Candor is another
aspect, and it is difficult to expect people to criticize themselves on
a formal site visit.

Dr. Mold suggested practice-based research networks as other potential
users who could use knowledge about what innovations need to be tested.
Also, it would be helpful for AHRQ to set standards on how innovations
can be documented. He is still not sure of what is meant by innovations
and asked for examples. He wondered if the following are examples,
noting that much innovation occurs at the practice level, and individual
doctors in solo practices are doing innovative things:

-   People with diabetes hold the A1C monitor in their hand as the
    reading comes up, giving them ownership of the result.

-   Practices negotiate with a mammography center for a block of time,
    so the practice can schedule mammograms for its patients.

-   Home health agencies meet regularly and have documentation of
    certification.

-   A larger innovation would separate preventive services from the rest
    of a practice, and establish a prevention center run by a nurse.

Ms. Slutsky responded that those examples are exactly what is meant. She
agreed on the need to talk about both large and small innovations. She
asked if any evaluation of the measures has been done, that is, are
diabetes patients more successful if they hold the A1C monitor? This
particular item would require a patient education component to have an
impact.

Ms. Nelson polled her health care system in Long Island on the questions
circulated before this meeting, and the repeated theme was that people
are looking for a clearinghouse that has a filter to avoid
garbage-in/garbage-out. People want ideas that are tried and true.

Ms. Rogers reiterated that she represents a State health department with
the challenge of straddling the evidence base and trying new things,
which can be very expensive, with no guarantee that something will work.
She would appreciate a site that documents outcomes of the use of
innovative measures. Legislators need to see that something is both
evidence-based and practice-based.

Dr. Bartlett summarized the discussion so far with the following points:

-   It is clear that participants think there is a need for an
    Innovations Clearinghouse.

-   However, establishing one will involve a significant effort.

-   Just another mediocre Web site will not be a big contribution.

-   Many interests will be served, but in trying to serve many, do not
    slip into mediocrity.

-   The clearinghouse must present useful tools, and not be advertising
    or wish lists.

Dr. Kerner noted that the standard of reporting is very important; these
standards exist for research, but are much harder to determine and
implement for clinical practice innovations from the field. To prevent a
clearinghouse from becoming a tool for people to promote unproved ideas,
he suggested a contingent to submitting an innovation; the submitter
would be available to respond on a Web forum to questions from the field
about costs, possible resistance, and other aspects of the innovation.
Ms. Slutsky indicated there would be further discussion on this, and
AHRQ is working on getting clearance from the Office of Management and
Budget (OMB) for such a forum, where answers would go into a database.
The tension point is between the burden of the forum on the submitter
and the value of additional information.

Dr. Casey spoke of a quality awards program that has been implemented
and relates to instant messaging; it includes a mechanism for the
submitter to get feedback about strengths and weaknesses and is not
burdensome. Weak spots will be around evaluation, and specific
methodologies to document costs and benefits have not been thought
through. It is essential to know what works. For example, did direct A1C
readouts result in better diabetes control? Tools to provide
infrastructure support to enhance evaluation should also be provided.

Dr. Seamonds commented that some evidence-based and outcomes studies
have already been done. A concrete example is in the area of nutrition
monitoring and management, where there is a body of literature that
documents decreased lengths of stay and improved outcomes in hospital
patients who receive total parenteral nutrition. This provides a good
starting point to develop algorithms, because they already exist for
this procedure.

Ms. Ridley noted that the Homeland Security site on lessons learned
suggests some linkage possibilities. Massachusetts gets money from HRSA
for preparedness, which covers issues such as patient safety and
overcrowding and surge. At the State level, this funding could be used
to develop joint initiatives between various programs. With all the
challenges of the health care system, providing resources for safety and
preparedness has been added to the burden, and more one-stop shopping
sites would be helpful. Dr. Amofah agreed that hundreds of community
health centers and collaboratives would find such an initiative useful,
and efforts should be devoted to sharing methodologies. A training
component with video clips on how to use tools and methodologies would
be helpful.

Referring to submitters answering questions, Dr. Amofah felt AHRQ should
filter questions and feedback. Dr. Rosati noted that there are also
lessons to be learned from innovations that do not work, and it is
helpful to know the flip side of why something did not work.

Ms. Nix indicated AHRQ will launch measures that do not measure quality
but are associated with measures of quality, and it has some experience
trying to bring these types of information together. Ms. Greenberg
suggested an evidence pool, for example, linking the A1C innovation to
information about patient compliance and patient education.

**Discussion---Part 2: What Should Be in the Database and on the Web
Site?**

Dr. Bartlett noted that some of responses in the previous discussion
addressed this question, and the categories are somewhat fluid. He also
encouraged AHRQ representatives to make sure their questions are put on
the table.

Dr. Mittman returned to the topic of level of evidence, commenting that
the degree of impact on the effectiveness of an innovation might be
relatively small. It is impossible to say if something is effective or
ineffective without some information about context. Because something
works in the VA does not always mean it will work elsewhere, and this
could be impossible to predict. It will be a challenge for submitters to
think about factors they have not recognized.

Ms. Slutsky responded that innovations is a much more immature field
than guidelines, where some of that information is already available.
The proposed clearinghouse can guide the field in identifying factors
known to be associated with innovation and has the capacity to create a
global community that could help lead to standardization.

Dr. Mittman mentioned the danger of posting innovations on the Web and
people trying to adopt them and not succeeding. As the initiative gets
under way, warnings and disclosures are needed. Dr. Kerner remarked that
if the Web site is viewed as a tool to inform or share decisionmaking,
it is not necessary to have everything in place by the innovation
developer, because the user will come back with questions. An
educational component can focus on adaptation for individual settings.
Some innovations will end up not working, which is not necessarily a bad
thing, because it will develop feedback for improvement and nuances for
adaptability. There is value in creating a preliminary set of standards,
which are then complemented with program implementation guidelines.
Searching and browsing capabilities are very important. It is difficult,
but if viewed as a partnership between the innovator and the user, and
an iterative process, it is doable.

Dr. Barnett asked about other ancillary features of the Web site in
addition to building a global community; for example, people in a
certain field exchanging stories in real time. Dr. Hines cautioned that
evidence is critical for anything posted on the Web site, but the
standard should not be that only innovations backed by evidence can be
posted. That limitation would hide many things, and a discussion about
why things did not work can be more valuable than evidence that
something does work. Ms. Slutsky responded that the subcommittee agrees
that negative experiences should be incorporated, and the lack of
published negative results can be frustrating to clinical care
practitioners and bias differences in effectiveness and efficacy.

Dr. Mold suggested that the REAIM model be used as the basis for the
Clearinghouse; add another I, for impact. Dr. Kerner commented that a U
could be added for unintended consequence. Dr. Mold explained that not
many programs will have the desired quality of evidence of the
previously mentioned hospital nutrition program. Few clinical practice
guidelines have been tested in the field, and when they are, they often
fail. They may not fit into a clinical practice and can actually
increase costs.

Dr. Amofah noted that some innovations will be evidence-based, but some
will be experienced-based. The classification of innovations can address
this, and it is important that they can be easily searched through a
range of categories and scenarios.

Ms. Hansen-Turton reported that she works with the Steps to a
HealthierUS program, which includes both community-based and
evidence-based initiatives. The program is running into problems of
promising practices that do not work in communities, and tips on
adaptability would be useful. However, she added, for adaptability, the
onus is on users to know their own communities. Trying to adapt too many
tools could be expensive. Many States and other HHS agencies (for
example, HRSA's Bureau of Health Professions) are working with
innovations, and this clearinghouse needs to capture them. Another
important factor is a source of funding, to provide a sense of who is
paying for studies.

Dr. Rosati agreed that evidence, cost, and ease of implementation are
all important issues. He suggested establishing metrics around certain
parameters to rate evidence, cost, and other factors. Implementation
will be an issue, whether this is peer reviewed or not. Dr. Kerner
warned that the process of deriving objective peer reviewable criteria
can be difficult. A more practical approach is to provide guidance on
implementation, costs, unintended consequences, and other issues; then
it is up to the user.

Dr. Bartlett then asked what types of submissions would *not* make it
into the clearinghouse.

Ms. Nelson suggested a mechanism of comparison, so a user could filter
innovations for further investigation. What will or will not work will
be different for each user. She would like to see ideas that have
definitely worked on the Web site. Dr. Kerner added that how submissions
are classified is important. For example, one area could be an "epiphany
corner" for what seems like a great idea but has never been tried. And
another could be for something that has been replicated a dozen times.
He does not believe a peer review process is necessary for this
classification.

Ms. Nix noted that the internal workgroup discussed a scale to classify
innovations, and Dr. Collins Sharp has been in touch with experts in
this area. Paul Plsek, a consultant with a background in engineering,
has focused on bringing modern quality management techniques to health
care organizations, developed a scale to test level of innovation, and
is now testing it. Dr. Collins Sharp asked if participants thought that
setting a bar for evidence would help or inhibit use of the site.

Dr. Mittman suggested that if something has little evidence, start with
a pilot; with more evidence, an idea can be implemented on larger scale.
Dr. Amofah noted that some people are deterred by a certain level of
detail, and most users need to be convinced how well something will
work, with access to details and someone who can explain it. Dr. Kerner
questioned the use of restaurant review-type feedback, with users
invited to give their reviews of how something worked for them.
Voluntary feedback information could be very helpful. Dr. Bartlett
commented that Amazon.com and MedQIC both use this device. Another
participant noted that this type of feedback can come directly or
indirectly from vendors and questioned whether the site should ban
feedback from vendors. The other side of this is that feedback can also
come from competitors.

Ms. Slutsky responded that this is not a new issue. When vendors develop
guidelines or measures and quality tools, the question becomes one of
copyright---is it possible to provide sufficient information about an
innovation for users to see what it is without violating its copyright?
Copyrighted material must be described with the same detail as
noncopyrighted material. Ms. Williams emphasized the need for submitters
to be identified so the source of information is clear. Dr. Kerner added
that reviews are just reviews, with no mandate or official endorsement
implied.

Dr. Bartlett asked if it is important that innovations presented on the
Web site include a contact to answer questions and provide further
information. Participants were unanimously in favor of this feature. Dr.
Amofah reframed the question to ask participants if they would be
willing to be that contact person. While most responded affirmatively,
it was without the enthusiasm of the response to the first question.

Ms. Ridley mentioned a successful program that links to a Webcast on
preparedness and patient safety, with authors and key sponsors
identified, and with virtual real-time surveys used for some questions.
That might provide an opportunity to get questions answered without
burdening submitters with hundreds of telephone calls or e-mails.

Ms. Slutsky was struck by the fact that everyone wants a contact point,
but how much less enthusiasm there is about *being* a contact person.
Ms. Greenberg felt this could be a feasibility issue; contacts can
quickly become out of date. Ms. Slutsky responded that this has not been
a problem for the Guideline Clearinghouse. Dr. Amofah noted that direct
contact could be overwhelming. Ms Slutsky suggested a mailbox where
general or specific questions can be sent, as well as someone's
preferred method of contact for very specific questions.

Dr. Rosati concurred that the vendor issue is very important---if a
vendor wants to get an innovation out, people implementing it will be
the driving force. That is distinct from promoting something for
ulterior motives.

Dr. Casey recommended more focus on strategy and felt reducing health
disparities should be part of the strategy. He also suggested the
creation of risk assessment tools, so potential users can assess their
readiness to try something. Just because a large number of people failed
at an innovation does not mean it is a failure. For example, attempts to
improve hospital chronic care fail frequently, because people do not
want to make the necessary changes. When considering an evidence base,
be sure that evidence-based outcomes are necessary and important.

Ms. Williams reiterated that if something did not work, where it failed
should be examined. Factors that led to failure could be tweaked to lead
to success. Sometimes something that did not work in one system can be
implemented in another, but often ideas do not spread beyond the system
in which they originated. Dr. Bartlett suggested the Web site include
examples of failure, an analysis of shared experience, and broader
lessons learned. Ms. Slutsky responded that one reason AHRQ planners
wanted to use a structured database format was to allow people to look
across formats at the broader picture, and contact information should be
included for specific intervention information for users. When polled
again, participants were more positive about being identified as contact
points for information and voted unanimously that they would be willing
to serve in such a role. Dr. Bartlett qualified the vote with the caveat
that this might be a self-selected group.

Dr. Mittman suggested a moderated site, with clinical evidence that
lists randomized controlled trials, evidence tables, and a supporting
narrative, similar to the clinical evidence package that the *British
Medical Journal* presents. Evidence is packaged in easily usable
formats. This would require teams of experts responsible for each
clinical area, but it would provide a first point of contact to relieve
some of the burden on the innovator. It would be not so much a raw
database as a rolling systematic review. If the bar is set at good
clinical research evidence, he added, this would be an empty database.

Ms. Nix agreed that the vision Dr. Mittman described is shared by the
planning committee. His description is similar to how Web M&M works,
with governance that oversees cases that are submitted and experts who
do commentary. Dr. Mittman commented that M&M is more of a case study
format.

Dr. Collins Sharp noted that the counterbalance for a contact person is
how much information is on the Web site. Sufficient information to allow
implementation will decrease the burden of contacts, but puts it at the
front end with the need for adequate description. This can come up
against the desire of vendors to not provide a full description of a
product because they do no want to give it away for free.

Dr. Mold commented that if principles behind something are
evidence-based, the evidence can be linked to an innovation, and the
most important aspect is setting the standard so there is an incentive
to provide information. With standards for reporting, people will have
to collect evidence. Setting a standard may end up advancing the field
more than the innovations themselves. He gave the example of the Great
Harvest Bread Company, a national franchise with a rule that if a
franchisee discovers something better than what company is already
doing, it has an obligation to share it with other franchisees---the
company provides the means and funding for the dissemination. He asked
how many participants would be willing to have onsite visitors to learn
more about an innovation. About five people said they would, and Dr.
Mold added that if AHRQ is serious about improving things, it should
fund such onsite visits.

Dr. Kerner emphasized that an incentive is needed to motivate people to
submit an idea to a Web site, particularly researchers accustomed to
publishing in journals. He worked with a public health program that had
a small supplemental grant program that included submission to a Web
site as a requirement of funding. That might be unrealistic in this time
of shrinking funding; another approach is simply to emphasize the value
to the field of sharing work. Small business innovation research grants
could be another implementation mechanism, although Ms. Slutsky
cautioned that for the foreseeable future, a financial incentive is not
likely.

Ms. Slutsky added that while much of this discussion has been about an
evidence base for innovation, part of the goal of this project is to
change the evidence base. She asked if innovations known not to be
effective should be promoted.

Dr. Amofah asked to what extent the Innovations Clearinghouse might
migrate toward clinical guidelines, noting that some innovations could
easily deal with clinical practice guidelines. Ms. Slutsky responded
that the clearinghouse will not migrate to clinical practice guidelines.
Dr. Amofah also asked about different types of tools---tools to help
implement a change, tools to identify a problem, tools to manage change.
Ms. Greenberg commented that instinct says not to let anything into the
database without supporting evidence, but it should include things that
are not clinical. Ms. Slutsky agreed that the focus is not clinical. Dr.
Mold felt this discussion has confused him about what innovations are
and asked if the collaborative model would be an innovation. Ms. Slutsky
indicated it is. Dr. Mold responded that it is a very different model, a
change innovation, nonclinical with very little evidence behind it. Dr.
Mittman noted that the collaborative approach is a care model and
evidence for a care model could be transferable. He thought it makes
sense to put different types of models on the Web site, but a focus on
the care model might be more useful.

Ms. Bella mentioned that more help is needed on implementation of the
models. Another challenge is packaging the site to make it known to
people who might not otherwise be aware of it. To build a better system,
a care model and training are needed, Dr. Mittman explained, and this
program is better suited to provide a care model than management
experience. Ms. Rogers observed that defining the climate in which
something succeeded can be as important as the actual interventions, and
organizations that might benefit most may not have the wherewithal to
think about that climate and need a prompt. A filter question could sort
what might work for a certain type of organization in a certain climate.
Dr. Rosati added that this gets to the question of what the site is
about, and a collaborative is a good example. The challenge is to say,
within a certain set of circumstances, what tools work and provide the
specifics about precedents. Dr. Amofah suggested that most of what is
needed is the "how?"

**Discussion---Part 3: Rewards and Burden on Contributors**

Dr. Bartlett opened this session by encouraging Ms. Slutsky and Ms. Nix
to be very direct with their questions, because there has been no
indication from the meeting attendees so far that their hearing AHRQ's
ideas inhibits creativity. Ms. Slutsky explained that a core set of
information is needed, but she understands that much of this does not
exist in written form. Rather, it is in the heads of individuals who
might already be overstressed. AHRQ is looking for ways to provide
incentives and make it easier for individuals to submit their
innovations. A Web forum will include standardized information, but
there also will be a need for narrative information. AHRQ is willing to
capture any information that already exists.

Ms. Ridley shared that the Massachusetts Department of Public Health
established a Web site that profiled a tool for patient safety research
grants and cooperative agreements, with information provided through a
combination of drop-down lists and narrative. It was updated at least
annually. Dr. Mold indicated that the problem is data entry. Most people
with innovations do some documentation for their own purposes, and they
would rather just upload that to a Web site than use the tools of the
site.

Dr. Mittman admitted surprise at how many organizations submit to the
National Guideline Web site, and Ms. Slutsky responded that she was on
the road for a year with a slide show explaining the process. Her
philosophy was, if you build it, they will come. Eventually, it became a
standard, but not without overcoming some obstacles. There were early
problems with electronic versions: some groups had excellent guidelines
that were not written, quality measures were difficult, and sometimes
technical aspects were voluminous.

Dr. Hines questioned voluntary innovation reporting as the core of the
site, noting that IHI and MedQIC had problems with not enough
submissions. A trained team might be able to extract the needed
information more efficiently and accurately than a well-intentioned
volunteer. He suggested that the core of the clearinghouse be a
dedicated team that harvests reports on innovations and puts them into
format. Ms. Slutsky agreed, and Ms. Nix explained that there has been a
request for funding for this type of personnel from one AHRQ research
portfolio, and she was gratified that others were thinking in the same
direction.

Ms. Greenberg suggested information harvesters go one step further and
borrow from organizations that have abstracts on hand. The hurdle is
having people create new information, and there is not much incentive
for that now. Dr. Mittman felt there is a hierarchy of groups that could
submit, ranging from large health care firms to smaller groups to
private practitioners. Starting at the top and working with large
groups, seeding the site with useful materials through a highly
leveraged effort might allow for sufficient content to show smaller
systems what AHRQ is looking for.

Dr. Bartlett asked what could be learned from the less than overwhelming
response to IHI. Dr. Hines answered that some of the comments about
incentivizing participation are very valuable. With IHI, the only
incentive was the sense of contributing to the common good. Getting the
site to the point where it is enough of a draw for people to see the
value of contributing information is one goal. Without sufficient
content, the site will not draw people in, and without people there will
not be any submissions, leading to a downward spiral. Ms. O'Brien noted
that IHI's 100,000 Lives campaign has pulled people to the Web site and
has been extremely valuable.

Dr. Seamonds suggested that the Innovations Clearinghouse Web site be
set up so neophytes could look for possible problem areas, and it could
serve as a good educational tool. Ms. Ridley suggested that States now
have many agencies focused on safety and quality, and they need to be
engaged and included. She would like to see AHRQ identify partners in
each State who can play that role. Ms. Nix agreed that States are
important and reported that AHRQ has programmatic relationships with
States right now and wondered if these could perhaps be expanded to
include the Innovations Clearinghouse. Ms. Nix noted that by the end of
the month, AHRQ will release some State data related to the National
Health Care Quality Report, and the Agency hopes this will continue to
motivate States toward change and improvement.

Dr. Mold advised that when talking about large systems, professional
societies should not be forgotten. They are involved in maintenance of
certification, which will require members to work on QI. Tapping into
that could be valuable, as would CME credits. He added that little ideas
could be captured from what others, such as societies, have done to
incentivize input. Ms. Hansen-Turton observed that considerable
innovation has come from nursing, and associations have a vested
interest in positioning their own members. E-mail marketing is
inexpensive, and it works.

Ms. Bella asked about the relationship between AHRQ and CMS, and Ms.
Slutsky answered that the two agencies have a close partnership. As CMS
implements the Medicare bill, however, it is on a tight timeline---the
Innovations Clearinghouse will be useful to them, but the timeline may
anticipate it. There still is much room for AHRQ to work with CMS. This
pulls in a big audience, Ms. Bella observed.

Dr. Bartlett asked for suggestions on how to make an Innovations
Clearinghouse Web site work for all of the different types of audiences
that have been articulated. Dr. Mold suggested physicians want to see
something they can use in their practice, and therefore users should be
channeled. Ms. Slutsky noted that people often do not like channeling,
because they think they are only getting a subset of information. The
physician audience may perceive that it gets the best information, but
consumers may feel they are getting limited information. She added that
channeling seems to work well in Britain, but people in the United
States do not like being told what to do and will want to go where they
are told not to. Dr. Hines recommended giving people options, and where
they want to go would be a function of what questions they are trying to
answer. For example, questions about innovations related to
incentivizing quality will attract policymakers and purchasers, but not
clinicians. The right structure will allow people to go where they want
to go, without putting them in a situation where they resent not being
able to see all they want to see. Allow channeling by offering sorting.

Ms. Ridley commented that Massachusetts' patient safety matrix includes
all of the concepts mentioned, as well as a grid on how to navigate. Dr.
Rosati remarked that channeling is not as relevant as what is in the
database. If disciplines, cost factors, and other factors are laid out,
people will look for what they need. The integrity of the database is
what matters.

Ms. Slutsky asked participants if their organizations' innovations would
be difficult to submit because of medical/legal or liability issues. Ms.
Greenberg indicated that evaluation might be the most difficult aspect
because a program might not have been sufficiently evaluated to be
disseminated. Ms. Bella suggested that liability would not be a
deterrent if it was clear that a broad range of input at various stages
of development was welcome. Dr. Rosati added that copyright is also
important relative to whether information can be shared. Ms. Slutsky
acknowledged that has been dealt with in other clearinghouses with
copyright release and allowing submitters to have their own copyright
statement. Users must get permission to distribute, for example.
Grantees have the right to hold a copyright, but usually it must revert
back to the funder. A related issue concerns tools that generate income
for organizations.

Ms. O'Brien noted that in cases of patient harm, families want to know
that some good has come out of bad, and that can be an incentive for
sharing stories.

Ms. Nix asked if providing personal information via the Web would be a
barrier, as well as the need to contact a submitter to authenticate and
get additional information. (That information would not be posted on the
Web site, just used for authentication.) Ms. Slutsky brought up the
related issue of how to validate that a submitter is the real submitter.
There will be some governance on what is posted.

Ms. Ridley asked about security of the site, and who would have access.
Ms. Nix explained that it will be a secured server from a hacking
standpoint, but open to the public.

Dr. Hines asked about the relationship of an organization to an
innovation from an individual from that organization. Ms. Slutsky
indicated that the copyright aspect might require organizational
sign-off, and Dr. Hines added that if a patient safety or
waste/inefficiency issue is involved, any hesitation from the
organization's legal department would squelch posting the innovation.

Ms. Bella asked about turnaround time---how quickly could an innovation
get out? Dr. Rosati indicated 2 to 5 years. Dr. Mittman added that there
might be ways to speed it up, for example, publishing the details of an
innovation on the Web site, followed later by a journal article. Ms.
Slutsky noted that she worked with journals to establish the Guideline
Clearinghouse, and the problem was to get them to agree that posting on
a Web site is not prior publication. *JAMA* was not willing to do that,
but many others were, and an expedited review was possible from working
together.

Ms. Greenberg returned to the mission of the site, asking for
clarification that the goal is innovations that have been implemented in
the real world and could be again, not research breakthroughs. Examples
might be implantation of a new heart valve or a new way of coordinating
home visits, which journals are not interested in. Ms. Slutsky was not
sure that journal editors would agree.

**Discussion---Part 4: Outstanding Issues and Additional Comments**

Ms. Slutsky shared that the working name for the project is the
Clearinghouse for Health Innovation Change, with the acronym CHIC, but
she is open to other suggestions. Dr. Bartlett noted that a wealth of
good ideas have come from the meeting, and there is a better sense of
what the initiative might look like than there was when the meeting
began.

Dr. Rosati asked how the site would be maintained once it is set up,
with an enormous amount of work required to keep things from becoming
outdated. Ms. Slutsky explained that planners envision the same process
as the Guideline Clearinghouse, sweeping the database on regular
intervals and archiving old material. It might be more difficult for
innovations, with a number of questions remaining. How long does
something remain an innovation? Does evidence mean evidence of need for
something, or evidence that something works? Objective criteria are
needed to set parameters.

Dr. Mittman cautioned that systematic review and guidelines can become
outdated, but individual studies do not become outdated, they become
diluted. Individual innovations will not become outdated, but users must
consider their age and description. Dr. Bartlett suggested that some
sort of ranking by timeliness could be implemented. Ms. Slutsky added
that there might be communication with submitters every year or two
about enhancements or changes in innovation. Sustainability would be
another avenue to pursue, according to Dr. Mittman.

Ms. Ridley asked about the scope of the clearinghouse and if it is
limited to quality and safety information. Ms. Slutsky emphasized that
it is not limited to quality and safety, adding that bioterrorism
generally builds off existing public health programs. Ms. Williams noted
that for the scope of the audience, there have been comments about the
potential danger of being too broad. Can the same material be delivered
to a public policy audience and to a clinical audience? Ms. Greenberg
felt this challenge remains, and there are still different visions in
the room of what the Web site will be.

Dr. Hines asked if there is an estimate of how many innovations will
meet the criteria being discussed. If there are thousands, this may
become a resource issue. But his experience is that the actual number of
innovations is fairly sparse. Ms. Hansen-Turton spoke of the need to
bring networks that are out of the funding mainstream into the fold.

Dr. Amofah asked how often new entries would be announced, and Ms. Nix
responded that other clearinghouses announce what is new every week. Dr.
Amofah emphasized that most users look for a series of steps in a
process to make innovations more functional, with clusters of like
ideas. Ms. Nix felt that approach is being considered. Dr. Amofah added
that the site will be valuable to the extent to which people are
informed about what exists and get training to use the tools that are
offered.

Dr. Mold suggested that in identifying innovations, take a process,
break it into smaller steps, find people who have figured out pieces of
it, and put the pieces together. While some might like to believe that
innovations are large and encompassing, important innovations are very
small.

Ms. Slutsky suggested that the meeting conclude by asking each
participant what his or her take-home message is. The following comments
were made:

-   Ms. O'Brien: Focus on those we serve (patient and family).

-   Dr. Hines: Remember that users need more than information. They need
    tools on how to apply information, and they need information on how
    to motivate others. People need to be made aware, and they need to
    know how to apply that awareness.

-   Ms. Bella: The question is one of depth versus breadth, how to say
    no to some areas, and whether some users will have to sacrifice for
    it to be valuable.

-   Ms. Hansen-Turton: It is important that policymakers are aware of
    what is on the site and what the benefits can be (policy
    implications in particular; AHRQ owns that and should be responsible
    for that aspect). The ability to do State-by-State searches and
    analyses is very important.

-   Ms. Rogers: Make sure that people have thought through what their
    problems are before they come to find a solution.

-   Ms. Ridley: Incorporate State health departments as partners in the
    solutions, breaking down silos. Users should represent agencies and
    trade and professional organizations, and the process should
    incentivize all. Linkage should go beyond safety, quality, and
    preparedness to challenges of other parts of the health care system.
    A wealth of information exists, but teaching people how to apply
    existing tools and standards is a big job.

-   Dr. Mittman: This is a messy business without neat answers.
    Experience is required for organizational change, with all available
    resources applied. Disclaimers will be important.

-   Dr. Amofah: To get people to adopt an innovation, ensure they are
    informed and understand why something is important. Sell the
    importance of using this tool. The project is itself an innovation.

-   Dr. Rosati: Ease of use is critical. Anyone should be able to come
    in from various levels and find information. Side-by-side
    comparisons should be offered for similar innovations. Frequent
    updates are important; the key is to stay current.

-   Dr. Mold: Do not forget to include a research agenda.

-   Ms. Greenberg: Make it easy for users to find someone like them who
    has done something. Make innovations easy to implement, with steps
    laid out in an obvious way.

-   Ms. Line: This is a huge undertaking. Gear it to the user, and
    engage the user interactively, providing more than passive
    information. Leave the door open to include things that did not
    work---every innovation has something that did not work in front of
    it.

-   Dr. Seamonds: Make sure that the neophyte is comfortable with the
    project and can expand his or her knowledge. Also, provide a vehicle
    for simple grassroots improvements from simple observations that
    prompted a change in operation in an organization. Include an arena
    for anecdotal communication.

    **Conclusion**

Ms. Slutsky acknowledged that she had not been sure about what to expect
from this meeting and was surprised at what it has yielded. AHRQ asked
for comments on an idea and concept without filling in the blanks and
came away with strong threads that all seem to have consensus. She is
excited about being in on the ground floor of an idea, and Ms. Nix has a
huge task ahead of her.

Ms. Slutsky concluded that she hopes that participants will be available
for individual consultation to flesh out ideas and that they will become
ambassadors for what AHRQ is trying to do. She asked that if anyone sees
this project moving in the wrong direction, please speak up. AHRQ will
move forward with the project. It has been funded, and today's advice
will make it better. A request for proposals will be posted this spring.

Ms. Williams added that participants should not wait for AHRQ to call if
they think of a brilliant idea they did not mention today.

The meeting was adjourned.

INNOVATIONS CLEARINGHOUSE EXPERT USER MEETING SUMMARY

APPENDIX A

AGENDA

![](media/image3.png){width="1.1270833333333334in"
height="0.4583333333333333in"}*U.S. Department of Health and Human
Services*

### Agency for Healthcare Research and Quality 

*Presents*

# [Innovations Clearinghouse]{.smallcaps} 

# [Expert Panel Meeting]{.smallcaps}

##### Agenda

### 8:30 -- 8:45 a.m. Welcome, Introductions, and Purpose of the Meeting 

**8:45 -- 9:15 a.m.** **Presentation on Innovations Clearinghouse
Concept**

###  

**9:15 -- 10:15 a.m.** **Observations on the Proposed Innovations
Clearinghouse Concept**

[Discussion questions include]{.underline}:

-   How do you currently scan for innovations when planning quality
    improvement activities?

-   How could an Innovations Clearinghouse help meet your information
    needs?

-   Who in your organization might use an Innovations Clearinghouse and
    how could it help improve quality?

**10:15 -- 10:30 a.m. Break**

**10:30 -- noon What Should Be In the Database and On the Web Site?**

[Discussion questions include]{.underline}:

-   What kind of information and what level of detail is needed to
    replicate innovations?

-   How strong should evidence be that an innovation works before it's
    added to the database? What would you want to know about the nature
    of the evidence?

-   Would you want information about clusters of innovations that could
    be implemented simultaneously?

-   What kind of tools would you want to be able to access through the
    Web site and what kind of evidence would you want of a tool's
    effectiveness?

-   What kind of searching and browsing would you want to be able to do?

-   What types of online learning or networking opportunities would make
    this site valuable?

**Noon -- 12:30 p.m. Lunch**

**12:30 -- 2:15 p.m. What Would Encourage or Discourage You From Using
the Web Site?**

[Discussion questions include]{.underline}:

-   What encouragements/incentives could be provided for submissions?

-   Would proprietary concerns prevent posting of innovations or tools?

-   Would you be receptive to making online submissions if that included
    giving personal identifying information? \[Note: the information
    would not be displayed on the Web site but would be entered into the
    database. Also, the information would be used for the purposes of
    contacting the submitter to authenticate the submission and seek
    clarification where needed.\]

-   Would you submit to the Clearinghouse if you knew that you might be
    contacted by potential implementers who would like guidance,
    clarification, or more details?

-   Are there particular features that would make you more likely to use
    the Web site?

-   What are the best ways of letting you know about the Web site and
    about new entries of interest on the Web site?

**2:15 -- 2:30 p.m. Outstanding Issues and Additional Comments**

**2:30 Adjourn**

INNOVATIONS CLEARINGHOUSE EXPERT USER MEETING SUMMARY

APPENDIX B

PARTICIPANT LIST

![](media/image3.png){width="1.1270833333333334in"
height="0.4583333333333333in"}*U.S. Department of Health and Human
Services*

### Agency for Healthcare Research and Quality 

*Presents*

# [Innovations Clearinghouse]{.smallcaps} 

# [Expert Panel Meeting]{.smallcaps} 

#### Participant List

**St. Anthony Amofah, M.D., M.B.A.**

Medical Director

Health Choice Network

11535 SW 100 Terrace

Miami, FL 33176

Phone: (305) 519-8188

Fax: (305) 271-1057

Email: aamofah@hcnetwork.org

**Larry Bartlett, Ph.D.**

Director

Health Systems Research, Inc.

1200 18th Street, N.W., Suite 700

Washington, DC 20036

Phone: (202) 828-1500

Fax: (202) 728-9469

Email: lbarlett@hsrnet.com

**Melanie Bella, M.B.A.**

### Vice President for Policy

The Center for Health Care Strategies, Inc.

P.O. Box 3469

Princeton, NJ 08543-3469

Phone: (609) 895-8101

Fax: (609) 895-9648

Email: mbella@chcs.org

**Cindy Brach, M.P.P.**

Senior Health Policy Researcher

Center for Delivery, Organization,

and Markets

Agency for Healthcare Research and Quality

540 Gaither Road, Room 5129

Rockville, MD 20850

Phone: (301) 427-1444

Fax: (301) 427-1430

Email: cbrach@ahrq.gov

**Donald Casey, Jr., M.D., M.P.H., M.B.A.**

Senior Vice President and

Chief Medical Officer

Quality Management Team

Catholic Healthcare Partners

615 Elsinore Place

Cincinnati, OH 45202

Phone: (513) 639-2769

Fax: (513) 639-2762

Email: decasey@health-partners.org

**Beth A. Collins Sharp, Ph.D., R.N.**

Health Scientist Administrator

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

540 Gaither Road, Room 6317

Rockville, MD 20850

Phone: (301) 427-1503

Fax: (301) 427-1520

Email: bcsharp@ahrq.gov

**Margaret Coopey, M.P.S., M.G.A., R.N.**

Senior Health Policy Analyst

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

### 540 Gaither Road, Room 6353

Rockville, MD 20850

Phone: (301) 427-1618

Fax: (301) 427-1520

Email: mcoopey@ahrq.gov

**Liza Greenberg, R.N., M.P.H.**

Health Project and Management Consulting

Phone: (301) 928-5909

E-mail:

Lgreenberg@healthprojectconsulting.com

**Tine Hansen-Turton, M.G.A.**

Executive Director

National Nursing Centers Consortium

260 South Broad Street, 18th Floor

Philadelphia, PA 19102

Phone: (215) 731-7140

E-Mail : Tine@nncc.us

**Steve Hines, Ph.D.**

Senior Scientist

Delmarva Foundation

9240 Centerville Road

Easton, MD 21601

Phone: 410-310-7978

Fax: 410-822-7971

Email: hiness@dfmc.org

**Jon F. Kerner, Ph.D.**

Deputy Director for Research Dissemination and Diffusion

Division of Cancer Control and Population Sciences

National Cancer Institute

6130 Executive Boulevard, EPN 6144

Bethesda, MD 20892

Phone: (301) 594-7294

Fax: (301) 594-6787

Email: kernerj@mail.nih.gov

**Ann M. Lewis, M.P.H.**

Executive Director

CareSouth Carolina, Inc.

P.O. Box 1090

201 South Fifth Street

Hartsville, SC 29551

Phone: (843) 857-0111

Fax: (843) 857-0150

Email: annlewis@caresouth-carolina.com

**Laura Line, M.S.**

### Deputy Executive Director

Philadelphia Health

Management Corporation

260 South Broad Street, 18th Floor

Philadelphia, PA 19102

Phone: (267) 765-2363

Fax: (215) 765-2400

Email: lline@phmc.us

**Brian S. Mittman, Ph.D**

Senior Social Scientist

VA Center for the Study of

Healthcare Provider Behavior

VA Greater Los Angeles Healthcare System

16111 Plummer Street, Mailcode 152

Sepulveda, CA 91343

Phone: (818) 895-9544

Fax: (818) 895-5838

Email: Brian.Mittman@med.va.gov

**James W. Mold, M.D., M.P.H.**

Professor and Director

Research Division of the Department of

Family Medicine and Preventive Medicine

University of Oklahoma Health Sciences Ctr

900 Northeast 10th Street, Room 2207

Oklahoma City, OK 73104

Phone: (405) 271-2370

Fax: (405) 271-4125

Email: james-mold@ouhsc.edu

**Karen Nelson, R.N.**

Assistant Vice President

Quality Management

North Shore - LIJ Health System

145 Community Drive

Great Neck, NY 11021

Phone: (516) 465-8054

Fax: (516) 465-8376

Email: knelson@nshs.edu

**Mary P. Nix, M.S., M.T.(A.S.C.P), S.B.B.**

Project Officer

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

540 Gaither Road, Room 6361

Rockville, MD 20850

Phone: (301) 427-1624

Fax: (301) 427-1520

Email: mnix@ahrq.gov

**Susan O'Brien, M.S., R.N.**

St. Mary's Hospital Medical Center

707 S Mills Street

Madison, WI 53715

Phone: (608) 446-6874

Fax: (608) 259-5327

E-mail: sue_o'brien@ssmhc.com

**Nancy Ridley, M.S.**

Assistant Commissioner

Center for Patient Safety

Massachusetts Department of Public Health

250 Washington Street, 2nd Floor

Boston, MA 02108-4619

Phone: (617) 624-5280

Fax: (617) 624-5587

Email: nancy.ridley@state.ma.us

**Jill Rogers, M.S.N.**

Section Chief

Health Promotion and Disease Prevention

Division of Public Health

Delaware Health and Social Services

655 South Bay Road

Blue Hen Corp Center, Suite 201 B

Dover, DE 19901

Phone: (302) 741-2900

Fax: (302) 741-2910

Email: jill.rogers@state.de.us

**Robert J. Rosati, Ph.D.**

Director of Outcomes Analysis

and Research

Center for Home Care Policy and Research

Visiting Nurse Service of New York

5 Penn Plaza, 11th Floor

New York, NY 10001

Phone: (212) 609-5776

Fax: (212) 209-3756

Email: Robert.Rosati@vnsny.org

**Bette Seamonds, PhD, DABCC**

Director of Chemistry and

Point-of-Care Services

Pathology Department

Mercy Health Laboratory

1500 Landsdowne Avenue

Darby, PA 19023

Phone: (610) 237-7370

Fax: (610) 237-4017

Email: bseamonds@mercyhealth.org

**Archna Sharma, M.D., M.P.H.**

Quality Management Office, VISN 5

Veterans Administration

8207 Falstaff Road

McLean, VA 22102

Phone: (410) 691-1142

Fax: (410) 684-3169

E-mail: Archna.sharma@med.va.gov

**Jean Slutsky, P.A., M.S.P.H.**

Director

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

540 Gaither Road, Room 6218

Rockville, MD 20850

Phone: (301) 427-1601

Fax: (301) 427-1520

Email: jslutsky@ahrq.gov

**Christine G. Williams, M.Ed.**

Director

Office of Communications

and Knowledge Transfer

Agency for Healthcare Research and Quality

540 Gaither Road, Room 2323

Rockville, MD 20850

Phone: (301) 427-1850

Fax: (301) 427-1873

Email: cwilliam@ahrq.gov

**SOW Appendix C**

> **AHRQ Web Product and Web Site Development Guidelines**
>
>  
>
> The following list highlights basic issues that need to be addressed
> when developing Web tools or sites under contract that will be
> **publicly available** when launched to ensure deliverables are on
> target, in compliance with legal and policy requirements, and do not
> require expensive rework to meet Federal and Department of Health and
> Human Services requirements for information resources.
>
> **[Section 1:  Guidelines for Web-Based Products]{.underline}**
>
>  
>
> Retrofitting Web-based products after the fact is highly undesirable
> because it adds time and costs to the process of making these products
> publicly available. All products that are developed with the intent of
> being posted on the AHRQ Web site should meet the following minimum
> requirements:
>
>  
>
> **Titles of Products**
>
> Coordinate with your project officer on the titles of your products.
> They need to be concise and relevant to the purpose of the project,
> but cannot include the name of the contractor or grantee as the
> performing organization as part of the title. Report titles should be
> no more than 10-words maximum and Web-based tools should be no more
> than 5-words maximum (make every word count---eliminate initial
> articles such as "The" or "A" ). Titles need to be distinct enough to
> differentiate among similar sounding products.
>
> **Quality Control/Editorial Review**
>
> This involves checking for spelling and grammar mistakes, formatting
> issues, general consistency, and style. This should be done by the
> AHRQ grantee or contractor prior to submission of the final product
> for posting on the AHRQ Web site. Federal resources follow the GPO
> Style Manual which is available electronically at:
>
> <http://www.gpoaccess.gov/stylemanual/browse.html>
>
>  
>
> **Accessibility**
>
> As an agency of the Federal Government, AHRQ must ensure that anything
> that is posted on our Web site is in compliance with requirements for
> information resources under Section 508 of the Rehabilitation Act.
> Also, federally funded resources need to be generally available to
> users in multiple formats to ensure that we are not forcing a
> particular platform, operational system, or software package on users.
>
>  
>
> **Intellectual Property Rights**
>
> Before we can post a product on the AHRQ Web site, we must have a
> written explanation of the following four questions:

-   Who retains the copyright?

-   Who has licenses for what purposes and uses?

-   What are the constraints imposed?

-   Who grants permission for further use or adoption?

> **Technical Assistance**
>
> We cannot release a tool without providing the following:

-   Written instructions on the use of that tool and what to do if a
    user encounters problems in accessing and using it.

-   A contact name, telephone number, and e-mail address for technical
    assistance.

-   A feedback mechanism for errors, future updates, and revisions.

> This information must be provided in writing along with the tool or
> product to be posted.  Provision of technical assistance support
> should be included in the life-cycle costs of the product.
>
>  
>
> **Source Code**
>
> AHRQ's intent is to make tools available to the public, clinicians,
> health planners and providers, and other Federal, State and local
> government agencies as appropriate for their intended purposes. Any
> software and products resulting from these projects should be easily
> transportable to other users and developers. The best way to ensure
> adoption and implementation for these audiences is to have a Web-based
> final product that is platform independent. Coordinate with AHRQ on
> infrastructure requirements for housing any robust back-end
> applications before they are developed.
>
> Source code for any technical application must be delivered to the
> Agency with the product. This provides AHRQ with the knowledge of how
> the application was created by the original developers and enables the
> Agency to make corrections, updates, or conversions as necessary to
> keep pace with technological changes once the products are released.
>
> **Usability and Version Control**
>
> Web resources should include usability testing, evaluation, and
> modification as an integral and recurring part of the development
> effort to ensure they are effective for the electronic business
> processes they are designed to facilitate. A set of Research-Based Web
> Design and Usability Guidelines that should be consulted are available
> at: <http://www.usability.gov/guidelines/index.html>
>
> Beta testing prior to release is desirable, evaluating the product
> against usability heuristics. As feedback is received and products are
> updated, the revisions will need to be designated by version number
> and date of release.
>
>  
>
> **Privacy Act Protections**
>
> Web resources are subject to the Privacy Act and this can impact both
> the development of Web-based tools and the users of those tools.
> Persistent cookies should not be programmed into the functionality of
> a Web-based tool, although session cookies are allowed. Registration
> for use cannot be requested if this would involve collection of
> individual identifiers from the users. Although exemptions to both
> rules can be sought, this involves a strong justification and several
> levels of review for approval through the U.S. Department of Health
> and Human Services (HHS).
>
>  
>
> **[Section 2:  Guidelines for Web Sites]{.underline}**
>
>  
>
> Web sites being supported through contracts are considered Federal
> information resources and as such are required to be in compliance
> with laws, policies, and directives that affect such resources. This
> includes content requirements under the E-Gov Act. For recommendations
> and guidance on how to implement Office of Management and Budget
> policies, go to: <http://www.firstgov.gov/webcontent/shouldmenu.shtml>
>
> **Clearance**
>
> Web resources require clearance by HHS\--including justification
> against a set of criteria. Publications cleared for printing are
> cleared for Web uploading at the same time. Web resources must comply
> with the numerous laws and directives that affect federally funded
> electronic information resources. Web content loaded on a site by
> contractors must be appropriate and follow all laws and directives.
> AHRQ Offices and Centers must coordinate initial review
> through AHRQ\'s Office of Communications and Knowledge Transfer (OCKT)
> before launch, and OCKT will coordinate departmental clearance.
> ***OCKT responsibility: To initiate clearance for Web site content and
> Web-based resources through HHS. Initiating Office or Center
> responsibility: To ensure that subsequent Web site postings for which
> they have let contracts are constantly reviewed for appropriateness.
> If there are any questions about whether such material is appropriate,
> contact the OCKT Division of Public Affairs for approval.***
>
> **Domain Names**
>
> All domain names for any Web resource funded in whole or in part by
> Federal funds must be registered as .gov through HHS with the General
> Services Administration (GSA). Although other domains, such as .org,
> .net, .edu, .com may also be reserved by the Agency, the .gov domain
> must be registered and that domain name will need to be indexed by
> FirstGov, the GSA portal to government-funded resources. The FirstGov
> link is then required on the home page of the site. Coordinate with
> OCKT on domain name requests. ***AHRQ Webmaster (OCKT) and Division of
> Public Affairs responsibility: To obtain approval for domain names.***
>
> **Editorial Review**
>
> All content for upload needs to be reviewed to ensure consistency and
> compliance with best practices and established style and conventions.
> As a minimum, the copy needs to be production edited to ensure there
> are no typos and the GPO Style Manual is followed for punctuation,
> spelling, use of numerals, abbreviations, etc. Do not use unexplained
> acronyms; they need to be spelled out on first reference in any
> document or file. There should not be anything marked DRAFT on a
> public site. Once the materials are uploaded, they are published and
> considered in the \"public domain.\" Do not use placeholders for
> content that does not exist. Government funded sites should not have
> anything designated \"under construction.\" A process needs to be
> established for regular review of content and updating. Additional
> materials need to undergo editorial review and be approved before
> uploading. The GPO Style Guide is available electronically as a
> reference at: <http://www.gpoaccess.gov/stylemanual/browse.html>
>
> ***Contractor responsibility: To comply with GPO Style Manual and AHRQ
> Web site conventions.***
>
> **Accessibility**
>
> Under the Rehabilitation Act, Federal agencies have an obligation to
> provide equal access for the disabled to their information and
> services. Requirements are specified in section 504 for individual
> accommodation and more recently in section 508 for electronic and
> information technology, which includes Web sites and multimedia
> products. Equivalent alternatives are required for auditory and visual
> information, such as providing alternative descriptive text for images
> for the blind and providing captions for audio-video files for the
> deaf. Written transcripts are required for all streaming audio. PDF
> files can be offered in conjunction with accessible files, such at
> HTML versions, but avoid uploading PDF-only versions of documents.
> OCKT has software used to evaluate Web sites and can provide a report
> on any accessibility violations that would need to be addressed before
> launch. Specific requirements are available at:
> [http://www.section508.gov](http://www.section508.gov/)
>
> ***Contractor responsibility: To develop Web resources that conform to
> Section 508 requirements and respond to any violations determined
> during AHRQ assessment of the site. AHRQ project officer
> responsibility: To ensure compliance with Section 508 and the HHS
> Policy on Section 508 Implementation.*** 
>
> **Privacy**
>
> A privacy policy notice must be prominently displayed, and the Web
> site host has to follow it. Sites are periodically audited to ensure
> that they observe their stated privacy policy. A Privacy Act System
> notice may need to be prepared and published for users to register on
> a site if the registrations represent a group of records, under the
> control of the Agency (or a contractor), that can be retrieved by
> personal identifier. This notice must go through several levels of
> review\--including the Office of General Counsel\--and be published in
> the *Federal Register*. Persistent cookies cannot be used on Federal
> sites unless the Secretary of HHS grants an exemption, and this
> involves a strong justification and review process. ***Contractor
> responsibility: To work in coordination with AHRQ staff for submission
> of the Privacy Act System notice and to adopt or modify the general
> privacy policy of the AHRQ main Web site.***\
> **Web Site Mailbox**
>
> Every Web site must provide full contact information for the sponsor
> and have a Contact Us link for submission of comments or questions as
> a customer feedback mechanism. Web site e-mail is subject to the same
> privacy and records management issues that affect the overall Web site
> as well as departmental standards for handling inquiries and customer
> feedback. ***Contractor responsibility: To maintain the Web site
> mailbox according to HHS requirements for response times and
> confidentiality, to maintain an electronic archives of responses on an
> annual basis of retention and destruction, and to submit the number of
> inquiries handled on an annual fiscal year basis to the AHRQ Webmaster
> to include as Web metrics for Agency reporting under the Government
> Performance Reporting Act .***
>
> **Records Management**
>
> All content on the site and e-mail generated by the site must be
> archived electronically and handled according to records retention
> schedules and disposition authorities as established with the National
> Archives and Record Administration. This requirement also affects Web
> site log files and statistical reporting on Web site usage.
> ***Contractor responsibility: To comply with the records management
> requirements of the AHRQ main Web site and to submit Web site usage
> statistics on an annual fiscal year basis to the AHRQ Webmaster to
> include as Web metrics for Agency reporting under the Government
> Performance Reporting Act .***
>
> **Information Collection Budget**
>
> If a Web site is used to collect information from users, such as for
> surveys or evaluations, then the Office of Management and Budget must
> first approve the burden hours for such an effort for this collection.
> A notice must be posted on the Web site at the point of collection
> with the OMB approval number and a statement on the process of
> collection. ***AHRQ project officer responsibility: To submit requests
> for OMB approval.***
>
>  
>
> **Intellectual Property**
>
> Copyright and trademark protections need to be observed on Web sites.
> Permissions for use must be granted for any copyrighted information
> included and registered trademarks need to be reflected in copy. Any
> copyright or trademark constraints related to materials uploaded to a
> site must be specified for users. Public domain does not extend
> outside the borders of the United States. Therefore, foreign countries
> must request specific permission for use. Given the global nature of
> the Internet, citation as to source is a critical issue. ***Contractor
> responsibility: To coordinate with AHRQ on permission requests and
> follow trademark guidelines provided.***
>
> **Linking**
>
> External links constitute an implied endorsement and create a business
> advantage for the linked sites. OMB requires Agencies to do a risk
> assessment of external links, and potential links need to be assessed
> against the HHS and AHRQ linking policies and criteria. If a site
> deviates from these policies, then the specific review and selection
> criteria must be justified and posted on the Web site for full
> disclosure. Outside Web resources may link to Agency resources
> providing the link is not displayed in any way that would imply an
> endorsement by the Agency of a specific commercial product or service.
> ***Contractor responsibility: To assess links according to AHRQ
> linking policy requirements and evaluation checklist provided.***
>
> **Electronic FOIA**
>
> The Agency is required by law to have an electronic FOIA reading room
> and to provide materials that can be requested under the Freedom of
> Information Act in electronic form, if so requested. HHS will normally
> require that any Web resource funded by the Agency provide a link to
> the AHRQ electronic reading room, which is housed on the main AHRQ Web
> site. ***Contractor responsibility: To include link to AHRQ electronic
> reading room.***
>
> **Security**
>
> Web sites need to be monitored and protected against intrusion and
> corruption or compromise of content. This is especially critical if
> there are any business processes involved or financial transactions
> conducted on the Web site with users. Web resources are periodically
> audited and evaluated for security by the GSA. Security measures must
> be specifically delineated for any federally funded Web resources in
> existence or in development. Any attacks on Web resources must be
> documented and reported to the HHS Inspector General. ***Contractor
> responsibility: To establish and maintain security according to AHRQ
> and HHS policies and procedures.***
>
> **Usability**
>
> Web resources should include usability testing, evaluation, and
> modification as an integral and recurring part of the development
> effort to ensure they are effective for the electronic business
> processes they are supposed to facilitate. Go to
> [http://www.usability.gov](http://www.usability.gov/) as a reference
> for best practices in initial development or redesign of Web
> resources. ***Contractor responsibility: To address usability issues
> and to work in coordination with AHRQ staff on usability testing.***
>
> **Web Sponsor Identity**
>
> AHRQ has uniform principles to identify AHRQ as the primary sponsor of
> AHRQ-related Web sites. These principles reflect HHS best practices
> for a consistent look and feel of Web resources, reinforce
> credibility, and support HHS and Agency branding efforts. The four
> specific principles that should be consistent across all AHRQ-funded
> Web sites are:
>
>  
>
>      ***Web site URL name**:* The name of a Web site should always
> contain AHRQ in the URL. A Web resource should either be a folder on
> the main AHRQ Web site
> ([www.ahrq.gov/chiri](http://www.ahrq.gov/chiri)) or a third-level
> domain of the Web site
> ([www.webmm.ahrq.gov](http://www.webmm.ahrq.gov/)).
>
>      ***Title of Web site project:*** AHRQ\'s name should be part of
> the formal title and appear at the beginning of the Web site\'s
> project name. For example: AHRQ\'s Web Morbidity and Mortality online
> journal.
>
>      ***HHS and AHRQ logos**:* The HHS and AHRQ logos should be
> featured prominently on the Web site and in materials that are used to
> market that Web site.
>
>      ***Web site home page format**:* The Web site home page should
> have common design and navigation elements with the HHS Portal and the
> AHRQ Web site so that all Web sites look as though they belong to the
> Department and AHRQ Web family. All AHRQ domain sites must include a
> standard banner and footer that are branded for Web resources.
> Technical specifications and templates for developers to consult when
> designing Web resources are provided by the AHRQ Web Manager.
>
> ***Contractor responsibility: To develop Web resources which are
> consistent with identity principles and design specifications in
> coordination with AHRQ staff.***
>
> **Additional Information**
>
> To discuss specific issues or to get additional guidance on Web
> requirements, contact:
>
> Gerri Michael Dyer
>
> Electronic Dissemination Advisor
>
> E-mail: <gdyer@ahrq.gov>
>
> Phone: 301-427-1898
>
> Biff LeVee
>
> AHRQ Web Site Manager
>
> E-mail: <blevee@ahrq.gov>
>
> Phone: 301-427-1897
>
> Randie Siegel
>
> Director of Print and Electronic Publishing
>
> E-mail: <rsiegel@ahrq.gov>
>
> Phone: 301-427-1852
>
> ***Last Update: December 2005***

**SOW Appendix D**

**[Guidelines for Developing AHRQ Web-based Tools]{.underline}**

The Guidelines for Developing AHRQ Tools is based on a conceptual
methodology that describes the stages involved in an information system
development project, referred to as the Software Development Life Cycle
(SDLC).

**Planning**

Planning focuses on defining the project's objectives, scope, target
audiences, deliverables, resources, and schedule. Project planning is
the responsibility of the Project Manager or Principal Investigator and
takes place initially and throughout the life cycle of the project.

**Requirements**

This phase involves establishing an understanding and agreement with
AHRQ and documenting [what]{.underline} the system is supposed to do and
the specifications for the tool. These requirements are expected to be
managed throughout the life cycle of the project.

**Design**

This phase describes [how]{.underline} the system is supposed to work,
including system architecture, development platforms, underlying
databases, and user interfaces.

**Development**

This phase involves the actual development of the prototype, including
source code, to ensure compliance with design specifications and
usability.

**Testing**

This phase involves putting the components together and testing the
product to demonstrate to AHRQ that it meets the requirements, is error
free, and achieves the original objectives of the project. The general
types of testing performed are integration, system, performance, and
user acceptance testing. Deliverables should be certified as to the
quality assurance process undertaken.

The diagram below depicts the phases involved and the allocated
documentation AHRQ expects from grantees or contractors throughout the
life cycle.

![](media/image4.wmf){width="4.9847222222222225in"
height="3.6465277777777776in"}

>  

**Documentation**

This section contains the descriptions of the documentation required for
all AHRQ sponsored software Development efforts:

**Vision/Project Initiation Document**

The Program Initiation Document (PID) is intended to be a statement of
purpose and scope for initiating a given project and a guide to manage
expectations in both process and deliverables throughout the System
Development Lifecycle (SDLC). The PID defines the business case for the
project by defining the purpose, the milestones, resources, objectives,
costs, risks including mitigation strategies, and the artifacts and IT
technologies (architecture) utilized and produced for, and during, the
project. The PID serves as the formal funding commitment document
approved by the COTR and Stakeholders. Additionally, the PID must be
approved by AHRQ IT management, and in some cases, the AHRQ Information
Technology Review Board (ITRB) for technical viability, adherence to
Agency Enterprise Architecture (EA); technical standards and formal
Project Management requirements as derived from Departmental standards
and accepted Project Management Institute (PMI) Project Management Body
of Knowledge (PMBOK) standards.

**Project Work Plan**

The Project Work Plan (PWP) provides a method to assign and track the
project resources, hours and specific deliverables. This plan is
formatted in Microsoft Project provides the detailed Work Breakdown
Structure (WBS) and resource loading that can be used to identify
project costs and is intended for the project manager to track the
schedule and cost of a project, including development of Earned Value
Management (EVM) measures. The PWP is delineated by the phases of the
project which include Project Initiation, Generation of the PWP
Schedule, Requirements Gathering, System Design, System Development,
System Testing including User Acceptance, System Deployment and System
Support and production of project deliverables which require COTR or
Stakeholder acceptance and signoff to continue project tasks identified
in the PWP.

**Software Requirements Document**

This document contains the system requirements, use cases and
supplementary specifications that provide the basis for design and
development of the system. The following information is provided for
each requirement identified in the document:

-   Requirement ID, Name and Title

-   Requirement Description Requirement

-   Software Release Version

-   Use Case Model

-   Use Case Specifications

-   Supplementary Specifications

The conventions used in identification of requirements and Use Case and
Supplementary Specifications conform to the naming conventions and
structures defined in the AHRQ Configuration Management Plan and
associated Standard Operating Procedures (SOP's) and Working
Instructions (WI's).

## Requirements Traceability Matrix

The traceability of a project is captured in the Requirements
Traceability Matrix (RTM). This matrix is created at the beginning of a
project's lifecycle to trace the requirements from identification
through testing. The project elements are traced as they relate to other
project elements, especially those related to requirements. 

The purpose of establishing traceability is to help understand the
sources of requirements, manage the scope of the project, manage changes
to requirements; assess the project impact of a change in a requirement;
and verify that all requirements of the system are fulfilled by the
implementation. The RTM should be reviewed once more and baselined at
the end of the lifecycle before it is released to the client.

Traceability may be set up to help answer the following sample set of
queries:

-   Show the user needs that are not linked to product features;

-   Show the status of tests on all use cases in iteration #n;

-   Show all supplementary requirements linked to tests whose status is
    untested;

-   Show the results of all tests that failed, in order of criticality;

-   Show me the features scheduled for this release, which user needs
    they satisfy, and their status.

Figure 1 shows a sample of the data traced through a project's
lifecycle.

  ---------------- ------------- --------- ----------------- ---------- ---------
  **Requirement ID **Version**   **Trace   **Trace to Design **Trace to **Trace
  and Title**                    to UC**   Specification**   Test       to Source
                                                             Script**   Code**

  **Requirement    **System \#** **UC \#** **DS \#**         **TS \#**  **SC \#**
  \#**                                                                  
  ---------------- ------------- --------- ----------------- ---------- ---------

Figure 1: Criteria used in RTM for a project SDLC

Ideally, this traceability is maintained through the Rational toolset
(Requisite Pro, Clear Case, Rational Rose, Test Director), but can be
accommodated in any properly formatted RTM, appropriately maintained in
the Rational CM Libraries under CM control.

**System Design Document**

The document details the use cases developed in the Software
Requirements Document, and details the interaction which occurs between
a user and the system.

At the top-most level, the document describes the major responsibilities
that the software and the various roles within the system. It also
describes how the high-level components collaborate with each other and
provide a rationale for why the system was decomposed into these
components.

A more detailed description of the system should be incorporated into
the document as the project evolves. The document describes the general
nature of the system, and describes the architecturally significant
parts of the design model, such as its decomposition into subsystems and
packages. For each significant package, a section of the document should
detail its decomposition into classes and class utilities.
Architecturally significant classes should be introduced and a
description of their responsibilities should accompany the introduction.
Any significant relationships, operations, and attributes should be
detailed in this document.

The document should be organized by use case, so that it provides
traceability back to the use cases in the SRD, and the initial
requirements. The document must also contain a description of the
database model and data elements used to support the application. This
data can be referenced to an appropriately maintained Entity
Relationship Diagram (ERD) and data definitions which conform to CM
standards and are appropriately maintained in the Rational CM Libraries.

**Test Plan**

The purpose of this document is to develop a standard methodology to be
used by the Test Team. The Test Methodology provides guidance for
software quality assurance practices and procedures. The Test Pan is
accompanied by specific test scripts that provide repeatable and
predicable results for verification and validation of environmental
features (infrastructure, connectivity, device support, security,
standards including section 508, HHS, and OMB, etc.), specific
requirements, design, and development features traced through the RTM
and documented in the Project Deliverables (PWP). The Test Plan should
also be accompanied by specific test scripts that can be used as the
basis for user demonstration, acceptance testing and end user training.

**User Acceptance Testing**

The User Acceptance Test (UAT) Report document summarizes the results of
UAT. It displays a Test Case and script created by the system tester,
and followed by a user. The test case, and script, should be traced back
to the initial requirement, and use case, documented in the SRD.

The document contains two sections. The two sections which make up the
documented are listed below:

-   User Acceptance Test

-   User Acceptance Test Feedback

**User Acceptance Test**

The UAT should include a Test Action, Expected Result, a section for the
user to note whether the requirement met his expectation, a place to
document the Actual Result, and a column to document the Requirement ID
or Configuration Change Request Number.

**User Acceptance Test Feedback**

The Test Feedback section details the areas of the application that did
not meet the user's expectations. It provides a list of defects, or
failed expectations, which need to be addressed prior to the next round
of testing.

The results of additional UAT should be documented in subsequent
sections in the document.

**User Guide**

Provide a set of working instructions which an end user may use as a
reference, or instructional manual, to learn the system. The End User
Support Material may also include online help.

**Operations Manual**

This document provides guidance and defines procedures related to the
operational implementation of the system. At a minimum, the document
should contain the following:

-   System Overview

-   Statement of acceptable use of the system and information

-   Hardware and software descriptions

-   Interfaces with other Systems and Databases

-   Access requirements and maintenance

-   Security procedures including virus protection

-   Incident Reporting and Handling

-   System Startup and Recovery Procedures

-   Change Management Procedures

**Version Description Document**

This document identifies and describes the general release information,
and inventory of software released (Bill of Materials), for a specific
application, including prototype iterations. The document should include
the following sections listed below:

-   Introduction

-   General Release Information

-   Installation Instructions

-   Version Description

-   Recovery Instructions

**Introduction**

Describes the objective of the document, defines the release
identification and provides contact information.

**General Release Information**

Provide information about the specific release, including any interfaces
and dependencies.

**Installation Instructions**

Describes the steps required to install the software.

**Version Description**

Provides an inventory of List Objects and Module Types such as: class
files, SQL Scripts, HTML files, DTD and XML files.

**Recovery Instructions**

Describe the steps required to reconstruct the release from the product
baselines, established in the configuration management library.

**Other Supporting Documents**

The following sub-sections are a list of additional documentation that
are planned for, and gathered, during the lifecycle of a project. The
documentation supports the project and should be placed under
configuration management. The items listed below include the following
supporting documentation:

-   Meeting Minutes

-   Peer Review

-   Lessons Learned

**Meeting Minutes**

Minutes should be taken to document any, and every, meeting concerning a
project, from initial meetings between project team members and
stakeholders/users to project team meetings. The documenting of meetings
allows for any issues, or risks, which arise during a project's
development to be noted, addressed and tracked from the risks inception
to its resolution.

The meeting minutes may also serve to document the impact to schedule
changes resulting from project risks, or may serve to document
requirements which are elicited during meetings with a client.

**Peer Review**

Peer Reviews are to be scheduled and documented during a project's
lifecycle. Reviews are to take place on three key areas of a lifecycle.
Each review should be conducted to ensure that software development
processes instituted at AHRQ are being followed. The three key areas are
listed below:

-   Requirements Review

-   Design Review

-   Code Review

Each review should be conducted according to a checklist, which allows
each reviewer to independently assess the key area.

**Lessons Learned**

A Lessons Learned meeting should be documented for each release of an
application, whether the release is major or minor. The Lessons Learned
should cover each phase of the lifecycle, including the areas listed
below:

-   Project Management (Planning)

-   Requirements

-   Design

-   Testing

-   Deployment

APPENDIX E, Innovations Clearinghouse SOW

![](media/image5.png){width="6.25in" height="0.7965277777777777in"}

**AHRQ Technical Reference Model**

**Agency for Healthcare Research and Quality**

**Office of the Chief Information Officer**

**DRAFT**

**Version 0.1**

**January 24, 2006**

**Prepared by**

**Portfolio Management Consulting**

**Document Control**

Document Information

  ---------------------- ------------------------------------------------
                         **Information**

  Document Id            

  Document Owner         *Lenore Morrison*

  Issue Date             *January 24, 2006*

  Last Saved Date        *January 24, 2006*

  File Name              *AHRQ Technical Reference Model.doc*
  ---------------------- ------------------------------------------------

Document History

  ------------- ----------------- ---------------------------------------------
  **Version**   **Issue Date**    **Changes**

  *0.1*         *January 24,      *Initial Draft*
                2006*             

                                  

                                  

                                  
  ------------- ----------------- ---------------------------------------------

Document Approvals

+--------------------+--------------+----------------------+----------+
| **Role**           | **Name©**    | **Signature**        | **Date** |
+--------------------+--------------+----------------------+----------+
| AHRQ Project       | Brain        |                      |          |
| Sponsor            | O'Rourke     |                      |          |
+--------------------+--------------+----------------------+----------+
| Project Review     | Pat Bosco    |                      |          |
| Group              |              |                      |          |
|                    | Mike Funk    |                      |          |
|                    |              |                      |          |
|                    | Ralph        |                      |          |
|                    | Dicicco      |                      |          |
|                    |              |                      |          |
|                    | Andrew       |                      |          |
|                    | Koslov       |                      |          |
|                    |              |                      |          |
|                    | John Pedder  |                      |          |
+--------------------+--------------+----------------------+----------+
| PMC Project        | Charlie      |                      |          |
| Manager©           | Saunders     |                      |          |
+--------------------+--------------+----------------------+----------+

Contents

**1.0 Technical Reference Model Overview
[4](#technical-reference-model-overview)**

1.1 Background [4](#background)

1.2 Purpose [4](#purpose)

1.3 Scope [5](#scope)

2.0 AHRQ Technical Standards Profile
[6](#ahrq-technical-standards-profile)

2.0.1 Process to Produce Recommendations
[6](#process-to-produce-recommendations)

2.0.2 Criterion Definition [6](#criterion-definition)

2.0.3 Disposition Definition [7](#disposition-definition)

2.0.4 FEA TRM Definitions [7](#fea-trm-definitions)

2.1 Component Framework [7](#component-framework)

2.2 Service Access and Delivery [8](#service-access-and-delivery)

2.3 Service Interface and Integration
[8](#service-interface-and-integration)

2.4 Service Platform and Infrastructure
[9](#service-platform-and-infrastructure)

Appendices [10](#appendices)

APPENDIX A -- AHRQ Technical Standards Profile
[11](#appendix-a-ahrq-technical-standards-profile)

APPENDIX B -- References [19](#appendix-b-references)

# 1.0 Technical Reference Model Overview

## 1.1 Background

The Federal Enterprise Architecture (FEA) Technical Reference Model
(TRM) is a component-driven, technical framework categorizing the
standards and technologies to support and enable the delivery of
services and capabilities. It also unifies existing agency standards and
E-Gov guidance by providing a foundation to advance the reuse and
standardization of technology and services from a government-wide
perspective.

Aligning agency capital investments to the TRM leverages a common,
standardized vocabulary, allowing interagency discovery, collaboration,
and interoperability. Agencies and the federal government will benefit
from economies of scale by identifying and reusing the best solutions
and technologies to support their business functions, mission, and
target architecture.

Organized in a hierarchy, the TRM categorizes the standards and
technologies that collectively support the secure delivery, exchange,
and construction of business and applications that may be used and
leveraged in a component-based (CBA) or service-oriented architecture
(SOA). The defined Federal Enterprise Architecture (FEA) TRM consists
of:

-   **Service Areas** represent a technical tier supporting the secure
    construction, exchange, and delivery of Service Components. Each
    Service Area aggregates the standards and technologies into
    lower-level functional areas. Each Service Area consists of multiple
    Service Categories and Service Standards. This hierarchy provides
    the framework to group standards and technologies that directly
    support the Service Area.

-   **Service Categories** classify lower levels of technologies and
    standards with respect to the business or technology function they
    serve. In turn, each Service Category is comprised of one or more
    Service Standards.

-   **Service Standards** define the standards and technologies that
    support a Service Category. To support agency mapping into the FEA
    TRM, many of the Service Standards provide illustrative
    specifications or technologies as examples.

## 1.2 Purpose

The Agency for Healthcare Research and Quality (AHRQ) Technical
Reference Model (TRM) is designed to identify and select products that
will fulfill, or are likely to aid in fulfilling, the AHRQ architectural
vision and also identifies industry standards that support system
interoperability and consistency. These concepts make it much easier to
maintain and improve a diverse information technology environment. This
TRM documents the results of the identification and selection of the
standards and technologies to be used when planning and designing new
systems or upgrading current systems.

It is understood that new and innovative technologies are constantly
being brought to the market place. Therefore, it is plausible that a
product not on the standards list will be considered in the design or
upgrade of a system, if this is the case the program/project manager
will need to request a waiver from using the approved standard
technologies and provide justification as to why the new technology is
the best option. The appropriate review boards will consider the waiver
and, if approved, the new technology will be evaluated for inclusion on
the standards list.

## 1.3 Scope

This TRM presents only approved products and candidate approved
products. This does not include the entire baseline of products
currently in use at HHS or AHRQ. Rather, it establishes the common
standards for current and future information system projects. The
approved standards are presented in the AHRQ Technical Standards Profile
(TSP) found in Appendix A. This document does not address the standards
associated with desktop computing environment for two reasons; first,
this area of support falls under the Information Technology Support
Center (ITSC) contract under the Office of the Secretary and secondly,
the technologies in this are change rapidly and could possibly change
every quarter best on best market value.

# 

# 

#  

# 2.0 AHRQ Technical Standards Profile

**AHRQ TECHNICAL STANDARDS PROFILE (TSP)**

The Agency for Healthcare Research and Quality (AHRQ) Technical
Standards Profile (TSP) is a list of standards and products that are
approved for use in the agency. The list is organized according to the
FEA TRM taxonomy and will be used in two ways:

1.  Provide a reference point for evaluation of OMB 300s

2.  Provide a guide to new initiatives for what technologies should be
    used for specific areas

The adjoining table contains the adopted standards for the AHRQ TRM as
recommended by the review team comprised of staff from software
development, security and enterprise architecture groups and approved by
the CIO.

### 2.0.1 Process to Produce Recommendations

The technologies identified in this recommendation were produced using a
three step process:

1.  Identify standards and technologies currently in use at HHS and at
    AHRQ. The initial list of standards and technologies was extracted
    from the 2004 HHS TSP. This list was supplemented with technology
    inventories and standards provided by the AHRQ project teams.

2.  Synthesize data to identify which standards and technologies should
    be considered for current standards. An analysis was performed to
    determine which standards and products were proven market solutions,
    in widespread usage, and likely to have enterprise procurement
    vehicles.

3.  Working Session to determine recommendation list. Several working
    sessions along with review and comment cycles were conducted to
    evaluate the standards and technologies against the criteria
    outlined below. The results of these working sessions were presented
    for review to the CIO.

### 

### 2.0.2 Criterion Definition

The following criteria were used in determining if a technology should
be included in the AHRQ TSP:

**Interoperability (IO)** \-- A standard or technology that enables
software and hardware on different machines from different vendors to
communicate and share services. For example, standards for web services.

**Data Sharing (DS)** \-- A standard or technology that enables systems
to electronically exchange data. For example, a public standard XML
schema for exchanging health information.

**Overall Efficiency and Effectiveness of the Department as a Whole
(EFCT)** \-- A standard or technology that enables the department to
operate more efficiently and effectively. For example, a unified email
and calendaring package.

**Economies of Scale (EoS)** \-- A standard or technology that is in use
at multiple OPDIVs and may be less expensive to buy in bulk. For
example, a database or application server.

### Disposition Definition

The following definitions were used for classification of the
disposition of the service specifications:

**Standard** -- indicates the service specification is an approved
standard for use at AHRQ.

**Legacy** -- indicates the service specification is currently used at
AHRQ and will continue to be supported until it replaced or closed out
of service.

**Niche** -- indicates a service specification that is usually part of a
solution package or COTS product. For example, SQL Server is a database
used in several network and security management and monitoring
solutions, but Oracle is the approved preferred standard for relational
database technical solution.

**Evaluate** -- indicates a new version of a current standard or a new
technology that AHRQ intents to test and evaluate for possible standard
adoption.

### FEA TRM Definitions

The following sections are taken from the ***Consolidated Reference
Model*** of the Federal Enterprise Architecture Framework. They define
the Service Areas along with the corresponding Service Categories and
Service Standards.

## 2.1 Component Framework

The Component Framework Service Area defines the underlying foundation
and technical elements by which Service Components are built, integrated
and deployed across Component-Based and Distributed Architectures. The
Component Framework consists of the design of application or system
software that incorporates interfaces for interacting with other
programs and for future flexibility and expandability.

This includes, but is not limited to, modules that are designed to
interoperate with each other at runtime. Components can be large or
small, written by different programmers using different development
environments and may be platform independent. Components can be executed
on standalone machines, a LAN, Intranet or the Internet.

-   **Security.** Security defines the methods of protecting information
    and information systems from unauthorized access, use, disclosure,
    disruption, modification, or destruction in order to provide
    integrity, confidentiality and availability. Biometrics, two factor
    identification, encryption, and technologies based on the NIST
    FIPS140 standards are evolving areas of focus. See:
    http://csrc.nist.gov/cryptval/

-   **Presentation / Interface.** This defines the connection between
    the user and the software, consisting of the presentation that is
    physically represented on the screen.

-   **Business Logic.** Defines the software, protocol or method in
    which business rules are enforced within applications.

-   **Data Interchange**. Define the methods in which data is
    transferred and represented in and between software applications.

-   **Data Management.** Data management is the management of all
    data/information in an organization. It includes data
    administration, the standards for defining data and the way in which
    people perceive and use it.

## 2.2 Service Access and Delivery

The Service Access and Delivery Service Area defines the collection of
Access and Delivery Channels that will be used to leverage the Service
Component, and the legislative requirements that govern its use and
interaction.

-   **Access Channels.** Access Channels define the interface between an
    application and its users, whether it is a browser, personal digital
    assistant or other medium.

-   **Delivery Channels.** Delivery Channels define the level of access
    to applications and systems based upon the type of network used to
    deliver them.

-   **Service Requirements.** Service Requirements define the necessary
    aspects of an application, system or service to include legislative,
    performance, and hosting.

-   **Service Transport.** Service Transport defines the end to end
    management of the communications session to include the access and
    delivery protocols.

## 2.3 Service Interface and Integration

The Service Interface and Integration Service Area defines the
discovery, interaction and communication technologies joining disparate
systems and information providers. SOAs leverage and incorporate Service
Interface and Integration standards to provide interoperability and
scalability.

-   **Integration.** Integration defines the software services enabling
    elements of distributed business applications to interoperate. These
    elements can share function, content, and communications across
    heterogeneous computing environments. In particular, service
    integration offers a set of architecture services such as platform
    and service location transparency, transaction management, basic
    messaging between two points, and guaranteed message delivery.

-   **Interoperability.** Interoperability defines the capabilities of
    discovering and sharing data and services across disparate systems
    and vendors.

-   **Interface.** Interface defines the capabilities of communicating,
    transporting and exchanging information through a common dialog or
    method. Delivery Channels provide the information to reach the
    intended destination, whereas Interfaces allow the interaction to
    occur based on a predetermined framework.

## 2.4 Service Platform and Infrastructure

The Service Platform and Infrastructure Service Area define the
collection of platforms, hardware and infrastructure standards that
enable Component Based Architectures and Service Component reuse.

-   **Support Platforms.** Support platforms are hardware or software
    architectures. The term originally dealt with only hardware, and it
    is still used to refer to a CPU model or computer family.

-   **Delivery Servers.** Delivery Servers are front-end platforms that
    provide information to a requesting application. It includes the
    hardware, operating system, server software, and networking
    protocols.

-   **Software Engineering.** Software engineering covers the technology
    associated with building software systems as well as technical
    solutions supporting management issues, such as testing, modeling
    and versioning. The TRM is concerned with component technical
    architecture, not engineering processes.

-   **Database / Storage.** Database / Storage refers to a collection of
    programs that enables storage, modification, and extraction of
    information from a database, and various techniques and devices for
    storing large amounts of data.

-   **Hardware / Infrastructure.** Defines the physical devices,
    facilities and standards that provide the computing and networking
    within and between enterprises.

# 

# Appendices

**Appendix A -- AHRQ Technical Standards Profile**

**Appendix B -- References**

# APPENDIX A -- AHRQ Technical Standards Profile

  ------------- ---------------- ---------------- ----------------- -------------- -------- ---------- --------- -----------------
  **Service     **Service        **Service        **Service         **Criteria**                                 **Disposition**
  Area**        Category**       Standard**       Specification**                                                

                                                                    **IO**         **DS**   **EFCT**   **EoS**   

  **Component   **Business       **Platform       VB Script         ** **          ** **    **X**      ** **     Standard
  Framework**   Logic**          Dependent**                                                                     

                                                  C-Sharp (c#)      ** **          ** **    **X**      ** **     Standard

                                                  Visual Basic      ** **          ** **    **X**      ** **     Standard

                                                  Visual Basic .NET **X**          ** **    **X**      ** **     Standard

                                 **Platform       Java              **X**          ** **    **X**      ** **     Standard
                                 Independent**                                                                   

                                                  Java Script       ** **          ** **    **X**      ** **     Standard

                                                  C/C++             ** **          ** **    **X**      ** **     Standard

                **Data           **Data           ebXML             **X**          **X**    **X**      ** **     Standard
                Interchange**    Exchange**                                                                      

                                                  Simple Object     **X**          ** **    **X**      ** **     Standard
                                                  Access Protocol                                                
                                                  (SOAP)                                                         

                                                  Coldfusion (CFML) **X**          ** **    **X**      ** **     Niche

                                                  ISO 11179         **X**          **X**    **X**      ** **     Standard

                **Data           **Database       JDBC              **X**          ** **    ** **      ** **     Standard
                Management**     Connectivity**                                                                  

                                                  Open Database     **X**          ** **    ** **      ** **     Standard
                                                  Connectivity                                                   
                                                  (ODBC)                                                         

                                                  PL/SQL            **X**          ** **    ** **      ** **     Standard

                                                  Net 9             **X**          ** **    ** **      ** **     Standard

                                                  Net 8             **X**          ** **    ** **      ** **     Legacy

                                 **Reporting /    Microsoft Office  **X**          **X**    **X**      **X**     Standard
                                 Analysis**       Pro                                                            

                                                  Adobe Acrobat     **X**          **X**    **X**      **X**     Standard

                                                  SAS               **X**          **X**    **X**      **X**     Standard

                                                  Crystal Reports   **X**          **X**    ** **      ** **     Legacy

                                                  Oracle Reports    **X**          **X**    **X**      **X**     Standard

                **Presentation   **Content        XHTML             **X**          ** **    ** **      ** **     Standard
                Interface**      Rendering**                                                                     

                                                  Dynamic DHTML     **X**          **X**    ** **      ** **     Standard

                                                  Cascading Style   **X**          **X**    ** **      ** **     Standard
                                                  Sheets                                                         

                                 **Dynamic /      Java Server Pages **X**          ** **    ** **      ** **     Standard
                                 Server-Side                                                                     
                                 Display**                                                                       

                                                  ASP .NET          **X**          ** **    ** **      ** **     Niche

                                                  Active Server     **X**          ** **    ** **      ** **     Standard
                                                  Pages                                                          

                                 **Static         HTML              **X**          ** **    **X**      ** **     Standard
                                 Display**                                                                       

                                 **Wireless /     **No              ** **          ** **    ** **      ** **      
                                 Mobile / Voice** Specification                                                  
                                                  Recommended**                                                  
  ------------- ---------------- ---------------- ----------------- -------------- -------- ---------- --------- -----------------

  ------------- -------------- ------------------ ----------------- -------------- -------- ---------- --------- -----------------
  **Service     **Service      **Service          **Service         **Criteria**                                 **Disposition**
  Area**        Category**     Standard**         Specification**                                                

                                                                    **IO**         **DS**   **EFCT**   **EoS**   

  **Component   **Security**   **Certificates /   Digital           **X**                              ** **     Standard
  Framework**                  Digital            Certificate                                                    
                               Signatures**       Authentication                                                 

                                                  Secure Sockets    **X**                              ** **     Standard
                                                  Layer (SSL)                                                    

                                                  FIPS 186          **X**                              ** **     Standard

                                                  HHS PKI Program   **X**                              **X**     Standard

                               **Supporting       Secure            **X**                              ** **     Standard
                               Security           Multipurpose                                                   
                               Services**         Internet Mail                                                  
                                                  Extensions                                                     
                                                  (S/MIME)                                                       

                                                  Transport Layer   **X**                              ** **     Standard
                                                  Security (TLS)                                                 

                                                  Web Services      **X**                              ** **     Standard
                                                  Security                                                       
                                                  (WS-Security)                                                  

                                                  Security          **X**                              ** **     Standard
                                                  Assertion Markup                                               
                                                  Language (SAML)                                                

  **Component   **Security**   **Security         Simple Key        **X**                              ** **     Standard
  Framework**                  Services**         Management                                                     
                                                  Protocol (SKIP)                                                

                                                  Secure Shell      **X**                              ** **     Standard
                                                  (SSH)                                                          

                                                  Applications      **X**                              ** **     Standard

                                                  Cryptography      **X**                              ** **     Standard

                                                  Environment       **X**                              ** **     Standard
                                                  Management                                                     

                                                  Intrusion         **X**                              ** **     Standard
                                                  Detection                                                      

                                                  Security Layers   **X**                              ** **     Standard
                                                  (Physical, Link,                                               
                                                  Network)                                                       

  **Service     **Access       **Collaboration    Microsoft         **X**          **X**    **X**      **X**     Standard
  Access and    Channels**     Communications**   Exchange Client                                                
  Delivery**                                                                                                     

                                                  Outlook           **X**          **X**    **X**      **X**     Standard

                                                  Outlook Web       **X**          **X**    **X**      **X**     Standard
                                                  Access                                                         
  ------------- -------------- ------------------ ----------------- -------------- -------- ---------- --------- -----------------

  --------------- ---------------- --------------- ----------------- -------------- -------- ---------- --------- -----------------
  **Service       **Service        **Service       **Service         **Criteria**                                 **Disposition**
  Area**          Category**       Standard**      Specification**                                                

                                                                     **IO**         **DS**   **EFCT**   **EoS**   

  **Service       **Access         **Other         X-windows, X11R6  **X**          ** **    ** **      ** **     Standard
  Access and      Channels**       Electronic                                                                     
  Delivery**                       Channels**                                                                     

                                                                     ** **          ** **    ** **      ** **      

                                                                     ** **          ** **    ** **      ** **      

                                   **Web           Internet Explorer **X**          **X**    **X**      ** **     Standard
                                   Browsers**      6.0                                                            

                                                   Netscape          **X**          **X**    **X**      ** **     Standard
                                                   Communicator                                                   

                                   **Wireless /    Blackberry 4.0    ** **          **X**    **X**      ** **     Standard
                                   PDA**                                                                          

                  **Service        **Hosting**     **No              ** **          ** **    ** **      ** **     ** **
                  Requirements**                   Specification                                                  
                                                   Recommended**                                                  

                                                   **No              ** **          ** **    ** **      ** **     ** **
                                                   Specification                                                  
                                                   Recommended**                                                  

                                   **Legislative   **No              ** **          ** **    ** **      ** **     ** **
                                   Compliance**    Specification                                                  
                                                   Recommended**                                                  

                  **Service        **Service       FTP               **X**          **X**    ** **      ** **     Standard
                  Transport**      Transport**                                                                    

                                                   HTTP              **X**          **X**    **X**      ** **     Standard

                                                   HTTPS             **X**          **X**    **X**      ** **     Standard

                                                   TCP/IP            **X**          **X**    **X**      ** **     Standard

                                                   Telnet            **X**          ** **    ** **      ** **     Legacy

                                                   ICMP              ** **          ** **    ** **      ** **     Standard

                  **Service        **Supporting    DHCP              **X**          ** **    **X**      ** **     Standard
                  Transport**      Network                                                                        
                                   Services**                                                                     

                                                   DNS               **X**          ** **    **X**      ** **     Standard

                                                   IMAP/POP3         **X**          ** **    ** **      ** **     Standard

                                                   LDAP              **X**          ** **    **X**      ** **     Standard

                                                   MIME              **X**          ** **    ** **      ** **     Standard

                                                   SMTP              **X**          ** **    **X**      ** **     Standard

                                                   SNMP              **X**          ** **    **X**      ** **     Standard

                                                   SunOne LDAP       **X**          ** **    **X**      **X**     Standard
                                                   Server                                                         

                                                   X.500             **X**          ** **    ** **      ** **     Standard

                                                   Microsoft Active  **X**          ** **    **X**      **X**     Standard
                                                   Directory                                                      

  **Service       **Interface**    **Service       **No              ** **          ** **    ** **      ** **     ** **
  Interface and                    Description     Specification                                                  
  Integration**                    Interface**     Recommended**                                                  

                                   **Service       **No              ** **          ** **    ** **      ** **     ** **
                                   Discovery**     Specification                                                  
                                                   Recommended**                                                  
  --------------- ---------------- --------------- ----------------- -------------- -------- ---------- --------- -----------------

  --------------- ---------------------- ------------------ ----------------- -------------- ------- -------- ------- ---------- ------- --------- ------- ----------------- ----------
  **Service       **Service Category**   **Service          **Service         **Criteria**                                                                 **Disposition**   
  Area**                                 Standard**         Specification**                                                                                                  

                                                                              **IO**                 **DS**           **EFCT**           **EoS**                             

  **Service       **Integration**        **Enterprise       Plumtree Portal                  ** **            **X**              **X**             **X**                     Standard
  Interface and                          Application                                                                                                                         
  Integration**                          Integration**                                                                                                                       

                                                            Citrix Terminal                  ** **            ** **              **X**             **X**                     Standard
                                                            Server                                                                                                           

                                         **Middleware**     ActiveX                          **X**            ** **              ** **             ** **                     Standard

                                                            Object Request                   ** **            ** **              ** **             ** **                      
                                                            Broker (ORB):                                                                                                    
                                                            COM+                                                                                                             

                  **Integration**        **Middleware:**    Net IQ Webtrends                 **X**            **X**              ** **             ** **                     Standard
                                         *Transaction                                                                                                                        
                                         Process                                                                                                                             
                                         Monitoring*                                                                                                                         

                                                            Rational                         **X**            ** **              **X**             **X**                     Standard
                                                            Performance                                                                                                      
                                                            Studio                                                                                                           

                  **Interoperability**   **Data             Electronic Data                  **X**            ** **              ** **             ** **                     Standard
                                         Transformation**   Interchange (EDI)                                                                                                

                                                            XML                              **X**            **X**              **X**             ** **                     Standard

                                                            XSLT                             ** **            **X**              ** **             ** **                     Standard

                                                            Informatica                      ** **            **X**              ** **             **X**                     Standard

                                                            Adobe Acrobat                    **X**            ** **              ** **             **X**                     Standard

                                                            Lightweight                      **X**            ** **              ** **             ** **                     Standard
                                                            Directory                                                                                                        
                                                            Interchange                                                                                                      
                                                            Format (LDIF)                                                                                                    

                                         **Data Types       XML                              **X**            **X**              ** **             ** **                     Standard
                                         /Validation**                                                                                                                       

                                                                                             ** **            ** **              ** **             ** **                      

                  **Interoperability**   **Data Format /    XML                              **X**            **X**              ** **             ** **                     Standard
                                         Classification**                                                                                                                    

                                                            PDF                              ** **            **X**              **X**             **X**                     Standard

                                                            ACS X12                          ** **            **X**              ** **             ** **                     Standard

                                                            HL7 v2.3+                        ** **            **X**              ** **             ** **                     Standard

                                                            IEEE 1073 (MIB)                  **X**            ** **              ** **             ** **                     Standard

                                                            DICOM                            **X**            ** **              ** **             ** **                     Standard

                                                            LOINIC                           ** **            **X**              ** **             ** **                     Standard

                                                            SNOMED CT                        ** **            **X**              ** **             ** **                     Standard

                                                            HUGN                             ** **            **X**              ** **             ** **                     Standard

                                                            HIPPA                            ** **            **X**              ** **             ** **                     Standard

                                                            Healthcare                       **X**            **X**              **X**             ** **                      
                                                            Electronic                                                                                                       
                                                            Transaction                                                                                                      
                                                            Standard Format                                                                                                  
  --------------- ---------------------- ------------------ ----------------- -------------- ------- -------- ------- ---------- ------- --------- ------- ----------------- ----------

  ------------------ ------------------ -------------- ----------------- ------------- -------------- -------- ------- ---------- ------- --------- -----------------
  **Service Area**   **Service          **Service      **Service                       **Criteria**                                                 **Disposition**
                     Category**         Standard**     Specification**                                                                              

                                                                                       **IO**         **DS**           **EFCT**           **EoS**   

  **Service Platform **Database /       **Database**                     Oracle        **X**          ** **    **X**              **X**             Standard
  and                Storage**                                                                                                                      
  Infrastructure**                                                                                                                                  

                                                                         MS Access     **X**          **X**    **X**              **X**             Legacy / Niche

                                                                         SQL Server    **X**          ** **    **X**              ** **             Niche

                                                                         MY SQL Server ** **          ** **    ** **              ** **             Legacy /Niche

                                        **Storage**                      Clariion SAN  **X**          ** **    **X**              ** **             Standard
                                                                         CX7000                                                                     

                                                                         HP RAID       ** **          **X**    ** **              **X**             Standard

                     **Delivery         **Web                            Apache        ** **          **X**    **X**              ** **             Niche
                     Servers**          Servers**                                                                                                   

                                                                         Oracle        **X**          **X**    ** **              **X**             Standard
                                                                         Application                                                                
                                                                         Server 10g                                                                 

                                                                         Microsoft     **X**          ** **    **X**              **X**             Standard
                                                                         Internet                                                                   
                                                                         Information                                                                
                                                                         Server 6.0                                                                 

                                        **Media                          Real Player   **X**          **X**    **X**              **X**             Standard
                                        Servers**                        10                                                                         

                                                                         MS Windows    **X**          **X**    **X**              **X**             Standard
                                                                         Media Player                                                               
                                                                         10                                                                         

  **Service Platform **Hardware /       **Embedded                       Dual Intel    ** **          ** **    ** **              ** **             Standard
  and                Infrastructure**   Technology                       Xeon                                                                       
  Infrastructure**                      Devices**                        Processors                                                                 

                                                                         Dual Fiber    ** **          ** **    ** **              ** **             Standard
                                                                         Channel Host                                                               
                                                                         bus adapters                                                               

                                        **Local Area                     Cisco Family  **X**          ** **    **X**              **X**             Standard
                                        Network**                        of routers                                                                 
                                                                         and switches                                                               

                                                                         802.3, 802.11 ** **          ** **    **X**              ** **             Standard
                                                                         A/B/G                                                                      

                                                                         Dual          ** **          ** **    **X**              ** **             Standard
                                                                         10/100/1000                                                                
                                                                         NIC                                                                        
                                                                         (Ethernet)                                                                 

                                        **Wide Area                      Cisco Family  **X**          ** **    **X**              **X**             Standard
                                        Network**                        of routers                                                                 
                                                                         and switches                                                               

                                                                         Steel Belted  ** **          **X**    ** **              ** **             Standard
                                                                         Radius                                                                     

                                                                         PPP           **X**          ** **    ** **              ** **             Standard

                                                                         Dial-up       **X**          ** **    ** **              ** **             Standard

                                                                         100 mb Fiber  **X**          ** **    ** **              ** **             Standard
                                                                         Optic                                                                      
                                                                         Backbone                                                                   

                                                                         CSMA/CD       **X**          ** **    ** **              ** **             Standard

                                                                         Aventail VPN  ** **          **X**    ** **              ** **             Standard
                                                                         Appliance                                                                  

                                                                                                                                                    
  ------------------ ------------------ -------------- ----------------- ------------- -------------- -------- ------- ---------- ------- --------- -----------------

  ------------------ ------------------ ------------- --------------- ----------------- ----------------- -------------- -------------- -------- -------- ---------- ---------- --------- --------- ----------------- ----------------- --
  **Service Area**   **Service                        **Service                         **Service                        **Criteria**                                                                                 **Disposition**   
                     Category**                       Standard**                        Specification**                                                                                                                                 

                                                                                                                                                                                                                                        

                                                                                                                         **IO**                  **DS**              **EFCT**             **EoS**                                       

  **Service Platform **Hardware /                     **Network                         Cisco Family of                  **X**                   ** **               **X**                **X**                       Standard          
  and                Infrastructure**                 Devices /                         routers and                                                                                                                                     
  Infrastructure**                                    Standards**                       switches                                                                                                                                        

                                                                                        Brocade Fiber                    ** **                   ** **               **X**                ** **                       Standard          
                                                                                        Channel Switches                                                                                                                                

                                                                                        Aventail VPN                     ** **                   **X**               ** **                ** **                       Standard          
                                                                                        Appliance                                                                                                                                       

                                                                                        Firewall                         ** **                   ** **               ** **                ** **                       Standard          
                                                                                        (Netscreen)                                                                                                                                     

                                                      Peripherals                       HP Printers                      **X**                   ** **               ** **                **X**                       Standard          

                                                                                        Cannon Image                     ** **                   ** **               ** **                ** **                       Niche             
                                                                                        Runners                                                                                                                                         

                                                                                        Sharp Printers                   ** **                   ** **               ** **                ** **                       Niche             

                                                                                        HP Network                       **X**                   ** **               ** **                **X**                       Standard          
                                                                                        Senders                                                                                                                                         

                                                      Video                             **No                             ** **                   ** **               ** **                ** **                       ** **             
                                                      Conferencing                      Specification                                                                                                                                   
                                                                                        Recommended**                                                                                                                                   

                                                      Servers /                         HP Family of                     **X**                   ** **               **X**                **X**                       Standard          
                                                      Computers                         Servers                                                                                                                                         

                                                                                                                         ** **                   ** **               ** **                ** **                                         

                     **Software                       Integrated                        Oracle Jdeveloper                **X**                   **X**               ** **                **X**                       Standard          
                     Engineering**                    Development                                                                                                                                                                       
                                                      Environment                                                                                                                                                                       

                                                                                        Visual Studio                    ** **                   **X**               ** **                **X**                       Standard          

                                                                                        Macromedia Studio                ** **                   **X**               ** **                ** **                       Standard          
                                                                                        MX                                                                                                                                              

                                                      Modeling                          Visio                            ** **                   **X**               ** **                **X**                       Standard          

                                                                                        Rational Rose                    ** **                   **X**               ** **                **X**                       Standard          

                                                                                        Métis                            ** **                   **X**               ** **                **X**                       Standard          

                                                                                        Erwin                            ** **                   **X**               ** **                **X**                       Standard          

                                                                                        All Fusion                       ** **                   ** **               ** **                ** **                       Legacy            
                                                                                        st./BPWin                                                                                                                                       

                                                      Software                          Rational                         **X**                   **X**               **X**                **X**                       Standard          
                                                      Configuration                     ClearCase                                                                                                                                       
                                                      Management                                                                                                                                                                        

                                                                                        Rational                         **X**                   **X**               **X**                **X**                       Standard          
                                                                                        ClearQuest                                                                                                                                      

                                                                                        Microsoft Project                ** **                   **X**               ** **                **X**                                         

                                                      Test Management                   Rational Test                    **X**                   ** **               **X**                **X**                       Standard          
                                                                                        Manager                                                                                                                                         

                                                                                        Rational Robot                   **X**                   ** **               ** **                **X**                       Standard          

                                                                                        Infocus 508                      ** **                   ** **               ** **                ** **                       Niche             

  **Service Area**   **Service          **Service                     **Service                           **Criteria**                                                                              **Disposition**                     
                     Category**         Standard**                    Specification**                                                                                                                                                   

                                                                                                          **IO**                        **DS**            **EFCT**              **EoS**                                                 

  **Service Platform **Supporting       Platform                      Windows 2003                        ** **                         ** **             **X**                 ** **               Standard                            
  and                Platforms**        Dependent                                                                                                                                                                                       
  Infrastructure**                                                                                                                                                                                                                      

                                                                      Windows 2000                        ** **                         ** **             ** **                 ** **               Legacy                              

                                                                      Windows NT                          ** **                         ** **             ** **                 ** **               Legacy                              

                                                                      Mac OS X                            ** **                         ** **             **X**                 ** **               Standard                            

                                                                      z/OS                                ** **                         ** **             **X**                 ** **               Standard                            

                     **Supporting       Platform                      Red Hat Linux 7.3                   ** **                         ** **             ** **                 ** **               Legacy                              
                     Platforms**        Independent                                                                                                                                                                                     

                                                                      Solaris 8                           ** **                         ** **             ** **                 ** **               Legacy                              

                                                                      Solaris 9                           ** **                         ** **             ** **                 ** **               Legacy                              

                                                                      POSIX                               ** **                         ** **             **X**                 ** **               Standard                            

                                                                      Red Hat Linux                       ** **                         ** **             **X**                 ** **               Standard                            
                                                                      Server                                                                                                                                                            

                                                                      Red Hat Linux                       **X**                         ** **             **X**                 ** **               Evaluate                            
                                                                      10.0                                                                                                                                                              

                                                                      Solaris 10                          **X**                         ** **             **X**                 ** **               Evaluate                            

                                        Wireless /                    Blackberry 4.0                      **X**                         **X**             ** **                 **X**               Standard                            
                                        Mobile                                                                                                                                                                                          
  ------------------ ------------------ ------------- --------------- ----------------- ----------------- -------------- -------------- -------- -------- ---------- ---------- --------- --------- ----------------- ----------------- --

# 

# 

# 

# APPENDIX B -- References

1.  AHRQ's Enterprise Architecture. Microsoft PowerPoint Presentation by
    Brian O'Rourke, Chief Technology Officer, Agency for Healthcare
    Research and Quality, 2004.

2.  Clinger-Cohen Act of 1996, Public Law 104-106, section 5125, 110
    Stat. 684 (1996)

3.  Enabling Citizen-centric Electronic Government. 2005-2006 FEA PMO
    Action Plan. OMB, March 2005.

4.  A Framework for Assessing and Improving Enterprise Architecture
    Management (Version 1.1). General Accounting Office, April 2003.

5.  Federal Enterprise Architecture Consolidated Reference Model.
    Executive Office of the President of the United States. May 2005.

6.  FY 06 A-11 FEA Additional Requirements and Guidelines. OMB, FEA PMO,
    July 30, 2004.

7.  Guidelines for Enterprise Architecture Assessment Framework. OMB FEA
    PMO, April 2004.

8.  HHS Enterprise Architecture -- Framework (version 8.0). HHS
    Enterprise Architecture Program Management Office, December 31,
    2005.

9.  HHS Technical Standards Profile (version 1). HHS Enterprise
    Architecture Program Office, December 2004.

10. NASCIO EA Development Tool-kit: Introduction and Architecture
    Governance, Version 3.0, National Association of State Chief
    Information Officers, October 2004.

Page Blank on Purpose

Page Blank on Purpose

**ATTACHMENT 2**

**PAST PERFORMANCE QUESTIONNAIRE**

**PART ONE: INSTRUCTIONS**

The offeror listed below has submitted a proposal in response to the
Agency for Healthcare Research and Quality (AHRQ) Solicitation No.
AHRQ-06-00029, entitled "Accelerating Change and Transformation in
Organizations and Networks (ACTION)." Past performance is an important
part of the evaluation criteria for this acquisition, so input from
previous customers of the offeror is important. This office would
greatly appreciate you taking the time to complete this form. **This
information is to be provided to Mary Haines, the AHRQ Contracting
Officer and is NOT to be disclosed to the offeror either verbally or in
writing.** Please provide an honest assessment and return to AHRQ to the
address shown below, no later than **July 25, 2006, 12 noon EST**. If
you have any questions, please contact Mary Haines via e-mail at
<mhaines@ahrq.gov> or the new e-mail address mary.haines@ahrq.hhs.gov.

Mary Haines

Agency for Healthcare Research and Quality

Division of Contracts Management

540 Gaither Road

Rockville, Maryland 20850

FAX: (301) 427-1740

NAME OF
OFFEROR:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

(Please print or type)

ADDRESS:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**INCLUDE THIS PAGE WHEN PROVIDING PAST PERFORMANCE EVALUATION TO AHRQ**

**[Contractor Performance Form]{.underline}**

1\. Name of
Contractor:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

2\.
Address:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

3\. Contract/Grant Number:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

4\. Contract/Grant Value (Base Plus Options):
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

5\. Contract/Grant Award Date:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

6\. Contract/Grant Completion Date:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

7\. Type of Contract/Grant: (Check all that apply) ( )FP ( )FPI (
)FP-EPA

( ) Award Fee ( ) CPFF-Completion ( ) CPFF-Term ( ) CPIF ( ) CPAF

( ) IO/IQ ( ) BOA ( ) Requirements ( ) Labor-Hour ( )T&M ( ) SBSA

( )8(a) ( )SBIR ( ) Sealed Bid( )Negotiated( )Competitive (
)Non-Competitive

8\. Description of Requirement:

**[CONTRACTOR'S PERFORMANCE RATING]{.underline}**

Ratings: Summarize contractor performance and circle in the column on
the right the number which corresponds to the performance rating for
each rating category. Please see reverse page for explanation of rating
scale.

+------------------+-----------------------------------------------+---+
| Quality of       | Comments                                      | 0 |
| Product or       |                                               |   |
| Service          |                                               | 1 |
|                  |                                               |   |
|                  |                                               | 2 |
|                  |                                               |   |
|                  |                                               | 3 |
|                  |                                               |   |
|                  |                                               | 4 |
|                  |                                               |   |
|                  |                                               | 5 |
+------------------+-----------------------------------------------+---+
| Cost             | Comments                                      | 0 |
|                  |                                               |   |
| Control          |                                               | 1 |
|                  |                                               |   |
|                  |                                               | 2 |
|                  |                                               |   |
|                  |                                               | 3 |
|                  |                                               |   |
|                  |                                               | 4 |
|                  |                                               |   |
|                  |                                               | 5 |
+------------------+-----------------------------------------------+---+
| Timeliness of    | Comments                                      | 0 |
|                  |                                               |   |
| Performance      |                                               | 1 |
|                  |                                               |   |
|                  |                                               | 2 |
|                  |                                               |   |
|                  |                                               | 3 |
|                  |                                               |   |
|                  |                                               | 4 |
|                  |                                               |   |
|                  |                                               | 5 |
+------------------+-----------------------------------------------+---+
| Business         | Comments                                      | 0 |
| Relations        |                                               |   |
|                  |                                               | 1 |
|                  |                                               |   |
|                  |                                               | 2 |
|                  |                                               |   |
|                  |                                               | 3 |
|                  |                                               |   |
|                  |                                               | 4 |
|                  |                                               |   |
|                  |                                               | 5 |
+------------------+-----------------------------------------------+---+
|                  |                                               |   |
+------------------+-----------------------------------------------+---+

Customer Satisfaction - Is/was the Contractor committed to customer
satisfaction? Yes No ; Would

you use this Contractor again? Yes No

Reason:

**NAME OF EVALUATOR:**
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**(Please Print)**

**TITLE OF EVALUATOR:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**SIGNATURE OF
EVALUATOR:**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**DATE:**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**MAILING ADDRESS:** **Include name of organization/ federal agency**

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**PHONE
#:**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**E-MAIL
:**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**Rating Guidelines:** Summarize contractor performance in each of the
rating areas. Assign each area a rating 0(Unsatisfactory), 1(Poor),
2(Fair), 3(Good), 4(Excellent) 5(Outstanding). Use the following
instructions as guidance in making these evaluations.

+------------+-------------+-----------+------------+----------------+
|            | **Quality** | **Cost    | **         | **Business     |
|            |             | Control** | Timeliness | Relation**     |
|            |             |           | of**       |                |
|            |             |           |            |                |
|            |             |           | **Per      |                |
|            |             |           | formance** |                |
+------------+-------------+-----------+------------+----------------+
|            | -Compliance | -Within   | -Met       | -Effective     |
|            | with        | bud       | interim    | management     |
|            | contract    | get(over/ | milestones |                |
|            |             |           |            | -Businesslike  |
|            | r           | under     | -Reliable  | correspondence |
|            | equirements | target    |            |                |
|            |             | costs)    | -          | -Responsive to |
|            | -Accuracy   |           | Responsive | contract       |
|            | of reports  | -Current, | to         |                |
|            |             | accurate, | technical  | requirements   |
|            | -Technical  | and       |            |                |
|            | excellence  |           | direction  | -Prompt        |
|            |             | complete  |            | notification   |
|            |             | billings  | -Completed | of problems    |
|            |             |           | on time,   |                |
|            |             | -Rel      |            | -Reasonab      |
|            |             | ationship | including  | le/cooperative |
|            |             | of        | wrap-up    |                |
|            |             |           | and        | -Flexible      |
|            |             | n         |            |                |
|            |             | egotiated | contract   | -Pro-active    |
|            |             | costs to  | adm        |                |
|            |             |           |            | -Effective     |
|            |             | actual    | -No        | small/small    |
|            |             |           | liquidated |                |
|            |             | -Cost     | damages    | disadvantaged  |
|            |             | eff       |            | business sub-  |
|            |             | iciencies | assessed   |                |
|            |             |           |            | contracting    |
|            |             | -Change   |            | program        |
|            |             | orders    |            |                |
|            |             | issue     |            |                |
+------------+-------------+-----------+------------+----------------+
| 0-unsa     | Nonc        | Ability   | Delays are | Response to    |
| tisfactory | onformances | to manage | je         | inquiries,     |
|            | are         | cost      | opardizing |                |
|            |             |           |            | tech           |
|            | j           | issues is | the        | nical/service/ |
|            | eopardizing | jeo       | a          | administrative |
|            | the         | pardizing | chievement |                |
|            |             |           | of         | issues is not  |
|            | achievement | pe        |            | effective      |
|            | of contract | rformance | contract   |                |
|            |             | of        | req        |                |
|            | re          | contract  | uirements, |                |
|            | quirements, |           |            |                |
|            | despite use | requ      | despite    |                |
|            |             | irements, | use of     |                |
|            | of Agency   | despite   | Agency's   |                |
|            | resources   |           |            |                |
|            |             | use of    | resources  |                |
|            |             | Agency    |            |                |
|            |             |           |            |                |
|            |             | resources |            |                |
+------------+-------------+-----------+------------+----------------+
| 1-Poor     | Overall     | Ability   | Delays     | Response to    |
|            | compliance  | to manage | require    | inquiries,     |
|            |             | cost      | major      |                |
|            | requires    |           |            | tech           |
|            | major       | issues    | Agency     | nical/service/ |
|            | Agency      | requires  | resources  | administrative |
|            |             | major     | to         |                |
|            | resources   |           |            | issues is      |
|            | to ensure   | Agency    | ensure     | marginally     |
|            |             | resources | a          | effective      |
|            | achievement | to        | chievement |                |
|            | of contract |           | of         |                |
|            |             | ensure    |            |                |
|            | r           | ac        | contract   |                |
|            | equirements | hievement | re         |                |
|            |             | of        | quirements |                |
|            |             |           |            |                |
|            |             | contract  |            |                |
|            |             | req       |            |                |
|            |             | uirements |            |                |
+------------+-------------+-----------+------------+----------------+
| 2-Fair     | Overall     | Ability   | Delays     | Response to    |
|            | compliance  | to manage | require    | inquiries,     |
|            |             | cost      | minor      |                |
|            | requires    |           |            | tech           |
|            | minor       | issues    | Agency     | nical/service/ |
|            | Agency      | requires  | resources  | administrative |
|            |             | minor     | to         |                |
|            | resources   |           |            | issues is      |
|            | to ensure   | Agency    | ensure     | somewhat       |
|            |             | resources | a          | effective      |
|            | achievement | to        | chievement |                |
|            | of contract |           | of         |                |
|            |             | ensure    |            |                |
|            | r           | ac        | contract   |                |
|            | equirements | hievement | re         |                |
|            |             | of        | quirements |                |
|            |             |           |            |                |
|            |             | contract  |            |                |
|            |             | req       |            |                |
|            |             | uirements |            |                |
+------------+-------------+-----------+------------+----------------+
| 3-Good     | Overall     | M         | Delays do  | Response to    |
|            | compliance  | anagement | not impact | inquiries,     |
|            | does        | of cost   |            |                |
|            |             |           | a          | tech           |
|            | not impact  | issues    | chievement | nical/service/ |
|            | achievement | does not  | of         | administrative |
|            | of          | impact    | contract   |                |
|            |             |           |            | issues is      |
|            | contract    | ac        | re         | usually        |
|            | r           | hievement | quirements | effective      |
|            | equirements | of        |            |                |
|            |             | contract  |            |                |
|            |             |           |            |                |
|            |             | req       |            |                |
|            |             | uirements |            |                |
+------------+-------------+-----------+------------+----------------+
| 4          | There are   | There are | There are  | Response to    |
| -Excellent | no quality  | no cost   | no delays  | inquiries,     |
|            |             |           |            |                |
|            | problems    | m         |            | tech           |
|            |             | anagement |            | nical/service/ |
|            |             | issues    |            | administrative |
|            |             |           |            |                |
|            |             |           |            | issues is      |
|            |             |           |            | effective      |
+------------+-------------+-----------+------------+----------------+

5-Outstanding. The Contractor has demonstrated an outstanding
performance level that justifies adding a point to the score. It is
expected that this rating will be used in those rare circumstances where
Contractor performance clearly exceeds the performance levels described
as "Excellent."

***ATTACHMEMNT 3***

> ***PROPOSAL INTENT RESPONSE & APPROVAL TO RELEASE NAME FOR
> SUBCONTRACTING OPPORTUNITIES ( 1 PAGE, 2 PURPOSES!)***
>
> **[RFP No. AHRQ-06-00002]{.underline}**
>
> **[INNOVATIONS CLEARINGHOUSE]{.underline}**
>
> **Your expression of intent is not binding but will greatly assist us
> in planning for the proposal evaluation. Furnish the information
> requested below and return this page by JUNE 7, 2006.**
>
> **\[ \] INTEND TO SUBMIT A PROPOSAL**
>
> **\[ \] DO NOT INTEND TO SUBMIT A PROPOSAL FOR THE FOLLOWING
> REASONS:**
>
> **\[ \] I GRANT PERMISSION TO THE AGENCY FOR HEALTHCARE RESEARCH AND
> QUALITY, CONTRACTS OFFICE TO ADD THE CONTACT INFORMATION BELOW TO A
> BIDDERS LIST TO PROVIDE TO OTHER INTERESTED OFFERORS FOR
> SUBCONTRACTING OPPORTUNITIES. (\*MUST INCLUDE AUTHORIZED SIGNATURE)**
>
> **COMPANY/INSTITUTION NAME:**
>
> **\*AUTHORIZED SIGNATURE:**
>
> **TYPED NAME AND TITLE:**
>
> **DATE:**
>
> **\[ \] PLEASE [DO NOT]{.underline} RELEASE THE CONTACT INFORMATION.**
>
> **Please return to:**
>
> **Mary Haines**
>
> **Agency for Healthcare Research and Quality**
>
> **Contracts Management**
>
> **540 Gaither Road**
>
> **Rockville, Maryland 20850**
>
>  

**Attachment 4**

Number range CHAPTER**[BREAKDOWN OF PROPOSED ESTIMATED COST (PLUS FEE)
AND LABOR HOURS]{.underline}**

**[INSTRUCTIONS FOR USE OF THE FORMAT]{.underline}**

**1. Refer to Business Proposal Instructions, Section L of this
solicitation. The Instructions contain the requirements for proper
submission of cost/price data which must be adhered to.**

**2. This sample format has been prepared as a universal guideline for
all solicitations. It may require amending to meet the specific
requirements of this solicitation. For example, this solicitation may
require the submission of cost/price data for three years listed on this
form. (See Section L, Instructions, Conditions and Notices to Offerors,
for the estimated duration of this project.) If this solicitation is
phased, identify each phase in addition to each year. Total each year,
phase, and sub-element.**

**3. This format must be used to submit the breakdown of all proposed
estimated cost elements. List each cost element and sub-element for
direct costs, indirect costs and fee, if applicable. In addition,
provide detailed calculations for all items. For example:**

> **a. For all personnel, list the name, title, rate per hour and number
> of hours proposed. If a pool of personnel is proposed, list the
> composition of the pool and how the cost proposed was calculated. List
> the factor used for prorating Year One and the escalation rate applied
> between years.**
>
> **Offeror\'s proposal should be stated in the same terms as will be
> used to account for and record direct labor under a contract (i.e.
> percentage of effort is used for most faculty and professional
> employees at educational institutions). If percentages of effort are
> used, the basis to which such percentages are applied
> [must]{.underline} also be submitted by the offeror. The attached
> format should be revised to accommodate direct labor proposed as a
> percentage of effort.**
>
> **b. For all materials, supplies, and other direct costs, list all
> unit prices, etc., to detail how the calculations were made.**
>
> **c. For all indirect costs, list the rates applied and the base the
> rate is applied to.**
>
> **d. For all travel, list the specifics for each trip.**
>
> **e. For any subcontract proposed, submit a separate breakdown
> format.**
>
> **f. Justification for the need of some cost elements may be listed as
> an attachment, i.e., special equipment, above average consultant fees,
> etc.**

**4. If the Government has provided \"uniform pricing assumptions\" for
this solicitation, the offeror must comply with and identify each
item.**

**Attachment 4**

**RFP Number:**

**Organization:**

**Date:**

**[BREAKDOWN OF PROPOSED ESTIMATED COST (PLUS FEE) AND LABOR
HOURS]{.underline}**

**Option Option**

**[COST ELEMENT]{.underline} [Year 1 Year 2 Year 3 Year 1 Year 2
Total]{.underline}**

**[DIRECT LABOR:]{.underline}**

**[Hours]{.underline} [Hours]{.underline} [Hours]{.underline}
[Hours]{.underline} [Hours]{.underline} [Hours]{.underline}**

**[Labor Category]{.underline} [Rate]{.underline} [Amt]{.underline}
[Amt]{.underline} [Amt]{.underline} [Amt]{.underline} [Amt]{.underline}
[Amt]{.underline}**

**(Title and Name\--**

**use additional**

**pages as necessary)**

**[DIRECT LABOR COST]{.underline}: \$ \$ \$ \$ \$ \$**

**[MATERIAL COST]{.underline}: \$ \$ \$ \$ \$ \$**

**[TRAVEL COST]{.underline}: \$ \$ \$ \$ \$ \$**

**[OTHER (Specify)]{.underline} \$ \$ \$ \$ \$ \$**

**[OTHER (Specify)]{.underline} \$ \$ \$ \$ \$ \$**

**TOTAL [DIRECT]{.underline} COST: \$ \$ \$ \$ \$ \$**

**[FRINGE BENEFIT COST]{.underline}:**

**(if applicable)**

**% of Direct Labor Cost \$ \$ \$ \$ \$ \$**

**[INDIRECT COST]{.underline}:**

**% of Total Direct Cost \$ \$ \$ \$ \$ \$**

**[TOTAL COST]{.underline}: \$ \$ \$ \$ \$ \$**

**[FEE]{.underline}:**

**(if applicable)**

**% of Total Est. Cost \$ \$ \$ \$ \$ \$**

**[GRAND TOTAL EST COST]{.underline} \$ \$ \$ \$ \$ \$**

> **EXHIBIT 1 PERFORMANCE REQUIRMENTS SUMMARY** will be provided as an
> amendment to the solicitation.

[^1]: <http://www.ahrq.gov/hrqa99a.htm>

[^2]: Rogers EM. Diffusion of innovations. Fifth Edition. New York, NY:
    Free Press. 2003

[^3]: Greenhalgh T, Robert G, Bate P. How to spread good ideas. A
    systematic review of the literature on diffusion, dissemination, and
    sustainability of innovations in health service delivery and
    organization. Report for the National Co-ordinating Centre for NHS
    Service Delivery and Organisation R&D (NCCSD). April 2004.
    <http://www.sdo.lshtm.ac.uk/changemanagement.htm> Downloaded and
    accessed 12/30/2004.

[^4]: Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV.
    Effects of a medical

    emergency team on reduction of incidence of and mortality from
    unexpected cardiac arrests in

    hospital: preliminary study. *BMJ.* 2002;324:387-390.

[^5]: Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart G, Opdam H,
    Silvester W, Doolan L, Gutteridge G. Prospective controlled trial of
    effect of medical emergency team on postoperative morbidity and
    mortality rates. *Crit Care Med* 2004; 32(4),916-921.

[^6]: Institute for Healthcare Improvement Web site, accessed 1/25/05;
    Landro, *Wall Street Journal*, 12/1/04.

[^7]: \[No authors listed\] Discharge planning advisor. Innovative
    program helps homeless, frees up beds. Hosp Case Manag. 2005
    Oct;13(10):155-7.

[^8]: Berenholtz SM, Pronovost PJ, Lipsett PA, Hobson D, Earsing K,
    Farley JE, Milanovich S, Garrett-Mayer E, Winters BD, Rubin HR,
    Dorman T, Perl TM. Eliminating catheter-related bloodstream
    infections in the intensive care unit. Crit Care Med. 2004
    Oct;32(10):2014-20.

[^9]: <http://www.whitehouse.gov/omb/inforeg/infocoll.html#fapraf>

[^10]: <http://www.archives.gov/federal_register/public_laws/paperwork_reduction_act/3507.html>