**DEPARTMENT**

> **OF HEALTH**
>
> **AND HUMAN**
>
> **SERVICES**

**Agency for Health Care Research and Quality**

FISCAL YEAR 2003 PERFORMANCE PLAN

FISCAL YEAR 2002 REVISED FINAL PLAN

FISCAL YEAR 2001 PERFORMANCE REPORT

Contact: Kathleen Kendrick, R.N., M.S.

> Director, Program Development and Evaluation
>
> 301-594-7196

TABLE OF CONTENTS

Part I - Agency Context for Performance Measurement

Introduction...........................................................................................
5

1.1 Overview of the Context of GPRA in
AHRQ....................................... 5\
Strategic Management at
AHRQ...................................................... 6

1.2 Agency Vision, Mission, and Strategic
Goals....................................... 7

1.3 Organization, Programs, Operations,
Strategies.................................. 8

Research
Pipeline.........................................................................
8

AHRQ
Audiences..........................................................................
9

AHRQ Cycle of
Research...............................................................
9

Needs
Assessment.......................................................................
10

Knowledge
Creation......................................................................
10

Translation and
Dissemination.........................................................
10

Evaluation
..................................................................................
10

Priority Populations
......................................................................
.10

Training
.......................................................................................11

1.4 Partnerships and Coordination with other Federal
Agencies.................... 11

5.  Summary of FY2001 Performance Report: Accountability Through
    ..........12

> Performance Measurement

Part II - Program Planning and Assessment

Introduction.............................................................................................14

Budget and Program
Aggregation................................................................17

FY 2001 Performance Report Results Executive
Summary...............................19

Fiscal Year 2001 Performance Report and FY 2002 Performance
Plan................27

1.  Budget Line (1) Research on Health Costs, Quality, and
    Outcomes...........29

[Goal 1]{.underline}
..........................................................................................29

Establish future research agenda based on user's needs

FY 2001 Performance Report
Results..................................................30

[Goal
2]{.underline}...........................................................................................31

Make significant contributions to the effective

functioning of the U.S. health care system through

the creation of new knowledge

FY 2001 Performance Report
Results.................................................32

[Goal 3]{.underline}
.........................................................................................38

Foster translation and dissemination of new knowledge

into practice by developing and providing information,

products, and tools on outcomes; quality; and access, cost,

and use of care

FY 2001 Performance Report
Results................................................39

[Goal
4]{.underline}.......................................................................................\...48

Evaluate the effectiveness and impact of AHRQ research

and associated activities

FY 2001 Performance Report
Results................................................49

[Goal
5]{.underline}........................................................................................64

Support Initiative to Improve Health Care Quality through

leadership and research

FY 2001 Performance Report
Results...............................................65

2.  Budget Line (2) Medical Expenditure Panel
    Surveys............................67

> [Goal
> 6]{.underline}........................................................................................68
>
> Produce and release information from MEPS on health care
>
> access, cost, use, and quality
>
> FY 2001 Performance Report
> Results..............................................69

3.  Budget Line (3) Program
    Support....................................................72

> [Goal
> 7]{.underline}.......................................................................................72

Support the overall direction and management of AHRQ

FY 2001 Performance Report
Results..............................................72

Fiscal Year 2003 Performance
Plan..........................................................75

Appendices to the Performance
Plan........................................................111

A.1 Changes and Improvements Over Previous
Year..............................113

A.2 Linkage to HHS Strategic
Plan......................................................119

A.3 Performance Measurement Linkages with Budget,
...........................124

Cost Accounting, Information Technology Planning,

Capital Planning and Program Evaluation

Fiscal Year 2001 Performance
Summary..................................................127

Brief List of Patient Safety
Awards...........................................................155

Part I - AGENCY CONTEXT FOR

PERFORMANCE MEASUREMENT

*"Knowing is not enough; we must apply*

*Willing is not enough; we must do"*

\--Goethe

**Introduction:**

The Agency for Healthcare Research and Quality (AHRQ) Performance Plan
is a companion piece to the AHRQ Strategic Plan and to the FY 2003
Budget Request. In this document the initial FY 2003 and revised FY 2002
Performance Plans have been merged with the FY 2001 Performance Report
to comply with the format developed by the Department of Health and
Human Services (HHS).

The 2003 Performance Plan being submitted has undergone an extensive
review, reorganization and revision that we believe will increase its
clarity and usefulness as a strategic management tool. This document
reflects the agencies transition from goals which were closely aligned
with the "Cycle of Research" to goals which are more closely reflect the
agency's vision, mission and strategic goals. As a result, the FY2001
Performance Report and the FY 2002 Performance Plan will continue to be
organized around the seven goals identified in the FY 2002 Congressional
Justification. Beginning with the 2003 Performance Plan, however,
performance goals will be more closely aligned with the agencies
strategic goals and performance measures will include both output and
outcome measures. Specifically, these changes include:

-   Individual sections devoted to a single strategic goal. Each section
    will contain the Strategic Goal and Strategic Objectives, followed
    by the Performance Goal, Performance Measure, and Strategies.

```{=html}
<!-- -->
```
-   Use of the results by AHRQ and issues related to data availability
    and integrity as well as the identification of key factors that
    influence our success have been rewritten and incorporated with the
    performance goals they support.

```{=html}
<!-- -->
```
-   Strategies receive a greater focus, as they give direction and
    guidance to AHRQ staff and outline how we will achieve our goals.

We believe that these revisions will enable us to determine how well the
basic knowledge which forms the core of AHRQ's work provides information
which can be turned into actions by those who make clinical decisions,
purchasers and providers who make decisions about what services to use,
pay for and how to structure those services, as well as by policy
makers.

**1.1 Overview of the Context of GPRA in AHRQ**

General program direction and strategic planning is accomplished through
the collaboration of the Office of the Director (with its four
administrative offices) and six Research Centers, which have
programmatic responsibility for portions of the Agency's

research portfolio. The Agency planning processes, facilitated by the
electronic Planning System completed in FY2000, are linked to budget
planning and performance management through GPRA.

The agency's strategic plan guides the overall management of the agency.
Each Office and Center (O/C) have individual strategic and operations
plans. The annual operations plans identify critical success factors
that illustrate how each O/C contributes to AHRQ achieving its strategic
and annual performance plan goals, as well as internal O/C management
goals. In turn these critical success factors serve as the basis for
each employee's annual performance plan. This nesting of plans allows
the individual employee to see how her or his job and accomplishments
further the respective unit's goals and the Agency's mission. At the end
of each year, the Office and Center Directors and their staffs review
their accomplishments in relation to the annual operations plans and
draft the next year's plans. The results of the reviews contribute
significantly to the performance reports, which are influential in
revising the operations plans and in turn, the agency strategic plan.

# Strategic Management at AHRQ

As a result of the increased emphasis on strategic planning, evaluation
activities have

taken on greater focus. Evaluations are used to demonstrate the impact
of Agency work on the health care system, to test and improve the
usefulness and usability of Agency products, and to assess the
effectiveness and efficiency of internal operations. The results of the
evaluation studies are used to make planning, budget, and operations
decisions in subsequent years, as well as for GPRA reporting purposes.

**1.2 Agency Vision, Mission, and Strategic Goals**

**Vision**

The vision of the Agency for Healthcare Research and Quality (AHRQ) is
to foster health care research that helps the American health care
system provide access to high quality, cost-effective services; be
accountable and responsive to consumers and purchasers; and improve
health status and quality of life.

**Mission**

The Agency's mission is to improve the outcomes and quality of health
care services, reduce its costs, address patient safety, and broaden
access to effective services, through the establishment of a broad base
of scientific research and through the promotion of improvements in
clinical and health system practices, including the prevention of
diseases and other health conditions.

The Agency promotes health care quality improvement by conducting and
supporting health services research that develops and presents
scientific evidence regarding all aspects of health care. Health
services research addresses issues of "organization, delivery,
financing, utilization, patient and provider behavior, quality,
outcomes, effectiveness and cost. It evaluates both clinical services
and the system in which these services are provided. It provides
information about the cost of care, as well as its effectiveness,
outcomes, efficiency, and quality. It includes studies of the structure,
process, and effects of health services for individuals and populations.
It addresses both basic and applied research questions, including
fundamental aspects of both individual and system behavior and the
application of interventions in practice settings."[^1]

**AHRQ Strategic Plan**

The strategic plan will serve as the road map for AHRQ activities
through 2002. The current plan was released in December 1998 after an
extensive planning process and was made widely available for comment. In
2001, AHRQ published its second "Request for Ideas" (RFI) soliciting
ideas from the Agency's customers and the general public for priorities
in the context of planning for the new Strategic Plan. In addition, the
National Advisory Committee (NAC) to AHRQ is providing feedback to the
agency, and several NAC members have agreed to participate in an ad hoc
advisory planning group.

AHRQ assesses the progress made toward achieving each of the goals as
part of the annual planning and budget development process. These
assessments are integral to AHRQ's compliance with the Government
Performance and Results Act of 1993 and provide the backdrop against
which the next year's activities are planned. The completion of the 2000
performance report provided valuable information to the Agency on
progress toward strategic goals. Results have been used to gather new
knowledge, improve research management, and strengthen dissemination
activities.

AHRQ has identified three strategic goals, each of which will contribute

to improving the quality of health care for all Americans.

###### AHRQ Goal 1. Support Improvements in Health Outcomes

AHRQ seeks to support research to understand and improve decision-making
at all levels of the health care system, the outcomes of health care
and, in particular, what works, for whom, when, and at what cost.

######## AHRQ Goal 2. Strengthen Quality Measurement and Improvement

AHRQ is interested in a broad array of research topics, including
studies to develop valid and reliable measures of the process and
outcomes of care, causation and prevention of errors in health care,
strategies for incorporating measures of quality improvement into
programs, and dissemination and implementation of validated quality
improvement mechanisms.

> **AHRQ Goal 3. Identify Strategies To Improve Access, Foster
> Appropriate Use, and Reduce Unnecessary Expenditures**

AHRQ will focus on whether particular approaches to health care delivery
and financing, or characteristics of the health care market, alter
behaviors in ways that improve access and promote cost-effective use of
health care resources.

**1.3 Organization, Programs, Operations**, **and Strategies**

The main focus of AHRQ research is on the delivery of health care and
identifying ways to measure and improve it. Most of the Agency's
research portfolio consists of extramurally funded work from leading
universities and other research institutions throughout the Nation. The
portfolio also contains an impressive body of intramural research.
Issues related to the quality, cost and use of, as well as access to,
health care are studied through extramural and intramural research.
Extramural research is the primary source of studies on outcomes and
effectiveness. AHRQ sponsored and conducted research measures the
effectiveness of the services that deliver preventive, diagnostic, and
therapeutic care, compares them with existing practice, and evaluates
the ability of the health care system to deliver them effectively. The
Agency has an increasing focus on patient safety and the reduction in
medical errors, as well as research that will accelerate and magnify the
impact of research on clinical practice and patient outcomes. Through
the Translating Research Into Practice (TRIP) agenda, the Agency
sponsors applied research to develop sustainable and replicable models
and tools to improve the quality, outcomes, effectiveness, efficiency,
and cost effectiveness of health care.

**The Research Pipeline**

New Knowledge New Tools and Talent Translating Research

On Priority for a New Century Into Practice

Health Issues

The AHRQ portfolio reflects a "pipeline" of activities that together
build the infrastructure, tools, and knowledge for improvements in the
American health care system. This pipeline begins with the funding of
new research that answers important questions about what works in
American health care (*New Knowledge on Priority Health Issues*).

The second section in the pipeline (*New Tools and Talent for a New
Century*) is focused on more applied research and translates new
knowledge into instruments for measurement, databases, informatics, and
other applications that can be used to assess and improve care.

The final section of the pipeline is where the first two investments
come together by closing the gap between what we know and what we do
*(Translating Research Into Practice*). AHRQ funds research and
demonstrations to translate the knowledge and tools into measurable
improvements in the care Americans receive.

# AHRQ Audiences

######## 

Agency activities begin and end with the end-users of Agency research.
AHRQ customers require evidence-based information to inform health
policy decisions.

Health policy choices in this context represent three general levels of
decision-making:

*Clinical Decisions* - Information is used every day by clinicians,
consumers, patients, and health care institutions to make choices about
what works, for whom, when, and at what cost.

*Health Care Organizations Policy Decisions* - Health plan and system
administrators, policymakers, and purchasers are confronted daily by
choices on how to improve the health care system's ability to provide
access to and deliver high-quality, high-value care.

*Public Policy Decisions* - Policymakers to expand their capability to
monitor and evaluate the impact of system changes on outcomes, quality,
access, cost, and use of health care use Information and to devise
policies designed to improve the performance of the system. These
decisions include those made by Federal, State, and local policymakers
and those that affect the entire population or certain segments of the
public.

**AHRQ Cycle of Research**

![](media/image1.png){width="2.8in" height="2.2118055555555554in"}

Producing meaningful contributions to the Nation and to research on
health care requires continuous activity focused on iterative
improvement in priority setting, on developing research initiatives, and
on research products and processes. The following research cycle
describes the processes AHRQ uses to conduct its ongoing activities in
order to make the most productive use of its resources.

**Needs Assessment**

Agency activities begin and end with the end-users of Agency research.
The research agenda is based on an assessment of gaps in the knowledge
base and on the needs of patients, clinicians, institutions, plans,
purchasers, and State and Federal policymakers for evidence-based
information. Input gained during the needs assessments feeds directly
into the research initiatives undertaken by the Agency, as well as the
products developed from research findings to facilitate use in health
care.

**Knowledge Creation**

AHRQ will support and conduct research to produce the next generation of
knowledge needed to improve the health care system. Building on the last
12 years of investment in outcomes and health care research, AHRQ will
focus on national priority areas for which much remains unknown.

**Translation and Dissemination**

Simply producing knowledge is not sufficient; findings must be useful
and made widely available to practitioners, patients, and other decision
makers. The Agency will systematically identify priority areas for
improving care through integrating findings into practice and will
determine the most effective ways of doing this. Additionally, AHRQ will
continue to synthesize and translate knowledge into products and tools
that support its customers in problem solving and decision making. It
will then actively disseminate the knowledge, products, and tools to
appropriate audiences. Effective dissemination involves forming
partnerships with other organizations and leveraging resources.

**Evaluation**

Knowledge development is a continuous process. It includes a feedback
loop that depends on evaluation of the research's utility to the end
user and impact on health care. In order to assess the ultimate outcomes
of AHRQ research, the Agency will place increased emphasis on evaluation
of the impact and usefulness of Agency-supported work in health care
settings and policymaking. The evaluation activities will include a
variety of projects, from smaller, short-term projects that assess
process, outputs, and interim outcomes to larger, retrospective projects
that assess the ultimate outcomes/impact of AHRQ activities on the
health care system.

######## Priority Populations

Health services research has consistently documented the persistent, and
at times great, disparities in health status and access to appropriate
health care services for certain groups. The Agency's reauthorization
legislation, P.L. 106-129, the Healthcare Research and Quality Act of
1999, mandated the creation of an Office of Priority Populations to
continue and build the research and associated activities that AHRQ
undertakes on health care for priority populations to eliminate
disparities. These populations include racial and ethnic minorities,
women, children (including adolescents), the elderly, people with
special needs (disabilities, chronic illness, end-of-life issues), low
income populations and those from inner-city and rural (including
frontier) areas with health care delivery issues. In FY 2000, AHRQ
developed the structure for the Agency's Office of Priority Populations
Research (OPPR) to continue focusing on developing science-based
information to address issues of access to care, outcomes, quality, and
the cost and use of services for each of these priority populations. In
2001 the Office of Priority Populations Research was established. A
nationwide search is currently underway to recruit a leader in the field
to serve as the Director of that office. Until a Director is recruited,
the Deputy Director of the Agency has responsibility for overseeing OPPR
priorities.

######## Training

AHRQ assures a strong infrastructure for health services research
through investments in training and the support of young investigators.
Critical areas of emphasis include:

o bringing diversity to the health services research workforce by
increasing

the number of trained minority researchers;

o instituting training programs to build research capacity in states
that have

not traditionally been involved in health services research, but are
interested in developing the needed infrastructure;

o training programs for junior-level researchers and mid-career
scientists to

emerging and innovative research methods; and

o training programs that build curricula and foster innovative training

approaches.

**1.4 Partnerships and Coordination with Other Federal Agencies**

AHRQ is not able to accomplish its mission alone. Partnerships formed
with the agencies within the Department of Health and Human Services,
with other components of the Federal Government, with State and local
governments, and with private sector organizations play a critical role
in enabling the Agency to achieve its goals.

Most of the Agency's partnerships are related to:

-   The development of new research knowledge

o AHRQ co-funds individual research projects and sponsors joint research
solicitations with agencies within HHS, such as NIH, CDC, and SAMHSA

o AHRQ co-funded research with the David and Lucille Packard Foundation.

-   The development of tools, measures, and decision support mechanisms

o HRSA and AARP partnered with AHRQ to develop the Put Prevention into
Practice Personal Health Guide for Adults Over 50.

o An increasing number of agencies (such as NIH, CMS, and the VA) are
working closely with AHRQ's Evidence-based Practice Centers to develop
assessments of existing scientific evidence to guide their work.

o Evidence reports are being used to develop clinical practice
guidelines by organizations such as the American Psychiatric
Association, American Academy of Pediatrics, American College of
Obstetrics and Gynecology, American Academy of Physicians, the
Consortium for Spinal Cord Medicine, American Academy of Cardiology, and
American Heart Association.

o The Healthcare Cost and Utilization Project (HCUP) is a long standing
public-private partnership between AHRQ and 22 partner states to build a
multi-state data system

-   The translation of research into practice

o 14 companies/organizations have joined AHRQ in disseminating its
Quality Navigational Tool designed to assist individuals apply research
findings on quality measures and make major decisions regarding health
plans, doctors, treatments, hospitals, and long-term care (e.g. Midwest
Business Group on Health, IBM, United Parcel Service, National Consumers
League).

o 14 organizations/companies have joined AHRQ in disseminating smoking
cessation materials (e.g. American Cancer Society, American Academy of
Pediatrics, Michigan Department of Community Health, Utah Tobacco
Prevention and Control System).

**1.5 Summary FY 2001 Performance Report:**

**Accountability through Performance Measurement**

A summary table of program performance for FY 2001 follows:

######## STATUS OF AHRQ FY 1999-2003 PERFORMANCE MEASURES

+------------------+--------+--------------+-------------+------------+
| Goal             | Number | Data Not     | Number Of   | Number of  |
|                  | Of     | Available    | Measures    | Measures   |
|                  | Me     |              | Where       | where      |
|                  | asures |              | Targets     | Targets    |
|                  |        |              | Were        | Were Not   |
|                  |        |              | Achieved Or | Achieved   |
|                  |        |              | Exceeded\*  |            |
+------------------+--------+--------------+-------------+------------+
| Goal 1:          | FY 01: | FY 01: 0     | FY 01: 1    | FY 01: 0   |
| Establish future | 1      |              |             |            |
| research needs   |        | FY 00: 0     | FY 00: 1    | FY 00: 0   |
| based on user's  | FY 00: |              |             |            |
| needs.           | 1      | FY 99: 0     | FY 99: 1    | FY 99: 0   |
|                  |        |              |             |            |
| (HCQO)           | FY 99: |              |             |            |
|                  | 1      |              |             |            |
+------------------+--------+--------------+-------------+------------+
| Goal 2: Make     | FY 01: | FY 01: 0     | FY 01: 8    | FY 01: 0   |
| significant      | 8      |              |             |            |
| contributions to |        | FY 00: 0     | FY 00: 7    | FY 00: 0   |
| the effective    | FY 00: |              |             |            |
| Functioning of   | 7      | FY 99: 0     | FY 99: 5    | FY 99: 0   |
| the U.S. health  |        |              |             |            |
| care system      | FY 99: |              |             |            |
| through the      | 5      |              |             |            |
| creation of new  |        |              |             |            |
| knowledge.       |        |              |             |            |
| (HCQO)           |        |              |             |            |
+------------------+--------+--------------+-------------+------------+

+------------------+--------+--------------+-------------+------------+
| Goal 3: Foster   | FY 01: | FY 01: 0     | FY 01: 12   | FY 01: 0   |
| translation and  | 12     |              |             |            |
| dissemination of |        | FY 00: 0     | FY 00: 17   | FY 00: 0   |
| new knowledge    | FY 00: |              |             |            |
| into practice by | 17     | FY 99: 0     | FY 99: 15   | FY 99: 0   |
| developing and   |        |              |             |            |
| providing        | FY 99: |              |             |            |
| information,     | 15     |              |             |            |
| products, and    |        |              |             |            |
| tools on         |        |              |             |            |
| outcomes;        |        |              |             |            |
| quality; and     |        |              |             |            |
| access, use, and |        |              |             |            |
| cost of care.    |        |              |             |            |
| (HCQO)           |        |              |             |            |
+------------------+--------+--------------+-------------+------------+
| Goal 4: Evaluate | FY 01: | FY 01: 0     | FY 01: 17   | FY 01: 0   |
| the              | 17     |              |             |            |
| effectiveness    |        | FY 00: 0     | FY 00: 11   | FY 00: 0   |
| and impact of    | FY 00: |              |             |            |
| AHRQ research    | 11     | FY 99: 0     | FY 99: 7    | FY 99: 0   |
|                  |        |              |             |            |
| and associated   | FY 99: |              |             |            |
| activities.      | 7      |              |             |            |
| (HCQO) (Note:    |        |              |             |            |
| All Agency       |        |              |             |            |
| evaluation       |        |              |             |            |
| activities,      |        |              |             |            |
| including        |        |              |             |            |
| MEPS-related     |        |              |             |            |
| studies, are     |        |              |             |            |
| included under   |        |              |             |            |
| Goal 4. This is  |        |              |             |            |
| because the MEPS |        |              |             |            |
| budget line      |        |              |             |            |
| covers only      |        |              |             |            |
| costs associated |        |              |             |            |
| with data        |        |              |             |            |
| design, data     |        |              |             |            |
| collection and   |        |              |             |            |
| analysis, and    |        |              |             |            |
| data products.)  |        |              |             |            |
+------------------+--------+--------------+-------------+------------+
| GOAL 5: Support  | FY 01: | FY 01: 0     | FY 01: 6    | FY 01: 0   |
| of initiative to | 6      |              |             |            |
| improve health   |        | FY 00: 0     | FY 00: 6    | FY 00: 0   |
| care quality     | FY 00: |              |             |            |
| through          | 6      | FY 99: 0     | FY 99: 6    | FY 99: 0   |
| leadership and   |        |              |             |            |
| research. (HCQO) | FY 99: |              |             |            |
|                  | 6      |              |             |            |
+------------------+--------+--------------+-------------+------------+
| Goal 6: Produce  | FY 01: | FY 01: 0     | FY 01: 4    | FY 01: 0   |
| and release      | 4      |              |             |            |
| information from |        | FY 00: 0     | FY 00: 6    | FY 00: 0   |
| MEPS on          | FY 00: |              |             |            |
|                  | 6      | FY 99: 0     | FY 99: 11   | FY 99: 0   |
| health care      |        |              |             |            |
| access, cost,    | FY 99: |              |             |            |
| use, and         | 11     |              |             |            |
| quality.         |        |              |             |            |
+------------------+--------+--------------+-------------+------------+
| Goal 7: Support  | FY 01: | FY 01: 0     | FY 01: 6    | FY 01: 0   |
| the overall      | 6      |              |             |            |
| direction and    |        | FY 00: 0     | FY 00: 5    | FY 00: 1   |
| management of    | FY 00: |              |             |            |
| AHRQ.            | 5      | FY 99: 0     | FY 99: 5    | FY 99: 0   |
|                  |        |              |             |            |
|                  | FY 99: |              |             |            |
|                  | 5      |              |             |            |
+------------------+--------+--------------+-------------+------------+

**\*Note: Includes performance which is within 5% of estimated target.**

##### 

##### As the table demonstrates, AHRQ performance for the output and process measures in the earlier performance plans has been excellent, and serves as a strong foundation for building future program performance goals. The future challenge will be to demonstrate excellent program performance as the plan evolves from one focused on outputs and process to goals with more outcome focus. More detail on AHRQ's 01 performance can be found in the appendix of this document. 

##### PART II -- PROGRAM PLANNING AND ASSESSMENT

\"What we really want to get at is not how many reports have been done,
but how many people\'s lives are being bettered by what has been
accomplished. In other words, is it being used, is it being followed, is
it actually being given to patients? \[W\]hat effect is it having on
people?"

**Congressman John Porter, 1998**

**Chairman, House Appropriations Subcommittee on Labor, HHS, and
Education**

Introduction and Rationale:

The purpose of research is to produce information and knowledge that can
inform decision-making. There is increasing awareness among those who
provide and receive health care services, those who pay for those
services and those who are making policy decisions that health care
should be research led and that the services which are provided should
be evidence based. As a result, research agencies must find a way to
demonstrate the benefits of the research produced, not only in terms of
how many research finding are published in professional journals but how
the investment in research results in practical everyday applications
which can be used by people who need information to make decisions about
health care.

Demonstrating that research has led to tangible effects in the care
provided to individual patients is difficult. Impact is rarely
immediate. For example, we may learn that a health care organization has
adopted a policy based on research funded by AHRQ, however, it might be
several years before we know what effect it has had on patient care
overall. Knowing that some clinicians, or health systems are changing
their practice is different from knowing how overall practice patterns
are being influenced and what the effect is on clinical outcomes.

To address the need to demonstrate the impact of research on people's
health, AHRQ staff and consultants developed a "pyramid of outcomes"
model that includes four different levels of impact beginning at the
base with impact on knowledge and further research development and
progressing to the impact on health outcomes.

**Outcomes' Impact Pyramid**

Stryer et al. (2000)

This model of assessing impact of AHRQ sponsored research forms the
basis for the development of performance measures. Similarly, AHRQ must
ensure that performance measures are developed to assess the impact of
the research investment at all levels of the pyramid.

Performance measures aimed at the base of the pyramid focus on research
that contributes to the health care knowledge base, leads to future
research or both. Research at this level includes the development of
tools and methods for research, instruments and techniques to assist
clinical decision-making, and identify areas which do not have a
sufficient evidence base. The process indicators that are developed to
measure performance at this level of the pyramid assess the quantity or
quality of activities that have the potential to contribute, at least
indirectly, to helping AHRQ meet it's strategic goals, or to monitor the
establishment of major new initiatives or implementation of improvements
in core activities where significant resources are involved or the
potential for significance of the ultimate impact is high.

**Output indicators** are developed aimed at capturing the impact at the
second and third level of the pyramid. Research at the second level of
impact is research that results in the creation of a policy or program
by a professional organization, health plans, hospitals, legislative
bodies, regulators or accrediting organizations. Similarly, impact at
level three of the pyramid is defined as research that results in a
change in what clinicians or patients do, or changes in a pattern of
care.

##### AHRQ has developed outcome indicators to measure impact at the fourth level of the pyramid, that is impact on the quality of care, actual health outcomes, cost of treatment or access to health care. Often, however, the connection between a particular research project and health outcomes is indirect and can take years to emerge. As a result, AHRQ has developed measures that utilize a "convergence of evidence" approach to establish a connection between research and outcomes. This involves identifying bodies of research which, when considered together, establish a connection between research and outcomes.

**AHRQ Performance Indicators**

+----------+---------------+------------------------+-----------------+
| **Phase  | Year One --   | Years 3 - 5 -- results | Years 3 - 10 -- |
| of       | research      | received               | results used in |
| init     | initiative    |                        | health care     |
| iative** | starts        |                        | system          |
+----------+---------------+------------------------+-----------------+
| **I      | Process       | Output indicators      | Outcome         |
| ndicator | indicators    |                        | indicators      |
| type**   |               |                        |                 |
+----------+---------------+------------------------+-----------------+
| **I      | Grants        | Publications, web      | Results of      |
| ndicator | funded,       | site,                  | evaluation      |
| ex       | creation of   |                        | studies, users  |
| amples** | reports,      | dissemination,         | stories,        |
|          | partnerships  | research findings,     | analysis of     |
|          | formed        | reports, products      | trend/other     |
|          |               | available for use in   | data            |
|          |               | health care system     |                 |
+----------+---------------+------------------------+-----------------+

BUDGET AND PROGRAM AGGREGATION:

The AHRQ GPRA annual performance report and plans are aligned with the
Agency's three budget lines:

\(1\) Research on Health Care Costs, Quality, and Outcomes,

\(2\) Medical Panel Expenditure Surveys, and

\(3\) Program Support.

The first two budget lines are where Agency programs are funded. The
goals associated with each of the budget lines represent core activities
funded in each. The following two tables illustrate how the GPRA goals
are aligned with the AHRQ budget lines.

Table I, representing the GPRA goals for FY 2001 and FY 2002 uses the
cycle of research as a basic framework underpinning the development of
goals and measures for AHRQ's budget line: Research on Health Care
Costs, Quality and Outcomes.

Table 1: GPRA Framework FY 2001-2002

  ------------------ ----------------------------------------------------
  **What the         **GPRA Goal**
  Indicators         
  Address**          

  **Budget line 1:   
  Research on Health 
  Care Costs,        
  Quality, and       
  Outcomes**         

  Cycle of Research  **GPRA Goal 1:** Establish Future Research Agenda
  Phase 1: Needs     Based on User's Needs
  Assessment         

  Cycle of Research  **GPRA Goal 2:** Make significant contributions to
  Phase 2: Knowledge the effective functioning of the US health care
  Creation           system through the creation of new knowledge.

  Cycle of Research  **GPRA Goal 3:** Foster translation of new knowledge
  Phase 3:           into practice by developing and providing
  Translation and    information, products, and tools on outcomes,
  Dissemination      quality, access, cost and use of care.

  Cycle of Research  **GPRA Goal 4:** Evaluate the effectiveness and
  Phase 4:           impact of AHRQ research and associated activities.
  Evaluation         

  Lead role for      **GPRA Goal 5:** Support Department-wide Initiative
  quality initiative to Improve Health Care Quality through leadership
                     and research.

  **Budget line 2:   
  Medical Panel      
  Expenditure        
  Surveys**          

  Core MEPS          **GPRA Goal 6:** Collect current data and create
  activities         data tapes and associated products on health care
                     use and expenditures for use by public and
                     private-sector decision makers and researchers.

  **Budget line 3:   
  Program Support**  

  Agency management  **Goal 7:** Support the overall direction and
  activities:        management of AHRQ
  contracts          
  management and the 
  AHRQ Intranet.     
  ------------------ ----------------------------------------------------

##### Beginning in FY 2003, AHRQ has redesigned our strategic management system and revised our GPRA goals to align more closely with the Agencies strategic plan. Table 2 shows this revision and realignment.

Table 2: FY 2003 GPRA Framework

+------------------+---------------------------------------------------+
| **What the       | **GPRA Goal**                                     |
| Indicators       |                                                   |
| Address**        |                                                   |
+------------------+---------------------------------------------------+
| **Budget line 1: |                                                   |
| Research on      |                                                   |
| Health Care      |                                                   |
| Costs, Quality,  |                                                   |
| and Outcomes**   |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | **To have measurable improvement in the quality   |
| 1:               | and safety of healthcare for Americans**.         |
|                  |                                                   |
| Strengthen       |                                                   |
| Quality          |                                                   |
| Measurement and  |                                                   |
| Improvement      |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | **To have measurable improvement in the type of   |
| 2:               | delivery system or processes by which care is     |
|                  | provided and their effects on health care         |
| Support          | outcomes.**                                       |
| Improvements in  |                                                   |
| Health outcomes  |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | **To develop the evidence base for policy makers  |
| 3: Identify      | and health systems to use in making decisions     |
| Strategies to    | about what services to pay for, how to structure  |
| Improve Access,  | those services, and how those services are        |
| Foster           | accessed.**                                       |
| Appropriate Use  |                                                   |
| and Reduce       |                                                   |
| Unnecessary      |                                                   |
| Expenditures     |                                                   |
+------------------+---------------------------------------------------+
| **Budget line 2: |                                                   |
| Medical Panel    |                                                   |
| Expenditure      |                                                   |
| Surveys**        |                                                   |
+------------------+---------------------------------------------------+
| Core MEPS        | **To provide comprehensive, relevant and timely   |
| activities       | data on health care use and expenditures for use  |
|                  | by public and private sector decision makers and  |
|                  | researchers.**                                    |
+------------------+---------------------------------------------------+
| **Budget line 3: |                                                   |
| Program          |                                                   |
| Support**        |                                                   |
+------------------+---------------------------------------------------+
| Enhance the      | **Maximize the value of AHRQ by developing        |
| value of AHRQ as | efficient and responsive business processes,      |
| the leader in    | aligning human capital policies and practices     |
| Healthcare       | with AHRQ's mission, building an integrated and   |
| Outcomes,        | reliable information technology infrastructure**  |
| Quality, Cost,   |                                                   |
| Use and Access   |                                                   |
+------------------+---------------------------------------------------+

We believe that this realignment will help us determine how well the
basic knowledge which forms the core of AHRQ's work provides information
which can be turned into actions by clinical decision makers; purchasers
and providers who make decisions about what services to use and pay for
and how to structure those services; as well as by policy makers.

NOTE: A complete FY2001 performance report has been included in the
Appendix.

# FY 2001 GPRA Performance Report Results

# Executive Summary

The mission of the Agency for Healthcare Research and Quality is to
conduct and sponsor research that will help improve the outcomes and
quality of health care, reduce costs, address patient safety and medical
errors, and broaden access to effective services. AHRQ\'s ability to
sustain a high level of performance during fiscal year 2001 is evidenced
by how its research has been used to provide better health care and the
impact it has had on the delivery of health care services.

Among other activities, in fiscal year 2001, AHRQ:

-   AHRQ's Evidence-Based Practice Centers (EPCs) researched 28 new
    topics. These included research into: seasonal allergies; diabetes;
    venous thrombosis; hyperbaric oxygen therapy treatment for brain
    injury and stroke; the management of bronchiolitis and coronary
    heart disease in women; vaginal birth following Caesarian sections;
    and, the diagnosis and treatment of Attention Deficit/Hyperactivity
    Disorder(ADHD) in children;

-   The Integrated Delivery System Research Network (IDSRN) was charged
    with 16 projects focusing moreover on improving patient safety and
    working conditions for healthcare workers and reducing disparities
    in health care delivery;

-   State participation in the Healthcare Cost and Utilization Project
    (HCUP) increased by 15% this past fiscal year or to 25 states in
    sum. HCUP was also expanded to include hospital-based ambulatory
    surgery databases. The pilot of emergency department databases was
    expanded from one to five states. AHRQ also announced the
    availability of the Kids' Inpatient Database (KID), the first
    comprehensive research database exclusively concerned with inpatient
    care of children and adolescents in the nation's community
    hospitals. Finally, HCUP data was made more available to the public
    through HCUPnet;

-   AHRQ established nine Excellence Centers to Eliminate Ethnic/Racial
    Disparities (EXCEED). This 5-year effort will bring together teams
    of new and experienced investigators to analyze the factors that
    contribute to ethnic and racial inequities in health care and to
    identify practical tools and strategies to eliminate disparities;

-   In 2001, AHRQ invested funds to evaluate health system responses to
    domestic violence. These studies are the first of their kind that
    will move beyond studying prevalence, screening, and training to
    take a rigorous look at health care interventions for domestic
    violence and their effectiveness. Victims will be evaluated over
    time to identify interventions that improve their health and safety,
    predict and improve health care use, prevent and reduce the
    occurrence of domestic violence, and develop better techniques to
    identify those at risk for domestic violence;

-   In 2001, AHRQ supported child-relevant studies focused on outcomes,
    quality and patient safety, cost use and access;

-   Concerning bioterrorism, this past year AHRQ funded, for example,
    research at the University of Alabama at Birmingham and the Research
    Triangle Institute that yielded the development of Web-based
    training modules to teach health professionals how to address varied
    biological agents. Separate modules were created for emergency room
    practitioners, radiologists, pathologists, and infection control
    specialists. Clinicians can obtain continuing medical education
    (CME) credit via:
    [http://www.bioterrorism.uab.edu](http://www.bioterrorism.uab.edu/);

# Based on the promising results from a pilot project, in 2001 AHRQ initiated a contract to continue funding the collaborative HIV Research Network project through 2005. Also, using AHRQ's Comprehensive Health Enhancement Support System (CHESS), an online services model for people facing major life crises, the University of Wisconsin at Madison showed that HIV-positive patients who are provided with CHESS software on home-based computer systems are more efficient users of health care. Having access to information, decision support, and connections to experts and other patients enabled 465 study patients to better monitor their health and alert their doctors when signs of serious illness appeared. These patients had lower health care costs, fewer hospitalizations, and shorter hospital stays than patients without access to CHESS. Treatment costs in one study were reduced by \$400 per month and patients spent 15 percent less time in the doctor's office because their self-management improved.

-   The U.S. Preventive Services Task Force (USPSTF) this past year
    issued four recommendations covering chlamydia screening, lipid
    (high cholesterol) screening, skin cancer, and bacterial vaginosis
    in pregnancy. These are the first of 12 recommendations the Third
    USPSTF will release. The remainder are: chemoprevention (for
    example, tamoxifen and related drugs) to prevent breast cancer (new
    topic); vitamin supplementation to prevent cancer or coronary heart
    disease (vitamin E, folate, beta carotene, and vitamin C) (new
    topic); developmental screening in children (new topic); screening
    for diabetes mellitus (updated topic); newborn hearing screening
    (updated topic); counseling to prevent unintended pregnancy (updated
    topic); postmenopausal hormone therapy (updated topic); and,
    screening for depression (updated topic).

-   As of 2001 year end, work on the National Quality Report (NQR)
    continues. A 14-member Institute of Medicine (IOM) committee of
    leading experts in quality and quality measurement was formed and
    recommended that the NQR quality monitoring system be based on a
    conceptual framework that addresses both dimensions of patient care
    (e.g., safety, effectiveness, patient centeredness, timeliness,
    equity) and patient needs (e.g., staying healthy, getting better,
    living with illness, coping with end of life issues). AHRQ initiated
    a call for measures to relevant federal agencies to identify
    candidate measures to populate the conceptual framework. The IOM
    issued a complementary call for measures to the private sector. Over
    400 measures were submitted. An interagency workgroup is evaluating
    the candidate measures for inclusion in the first report using
    criteria suggested by the IOM (e.g., significance, scientific
    soundness, feasibility).

-   Funded \$50 million in 94 new research grants, contracts and other
    projects to reduce medical errors and improve patient safety;

-   Funded over 30 projects that will examine the effect of working
    conditions on health care workers' ability to provide safe,
    high-quality care in ambulatory, inpatient (both hospital and long
    term care institutions) and in home care settings;

The above listing only begins to capture the nature of AHRQ research and
the Agency\'s role within the federal health care infrastructure. As the
only federal agency specifically charged to provide information to those
whom make decisions about health care, the research AHRQ sponsors and
conducts supports the work of several federal agencies such as: the Food
and Drug Administration (FDA); the National Institutes of Health (NIH);
the Centers for Disease Control & Prevention (CDC); the Substance Abuse
and Mental Health Services Administration (SAMHSA); the Center for
Medicaid and Medicare Services; and, others. Leaders in these agencies
and other federal and state government, private and not for profit
leaders decision-makers are AHRQ's key constituent members.

Constituent or User Input is Key

Key to the Agency\'s success in carrying out its mission is its
constituent or user-driven agenda. AHRQ regularly requests input from
its users or stakeholders through a variety of vehicles, including: the
National Advisory Council; expert meetings; Federal Register notices;
and, public comments submitted through its web site, www.ahrq.gov. This
user-driven research agenda may be thought of as a three-part research
pipeline that:

1.  Identifies and supports new research on priority health care issues;

2.  Develops the tools and talent for knowledge creation; and,

3.  Translates and disseminates research into practice.

New Research

The first pipeline segment is about health service delivery research
that address important questions and/or inadequacies in the health care
system. Further, it is about moving from the use of conventional
practices in medicine to the use of the most valid or evidence-based
scientific information available. The research is about outcomes, about
links between processes and outcome, about how to measure quality and
health expenditures.

In Fiscal Year 2001, AHRQ provided \$136 million to fund more than 401
new grants that were investigator initiated. The topics cover a range of
health services research and effectively allows AHRQ to fund important
research that may not fit within the parameters of an AHRQ-issued RFA
(requests for applications). Additionally, the Agency funded 45 small
conference grants (\$1.6 million) and 15 dissertation grants (\$500,000)
also covering an array of topics.

Developing Tools and Talent

The second segment of AHRQ\'s pipeline concerns developing instruments
and human resources to translate new knowledge into practice. Among the
tools AHRQ uses in this regard are CAHPS (Consumer Assessment of Health
Plans) and its Computerized Needs-Oriented Quality Measurement
Evaluation System (CONQUEST), the quality measures derived from the
Healthcare Cost and Utilization Project (HCUP) and Q-SPAN (Expansion of
Quality Measures) and the quality improvement tools created by the use
of Evidence Reports (ERs) and Technology Assessments (TAs). In the area
of preventive care, AHRQ provides technical support for the U.S.
Preventive Services Task Force (USPSTF), an independent panel of
preventive health experts charged with evaluating the scientific
evidence for the effectiveness of a range of clinical preventive
services.

AHRQ has partnered with a wide variety of Federal agencies, academic
institutions and health care organizations. Decision-makers use the
evidence developed through AHRQ-sponsored research and the tools
developed from those findings to help make informed decisions about what
works, for whom, when and at what cost.

Among the most visible examples of AHRQ\'s partnerships are its
Evidence-Based Practice Centers (EPCs). For the past five years, AHRQ
has been a science partner with leading public and private research
institutions nationwide in conducting scientific reviews and syntheses
of scientific literature. Each EPC has a five year contract to review
assigned specific topics in clinical care. Nominations for these topics
are routinely solicited from professional organizations, delivery
systems and others as well as accepted on an on-going basis. Among the
criteria for topic selection are whether these clinical care topics are
common, expensive, and/or significantly relevant to Medicare and
Medicaid populations.

With regard to talent, in fiscal year 2001 AHRQ created a new fellowship
program in honor of Kerr White, a founder in the filed of health
services research. Along with the National Institutes of Health, AHRQ
also expanded opportunities available under the National Research
Service Award program to include sponsorship of individual pre-doctoral
fellowships for under-represented minority students.

Translating Research

The final pipeline segment translates research into measurable
improvements in health care. New knowledge with improved tools and
talent are used to close the gap between what we know and what we do to
improve health care cost, outcomes and quality.

For example, Translating Research into Practice (TRIP) grants are now in
their second round. TRIP II, amounting to \$5.7 million and double the
number of the first round of TRIP grants, focuses on seven specific
areas: six found in the race and disparities initiative (infant
mortality, cancer screening and management, cardiovascular disease,
diabetes, HIV infection/AIDS and immunizations); and, pediatric asthma.
Each of these TRIP II grants requires partnerships among researchers,
health care systems and organizations to evaluate strategies for
improving quality of care. The TRIP program will result in more rapid
uptake of research results by providing an incentive for health care
organizations to evaluate alternative improvement strategies.

To complement this initiative and further foster partnerships in Fiscal
Year 2001, AHRQ awarded 18 planning grants equally \$1.3 million. This
funding supports the efforts of the PBRNs (Practice-Based Research
Networks) to design systems that will facilitate the translation of
research into practice and to assess the impact of these systems on care
delivered.

Additionally, AHRQ has partnered with nine Integrated Delivery System
Networks (IDSRN) to link the nation\'s top researchers with the some of
the country\'s largest health care systems. As a group, the networks
provide health services to over 34 million Americans, including the
privately insured, Medicare and Medicaid patients and the uninsured.
This new model of field-based research will enable AHRQ to accelerate
the pace of its research regarding key concerns such as medical care
quality and safety, access to services and costs.

Finally, AHRQ oversees the Put Prevention Into Practice (PIPP)
initiative, which serves as the implementation vehicle for USPSTF\'s
age-specific and risk factor-specific recommendations on preventive
care.

Evaluations

To understand AHRQ's impact on health service delivery, additional
emphasis is being placed on evaluation activities. As a result, AHRQ was
able to report on both process, output and interim outcome goals for its
major initiatives. The Fiscal Year 2001 evaluation portfolio included a
number of evaluations that assessed the impact of research products used
to inform customers, measure quality, and make policy decisions.
Evaluations completed this year are:

1.  Assessment of Past Agency Activities on Disabilities

2.  Background Research in Support of the Development of an
    Institutional CAHPS Survey

3.  Development and Analysis of HCUP- The Healthcare Cost and
    Utilization Project

4.  Development of a Conceptual Framework for a Healthcare Disparities
    Report

5.  Development of Point of Service Assessments for the Office of
    Management

6.  Evaluation of Data Sources Used in Research on Health Care Markets

7.  Evidence Report on the Efficacy of Behavioral Dietary Interventions
    to Reduce Cancer Risks

8.  Health Services Research Education: Assessing Customer Satisfaction
    & Program Needs

9.  Improving the Process for Tracking Research Impact

10. Study of the Per Patient Cost and Efficacy of Treatment for
    Temporomandibular Joint (TMJ) Disorders

Leadership on Health Care Quality

The Agency frequently uses the phrase, \"quality research for quality
healthcare.\" This underscores AHRQ's commitment to research on quality
measurement as well as quality improvement. In this area, over 80
AHRQ-funded investigators are now studying issues as far ranging as:

1.  Quality improvement in caring for newborns with jaundice;

2.  The comprehensiveness of prescription drug coverage as a measure of
    quality care among elderly beneficiaries with chronic health
    conditions; and,

3.  Identification of hospital-based quality improvement interventions
    that are most effective in increasing use of beta-blockers after
    acute myocardial infarction.

AHRQ has taken a lead role in the QuIC (Quality, Inter-Agency
Coordination) Task Force efforts to address medical errors and patient
safety in the U.S. Sadly, every hour, 10 Americans die in a hospital due
to avoidable errors and another 50 are disabled. As part of its efforts
to improve patient safety and reduce medical errors, the QuIC Task Force
has published Five Steps to Safer Health Care. The five steps were
distilled from an earlier AHRQ publication, \"20 Tips to Reduce Medical
Errors.\" Those evidence based recommendations provide patients with
guidance on how to improve their safety and have been widely adopted
across the government through the QuIC. As an example, the OPM (Office
of Personnel Management) has included them in this years health benefits
brochure. AHRQ also coordinated publication of the landmark report,
*Doing What Counts for Patient Safety---Federal Actions to Reduce
Medical Errors and Their Impact*, a response to the 1999 Institute of
Medicine (IOM) report, *To Err is Human*.

As the Department\'s leader on healthcare quality, AHRQ has continued
funding patient safety research. The agency in FY 2001 awarded 94 grants
(\$50 million) covering a diverse array of topics in the field.

In collaboration with the National Center for Health Statistics (NCHS)
and other agencies, AHRQ continues its foundational work for the first
annual report ever produced on U.S. healthcare quality, the National
Quality Report (NQR). When completed in FY 2003, the NQR will show how
the system is faring and where improvements will be needed.

Health Care Disparities

A similar agency effort is underway to produce a National Disparity
Report. This work, also targeted for release in 2003, will address
prevailing disparities in health care delivery as it relates to racial
and socioeconomic factors in the priority populations of rural,
inner-city, low-income and minority groups; women; children; the
elderly; and individuals with special health care needs. Life expectancy
and overall health have improved for many Americans but too many racial
and ethnic minorities still suffer disproportionately from diabetes,
cancer, and other diseases. We believe that through our research
partnerships, we can expand the magnitude of our efforts to ensure that
all Americans receive high quality and necessary health care services.

MEPS

Concerning cost and utilization, AHRQ continues to conduct the Medical
Expenditure Panel Survey (MEPS) that provides up-to-date, highly
detailed information on how Americans as a whole, as well as different
segments of the population, use and pay for health care.

MEPS data has been used to inform policy decisions in numerous public
and private sector agencies. For example, MEPS data was used to
establish baseline measures for the Healthy People 2010 objective on
oral health and preventive dental visits; to compare estimates of prices
paid for drugs by elderly and nonelderly persons with and without health
insurance for prescribed medications; to validate actuarial models; to
help create a profile of the population living with chronic illness; to
estimate national health expenditure rates for the elderly; and, to
inform estimates of out-of-pocket expenditures by individuals not
covered by the government or their own insurance policies. Customer
satisfaction in using MEPS has been rated high (90%) and feedback from
recipients of MEPS data indicated that the data were timely and useful.

More generally, the data currently collected from MEPS will support
quality of health care research directed to the following broad areas:
access to care; patient/customer satisfaction; health insurance
coverage; health status; and, health services utilization and
expenditures. The planned MEPS enhancements call for a significant
household survey sample expansion of individuals with certain illnesses
of national interest in terms of patient satisfaction with care
received, the quality of the care and the burden of disease. The intent
of this planned enhancement is to permit more focused analyses of the
quality of care received for these special populations. It was
recommended that the following medical conditions be given special
attention for implementing MEPS healthcare quality enhancements:
diabetes; asthma; hypertension; stroke; ischemic heart disease;
arthritis; and, COPD (chronic obstructive pulmonary disease). Diabetes
and ischemic heart were identified because these diseases are also
priorities of the national health disparities research.

> **DEPARTMENT**
>
> **OF HEALTH**
>
> **AND HUMAN**
>
> **SERVICES**

**Agency for Health Care Research and Quality**

FISCAL YEAR **2002** REVISED FINAL PLAN

FISCAL YEAR **2001** PERFORMANCE REPORT

  -----------------------------------------------------------------------
  **2.1 Budget line 1 \-- Research on Health Care Costs, Quality, and
  Outcomes**

  -----------------------------------------------------------------------

Funding Levels: FY 2001 \$226,385,000 (Actual -- Current Law)

FY 2001 \$227,897,000 (Actual -- Proposed Law)

FY 2002 \$247,645,000 (Appropriation)

FY 2002 \$249,171,000 (Current Estimate)

FY 2003 \$194,000,000 (Request -- Current Law)

FY 2003 \$195,611,000 (Request -- Proposed Law)

This budget line represents the bulk of the Agency's research
(extramural and intramural) portfolio. Dissemination and evaluation
activities are also included. The first five of the annual performance
plan's six goals are used to track Agency performance in these areas.

+-----------------------------------------------------------------------+
| **GPRA GOAL 1: Establish Future Research Needs Based on User's        |
| Needs.**                                                              |
|                                                                       |
| > **(HCQO)**                                                          |
+-----------------------------------------------------------------------+

# Strategy: Cycle of Research Phase 1:Needs Assessment

In the field of health services research, the user of the information
plays a critical role. If health services research is to improve the
quality of health care, it must provide answers to the questions and
issues that represent the barriers to improvement. AHRQ emphasizes open
communication with users of its research to ensure that it is addressing
important questions. Through continued emphasis on the first phase of
the cycle of research, needs assessment, AHRQ will continue to assure
that the Agency's research *begins and ends with the user*.

**Types of Indicators: Output**

**Use of Results** **by AHRQ:**

Input received on specific topic areas and health care issues are used
in the Agency's

program and budget development activities. The result is research
agendas and

program initiatives that are informed by the real needs of the user
community.

**Data Issues:**

To provide context for reviewing the advice received from users, AHRQ
reviews major articles in the research literature pertaining to a
particular subject area. This allows the Agency to assess where the user
input fits into the current body of research and how best to proceed.
Through a 1999 study conducted by the Lewin Group, we learned that user
input loses much of its critical meaning when aggregated. The Agency
has, therefore, created a data management system that will, in its final
stage, electronically store the source document and have word search
capability so that staff can identify relevant documents and access them
efficiently from their desk top computers when performing program and
budget development activities. Additionally, to ensure that the input
from users is incorporated into Agency activities, a number of check
points have been integrated into the planning processes where user input
is explicitly identified and assessed in relation to the proposed
activities.

**Objective 1.1:** Define direction of FY project funding priorities, in
large part, by

> needs assessment activities.

**1^st^ Indicator:** Agency research agenda covering strategic goal
areas for FY

2001 priorities (patient safety and informatics) is documented based on
consultations with various groups.

**Results:** A Patient Safety Reporting Summit with relevant

public and private sector stakeholders was held on April 23-24, 2001 in
Restin, VA. Secretary Thompson opened the summit, with over 300
attendees.

AHRQ sponsored IT expert meetings at the spring and fall

American Medical Informatics Association (AMIA) conferences on
electronic medical records and a expert meeting on child health and
information technology. AHRQ participated in IT meetings at Ambulatory
Pediatrics Association and American Medical Association (e-health)
annual meetings. In addition the agency contracted Health Strategies
Consultancy for an assessment of AHRQ's role in IT within DHHS.

  -----------------------------------------------------------------------
  **GPRA Goal 1 -- FY 2002 Indicators**

  -----------------------------------------------------------------------

+-------------------------+--------------------------------------------+
| #                       | #                                          |
|                         |                                            |
| # Goal 1 Objectives     | # FY 2002 Indicator                        |
+-------------------------+--------------------------------------------+
| **Objective 1.1:**      | Agency research agenda covering strategic  |
| Define direction of FY  | goal areas for FY 2002 priorities          |
| project funding         | (investigator-initiated research, national |
| priorities, in large    | quality report, national disparities       |
| part, by needs          | report) is documented by July 2002 based   |
| assessment activities.  | on consultations with various groups.      |
+-------------------------+--------------------------------------------+

+-----------------------------------------------------------------------+
| **GPRA GOAL 2: Make significant contributions to the effective**      |
|                                                                       |
| **Functioning of the U.S. health care system through the creation of  |
| new knowledge. (HCQO)**                                               |
+-----------------------------------------------------------------------+

**Strategy: Cycle of Research Phase 2: Knowledge**

There are many gaps in knowledge in all areas of health care. New
questions emerge as new technologies are developed, the population's
demographics change, areas of inquiry previously under-emphasized take
on greater importance, and research previously undertaken identifies
further areas that need attention. Therefore, AHRQ will continue to
focus on creating new knowledge through its peer reviewed extramural and
intramural research and assessing the findings that result from
completed projects.

**Type of Indicators:** Process and Output

**Use of Results by AHRQ**: AHRQ uses three approaches to illustrate how
it addresses its core activity of creating new knowledge. First, the
Agency documents science advances that have resulted from its investment
of funds, staff, and other resources. This enables the Agency to do the
following:

-   identify and highlight significant research findings from research
    funded or sponsored by AHRQ;

-   focus its translation and dissemination activities to maximize the
    potential use of critical findings in the health care system;

-   annually assess progress toward filling the gap between what we know
    and what we need to know about health care.

Second, AHRQ documents coverage in major journals and/or evidence of use
of research findings. Coverage by popular and professional media is
highly competitive. AHRQ's receiving coverage is an initial indication
that its investment in research has the potential for significant impact
when disseminated and implemented widely. The actual use of the findings
by purchasers, professional associations, managed care organizations,
and/or insurers also signals that the new knowledge has the potential to
make a difference. The ultimate outcome or impact will be evaluated
after the finding has been implemented over a period of time.

Third, the statistics on the number of grants funded and dollars
invested in particular areas are used to determine whether the AHRQ
portfolio has a significant body of work underway to begin to inform the
field. They are also used in gauging the investment in these areas vs.
other programs as AHRQ allocates its resources.

**Data Issues:**

Project officers in consultation with grantees largely do collecting
data and/or anecdotes on the use of research results or tools through
searches of the literature, media outlets, and Internet listings and
tracking. The information is captured through regular communications
with partners, researchers, associations, and Federal, State, and local
governments. Anecdotal information is used only when it can be verified
with the actual user. Documentation of the use is sought whenever
possible. AHRQ continues to look for ways to introduce efficiencies in
this labor-intensive effort.

  -----------------------------------------------------------------------
  **GPRA Goal 2 -- FY 2001 Results**

  -----------------------------------------------------------------------

Objective 2.1: Determine annually the salient findings from research in
each

of the three areas (outcomes; quality; and cost, access, and use) and
develop plan for next steps translation and dissemination.

**1^st^ Indicator:** Produce an annual report that describes at least 12
science advances covering the three research goal areas (outcomes;
quality; cost, access, and use). For each finding, specific steps in
translation and dissemination are identified and initiated.

**Results:** AHRQ has identified at least 12 science advances. These
findings are highlighted in the AHRQ Accountability Report which is
being submitted to Congress under separate cover. The following list
exemplifies the types of findings contained in this report as well as
how the findings were disseminated:

> **ROUTINE MEDICAL TESTING PRIOR TO CATARACT SURGERY (Effectiveness):
> Cataract PORT II**
>
> Finding: Routine medical testing prior to cataract surgery doesn't
> improve outcomes and in most cases is unnecessary.
>
> **EDUCATIONAL PROGRAM FOR NURSING HOME PHYSICIANS AND STAFF TO REDUCE
> USE OF NON-STEROIDAL ANTI-INFLAMMATORY DRUGS AMONG NURSING HOME
> RESIDENTS: A CONTROLLED TRIAL**
>
> Finding: The educational program and algorithm described in this study
> could help reduce the overuse of NSAIDs in elderly nursing home
> patients. May 2001 *Medical Care*
>
> **A QUALITATIVE STUDY OF INCREASING BETA-BLOCKER USE AFTER MYOCARDIAL
> INFARCTION (Quality)**
>
> Finding: A newly tested classification system may help hospitals to
> identify areas of improvement and then to make successful quality care
> improvements. May 23, 2001 *Journal of the American Medical
> Association*
>
> **A Randomized Clinical Trial of Outpatient Geriatric Evaluation and
> Management**
>
> Finding: Targeting outpatient geriatric evaluation and management
> (GEM) slows a patient's functional decline in his or her daily
> activities.
>
> April 2001 *Journal of the Am. Geriatrics Society*
>
> **MANAGEMENT OF DENTAL PATIENTS WHO ARE HIV POSITIVE**
>
> Finding: The antifungal medication, fluconazole, may help prevent the
> development of thrush in the mouths of HIV-infected patients. *August
> 2001 Oral Surgery, Oral Medicine, Oral Pathology.*

**2^nd^ Indicator:** Generate 2 - 3 synthesis reports on research
findings and

practical applications on Agency priority topics.

**Results:** AHRQ has, to date, completed two synthesis reports
highlighting research findings related to Patient Safety and End of Life
Care for patients. In addition a Fact Sheet on Diabetes was developed
for use by clinicians and their patients. The research synthesis
entitled \"Reducing and Preventing Adverse Drug Events To Decrease
Hospital Costs\" can be found at the following URL
<http://www.ahrq.gov/qual/aderia/aderia.htm>. 

**Objective 2:2:** Achieve significant findings from AHRQ sponsored and
conducted

research.

**1^st^ Indicator:** Findings from at least 40 AHRQ sponsored or
conducted research

> are used by public and private partners to improve health care.

**Results:** Numerous examples of AHRQ sponsored and conducted research
that has either been published in the peer review literature or that
illustrates how findings are used by public and private partners to
improve health care can be found in AHRQ's electronic publication
"Research Activities". This publication can be accessed using the
following URL: <http://www.ahrq.gov/>.

The following is a brief list of articles and projects that can be found
in this publication:

Ayanian J. (2001) Increased mortality among middle-aged women after
myocardial infarction: Searching for mechanisms and solutions. Annals of
Internal Medicine. 134:239-241.

Basu J. (2001) Access to primary care: The role of race and income. A
study of municipal health services clinics. Journal of Health and Social
Policy. 13(4):57-73.

Burstin H, Lewin, D and Hubbard, H. (2001) Future directions in primary
care research: Special issues for nurses Policy, Politics, and Nursing
Practice. 103:103-107.

Coppola K, Ditto P, Danks J, and Smucker W. (2001) Accuracy of primary
care and hospital-based physician\'s prediction of elderly outpatient\'s
treatment preferences with and without advance directives. Archives of
Internal Medicine. 161:431-440.

> Ditto P, Danks J, Smucker W, and others. (2001) Advance Directives as
> acts of communication. Archives of Internal Medicine. 161:421-430.
>
> Fernandez A, Grumbach K, Vranizan K, and others. (2001) \'Primary care
> physicians\' experience with disease management programs. Journal of
> General Internal Medicine. 16:163-167.
>
> Fink K, Baldwin L, Lawson H and others. (2001) The role of
> gynecologists in providing primary care to elderly women. Journal of
> Family Practice. 50(2):153-158.
>
> Franks P. (2001) Impact of patient socioeconomic status on physician
> profiles: A comparison of census-derived and individual measures.
> Medical Care. 39(1): 8-14.
>
> Frazier L, Colditz G, Fuchs C and Kuntz K. (2000-Oct)
> Cost-effectiveness of screening for colorectal cancer in the general
> population. Journal of the American Medical Association. 284(15):
> 1954-1961.
>
> Galsgow R and Bull S. (2001) Making a difference with interactive
> technology: Consideration in using and evaluating computerized aids
> for diabetes self management education. Diabetes Spectrum. 14(2):
> 99-106.
>
> Kiefe K. (2001). Improving Quality Improvement Using Achievable
> Benchmarks Of Care Feedback. Journal of the American Medical
> Association. June 13, 2001
>
> Malkin J, Broder M and Keeler E. (2000-Oct) Do longer postpartum stays
> reduce newborn readmissions? Analysis using instrumental variables.
> Health Services Research. 35(5 pt 2):1071-91.
>
> Murphy J, Chaing H, Montgomery J, Rogers W and Safran D. (2001) The
> quality of physician-patient relationships. Journal of Family
> Practice. 50(2): 123-129.
>
> Pathman DE. (2000). State Scholarship, Loan Forgiveness, And Related
> Programs: The Unheralded Safety Net. Journal of the American Medical
> Association. October 25 2000
>
> Rollman B, Gilbert T and others. (2001). The electronic medical
> record. Archives of Internal Medicine. 161:189-197. (R01 HS09421)
>
> Safran D, Montgomery J, Chang H, Murphy J and Rogers W. (2001).
> Switching doctors: Predictors of voluntary disenrollment from a
> primary physician\'s practice. The Journal of Family Practice. 50(2):
> 130-135
>
> Siminoff LA. (2001). Factors Influencing Families Consent To Donation
> Of Solid Organs For Transplantation. Journal of the American Medical
> Association. July 4, 2001

In addition, AHRQ has issued press releases describing the findings and
conclusions of 45 studies that were funded or sponsored by AHRQ. Media
coverage of these studies includes at least 996 newspaper and magazine
articles and over 500 online news reports and television and radio
stories. Coverage ranged from 225 newspaper articles, 175 television and
radio reports and numerous online news stories for a study on ear
surgery to 10 key health care trade press reports for a study of
declining health insurance enrollment.

**Objective 2.3:** Initiate FY Research Initiatives.

> Funding of a minimum of 60 projects in the following areas:

-   Reducing Medical errors and Enhancing Patient Safety

-   Using Computers and Information Technology to Prevent Medical Errors

-   Working Conditions

**Results:** AHRQ has invested \$50 million to fund 94 new research
grants, contracts and other projects to reduce medical errors and
improve patient safety. This initiative represents the federal
government's largest single investment to address the estimated 44,000
to 98,000 patient deaths related to medical errors each year. The 94
projects will be carried out at state agencies, major universities,
hospitals, outpatient clinics, nursing homes, physicians' offices,
professional societies and other organizations across the country. These
projects will address key unanswered questions about how errors occur
and provide science-based information on what patients, clinicians,
hospital leaders, policymakers and others can do to make the health
system safer. The results of this research will identify improvement
strategies that work in hospitals, doctors' offices, nursing homes and
other health care settings across the nation.

This \$50 million research initiative is the first phase of a multi-year
effort. Many institutions will receive additional funds to continue
their work in future years. These projects reflect the input of
consumers, health care providers and policymakers from a national
research summit last year led by AHRQ and its partners on the Quality
Interagency Coordination (QuIC) Task Force. These projects fall into the
following six major categories of awards:

**Supporting Demonstration Projects to Report Medical Errors Data:**
These activities include 24 projects for \$24.7 million to study
different methods of collecting data on errors or analyzing data that
are already collected to identify factors that put patients at risk of
medical errors.

**Using Computers and Information Technology to Prevent Medical
Errors:** These activities include 22 projects for \$5.3 million to
develop and test the use of computers and information technology to
reduce medical errors, improve patient safety, and improve quality of
care.

**Understanding the Impact of Working Conditions on Patient Safety:**
These activities include eight projects for \$3 million to examine how
staffing, fatigue, stress, sleep deprivation, and other factors can lead
to errors. These issues---which have been studied extensively in
aviation, manufacturing and other industries---have not been closely
studied in health care settings.

**Developing Innovative Approaches to Improving Patient Safety:** These
activities include 23 projects for \$8 million to research and develop
innovative approaches to improving patient safety at health care
facilities and organizations in geographically diverse locations across
the country.

**Disseminating Research Results:** These activities include seven
projects for \$2.4 million to help educate clinicians and others about
the results of patient safety research. This work will help develop,
demonstrate and evaluate new approaches to improving provider education
in order to reduce errors, such as applying new knowledge on patient
safety to curricula development, continuing education, simulation
models, and other provider training strategies.

**Additional Patient Safety Research Initiatives:** AHRQ will use the
remaining \$6.4 million for 10 other projects covering other patient
safety research activities, including supporting meetings of state and
local officials to advance local patient safety initiatives and
assessing the feasibility of implementing a patient safety improvement
corps.

A brief list of specific projects funded can be found in the appendix of
this report.

+-----------------------------------------------------------------------+
| ###### GPRA Goal 2 -- FY 2002 Indicators                              |
+-----------------------------------------------------------------------+

+-----------------------+----------------------------------------------+
| **Goal 2 Objectives** | **FY 2002 Indicator**                        |
+-----------------------+----------------------------------------------+
| **Objective 2.1:**    | Produce an annual report that reports on at  |
| Determine annually    | least 18 science advances covering the three |
| the salient findings  | research goal areas (outcomes; quality;      |
| from research in each | cost, access, and use).                      |
| of the three areas    |                                              |
| (outcomes; quality;   | For each finding, specific steps in          |
| and cost, access, and | translation and dissemination are identified |
| use) and develop plan | and initiated.                               |
| for next steps        |                                              |
| translation and       | Increase by 15% (relative to FY01 baseline)  |
| dissemination.        | the number of synthesis reports generated on |
|                       | research findings and practical applications |
|                       | on Agency priority topics.                   |
|                       |                                              |
|                       | ####                                         |
|                       |                                              |
|                       | #### Patient Safety                          |
|                       |                                              |
|                       | Establish baseline for number of science     |
|                       | advances in patient safety covering the      |
|                       | three research goal areas (outcomes;         |
|                       | quality; and cost, access, and use) that     |
|                       | will be included in future accountability    |
|                       | reports to the Congress.                     |
|                       |                                              |
|                       | Generate at least one synthesis report on    |
|                       | research findings and practical application  |
|                       | in the area of patient safety                |
+-----------------------+----------------------------------------------+
| **Objective 2:2:**    | Discontinued                                 |
| Achieve significant   |                                              |
| findings from AHRQ    | (This measure is combined with 3.1 in FY     |
| sponsored and         | 2002.)                                       |
| conducted research.   |                                              |
|                       |                                              |
| (This objective is    |                                              |
| combined with         |                                              |
| Objective 3.1 in FY   |                                              |
| 2002)                 |                                              |
+-----------------------+----------------------------------------------+
| **Objective 2.3:**    | Fund or conduct a minimum of 50 projects in  |
| Initiate FY Research  | the following areas:                         |
| Initiatives.          |                                              |
|                       | -   Patient Safety                           |
|                       |                                              |
|                       | -   Translating Research into practice       |
|                       |                                              |
|                       | -   Healthcare Cost and Utilization Project  |
|                       |                                              |
|                       | -   Medical Expenditure Panel                |
+-----------------------+----------------------------------------------+

+-----------------------------------------------------------------------+
| **GPRA GOAL 3: Foster translation and dissemination of new            |
| knowledge**                                                           |
|                                                                       |
| into practice by developing and providing information, products, and  |
| tools on outcomes; quality; and access, use, and cost of care. (HCQO) |
+-----------------------------------------------------------------------+

# Strategy: Cycle of Research Phase 3: Translation and Dissemination

AHRQ is committed to ensuring that the knowledge gained through health
care research is translated into measurable improvements in the American
health system. AHRQ is focusing on closing the gap between what we know
and what we do. Under the "Translating Research Into Practice"
initiative, the Agency invests in demonstration projects, public
(Federal, state, and local government) and private-sector partnerships,
and targeted dissemination activities to develop and test implementation
strategies in different settings in the health care system and
demonstrate their applicability to widespread dissemination in other
areas of the system.

**Types of Indicators:** Output and process.

**Use of Results by AHRQ:**

The indicators regarding number of partnerships, attendees at User
Liaison Program meetings, or hits on the AHRQ web site help the Agency
determine whether what it produces is of use to major audience segments.
The Agency evaluates the results of the GPRA plan indicators in
combination with other information such as details about what products
were released, feedback from attendees at programs, where the hits are
on the web site, and feedback from customers to manage and improve its
dissemination efforts.

Frequently, the results of research are not readily implemented in the
health care system without an interim step such as the creation of a
tool that facilitates use. A major focus for the Goal 3 indicators,
therefore, is to look at the creation and use of tools. The indicators
for the Agency's investment in training helps the Agency track its
success in furthering the field of health services research by fostering
new talent. The numbers of trainees funded are a reflection of Agency
commitment and the success of the training programs in attracting
successful candidates. This data can be used in combination with other
information about individual trainees, their research projects,
professional credentials of professors and mentors, etc. to assess the
overall success of the program.

**Data Issues:**

The AHRQ has implemented several computer-based reporting tools to
monitor usage of Agency information systems and websites. Accurate
statistics are recorded on the usage of the National Guideline
Clearinghouse, Publications Clearinghouse, and various other Agency
websites and systems using commercially available reliable and accurate
tools, e.g., WebTrends. These tools are used by many corporations and
government agencies nationwide to monitor usage and have been certified
by various information technology testing and review groups. Information
on all grants, which can be word searched, is included in the Agency
Management Information System.

AHRQ tracks print media (newspapers, health care-related trade journals
and newsletters, and consumer magazines) and on-line news services for
stories about or involving the Agency through a contractor, Burrelle\'s
\-- one of the Nation\'s largest and oldest news clipping services. The
contractor, Video Monitoring Service, monitors TV and radio news reports
on selected studies in major markets around the United States.

Agency program staff maintains other statistics during the normal
monitoring of contracts and grants. Certain items, such as the release
of a CONQUEST product, are documented on the AHRQ web site when ready so
that consumers are aware of the availability. Other items such as the
statistics on the ULP program are monitored through the management of
the support contracts, travel arrangements, and other records kept in
administering the program. Anecdotal information is verified with the
primary source before being used by the Agency in this report or for any
other uses.

  -----------------------------------------------------------------------
  **GPRA Goal 3 -- FY 2001 Results**

  -----------------------------------------------------------------------

**Objective 3.1:** Achieve significant findings from AHRQ sponsored and

> Conducted research and maximize dissemination of information, tools,
> and products developed from research results for use in practice
> settings.

**1^st^ Indicator:** At least 5 public-private partnerships are formed
to implement

research findings for decision makers.

**Results:** The following examples of public-private partnerships

-   A meeting was held in June 2001 to discuss opportunities for
    collaboration and to attempt to match up select TRIP-IIs and PROs.
    Participants included staff from both AHRQ and CMS (HCFA). The focus
    of the meeting was to explore how AHRQ could be more strategic in
    working with PROs to implement findings from TRIP-II as well as the
    broader array of TRIP related projects. Three potential areas for
    closer interaction are: 1) improving our understanding of the
    effectiveness of implementation interventions; 2) involving PROs in
    research; and, 3) developing a research agenda for implementation
    research. Specific plans were made to hold a meeting hosted by AHRQ
    and CMS to focus on defining what information the PROs and their
    customers need in their quality improvement activities and setting a
    research agenda.

```{=html}
<!-- -->
```
-   In cooperation with the Kantor Family Foundation, AHRQ developed a
    task order to clarify important issues for patients and clinicians
    relevant to developing a national outcomes database to inform
    clinical decisions about treatment alternatives based on knowing
    what happens to "People Like Me".

-   AHRQ worked with Blue Cross and Blue Shield of Minnesota to develop
    a provider resources kit, based on the Public Health Service (PHS)
    Guideline, *Treating Tobacco Use and Dependence,* which is currently
    being focus tested with several physicians within the plan's
    network. This resources kit contains a number of smoking cessation
    materials to aid physicians in treating patients who smoke and will
    be distributed to the entire network of physicians in Minnesota.

```{=html}
<!-- -->
```
-   Identified and implemented two specific opportunities for promoting
    research that uses CEA to inform decisions.

**2^nd^ Indicator:** Formation of a minimum of 10 partnerships to
support

> dissemination of AHRQ products through intermediary organizations,
> such as health plans and professional organizations. AHRQ has
> completed seven public-private partnerships. The partners' activities
> include, but are not limited, to reprinting and disseminating the
> information to their constituencies, web linking, and the electronic
> dissemination of AHRQ materials.

**Results:** AHRQ continues to be successful in forming partnerships
with intermediary organizations that support dissemination of AHRQ
products. To date, AHRQ has completed seven public-private partnerships.
The partners' activities include, but are not limited, to reprinting and
disseminating the information to their constituencies, web linking, and
the electronic dissemination of AHRQ materials. Examples of these
partnerships include:

-   American Academy of Family Physicians

```{=html}
<!-- -->
```
-   American Medical Informatics Association

```{=html}
<!-- -->
```
-   Society of Academic Emergency Medicine

-   American Association of Health Plans

-   Merck's Institute for Aging Research

-   Department of Veterans Affairs Health Services Research and
    Development QUERI program

-   Centers for Medicare and Medicaid

AHRQ's successful collaboration with these partners resulted in the
following activities:

-   AHRQ, through collaborative efforts with the agency's National
    Advisory Council (NAC), nursing organizations, AHRQ nurses and
    others, is establishing outreach paths to the nursing research
    community. The hope is to facilitate dissemination of AHRQ products
    through all segments of the health care arena, and, at the same
    time, to enhance nursing research at AHRQ.

-   In March 2001, AHRQ sponsored a meeting that included nurses

> from AHRQ, NAC, the American Academy of Nurses, and the Council on
> Advancement of Nursing Sciences. At this meeting two lists were
> developed for use by the review staff at AHRQ and members of peer
> review groups. One list was of nursing journals to be used as
> peer-reviewed journals when referencing nursing research. The other
> list was of nurses that can be recruited to serve as peer reviewers
> for AHRQ-sponsored research proposals. In addition, a series of steps
> were developed to promote research opportunities for nurses. Following
> the meeting, in May 2001, as further encouragement for nursing
> research at AHRQ, an article was written by AHRQ staff and published
> in *Policy, Politics, and Nursing Practice* entitled, "Future
> Directions in Primary Care Research: Special Issues for Nurses". A
> follow-up meeting is planned for FY02.

-   AHRQ partnered with the American Association of Health Plans to
    convene a conference, "Building Bridges VII: Assessing Policy
    Decisions and Their Impact on Health Care Delivery," April
    25-27, 2001. Other partners included CDC, BlueCross BlueShield
    Association, and the National Institute on Disability and
    Rehabilitation Research. AHRQ disseminated information about how
    health plan decision makers and researchers could use both
    Healthcare Cost and Utilization Project (HCUP) data and Consumer
    Assessment of Health Plans (CAHPS) data for comparative analyses,
    forecasting, and benchmarking within managed care settings.

-   AHRQ worked with the Tobacco-Free Coalition of Oregon to develop a
    smoking cessation tools kit, based on the PHS Guideline, *Treating
    Tobacco Use and Dependence,* which will be distributed in bulk
    quantities to all health plans in Oregon.

-   AHRQ worked with the National Educational Association (NEA) to place
    a tobacco ad in their newsletter and promote the entire product line
    in the March issue of \"NEA Today.\" The NEA also distributed a
    flier, which lists the entire PHS smoking cessation product line, to
    NEA staff.

-   AHRQ worked with the American Cancer Society to distribute 550
    copies of the *You Can Quit Smoking* consumer guides to 11 GIANT
    Food Store Pharmacies on the East Coast in time for the Great
    American Smokeout.

-   AHRQ collaborated with Smoke Free Families to develop an
    intervention piece for the pregnant smoker, based on the PHS
    Guideline *Treating Tobacco Use and Dependence.* This piece will be
    distributed to OB/GYNs and will be promoted in talks with other
    groups interested in addressing pregnant smokers.

-   AHRQ worked with the Kansas State Employees Health Care Commission
    to distribute the following AHRQ publications: *Staying Healthy at
    50+, Child Health Guide, Personal Health Guide,* and *Your Guide to
    Choosing Quality Health Care* to all Kansas state employees.

-   AHRQ worked with the American Association of Health Plans (AAHP) to
    facilitate dissemination of *Staying Healthy at 50+* to the HMO
    trade group's 1,000 member health plans by supplying them with
    sample copies of the publication and by announcing the availability
    of the publication in its member newsletters.

-   AHRQ worked with the Connecticut Department of Social Services,
    Elderly Services Division, to distribute the following PPIP
    publications at various seminars and presentations for area care
    facilities: *Staying Healthy at 50+* (English and Spanish), Personal
    *Health Guide* (English and Spanish) and the *Adult Preventive Care
    Timeline*.

-   AHRQ worked with Health First Health Plans in Florida to distribute
    the English-language versions of *Staying Healthy at 50+, Child
    Health Guide,* and the *Personal Health Guide* to its members. The
    publications were distributed via separate mailings to its Medicare,
    Commercial and Youth populations accompanied by a letter from the
    plans' Quality Management Director.

-   AHRQ worked with John Deere Health to distribute *Staying Healthy at
    50+* to John Deere Health's senior Risk enrollees in the
    post-enrollment packets along with their benefit booklet and member
    ID card.

**3^rd^, 4^th^ and 5^th^ Indicator:** The number of hits on the Web
site, the number of documents uploaded and the number of inquiries
handled on the web site.

[]{.mark}

**Results:** AHRQ has invested significant resources in developing a
user friendly web site that allows real time access to a variety of
products produced by AHRQ. Use of AHRQ's web site continued to increase
in FY2001 with more than 22.6 million hits compared with 18.8 million
hits the previous year. User sessions also rose by almost a million
visits -- 2.4 million up form 1.5 million in FY2000. Page views
increased substantially for FY2001 at 8.9 million compared with 5.4
million the previous year. Overall workload increased with 5,068 files
and documents uploaded to the Web site, compared with 4,400 for FY2000.

In addition, AHRQ handled 4,006 electronic inquiries during FY2001
through its Web site Mailbox, up from 3,500 the previous year. These
inquiries included requests for Agency information products, funded
research, consumer health issues and concerns, technical assistance,
referrals to other resources, and requests to use AHRQ electronic
content on other Web sites or in electronic or print products.

**6^th^ Indicator:** Number of State and local governments trained in
the

> understanding and use of health services research findings through ULP
> Workshops.

**Results:** In FY 2001, ULP sponsored 23 different activities,
including 12 National workshops, 7 state-based workshops, one telephone
conference and one Web-assisted audio-conference. These activities
provided over 2300 health care policy makers from all 50 states, the
District of Columbia, American Samoa, the Virgin Islands and Guam with
access to research findings they could use to make evidence-based
decisions about health care.

**Objective 3.2:** Develop and facilitate the use of new tools, talent,
products, and

> implementation methodologies stemming from research portfolio.

**1^st^ Indicator:** Produce evidence summaries for use in Federal
direct care

providers' efforts to create guidelines.

**Results:** Nine evidence summaries that were produced in FY 01 were
used by Federal direct care providers to create guidelines including:

-   Impact of Cancer-related Decision Aids

-   Defining/Management of Chronic Fatigue Syndrome

-   Diagnosis/Management of Parkinson's Disease

-   Management of Post-Term Pregnancy

-   Evaluation/Management of Stroke, Effectiveness and Cost
    Effectiveness of Echocardiography and Carotid Ultrasound

-   Making Health Care Safer: A Critical Analysis of Patient Safety
    Practices

-   Management of Chronic Asthma

-   Otitis Media with Effusion

-   Management of Neurogenic Pain

The findings from AHRQ sponsored and conducted research has been used by
public and private partners to inform the development of guidelines or
other quality improvement tools. The following four examples illustrate
how public and private partners have used evidence developed by AHRQ to
improve the quality of care received:

-   **Treatment of Pulmonary Disease Following Spinal Chord Injury**
    -The VA Spinal Chord Injury Quality Enhancement Research initiative
    (SCI QUERI) is considering this evidence report as it develops
    quality improvement strategies for translating evidence into
    practice for patients with spinal chord injury treated within the VA
    healthcare system.

-   **Making Health Care Safer: A Critical Analysis of Patient Safety
    Practices** -This evidence report has been disseminated to members
    of the National Forum for Health Care Quality Measurement and
    Reporting (NQF), which includes a variety of stakeholders, including
    Federal direct care providers such as the Veteran's Administration
    and the Department of Defense. This report should be useful to these
    providers as well as private and public purchasers, employers,
    consumers, and accrediting bodies, in developing effective
    strategies for reducing medical errors and improving patient safety.

-   **Evaluation and Treatment of New Onset Atrial Fibrillation in the
    Elderly** -The North Carolina Peer Review/Quality Improvement
    Organization (PRO), as part of its efforts under the National Stroke
    Project sponsored by the Centers for Medicare and Medicaid Services,
    has widely disseminated copies of the AHRQ-sponsored evidence report
    on Atrial Fibrillation to physicians and hospitals that treat atrial
    fibrillation patients across the state. In addition, a physician
    opinion leader presented the findings of the report to an audience
    of 246 health care professionals during a teleconference on
    preventing strokes in patients with atrial fibrillation.
    Participants had the opportunity to discuss the finding with an
    expert panel. Rates of warfarin prescription prior to discharge of
    patients with atrial fibrillation have been measured for the state
    and will serve as a baseline for assessing improvements in practice
    over time. The PRO is also providing a data abstraction tool to
    hospitals in the state that will enable them to assess their
    performance on this measure and assist in targeting quality
    improvement efforts.

-   **US Preventive Services Task Force (USPSTF)** -The Centers for
    Medicare and Medicaid Services (CMS) is using USPSTF recommendations
    based on AHRQ-sponsored systematic evidence reviews to develop
    messages (based on focus group testing) for consumers and clinicians
    regarding Medicare-covered services for osteoporosis, cervical
    cancer, prostate cancer and breast cancer.

**2^nd^ Indicator:** EPCs will produce a minimum of 12 evidence reports
and

> technology assessments that can serve as the basis for interventions
> to enhance health outcomes and quality by improving practice.

**Results:** In FY 01, AHRQ-supported EPC's produced the following 18
Evidence Reports and 3 Technology Assessments that can serve as the
basis for interventions to enhance health outcomes and quality by
improving practice.

> **Evidence Reports**

-   Impact of Cancer-related Decision Aids (McMaster U. EPC)

-   Chronic Fatigue Syndrome (San Antonio EPC)

-   Diagnosis/Management of Parkinson's Disease (MetaWorks EPC)

-   Management of Post-Term Pregnancy (Duke EPC)

-   Methods to Rate Strength of Scientific Evidence (RTI/UNC EPC)

-   Evaluation/Management of Stroke, Effectiveness and Cost
    Effectiveness of Echocardiography and Carotid Ultrasound (OHSU EPC)

-   Utilization of Physician Services (RAND EPC)

-   Refinement of HCUP Quality Indicators (UCSF/Stanford EPC)

-   Making Health Care Safer: A Critical Analysis of Patient Safety
    Practices (UCSF/Stanford EPC)

-   Management of Chronic Asthma (BC/BS EPC)

-   Otitis Media With Effusion (RAND EPC)

-   Management of Neurogenic Pain (McMaster U. EPC)

-   Ayruvedic Interventions for Diabetes Mellitus (RAND EPC)

-   Feasibility Study: Use of Hyperbaric Oxygen Therapy for Stroke and
    Brain Injury (OHSU EPC)

-   Management of Allergic Rhinitis (NEMC)

-   Diagnosis and treatment of Repetitive Motion Disorders (ECRI)

-   Bioterrorism, Training for Rare Public Health Events (JHU)

> **Technology Assessments**

-   Actinic Keratosis (OHSU EPC)

-   PET for Breast Cancer (BC/BS EPC)

-   Pneumatic Compression for Venous Insufficiency (intramural)

Note: Summaries of Evidence Reports, Final Reports and Technical
Supplements can be found at the following ULP:
[www.ahrq.gov](http://www.ahrq.gov/)

**3^rd^ Indicator:** Support a minimum of 165 pre- and post-doctoral
trainees.

**Results:** AHRQ provided support for approximately 176 pre and
postdoctoral students through a variety of mechanisms including
individual pre-doctoral fellowships

(4 students), individual postdoctoral fellowships (7 students),
dissertation grants

(15 students) and institutional training program grants (75 pre-doctoral
students and 75 postdoctoral students).

**4^th^ Indicator:** Support up to 3 Minority Research Infrastructure
Support Program

(M-RISP) grants in order to develop the health services research
capabilities of traditionally minority-serving institutions.

**Results:** Three M-RISP projects will be supported, reflecting a
multidisciplinary blend of clinical and nonclinical perspectives and
partnerships. Support will be provided for the development of health
services research infrastructure in these institutions. Awardees include
the University of Hawaii, Tennessee State/Meharry Universities, and the
University of Texas at Galveston, which respectively serve large numbers
of Asian/Pacific Islanders, African Americans, and Hispanic students.
Research foci of funded centers will include preventative medicine,
health disparities, quality of care, and medical effectiveness.

**5^th^ Indicator:** Support up to 6 Building Research Infrastructure
and Capacity

> (BRIC) two-year planning grants in EPSCOR states and states which
> historically have received little or no research support from AHRQ.

**Results:** AHRQ will provide support for planning associated with the
development of research infrastructures in four institutions located in
states that historically have not received significant health services
research funding. Recipients will include institutions located in states
such as Kentucky, Louisiana, Utah, and a consortium of the intermountain
states of Nevada, Montana, Idaho and Wyoming. These projects emphasize a
variety of health services research interests that draw upon
multidisciplinary perspectives to address critical issues pertaining to
child health, primary care, health care quality, and rural and frontier
health care.

**6^th^ Indicator:** Fund at least 10 projects in tool development.

**Results:** AHRQ, through intramural and extramural research, has
funded more than 10 projects in the development of new tools and
facilitated their use in diverse health care delivery systems. The
following are selected examples:

-   Development of a series of Web-based, user-friendly quality
    indicators for improving the quality of inpatient and ambulatory
    health care to include: (1) AHRQ's prevention/access indicators,
    and (2) Mortality/utilization indicators. These indicators resulted
    from a refinement and expansion of the original HCUP Quality
    Indicators.

```{=html}
<!-- -->
```
-   Development of clinical classifications software (ICD-10). This tool
    includes categories that can be employed in many types of projects
    analyzing data on diagnoses. For example, they can be used to
    identify populations for disease -- or procedure -- or specific
    duties, provide statistical information about relatively specific
    conditions, and define comorbidities.

```{=html}
<!-- -->
```
-   Publication of the HCUP Fact Book. This Fact Book was produced as a
    tool to provide statistics and information on procedures performed
    in U.S. hospitals. The book provides information on the most common
    procedures, characteristics of patients who receive procedures,
    procedures associated with the longest hospital stays, the most
    expensive hospitalizations, and the highest in-hospital mortality.
    Elixhauser A, Klemstine K, Steiner C, Bierman A. Procedures in U.S.
    Hospitals, 1997. Rockville (MD): Agency for Healthcare Research and
    Quality: 2001. HCUP Fact Book No.2; AHRQ Publication NO. 01-0016ISBN
    1-58763-029-X.

```{=html}
<!-- -->
```
-   Kids' Inpatient Database (KID): Nation's First Database Devoted
    Solely to Children's Hospital Care. The KID is the only dataset on
    hospital use, outcomes, and charges for children age 28 and younger,
    including newborns, regardless of whether they were privately
    insured, received public assistance, or had no health insurance. The
    KID is the most recent tool in a family of databases, Web-based
    products, and user friendly software developed as part of HCUP.

```{=html}
<!-- -->
```
-   Release of the 1998 and 1999 State Inpatient Databases (SID -- 15
    states) and State Ambulatory Surgery Databases (SASD -- 7 states) --
    powerful health care research and policy analysis tools. The SID
    contains information for every inpatient hospitalization in each
    participating state, while the SASD captures information on
    surgeries performed at hospital-affiliated ambulatory surgery sites.
    These research resources are part of a family of databases and
    software tools developed as part of HCUP.

  -----------------------------------------------------------------------
  **GPRA Goal 3 -- FY 2002 Indicators**

  -----------------------------------------------------------------------

+--------------------+-------------------------------------------------+
| **Goal 3           | **FY 2002 Indicator**                           |
| Objectives**       |                                                 |
+--------------------+-------------------------------------------------+
| **Objective 3.1:** | Increase by 15% (relative to FY 2001 baseline)  |
| Maximize           | the number of state and local governments       |
| dissemination of   | trained and/or receiving technical assistance   |
| information,       | through User Liaison Program (ULP).             |
| tools, and         |                                                 |
| products developed | Identify at least 5 new and important on-going  |
| from research      | partnerships with public and private-sector     |
| results for use in | organizations to disseminate and implement AHRQ |
| practice settings. | sponsored or conducted research findings.       |
|                    |                                                 |
|                    | Synthesis of at least 5 grant portfolio areas   |
|                    | on quality of care across Agency's goals for    |
|                    | persons with chronic care needs produced and    |
|                    | disseminated with particular focus on outreach  |
|                    | to managed care executives.                     |
|                    |                                                 |
|                    | Patient safety research findings will be        |
|                    | disseminated to 1000 providers through the      |
|                    | Patient safety dissemination and education      |
|                    | program.                                        |
|                    |                                                 |
|                    | Augment the family of CAHPS instruments to      |
|                    | include a Group Practice Level Survey.          |
|                    |                                                 |
|                    | Increase by 15% (relative to the FY 2001        |
|                    | baseline) the number of publications of AHRQ    |
|                    | funded and conducted research.                  |
|                    |                                                 |
|                    | Establish a baseline of electronically          |
|                    | available tools and publications that bring     |
|                    | AHRQ research to our intended audiences -       |
|                    | policymakers, clinicians, State and local       |
|                    | officials, health care systems and researchers. |
+--------------------+-------------------------------------------------+
| **Objective 3.2:** | Fund or conduct at least 5 projects in tool and |
| Develop and        | data development.                               |
| facilitate the use |                                                 |
| of new tools,      | Fund or conduct at least 5 projects in tool and |
| talent, products,  | data use.                                       |
| and implementation |                                                 |
| methodologies      | Provide technical assistance to 30 new users of |
| stemming from      | CAHPS tools.                                    |
| research           |                                                 |
| portfolio.         | Obtain data for the National CAHPS Benchmarking |
|                    | Database from 10 new sponsors of CAHPS surveys. |
| (This is objective |                                                 |
| 3.3 in FY 99-00).  | HIV Research Network data will be used to       |
|                    | produce 3 analyses on disparities or quality.   |
|                    |                                                 |
|                    | ***Training Programs***                         |
|                    |                                                 |
|                    | Establish at least 1 Patient Safety             |
|                    | Investigator training programs.                 |
|                    |                                                 |
|                    | Support at least 5 new career development or    |
|                    | research infrastructure grants to help to       |
|                    | ensure the existence of an adequate cadre of    |
|                    | future health services researchers to address   |
|                    | existing and emerging research priorities.      |
|                    |                                                 |
|                    | ***Evidence-based Practice Centers***           |
|                    |                                                 |
|                    | Produce evidence summaries for use in Federal   |
|                    | direct care providers' efforts to increase      |
|                    | evidence-based care.                            |
|                    |                                                 |
|                    | Produce a minimum of 18 evidence reports,       |
|                    | systematic evidence reviews, and technology     |
|                    | assessments that may serve as the basis for     |
|                    | interventions by Federal and other direct care  |
|                    | providers, professional associations, and other |
|                    | healthcare organizations to increase            |
|                    | evidence-based practice and enhance health      |
|                    | outcomes and quality.                           |
|                    |                                                 |
|                    | For at least four evidence reports or           |
|                    | technology assessments work with partners to    |
|                    | measure how the reports or assessments were     |
|                    | used and what impact they had on clinical       |
|                    | decision making and patient care.               |
+--------------------+-------------------------------------------------+

+-----------------------------------------------------------------------+
| # GPRA Goal 4: Evaluate the effectiveness and impact of AHRQ research |
|                                                                       |
| and associated activities. (HCQO) (Note: All Agency evaluation        |
| activities, including MEPS-related studies, are included under Goal   |
| 4. This is because the MEPS budget line covers only costs associated  |
| with data design, data collection and analysis, and data products.)   |
+-----------------------------------------------------------------------+

**Strategy: Cycle of Research Phase 4:Evaluation**

As explained in other portions of this document, interim outcomes of
research can be evaluated on a relatively short-term basis. However, the
ultimate outcome of how the research affects people receiving health
care or people interacting with the system requires large, expensive
retrospective studies. AHRQ is implementing a growing portfolio of
evaluations that will show, iteratively, the outcomes of the investments
of Agency funds.

**Types of indicators:** Interim outcomes of research

**Use of results by AHRQ:**

AHRQ conducts evaluations of its major programs or products to achieve
one or more of the following:

-   evaluate the current state of the program or product including
    impact in health care;

-   improve customer satisfaction with the program or product;

-   target or prioritize future activities to increase their usability
    or usefulness.

**Data Issues:**

Many of the evaluations are conducted with the assistance of consultants
who are highly skilled in evaluation research and/or the subject matter.
Some are done through surveys for customer satisfaction that were
cleared through OMB. The third category is evaluations conducted through
consultations with experts and users to obtain direct feedback on a
particular product. In order to ensure the integrity of the evaluations,
the AHRQ staff assigned to these projects are not program staff
responsible for the day-to-day administration of the program.
Additionally, advice on the evaluation questions as well as on the
interpretation and use of the results is often sought from experts on
the AHRQ National Advisory Council.

  -----------------------------------------------------------------------
  **GPRA Goal 4 -- FY 2001 Results**

  -----------------------------------------------------------------------

**Objective 4.1:** Evaluate the impact of AHRQ sponsored products in
advancing

methods to measure and improve health care.

Evidence Reports:

**1^st^ Indicator:** Use of evidence reports and technology assessments
to create

> quality improvement tools in at least 10 organizations.

**Results:** Twenty-one examples are listed of Organizations and their
uses of evidence reports to create tools for improving health care
quality:

-   **Treatment of Attention Deficit and Hyperactivity Disorder (ADHD)**
    -The American Academy of Pediatrics (AAP) is developing a clinical
    practice guideline based on the AHRQ-sponsored evidence report on
    this topic that is expected to be issued in October 2001.

-   **Venous Thromboembolism** -The Eastern Association for Surgery of
    Trauma has developed a guideline submitted to the Journal of Trauma
    Surgery based on an AHRQ-sponsored evidence report on this topic.

```{=html}
<!-- -->
```
-   **Anesthesia Management During Cataract Surgery** and **Treatment of
    Co-Existing Cataract and Glaucoma** -The American Academy of
    Opthalmology (AAO) has considered this evidence report in developing
    a Preferred Practice Parameter for Cataract in the Adult Eye. The
    practice parameter will be issued in September 2001 and will be
    posted on the AAO's website and disseminated to new opthalmic
    residents.

-   **Treatment of Co-Existing Cataract and Glaucoma** -The American
    Academy of Opthalmology (AAO) has considered this evidence report in
    developing a Preferred Practice Parameter for Cataract in the Adult
    Eye. The practice parameter will be issued in September 2001 and
    will be posted on the AAO's website and disseminated to new
    opthalmic residents.

-   **Use of Epoetin in Oncology** -The American Society of
    Hematology/American Society of Clinical Oncology (ASH/ASCO) has
    developed a guideline based on an AHRQ-sponsored evidence report on
    this topic.

-   **Pharmacotherapy for Alcohol Dependence** -The American Society of
    Adolescent Medicine (ASAM) has developed a guideline based on an
    AHRQ-sponsored evidence report on this topic that will be published
    in late 2001.

-   **Evaluation and Treatment of New Onset Atrial Fibrillation in the
    Elderly** -The ACP-ASIM, with representation from AAFP, has
    developed a guideline based on the AHRQ-sponsored evidence report on
    this topic. The American College of Cardiology (ACC), in conjunction
    with the European Society of Cardiology, is also planning to develop
    a guideline based on this report.

-   **Management of Neurogenic/Neuropathic Pain Following Spinal Cord
    Injury** -In response to the findings of this evidence report
    revealing insufficient information on which to base an
    evidence-based guideline, the Paralyzed Veterans of America Steering
    Committee has approved a proposal to use the evidence report to plan
    a strategy for stimulating collaborative research to fill the
    evidence gaps in this field.

-   **Otitis Media with Effusion** -The American Academy of Pediatrics
    (AAP) has convened a panel to develop guidelines based on an
    AHRQ-sponsored evidence report on this topic.

-   **Acute Bacterial Sinusitis in Children** -The American Academy of
    Pediatrics (AAP) developed a practice guideline based on the
    AHRQ-sponsored evidence report that will appear in *Pediatrics* in
    September. There will be a session to promote the guideline at the
    next AAP annual meeting and a CME program on the guideline is being
    developed. This evidence report was also considered in the
    development of guidelines by the American College of Physicians
    (ACP) and the CDC, as well as the Sinus and Allergy Health
    Partnership (a partnership composed of the American Academy of
    Otolaryngic Allergy, the American Academy of Otolaryngology-Head and
    Neck Surgery and the American Rhinologic Society).

-   **Criteria for Referral of Patients with Epilepsy** -The American
    Academy of Neurology (AAN) and the American Epilepsy Society (AES)
    sponsored an Epilepsy Guideline Topic Forum at the most recent AES
    meeting (December 2000) and plan to develop a guideline on this
    topic, based on the AHRQ-sponsored evidence report, beginning in
    September 2001.

-   **Management of Acne** -AHRQ partners, the American Academy of
    Pediatrics (AAP) and the American Academy of Dermatology (AAD), are
    currently using the evidence report in the development of guidelines
    for the treatment of acne. The completed guidelines is expected by
    the end of 2002. These guidelines will be disseminated through each
    professional society, including publication in the Journal of AAD
    and in Pediatrics.

-   **Acute Otitis Media** -The American Academy of Pediatrics (AAP) and
    the American Association of Family Practitioners has convened an
    expert panel that is in the final stages of developing a clinical
    practice guideline based on this evidence report.

-   **Management of Post-operative Atrial Fibrillation** -When this
    evidence report is completed it will be used by the American College
    of Chest Physicians to develop a clinical practice guideline.

-   **Stable Angina** -The American College of Cardiology/American Heart
    Association has developed a guideline based on this evidence report.

-   **Use of Epoetin in Oncology** -The American Society of
    Hematology/American Society of Clinical Oncology Guideline
    Development Committee is using this evidence report as the basis for
    development of a clinical practice guideline.

-   **Cervical Cytology** -The methodology used for this evidence report
    has been adapted for use by an expert-working group (Consensus Panel
    on Cervical Cancer Screening) advising the American Cancer Society
    on development of new guidelines for cervical cancer screening.

-   **Management of Cancer Pain** -The American Pain Society will use
    the evidence report to update their guideline on management of
    cancer pain. The evidence report was cited by a recently published
    IOM report titled *Improving Palliative Care for Cancer*.

-   **Management of Acute Exacerbations of Chronic Obstructive Pulmonary
    Disorder (COPD)** -This evidence report was used by the American
    College of Chest Physicians and the American College of
    Physicians-American Society of Internal Medicine Joint Expert Panel
    on Chronic Obstructive Pulmonary Disease to develop a clinical
    practice guideline for management of acute exacerbations of chronic
    obstructive pulmonary disease.

-   **Pulmonary Complications after Spinal Cord Injury** -The Spinal
    Cord Injury Consortium (funded by Paralyzed Veterans of America) is
    in the process of convening an expert panel to develop a clinical
    practice guideline based on evidence summarized in the report.

-   **US Preventive Services Task Force (USPSTF)** -The American
    Association of Family Practitioners (AAFP) is using new USPSTF
    recommendations based on four AHRQ-sponsored systematic evidence
    reviews in updating AAFP recommendations for preventive care. AAFP
    has also adapted the new USPSTF evidence rating system, released in
    the April 2001 issue of the *American Journal of Preventive
    Medicine* for developing prevention recommendations and policies for
    its members.

**2^nd^ Indicator:** For at least four evidence reports or technology
assessments per

> year, work with partners to measure how the reports or assessments
> were used and what impact they had on clinical.

**Results:** The following six examples show how AHRQ worked with
various partners to measure the impact of evidence reports on clinical
practice:

-   **Diagnosis and Treatment of Attention Deficit/Hyperactivity
    Disorder (ADHD)** --AHRQ's partner, the American Academy of
    Pediatrics (AAP) is launching a three-pronged campaign to promote
    implementation of its guidelines on diagnosis and treatment of ADHD.
    The guidelines are based on the two AHRQ-sponsored evidence reports
    on these topics. The campaign will address providers, parents, and
    payers/purchasers. Tool kits will be developed for providers,
    containing resources such as reminder cards and flow sheets, to
    facilitate adoption of guideline recommendations within pediatric
    practices. In addition, a module on ADHD will be added to the AAP's
    Educational Quality Improvement Program (EQuIP), a web-based
    interactive training tool and database that will offer educational
    case studies and enable practitioners to compare practice patterns
    related to the guidelines with those of other practitioners. A
    companion parent education pamphlet will also be developed and
    disseminated via AAP members and a more detailed informational book,
    written for a lay audience, will be distributed. Outreach to payers
    and purchasers will be accomplished through a series of regional
    meetings. AAP hopes to evaluate the effectiveness of the program,
    particularly the provider tool kits, in a second phase using the
    PROS (Pediatric Research in Office Settings).

-   **Acute Otitis Media** -AHRQ has awarded a task order, under the
    ISDRN contract, to the Lovelace Clinic Foundation to facilitate
    implementation and evaluation of guidelines based on the
    AHRQ-sponsored evidence report on Acute Otitis Media. The project
    involves:1) development of a clinical practice guideline based on
    the evidence report; 2) implementation of the guideline within the
    Lovelace Health Systems, a large integrated managed care delivery
    system in Albuquerque, New Mexico, based on an Episode of Care (EOC)
    model; measurement of the impact and evaluation of the effectiveness
    of the implementation strategies; and assessment of the difference
    in outcomes by ethnicity and socioeconomic status. The goal of the
    project is to enhance understanding of the most effective methods of
    changing physician and patient behavior to promote adherence to
    evidence-based guidelines.

-   **Venous Thromboembolism** -Based on the research gaps identified in
    the AHRQ-sponsored evidence report on this topic, a multi-centered
    trial (US and Canada) has been launched to obtain more definitive
    evidence regarding best treatment practices for venous
    thromboembolism.

-   **Evaluation and Treatment of New Onset Atrial Fibrillation** -The
    North Carolina Peer Review/Quality Improvement Organization (PRO),
    as part of its efforts under the CMS-sponsored National Stroke
    Project, has widely disseminated copies of the AHRQ-sponsored
    evidence report on Atrial Fibrillation to physicians and hospitals
    that treat atrial fibrillation patients across the state. In
    addition, a physician opinion leader presented the findings of the
    report to an audience of 246 health care professionals during a
    teleconference on preventing strokes in patients with atrial
    fibrillation. Participants had the opportunity to discuss the
    finding with an expert panel. Rates of warfarin prescription prior
    to discharge of patients with atrial fibrillation have been measured
    for the state and will serve as a baseline for assessing
    improvements in practice over time. The PRO is also providing a data
    abstraction tool to hospitals in the state that will enable them to
    assess their performance on this measure and assist in targeting
    quality improvement efforts.

-   **Diagnosis and Treatment of Dysphagia/Swallowing Problems in the
    Elderly** -Based on the results of this evidence report, AHRQ is
    working with relevant healthcare and research organizations to
    develop a research agenda that would facilitate filling in the gaps
    in the evidence in order to improve clinical practice surrounding
    treatment of swallowing problems in the elderly.

-   **Use of Ethropoeitin in Hematology and Oncology** -AHRQ provided
    grant funding for an investigator from Northwestern University to
    conduct a baseline survey of EPO use patterns among clinician
    members of the American Society for Clinical Oncology (ASCO) and the
    American Society of Hematology (ASH). Results of this survey will
    serve as a benchmark that will be useful in measuring and evaluating
    how the introduction of the ASH/ASCO clinical practice guideline,
    based on the AHRQ-sponsored evidence report, are affecting clinical
    practice.

**3^rd^ Indicator:** Findings from at least 3 evidence reports or
technology

> assessments will affect State or Federal health policy decisions.

**Results:** The following twelve examples are listed of the effects of
evidence reports and technology assessments on the health policy
decisions of State and Federal agencies:

-   **Efficacy of Interventions to Modify Dietary Behavior Related to
    Cancer Risk** \-- The National Cancer Institute served as partner
    and funder of the AHRQ-sponsored evidence report on this topic. The
    evidence report was incorporated into a Review for the National
    Cancer Policy Board (a joint NCI/IOM collaboration) on behavioral
    interventions in cancer prevention and early detection. In June,
    2001 the Research subcommittee of the 5 A Day Steering Committee
    noted the evidence report in developing a strategic plan for new
    research priorities. The NCI has broadly disseminated the report.
    They have posted a link to the report on their website and have
    highlighted it at a variety of investigator meetings, including the
    NCI Nutrition Behavior Change investigators meeting and the American
    Institute for Cancer Research Conference.

-   **Technology Assessment on Actinic Keratosis** \-- This technology
    assessment was used by the Centers for Medicare and Medicaid
    Services (CMS) as a basis for a coverage decision. On July 19, 2001
    CMS announced that the Coverage Issues Manual will be revised to
    indicate that Medicare will cover the destruction of actinic
    keratosis, without restrictions based on lesion or patient
    characteristics, using surgical or medical treatment methods,
    including but not limited to: cryosurgery with liquid nitrogen;
    curettage; excision; and photodynamic therapy.

-   **Diagnosis and Prevention of Dental Caries** \-- This evidence
    report was used as the basis for a NIH Consensus Development
    Conference (CDC). This was the first Federal CDC to base its
    findings on an evidence-based systematic review. Subsequent to the
    success of this effort, the NIH Office of Medical Applications of
    Research Consensus Development Conference program and AHRQ developed
    a clearly defined coordination process to provide systematic reviews
    on an ongoing basis for NIH CDCs. Prior to every Consensus
    Development Conference, an AHRQ Evidence-based Practice Center (EPC)
    develops an evidence report based on a rigorous systematic review
    and analysis of the scientific literature on the Conference topic.
    The EPC clinical experts and methodologists present their report
    findings and conclusions to the Consensus Conference panel and at
    the Consensus Conference. The Conference panel then can incorporate
    the findings of the EPC into their final Conference report. This
    insures that they have an impartial review of the best evidence when
    they are making their recommendations.

-   **Oral Health and HIV** \-- This evidence report was the basis of a
    State of the Science conference sponsored by NIH and NIDCR in
    December 2000.

-   **Telemedicine for the Medicare Population** \-- CMS is using this
    evidence report to develop a research agenda.

-   **Criteria to Determine Disability in Patients with Chronic Renal
    Failure** \--

> Based on this evidence report the SSA concluded there is insufficient
> evidence to change their policy regarding disability determination for
> End Stage Renal Disease (ESRD) patients.

-   **Management of Hepatitis C** \-- This project will generate a
    report that will be used by an NIH-sponsored Consensus Development
    Conference on Management of Hepatitis C.

-   **US Preventive Services Task Force (USPSTF) \--** Consistent with
    recommendations in a recent Institute of Medicine (IOM) report, Sen.
    Bob Graham has introduced legislation in Congress that would
    establish a fast-track approval process for decisions on coverage of
    clinical preventive services recommended by the USPSTF, based on
    AHRQ-sponsored systematic evidence reviews.

-   **Diagnosis and Monitoring of Osteoporosis in Postmenopausal Women**
    -An AHRQ-sponsored evidence report on this topic was used by the
    panel of the National Institutes of Health Consensus Development
    Conference on Osteoporosis Prevention, Diagnosis and Therapy, held
    March 27-29, 2000.

-   **Cervical Cytology** \-- Findings from this evidence report have
    been considered by the National Breast and Cervical Cancer Detection
    Program and the Expert Panel on HPV Infection and Sequelae at the
    CDC, as well as the Advisory Panel on Microbiological Devices of the
    FDA. They were also used by the US Preventive Services Task Force in
    developing recommendations for cervical cancer screening.

-   **Management of Cancer Pain** \-- This evidence report will be
    updated and used in the upcoming National Institutes of Health
    State-of-the-Science Conference on Symptoms Management in Cancer.

-   **Management of Uterine Fibroids \--** The Steering Committee of the
    Cooperative Multicenter Reproductive Medicine Network of the
    National Institute of Child Health and Human Development (NICHD) is
    using this evidence report to help develop research projects for the
    Network. The results of the report were also presented to the Annual
    Meeting of Directors of Programs in Reproductive Medicine who
    receive grants funded by NICHD. The results were also presented to
    the Advisory Council of the National Institute of Environmental
    Health Sciences, for assistance in strategic planning of the
    research agenda for the Institute. This report is also being used by
    the Obstetrics and Gynecology Devices Branch of the Food and Drug
    Administration to evaluate issues in the regulatory evaluation of
    medical devices used in the treatment of women with fibroids.

**4^th^ Indicator:** Use of evidence reports or technology assessments
and access to

> NGC site informed organizational decisionmaking in at least 4 cases
> and resulted in changes in health care procedures or health outcomes.

**Results:** The impact of AHRQ-sponsored research is evident in the
following examples of organizational decision making informed by
evidence reports, technology assessments and data availability at the
NGC Web site:

-   **US Preventive Services Task Force (USPSTF)** \-- USPSTF
    recommendations, based on AHRQ-sponsored systematic evidence reviews
    (SER's), formed the basis of an effort by the nonprofit group
    Partnership for Prevention to prioritize recommended clinical
    preventive services to facilitate decision making among purchasers,
    payers, and health plans about coverage and delivery of clinical
    preventive services. The study, released in June 2001, prioritizes
    clinical preventive services based on their health impact and cost
    effectiveness. The release of this report received significant
    national media attention and the National Healthcare Purchasing
    Institute will distribute 4,000 copies of a related monograph for
    employer-purchasers. It is expected that this report will have
    widespread impact on policy makers and organizational decision
    makers in considering issues related to coverage of clinical
    preventive services.

-   **Rehabilitation for Traumatic Brain Injury in Children and
    Adolescents** \-- Based on results of the AHRQ-supported evidence
    report on this topic, Aetna U.S. Healthcare announced a change in
    national coverage policy. As of February 2, 2000, consistent with
    the findings of the evidence report, Aetna allows payment for
    cognitive rehabilitation as adjunctive treatment of cognitive
    deficits when specified criteria are met.

-   **Cervical Cytology** \-- The findings of this evidence report have
    been considered by two large HMOs \-- Excellus Healthcare (Blue
    Cross/Blue Shield of upstate NY) and Henry Ford (Detroit) -- to
    develop coverage policies with respect to cervical cancer screening
    using the new liquid-based technologies. In addition, the
    cost-effectiveness model from this report is being used by a major
    pharmaceutical company to assist in the planning and evaluation of a
    human papillomavirus vaccine.

-   **Management of Uterine Fiborids** \-- This evidence report is being
    used by the Society for Cardiovascular and Interventional
    Radiologists to design a prospective registry for measuring outcomes
    in women undergoing uterine artery embolization. Study design and
    choice of outcome measures were largely guided by the findings of
    the evidence report. Intermountain Health Care, a large integrated
    health system in Utah, is considering use of the report to develop
    guidelines for management of women with fibroids, as well as
    potential evaluations of the impact of the guidelines on patient
    care.

-   **Criteria for Clinical Guidelines** \-- An article based on this
    AHRQ-sponsored EPC report that reviews AHCPR's former guidelines and
    provides criteria for when clinical guidelines should be updated
    an/or withdrawn, will be published in the British Medical Journal
    and will be useful to organizations that develop and adapt clinical
    practice guidelines in making decisions about updating and
    withdrawing outdated guidelines.

# Research

**5^th^ Indicator:** At least 3 examples of how research informed
changes in policies

> or practices in other Federal agencies.

**Results:** AHRQ research has been used by several federal agencies to
structure policies and practices. Examples are listed below:

-   AHRQ's research portfolio on low income populations has informed
    HRSA's work on evaluating and improving the health care safety net.

-   AHRQ-funded research on guideline development and implementation has
    influenced the recommendations on professional education/performance
    measurement made by the Secretary's Advisory Committee on Genetic
    Testing (an NIH-led effort).

-   AHRQ-sponsored research conducted by Richard Zimmerman, M.D. on the
    effect of the Free Vaccine program on delivery of immunizations in
    primary care settings has supported the efforts of CDC to continue
    that program.

-   The National Cancer Policy Board issued a white paper on the
    volume-outcome relationship in cancer surgery on July 12 that
    includes a table based on HCUP analyses. The paper contains research
    recommendations that were informed by the clarity HCUP data brought
    to the understanding of potential impact of selective referral
    recommendations.

In addition, AHRQ is working to document impact case studies of how
research has facilitated changes in policies or practices in other
federal agencies. AHRQ-sponsored workshops, presentations, and technical
assistance that address use of AHRQ tools generate a great deal of
interest in products resulting from AHRQ investments. The following
exemplify this:

-   AHRQ conducted a 1.5-hour methods workshop on HCUP data at the
    AHSRHP conference. Despite being held as one of the last sessions,
    there were over 55 participants and there was heightened interest in
    the data.

-   At another session AHSRHP, AHRQ's staff did a presentation on the
    use of HCUP data and the Quality Indicators for the National Quality
    Report.

-   In addition, staff participated in two sessions at the Child Health
    Services Research conference, doing a presentation on the use of
    HCUP data and tools for research on children\'s hospitalizations,
    and moderating a session on state data for child health services
    research.

AHRQ-supported research was the information source for several
government agencies. The following demonstrate how diverse these
agencies are:

-   CBO -- received information on trends in mental health
    hospitalizations from 1990-1997. These data are useful to work they
    are doing on parity in coverage of mental health services (based on
    NIS data).

-   The National Quality Forum -- received information on the most
    common reasons for hospitalization by race for 7 HCUP SID states.

-   ASPE -- received information on quality of care for Medicare
    patients.

## 

## Quality Measures

**6^th^ Indicator:** Achievable Benchmarks of Care are used for quality
improvement

> activities by Peer Review Organizations.

**Results:** A new CAHPS RFA was released in May, 2001 that will include
the development of a module for use in quality improvement activities.

**7^th^ Indicator:** Use of dental performance measures by dental
service and

> insurance organizations.

**Results:** Draft instruments for Nursing Home CAHPS, Dental CAHPS,
ECHO are currently undergoing testing. In addition, AHRQ has obtained
expert input into the development of a CAHPs module for persons with
mobility impairments.

**8^th^ Indicator:** HCUP quality indicators incorporated into efforts
by hospital

> associations and hospitals to improve the quality of care.

**Results:** Quality indicators from AHRQ-funded research, used to
assess health care quality, are increasingly an integral part of
providers' efforts toward improving the level health care quality in the
nation. For example, the Healthcare Association of New York State
(HANYS) provides feedback to hospitals using the HCUP Quality Indicators
and has described a number of examples of how hospitals have changed
internal practices or have entered into collaborations with surrounding
communities to improve the quality of care:

-   One New York hospital reported that a hospital-wide intervention to
    prevent venous thrombosis was initiated in response to higher than
    average rates of post-surgical venous thrombosis reported through
    the HCUP QIs.

-   A group of hospitals worked with primary care providers in the

> community to improve the outpatient treatment of diabetes because the
> rates of admission for diabetes complications measured using the HCUP
> QIs were higher than for other communities.

-   Another group of hospitals worked with the state chapter of the
    College of Obstetricians and Gynecologists to reduce high rates of
    low birth weight identified using the HCUP QIs by fostering
    appropriate prenatal care in the surrounding community.

## National Guideline Clearinghouse

**9^th^ Indicator:** At least 10 users of the National Guideline
Clearinghouse will use

> NGC site to inform clinical care decisions.

**Results:** The following are examples from the June 2001 NGC Second
Annual Customer Satisfaction Survey of physician comments about how they
use the NGC to inform clinical care:

-   I can find appropriate choice of therapies for the patient I am
    taking care of.

```{=html}
<!-- -->
```
-   It is help in my clinical practice.

```{=html}
<!-- -->
```
-   This is a way to keep updated.

```{=html}
<!-- -->
```
-   I use it to search for guidelines for treatment and diagnosis to
    formulate the best approach for \[my patients\].

```{=html}
<!-- -->
```
-   Advice for decision making in daily routine.

```{=html}
<!-- -->
```
-   An excellent scientific source for my practice.

```{=html}
<!-- -->
```
-   Answering clinical questions.

```{=html}
<!-- -->
```
-   Apply evidence-based medicine in clinical practice.

```{=html}
<!-- -->
```
-   As a handy source for clinical question resolution.

```{=html}
<!-- -->
```
-   \[I use the NGC\] as a pediatrician doing patient care and an
    administrator developing a system for EBM at point of care.

-   I am an academic Physician (Endocrinologist) and I need a rapid
    access to all kinds of Medical information pertinent to Internal
    Medicine and Endo/Metaboloism. I recently discovered the NGC site,
    which, I believe, is the most "cost/effective" site available in
    Internet for a rapid, comprehensive review of the state of art about
    Medical Guide.

**10^th^ Indicator:** Guideline development or quality improvement
efforts by users will

> be facilitated through use of NGC in at least 5 cases.

**Results:** The following are examples from the June 2001 NGC Second
Annual Customer Satisfaction Survey of physician comments about how they
use the NGC for guideline development or quality improvement efforts:

-   As a critical care anesthesiologist in charge of a department I
    found it very helpful to base our protocols on solid ground.

```{=html}
<!-- -->
```
-   As Chief of the Medical Staff at a military hospital, I use the
    guidelines to help standardize rational approaches to health care
    and treatment for the physicians that work in my facility.

```{=html}
<!-- -->
```
-   Assistance in developing new local guidelines

```{=html}
<!-- -->
```
-   Development of guidelines for a pediatric care network

```{=html}
<!-- -->
```
-   Develop a strategic plan for my hospital re methods of
    standardization of clinical care

```{=html}
<!-- -->
```
-   Developing clinical guidelines for my group practice

```{=html}
<!-- -->
```
-   Developing critical paths in my hospital

```{=html}
<!-- -->
```
-   Developing guidelines for a managed care organizations

```{=html}
<!-- -->
```
-   Developing guidelines for our community health center?s use

```{=html}
<!-- -->
```
-   Developing physician quality assessment based on compliance with
    guidelines

**11^th^ Indicator:** NGC information will be used to inform health
policy decisions in

> at least 2 cases.

**Results:** The following are examples from the June 2001 NGC Second
Annual Customer Satisfaction Survey of Hospital/Health Administrator,
Purchaser/Employer, and Policymaker comments about how NGC has been used
to inform health policy decision:

-   As a resource for policy formation at my health care facility.

```{=html}
<!-- -->
```
-   In the development of medical policies and clinical guidelines.

```{=html}
<!-- -->
```
-   Review of national guidelines for medical policy development.

```{=html}
<!-- -->
```
-   Coverage decision, clinical policy and guideline development.

```{=html}
<!-- -->
```
-   Reviewing it for reference in our Health Alaskans 2010 planning
    documents.

**12^th^ Indicator:** Improvements in clinical care will result from
utilization of NGC

> information in at least 3 cases.

**Results:** The following physician comments about improvements in
clinical care resulting from use of the NGC are excerpted from the June
2001 NGC Second Annual Customer Satisfaction Survey:

-   \[I use the NGC because\] I want to improve my clinical practice
    level

```{=html}
<!-- -->
```
-   \[I use it to search\] for guidelines for treatment and diagnosis to
    formulate the best approach for \[my patients\]

```{=html}
<!-- -->
```
-   As Chief of the Medical Staff at a military hospital, I use the
    guidelines to help standardize rational approaches to health care
    and treatment for the physicians that work in my facility.

## 

## Training Programs

**13^th^ Indicator:** Two thirds of former pre- and postdoctoral
institutional award

> trainees are active in the conduct or administration of health
> services research.

**Results:** A comprehensive evaluation of career paths and productivity
of former pre-and postdoctoral trainees supported on institutional
training grants was undertaken. Data indicate that former trainees are
employed and actively involved in the conduct or administration of
health services research. Results have demonstrated that 2/3 of
pre-doctoral students are actively engaged in health services research
activities (the majority of the remaining 1/3 have not yet completed
their training). Over 90% of postdoctoral trainees supported by AHRQ are
employed in research-related positions. The majority of former trainees
are primarily in academia, though with an increased number opting for
careers in nonacademic settings. Overall, three quarters continue to
publish in academic journals since completion of their training, 1/3
have published books or chapters in books, and approximately ¼ have
published technical reports. Over 50% were investigators on grants or
contracts related to health services research projects, a rate which
escalates to 2/3 among former postdoctoral trainees.

In the spirit of continuous quality improvement, AHRQ also queried
former trainees on their training experiences. Satisfaction rates were
captured across multiple dimensions ranging from program content, to
mentoring, to career relevancy. All were extremely high (over 90% were
satisfied). Feedback provided by former students served as a basis for
the development and refinement of career development programs sponsored
by AHRQ. A conference on curriculum development was convened, at which a
variety of educational institutions exchanged approaches to and
innovations in preparing pre as well as postdoctoral students for
careers in health services research. Attention was paid to the formation
of public/private partnerships, responsiveness to community interests,
and training in cultural competency. In addition, AHRQ embarked on
programs designed to foster the research career development of newly
minted researchers in order to prepare them to compete successfully with
established researchers for support. Effort was also placed on building
and extending a community of health services researchers across current
training programs, as well as in states which do not traditionally
receive significant HSR funding and institutions which traditionally
serve minority students.

**Patient Safety**

**14^th^ Indicator:** Fund the establishment of a patient safety program
evaluation

center that shall develop and implement an overall evaluation plan for
patent safety projects funded by AHRQ.

**Results:** A task order has been issued to establish an implementation
plan for a patient safety program evaluation center. Proposals were due
by 8/30/2001 and are currently under review.

**Objective 4.2**: Evaluate the impact of MEPS data and associated
products on

> policymaking and research products.

**1^st^ Indicator:** Use of MEPS data in AHRQ research applications will
increase by

> 10 percent over number received in baseline period of 2000.

**Results:** The number of grant applications submitted to AHRQ that
proposed the use

Of MEPS data decreased from 4.7% in 2000, to 2.5% in 2001. Out of a
total of 715 grant applications submitted to AHRQ in 2001, 18 required
the use of MEPS data. 3 of these applications were funded. This decrease
may have been due to AHRQ targeting the funding of grants in 2001 on
research areas that are not supported by MEPS data, such as patient and
worker safety. To improve access to technical assistance CCFS conducted
12 user workshops and answered 920 technical assistance questions.
Approximately 100 users were added to the MEPS LIST SERVE increasing the
subscriber number to 400. CCFS staff also staffed the AHRQ booth at
professional meetings to answer MEPS related questions.

**2^nd^ Indicator:** Feedback from MEPS workshop participants indicating
that the

> workshops were useful and timely.

**Results:** In 2001 CCFS conducted 12 MEPS user workshops. 95 percent
of

participants completing workshop evaluations said that the workshop met
their primary

goal for attending the workshop.

**3^rd^ Indicator:** At least 5 examples of how research using MEPS has
been used

> to inform decisions by Federal, state and private sector policymakers.

**Results:** In 2001 CCFS published 47 journal articles in peer reviewed
journals, 3 Chart Books, 1 Methodology Report, 2 Research Findings
Reports, and 1 Highlight publication. (see attachments). All of these
publications provide valuable input into health care policy formulation.
In 2001 CCFS staff also made more than 100 MEPS related presentations at
professional meetings or for heath related organizations. The following
examples highlight the use of MEPS data for policy decision:

-   The staff of Representative Istook (Oklahoma) requested a tabulation
    of the top health conditions ranked according to condition specific
    health expenditures.

```{=html}
<!-- -->
```
-   In June, during the Congressional debates on the patients Bill of
    Rights, AHRQ received several inquiries about MEPS-IC health
    insurance data, including requests from Senator Kennedy's office and
    from the Heritage Foundations. A follow-up call to Senator Kennedy's
    office confirmed that they found the information on our web site to
    be informative and useful. According to Ralph Rector of the Heritage
    Foundation "When it comes to health insurance premium information,
    the MEPS-IC is the only game in town."

```{=html}
<!-- -->
```
-   Two States, Wisconsin and Arkansas purchased additional sample in
    the MEPS-IC this year to improve the MEPS-IC estimates for their
    states. This is related to work we have been doing for 20 States
    that are studying ways to expand health insurance coverage for low
    income children, workers, and families by coordinating public
    subsidies with private (employment-based) coverage, both under SCHIP
    and potentially more broadly. Over the past 6 months, we have
    provided special data tabulations to representatives from these
    States that provides additional information from the MEPS-IC survey
    to help them understand the structure of employment-based coverage
    in their States.

```{=html}
<!-- -->
```
-   AHRQ staffers prepared a literature review entitled "Does Spending
    on Prescription Drugs Reduce Spending on Other Health Care Services?
    A Preliminary Literature Review." This literature review, which was
    used as the basis for briefing staff in the Office of the Secretary,
    DHHS in April 2001. Featured among the papers reviewed was a recent
    study on this topic using MEPS data by researchers at Columbia
    University.

```{=html}
<!-- -->
```
-   AHRQ responded to a request from the Council of Economic Advisors
    for information on non-group premiums from the 1996 MEPS. The
    Council was provided with estimates by age group and single and
    family policies for the non-elderly population. The Council was also
    interested in more information on the type of policies purchased in
    the nongroup market, specifically whether these were high deductible
    plans. For those policies with linked MEPS HIPA data, AHRQ was able
    to show that many of the nongroup plans purchased in 1996 were high
    deductible policies. AHRQ also provided 1996 MEPS HC-IC estimates of
    private, non-group health insurance premiums by age group in
    response to requests from the Treasury, CBO, ASPE (to be included in
    a memo to the Assistant Secretary for Policy and Evaluation).

```{=html}
<!-- -->
```
-   AHRQ staff presented a paper on children's insurance at the National
    Academy of Sciences' "Workshop on Enhancing Methods for the State
    Children's Health Insurance Program" in June. The paper is meant to
    help states understand issues underlying the design of public health
    insurance programs for low-income children.

```{=html}
<!-- -->
```
-   MEPS data were used to formulate Recommendations in the Institute of
    Medicine Report -- Crossing the Quality Chasm: A new Health System
    for the 21st Century.

```{=html}
<!-- -->
```
-   MEPS data were used to formulate recommendations in the Institute of
    Medicine publication on the National Quality Report.

+-----------------------------------------------------------------------+
| ###### GPRA Goal 4 -- FY 2002 Indicators                              |
+-----------------------------------------------------------------------+

+---------------------+------------------------------------------------+
| #                   | #                                              |
|                     |                                                |
| # Goal 4 Objectives | # FY 2002 Indicator                            |
+---------------------+------------------------------------------------+
| **Objective 4.1**   | Fund evaluation of the impact of the CERTS     |
| Evaluate the impact | program in disseminating information regarding |
| of AHRQ sponsored   | therapeutics to at least 3 health care         |
| products in         | providers or others in order to improve        |
| advancing methods   | practice.                                      |
| to measure and      |                                                |
| improve health      | As a pilot approach to program evaluation,     |
| care.               | report on a citation analysis of results of    |
|                     | one major research initiative to assess        |
|                     | productivity and potential impact.             |
|                     |                                                |
|                     | Fund evaluation of private sector use of AHRQ  |
|                     | findings to identify at least 5 private sector |
|                     | uses of AHRQ findings, and describe any        |
|                     | assessment of the impact on clinical practice  |
|                     | and/or patient care.                           |
|                     |                                                |
|                     | Fund evaluation of the number and types of     |
|                     | patient safety events reported to system       |
|                     | grantees.                                      |
|                     |                                                |
|                     | Fund the interim evaluation of the extent to   |
|                     | which patient safety best practices,           |
|                     | identified in July 2001 EPC report, have been  |
|                     | adopted by health care institutions.           |
|                     |                                                |
|                     | Report on the evaluation of HCUPnet and the    |
|                     | HCUP Central Distributor and implement         |
|                     | appropriate changes to each based on           |
|                     | evaluation results.                            |
|                     |                                                |
|                     | Fund evaluation of how AHRQ funded and         |
|                     | conducted research on low-income populations   |
|                     | is used by policymakers.                       |
|                     |                                                |
|                     | Initiate evaluation of the effects of AHRQ's   |
|                     | investment in Practice-Based Research          |
|                     | Networks.                                      |
|                     |                                                |
|                     | # Evidence-based Practice Centers              |
|                     |                                                |
|                     | Use of evidence reports and technology         |
|                     | assessments to create quality improvement      |
|                     | tools in at least 10 organizations.            |
|                     |                                                |
|                     | Findings from at least 3 evidence reports or   |
|                     |                                                |
|                     | technology assessments will affect State or    |
|                     | Federal health policy decisions.               |
|                     |                                                |
|                     | Use of evidence reports or technology          |
|                     | assessments                                    |
|                     |                                                |
|                     | or access to NGC site informed organizational  |
|                     | decision making in at least 4 cases and        |
|                     | resulted in changes in health care processes,  |
|                     | quality, or health outcomes.                   |
|                     |                                                |
|                     | **Patient Safety**                             |
|                     |                                                |
|                     | User's panel (with composition similar to that |
|                     | of September 2000 patient safety research      |
|                     | summit) will evaluate progress on patient      |
|                     | safety research agenda defined by September    |
|                     | 2000 summit. Evaluation criterion will be      |
|                     | progress made in at least 50% of the research  |
|                     | priorities as a result of the Agency's FY01    |
|                     | RFAs.                                          |
|                     |                                                |
|                     | **Training Programs**                          |
|                     |                                                |
|                     | Establish continuous quality improvement       |
|                     | systems for conducting process and outcome     |
|                     | evaluations of new training and career         |
|                     | development initiatives to help to ensure they |
|                     | are responsive to emerging needs and demands   |
|                     | of the health care delivery system.            |
+---------------------+------------------------------------------------+
| **Objective 4.2**:  | Have a fully functional MEPS-based MEDSIM      |
| Evaluate the impact | model to allow simulation of the potential     |
| of MEPS data and    | impact of programmatic changes in health care  |
| associated products | financing and delivery Dec 2002.               |
| on policymaking and |                                                |
| research products.  | Produce baseline FY statistics on number of    |
|                     | MEPS-based articles published in peer review   |
|                     | journals.                                      |
|                     |                                                |
|                     | Conduct customer satisfaction survey for MEPS  |
|                     | workshop participants to assess how MEPS data  |
|                     | is being used to inform research and public    |
|                     | policy.                                        |
|                     |                                                |
|                     | Develop marketing plan to promote the MEPS-IC  |
|                     | data to state officials Dec 2002.              |
|                     |                                                |
|                     | Increase by 15% over FY01 baseline the number  |
|                     | of examples of how research using MEPS has     |
|                     | informed decisions by Federal, state and       |
|                     | private sector policymakers.                   |
+---------------------+------------------------------------------------+

+-----------------------------------------------------------------------+
| **GPRA GOAL 5:** **Support of initiative to improve health care**     |
|                                                                       |
| **Quality through leadership and research. (HCQO)**                   |
+-----------------------------------------------------------------------+

**Strategy: Quality Interagency Coordination Task Force (QuIC)**

The President mandated the establishment of the Quality Interagency
Coordination Task Force (QuIC) as a vehicle for promoting collaboration
among the Federal Agencies with health care responsibilities to improve
the quality of care in America. The Secretaries of Health and Human
Services and of Labor co-lead this activity and the AHRQ Director serves
as operating chair. The QuIC is working to improve patient and consumer
information, quality measurement systems, the workforce's ability to
deliver high quality care, and the information systems needed to support
the analysis of the care provided. Input gathered through these
coordinated activities contributes significantly to the development of
quality-related research conducted and sponsored by AHRQ.

**Types of Indicator:** Process and output.

**Use of Results by AHRQ:**

The QuIC provides AHRQ with opportunities to further two major Agency
goals. (1) In working with the Federal agencies that provide and/or
purchase health care for millions of Americans, AHRQ is learning what
major users of health services research on quality, evidence-based
medicine and other topics need. This provides AHRQ with an invaluable
source of real-time user input and directly influences the Agency's
research agenda and product development. (2) The QuIC provides AHRQ with
unparalleled opportunities to advance its *Translating Research Into
Practice* agenda. The Agency is able to inform the Federal health care
community about the existence of research and products that currently
are in the portfolio and are relevant to the issues the community is
wrestling with.

**Data Issues:**

The results for these indicators are largely completed work products and
success in meeting project milestones. As the Director of AHRQ is the
QuIC operational chair the AHRQ Coordinator for Quality Activities is
assigned to monitor progress of the various workgroups and maintains to
all the pertinent data. The majority of the work products of the group
are available upon completion to the public. In February 2000, the QuIC
website became operational at

  -----------------------------------------------------------------------
  **GPRA Goal 5 --FY 2001 Results**

  -----------------------------------------------------------------------

**Objective 5.1:** Conduct research to help to measure the current
status

> health of care quality in the Nation.

**1^st^ Indicator:** QI Taxonomy meeting held under the auspices of the
QuIC.

Results: A taxonomy is being published in a peer-reviewed journal in
cooperation

With CMS on quality improvement efforts

.

**2^nd^ Indicator:** Number of grants and contracts funded in FY2001
that will help to

> fill gaps in the information available to assess the national quality
> of care, or will help to expand the use of current measures to provide
> a broader or richer picture of quality.

**Objective 5.2:** Facilitate use of quality information to improve
health care in the

> Nation.

**1^st^ Indicator:** Number of grants to improve patient safety.

**Results:** A complete list of grants designed to improve patient
safety can be found in the appendices of this document. The following
provides a summary of the types and number of grants approved for
funding in FY 2001:

-   Funded 17 DCERPS

-   Funded 3 Centers of Excellence in Patient Safety Research

-   Funded 16 event reporting demonstrations

-   Funded 6 cooperative agreements to provide dissemination and
    education in patient safety.

-   Funded a coordinating center to assist in the dissemination of
    research results and best practices.

**2^nd^ Indicator:** Adoption of Agency sponsored research and tools
developed by

> one or more users to facilitate consumers/purchaser/decision- maker
> use of information about quality.

**Results:** CDC is now using NIS data (from HCUP) instead of NHDS data.
Two articles are to be published in the Nov. 2001 issue of IMWR based on
NIS data. In addition, an AHRQ grantee, a researcher at Dartmouth, is
working for CMS (formerly HCFA) on the proportion of procedures
performed for Medicare patients compared with all other patients. The
researcher is studying the volume-outcome relationship in the Medicare
population for a number of procedures. This is the initial information
AHRQ will be providing as part of an investigation into whether CMS
should publicly report information on hospital volume.

**Objective 5.3:** Improve quality measurement.

**1^st^ Indicator:** Identification of collaborators for research
projects on electronic

> medical records integrated with guidelines (e.g., from the Guideline
> Clearinghouse) or QI indicators (e.g., CONQUEST, QI Taxonomy project,
> HCUP measures).

**Results:** AHRQ has extended its collaborative partnerships throughout
the country. The agency has been very successful in its efforts to
collaborate on projects related to integrating guidelines from funded
research with electronic medical records. The following grants that
address implementation of guidelines, quality improvement indicators, or
performance measures through the use of electronic medical records have
been awarded by AHRQ:

Brigham & Women's Hospital Improving Quality with Outpatient Decision
Support

(U18 HS11046)

Baylor College of Medicine Baylor Practice-Based Research Network

(P20 HS11187)

Children's Hospital Family Linkages Supporting Hyperbilirubin

Boston, Ma. Guidelines

(R01 HS09390)

Duke University Med. Center Interactive, Guideline-Based Decision
Support on

(R01 HS09436) the Web

Indiana University An Inner-City Primary Care Research Network

(P20 HS11226)

Medical University of Primary and Secondary Prevention of CHD and

South Carolina Stroke

(U18 HS11132)

University of Pittsburgh Depression Care Using Computerized Decision

(R01 HS09421) Support

**2^nd^ Indicator:** Adoption of Living With Illness children\'s health
measure by

> NCQA.

**Results:** In FY2001, NCQA adopted a measure found in the CAHPS 2.0
related to living with illness for children's health. In addition, AHRQ
has worked with accrediting bodies to:

-   Establish a process for reviewing new surveys proposed for inclusion
    in the set of CAHPS surveys.

-   Maintain an AHRQ representation on key external bodies including the
    NCQA Committee on Performance Measurement, JCAHO Board of
    Commissioners and Performance Measure Advisory Committee, the
    Consortium (formerly known as AMAP), and the National Quality Forum
    research subcommittee.

  -----------------------------------------------------------------------
  **GPRA Goal 5 -- FY 2002 Indicators**

  -----------------------------------------------------------------------

+---------------------+------------------------------------------------+
| #                   | **FY 2002 Indicator**                          |
|                     |                                                |
| # Goal 5 Objectives |                                                |
+---------------------+------------------------------------------------+
| **Objective 5.1:**  | Sponsor and conduct research to help measure   |
| Conduct research to | the overall safety culture within health care  |
| help to measure the | institutions.                                  |
| current status of   |                                                |
| health care quality | Identify at least one private sector data      |
| in the Nation.      | source which will be incorporated into the     |
|                     | national quality report by 31 December 2002.   |
+---------------------+------------------------------------------------+
| **Objective 5.2:**  | Fund or conduct at least 3 projects bringing   |
| Facilitate use of   | healthcare                                     |
| quality information |                                                |
| to improve health   | information to the public in an                |
| care in the Nation. | understandable, user friendly manner that      |
|                     | facilitates its use in decision making.        |
+---------------------+------------------------------------------------+
| **Objective 5.3:**  | Adoption of at least one quality measure to be |
| Improve quality     | developed from our vulnerable populations RFA  |
| measurement.        | by a national accrediting organization.        |
|                     |                                                |
|                     | Identify 3 examples of how the Quality         |
|                     | Indicators were utilized by the public.        |
+---------------------+------------------------------------------------+

  -----------------------------------------------------------------------
  **2.2 Budget Line (2) \-- Medical Expenditure Panel Surveys (MEPS)**

  -----------------------------------------------------------------------

Funding Levels: FY 2001 \$40,850,000 (Actual -- Current Law)

FY 2001 \$40,850,000 (Actual -- Proposed Law)

FY 2002 \$48,500,000 (Appropriation)

FY 2002 \$48,500,000 (Current Estimate)

FY 2003 \$53,300,000 (Request -- Current Law)

FY 2003 \$53,300,000 (Request -- Proposed Law)

This budget line reflects AHRQ funding for the data collection and
related activities conducted through the Medical Expenditure Panel
Surveys.

+-----------------------------------------------------------------------+
| **GPRA Goal 6: Produce and release information from MEPS on**         |
|                                                                       |
| > **health care access, cost, use, and quality.**                     |
+-----------------------------------------------------------------------+

**Strategy:** AHRQ\'s Medical Expenditure Panel Surveys collects
detailed information regarding the use and payment for health care
services from a nationally representative sample of Americans. No other
surveys supported by the Federal Government or the private sector
provide this level of detail regarding: the health care services used by
Americans at the household level and their associated expenditures (for
families and individuals); the cost, scope, and breadth of private
health insurance coverage held by and available to the U.S. population;
and the specific services that are purchased through out-of-pocket
and/or third-party payments.

This level of detail enables public and private-sector economic models
to develop national and regional estimates of the impact of changes in
financing, coverage, and reimbursement policy and estimates of who
benefits and who bears the cost of a change in policy. No other survey
provides the foundation for estimating the impact of changes on
different economic groups or special populations of interest, such as
the poor, elderly, veterans, the uninsured, or racial/ethnic groups.

AHRQ will continue to assess the essential components of the MEPS
program -- development of new, updated, or otherwise enhanced databases;
creation of products for use by researchers and policymakers outside
AHRQ; and facilitation of the use of MEPS-related products.

**Types of Indicators:** Process and output

(Data collection begins with the treatment of common clinical conditions
over time for a nationally representative portion of the population in
support of the National Healthcare Quality Report.)

**Use of Results by** **AHRQ**:

The results provide the Agency with a thorough review of AHRQ's data
collection and development activities and release of data products and
publications associated with MEPS database. AHRQ uses the results for
the management of the program.

**Data Issues:**

Many of these indicators are yes/no indicators where the data collection
or product release happened as scheduled, or didn't. The evidence of
successful completion of the indicators will be available on the AHRQ
web site, where products can be accessed. Other data will come from
contract monitoring files. Where deadlines have been missed, the Agency
determined the cause for the delays and is making the necessary
corrections. Beginning with the FY 2000 performance report, AHRQ will
continue to include the results of evaluations of the use of the MEPS
products in Section 4.2.

  -----------------------------------------------------------------------
  **GPRA Goal 6 --FY 2001 Results**

  -----------------------------------------------------------------------

**Objective 6.1:** Release and disseminate MEPS data and information
products in

timely manner for use by researchers, policy makers, purchasers, and
plans.

**1^st^ Indicator:** In FY2001, 1997 Use and Expenditures, 2000
Point-in-Time, and

> 1998 Health Insurance and Demographics MEPS public use data files will
> be released.

**Results:** The following data files were released according to
schedule:

-   1997 Use and expenditures were released in May 2001

```{=html}
<!-- -->
```
-   1998 Health Insurance and Demographics were released in June 2001

```{=html}
<!-- -->
```
-   2000 Point-in-time was released in July 2001

```{=html}
<!-- -->
```
-   1996/97 longitudinal file was released in July 2001

**2^nd^ Indicator:** Response time for requests received for
information, assistance or

> specific products is as promised 95 percent of time.

**Results:** AHRQ continues to maintain a rapid response time to
requests forMEPS data, with 95 percent of all technical assistance
responded to within 5 working days.

**Objective 6.2**: Facilitate use of MEPS data and associated products
as tools by

extramural researchers, policy makers, purchasers, and plans.

**1^st^ Indicator:** Establish baseline for Data Center use capacity.

**Results:** Between January and July 2001, 13 data center projects were
approved. Applications are approved on a quarterly basis with the next
review cycle being held in October 2001.

**Objective 6.3:** Modify MEPS to support annual reporting on quality,
health care

disparities, and research on long-term care in adults and children with
special needs.

**Quality**

**1^st^ Indicator:** Data collection begins on the treatment of common
clinical

conditions over time for a nationally representative portion of the
population in support of the National Healthcare Quality Report.

**Results:** Beginning in February 2001, questions were added to the
MEPS on priority conditions, accountability measures, and preventive
care. An Adult self-administered questionnaire (SAQ),
parent-administered questionnaire (PAQ), and Diabetes Care Supplement
were administered to collect health status, satisfaction, and attitude
information. These data will be used in concert with the point-in-time
file to populate tables to be included in the National Quality Report.

The IOM has completed a recommended conceptual framework for the
National Quality Report. AHRQ issued a call for measures to federal
agencies to populate the framework and proposed measures have been
received. An interagency workgroup is evaluating the proposed measures
using criteria suggested by the IOM. A recommended set of measures for
the first report is scheduled to be available for vetting in the
Department and with external stakeholders in early Dec, 2001.

**LTC:**

**2^nd^ Indicator:** Have developed data use agreements (DUA) with CMS
to

assess and begin data development related to the MDS.

**Results:** The LTC enhancements in MEPS for 2001 were not funded.
However, AHRQ did modify the SSS data processing contract to procure the
statistical support necessary to start work on developing an over
sampling strategy for adults with functional limitations and children
with special needs. A Data Use Agreement (DUA) has been drafted and
accepted by HCFA/CMS for use of Minimum Data Set (MDS) data. An initial
set of data has been delivered covering 6 states (CA, OH, NY, TX, WA,
AL) representing 1,085,720 people who had at least 1 MDS assessment
during July 1999 to July 2000.

An analysis dataset is being created from this raw data to assess the
overall MDS data quality. The results of this analysis will help frame a
future agreement with HCFA/CMS regarding full access for all 50 states.
These data are being considered for use as quality indicators in the
institutional population in the AHRQ National Quality Report.

The data were received in late November, 2000. Social and Scientific
Systems, Inc., data processing contractors, started processing them in
late March, 2001. Preliminary data processing tasks have been happening
since then and construction of the analysis dataset was initiated in
July 2001. Initial analysis of the data was completed by Fall, 2001.

**3^rd^ Indicator:** Design MEPS over sample of adults with functional
limitations and

children with special needs.

**Results:** The LTC enhancements in MEPS for 2001 were not funded as a
result, this MEPS enhancement did not occur as planned.

**4^th^ Indicator:** Produce one report related to LTC.

**Results:** The final report of the Expert meeting in community-based
Long- Term Care is in its final stages of preparation. It has been sent
for review internally and is expected to be finalized by September,
2001. This expert meeting was a gathering of researchers who identified
gaps in knowledge and research priorities in the area of community-based
long-term care. The report will be a summary of the meeting and will
highlight some recommendations. It is intended to become an AHRQ
publication.

**5^th^ Indicator:** Have developed IAA with NCHS for LTC frame
development

activities.

**Results:** The LTC enhancements in MEPS for 2001 were not funded as a
result, this MEPS enhancement did not occur as planned.

**6^th^ Indicator:** Submit at least one peer-reviewed publication in
the area of LTC.

**Results:** The following article appeared in a peer-reviewed
publication:

-   Rhoades, J. A., and Altman, B.M. Personal Characteristics and
    Contextual Factors Associated with Residential Expenditures for
    Individuals with Mental Retardation. *American Association on Mental
    Retardation*, 2001;39(2):114-29.

+----------------------+-----------------------------------------------+
| **Goal 6 Objective** | **FY 2002 Indicator**                         |
+----------------------+-----------------------------------------------+
| **Objective 6.1:**   | Develop a method to facilitate users' custom  |
| Release and          |                                               |
| disseminate MEPS     | cross tabulations of MEPS data.               |
| data and information |                                               |
| products in timely   | Conduct six MEPS data user workshops.         |
| manner for use by    |                                               |
| researchers, policy  | Expand MEPS list-server participation by 20%. |
| makers, purchasers,  |                                               |
| and plans.           | Produce 4 descriptive data products (ie       |
|                      | findings,                                     |
|                      |                                               |
|                      | chartbooks, highlights, or table compendia).  |
|                      |                                               |
|                      | Develop Frequently Asked Questions Section    |
|                      | for                                           |
|                      |                                               |
|                      | MEPS web site.                                |
+----------------------+-----------------------------------------------+
| **Objective 6.2**:   | Determine the feasibility of existing         |
| Facilitate use of    | mechanisms to provide off-site access to      |
| MEPS data and        | confidential MEPS data.                       |
| associated products  |                                               |
| as tools by          | Expand data center capacity by 10% over FY 01 |
| extramural           | level.                                        |
| researchers, policy  |                                               |
| makers, purchasers,  |                                               |
| and plans.           |                                               |
+----------------------+-----------------------------------------------+

### Summary of Performance Objectives

  -----------------------------------------------------------------------
  **2.3 Budget line 3 -- Program Support**

  -----------------------------------------------------------------------

Funding Level: FY 2001 \$2,500,000 (Actual -- Current Law)

FY 2001 \$2,587,000 (Actual -- Proposed Law)

FY 2002 \$2,600,000 (Appropriation)

FY 2002 \$2,688,000 (Current Estimate)

> FY 2003 \$2,700,000 (Request -- Current Law)
>
> FY 2003 \$2,789,000 (Request --Proposed)

  -----------------------------------------------------------------------
  **GPRA Goal 7: Support the overall direction and management of AHRQ**

  -----------------------------------------------------------------------

This goal involves supporting the overall direction and management of
AHRQ through prudent acquisition performance management, capital asset
planning, personnel support and information technology planning.

**Types of Indicators:** Outcome indicators that document customer
satisfaction with two

> major functions within the Agency are used.

**Use of Results by** **AHRQ**:

The scores for each of the measures, in combination with the written
comments received in the survey, continue to be used to improve the
acquisition systems and the Intranet.

**Data Issues:**

The data collection is accomplished through customer surveys
administered to Agency staff annually.

  -----------------------------------------------------------------------
  **GPRA Goal 7 --FY 2001 Results**

  -----------------------------------------------------------------------

**Objective 7.4:** Establish and maintain a secure Agency computer
network

infrastructure.

**1^st^ Indicator:** Preliminary policies and procedures to reduce
security risks

> developed by the end of FY 2001.

**Results:** AHRQ published a final IT Security Plan on November 5,
2001. AHRQ further refined the security plan and published version 1.1
on December 27, 2001.

**2^nd^ Indicator:** Initial criteria for reporting security incidents
established by the

end of FY 2001.

**Results:** AHRQ published an Incident Response Plan documenting
criteria for reporting security incidents on November 6, 2001. The
Response Plan works in conjunction with security procedures detailed in
the Agency IT Security Plan issued on December 27, 2001.

**3^rd^ Indicator:** Initial procedures for responding to security
incidents established

by the end of FY 2001.

**Results:** AHRQ published an Incident Response Plan documenting
initial procedures for responding to security incidents on November 6,
2001

**4^th^ Indicator:** Implementation of a Secure Phase 1 LAN for analysis
of

intramural research and survey data will be completed by end of FY01.

**Results:** AHRQ has completed the implementation of Phase I of the
Secure LAN. Intramural research and survey data housed at AHRQ has been
evaluated based on NIST criteria and now resides on the Secure LAN. This
LAN is an air-gapped network which is protected by various security
measures which include, but are not limited to: media-less workstations,
Proximity Devices, Secure Printing Facility, Video Monitoring Equipment,
and Fiber from the desktop to the Secure LAN Cabinets.

**5^th^ Indicator:** Implementation of a Phase 1 firewall, intrusion
detection and virus

> control system will be in place by Dec 2001.

**Results:** AHRQ has completed the implementation of a Phase I
firewall, intrusion detection and virus control system. The following
features are operational:

-   Two Raptor Firewalls V6.5, one primary and one secondary, were
    installed and are currently in the Phase 2 hardening stage. An
    independent validation and verification of high-level security
    measures revealed \"no glaring issues at this time.\"

```{=html}
<!-- -->
```
-   Two servers running Real Secure\'s Intrusion Detection System,
    Internet Security System (ISS) V5.5, have been implemented and are
    monitoring the internal AHRQ network traffic and the external
    traffic attempting to come through AHRQ\'s firewall.

```{=html}
<!-- -->
```
-   Various virus-control systems have been implemented. The agency
    desktops are running McAfee\'s Virus Scan V4.0.3. The Exchange
    Server is running Trend Micro\'s Scan Mail 3.5.1, which scans both
    incoming and outgoing mail. McAfee\'s Netshield V4.1 is run against
    the file servers.

**6^th^ Indicator:** Initial security awareness training will begin by
Dec 2001.

**Results:** AHRQ implemented web-based IT Security Awareness Training
on December 28, 2001.

  -----------------------------------------------------------------------
  **GPRA Goal 7 -- FY 2002 Indicators**

  -----------------------------------------------------------------------

+---------------------+------------------------------------------------+
| **Goal 7            | **FY 2002 Indicator**                          |
| Objective**         |                                                |
+---------------------+------------------------------------------------+
| **02 Objective      | Perform initial tests, (periodically,          |
| 7.4.** Establish    | beginning in 2nd quarter of FY 2002) to        |
| and maintain a      | evaluate the preliminary policies and          |
| secure Agency       | procedures.                                    |
| computer network    |                                                |
| infrastructure.     |                                                |
+---------------------+------------------------------------------------+
| **02-03 Objective   | Completed reports of findings and              |
| 7.5.** Optimize     | recommendations of at least one business       |
| work processes      | process re-engineering studies.                |
| through             |                                                |
| implementation of   | AHRQ passes all items on GISRA self-assessment |
| AHRQ's workforce    | checklist                                      |
| restructuring plan. |                                                |
|                     |                                                |
| (This measure will  |                                                |
| be initiated in     |                                                |
| FY02.)              |                                                |
+---------------------+------------------------------------------------+

> **DEPARTMENT**
>
> **OF HEALTH**
>
> **AND HUMAN**
>
> **SERVICES**

**Agency for Health Care Research and Quality**

FISCAL YEAR **2003** PERFORMANCE PLAN

BUDGET AND PROGRAM AGGREGATION:

The AHRQ GPRA annual performance report and plans are aligned with the
Agency's three budget lines:

\(1\) Research on Health Care Costs, Quality, and Outcomes,

\(2\) Medical Panel Expenditure Surveys, and

\(3\) Program Support.

The first two budget lines are where Agency programs are funded. The
goals associated with each of the budget lines represent core activities
funded in each. The following two tables illustrate how the GPRA goals
are aligned with the AHRQ budget lines.

##### Beginning in FY 2003, AHRQ has redesigned our strategic management system and revised our GPRA goals to align more closely with the Agencies strategic plan. Table 2 shows this revision and realignment and pages 16 and 17 provide the FY2001-2002 framework along with the FY2003 framework.

Table 2: FY 2003 GPRA Framework

+------------------+---------------------------------------------------+
| **What the       | **GPRA Goal**                                     |
| Indicators       |                                                   |
| Address**        |                                                   |
+------------------+---------------------------------------------------+
| **Budget line 1: |                                                   |
| Research on      |                                                   |
| Health Care      |                                                   |
| Costs, Quality,  |                                                   |
| and Outcomes**   |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | To have measurable improvement in the type of     |
| 1:               | delivery system or processes by which care is     |
|                  | provided and their effects on health care         |
| Support          | outcomes.                                         |
| Improvements in  |                                                   |
| Health outcomes  |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | To have measurable improvement in the quality and |
| 2:               | safety of healthcare for Americans.               |
|                  |                                                   |
| Strengthen       |                                                   |
| Quality          |                                                   |
| Measurement and  |                                                   |
| Improvement      |                                                   |
+------------------+---------------------------------------------------+
| Strategic Goal   | To develop the evidence base for policy makers    |
| 3: Identify      | and health systems to use in making decisions     |
| Strategies to    | about what services to pay for, how to structure  |
| Improve Access,  | those services, and how those services are        |
| Foster           | accessed.                                         |
| Appropriate Use  |                                                   |
| and Reduce       |                                                   |
| Unnecessary      |                                                   |
| Expenditures     |                                                   |
+------------------+---------------------------------------------------+
| **Budget line 2: |                                                   |
| Medical Panel    |                                                   |
| Expenditure      |                                                   |
| Surveys**        |                                                   |
+------------------+---------------------------------------------------+
| Core MEPS        | To provide comprehensive, relevant and timely     |
| activities       | data on health care use and expenditures for use  |
|                  | by public and private sector decision makers and  |
|                  | researchers.                                      |
+------------------+---------------------------------------------------+

  ------------------ ----------------------------------------------------
  **Budget line 3:   
  Program Support**  

  Enhance the value  Maximize the value of AHRQ by developing efficient
  of AHRQ as the     and responsive business processes, aligning human
  leader in          capital policies and practices with AHRQ's mission,
  Healthcare         building an integrated and reliable information
  Outcomes, Quality, technology infrastructure
  Cost, Use and      
  Access research    
  ------------------ ----------------------------------------------------

##### We believe that this realignment will not only help us determine how well the basic knowledge which forms the core of AHRQ's work provides information which can be turned into actions by those who make clinical decisions, purchasers and providers who make decisions about what services to use, pay for and how to structure those services, as well as by policy makers but will allow us, in time to answer the question, "What difference has AHRQ sponsored research made?" 

2.1 Research on Health Care Costs, Quality and Outcomes

Funding Levels: FY 2001 \$226,385,000 (Actual -- Current Law)

FY 2001 \$227,897,000 (Actual -- Proposed Law)

FY 2002 \$247,645,000 (Appropriation)

FY 2002 \$249,171,000 (Current Estimate)

FY 2003 \$194,000,000 (Request -- Current Law)

FY 2003 \$195,611,000 (Request -- Proposed Law)

**2.1.1 Performance Goal: To have measurable improvement in the type of
delivery system or processes by which care is provided and their effects
on health care outcomes.**

# Program Description and Context

At its most basic level, providing high quality health care is doing the
right thing, at the right time, in the right way, for the right person.
The challenge that health care providers and health systems managers
face everyday is knowing what the right thing is, when the time is
right, and what the right way is. Patients and their families are also
faced with making choices about treatments and care settings with little
information on the relative quality, risks and benefits available to
them.

One of the most important priorities of AHRQ is to translate and
disseminate the findings of research supported by the Agency into tools
and information that can be used by its customers to make good health
care decisions and to improve the outcomes of care. The research
supported by AHRQ has historically concentrated on conditions that are
common, costly, and for which there is substantial variation in
practice. This research includes many of the conditions that represent
major expenditures for Medicare and Medicaid. AHRQ's research attempts
to reduce inappropriate variation and provide our health care decision
maker with information on what care is appropriate, which clinical
services work best in what circumstances and for which patients, how
much is enough, and what resources are used to provide it. Outcomes
research also attempts to help decision makers understand the
implications of structural and financing changes in the health care
system on the outcomes of care delivered in the system.

Outcomes and effectiveness research seeks to understand the end results
of particular health care practices and interventions. Outcomes include
effects that people experience and care about, such as change in the
ability to function. In particular, for individuals with chronic
conditions---where cure is not always possible---end results include
quality of life as well as mortality. By linking the care people get to
the outcomes they experience, outcomes research has become the key to
developing better ways to monitor and improve the quality of care.
Supporting improvements in health outcomes is a strategic goal of the
Agency for Healthcare Research and Quality.

**Strategies to Improve Health Outcomes**:

At the requested level, AHRQ's programs will assist with Secretarial
Initiatives, particularly in the area of chronic illness, such as
diabetes and asthma. Comprehensive primary care services can reduce the
morbidity associated with these illnesses. Hospital admission rates for
"ambulatory care sensitive conditions" serve as a marker for both
impaired access to primary care and suboptimal quality of primary care
delivered. Disparities in admission rates for racial and ethnic
minorities and low-income populations have been well documented.

Programs that support AHRQ's efforts to improve the outcomes include:
Centers for Education and Research on Therapeutics (CERTS),
Evidence-based Practice Centers (EPCs), U.S. Preventive Services Task
Force (USPTF); National Guidelines Clearinghouse (NGC) and Patient
Outcomes Research Teams (PORTs).

Centers for Education and Research on Therapeutics (CERTS):

The CERTS program is a national initiative designed to increase the
awareness of the benefits and risks of new, existing, or combined uses
of therapeutics through education and research. The CERT's program grew
out of recognition that physicians need more information about the
therapies they prescribe. Although information is available through the
pharmaceutical industry, continuing medical education programs,
professional organizations, and peer reviewed literature, comparative
information about the risks and benefits of new and older agents and
about drug interactions is limited.

The research conducted by the CERT's program contributes to improving
the health of Americans by increasing awareness of both the uses and
risks of new drugs and drug combinations, biological products, and
devices as well as of mechanisms to improve their safe and effective
use; providing clinical information to patients and consumers; health
care providers; pharmacists, pharmacy benefit managers, and purchasers;
health maintenance organizations and health care delivery systems;
insurers; and government agencies; and improving quality while reducing
the cost of care by increasing the appropriate use of pharmaceuticals
and biological products and devices and by preventing their adverse
effects and the consequences of these effects (such as unnecessary
hospitalizations).

***Evidence-based Practice Centers (EPC's):***

As the lead Federal agency for enhancing the quality, appropriateness,
and effectiveness of healthcare services and access to such services,
AHRQ conducts and funds research that develops and presents
evidence-based information on healthcare outcomes, quality, cost, use
and access. Included in AHRQ's legislative mandate is support of
syntheses and widespread dissemination of scientific evidence, including
dissemination of methods or systems for rating the strength of
scientific evidence. These research findings and syntheses assist
providers, clinicians, payers, patients, and policymakers in making
evidence-based decisions regarding the quality and effectiveness of
health care.

As a part of its charge to improve the quality and effectiveness of
health care through the development of state-of-the-art health care
information, and to respond to significant changes within the health
care industry, AHRQ established 12 Evidence-based Practice Centers in
1997. AHRQ has become a science partner with private and public-sector
organizations in their efforts to improve the quality, effectiveness and
appropriateness of clinical practice. Since 1997, the EPC's have
conduced more than 80 systematic reviews and analyses of scientific
literature on a wide spectrum of topics, incorporating the results and
conclusions into evidence reports and technology assessments.

The EPC program contributes to AHRQ's goal of improving the health of
the American public by producing synthesis of clinical research findings
through systematic methods for searching, reviewing, and evaluating the
clinical literature. Evidence reports and technology assessments can be
used by systems of care, professional societies, health plans, public
and private purchasers, States, and other entities as the scientific
foundation for development and implementation of their own clinical
practice guidelines, clinical pathways, review criteria, performance
measures, and other clinical quality improvement tools, as well as for
formulation of evidence-based policies related to specific health care
technologies.

U.S. Preventive Services Task Force (USPSTF):

Premature deaths and disabilities due to preventable causes continue to
extract a large toll in the United States. Health care providers and
health care organizations play an essential role in national prevention
efforts, by delivering effective vaccines, screening patients for early
disease or risk factors, counseling about health lifestyles, and
prescribing preventive medications. Despite steady progress in the
delivery of effective preventive care, important gaps remain -- as of
1998, more than one third of women over 50 had not had a mammogram and
breast exam in the last 2 years to screen for breast cancer, and more
than a third of older adults had not received a flu shot that year.
Inequities in preventive care also contribute to the disparities in the
health of specific populations, such as racial and ethnic minorities,
the elderly, and the poor and disabled.

The U.S. Preventive Services Task Force (USPSTF) and Put Prevention Into
Practice (PPIP), two major programs at AHRQ, make significant
contributes to the Department of Health and Human Services prevention
activities. The U.S Preventive Services Task Force, first convened in
1984, is charged with systematically reviewing the evidence of the
effectiveness of a wide range of clinical preventive services, including
screening tests, counseling, immunizations, and chemoprophylaxis. The
Put Prevention Into Practice program then works to translate information
from USPSTF reports into a format that meets the needs of a wide variety
of patients, clinicians, health plans, and health care purchasers.

National Guideline Clearinghouse (NGC):

The National Guideline Clearinghouse is a publicly available, web-based
database of evidence-based clinical practice guidelines and related
documents. Updated weekly with new content, the NGC is a partnership
between AHRQ, the American Medical Association (AMA) and the American
Association of Health Plans (AAHP) Foundation.

In its two years of operation, the NGC has become a resource for
physicians, nurses and other health care professionals as well as
purchasers and policy makers. Individual physicians and other clinicians
can review and use the NGC in clinical decision making and patient
counseling; health care organizations and integrated delivery systems
can use information accessible through the NGC to adopt or adapt
guidelines for their networks; medical specialty and professional
societies can use NGC resources in their own guideline development
efforts; employers and other large purchasers can use information from
the NGC to assist them in making health care benefits purchasing
decisions; educational institutions can incorporate information
accessible through the NGC into their curricula and continuing education
activities; and State and local governments can use the NGC in their
quality assurance and program oversight efforts.

Patient Outcomes Research Teams (PORT's)

Beginning in 1989, the AHRQ (formerly the Agency for Health Care Policy
and Research) funded a set of ambitious research projects known as
patient outcomes research teams (PORTs). These 14 projects were
succeeded, starting in 1993, by a \"second generation\" of projects
known as PORT-II. Together, AHRQ's PORTs and PORT-IIs represent a total
investment of more than \$100 million to answer critical questions about
the effectiveness and cost-effectiveness of available treatments for
common clinical conditions. A recent article by researchers from several
PORTs, as well as Carolyn M. Clancy, M.D., Director of AHRQ\'s Center
for Outcomes and Effectiveness Research, describes the purpose and
accomplishments of the PORTs.

The PORTs were designed to take advantage of readily available data and
to focus on common clinical conditions that are costly to the Medicare
and Medicaid programs and for which there is regional variability in
outcomes and use of resources. The PORTs are made up of a
multidisciplinary team of researchers ranging from health economists and
clinicians to quality-of-life experts and epidemiologists. PORT
investigators were instructed to answer the following questions: What
works and at what cost? For which patients or subgroups of patients?
When? Why is there variation in the use of treatments? What can be done
to reduce inappropriate variation? From whose perspective---i.e., the
patient is the ultimate judge of effectiveness? Is there a potential for
development and use of patient-reported outcome measures?

PORT researchers advanced the methods and applications of outcomes and
effectiveness research, that is, the examination of the end results of
medical interventions when applied to patients in everyday practice.
Effectiveness studies often are designed to monitor patients over time,
wherever they get their care, and to chart the outcomes of that care.
PORT researchers use Medicare and Medicaid data and data from insurance
claims and other large administrative databases to characterize patterns
of care, develop general and disease-specific outcome measures, and
disseminate important outcome information to patients and physicians to
reduce inappropriate practice variation and improve outcomes.

**FY2003 OUTCOME MEASURES FOR MEASURING IMPROVEMENTS IN HEALTH
OUTCOMES:**

The three outcome measures selected represent common problems
encountered in primary care and allow monitoring for children (asthma),
the elderly (pneumonia and influenza), and high-risk infants
(respiratory distress syndrome). For each condition there is evidence
that interventions can reduce hospitalization rates. These conditions
were chosen because reduction of morbidity can be obtained by
coordination of community preventive services, public health
interventions, clinical preventive services and primary care.

Interventions that improve access to and quality of care in population
groups at high risk for avoidable hospitalizations and coordination of
these efforts with community-based public health activities have the
potential to reduce the disparities identified here. The national goal
of a 25% reduction in avoidable hospitalizations for these conditions
can be achieved through targeting these high-risk populations. The
objective is to reduce the need for admission, and therefore the
morbidity and costs associated with hospitalization, through improved
primary care and preventive services.

##### 

##### 

##### 

##### 

##### 

##### 

##### 

##### 

##### 

##### 

##### 

**Program Outputs needed to achieve expected results**:

At the budget level proposed in the President's Budget, AHRQ will
maintain funding for Translating Research into Practice grants and
contracts. At this level, expiring grants are not continued and the
dollar amount reduction will be spread across non-patient safety and
training grants.

-   Maintain the number of synthesis reports generated on research
    findings and practical applications on Agency priority topics at FY
    2001 levels.

-   Report on impact of 5 uses of AHRQ-sponsored evidence reports,
    systematic evidence reviews, technology assessments, and the
    National Guideline Clearinghouse for which Federal or other
    government agencies, professional associations, or other private
    healthcare organizations are measuring or assessing the impact on
    clinical practice and/or patient outcomes.

-   Improve the evidence available to Federal and other direct care
    providers, professional associations, and other healthcare
    organizations to increase evidence-based practice and enhance health
    outcomes and quality.

**AHRQ will use the following indicators to report success in this
area**:

-   Using data found in HCUP, track the number of premature infants who
    develop RDS

-   Using data found in HCUP, track the number of patients 65 and older
    hospitalized for immunization-preventable pneumonia or influenza

-   Using data found in HCUP, track the number of children hospitalized
    for complications related to pediatric asthma

-   Track the number of research findings that have been translated into
    practice

-   Track the use of evidence by increasing the more effective
    partnerships with those who nominate topics for the EPCs, etc)

    2.  **Performance Goal 2: To have measurable improvement in the
        quality and safety of healthcare for Americans**

> ***2.1.2.1: Measuring Improvements in Quality Program Description
> and*** ***Context***

The United States has many of the world's finest health care
professionals, academic health centers, and other research institutions.
Patients sometimes receive excessive services that undermine the quality
of care and needlessly increase costs. At other times, they do not
receive the services that have proven to be effective at improving
health outcomes and even reducing costs.

A recent Institute of Medicine (IOM) report states "Quality problems are
everywhere, affecting many patients. Between the health care we have and
the care we could have lies not just a gap but a chasm." In this report,
the IOM proposes 13 recommendations to build a stronger healthcare
system over the next decade.

One of AHRQ's strategic goals is to strengthen the quality of healthcare
measurement and track improvements in the care available to Americans.
To achieve this goal AHRQ has invested in the development and testing of
measures of quality, as well as studies of the best ways to collect,
compare and communicate these data, and identifying and widely
disseminating effective strategies to improve quality of care. To
facilitate the use of this information in the health care system, the
Agency focuses on research that determines the most effective ways to
improve health care quality, including promoting the use of information
on quality through a variety of strategies, such as information
dissemination and assessing the impact of health care organization and
financing.

Meeting the needs of consumers, practitioners, and policymakers will
depend in part on the availability of valid and useable measures of the
quality of care. AHRQ will translate the findings from health services
research, effectiveness studies, technology assessments, and clinical
practice guidelines into quality of care measures and methods for
everyday settings. Basic research will develop more refined measures and
improvement strategies. Applied research and development will test the
validity and reliability of the measurement instruments and facilitate
their use in different population subgroups (such as minority groups,
chronically ill, disabled, elderly, and children) and care settings.
Demonstration projects will assess the use of measures and tools in
performance management systems and quality improvement activities.

**Strategies to Improve the Quality of Health Care**:

***The National Quality Report***:

The Agency's reauthorization calls for the development of a national
report on the quality of healthcare in the U.S. In developing this
report, the Agency is called on by the legislation to expand the Medical
Expenditure Panel Surveys (MEPS) to collect information on quality. In
addition, the Agency is charged with assuring coordination with the
private sector in the development of the report. In its report on AHRQ's
reauthorization, the Senate Health Education Labor and Pension Committee
provided the following guidance:

> "Beginning in FY 2003, the Secretary, acting through the Director, is
> to submit an annual report on national trends in health care quality,
> drawing upon the enhanced MEPS survey and other available data. The
> Committee expects the Agency to use a variety of measures to develop
> this annual picture of how health care quality is faring. The
> legislation directs the Agency to take into account any outcomes
> measurements generally collected by private sector accreditation
> organizations to assure that the reported information is not
> inconsistent with what is being collected through other programs. The
> committee hopes that this annual report will provide an opportunity
> for quality performance comparisons."

The Agency's coordination of the development of a National Healthcare
Quality Report is also responsive to the President's Quality Commission
report, which concluded, "The lack of comprehensive information on the
quality of American health care is unacceptable." Finally, the recent
Institute of Medicine's Commission on medical errors called for a
national report on progress on that issue. These mandates envision a
report that goes beyond a compilation of available data and statistics
to provide a framework for those public and private entities with an
interest in improving the quality and safety of patient care.

***HCUP Quality Indicators*:**

Health care decision makers need user-friendly data and tools to help
them assess the effects of health care program and policy changes,
accurately measure outcomes, community access to care, utilization and
cost of care as well as to guide future health care policy making.

To meet this need, AHRQ has developed a set of Quality Indicators (QIs),
which use hospital administrative data to highlight potential quality
concerns, identify areas that need further study and investigation, and
track changes over time. These indicators represent a refinement and
further development of the Quality Indicators developed in the early
1990's as part of the Healthcare Cost and Utilization Project (HCUP).

The AHRQ QIs are a set of quality indicators that have been organized
into three modules: Prevention, Inpatient and Patient Safety QIs.

Using these modules in FY2003, hospital and hospital systems can use
AHRQ QIs to help answer questions such as:

-   How does our hospital's cesarean section rate compare to the State
    or Nation?

-   Do other hospitals have similar mortality rates following hip
    replacement?

-   How does the volume of coronary artery bypass graft in my hospital
    compare with other hospitals?

State data organizations and community health partnerships can use AHRQ
QIs to ask questions that provide initial feedback about clinical areas
appropriate for further, more in-depth analysis, such as:

-   What can the pediatric AHRQ QIs tell us about the adequacy of
    pediatric primary care in our community?

-   How does the hysterectomy rate in our area compare with the State
    and national average?

State hospital associations can use the AHRQ QIs to do quick hospital
quality and primary care access screens. Other potential users include
managed care organizations, business-health coalitions, state data
organizations, and others poised to begin assessments using hospital
discharge data to answer questions such as:

-   Can we design community interventions in areas surrounding hospitals
    that have higher rates of diabetes complications?

-   Which Quality Indicators can be incorporated into performance
    management initiatives for our member hospitals?

Finally, Federal policymakers can use the AHRQ QIs to track health care
quality in the United States over time and to assess whether health care
quality is improving, for example:

-   How does the rate of coronary artery bypass grafts vary over time
    and across regions of the United States?

-   What is the national average for bilateral cardiac catheterization
    (a procedure generally not recommended) and how has this changed
    over time?

Translating Research Into Practice (TRIP)

Translating research findings into sustainable improvements in clinical
outcomes and patients' outcomes remains a substantial obstacle to
improving the quality of health care in America. What has been learned
in the research setting is often not incorporated into daily clinical
practice. To address this gap, in September 2000 AHRQ funded 13 new
projects to evaluate different strategies for translating research
findings into clinical practice. The aim of these 3-year cooperative
agreements is to identify sustainable and reproducible strategies to:

-   Help accelerate the impact of health services research on direct
    patient care

-   Improve the outcomes, quality, effectiveness, efficiency, and/or
    cost effectiveness of care through partnerships between health care
    organizations and researchers.

These new projects join 14 others funded in 1999 as part of a major
initiative by AHRQ to close the gap between knowledge and
practice-between what we know and what we do-to ensure continuing
improvements in the quality of the Nation's health care.

The 14 studies supported in the initial phase of the TRIP initiative
address a variety of health care problems, primarily through randomized
controlled trials. These studies, which represent important prototypes
of what is possible under ideal circumstances, generally require an
elaborate strategy for superimposing data collection on the demands of
routine practice.

The second phase of this initiative, TRIP-II is aimed at applying and
assessing strategies and methods that were developed idealized practice
settings or that are in current use but have not been evaluated.
Furthermore, increased demands for accountability in health care,
including reporting of clinical performance using standardized quality
measures, have created a sense of urgency regarding improvement within
health care organizations. With this as a basis, TRIP-II focuses on
implementation techniques and factors-such as organizational and
clinical characteristics- associated with successfully translating
research findings into diverse applied settings, including AHRQ
sponsored Practice Base Research Networks (PBRNs).

***Consumer Assessment of Health Plans (CAHPS®)***

Consumer Assessment of Health Plans makes available consumers' ratings
of the quality of care and services they get from their health plans.
This information is used by other consumers to make informed choices
among health plans, by health care purchasers -- such as employers or
Medicaid programs -- to select plans to offer their employees or
beneficiaries, and by plans for quality monitoring and improvement.
CAHPS® already has been used by more than 20 states, 10 employer groups
and a wide range of health plans and companies. For example, CAHPS® was
adopted by the Office of Personnel Management for use by the Federal
Employees Health Benefits Program to survey Federal employees and report
the findings to help about 9 million Federal Employees choose health
plans during the federal open season. CAHPS® also merged with the HEDIS
(Health Plan Employer Data and Information Set) Member Satisfaction
Survey and will be used by the National Committee for Quality Assurance
to evaluate and accredit managed care plans for 40 million Americans.
Finally, the Centers for Medicare and Medicaid Services (CMS) has used a
specially developed version of CAHPS® to survey over 130,000 Medicare
enrollees in managed care plans. The results of this survey, which was
released in February 1999, was available to help CMS's 39 million
beneficiaries who will be selecting a health plan.

These, and other successful uses of CAHPS®, is a testament to the
importance of sustained basic and applied health services research in
producing practical information for everyday health decision making.

**FY2003 Outcome Measures for Measuring Improvement in the Quality of
Healthcare:**

##### 

##### 

##### 

##### 

**Program Outputs needed to achieve expected results**:

At the budget level requested in the FY 2003 President's Budget, AHRQ
will be able to maintain grant and contract commitments which address
translating research into practice, MEPS, HCUP, CAHPS and patient
safety. Reductions will be made to grants and contracts in other areas.

-   Maintain the number of projects being funded that bring healthcare
    quality information to the public in an understandable,
    user-friendly manner that facilitates its use in decision-making.
    Baseline: FY2001

-   Identify private sector data to be used in future National Quality
    Reports

-   Identify a core set of quality measures to be used to report on
    progress in improving the overall quality of health care.

**Indicators AHRQ will use to measure success in this area over the next
two fiscal years**:

-   Track improvements in the quality of care as measured by a core set
    of measures that have been proven to be reliable and valid.

-   Track how the public utilizes quality measures

##### 2.1.2.B Program Description and Context: Measuring Improvements in Patient Safety:

Patient safety is a top priority in the nation today. It is estimated
that up to 98,000 Americans die each year as a result of preventable
medial errors (Kohn 1999). The majority of these errors are a result of
systemic problems rather than poor performance by individual providers.
Although the Untied States provides some of the best health care in the
world, there are a significant number of patients that are being harmed
as a result of the process of health care.

In Fiscal year 2001, AHRQ re-named the Center for Quality Measurement
and Improvement (CQMI) the Center for Quality Improvement and Patient
Safety (CQuiPS). This was tangible evidence of AHRQ's efforts to
re-focus activities to improve the quality of health care Americans
receive, and address preventable patient errors by reducing the risk of
patients being harmed by the process of health care.

The *goal of patient safety* is to reduce the risk of injury and harm
from preventable medical errors. This goal can be accomplished by the
removal or minimizing of hazards, which increase the risk of injury to
patients. Three steps must be followed to ensure that the "epidemic" of
medical errors is contained. These steps are:

-   *Identify* the causes of preventable errors and the hazard that
    increase the risk of injury to patients

-   *Implement* patient safety practices that eliminate known hazards
    and reduce the risk of injury to patients and create a positive
    safety culture.

-   Maintain *Vigilance* to ensure that a safe environment continues and
    a positive safety cultures are maintained.

Key phrases are risk, adverse events, no-harm events, near miss, and
detection.

*Risk* is the possibility/probability of occurrence or recurrence of an
event multiplied by the severity of the event. *Hazard* is anything that
can cause harm (NHS 2000). An event is a deviation in an activity or
technology, which leads towards an unwanted, negative consequence.
(Freitag and Hale 1997). Events can be classified in three different
categories (Battles 2001).

An *adverse event*/*misadventure* is an occurrence during clinical care
which results in physical or psychological injury or harm to a patient
or harm to the mission of the organization. A *no-harm event* is an
event that has occurred but resulted in no actual harm although the
potential for harm may have been present. Lack of harm may be due to the
robust nature of human physiology or pure luck. An example of such a
no-harm event would be the issuing of an ABO incompatible unit of blood
for a patient, but the unit was not transfused and was returned to the
blood bank.

A *near miss* as defined by Van der Schaaf (1991) is an event in which
the unwanted consequences were prevented because there was a recovery by
identification and correction of the failure, either planned or
unplanned. Heinreich (1941) developed the iceberg concept of accidents
and errors. The part of the iceberg above the water represents events
that cause major harm; below the water are no-harm events as well as
near misses.

![](media/image2.png){width="3.595833333333333in"
height="2.6958333333333333in"}

**Strategies for Improving Patient Safety**:

The overall goal of reducing medical errors and improving quality and
patient safety is an overall HHS goal, which is shared among AHRQ, Food
and Drug Administration (FDA), Center for Medicare and Medicaid Services
(CMS), HRSA and Indian Health Service (IHS). AHRQ's research
contribution focuses on creating basic knowledge and evaluating the
strength of existing evidence which provides information that can be
turned into actions by those who make clinical decisions, purchasers and
providers who make decisions about what services to use, pay for and how
to structure those services, as well as by public policy makers.

In cooperation with other DHHS partners AHRQ will seek to improve the
quality healthcare and reduce medical errors by 1) accelerating the
implementation of existing quality measures and safety practices 2)
developing capacity and new practices for Quality and Safety
Improvements and 3) detecting safety hazards and monitoring improvements
in healthcare Safety and Quality.

The Agency believes the best way to approach improving patient safety is
through an integrated set of activities to design and test best
practices for reducing errors in multiple settings of care; develop the
science base to inform these efforts, as well as to improve provider
training in the reduction of errors; capitalize on the advances in
information technology to translate proven effective strategies into
widespread practice; and build the capacity to further reduce errors in
the future. AHRQ's goal of improving the quality and safety of the
healthcare that Americans receive cannot be achieved in a single year,
but must follow a systematic progression of activities over a number of
years. In addition, AHRQ's activities should not be viewed in isolation,
but should be combined with similar efforts in other DHHS departments,
to produce the desired improvements in quality and safety.

In FY 2003, activities will build on the progress made in FY '01 and '02
and activities directed at stage two of the medical error epidemic will
be added to the portfolio, with a primary focus on minimizing the risk
of preventable injuries.

***Detecting safety hazards and monitoring improvement in Safety and
Quality*:**

*Detection* is the first step in error management. From an
organizational point of view it is important that error detection rate
be high because errors that are not detected can have disastrous
consequences (Zapt & Reason 1994). Thus one of the goals of error
management is to increase detection and reporting rates to decrease risk
of harm to patients. *Detection Sensitivity Level (DSL*) is the number
of events reported to an organization*.* High reporting rates indicate a
high DSL while few events reported indicates a low DSL. To achieve a
high DSL an organization must eliminate impediments to reporting. *Event
Severity Level (ESL)* is the level risk of the events reported. Over
time the event severity level should go down as an indicator of
successful error management while the detection sensitivity level (DSL)
remains high (Kaplan 1998). DSL represents information while ESL
represents risk.

###### Risk

![](media/image3.wmf){width="2.2338582677165353in"
height="1.043700787401575in"}

**Information**

Developing a system which will consistently and reliably identify
potential hazards to patient safety will require developing a national
system of patient safety reporting and monitoring. For example, existing
data collection mechanisms, if properly coordinated and standardized,
could provide a powerful national asset as an early warning system for
patient safety hazards, providing pooled data for analysis to develop
solutions to patient safety problems, and for tracking progress in
patient safety.

In addition, a successful monitoring system will require a direct
investment in IT infrastructure and enhancements to improve safety
monitoring. Healthcare is well behind other industries in harnessing the
potential of information technology. Developing the healthcare IT
infrastructure is key to monitoring the improvements in patient safety.
The development of options and information to support investments in IT
infrastructure, including cooperative agreements to encourage IT
investment and develop national models of IT infrastructure.

Finally, developing data standards and vocabulary to ensure that patient
safety information can be pooled, analyzed, and used to identify hazards
and safety practices will be essential. The lack of standardized coding
and language could result in a healthcare information system "Tower of
Babel". This initiative, in conjunction with the Department and sister
agencies, will strive toward the development of patient safety
information standards, development of voluntary consensus on those
standards, and incorporation of those standards into existing
Departmental databases.

***Accelerate the implementation of existing patient safety
practices*:**

In July 2001, the Agency released a report, entitled Making Health Care
Safer: A Critical Analysis of Patient Safety Practices, which examined
the evidence supporting 79 promising safety interventions. Many of these
proven lifesaving interventions, such as the use of medications to
reduce the risk of heart attacks and infections in surgical patients,
are not consistently implemented.

Key to developing and implementing this strategy will be implementing
local safety improvement priorities and improving the information
available to the public on patient safety. Healthcare organizations and
systems must be challenged to implement proven patient safety practices.
Participating organizations and systems should be encouraged develop
their own practice implementation practices from those identified in the
Agencies FY 2001 report and will be provided funding to facilitate the
system's investment in implementing those practices. Finally, these
organizations and systems will be required to generate information on
the barriers faced in implementing the practices as well as measure the
impact of the intervention on patient safety. Cooperative agreements
will include a provision to make continued funding of the program
contingent upon a demonstrated ability to implement the identified
priorities and measure their impact.

In addition, to order to improve the information available to the public
on patient safety the Agency will use its Consumer Assessment of Health
Plans Survey (CAHPS) program, which is responsible for the development
of state of the art survey and reporting instruments, to provide
information about patient's experience of care; to improve the relevance
of systems based information on patient safety for the public; support
value-based purchasing initiatives; and develop mechanisms for public
reporting on evidence-based safety interventions.

***Developing Capacity and New Practices for Safety Improvement*:**

The Agency's July 2001 report on patient safety interventions identified
a number of promising interventions for which more effectiveness
evidence is required prior to general implementation. Many potential
patient safety practices drawn primarily from non-medical fields, such
as the use of simulators, bar coding, "swipe" technology, and crew
resource management, require additional study to clarify their value in
the health care environment. The success of this strategy will depend on
the continuing development and evaluation of new safety interventions,
improving local patient safety infrastructure and capacity, and
developing reliable information for purchasers and the public to support
wise decision-making.

**FY2003 OUTCOME MEASURES FOR MEASURING IMPROVEMENT IN HEALTHCARE
SAFETY:**

AHRQ is committed to identifying ways of continually improving patient
safety in an environment of high quality health care. Its investments
sponsor research that helps to measure the overall safety culture within
health care institutions. The agency's performance goals and measures
will be used to gauge progress as advances are made toward increasing
event reporting and reducing risk of harm.

**Program Outputs needed to achieve expected results**:

At the budget level proposed in the President's Budget AHRQ will
maintain funding for our Translating Research into Practice grants and
contracts. At this level expiring grants are not continued and the
dollar amount reduction will be spread across non-patient safety and
training grants.

-   Invest \$3 million in new activities designed implement local safety
    improvements priorities by providing incentives to put systems-based
    interventions in place in healthcare organizations.

-   Invest \$2 million in new programs designed to provide technical
    assistance to states and provide on-site patient safety experts to
    improve local capacity.

-   Produce at least one synthesis of research findings and practical
    applications of AHRQ's research which will be available to clinical
    decision makers, health systems and policy makers

```{=html}
<!-- -->
```
-   Report the interim data on the extent the patient safety best
    practices identified in the July 2001 EPC report have been adopted
    by health care institution

-   Identify the number and types of adverse events, no-harm events, and
    near miss events reported in demonstration projects (Baseline
    established)

**Indicators AHRQ will use to measure success in this area**:

-   Track the number and types of adverse events, no-harm events, and
    near miss events reported.

-   Track the number of patient safety interventions being put into
    practice.

***External Factors influencing success: AHRQ's ability to meet its
target goals is dependent upon complementary programs being implemented
in CMS and OASPE.***

**2.1.3** **Performance Goal 3: To develop the evidence base for policy
makers and health systems to use in making decisions about what services
to pay for, how to structure those services, and how those services are
used and accessed.**

# Program Description and Context

**2.1.3.1: Develop Strategies for Making Decisions**

Adequate access to health care services continues to be a challenge for
many Americans. This is particularly true for the poor, the uninsured,
and members of minority groups, rural residents, and other priority
populations. In addition, the changing organization and financing of
care has raised new questions about access to a range of health
services, including emergency and specialty care. At the same time,
examples of inappropriate care, including over utilization and misuse of
services, continue to be documented. Through ongoing development of
nationally representative and more specialized databases, the production
of public-use data products, and research and analyses conducted by AHRQ
staff and outside researchers, the Agency addresses critical policy
issues pertaining to the access, cost and use of health care.

Research related to the effects of payment and organizational structures
and processes on the cost, quality and equity of health care services
are essential components of this research initiative. Research results
are intended to 1) improve clinical practice, 2) improve the health care
system\'s ability to provide access to and deliver high quality,
high-value health care, and 3) provide policymakers with the ability to
assess the impact of payment and organizational changes on outcomes,
quality, access, cost, and use of health care services.

Responding to the Institute of Medicine's (IOM) report, "Crossing the
Quality Chasm," AHRQ will fund applications that provide rigorous,
objective, and essential evidence required by public and private
decision-makers seeking to understand and improve the health care
system, to make changes in health care delivery, insurance, and
financing, and to manage the system in a manner that would induce
efficient, effective, equitable, accessible and timely health care.

Important issues to be addressed by such research include: 1) How do
different payment methodologies and financial incentives within the
health care system affect health care quality, costs, and access? a) How
do payment methodologies affect the behavior of health care
organizations and individual providers? b) Which payment arrangements
among patients, providers, and health plans enhance patient-centered
knowledge of and involvement with treatment regimens? c) How do payment
policies affect decisions about the purchase and selection of health
services and health insurance? What is the

role of quality in such decisions? What are the effects of such
decisions on health care costs? 2) What has been the impact of purchaser
and public sector initiatives on quality, costs, and access to health
care and health insurance? Of particular interest would be the impact of
employer and coalition efforts on the quality and cost-effectiveness of
care in the marketplace, the impact of State efforts to monitor and
improve access and quality, and the impact of public and private payment
changes on access to health care and to health insurance for vulnerable
populations. 3) What organizational structures and processes are most
likely to sustain high-quality, efficient, effective, timely, and
accessible health care? 4) How do different patterns and levels of
market competition affect the quality and cost of care?

In addition, AHRQ will fund projects in basic methodological work to
support such research, including: development of payment methodologies;
improvements in analytical and empirical methods required to
simultaneously address issues of efficiency, quality, and equity; and
improvement in data collection methods and qualitative methods needed to
understand the structure of new health care organizations and an
evolving health care system.

**Strategies to Improve Access, Foster Appropriate Use and Reduce
Unnecessary Expenditures:**

***Excellence Centers to Eliminate Ethnic/Racial Disparities (EXCEED)***

The health of Americans has improved over the past few decades, but not
all Americans have benefited equally. Many ethnic and racial groups have
not shared in the advances in health outcomes and health care. To
understand the causes and factors of these inequalities, in September
2000, AHRQ awarded grants to nine Centers to conduct a series of related
studies. Now known as "Excellence Centers to Eliminate Ethnic/Racial
Disparities" (EXCEED), the grants bring together teams of both new and
experienced investigators in a 5-year effort to analyze underlying
causes and contributing factors for racial and ethnic disparities in
health care and to identify and implement strategies for reducing and
eliminating them. The Centers of Excellence approach enables assembly of
a critical mass of investigators to address a group of projects linked
by a central theme, such as communication or cultural competency. In
addition, the Centers are able to train new investigators with an
interest in minority health services research.

The EXCEED program joins a series of AHRQ initiatives aimed at improving
the health and health care of priority populations and eliminating
racial and ethnic disparities in health outcomes and in health care
access and service delivery. AHRQ expects that lessons learned from
understanding and eliminating racial and ethnic disparities in health
and health care as well as the practical tools and strategies to
eliminate these disparities would be generalizable beyond the
communities studied. By focusing on the attributes of the ethnic and
racial groups, the underlying etiologies for the disparities, and
components and conditions of interventions to eliminate the disparities,
these projects should produce findings that are widely applicable for
minority as well as majority populations across the county.

Integrated Delivery System Research Networks (IDSRNs)

The Integrated Delivery System Research Networks (IDSRN) is a new model
of field-based research that links the Nations top researchers with some
of the largest health care systems to conduct research on cutting edge
issues in health care on an accelerated timetable.

Designed by AHRQ, the IDSRN was developed explicitly to capitalize on
the research capacity of, and research opportunities occurring within,
integrated delivery systems. The network creates, supports, and
disseminates scientific evidence about what works, and what does not
work in terms of data and measurement systems and organizational "best
practices" related to care delivery and research diffusion. It also
provides a cadre of delivery-affiliated researchers and sites to test
ways to adapt and apply existing knowledge.

As a group, the IDSRN provides health services in a wide variety of
organizational care settings to over 34 million Americans, including the
privately insured, Medicare and Medicaid patients, the uninsured, ethnic
and racial minorities, and rural and inner-city residents. Moreover,
each of the nine IDSRN partners have the following three unique
attributes that make it particularly well suited for conducting
time-sensitive research:

-   Available Data -- each of the IDSRN partners collect and maintain
    administrative, claims, encounter and other data on large
    populations that are clinically, demographically and geographically
    diverse

-   Research Experience -- IDSRN partners include some of the country's
    leading health services researchers, with expertise in quantitative
    and qualitative methodologies as well as first hand knowledge of
    emerging issue areas which have critical policy or managerial
    implications

-   Management authority to implement health care interventions -- IDSRN
    partners have responsibility for managing delivery systems and are
    in a position to implement financial and organizations strategies
    (e.g., selective contracting with physicians and hospitals for
    specific approaches to diffuse clinical guidelines within the
    system) with an evaluation component.

Primary Care Based Research Networks (PBRNs).

Over the past decade, primary care practice-based research networks
(PBRNs) have emerged as a promising approach to the scientific study of
primary care. A PBRN is a group of ambulatory practices devoted
principally to the primary care of patients, affiliated with each other
(and often with an academic or professional organization) in order to
investigate questions related to community-based practice. Typically,
PBRNs draw on the experience and insight of practicing clinicians to
identify and frame research questions whose answers can improve the
practice of primary care. By linking these questions with rigorous
research methods, the PBRN can produce research findings that are
immediately relevant to the clinician and, in theory, more easily
assimilated into everyday practice. In addition, data is readily
available which will provide useful information on who receives care,
where and how often.

In September 2001, 18 of these networks were awarded continuation grants
(cooperative agreements) to conduct network-defining surveys, using
standardized instruments. The surveys will provide baseline data on the
clinicians enrolled in each network, the services provided, and the
characteristics of patients receiving those services. Potential uses of
the data include practice benchmarking and guiding the selection and
design of specific PBRN research projects. Four of the networks were
awarded additional funds to pilot test and evaluate electronic methods
of collecting and aggregating practice-derived research data. Two
networks were awarded additional funds to assess clinician and patient
knowledge and attitudes about protecting the privacy and confidentiality
of research data.

AHRQ\'s overall goal is to improve the capacity of PBRNs to expand the
primary care knowledge base and to establish mechanisms that will assure
that new knowledge is incorporated into actual practice and its impact
is assessed. Additional information on PBRNs is available at:
<http://www.ahrq.gov/research/pbrnfact.htm>

## HIV Research Network

As HIV/AIDS spreads into different communities and as new therapies
become available, policymakers require reliable information about the
type and costs of the health care services that persons with HIV disease
are receiving, so that informed resource-allocation decisions can be
made. The HIV Cost and Services Utilization Study (HCSUS) is the first
major research effort to collect information on a nationally
representative sample of people in care for HIV infection. HCSUS is
examining costs of care, utilization of a wide array of services, access
to care, quality of care, quality of life, unmet needs for medical and
nonmedical services, social support, satisfaction with medical care, and
knowledge of HIV therapies.

HCSUS is composed of a core study and several supplemental studies. The
core study has enrolled a national probability sample of 2,864
HIV-infected adults who were receiving ongoing or regular medical care
in the first 2 months of 1996. Respondents were sampled from 28 urban
areas and 24 clusters of rural counties in the continental United
States. Patients receiving services in hospitals, clinics, and private
practice settings were enrolled. HCSUS oversampled women and members of
staff model health maintenance organizations, to obtain more precise
information on these specific populations. Supplemental studies are
examining HIV care delivery in rural areas, prevalence of mental and
substance abuse disorders, oral health of HIV-positive individuals, and
issues related to HIV-infected persons over 50 years of age.

HCSUS is funded through a cooperative agreement between the Agency for
Health Care Policy Research (AHCPR) and RAND---a private nonprofit
research institution in Santa Monica, CA. Supplemental funding has been
provided by the [Health Resources and Services
Administration](http://www.hrsa.dhhs.gov/) (HRSA), the [National
Institute of Mental Health](http://www.nimh.nih.gov/) (NIMH), the
[National Institute of Aging](http://www.nih.gov/nia/) (NIA), the
[National Institute of Drug Abuse](http://www.nida.nih.gov/) (NIDA), the
[National Institute of Dental and Craniofacial Research
(NIDCR)](http://www.nidr.nih.gov/) (NIDR), and the [Office of Minority
Health Research](http://www.omhrc.gov/) (OMR). A consortium of private
and Government institutions, centered at RAND, are conducting HCSUS.
Local and national advisory groups have been established to facilitate
communication between the HIV community and the research consortium.

In Fiscal Year 1999 and 2000, AHRQ and sister agencies (HRSA, SAMHSA,
and the Office of AIDS Research at NIH) provided support to develop the
first phase of the HIV Resource Utilization Data Coordinating Center
(DCC). In Fiscal Year 2001, based on the promising results of the first
phase, support from these agencies has continued, and plans have been
made to expand the scope of this project.

The goal of this project is to provide timely and relevant information
to policymakers about the resources utilized to treat persons with HIV
disease. It is important to obtain timely data because quickly occurring
therapeutic advances, the shifting epidemiology of HIV infection, and
the rapid diffusion of managed care are having a major effect on
patterns of health care resource utilization for HIV disease. More than
\$7 billion spent each year by Medicaid, Medicare, the Department of
Veterans Affairs, and the Ryan White CARE Act to treat people with HIV
disease. Information is needed by Federal and State policymakers to
ensure that all persons with HIV disease have access to appropriate
health care services.

The mechanism developed to address these issues is a multi-site HIV
provider network, which is linked by a centralized data-coordinating
center. The Data Coordinating Center (DCC) receives data from a large
number of HIV providers around the nation in order to provide valid and
reliable information about the determinants of resource utilization by
persons with HIV disease. The Data Coordinating Center has the
capability to receive, consolidate, analyze, and disseminate information
about the care provided to persons with HIV disease, eventually
operating in a continuous data receipt mode.

Healthcare Cost and Utilization Project (HCUP)

The unprecedented volume and pace of change in the U.S. health care
system, and the fact that changes are not occurring uniformly across the
country, require a new information paradigm in which scientifically
sound, standardized databases and tools for using them are made
available at the national, regional, and State levels.

The [Healthcare Cost and Utilization
Project](http://www.ahrq.gov/data/hcup/.) (HCUP) is a
Federal-State-industry partnership to build a standardized, multi-State
health data system. HCUP is maintained by the Agency for Healthcare
Research and Quality (AHRQ, formerly the Agency for Health Care Policy
and Research). AHRQ has taken the lead in developing HCUP databases,
Web-based products, and software tools and making them publicly
available.

HCUP comprises a family of administrative longitudinal
databases---including State-specific hospital-discharge databases and a
national sample of discharges from community hospitals---and powerful,
user-friendly software that can be used with both HCUP data and with
other administrative databases*.*

HCUP databases fill a unique niche and are increasingly being tapped by
analysts and researchers interested in hospital utilization, access,
charges, quality, and outcomes. The data are used to:

-   Describe patterns of care for uncommon as well as common diseases.

-   Analyze hospital procedures.

-   Study the care of population subgroups such as minorities, children,
    women, and the uninsured.

Researchers and policymakers use HCUP data to identify, track, analyze,
and compare trends at the national, regional, and State levels.

**FY2003 OUTCOME MEASURES FOR MEASURING HEALTHCARE COST, USE, AND
ACCESS:**

**Program Outputs needed to achieve expected results:**

At the budget level proposed in the President's Budget, AHRQ will
maintain funding for Translating Research into Practice grants, HCUP
grants and contracts, patient safety and MEPS. At this level, expiring
grants are not continued and the dollar amount reduction will be spread
across non-patients safety and training grants. At this level AHRQ will:

-   Maintain funding for grant and contract commitments supporting the
    Healthcare Cost and Utilization Project

**Indicators AHRQ will use to measure success in this area**:

-   The number of IDSRN finding used by health plans and others

-   The number of HCUP journal articles or policy tools which use HCUP
    data

-   The number of HIV research network findings being used by Network
    providers to improve the delivery of care

2.1.4 Performance Goal: Improving the Quality of Health Care Delivery
through Research and Training:

# 

# Program Description and Context: Research and Training

Today's health environment continues to change rapidly. Consolidation of
health plans, movement of patients and providers into managed care
settings, efforts to contain rising health care costs, fears that cost
containment measures will lower the quality of care, and persistent
problems in access to care and health insurance coverage for many
Americans are issues of great concern.

Public and private purchasers are experimenting with new, untested
financing strategies, organizational arrangements, and delivery
approaches. Health plans and providers are seeking to measure and
improve the effectiveness and cost-effectiveness of the care they
purchase or provide.

Decisionmakers at all levels in the health care system rely on this
critical knowledge to inform effective choices. For example:

-   Purchasers are looking for value, for high-quality care at a
    reasonable cost

-   Patients and caregivers want to make informed decisions about
    preventive and other primary care services, treatments, providers,
    and health care plans.

-   Clinicians need information to make the best possible decisions for
    and with their patients.

-   Health plans need information to determine which services to cover.

-   Institutional providers (hospitals, groups, and systems of care)
    need information to make decisions.

-   Policymakers need to understand the ramifications of available
    policy options.

Health services research addresses these issues to provide information
to help people make decisions at the clinical, system and policy levels.
To ensure there are adequate numbers of highly qualified scholars to
address emerging needs in delivery of high-quality health care, AHRQ
continues its commitment to support the research education of future
leaders in the field through research and institutional training
programs. In order to achieve this goal, concerted attention is placed
on nurturing the development of a strong, visible, and integrated cadre
of health services researchers. Emphasis is given to supporting the
education and career development of new investigators through a variety
of individual and institutional grant programs. Combining didactic
training with experiential research opportunities, these initiatives
produce both a cadre of researchers and a body of research that address
issues related to the cost and financing of health care, access to
health care, quality and outcomes of health care, and the translation of
clinical research into health care policy and practice geared toward the
provision of effective, safe health care.

**Strategies to Improve the Quality of Health Care Delivery through
Research and Training**:

***National Research Service Awards (NRSAs)***:

AHRQ supports 24 U.S. academic institutions that offer advanced training
to people with a strong interest in health services research who want to
prepare for careers in the organization, provision and financing of
health care services. These programs provide tuition support and
stipends to highly qualified predoctoral and postdoctoral candidates.
The goals of the research education programs are to help ensure that
there will be enough well-trained health service researchers to improve
quality, assure value for health dollars, and enhance access to services
and to equip scholars with the necessary knowledge, skills and
experience to conduct research that will meet the evolving needs of
patients, providers, health care plans, purchasers, and/or
policy-makers.

Dissertation Research Grants:

Dissertation research funding is available for students conducting
doctoral-level research on some aspect of the health care system. These
grant awards are often the first step toward establishing a career in
health services research.

## Career Development Awards

AHRQ began to support career development activities in FY 2000. These
awards, provided to individuals embarking on a research career, allow
individuals time and resources to gain experience in carrying out actual
research. The intent is to provide transitional support for newly
trained investigators in order to launch them on research careers. This
program will nurture the next generation of health services researchers.
AHRQ supports two career development programs:

-   [Mentored Clinical Scientist Development Award (K08) in Health
    Services Research]{.underline}. This research award provides support
    for the development of outstanding clinician research scientists who
    are committed to a career in health services research, with a focus
    on development as an independent scientist.

-   [Independent Scientist Award (K02) in Health Services
    Research]{.underline}. The K02 provides support for newly
    independent investigators with a clinical or research doctoral
    degree to enable them to develop their research careers.

Minority Research Infrastructure Support Program (M-RISP)

The Agency is committed to the Department's Initiative to Eliminate
Racial and Ethnic Disparities in Health and the complementary Healthy
People 2010 Goal to eliminate disparities in health. A critical
component in achieving these goals is to expand the nation's health
workforce to be more diverse and representative of the racial and ethnic
populations in America. This includes bringing needed diversity to the
health services research workforce.

In FY 2001, AHRQ launched the Minority Research Infrastructure Support
Program (M-RISP). The goals of the M-RISP program are to increase the
number of minority health services researchers, and to build capacity
for institutions to conduct health services research intended to improve
health for racial and ethnic minorities.

Building Research Infrastructure & Capacity Program (BRIC)

In order to build research capacity in States that have not
traditionally been involved in health service research, the Building
Research Infrastructure & Capacity Program (BRIC), AHRQ has funded four
two-year planning grants to pilot test the feasibility of developing a
new program to broaden the geographic distribution of AHRQ funding and
enhance the competitiveness for research funding of institutions located
in States that have a low success rate for grant applications from AHRQ.
These grants will stimulate sustainable improvements in capacity and/or
multi-disciplinary centers supporting investigators and multiple
research projects with a thematic focus.

FY2003 OUTCOME MEASURES FOR MEASURING HEALTHCARE RESEARCH AND TRAINING:

**Program Outputs needed to achieve expected results**:

At the budget level requested in the FY 2003 Presidents Budget, AHRQ
will be able to maintain only those grant and contract commitments which
address translating research into practice, MEPS, HCUP, CAHPS and
patient safety. Reductions will be made to ongoing grant and contract
commitments in the research and training portfolio, however at the
requested budget level AHRQ will:

-   Maintain NRSA grants at the level of the NIH set-aside

-   Reduce funding to current training programs. The exact amount and
    form the reductions take will be determined in FY 2002 and final
    outputs will be conveyed in the Agency's '03 Final Performance Plan.

**2.2 Budget Line (2) \-- Medical Expenditure Panel Surveys (MEPS)**

Funding Levels: FY 2001 \$40,850,000 (Actual -- Current Law)

FY 2001 \$40,850,000 (Actual -- Proposed Law)

FY 2002 \$48,500,000 (Appropriation)

FY 2002 \$48,500,000 (Current Estimate)

FY 2003 \$53,300,000 (Request -- Current Law)

FY 2003 \$53,300,000 (Request -- Proposed Law)

**2.2.1:** **Performance Goal: To provide comprehensive, relevant and
timely data on health care use and expenditures for use by public and
private sector decision makers and researchers.**

**2.2.1.1: Strengthen Core MEPS Activities**

The Medical Expenditure Panel Survey (MEPS) is designed to continually
provide policymakers, health care administrators, businesses, and others
with timely, comprehensive information about health care use and costs
in the United States, and to improve the accuracy of their economic
projections.

MEPS collects data on the specific health services that Americans use,
how frequently they use them, the cost of these services and how they
are paid for, as well as data on the cost, scope, and breadth of private
health insurance held by and available to the U.S. population.

MEPS is unparalleled for the degree of detail in its data, as well as
its ability to link data on health services spending and health
insurance to the demographic, employment, economic, health status, and
other characteristics of survey respondents. Moreover, MEPS is the only
national survey that provides a foundation for estimating the impact of
changes in sources of payment and insurance coverage on different
economic groups or special populations of interest, such as the poor,
elderly, families, veterans, the uninsured, and racial and ethnic
minorities.

MEPS is designed to help understand how the dramatic growth of managed
care, changes in private health insurance, and other dynamics of
today\'s market-driven health care delivery system have affected, and
are likely to affect, the kinds, amounts, and costs of health care that
Americans use. MEPS also is necessary for projecting who benefits from,
and who bears the cost of, changes to existing health policy and the
creation of new policies.

AHRQ medical expenditure survey data have been used by:

-   The Health Care Financing Administration and other components of the
    Department of Health and Human Services, the Congressional Budget
    Office, Office of Management and Budget, Department of the Treasury,
    Physician Payment Review Commission, Prospective Payment Assessment
    Commission, and other Federal Government agencies.

-   The Heritage Foundation, Lewin-VHI, Urban Institute, RAND
    Corporation, Project Hope, and other foundations and
    \"think-tanks.\"

-   Health insurance companies, pharmaceutical firms, health care
    consultants, and other health-related businesses.

-   Academic institutions and individual researchers.

MEPS provides answers to hundreds of questions, including:

-   How health care use and spending vary among different sectors of the
    population, such as the elderly, veterans, children, disabled
    persons, minorities, the poor, and the uninsured.

-   How the health insurance of households varies by demographics,
    employment status and characteristics, geographic locale, and other
    factors.

MEPS also answers key questions about private health insurance costs and
coverage, such as how employers\' costs vary by region. The answers to
these and other MEPS questions enable Congress, the Federal
Government\'s executive branch, and other public- and private sector
policymakers to:

-   Make timely national estimates of individual and family health care
    use and spending, private and public health insurance coverage, and
    the availability, costs, and scope of private health insurance among
    Americans.

-   Evaluate the growing impact of managed care and of enrollment in
    different types of managed care plans.

-   Examine the effects of changes in how chronic care and disability
    are managed and financed.

-   Assess the impact of changes in employer-supported health insurance.

-   Evaluate the impact of changes in Federal and State health care
    policies.

-   Examine access to and the costs of health care for common diseases
    and conditions, prescription drug use, and other health care issues.

**Strategies to Improve Information available to Decisionmakers**:

### Household Component (HC)

The HC collects data on approximately 10,000 families and 24,000
individuals across the Nation, drawn from a nationally representative
subsample of households that participated in the prior year\'s NCHS
National Health Interview Survey.

The objective is to produce annual estimates for a variety of measures
of health status, health insurance coverage, health care use and
expenditures, and sources of payment for health services. These data are
particularly important because statisticians and researchers use them to
generalize to people in the civilian noninstitutionalized population of
the United States, as well as to conduct research in which the family is
the unit of analysis.

The panel design of the survey, which features several rounds of
interviewing covering 2 full calendar years, makes it possible to
determine how changes in respondents\' health status, income,
employment, eligibility for public and private insurance coverage, use
of services, and payment for care are related.

Because the data are comparable to those from earlier medical
expenditure surveys, it is possible to analyze long-term trends. The
information from the Household Component also permits analysts and
policymakers to address the health care policy issues facing the Nation
in the 21st century.

### 

### Medical Provider Component (MPC)

The MPC covers approximately 3,000 hospitals, nearly 17,000 physicians,
and 500 home health care providers. Its purpose is to supplement
information received from respondents to the MEPS HC. The MPC also
collects additional information that can be used to estimate the
expenses of people enrolled in health maintenance organizations and
other types of managed care plans.

### Insurance Component (IC)

The IC consists of two subcomponents, the household sample and the list
sample. The household sample collects detailed information on the health
insurance held by and offered to respondents to the MEPS HC. The number
of employers and union officials interviewed varies from year to year,
as the number of respondents in the previous year\'s HC varies. These
data, when linked back to the original household respondent, allow for
the analysis of individual behavior and choices made with respect to
health care use and spending.

The list sample consists of a sample of approximately 40,000 business
establishments and governments throughout the United States. From this
survey, national, regional, and State-level estimates (for approximately
40 States each year) can be made of the amount, types, and costs of
health insurance available to Americans through their workplace.

**FY2003 OUTCOME MEASURES FOR MEDICAL EXPENDITURE PANEL SURVEY:**

**Program Outputs needed to achieve expected results**:

At the budget level requested in the FY 2003 Presidents Budget, AHRQ
will:

-   Develop and maintain MEPS-NET

-   Conduct 8 MEPS user workshops.

-   Expand MEPS list-server participation by 20%.

-   Produce 8 Descriptive Data Products (i.e. findings, chartbooks,
    highlights, or table compendia).

-   Update frequently asked question section of MEPS Web site.

-   In concert with ULP, develop an IC module for a workshop to promote
    the use of MEPS-IC data to state policymakers.

-   Increase by 15% the number of MEPS-based articles published in peer
    reviewed journals relative to baseline year (FY02).

-   Determine the feasibility of existing mechanisms to provide off-site
    access to confidential MEPS data by conducting a review of
    arrangements other statistical agencies have implemented.

-   Expand data center capacity by 10% over FY 02 level.

-   Modify MEPS to support annual reporting on quality, health care
    disparities, and research on long-term care in adults and children
    with special needs.

**Indicators AHRQ will use to measure success in this area**:

-   The number of attendees at MEPS user workshops

-   The number of policy makers who use MEPS analysis in policy
    decisions

-   Satisfaction ratings of participants in MEPS workshops

-   Data Center usage

    3.  **Budget line 3 -- Program Support**

Funding Level: FY 2001 \$2,500,000 (Actual -- Current Law)

FY 2001 \$2,587,000 (Actual -- Proposed Law)

FY 2002 \$2,600,000 (Appropriation)

FY 2002 \$2,688,000 (Current Estimate)

> FY 2003 \$2,700,000 (Request -- Current Law)
>
> FY 2003 \$2,789,000 (Request --Proposed)

**2.3.1: Performance Goal: Maximize the value of AHRQ by developing
efficient and responsive business processes, strategic management of
human capital by aligning human capital policies and practices with
AHRQ's mission, building an integrated and reliable information
technology infrastructure**

**2.3.1.1: Enhance Efforts on Health Outcomes, Quality, Cost, Use and
Access**

In FY 2000, AHRQ conducted an Agency-wide workforce and workload
analysis that identified major Agency work processes and functions,
captured data on the competencies of the current workforce and estimated
levels of future workforce capacities. In FY 2001, the Agency began
following up on the 2000 Study in four areas: 1) a refinement and
expansion of Agency work functions and activities that captures more
detailed administrative and operational processes; 2) refinement of the
staff competencies needed to perform the work of the Agency; 3) review
of the Agency's recruitment and selection activities and processes in
order to make recommendations for improvement; 4) the development of a
multi-year strategy to systematically address the Agency's structure,
technology and workforce related issues.

**Strategies to Maximize the Value of AHRQ**

## Developing efficient and responsive business processes

The Agency's five-year workforce restructuring plan, as approved by the
Office of the Secretary, focuses on periodic analysis of the manner in
which the Agency conducts its work and how well the existing structures,
technology, and systems support this work.

-   Develop and implement a plan for identifying, obtaining, storing and
    using programmatic and operations management and performance data to
    help inform resource allocation decisions.

-   Prioritize established work processes and develop a plan to conduct
    business process reviews. These reviews will be designed to ensure
    that the following issues are addressed: Are there unnecessary
    managerial and organizational levels?; Is decision making authority
    appropriately delegated?; and Do managers have sufficient authority
    to carry out their responsibilities and manage their programs.

-   Initiate a business process review of the AHRQ grants process from
    conception of a grant announcement to tracking of publications
    coming from completed grants with the goal being to make this
    process more effective and efficient.

***Strategic Management of Human Capital*:**

In FY 2000, AHRQ conducted an Agency-wide workforce analysis that
identified major Agency work processes and functions, captured data on
the competencies of the current workforce, and estimated future
workforce requirements based on projected workload, retirement, and
attrition trends. In FY 2001, the Agency is building on the outcomes of
the 2000 Study by continuing to refine their workforce planning tools
and conduct additional organizational assessments through four
initiatives: 1) development of a more detailed model of the Agency\'s
work that captures the administrative, operational, and programmatic
functions and activities; 2) refinement of the core, technical, and
leadership competencies needed to perform functions and activities
outlined in the model of the Agency\'s work; 3) review of the Agency's
recruitment and selection processes to include recommendations for
improvements and; 4) development of a multi-year strategy designed to
identify resources, tools, and information needed to meet the changing
demands of their work, as well as streamline costs.

The first initiative in FY 2001 involved the development of a detailed
model of AHRQ\'s functions and activities to outline AHRQ\'s current
work. The agency will use this model as an assessment tool for such
things as future workload analyses and assessment of gaps in work
activities.

In the second initiative, the competencies identified in the Spring 2000
effort are currently being refined and validated to create competency
models based on the more detailed functions and activities model. These
models will assist the agency in recruiting, selection, training,
development, and performance assessment activities.

The third initiative, the recruiting and selection process assessment,
benchmarking, and recommendations task, resulted in four primary
recommendations for improving AHRQ\'s processes for recruiting and
selecting new employees. These recommendations are currently being
evaluated for implementation.

## Building an integrated and reliable information technology infrastructure

AHRQ is undertaking a comprehensive review of our business processes and
products. This will undoubtedly result in changes in not only how we do
our daily work, but also how information flows within the organization.
Therefore, we must also redefine our information technology architecture
so that not only do we meet requirements multiple statutory
requirements, such as GISRA, Clinger-Cohen, but also to ensure that it
meets the management and organizational needs of the agency. Evolving
technologies, such as Web-based applications, provide opportunities for
us to accomplish our work in new, more efficient and timely ways. The IT
infrastructure we develop must be adaptable to meet new requirements and
while maintaining information security and critical infrastructure
protections.

**FY2003 OUTCOME MEASURES FOR RESEARCH MANAGEMENT**

**Program Outputs to meet required goals**:

-   Establish profile specific competency requirements for each of the
    agency's key leadership positions

-   Define diversity goals and identify sources to target recruiting
    efforts

-   Develop communication strategies for PSC HR and hiring managers that
    facilitates ongoing identification of staffing needs; consideration
    of training existing staff or hiring new staff to meet needs;
    identifying the most effective resources for recruiting the new
    hires and the timeframes required for obtaining new hires

-   Establish criteria to assess success of recruiting and selection
    efforts

-   Complete Business Process Review of AHRQ's grants process and design
    implementation plan for recommendations

-   Identify IT mission critical systems and services, measure current
    performance and establish systems to monitor ongoing system
    integrity

> **DEPARTMENT**
>
> **OF HEALTH**
>
> **AND HUMAN**
>
> **SERVICES**

**Agency for Health Care Research and Quality**

APPENDICES

#  {#section-27 .unnumbered}

# 

# A.1 Changes and Improvements Over Previous Year

**Summary**

The basic structure of the AHRQ FY 02 performance plan for FY 02 is
unchanged. The titles for Goals Five and Six were revised to simplify
the presentation, but the program objectives represented under each goal
remain the same.

The indicators presented under each objective were updated to reflect
where Agency programs are expected to be in FY 2003 and the anticipated
level of funding. The update resulted in output and outcome measures
being added, particularly in Goals Three and Four, for programs and
research initiatives that are expected to come to fruition during that
period. Additionally, the details of some indicators have changed based
on results received in FY 2000.

**Detailed Presentation of Goals and Objectives**

(Indicators can be seen in detail in section 1.4 of this report.)

**Goal 1: Establish Future Research Needs Based on User's Needs**

  ----------------------------- --------------------- -------------------
  **01 Performance Objective**  **02**                **Comments**

  Objective1.1: Define          Same                  
  direction of FY project                             
  funding priorities, in large                        
  part, by needs assessment                           
  activities.                                         
  ----------------------------- --------------------- -------------------

**Goal 2: Make significant contributions to the effective functioning of
the US health care system through *the creation of new knowledge*.**

  ---------------------------- --------------------- ---------------------
  **01 Performance Objective** **02**                **Comments**

  Determine annually the       Same                  
  salient findings from                              
  research in each of the                            
  three areas (outcomes;                             
  quality; and cost, access,                         
  and use) and develop plan                          
  for next steps translation                         
  and dissemination.                                 
  ---------------------------- --------------------- ---------------------

  ---------------------------- --------------------- ---------------------
  Objective 2:2: Achieve       Discontinued          This measure is
  significant findings from                          combined with 3.1 in
  AHRQ sponsored and conducted                       FY 2002.
  research.                                          

  Objective 2.3 Initiate FY    Same                  
  Research Initiatives.                              
  ---------------------------- --------------------- ---------------------

**Goal 3: Foster translation of new knowledge into practice by
developing and providing information, products, and tools on outcomes,
quality, and access, cost, and use of care.**

  ---------------------------- --------------------- ---------------------
  **01Performance Objective**  **02**                **Comments**

  Objective 3.1: Maximize      Achieve significant   This measure is
  dissemination of             findings from AHRQ    combined with 2.2 in
  information, tools, and      sponsored and         FY 2002. Measure 2.2
  products developed from      conducted research    is discontinued in FY
  research results for use in  and maximize          2002.
  practice settings.           dissemination of      
                               information, tools,   
                               and products          
                               developed from        
                               research results for  
                               use in practice       
                               settings.             

  Objective 3.2: Develop and   Same                  This is objective 3.3
  facilitate the use of new                          in FY 99-00.
  tools, talent, products, and                       
  implementatio methodologies                        
  stemming from research                             
  portfolio.                                         

  Objective 3.3: Develop and   DISCONTINUED          Objective 3.3 is
  facilitate the use of new                          discontinued in FY
  tools, talent, products, and                       01. (This becomes
  implementation methodologies                       objective 3.2 in FY
  stemming from research                             01.)
  portfolio.                                         
  ---------------------------- --------------------- ---------------------

# Goal 4: Evaluate the effectiveness and impact of AHRQ research and 

> **associated activities.**

+-----------------------------------+------------------+---------------+
| **01 Performance Objective**      | **02**           | **Comments**  |
+-----------------------------------+------------------+---------------+
| Objective 4.1: Evaluate the       | Same             | Objectives    |
| impact of AHRQ sponsored products |                  | 4.1 and 4.2   |
| in advancing methods to measure   |                  | have been     |
| and improve health care.          |                  | consolidated  |
|                                   |                  | in the FY 01  |
|                                   |                  | plan          |
|                                   |                  |               |
|                                   |                  | The FY 00     |
|                                   |                  | plan          |
|                                   |                  | inadvertently |
|                                   |                  | had two       |
|                                   |                  | objectives of |
|                                   |                  | the same      |
|                                   |                  | title. That   |
|                                   |                  | redundancy    |
|                                   |                  | has been      |
|                                   |                  | removed. With |
|                                   |                  | that          |
|                                   |                  | exception,    |
|                                   |                  | the           |
|                                   |                  | objectives    |
|                                   |                  | remain the    |
|                                   |                  | same as FY00. |
+-----------------------------------+------------------+---------------+
| Objective 4.2: Evaluate the       | Same             |               |
| impact of MEPS data and           |                  |               |
| associated products on            |                  |               |
| policymaking and research         |                  |               |
| products.                         |                  |               |
+-----------------------------------+------------------+---------------+

# 

# Goal 5: Support initiative to improve health care quality through leadership and research.

  ---------------------------- ------------------- ----------------------
  **01 Performance Objective** **02**              **Comments**

  Objective 5.1: Conduct       Same                
  research to help to measure                      
  the current status health of                     
  care quality in the Nation.                      

  Objective 5.2: Facilitate    Same                
  use of quality information                       
  to improve health care in                        
  the Nation.                                      

  Objective 5.3: Improve       Same                
  quality measurement.                             
  ---------------------------- ------------------- ----------------------

# 

# Goal 6: Medical Expenditure Panel Surveys

  ------------------------- ------------------- -------------------------
  **01 Performance          **02**              **Comments**
  Objective**                                   

  Objective 6.1: Release    Same                
  and disseminate MEPS data                     
  and information products                      
  in timely manner for use                      
  by researchers, policy                        
  makers, purchasers, and                       
  plans.                                        

  Objective 6.2: Facilitate Same                
  use of MEPS data and                          
  associated products as                        
  tools by extramural                           
  researchers, policy                           
  makers, purchasers, and                       
  plans.                                        

  Objective 6.3: Modify     Same                The name changed slightly
  MEPS to support annual                        to make the objective
  reporting on quality,                         more generic from year to
  health care disparities,                      year. This objective is
  and research on long-term                     where the Agency will
  care in adults and                            note the ongoing and
  children with special                         important role that MEPS
  needs.                                        is playing in measuring
                                                national quality.
  ------------------------- ------------------- -------------------------

**GOAL 7: Support the overall direction and management of AHRQ**

Budget line: Program Support

+-----------------------+------------------+--------------------------+
| **01 Performance      | **02**           | **Comments**             |
| Objective**           |                  |                          |
+-----------------------+------------------+--------------------------+
| Objective 7.1 is      | Discontinued     | The measures for these   |
| manditory (Capital    |                  | objectives are for       |
| Assets) but not       |                  | internal management and  |
| applicable to AHRQ.   |                  | do not warrant being     |
|                       |                  | reported outside AHRQ.   |
|                       |                  |                          |
|                       |                  | The measures for this    |
|                       |                  | area also remain in      |
|                       |                  | place internally in the  |
|                       |                  | Office of Management     |
|                       |                  | Operations Plan and      |
|                       |                  | employee performance     |
|                       |                  | plans for the managers   |
|                       |                  | and staff.               |
+-----------------------+------------------+--------------------------+

+-----------------------+------------------+--------------------------+
| Objective 7.2:        | Discontinued     | The measures for these   |
| Maintain acquisition  |                  | objectives are for       |
| performance           |                  | internal management and  |
| management system to  |                  | do not warrant being     |
| ensure: (1) timely    |                  | reported outside AHRQ.   |
| completion of         |                  |                          |
| transactions, (2)     |                  | The measures for this    |
| vendor and customer   |                  | area also remain in      |
| satisfaction, and (3) |                  | place internally in the  |
| efficient and         |                  | Office of Management     |
| effective use of      |                  | Operations Plan and      |
| resources.            |                  | employee performance     |
|                       |                  | plans for the managers   |
|                       |                  | and staff.               |
+-----------------------+------------------+--------------------------+
| Objective 7.3:        | Discontinued     | The measures for these   |
| continued enhancement |                  | objectives are for       |
| and expansion of      |                  | internal management and  |
| Agency intranet site  |                  | do not warrant being     |
| to ensure staff have  |                  | reported outside AHRQ.   |
| immediate access to   |                  |                          |
| all current           |                  | The measures for this    |
| information.          |                  | area also remain in      |
|                       |                  | place internally in the  |
|                       |                  | Office of Management     |
|                       |                  | Operations Plan and      |
|                       |                  | employee performance     |
|                       |                  | plans for the managers   |
|                       |                  | and staff.               |
+-----------------------+------------------+--------------------------+
| Objective 7.4:        | Same             | This objective was       |
|                       |                  | initiated in FY01.       |
| Establish and         |                  |                          |
| maintain a secure     |                  |                          |
| Agency computer       |                  |                          |
| network               |                  |                          |
| infrastructure.       |                  |                          |
+-----------------------+------------------+--------------------------+
| Objective 7.5.        | .                | This measure will be     |
| Optimize work         |                  | initiated in FY02.       |
| processes through     |                  |                          |
| implementation of     |                  |                          |
| AHRQ's workforce      |                  |                          |
| restructuring plan    |                  |                          |
+-----------------------+------------------+--------------------------+

**A.2 Linkage to HHS Strategic Plan** - AHRQ is guided by and supports
the strategic plan goals of the Department of Health and Human Services
(HHS).

The Agency's activities contribute to five of the six HHS goals.

> **HHS Goal 2:** Improve the Economic and Social Well-Being of
> Individuals,

Families, and Communities in the United States -- For example, through
the Agency's research on children's health

> **HHS Goal 3:** Improve Access to Health Services and Ensure the
> Integrity of the Nation's Health Entitlement and Safety Net Programs
> -- For example, through the activities of the Agency's Center for
> Primary Care Research
>
> **HHS Goal 4:** Improve the Quality of Health Care and Human Services
> -- For example, through numerous Agency activities including quality
> measurement research and data development
>
> **HHS Goal 5:** Improve the Nation's Public Health System -- For
> example,

through the Agency's data development and monitoring activities and
investments

> **HHS Goal 6:** Strengthen the Nation's Health Sciences Research
> Enterprise and Enhance Its Productivity -- For example, through the
> Agency's research, data development, translation, and dissemination
> activities

A detailed summary of AHRQ's contributions to the HHS strategic plan
follows, however, the Agency's contributions to Goals 4 and 6 are of
particular note.

AHRQ is the Department's lead Agency for health care quality activities
and is a major contributor to Goal 4. Research on the determinants of
health care quality, effective and cost effective ways to improve health
care quality, and how to measure health care quality will be
instrumental in achieving success in the goal's objectives.
Additionally, the new information developed, and then implemented in the
health care system, from research on outcomes and effectiveness of care,
as well as access to, cost, and use of health care, will help close the
gaps between what we know and what we need to know, on the one hand, and
what we know and what we do in health care, on the other hand. AHRQ also
supports HHS Goal 4 through leadership of the Secretary's quality
initiative, support of the Quality Interagency Task Force, contributions
to the HHS Race and Health Disparities Initiative, and close working
relationships with other HHS Agencies and the Office of the Secretary.

Within Goal 6, AHRQ's expanding portfolio in outcomes and effectiveness,
quality, primary care, and other practice-based research, as well as
extramural and intramural studies of issues pertaining to access, cost,
organization, and delivery of health care, will contribute to achieving
objective 6.4. Additionally, AHRQ's substantial investment in the
development of databases will enable others to perform research and
analyses to answer questions critical to understanding the dynamics of
the health care system.

AHRQ maintains its commitment to building the infrastructure needed to
continue to conduct high-quality, cutting-edge health care research for
the next century through national training programs (including the
National Research Service Awards program), service fellows and summer
intern programs, mentoring programs, and the provision of technical
assistance to a variety of its audiences. AHRQ will continue to support
and expand efforts to attract trainees from racial and ethnic minorities
into the field of health services research. These activities are aligned
with Objective 6.6.

For the Nation's investment in research to reach its full potential, the
results must be widely disseminated (Objective 6.7) and implemented. The
AHRQ Cycle of Research presented in section E.2. of this plan
illustrates the important role that dissemination will play in all areas
of activity within the Agency. This focus aligns with Objective 6.7.

**Examples of AHRQ's Contributions to the HHS Strategic Plan:**

**HHS Goal 2: Improve the Economic and Social Well-Being of Individuals,
Families, and Communities in the United States**

> HHS Objective 2.5: Increase Opportunities for Seniors to Have an
> Active and Healthy Aging Experience

HHS Objective 2.6: Expand Access to Consumer-Directed, Home and
Community-Based Long-Term Care and Health Services

> AHRQ will conduct research in areas relevant to improving the aging
> experience in such areas as: 1) conditions of particular importance to
> the Medicaid population; and 2) quality measurement issues and tool
> development for institutional settings. In FY 2001, AHRQ has supported
> important new initiatives to improve the knowledge and tools to
> improve health care quality for Older Americans and optimize their
> functional outcomes.

**HHS Goal 3: Improve Access to Health Services and Assure the Integrity
of the Nation's Health Entitlement and Safety Net Programs**

> HHS Objective 3.2: Increase the Availability of Primary Health Care
> Services

To test the effectiveness of health care improvement approaches, AHRQ
will study such topics as: 1) the implementation of evidence-based
information in diverse health care settings to determine effective
strategies for enhancing practitioner behavior change and improving
patient behavior, knowledge and satisfaction; 2) the factors which
determine the success of quality improvement strategies and to what
extent these vary by the nature of the problem addressed and the target
population; and 3) the factors which influence access to primary care
services, and transitions between primary and specialty services.

Projects to evaluate the impact of managed care will constitute a
systematic effort to determine the impact of managed care and other
changes in the organization of care on health care quality; outcomes;
and cost, use and access.

HHS Objective 3.3: Improve Access to and the Effectiveness of Health
Care Services for Persons with Specific Needs

AHRQ will examine how various clinical and system characteristics affect
the health outcomes, quality access, and satisfaction, for the elderly
and chronically ill. In FY 2001, added emphasis will be given to
research and tool development to improve care for persons with chronic
conditions, including mental health conditions, and disabilities.

HHS Objective 3.4: Protect and Improve Beneficiary Health and
Satisfaction with Medicare and Medicaid

AHRQ activities supporting Medicare and Medicaid beneficiaries will
include:

1\) research on conditions that are common, costly, and for which there
is substantial variation in practice, conditions that represent major
Medicare or Medicaid expenditures; 2) providing objective,
science-based, timely information to health care decision makers\--
patients and clinicians, health system leaders, and policy makers; 3)
health care surveys, such as CAHPS® and MEPS, that provide information
supporting health plan choices and coverage decisions; 4) new tools to
assist beneficiaries to choose the most appropriate care for their
needs; 5) new tools to measure and improve quality of life for Medicare
and Medicaid beneficiaries; and 4) studying the national impact public
programs on access and cost of care for children.

**HHS Goal 4: Improve the Quality of Health Care and Human Services**

> HHS Objective 4.1: Promote the Appropriate Use of Effective Health
> Services

AHRQ is implementing a growing portfolio of evaluations that will show
over time the outcomes of the investments of Agency funds. The Agency
will evaluate the use and usability of the Medical Expenditures Panel
Survey (MEPS) databases for their intended purposes. Other evaluations
will assess: 1) evidence reports and technology assessments of
evidence-based practice centers; and 2) products that advance methods to
measure and improve health-care quality, including clinical quality
improvement software (CONQUEST), the Consumer Assessment of Health Plans
Survey, and the Expansion of Quality of Care Measures project (Q-SPAN).
In FY 2001, be placed added emphasis on developing the tools, systems,
and partnerships that enable quality improvement strategies to be
effective in diverse settings, including public providers.

> HHS Objective 4.2: Reduce Disparities in the Receipt of Quality Health
> Care Services

AHRQ will expand its commitment to conducting health services research
that will help reduce disparities that exist for racial and ethnic
minorities. In particular, the Agency will sponsor research to
understand the contribution that racial discrimination makes to existing
patterns of care for minority Americans. The Agency will also focus on
building infrastructure for relevant research by training minority and
other investigators to address issues for minority populations.

> HHS Objective 4.3: Increase Consumers' Understanding of Their Health
> Care Options

The Consumer Assessment of Health Plans (CAHPS®) is a tool for surveying
members of health plans about their experience with and assessment of
the quality of health care they receive, and for reporting the results
to other consumers who are choosing a plan. AHRQ will continue its
efforts to enhance CAHPS® and evaluate its use. In FY 2001, additional
research will be sponsored to better understand how consumers use
quality information.

**HHS Goal 5: Improve the Public Health System**

> HHS Objective 5.1: Improve the Public Health Systems' Capacity to
> Monitor The Health Status and Identify Threats to the Health of the
> Nation's Population
>
> AHRQ will develop a fully integrated strategy to create the national
> capacity to monitor the quality of care, particularly for populations
> of national interest, including the chronically ill, poor,
> racial/ethnic minorities, and children. This includes an increase in
> the scope of the *Medical Expenditure Panel Survey* (MEPS),
> partnerships with other Federal Agencies and the private sector.

AHRQ will complete the redesign and expansion of the Healthcare Cost and
Utilization Project (HCUP) providing state and community decision-makers
a powerful set of linked databases they can use to monitor the impact of
major system changes on access, quality, outcomes and cost in their
states and communities, and to compare these against the progress of
other states and communities.

AHRQ will build on past investments in tool development by focusing on
expanding the toolbox. These tools will enable purchasers, policymakers,
health plans, providers, and patients to improve care.

**HHS Goal 6: Strengthen the Nation's Health Sciences Research
Enterprise and EnhanceI ts Productivity**

> HHS Objective 6.2: Improve the Prevention, Diagnosis, and Treatment of
> Disease and Disability

AHRQ will fund research on the prevention of functional decline in the
elderly, the quality of clinical preventive services, and the prevention
of medication and other medical errors.

HHS Objective 6.4: Increase the Understanding of and Response to the
Major Issues Related to the Quality, Financing, Cost, and
Cost-Effectiveness of Health Care Services

There are many gaps in knowledge in all areas of health care. New
questions emerge as new technologies are developed, the population's
demographics change, areas of inquiry previously under-emphasized take
on greater importance, and research previously undertaken identifies
further areas that need attention. Therefore, AHRQ will continue to
focus on creating new knowledge and assessing the findings that result
from completed projects.

> HHS Objective 6.6: Improve the Quality of Medical and Health Science
> Research by Strengthening the Base of Highly Qualified Scientific
> Investigators

AHRQ will invest in a number of programs to further the training of
health services researchers to address the research and analytic needs
of the changing health care system. These priorities will build on prior
efforts to make both curricula and practical research experiences more
relevant to decision makers' concerns about the effectiveness of health
care and issues of cost, quality, and access. In addition, an added
emphasis is being made on increasing the number of minority
investigators at both majority and minority institutions.

> HHS Objective 6.7: Ensure That Research Results Are Effectively
>
> Communicated to the Public, Practitioners, and the Scientific
> Community

Building on the previous 10 years of research findings, AHRQ will
identify ongoing gaps between what we know now and what we do in health
care and will begin to close those gaps through research and
demonstrations that develop and test implementation strategies in
different settings in the health care system. A major focus within this
is identifying existing implementation strategies in use in health care
settings and demonstrating their applicability to wide spread
dissemination in other areas of the system.

AHRQ places considerable focus on developing tools and products that
facilitate the transfer of research findings into practice. The Agency
has a well developed dissemination system that includes publications
development, the Publications Clearinghouse, and an award winning Web
site. This emphasis is critical to the Agency's success. Ongoing plans
include incorporating regular customer feedback into our operations to
continue to improve our efforts.

**A.3 Performance Measurement Linkages with Budget, Cost Accounting,
Information Technology Planning, Capital Planning and Program
Evaluation**

**BUDGET LINKAGE**

The performance goals and measures are linked directly to two of the
Agency's three budget lines. As described previously, the budget line
for Research on Health Care Costs, Quality, and Outcomes has five of the
six performance plan goals tied to it. The budget line for the Medical
Panel Expenditure Surveys is represented in the sixth goal. The
Introduction to Part 2 of this document describes these alignments in
detail.

The third budget line, Program Support, has been removed from the
performance goals in the AHRQ performance plan. The measures previously
reported for Program Support focused on internal management issues for
contracts management and information system development. We are dropping
the measures because they do not rise to the level of being one of the
"critical few" measures that should be reported by the Agency in the
GPRA plan. The measures continue to be important, however, and remain in
place for internal accountability in the Office of Management Operations
Plan and performance plans for the managers and staff.

The Agency provides page numbers, both in the body of the performance
plan and in the summary table in section 1.4, to crosswalk the specific
measures that are related to the programs proposed in the budget
document. Many of the measures are for projects and initiatives that
were started in past years and are continuing. This has been noted by
identifying the budget connection as "commitment base."

**COST ACCOUNTING**

AHRQ allocates the full cost of all associated research activities,
including overhead costs, by our three budget activities: Research on
Health Costs, Quality and Outcomes, Medical Expenditure Panel Surveys,
and Program Support.

**INFORMATION TECHNOLOGY**

AHRQ is currently completing the infrastructure for a new integrated
information system, the Agency Management Information System (AMIS).

**PLANNING**

The goal of AMIS is to provide access to an integrated set of project
information that includes the information currently input through
multiple systems. When the AMIS is fully operational all Agency staff
will have on‑demand access to current information for the entire AHRQ
research grant, contract, intramural project, and IAA portfolio. This
system is one of the major methods the Agency will use to collect data
to document its success in meeting its GPRA goals.

**CAPITOL PLANNING** Does not apply to AHRQ.

**PROGRAM** **EVALUATION**

Goal 4 in the AHRQ annual performance plan is devoted to reporting the
evaluations that the Agency will do in FY 2002. The evaluations are
focused on demonstrating the use of Agency research and products and the
resulting impact on organizations within the health care system.

> **DEPARTMENT**
>
> **OF HEALTH**
>
> **AND HUMAN**
>
> **SERVICES**

**Agency for Health Care Research and Quality**

FISCAL YEAR **2001** PERFORMANCE Summary

# 

# 

# 

# 

#  {#section-35 .unnumbered}

# 

**Summary of Performance Objectives**

**Budget Line 1: Research on Health Costs, Quality, and Outcomes**

  -----------------------------------------------------------------------
  **GPRA Goal 1: Establish research agenda based on users= needs.**

  -----------------------------------------------------------------------

> **NOTE: B: is Budget; Page: will have entered a page number; CB is
> commitment base. Under Actual Performance column the corresponding
> page (p., pp.) #s of the GPRA report are noted.**

+------------------+-------------------------------------+-------+----+
| **Performance    | **FY Targets**                      | **A   | ** |
| Objective**      |                                     | ctual | Re |
|                  |                                     | Per   | fe |
|                  |                                     | forma | r- |
|                  |                                     | nce** | en |
|                  |                                     |       | ce |
|                  |                                     |       | ** |
+------------------+-------------------------------------+-------+----+
| **Objective 1.1: | **02:**                             | **Co  | ** |
| Define direction |                                     | mplet | CB |
| of FY project    | -   **Agency research agenda        | ed.** | ** |
| funding          |     covering strategic goal areas   |       |    |
| priorities, in   |     for FY 2002 priorities          | **Co  | ** |
| large part, by   |     (investigator-initiated         | mplet | CB |
| needs assessment |     research, national quality      | ed.** | ** |
| activities.**    |     report, national disparities    |       |    |
|                  |     report) is documented by March  | **    | *  |
|                  |     2002 based on consultations     | Compl | *B |
|                  |     with various groups.**          | eted. | :3 |
|                  |                                     | P. 40 | ** |
|                  | **01:**                             | and   |    |
|                  |                                     | App   |    |
|                  | -   **Agency research agenda        | endix |    |
|                  |     covering strategic goal areas   | 5.**  |    |
|                  |     for FY 2001 priorities (patient |       |    |
|                  |     safety and informatics) is      |       |    |
|                  |     documented based on             |       |    |
|                  |     consultations with various      |       |    |
|                  |     groups.**                       |       |    |
|                  |                                     |       |    |
|                  | **00:**                             |       |    |
|                  |                                     |       |    |
|                  | -   **Agency research agenda        |       |    |
|                  |     covering the 3 strategic        |       |    |
|                  |     research goals and the new FY   |       |    |
|                  |     2000 closing the gap            |       |    |
|                  |     initiatives are documented      |       |    |
|                  |     based on consultations with     |       |    |
|                  |     various groups.**               |       |    |
|                  |                                     |       |    |
|                  | **99:**                             |       |    |
|                  |                                     |       |    |
|                  | -   **Agency research agenda        |       |    |
|                  |     covering the 3 strategic        |       |    |
|                  |     research goals is developed in  |       |    |
|                  |     FY 99 and documented based on   |       |    |
|                  |     consultations with various      |       |    |
|                  |     groups.**                       |       |    |
+------------------+-------------------------------------+-------+----+

+-----------------------------------------------------------------------+
| **GPRA Goal 2: Make significant contributions to the effective        |
| functioning of the**                                                  |
|                                                                       |
| **U.S. health care system through the creation of new knowledge.**    |
+-----------------------------------------------------------------------+

+------------------+-------------------------------------+-------+----+
| **Performance    | **FY Targets**                      | **A   | ** |
| Objective**      |                                     | ctual | Re |
|                  |                                     | Per   | fe |
|                  |                                     | forma | r- |
|                  |                                     | nce** | en |
|                  |                                     |       | ce |
|                  |                                     |       | ** |
+------------------+-------------------------------------+-------+----+
| **01 - 02        | **02:**                             | **Co  | ** |
| Objective 2.1:   |                                     | mplet | CB |
| Determine        | -   **Produce an annual report on   | ed.** | ** |
| annually the     |     at least 18 science advances    |       |    |
| salient findings |     covering the three research     | **Co  | ** |
| from research in |     goal areas (outcomes; quality;  | mplet | CB |
| each of the      |     cost, access, and use).**       | ed.** | ** |
| three areas      |                                     |       |    |
| (outcomes;       | -   **For each finding, specific    |       |    |
| quality; and     |     steps in translation and        |       |    |
| cost, access,    |     dissemination are identified    |       |    |
| and use) and     |     and initiated.**                |       |    |
| develop plan for |                                     |       |    |
| next steps       | -   **Generate 2 - 3 synthesis      |       |    |
| translation and  |     reports on research findings    |       |    |
| dissemination.** |     and practical applications on   |       |    |
|                  |     Agency priority topics.**       |       |    |
|                  |                                     |       |    |
|                  | **01:**                             |       |    |
|                  |                                     |       |    |
|                  | -   **Produce an annual report on   |       |    |
|                  |     at least 12 science advances    |       |    |
|                  |     covering the three research     |       |    |
|                  |     goal areas (outcomes; quality;  |       |    |
|                  |     cost, access, and use) For each |       |    |
|                  |     finding, specific steps in      |       |    |
|                  |     translation and dissemination   |       |    |
|                  |     are identified and initiated.** |       |    |
|                  |                                     |       |    |
|                  | -   **Generate 2 - 3 synthesis      |       |    |
|                  |     reports on research findings    |       |    |
|                  |     and practical applications on   |       |    |
|                  |     Agency priority topics.**       |       |    |
+------------------+-------------------------------------+-------+----+
| **00 Objective   | **00:**                             | **Co  | ** |
| 2.1: Determine   |                                     | mplet | CB |
| annually the     | -   **Annual report on science      | ed.** | ** |
| salient findings |     advances in three research goal |       |    |
| from research in |     areas.**                        | **Co  |    |
| each of the      |                                     | mplet |    |
| three areas      | -   **At least four major findings  | ed.** |    |
| (outcomes;       |     in each area that have          |       |    |
| quality; and     |     potential to save significant   | **Co  |    |
| cost, access,    |     amounts of money, improve       | mplet |    |
| and use) and     |     quality, save lives or prevent  | ed.** |    |
| develop plan for |     physical suffering, or change   |       |    |
| next steps       |     the organization and delivery   |       |    |
| translation and  |     of health care.**               |       |    |
| dissemination.** |                                     |       |    |
|                  | -   **For each finding, specific    |       |    |
|                  |     steps in translation and        |       |    |
|                  |     dissemination are identified    |       |    |
|                  |     and initiated.**                |       |    |
+------------------+-------------------------------------+-------+----+
| **99 Objective   | **99:**                             | **    |    |
| 2.1: Determine   |                                     | Compl |    |
| the salient      | -   **A report produced that        | eted. |    |
| findings from    |     synthesizes research on the     | (App  |    |
| research for     |     major health concerns of at     | endix |    |
| three priority   |     least three priority            | 6).** |    |
| populations and  |     populations.**                  |       |    |
| develop plan for |                                     |       |    |
| next steps in    |                                     |       |    |
| translation and  |                                     |       |    |
| dissemination.** |                                     |       |    |
+------------------+-------------------------------------+-------+----+
| **Objective 2:2: | **02:**                             | **Co  | ** |
| Achieve          |                                     | mplet | CB |
| significant      | -   **Findings from at least 20     | ed.** | ** |
| findings from    |     AHRQ sponsored or conducted     |       |    |
| AHRQ sponsored   |     research are used by public and | *     | ** |
| and conducted    |     private partners to improve     | *400% | CB |
| research.**      |     health care.**                  | i     | ** |
|                  |                                     | ncrea |    |
|                  | **01: same as 00, except changed to | se:** | ** |
|                  | 40 findings**                       |       | CB |
|                  |                                     | **250 | ** |
|                  | **00:**                             | cita  |    |
|                  |                                     | tions |    |
|                  | -   **Findings from at least 25     | for   |    |
|                  |     AHRQ sponsored or conducted     | AHRQ  |    |
|                  |     research are published in major | fin   |    |
|                  |     peer reviewed professional      | dings |    |
|                  |     publications (*New England      | (20   |    |
|                  |     Journal of Medicine, Journal of | lis   |    |
|                  |     American Medical Association*,  | ted). |    |
|                  |     etc.); receive national press   | 32    |    |
|                  |     coverage; are used in Federal   | exa   |    |
|                  |     or State policymaking; are used | mples |    |
|                  |     by professional associations or | of    |    |
|                  |     health plans as the basis of    | major |    |
|                  |     strategies to achieve quality;  | media |    |
|                  |     or are used to establish        | cove  |    |
|                  |     coverage decisions by health    | rage; |    |
|                  |     care purchasers, managed care   | 7     |    |
|                  |     organizations, or insurers,     | exa   |    |
|                  |     including Medicare or           | mples |    |
|                  |     Medicaid.**                     | of    |    |
|                  |                                     | us    |    |
|                  | **99: Findings from at least 10     | age** |    |
|                  | AHRQ sponsored or funded research   |       |    |
|                  | are published in major peer         | **50  |    |
|                  | reviewed professional publications  | cita  |    |
|                  | (*New England Journal of Medicine,  | tions |    |
|                  | Journal of American Medical         | for   |    |
|                  | Association*, etc.); receive        | AHRQ  |    |
|                  | national press coverage; are used   | find  |    |
|                  | in Federal or State policymaking;   | ings; |    |
|                  | are used by professional            | 7     |    |
|                  | associations or health plans as the | exa   |    |
|                  | basis of strategies to achieve      | mples |    |
|                  | quality; or are used to establish   | of    |    |
|                  | coverage decisions by health care   | major |    |
|                  | purchasers, managed care            | media |    |
|                  | organizations, or insurers,         | cove  |    |
|                  | including Medicare or Medicaid.**   | rage; |    |
|                  |                                     | 7     |    |
|                  |                                     | exa   |    |
|                  |                                     | mples |    |
|                  |                                     | of    |    |
|                  |                                     | usa   |    |
|                  |                                     | ge.** |    |
+------------------+-------------------------------------+-------+----+
| **01 - 02        | **02: Funding of a minimum of 100   | **Co  | ** |
| Objective 2.3:   | projects; 30% of these projects**   | mplet | B: |
| Initiate FY      |                                     | ed.** | 64 |
| Research         | > **address priority populations.** |       | ** |
| Initiatives**    |                                     | **Co  |    |
|                  | **01: Funding of a minimum of 60    | mplet | *  |
|                  | projects in the following areas:**  | ed.** | *B |
|                  |                                     |       | :3 |
|                  | -   **40 projects in reducing       | **Co  | 1- |
|                  |     medical errors and enhancing    | mplet | 35 |
|                  |     patient safety**                | ed.** | ** |
|                  |                                     |       |    |
|                  | -   **10 projects in informatics    |       | ** |
|                  |     applications in health care**   |       | B: |
|                  |                                     |       | 33 |
|                  | -   **10 projects in quality        |       | ** |
|                  |     improvement through             |       |    |
|                  |     improvements in health care     |       | *  |
|                  |     working conditions**            |       | *B |
|                  |                                     |       | :3 |
|                  |                                     |       | 6- |
|                  |                                     |       | 37 |
|                  |                                     |       | ** |
+------------------+-------------------------------------+-------+----+
| **00 Objective   | -   **Funding of a minimum of 10    | **43  | *  |
| 2.3: Implement   |     projects that address gaps in   | pro   | *B |
| FY 2000 priority |     knowledge about the priority    | jects | :1 |
| (1)** A**New     |     problems faced by Medicare and  | fun   | 9- |
| Research on      |     Medicaid.**                     | ded** | 20 |
| Priority Health  |                                     |       | ** |
| Issues.**@       | -   **Funding of a minimum of 10    | *     |    |
|                  |     projects to address eliminating | *More | *  |
|                  |     disparities in health care with | then  | *B |
|                  |     particular emphasis on          | 30    | :5 |
|                  |     disparities that exist for      | pro   | 0- |
|                  |     racial and ethnic minorities.** | jects | 51 |
|                  |                                     | fun   | ** |
|                  |                                     | ded** |    |
+------------------+-------------------------------------+-------+----+
| **99 Objective   | **Funding of a minimum of 21        | **56  |    |
| 2.3 Initiate FY  | projects in:**                      | pro   |    |
| 99 Research      |                                     | jects |    |
| Initiatives**    | -   **consumers use of information  | fund  |    |
|                  |     on quality**                    | ed.** |    |
|                  |                                     |       |    |
|                  | -   **strengthen value-based        | **51  |    |
|                  |     purchasing**                    | pro   |    |
|                  |                                     | jects |    |
|                  | -   **measure national health care  | fund  |    |
|                  |     quality**                       | ed.** |    |
|                  |                                     |       |    |
|                  | -   **vulnerable populations**      | **De  |    |
|                  |                                     | tails |    |
|                  | -   **translating research into     | on p. |    |
|                  |     practice**                      | 47.** |    |
|                  |                                     |       |    |
|                  | **Funding of a minimum of 17        |       |    |
|                  | projects in:**                      |       |    |
|                  |                                     |       |    |
|                  | -   **Outcomes for the elderly and  |       |    |
|                  |     chronically ill**               |       |    |
|                  |                                     |       |    |
|                  | -   **Clinical preventive           |       |    |
|                  |     services**                      |       |    |
|                  |                                     |       |    |
|                  | -   **CERTS**                       |       |    |
|                  |                                     |       |    |
|                  | -   **Improving the quality of      |       |    |
|                  |     children**=**s health**         |       |    |
+------------------+-------------------------------------+-------+----+

  -----------------------------------------------------------------------
  **Goal 3: Foster translation and dissemination of new knowledge into
  practice by developing and providing information, products, and tools
  on outcomes; quality; and access, cost, and use of care.**

  -----------------------------------------------------------------------

+------------------+-------------------------------------+-------+---+
| **Performance    | **FY Targets**                      | **A   | * |
| Objective**      |                                     | ctual | * |
|                  |                                     | Per   | R |
|                  |                                     | forma | e |
|                  |                                     | nce** | f |
|                  |                                     |       | e |
|                  |                                     |       | r |
|                  |                                     |       | - |
|                  |                                     |       | e |
|                  |                                     |       | n |
|                  |                                     |       | c |
|                  |                                     |       | e |
|                  |                                     |       | * |
|                  |                                     |       | * |
+------------------+-------------------------------------+-------+---+
| **01 - 02        | **02:**                             | **Co  | * |
| Objective 3.1:   |                                     | mplet | * |
| Maximize         | -   **Number of state and local     | ed.** | C |
| dissemination of |     governments trained and/or      |       | B |
| information,     |     receiving technical assistance  | **Co  | * |
| tools, and       |     through User Liaison Program    | mplet | * |
| products         |     (ULP).**                        | ed.** |   |
| developed from   |                                     |       | * |
| research results | -   **At least 20 partnerships to   | **16. | * |
| for use in       |     disseminate and implement       | 7m.** | C |
| practice         |     research findings are formed    |       | B |
| settings.**      |     with public and private-sector  | **4,  | * |
|                  |     organizations.**                | 006** | * |
| **NOTE: in the   |                                     |       |   |
| FY 2001 plan,    | -   **Synthesis of at least 5 grant | **3,  | * |
| objective 3.1    |     portfolio areas on quality of   | 730** | * |
| and 3.2 have     |     care across Agency**=**s goals  |       | C |
| been             |     for persons with chronic care   | **    | B |
| consolidated.**  |     needs produced and disseminated | 425** | f |
|                  |     with particular focus on        |       | o |
|                  |     outreach to managed care        |       | r |
|                  |     executives.**                   |       | a |
|                  |                                     |       | l |
|                  | **01:**                             |       | l |
|                  |                                     |       | w |
|                  | -   **At least 5 public-private     |       | e |
|                  |     partnerships are formed to      |       | b |
|                  |     implement research findings for |       | s |
|                  |     decisionmakers.**               |       | i |
|                  |                                     |       | t |
|                  | -   **Formation of a minimum of 10  |       | e |
|                  |     partnerships to support         |       | m |
|                  |     dissemination of AHRQ products  |       | e |
|                  |     through intermediary            |       | a |
|                  |     organizations, such as health   |       | - |
|                  |     plans and professional          |       | s |
|                  |     organizations.**                |       | u |
|                  |                                     |       | r |
|                  | -   **Number of hits on the Web     |       | e |
|                  |     site**                          |       | s |
|                  |                                     |       | a |
|                  | -   **Number of inquiries handled   |       | n |
|                  |     on web site**                   |       | d |
|                  |                                     |       | U |
|                  | -   **Number of Uploaded            |       | L |
|                  |     documents.**                    |       | P |
|                  |                                     |       | * |
|                  | -   **Number of State and local     |       | * |
|                  |     governments trained in the      |       |   |
|                  |     understanding and use of health |       |   |
|                  |     services research findings      |       |   |
|                  |     through ULP Workshops .**       |       |   |
+------------------+-------------------------------------+-------+---+
| **00 & 99        | **00: Formation of a minimum of 5   | *     | * |
| Objective 3.1:   | partnerships to support             | *Over | * |
| Promote          | dissemination of AHRQ products      | 30    | C |
| distribution of  | through intermediary organizations, | publ  | B |
| AHRQ             | such as health plans and            | ic/pr | * |
| publications,    | professional organizations.**       | ivate | * |
| products, and    |                                     | and   |   |
| tools through    | **99: Formation of a minimum of 5   | pub   |   |
| intermediary     | partnerships to support             | lic/p |   |
| organizations.** | dissemination of AHRQ products      | ublic |   |
|                  | through intermediary organizations, | pa    |   |
|                  | such as health plans and            | rtner |   |
|                  | professional organizations.**       | ships |   |
|                  |                                     | form  |   |
|                  |                                     | ed.** |   |
|                  |                                     |       |   |
|                  |                                     | *     |   |
|                  |                                     | *Over |   |
|                  |                                     | 30    |   |
|                  |                                     | publ  |   |
|                  |                                     | ic/pr |   |
|                  |                                     | ivate |   |
|                  |                                     | and   |   |
|                  |                                     | pub   |   |
|                  |                                     | lic/p |   |
|                  |                                     | ublic |   |
|                  |                                     | pa    |   |
|                  |                                     | rtner |   |
|                  |                                     | ships |   |
|                  |                                     | for   |   |
|                  |                                     | med** |   |
+------------------+-------------------------------------+-------+---+
| **01 - 02        | **02:**                             |       | * |
| Objective 3.2:   |                                     |       | * |
| Develop and      | -   **Produce evidence summaries    |       | C |
| facilitate the   |     for use in Federal direct care  |       | B |
| use of new       |     providers**= **efforts to       |       | * |
| tools, talent,   |     create guidelines.**            |       | * |
| products, and    |                                     |       |   |
| implementation   | -   **Evidence-based practice       |       | * |
| methodologies    |     centers (EPCs) will produce a   |       | * |
| stemming from    |     minimum of 18 evidence reports  |       | C |
| research         |     and technology assessments that |       | B |
| portfolio. (This |     can serve as the basis for      |       | * |
| is objective 3.3 |     interventions to enhance health |       | * |
| in FY 99-00)**   |     outcomes and quality by         |       |   |
|                  |     improving practice.**           |       | * |
|                  |                                     |       | * |
|                  | -   **Fund at least 10 projects in  |       | C |
|                  |     tool and data development.**    |       | B |
|                  |                                     |       | * |
|                  |                                     |       | * |
+------------------+-------------------------------------+-------+---+
|                  | **01:**                             | **Co  | * |
|                  |                                     | mplet | * |
|                  | -   **Produce evidence summaries    | ed.** | C |
|                  |     for use in Federal direct care  |       | B |
|                  |     providers**= **efforts to       | **Co  | * |
|                  |     create guidelines.**            | mplet | * |
|                  |                                     | ed.** |   |
|                  | -   **EPCs will produce a minimum   |       |   |
|                  |     of 12 evidence reports and      | **Co  |   |
|                  |     technology assessments that can | mplet |   |
|                  |     serve as the basis for          | ed.** |   |
|                  |     interventions to enhance health |       |   |
|                  |     outcomes and quality by         | **Co  |   |
|                  |     improving practice.**           | mplet |   |
|                  |                                     | ed.** |   |
|                  | -   **Support a minimum of 165 pre- |       |   |
|                  |     and post-doctoral trainees.**   | **Co  |   |
|                  |                                     | mplet |   |
|                  | -   **Support up to 3 Minority      | ed.** |   |
|                  |     Research Infrastructure Support |       |   |
|                  |     Program IM-RISP) grants in      |       |   |
|                  |     order to develop the health     |       |   |
|                  |     services research capabilities  |       |   |
|                  |     of traditionally                |       |   |
|                  |     minority-serving                |       |   |
|                  |     institutions.**                 |       |   |
|                  |                                     |       |   |
|                  | -   **Support up to 6 Building      |       |   |
|                  |     Research Infrastructure and     |       |   |
|                  |     Capacity (BRIC) two-year        |       |   |
|                  |     planning grants in EPSCOR       |       |   |
|                  |     states and states which         |       |   |
|                  |     historically have received      |       |   |
|                  |     little or no research support   |       |   |
|                  |     from AHRQ.**                    |       |   |
|                  |                                     |       |   |
|                  | -   **Fund at least 10 projects in  |       |   |
|                  |     tool development.**             |       |   |
+------------------+-------------------------------------+-------+---+
| **99 - 00        | **00:**                             | *     | * |
| Objective 3.2:   |                                     | *18.8 | * |
| Maximize         | -   **Number of hits on the Web     | mi    | B |
| dissemination of |     site**                          | llion | : |
| information,     |                                     | h     | 5 |
| tools, and       | -   **Number of inquiries handled   | its** | 7 |
| products         |     on web site.**                  |       | * |
| developed from   |                                     | **3,  | * |
| research results | -   **Number of Uploaded            | 500** |   |
| for use in       |     documents**                     |       | * |
| practice         |                                     | **4,  | * |
| settings.        | -   **Reports from user surveys on  | 400** | B |
| (Becomes         |     how the information requested   |       | : |
| Objective 3/1 in |     was used.**                     | **C   | 5 |
| FY 01.)**        |                                     | omple | 7 |
|                  | -   **Number of State and local     | ted** | * |
|                  |     governments trained in the      |       | * |
|                  |     understanding and use of health | **St  |   |
|                  |     services research findings      | ate - | * |
|                  |     through User Liaison Program    | 50**  | * |
|                  |     (ULP) Workshops**               |       | B |
|                  |                                     | **Lo  | : |
|                  | -   **+ Meetings held.**            | cal - | 5 |
|                  |                                     | 29**  | 7 |
|                  | -   **+ Number of attendees**       |       | * |
|                  |                                     | **17  | * |
|                  | -   **+ States represented.**       | meeti |   |
|                  |                                     | ngs** | * |
|                  | -   **Reports from annual           |       | * |
|                  |     participants on how the         | *     | B |
|                  |     information was used in         | *1196 | : |
|                  |     decision-making.**              | atte  | 5 |
|                  |                                     | nd.** | 9 |
|                  | -   **Statistics on usage of        |       | * |
|                  |     National Guideline              | **50  | * |
|                  |     Clearinghouse including number  | Sta   |   |
|                  |     of hits, requests,              | tes + | * |
|                  |     organizations, and total        | D     | * |
|                  |     users.**                        | .C.** | C |
|                  |                                     |       | B |
|                  | -   **Survey of a sample of NGC     | **    | * |
|                  |     users to understand the impact  | Met** | * |
|                  |     of use on decisions and patient |       |   |
|                  |     care.**                         | **    | * |
|                  |                                     | Hits: | * |
|                  | -   **At least 10                   | 32    | C |
|                  |     purchasers/businesses use AHRQ  | ,234, | B |
|                  |     findings to make decisions.**   | 401** | * |
|                  |                                     |       | * |
|                  |                                     | **R   |   |
|                  |                                     | eques | * |
|                  |                                     | ts:** | * |
|                  |                                     |       | C |
|                  |                                     | **18  | B |
|                  |                                     | ,207, | * |
|                  |                                     | 430** | * |
|                  |                                     |       |   |
|                  |                                     | **    | * |
|                  |                                     | Org** | * |
|                  |                                     | =**s: | C |
|                  |                                     | 58,   | B |
|                  |                                     | 803** | * |
|                  |                                     |       | * |
|                  |                                     | *     |   |
|                  |                                     | *User | * |
|                  |                                     | sessi | * |
|                  |                                     | ons** | C |
|                  |                                     |       | B |
|                  |                                     | **1.5 | * |
|                  |                                     | mill  | * |
|                  |                                     | ion** |   |
|                  |                                     |       | * |
|                  |                                     | **902 | * |
|                  |                                     | res   | B |
|                  |                                     | ponde | : |
|                  |                                     | nts** | 2 |
|                  |                                     |       | 4 |
|                  |                                     | **    | - |
|                  |                                     | Evalu | 2 |
|                  |                                     | ation | 5 |
|                  |                                     | to be | * |
|                  |                                     | comp  | * |
|                  |                                     | leted |   |
|                  |                                     | in    | * |
|                  |                                     | mid   | * |
|                  |                                     | -year | B |
|                  |                                     | 2     | 2 |
|                  |                                     | 001** | 5 |
|                  |                                     |       | : |
|                  |                                     | **11  | * |
|                  |                                     | examp | * |
|                  |                                     | les** |   |
|                  |                                     |       |   |
|                  |                                     | **lis |   |
|                  |                                     | ted** |   |
+------------------+-------------------------------------+-------+---+
| **Goal 3         | **99:**                             | **15  |   |
| continued:**     |                                     | .5M** |   |
|                  | -   **Number of hits on the Web     |       |   |
| **Objective      |     site**                          | **2,  |   |
| 3.2**            |                                     | 950** |   |
|                  | -   **Number of inquiries handled   |       |   |
|                  |     on web site.**                  | **4,  |   |
|                  |                                     | 000** |   |
|                  | -   **Number of Uploaded            |       |   |
|                  |     documents**                     | **48  |   |
|                  |                                     | st    |   |
|                  | -   **Number of State and local     | ates; |   |
|                  |     governments trained in the      | 4     |   |
|                  |     understanding and use of health | te    |   |
|                  |     services research findings      | rrito |   |
|                  |     through User Liaison Program    | ries; |   |
|                  |     (ULP) Workshops**               | 30    |   |
|                  |                                     | c     |   |
|                  | -   **+ Meetings held.**            | ounty |   |
|                  |                                     | g     |   |
|                  | -   **+ Number of attendees**       | ov**= |   |
|                  |                                     | **ts; |   |
|                  | -   **+ States represented.**       | 9     |   |
|                  |                                     | city  |   |
|                  | -   **Statistics on usage of        | go    |   |
|                  |     National Guideline              | v**=* |   |
|                  |     Clearinghouse including number  | *ts** |   |
|                  |     of hits, requests,              |       |   |
|                  |     organizations, and total        | *     |   |
|                  |     users.**                        | *18** |   |
|                  |                                     |       |   |
|                  | -   **At least 5                    | **    |   |
|                  |     purchasers/businesses use AHRQ  | 834** |   |
|                  |     findings to make decisions.**   |       |   |
|                  |                                     | *     |   |
|                  |                                     | *48** |   |
|                  |                                     |       |   |
|                  |                                     | **13  |   |
|                  |                                     | ,590, |   |
|                  |                                     | 013** |   |
|                  |                                     |       |   |
|                  |                                     | **21  |   |
|                  |                                     | exa   |   |
|                  |                                     | mples |   |
|                  |                                     | list  |   |
|                  |                                     | ed.** |   |
|                  |                                     |       |   |
|                  |                                     | **Fu  |   |
|                  |                                     | rther |   |
|                  |                                     | det   |   |
|                  |                                     | ails, |   |
|                  |                                     | p     |   |
|                  |                                     | 54**  |   |
+------------------+-------------------------------------+-------+---+
| **Objective 3.3  | **00**                              | **    |   |
| (This becomes    |                                     | Met** |   |
| objective 3.2 in | -   **Demonstration of use of at    |       |   |
| FY 01. Objective |     least 3 AHRQ research findings  | **29  |   |
| 3.3 is           |     in systematic efforts to        | Pro   |   |
| discontinued in  |     Translate Research Into         | jects |   |
| FY 01.)**        |     Practice.**                     | fun   |   |
|                  |                                     | ded** |   |
| **99 - 00        | -   **Funding of a minimum of 5     |       |   |
| Objective 3.3:   |     major projects that will        | **23  |   |
| Develop and      |     develop products, tools, or     | Lis   |   |
| facilitate the   |     methodologies for implementing  | ted** |   |
| use of new       |     research findings into practice |       |   |
| tools, talent,   |     in significant segments of the  | **40% |   |
| products, and    |     health care system (i.e.,       | Incre |   |
| implementation   |     potential to be generalizable   | ase** |   |
| methodologies    |     across health care systems,     |       |   |
| stemming from    |     provider-types, or clinical     | **(fu |   |
| research         |     areas.)**                       | rther |   |
| portfolio.**     |                                     | det   |   |
|                  | -   **At least 2 new tools,         | ails, |   |
|                  |     products, or methodologies      | P.    |   |
|                  |     become available from projects  | 89)** |   |
|                  |     funded between FY 1993 and FY   |       |   |
|                  |     1996.**                         |       |   |
|                  |                                     |       |   |
|                  | -   **Support a five percent        |       |   |
|                  |     increase, at a minimum, in      |       |   |
|                  |     number of pre- and              |       |   |
|                  |     post-doctoral trainees.**       |       |   |
+------------------+-------------------------------------+-------+---+
| **Goal 3,        | **99**                              | **10  |   |
| continued:**     |                                     | prod  |   |
|                  | -   **Evidence-based Practice       | uced; |   |
| **Objective      |     Centers (EPCs) produce a        | 3**   |   |
| 3.3**            |     minimum of 12 evidence reports  | A**in |   |
|                  |     and technology assessments that | p     |   |
|                  |     can serve as the basis for      | ress* |   |
|                  |     interventions to enhance health | *@**; |   |
|                  |     outcomes and quality by         | 30    |   |
|                  |     improving practice (i.e.,       | under |   |
|                  |     practice guidelines, quality    | dev   |   |
|                  |     measures, and other quality     | elopm |   |
|                  |     improvement tools). At least    | ent** |   |
|                  |     four reports are being used by  |       |   |
|                  |     customers to develop practice   | **Rel |   |
|                  |     guidelines or other             | eased |   |
|                  |     interventions.**                | March |   |
|                  |                                     | 1999; |   |
|                  | -   **The AHRQ software product,    | con   |   |
|                  |     CONQUEST 2.0 released in FY     | tract |   |
|                  |     1999 containing new measures,   | to be |   |
|                  |     including measures for new      | aw    |   |
|                  |     conditions, and updated         | arded |   |
|                  |     measures. Contract awarded to   | 9     |   |
|                  |     create web-based product for    | /00** |   |
|                  |     more timely updating of         |       |   |
|                  |     information contained within    | **13  |   |
|                  |     product.**                      | exa   |   |
|                  |                                     | mples |   |
|                  | -   **Funding of a minimum of 5     | provi |   |
|                  |     major projects that will        | ded** |   |
|                  |     develop products, tools, or     |       |   |
|                  |     methodologies for implementing  | **15  |   |
|                  |     research findings into practice | exa   |   |
|                  |     in significant segments of the  | mples |   |
|                  |     health care system (i.e.,       | provi |   |
|                  |     potential to be generalizable   | ded** |   |
|                  |     across health care systems,     |       |   |
|                  |     provider-types, or clinical     | **167 |   |
|                  |     areas.)**                       | train |   |
|                  |                                     | ees** |   |
|                  | -   **At least two new tools,       |       |   |
|                  |     products, or methodologies      | **Fu  |   |
|                  |     become available from projects  | rther |   |
|                  |     funded between FY 1993 and FY   | det   |   |
|                  |     1996**                          | ails, |   |
|                  |                                     | p.    |   |
|                  | -   **Support a minimum of 150 pre- | 56**  |   |
|                  |     and post-doctoral trainees.**   |       |   |
+------------------+-------------------------------------+-------+---+

  -----------------------------------------------------------------------
  **GPRA Goal 4: Evaluate the effectiveness and impact of AHRQ research
  and associated activities.**

  -----------------------------------------------------------------------

+------------------+-------------------------------------+-------+---+
| **Performance    | **FY Targets**                      | **A   | * |
| Objective**      |                                     | ctual | * |
|                  |                                     | Per   | R |
|                  |                                     | forma | e |
|                  |                                     | nce** | f |
|                  |                                     |       | e |
|                  |                                     |       | r |
|                  |                                     |       | - |
|                  |                                     |       | e |
|                  |                                     |       | n |
|                  |                                     |       | c |
|                  |                                     |       | e |
|                  |                                     |       | * |
|                  |                                     |       | * |
+------------------+-------------------------------------+-------+---+
| **01 - 02        | **02:** **Evaluate the impact of    | **Co  | * |
| Objective 4.1    | the CERTS program in disseminating  | mplet | * |
| Evaluate the     | information regarding therapeutics  | ed.** | C |
| impact of AHRQ   | to at least 3 health care providers |       | B |
| sponsored        | or others in order to improve       | **Co  | * |
| products in      | practice.**                         | mplet | * |
| advancing        |                                     | ed.** |   |
| methods to       | -   **Evaluation to determine       |       | * |
| measure and      |     whether AHRQ funded studies in  | **Co  | * |
| improve health   |     methodological development have | mplet | C |
| care.**          |     been effective in developing at | ed.** | B |
|                  |     least 3 new research            |       | * |
| **NOTE: 99 - 00  |     techniques, whether the         |       | * |
| Objectives 4.1   |     techniques are being            |       |   |
| and 4.2 have     |     implemented, and how these      |       | * |
| been             |     studies could be improved.**    |       | * |
| consolidated in  |                                     |       | C |
| the FY 01        | -   **Evaluation of the outcomes of |       | B |
| plan.**          |     the pharmaceutical studies the  |       | * |
|                  |     Agency has funded to assess     |       | * |
|                  |     impact.**                       |       |   |
|                  |                                     |       |   |
|                  | -   **Interim assessment of the     |       |   |
|                  |     impact of the management system |       |   |
|                  |     for tracking project            |       |   |
|                  |     profiles.**                     |       |   |
|                  |                                     |       |   |
|                  | -   **Qualitative review by experts |       |   |
|                  |     of results of one major         |       |   |
|                  |     research initiative to assess   |       |   |
|                  |     quality and productivity and    |       |   |
|                  |     potential impact.**             |       |   |
|                  |                                     |       |   |
|                  | -   **Evaluate private sector use   |       |   |
|                  |     of at least 5 AHRQ findings.**  |       |   |
|                  |                                     |       |   |
|                  | -   **Identify at least 5 private   |       |   |
|                  |     sector uses of AHRQ findings,   |       |   |
|                  |     and describe any assessment of  |       |   |
|                  |     the impact on clinical practice |       |   |
|                  |     and/or patient care.**          |       |   |
|                  |                                     |       |   |
|                  | ***Evidence-based Practice          |       |   |
|                  | Centers***                          |       |   |
|                  |                                     |       |   |
|                  | -   **Use of evidence reports and   |       |   |
|                  |     technology assessments to       |       |   |
|                  |     create quality improvement      |       |   |
|                  |     tools in at least 10            |       |   |
|                  |     organizations.**                |       |   |
|                  |                                     |       |   |
|                  | -   **For at least four evidence    |       |   |
|                  |     reports or technology           |       |   |
|                  |     assessments per year, work with |       |   |
|                  |     partners to measure how the     |       |   |
|                  |     reports or assessments were     |       |   |
|                  |     used and what impact they had   |       |   |
|                  |     on clinical decision making and |       |   |
|                  |     patient care.**                 |       |   |
|                  |                                     |       |   |
|                  | -   **Findings from at least 3      |       |   |
|                  |     evidence reports or technology  |       |   |
|                  |     assessments will effect State   |       |   |
|                  |     or Federal health policy        |       |   |
|                  |     decisions.**                    |       |   |
|                  |                                     |       |   |
|                  | -   **Use of evidence reports or    |       |   |
|                  |     technology assessments and      |       |   |
|                  |     access to NGC site informed     |       |   |
|                  |     organizational decision making  |       |   |
|                  |     in at least 4 cases and         |       |   |
|                  |     resulted in changes in health   |       |   |
|                  |     care processes, quality, or     |       |   |
|                  |     health outcomes.**              |       |   |
|                  |                                     |       |   |
|                  | **01: *Evidence-based Practice      |       |   |
|                  | Centers***                          |       |   |
|                  |                                     |       |   |
|                  | -   **Use of evidence reports and   |       |   |
|                  |     technology assessments to       |       |   |
|                  |     create quality improvement      |       |   |
|                  |     tools in at least 10            |       |   |
|                  |     organizations.**                |       |   |
|                  |                                     |       |   |
|                  | -   **For at least four evidence    |       |   |
|                  |     reports or technology           |       |   |
|                  |     assessments per year, work with |       |   |
|                  |     partners to measure how the     |       |   |
|                  |     reports or**                    |       |   |
|                  |                                     |       |   |
|                  | > **assessments were used and what  |       |   |
|                  | > impact they had on clinical       |       |   |
|                  | > decision making and patient       |       |   |
|                  | > care.**                           |       |   |
|                  |                                     |       |   |
|                  | ***Research***                      |       |   |
|                  |                                     |       |   |
|                  | -   **At least 3 examples of how    |       |   |
|                  |     research informed changes in    |       |   |
|                  |     policies or practices in other  |       |   |
|                  |     Federal agencies.**             |       |   |
+------------------+-------------------------------------+-------+---+
| **Goal 4         | -   **Findings from at least 3      | **Co  | * |
| continued:**     |     evidence reports or technology  | mplet | * |
|                  |     assessments will effect State   | ed.** | C |
| **01 Objective   |     or Federal health policy        |       | B |
| 4.1 Evaluate the |     decisions.**                    | **Co  | * |
| impact of AHRQ   |                                     | mplet | * |
| sponsored        | -   **Use of evidence reports or    | ed.** |   |
| products in      |     technology assessments and      |       | * |
| advancing        |     access to NGC site informed     | **Co  | * |
| methods to       |     organizational decision-making  | mplet | C |
| measure and      |     in at least 4 cases and         | ed.** | B |
| improve health   |     resulted in changes in health   |       | * |
| care. *Cont.***  |     care procedures or health       | **Co  | * |
|                  |     outcomes.**                     | mplet |   |
|                  |                                     | ed.** | * |
|                  | ***Quality Measures***              |       | * |
|                  |                                     | **Co  | C |
|                  | -   **Achievable Benchmarks of Care | mplet | B |
|                  |     are used for quality            | ed.** | * |
|                  |     improvement activities by Peer  |       | * |
|                  |     Review Organizations**          | **Co  |   |
|                  |                                     | mplet | * |
|                  | -   **Use of dental performance     | ed.** | * |
|                  |     measures by dental service and  |       | C |
|                  |     insurance organizations.**      | **Co  | B |
|                  |                                     | mplet | * |
|                  | -   **HCUP quality indicators       | ed.** | * |
|                  |     incorporated into efforts by    |       |   |
|                  |     hospital associations and       | **Co  | * |
|                  |     hospitals to improve the        | mplet | * |
|                  |     quality of care.**              | ed.** | C |
|                  |                                     |       | B |
|                  | ***National Guideline               | **Co  | * |
|                  | Clearinghouse***                    | mplet | * |
|                  |                                     | ed.** |   |
|                  | -   **At least 10 users of the      |       |   |
|                  |     National Guideline              | **Co  |   |
|                  |     Clearinghouse will use site to  | mplet |   |
|                  |     inform clinical care            | ed.** |   |
|                  |     decisions**                     |       |   |
|                  |                                     |       |   |
|                  | -   **Guideline development or      |       |   |
|                  |     quality improvement efforts by  |       |   |
|                  |     users will be facilitated       |       |   |
|                  |     through use of NGC in at least  |       |   |
|                  |     5 cases.**                      |       |   |
|                  |                                     |       |   |
|                  | -   **NGC information will be used  |       |   |
|                  |     to inform health policy         |       |   |
|                  |     decisions in at least 2         |       |   |
|                  |     cases.**                        |       |   |
|                  |                                     |       |   |
|                  | -   **Improvements in clinical care |       |   |
|                  |     will result from utilization of |       |   |
|                  |     NGC information in at least 3   |       |   |
|                  |     cases.**                        |       |   |
|                  |                                     |       |   |
|                  | ***Training Programs***             |       |   |
|                  |                                     |       |   |
|                  | **Two thirds of former pre- and     |       |   |
|                  | postdoctoral institutional award    |       |   |
|                  | trainees are active in the conduct  |       |   |
|                  | or administration of health         |       |   |
|                  | services research.**                |       |   |
+------------------+-------------------------------------+-------+---+
| **00 Objective   | **00**                              | *     | * |
| 4.1 (& 4.2)\*:   |                                     | *HCUP | * |
| Evaluate the     | -   **AHRQ**=**s HCUP Quality       | QI*   | C |
| impact of AHRQ   |     Indicators(QI**=**s) will be    | *=**s | B |
| sponsored        |     redesigned based on             | defi  | * |
| products in      |     consultations with state policy | ned** | * |
| advancing        |     makers, researchers, hospital   |       |   |
| methods to       |     associations, and others about  | *     |   |
| measure and      |     their past use of the QI**=**s. | *Nati |   |
| improve health   |     By the end of March 2001, a new | onal- |   |
| care.**          |     set of quality indicators will  | level |   |
|                  |     be defined and feedback         | QI    |   |
| **\* Objectives  |     obtained from a new set of HCUP | i     |   |
| 4.1 and 4.2 were |     QI users. In addition, AHRQ     | nform |   |
| inadvertently    |     will provide access to recent   | ation |   |
| the same. They   |     national-level QI information   | p     |   |
| have been        |     via both the Internet and       | osted |   |
| consolidated to  |     through published reports, with | to    |   |
| simplify the     |     special focus on disseminating  | Int   |   |
| reporting.**     |     information to hospital users   | ernet |   |
|                  |     and organizations with          | Sept  |   |
|                  |     responsibility for hospital     | 2     |   |
|                  |     quality reporting.**            | 000** |   |
|                  |                                     |       |   |
|                  | -   **Use of evidence reports and   | **16  |   |
|                  |     technology assessments to       | exa   |   |
|                  |     create quality improvement      | mples |   |
|                  |     tools in at least 10            | lis   |   |
|                  |     organizations.**                | ted** |   |
|                  |                                     |       |   |
|                  | -   **For at least four evidence    | **4   |   |
|                  |     reports or technology           | exa   |   |
|                  |     assessments per year, work with | mples |   |
|                  |     partners to measure how the     | lis   |   |
|                  |     reports or assessments were     | ted** |   |
|                  |     used and what impact they had   |       |   |
|                  |     on clinical decisionmaking and  | **4   |   |
|                  |     patient care.**                 | exa   |   |
|                  |                                     | mples |   |
|                  | -   **At least three examples of    | lis   |   |
|                  |     how research informed changes   | ted** |   |
|                  |     in policies or practices in     |       |   |
|                  |     other Federal agencies.**       | **C   |   |
|                  |                                     | omple |   |
|                  | -   **AHRQ will report on the       | ted** |   |
|                  |     extent to which CONQUEST        |       |   |
|                  |     assists those who are charged   | **(de |   |
|                  |     with carrying out quality       | tails |   |
|                  |     measurement and improvement     | p.    |   |
|                  |     activities and the extent to    | 1     |   |
|                  |     which it helps further          | 02)** |   |
|                  |     state-of-the-art in clinical    |       |   |
|                  |     performance measurement.**      | **    |   |
|                  |                                     | Compl |   |
|                  | -   **CAHPS**7 **has assisted the   | eted. |   |
|                  |     Health Care Financing           | Re    |   |
|                  |     Administration (HCFA) in        | sults |   |
|                  |     informing Medicare              | to be |   |
|                  |     beneficiaries about their       | publ  |   |
|                  |     health care choices. The use    | ished |   |
|                  |     and impact of this information  | in    |   |
|                  |     is determined by surveying a    | July  |   |
|                  |     sample of these                 | 2     |   |
|                  |     beneficiaries.**                | 001** |   |
|                  |                                     |       |   |
|                  | -   **At least one quality measure  | **ABC |   |
|                  |     from Q-span (or instances where | S     |   |
|                  |     AHRQ research contributes to    | ystem |   |
|                  |     the development of measures)    | of    |   |
|                  |     are used in the Health Plan     | p     |   |
|                  |     Employer Data Information Set   | erfor |   |
|                  |     (HEDIS) by the National         | mance |   |
|                  |     Committee for Quality Assurance | prof  |   |
|                  |     (NCQA), measurement activities  | iling |   |
|                  |     of the Joint Commission for the | cited |   |
|                  |     Accreditation of Healthcare     | (de   |   |
|                  |     Organizations (JCAHO), or other | tails |   |
|                  |     organizations monitoring health | p.    |   |
|                  |     care quality.**                 | 1     |   |
|                  |                                     | 03)** |   |
+------------------+-------------------------------------+-------+---+
| **Goal 4         | **99**                              | **C   |   |
| continued:**     |                                     | omple |   |
|                  | -   **An evaluation of the outcomes | ted** |   |
| **99 Objective   |     of outcomes research and the    |       |   |
| 4.1 Evaluate the |     impact of AHRQ-supported        | **Pro |   |
| impact of AHRQ   |     outcomes and effectiveness      | gress |   |
| sponsored        |     research on clinical            | r     |   |
| products in      |     practice.**                     | eport |   |
| advancing        |                                     | (de   |   |
| methods to       | -   **An evaluation and synthesis   | tails |   |
| measure and      |     of (1) primary care research    | p.    |   |
| improve health   |     supported by AHRQ and (2) an    | 64)** |   |
| care. *Cont.***  |     assessment of the current state |       |   |
|                  |     of the science and future       | **C   |   |
|                  |     directions for primary care     | omple |   |
|                  |     research.**                     | ted** |   |
|                  |                                     |       |   |
|                  | -   **AHRQ**=**s state data         | **P   |   |
|                  |     strategy will be redesigned     | relim |   |
|                  |     based on consultations with     | inary |   |
|                  |     state policy makers,            | resul |   |
|                  |     researchers, hospital           | ts.** |   |
|                  |     associations, and others about  |       |   |
|                  |     their past use of data from the | **    |   |
|                  |     Healthcare Cost and Utilization | Final |   |
|                  |     Project (HCUP) as well as       | r     |   |
|                  |     additional data needs.**        | eport |   |
|                  |                                     | rec   |   |
|                  | -   **Results of the evaluation of  | eived |   |
|                  |     the Consumer Assessment of      | in    |   |
|                  |     Health Plan Study (CAHPS**7**)  | June  |   |
|                  |     will be used to improve the     | 20    |   |
|                  |     usability and usefulness of the | 00.** |   |
|                  |     tool. Findings are expected to  |       |   |
|                  |     show whether (a) the            |       |   |
|                  |     survey-based information from   |       |   |
|                  |     CAHPS**7 **helps consumers make |       |   |
|                  |     better health care              |       |   |
|                  |     decisions, (b) the information  |       |   |
|                  |     increases consumer confidence   |       |   |
|                  |     when choosing health care plan, |       |   |
|                  |     and (3) CAHPS**7 **is used by   |       |   |
|                  |     public and private              |       |   |
|                  |     organizations.**                |       |   |
|                  |                                     |       |   |
|                  | -   **Evaluation studies on: (1)    |       |   |
|                  |     the quality and usefulness of   |       |   |
|                  |     the evidence reports and        |       |   |
|                  |     technology assessments produced |       |   |
|                  |     by the Evidence-based Practice  |       |   |
|                  |     Centers and (2) the impact of   |       |   |
|                  |     the use of these products on    |       |   |
|                  |     the health care system will be  |       |   |
|                  |     developed and initiated in FY   |       |   |
|                  |     1999.**                         |       |   |
+------------------+-------------------------------------+-------+---+
| **01 - 02        | **02**                              | **Not | * |
| Objective 4.2:   |                                     | a     | * |
| Evaluate the     | -   **Have a fully functional       | ttain | C |
| impact of MEPS   |     MEPS-based MEDSIM model to      | ed.** | B |
| data and         |     allow simulation of the         |       | * |
| associated       |     potential impact of             | **Co  | * |
| products on      |     programmatic changes in health  | mplet |   |
| policymaking and |     care financing and delivery Dec | ed.** | * |
| research         |     2002.**                         |       | * |
| products.**      |                                     | **Co  | C |
|                  | -   **Produce baseline FY           | mplet | B |
|                  |     statistics on number of         | ed.** | * |
|                  |     MEPS-based articles published   |       | * |
|                  |     in peer review journals.**      |       |   |
|                  |                                     |       | * |
|                  | -   **Conduct customer satisfaction |       | * |
|                  |     survey for MEPS workshop        |       | C |
|                  |     participants to assess how MEPS |       | B |
|                  |     data is being used to inform    |       | * |
|                  |     research and public policy.**   |       | * |
|                  |                                     |       |   |
|                  | -   **Develop marketing plan to     |       | * |
|                  |     promote the MEPS-IC data to     |       | * |
|                  |     state officials Dec 2002.**     |       | C |
|                  |                                     |       | B |
|                  | -   **At least 5 examples of how    |       | * |
|                  |     research using MEPS has been    |       | * |
|                  |     used to inform decisions by     |       |   |
|                  |     Federal, state and private      |       | * |
|                  |     sector policymakers.**          |       | * |
|                  |                                     |       | C |
|                  | **01**                              |       | B |
|                  |                                     |       | * |
|                  | -   **Use of MEPS data in AHRQ      |       | * |
|                  |     research applications will      |       |   |
|                  |     increase by 10 percent over     |       | * |
|                  |     number received in baseline     |       | * |
|                  |     period of 2000**                |       | C |
|                  |                                     |       | B |
|                  | -   **Feedback from MEPS workshop   |       | * |
|                  |     participants indicating that    |       | * |
|                  |     they were useful and timely.**  |       |   |
|                  |                                     |       | * |
|                  | -   **At least 5 examples of how    |       | * |
|                  |     research using MEPS has been    |       | C |
|                  |     used to inform decisions by     |       | B |
|                  |     Federal, state and private      |       | * |
|                  |     sector policymakers.**          |       | * |
|                  |                                     |       |   |
|                  | **00**                              |       | * |
|                  |                                     |       | * |
|                  | > **See above 4.1 for 00**          |       | C |
|                  |                                     |       | B |
|                  |                                     |       | * |
|                  |                                     |       | * |
+------------------+-------------------------------------+-------+---+
| **99: Objective  | -   **AHRQ Clearinghouse customer   | **M   | * |
| 4.2: Evaluate    |     satisfaction rated at 98%.**    | et.** | * |
| major            |                                     |       | C |
| dissemination    | -   **Customer satisfaction data on | **81  | B |
| mechanisms.**    |     AHRQ consumer publications      | .3%** | * |
|                  |     (useful/relevant) rated at      |       | * |
|                  |     90%.**                          |       |   |
+------------------+-------------------------------------+-------+---+
| **01 Objective   | -   **Use of MEPS data in 1% of     | *     |   |
| 4.3 n/a**        |     research applications received  | *MEPS |   |
|                  |     by AHRQ.**                      | used  |   |
| **00 Objective   |                                     | in    |   |
| 4.3: Evaluate    | -   **Distribution of MEPS data     | 31%   |   |
| the impact of    |     sets to at least 2500           | of    |   |
| MEPS data and    |     requestors.**                   | f     |   |
| associated       |                                     | unded |   |
| products on      | -   **At least 5 examples of how    | proje |   |
| policymaking and |     research using MEPS has been    | cts** |   |
| research         |     used to inform decisions by     |       |   |
| projects.**      |     Federal, state, and private     | **    |   |
|                  |     sector policymakers.**          | 5,700 |   |
| **99: n/a**      |                                     | data  |   |
|                  | -   **Feedback from recipients of   | sets; |   |
|                  |     MEPS data indicating that the   | 379   |   |
|                  |     data were timely, useful, and   | CD    |   |
|                  |     of high significance.**         | R     |   |
|                  |                                     | OM**= |   |
|                  |                                     | **s** |   |
|                  |                                     |       |   |
|                  |                                     | **15  |   |
|                  |                                     | exa   |   |
|                  |                                     | mples |   |
|                  |                                     | gi    |   |
|                  |                                     | ven** |   |
|                  |                                     |       |   |
|                  |                                     | **M   |   |
|                  |                                     | et.** |   |
+------------------+-------------------------------------+-------+---+

  -----------------------------------------------------------------------
  **GPRA Goal 5: Support initiative to improve health care quality
  through leadership and research.**

  -----------------------------------------------------------------------

+--------------+-----------------------------------------+-------+----+
| *            | **FY Targets**                          | **A   | ** |
| *Performance |                                         | ctual | Re |
| Objective**  |                                         | Per   | fe |
|              |                                         | forma | r- |
|              |                                         | nce** | en |
|              |                                         |       | ce |
|              |                                         |       | ** |
+--------------+-----------------------------------------+-------+----+
| **00 - 02    | **02**                                  | **Co  | *  |
| Objective    |                                         | mplet | *B |
| 5.1: Conduct | -   **Integration of at least one       | ed.** | :3 |
| research to  |     private sector data source into the |       | 5- |
| help to      |     national**                          | **Co  | 36 |
| measure the  |                                         | mplet | ** |
| current      | > **quality report by 31 December       | ed.** |    |
| status**     | > 2002.**                               |       | ** |
|              |                                         | **    | CB |
| **health of  | **01**                                  | Final | ** |
| care quality |                                         | recom |    |
| in the       | -   **QI Taxonomy meeting held under    | mend. | ** |
| Nation.**    |     the auspices of the QuIC.**         | March | CB |
|              |                                         | 2     | ** |
| **99         | -   **Number of grants and contracts    | 001** |    |
| Objective    |     funded in FY2001 that will help to  |       | ** |
| 5.1: Provide |     fill gaps in the information        | **S   | CB |
| leadership   |     available to assess the national    | urvey | ** |
| for the      |     quality of care, or will help to    | compl |    |
| Executive    |     expand the use of current measures  | eted. | ** |
| Branch**=**s |     to provide a broader or richer      | Fi    | B: |
| Quality      |     picture of quality.**               | elded | 35 |
| Interagency  |                                         | d     | ** |
| Coordination | **00**                                  | uring |    |
| Task Force   |                                         | FY    | ** |
| (QuIC)**     | -   **Data sources identified that will | 2     | CB |
|              |     contribute information as part of   | 001** | ** |
|              |     the mosaic picture of quality of    |       |    |
|              |     care in the Nation.**               | **    |    |
|              |                                         | Final |    |
|              | -   **Develop and begin to test some    | r     |    |
|              |     questions to be added to existing   | eport |    |
|              |     data collection activities to       | 30    |    |
|              |     provide a better picture of         | March |    |
|              |     quality.**                          | 20    |    |
|              |                                         | 01.** |    |
|              | -   **Develop framework for National    |       |    |
|              |     Healthcare Quality Report.**        | *     |    |
|              |                                         | *Met: |    |
|              | -   **Provide leadership for the        | (de   |    |
|              |     Executive Branch**=**s Quality      | tails |    |
|              |     Interagency Coordination Task Force | pp.   |    |
|              |     (QuIC).**                           | 110-1 |    |
|              |                                         | 11)** |    |
|              | **99**                                  |       |    |
|              |                                         | **M   |    |
|              | -   **Collaborative work groups are     | et.** |    |
|              |     established under the QuIC under    |       |    |
|              |     take projects with direct           |       |    |
|              |     application to improving quality of |       |    |
|              |     care.**                             |       |    |
|              |                                         |       |    |
|              | -   **In addition to the work on        |       |    |
|              |     specific projects chosen by the     |       |    |
|              |     QuIC, communication is facilitated  |       |    |
|              |     on common issues such as: 1)        |       |    |
|              |     Implementation of the Bill of       |       |    |
|              |     Rights and Responsibilities from    |       |    |
|              |     the President**=**s Commission on   |       |    |
|              |     Consumer Protection and Quality in  |       |    |
|              |     the Health Care Industry; And 2)    |       |    |
|              |     organization or management          |       |    |
|              |     strategies to improve quality of    |       |    |
|              |     care.**                             |       |    |
+--------------+-----------------------------------------+-------+----+
| **Goal 5     | **02**                                  | **F   | ** |
| continued:** |                                         | unded | CB |
|              | -   **Funding of at least 5 projects    | \>    | ** |
| **00 - 02    |     bringing healthcare information to  | 40.** |    |
| Objective    |     the public in an understandable,    |       | *  |
| 5.2:         |     user friendly manner which          | **Co  | *B |
| Facilitate   |     facilitates its use in decision     | mplet | :3 |
| use of       |     making.**                           | ed.** | 1- |
| quality      |                                         |       | 35 |
| information  | **01**                                  | **3   | ** |
| to improve   |                                         | exa   |    |
| health care  | -   **Number of grants to improve       | mples | ** |
| in the       |     patient safety.**                   | g     | CB |
| Nation.**    |                                         | iven. | ** |
|              | -   **Adoption of Agency sponsored      | De    |    |
| **99         |     research and tools developed by one | tails |    |
| Objective    |     or more users to facilitate         | pp.   |    |
| 5.2: Conduct |     consumers/purchaser/decision- maker | 112** |    |
| research to  |     use of information about quality**  |       |    |
| expand the   |                                         | **M   |    |
| tool box of  | **00**                                  | et.** |    |
| measures and |                                         |       |    |
| risk         | -   **Development of at least one tool  | **M   |    |
| adjustment   |     that can be used by large group     | et.** |    |
| methods      |     purchasers in assisting their       |       |    |
| available    |     beneficiaries to choose the health  |       |    |
| help to      |     care plan, provider, or hospital    |       |    |
| measure the  |     that best meets their needs.**      |       |    |
| current      |                                         |       |    |
| status of    | **99**                                  |       |    |
| quality in   |                                         |       |    |
| the          | -   **Inventory of measures and risk    |       |    |
| nation.**    |     adjustment methods currently in use |       |    |
|              |     by Federal Agencies will be         |       |    |
|              |     developed.**                        |       |    |
|              |                                         |       |    |
|              | -   **Assessment of measures and risk   |       |    |
|              |     adjustment methods needed by        |       |    |
|              |     Federal Agencies will be            |       |    |
|              |     conducted.**                        |       |    |
+--------------+-----------------------------------------+-------+----+
| **00 - 02    | **02**                                  | **Co  | ** |
| Objective    |                                         | mplet | CB |
| 5.3: Improve | -   **Adoption of at least one quality  | ed.** | ** |
| quality      |     measure to be developed from our    |       |    |
| me           |     vulnerable populations RFA by a     | **Co  | ** |
| asurement.** |     national accrediting                | mplet | CB |
|              |     organization.**                     | ed.** | ** |
| **99         |                                         |       |    |
| Objective    | **01**                                  | *     | ** |
| 5.3: Inform  |                                         | *Met. | CB |
| health care  | -   **Identification of collaborators   | (De   | ** |
| or           |     for research projects on electronic | tails |    |
| ganizational |     medical records integrated with     | p.1   |    |
| leaders and  |     guidelines (e.g., from the          | 12)** |    |
| others how   |     Guideline Clearinghouse) or QI      |       |    |
| to design    |     indicators (e.g., CONQUEST, QI      | **M   |    |
| quality into |     Taxonomy project, HCUP measures).** | et.** |    |
| their        |                                         |       |    |
| systems**    | -   **Adoption of Living With Illness   |       |    |
|              |     children\'s health measure by       |       |    |
|              |     NCQA.**                             |       |    |
|              |                                         |       |    |
|              | **00**                                  |       |    |
|              |                                         |       |    |
|              | -   **Sponsor research to fill existing |       |    |
|              |     gaps in quality measures in areas   |       |    |
|              |     of high need.**                     |       |    |
|              |                                         |       |    |
|              | **99**                                  |       |    |
|              |                                         |       |    |
|              | -   **Review research conducted that    |       |    |
|              |     identifies appropriate ways of      |       |    |
|              |     redesigning health care delivery    |       |    |
|              |     systems to reduce errors.**         |       |    |
+--------------+-----------------------------------------+-------+----+
| **00:        | **99**                                  | **M   |    |
| Di           |                                         | et.** |    |
| scontinued** | -   **Research on effective             |       |    |
|              |     dissemination of information to     |       |    |
| **99         |     decisions makers including          |       |    |
| Objective    |     patients, clinicians,               |       |    |
| 5.4: Improve |     organizational leaders, purchasers, |       |    |
| u            |     and public policy makers            |       |    |
| nderstanding |     conducted.**                        |       |    |
| of how to    |                                         |       |    |
| ensure that  |                                         |       |    |
| research     |                                         |       |    |
| affects      |                                         |       |    |
| clinical     |                                         |       |    |
| practice as  |                                         |       |    |
| a            |                                         |       |    |
| ppropriate** |                                         |       |    |
+--------------+-----------------------------------------+-------+----+

**Budget Line 2: Medical Panel Expenditure Surveys**

+-----------------------------------------------------------------------+
| **GPRA Goal 6: Collect current data and create data tapes and         |
| associated products on**                                              |
|                                                                       |
| **health care use and expenditure for use by public and               |
| private-sector**                                                      |
|                                                                       |
| **decision makers and researchers. ( Medical Expenditure Panel        |
| Surveys)**                                                            |
+-----------------------------------------------------------------------+

+--------------------+-----------------------------------+-------+----+
| **Performance      | **FY Targets**                    | **A   | ** |
| Objective**        |                                   | ctual | Re |
|                    |                                   | Per   | fe |
|                    |                                   | forma | r- |
|                    |                                   | nce** | en |
|                    |                                   |       | ce |
|                    |                                   |       | ** |
+--------------------+-----------------------------------+-------+----+
| **99 - 02          | **02**                            |       | ** |
| Objective 6.1:     |                                   |       | CB |
| Release and        | -   **Develop a method to         |       | ** |
| disseminate MEPS   |     facilitate users**= **custom  |       |    |
| data and           |     cross tabulations of MEPS     |       |    |
| information        |     data.**                       |       |    |
| products in timely |                                   |       |    |
| manner for use by  | -   **Conduct six MEPS data user  |       |    |
| researchers,       |     workshops.**                  |       |    |
| policy makers,     |                                   |       |    |
| purchasers, and    | -   **Expand MEPS list-server     |       |    |
| plans.**           |     participation by 20%.**       |       |    |
|                    |                                   |       |    |
|                    | -   **Produce 4 Findings and at   |       |    |
|                    |     least one Chartbook.**        |       |    |
|                    |                                   |       |    |
|                    | -   **Develop Frequently Asked    |       |    |
|                    |     Questions Section for MEPS    |       |    |
|                    |     web site.**                   |       |    |
+--------------------+-----------------------------------+-------+----+
| **99 - 02          | **01**                            | **Co  | ** |
| Objective 6.1:     |                                   | mplet | CB |
| Release and        | -   **In FY2001, 1997 Use and     | ed.** | ** |
| disseminate MEPS   |     Expenditures, 2000            |       |    |
| data and           |     Point-in-Time, and 1998       | **Co  | ** |
| information        |     Health Insurance and          | mplet | CB |
| products in timely |     Demographics MEPS public use  | ed.** | ** |
| manner for use by  |     data files will be            |       |    |
| researchers,       |     released.**                   | *     | ** |
| policy makers,     |                                   | *Avai | CB |
| purchasers, and    | -   **Response time for requests  | lable | ** |
| plans. *Cont.***   |     received for information,     | w     |    |
|                    |     assistance or specific        | ithin | ** |
|                    |     products is as promised 95    | 12    | CB |
|                    |     percent of time**             | mos** | ** |
|                    |                                   |       |    |
|                    | **00**                            | **R   | ** |
|                    |                                   | eleas | CB |
|                    | -   **Core MEPS public use files  | ed:** | ** |
|                    |     (PUFs) available through Web  |       |    |
|                    |     site and CD-ROM within 9-18   | **    | ** |
|                    |     months after data collection  | -July | CB |
|                    |     completed.**                  | 2     | ** |
|                    |                                   | 000** |    |
|                    | -   **Specific products due in    |       | ** |
|                    |     FY2000:**                     | **    | CB |
|                    |                                   | -Avai | ** |
|                    | **+ 1999 point-in-time file**     | lable |    |
|                    |                                   | 1^st^ | ** |
|                    | **+ 1997 expenditure data**       | qu    | CB |
|                    |                                   | arter | ** |
|                    | **available**                     | 2     |    |
|                    |                                   | 001** |    |
|                    | **+ 1996 full panel file          |       |    |
|                    | available**                       | *     |    |
|                    |                                   | *-Jan |    |
|                    | -   **Customer satisfaction data  | 2     |    |
|                    |     from use of MEPS tapes and    | 000** |    |
|                    |     products rated at least       |       |    |
|                    |     90%.**                        | **    |    |
|                    |                                   | Rated |    |
|                    | -   **Response time for requests  | at    |    |
|                    |     received from policymakers,   | 90%** |    |
|                    |     purchasers and plans for MEPS |       |    |
|                    |     data tapes, analyses, and/or  | **96% |    |
|                    |     reports responded to within   | w     |    |
|                    |     promised time frames 95% of   | ithin |    |
|                    |     time.**                       | 4     |    |
|                    |                                   | d     |    |
|                    | **99**                            | ays** |    |
|                    |                                   |       |    |
|                    | -   **Core MEPS public use files  | **S   |    |
|                    |     (PUFs) available through      | ignif |    |
|                    |     Website and CD-ROM within     | icant |    |
|                    |     9-12 months after data        | pro   |    |
|                    |     collection completed.**       | gress |    |
|                    |                                   | ma    |    |
|                    | -   **Specific products due in FY | de.** |    |
|                    |     1999:**                       |       |    |
|                    |                                   | **De  |    |
|                    | **+ 1997 point-in-time file.**    | liver |    |
|                    |                                   | ed:** |    |
|                    | **+ 1996 full-year expenditure    |       |    |
|                    | file.**                           | **    |    |
|                    |                                   | March |    |
|                    | **+ 1996 full-year event, job,    | 1     |    |
|                    | and**                             | 999** |    |
|                    |                                   |       |    |
|                    | **condition files.**              | *     |    |
|                    |                                   | *Dec. |    |
|                    | **+ 1998 point- in- time file.**  | 1     |    |
|                    |                                   | 999** |    |
|                    |                                   |       |    |
|                    |                                   | **Job |    |
|                    |                                   | and   |    |
|                    |                                   | Cond  |    |
|                    |                                   | ition |    |
|                    |                                   | Files |    |
|                    |                                   | deli  |    |
|                    |                                   | vered |    |
|                    |                                   | Nov   |    |
|                    |                                   | ember |    |
|                    |                                   | 1999  |    |
|                    |                                   | and   |    |
|                    |                                   | A     |    |
|                    |                                   | ugust |    |
|                    |                                   | 1999  |    |
|                    |                                   | res   |    |
|                    |                                   | pecti |    |
|                    |                                   | vely; |    |
|                    |                                   | event |    |
|                    |                                   | files |    |
|                    |                                   | will  |    |
|                    |                                   | be    |    |
|                    |                                   | avai  |    |
|                    |                                   | lable |    |
|                    |                                   | by    |    |
|                    |                                   | M     |    |
|                    |                                   | arch, |    |
|                    |                                   | 2     |    |
|                    |                                   | 000** |    |
|                    |                                   |       |    |
|                    |                                   | *     |    |
|                    |                                   | *Dec. |    |
|                    |                                   | 1     |    |
|                    |                                   | 999** |    |
+--------------------+-----------------------------------+-------+----+
| **Goal 6           | -   **Research findings and       | **30+ | ** |
| continued:**       |     survey reports developed and  | pu    | B: |
|                    |     disseminated for use by       | blica | A5 |
| **Objective 6.1**  |     policy makers and researchers | tions | -A |
|                    |     including *MEPS* *Research    | re    | -6 |
|                    |     Findings*, *MEPS Highlights*, | lated | ** |
|                    |     chart books, peer-reviewed    | to    |    |
|                    |     journal articles, book        | M     | ** |
|                    |     published on contributions of | EPS** | CB |
|                    |     expenditure surveys to policy |       | ** |
|                    |     making, publications oriented | **Ra  |    |
|                    |     toward non-researchers.)**    | tings | ** |
|                    |                                   | be    | CB |
|                    | -   **Customer satisfaction data  | tween | ** |
|                    |     from use of MEPS tapes and    | 86-   |    |
|                    |     products rated at 85%.**      | 96%** |    |
|                    |                                   |       |    |
|                    | -   **Requests received from      | **Req |    |
|                    |     policy makers, purchasers and | uests |    |
|                    |     plans for MEPS data tapes,    | f     |    |
|                    |     analyses, and/or reports      | illed |    |
|                    |     responded to within promised  | w     |    |
|                    |     time frames 85% of the        | ithin |    |
|                    |     time.**                       | 5     |    |
|                    |                                   | days  |    |
|                    |                                   | un    |    |
|                    |                                   | iform |    |
|                    |                                   | ly.** |    |
|                    |                                   |       |    |
|                    |                                   | **p   |    |
|                    |                                   | .81** |    |
+--------------------+-----------------------------------+-------+----+
| **Goal 6           | **02**                            | **Co  | ** |
| continued:**       |                                   | mplet | CB |
|                    | -   **Determine the feasibility   | ed.** | ** |
| **99 - 02          |     of existing mechanisms to     |       |    |
| Objective 6.2:     |     provide off-site access to    | **Def | ** |
| Facilitate use of  |     confidential MEPS data.**     | erred | CB |
| MEPS data and      |                                   | to    | ** |
| associated         | -   **Expand data center capacity | Jan   |    |
| products as tools  |     by 10% over FY 01 level.**    | 2001  | ** |
| by extramural      |                                   | (De   | CB |
| researchers,       | **01**                            | tails | ** |
| policy makers,     |                                   | p.1   |    |
| purchasers, and    | -   **Establish baseline for Data | 16)** | ** |
| plans.**           |     Center use capacity.**        |       | CB |
|                    |                                   | **Inc | ** |
|                    | **00**                            | luded |    |
|                    |                                   | in 20 |    |
|                    | -   **Data centers operational**  | app   |    |
|                    |                                   | licat |    |
|                    | **+ \# requests for use of the    | ions, |    |
|                    | centers**                         | 5     |    |
|                    |                                   | fund  |    |
|                    | **+ \# user-days at the data      | ed.** |    |
|                    | centers**                         |       |    |
|                    |                                   | **M   |    |
|                    | **+ \# projects completed**       | et.** |    |
|                    |                                   |       |    |
|                    | **99**                            | **    |    |
|                    |                                   | Fully |    |
|                    | -   **Inclusion of MEPS data in   | up    |    |
|                    |     extramural research grants    | Feb.  |    |
|                    |     with AHRQ and other           | 20    |    |
|                    |     funders.**                    | 00.** |    |
|                    |                                   |       |    |
|                    | -   **Plan for extramural         |       |    |
|                    |     researcher access to MEPS     |       |    |
|                    |     data fully implemented**      |       |    |
+--------------------+-----------------------------------+-------+----+
| **01 - 02          | **02**                            | **Co  | ** |
| Objective 6.3:     |                                   | mplet | B: |
| Modify MEPS to     | -   **Process and make available  | ed.** | 78 |
| support annual     |     data to be included in the    |       | ** |
| reporting on       |     National Quality Report.**    | **Co  |    |
| quality, health    |                                   | mplet | ** |
| care disparities,  | -   **Begin data collection to    | ed.** | B: |
| and research on    |     support the disparities       |       | 78 |
| long-term care in  |     report Sept 2002.**           | **Not | ** |
| adults and         |                                   | fund  |    |
| children with      | **01**                            | ed.** | ** |
| special needs.**   |                                   |       | CB |
|                    | -   **Data collection begins on   | **Co  | ** |
|                    |     the treatment of common       | mplet |    |
|                    |     clinical conditions over time | ed.** | ** |
|                    |     for a nationally              |       | CB |
|                    |     representative portion of the | **Not | ** |
|                    |     population in support of the  | fund  |    |
|                    |     National Healthcare Quality   | ed.** |    |
|                    |     Report.**                     |       |    |
|                    |                                   | **Co  |    |
|                    | ```{=html}                        | mplet |    |
|                    | <!-- -->                          | ed.** |    |
|                    | ```                               |       |    |
|                    | -   **LTC Measures:**             |       |    |
|                    |                                   |       |    |
|                    | > **1. Have developed data use    |       |    |
|                    | > agreements (DUA) with HCFA to   |       |    |
|                    | > assess and begin data           |       |    |
|                    | > development related to the      |       |    |
|                    | > MDS.**                          |       |    |
|                    | >                                 |       |    |
|                    | > **2. Design MEPS over sample of |       |    |
|                    | > adults with functional          |       |    |
|                    | > limitations and children with   |       |    |
|                    | > special needs.**                |       |    |
|                    | >                                 |       |    |
|                    | > **3. Produce one report related |       |    |
|                    | > to LTC.**                       |       |    |
|                    | >                                 |       |    |
|                    | > **4. Have developed IAA with    |       |    |
|                    | > NCHS for LTC frame development  |       |    |
|                    | > activities.**                   |       |    |
|                    | >                                 |       |    |
|                    | > **5. Submit at least one        |       |    |
|                    | > peer-reviewed publication in    |       |    |
|                    | > the area of LTC.**              |       |    |
+--------------------+-----------------------------------+-------+----+
| **00 Objective     | **00:**                           | *     |    |
| 6.3: Modify and    |                                   | *Met. |    |
| enhance MEPS to    | -   **The design decisions        | (De   |    |
| enable reporting   |     necessary for the expansion   | tails |    |
| on the quality of  |     of MEPS databases in order to | pp.   |    |
| health care in     |     collect data that will        | 117-1 |    |
| America as part of |     support the National          | 19)** |    |
| FY 2000 Priority   |     Healthcare Quality Report are |       |    |
| (3),** A**New      |     completed by August 2000. The |       |    |
| Tools for a New    |     design decisions will be      |       |    |
| Century.**@        |     operationalized in the coming |       |    |
|                    |     fiscal years.**               |       |    |
+--------------------+-----------------------------------+-------+----+
| **99 Objective     | **99**                            | **M   |    |
| 6.3: Modify and    |                                   | et.** |    |
| enhance MEPS to    | -   **MEPS Household Survey:      |       |    |
| enable reporting   |     Interviews with 9,000         | **M   |    |
| on the quality of  |     previously surveyed families  | et.** |    |
| health care in     |     to obtain calendar year 1998  |       |    |
| America.**         |     health care data, and with    | **M   |    |
|                    |     5,600 new families.**         | et.** |    |
|                    |                                   |       |    |
|                    | -   **MEPS Medical Provider       | **M   |    |
|                    |     Survey: Interviews with       | et.** |    |
|                    |     approximately 3,000           |       |    |
|                    |     facilities, 12,000            | **p.  |    |
|                    |     office-based providers, 7,000 | 82**  |    |
|                    |     hospital-identified           |       |    |
|                    |     physicians, and more than 500 |       |    |
|                    |     home health providers.**      |       |    |
|                    |                                   |       |    |
|                    | -   **MEPS Insurance Component    |       |    |
|                    |     (MEPS-IC): Interviews with    |       |    |
|                    |     more than 40,000 employers    |       |    |
|                    |     and 1,000 insurance           |       |    |
|                    |     carriers.**                   |       |    |
|                    |                                   |       |    |
|                    | -   **MEPS data collection        |       |    |
|                    |     successfully moved to ongoing |       |    |
|                    |     survey mode from data         |       |    |
|                    |     collection every ten years.** |       |    |
+--------------------+-----------------------------------+-------+----+

**Budget Line 3: Program Support**

+-----------------------------------------------------------------------+
| **GPRA Goal 7: Support the overall direction and management of AHRQ** |
|                                                                       |
| **Summary of Performance Objectives**                                 |
+-----------------------------------------------------------------------+

+---------------------+----------------------------------+-------+----+
| **Performance       | **FY Targets**                   | **A   | ** |
| Objective**         |                                  | ctual | Re |
|                     |                                  | Per   | fe |
|                     |                                  | forma | r- |
|                     |                                  | nce** | en |
|                     |                                  |       | ce |
|                     |                                  |       | ** |
+---------------------+----------------------------------+-------+----+
| **Objective 7.1 is  |                                  |       |    |
| mandatory (Capital  |                                  |       |    |
| Assets) but not     |                                  |       |    |
| applicable to       |                                  |       |    |
| AHRQ.**             |                                  |       |    |
+---------------------+----------------------------------+-------+----+
| **00-99 Objective   | **01 DISCONTINUED**              | **    | ** |
| 7.2:Maintain        |                                  | Rated | CB |
| acquisition         | **00**                           | 4.2   | ** |
| performance         |                                  | (Deta |    |
| management system   | -   **Internal customer          | ils** | ** |
| to ensure: (1)      |     satisfaction rated at        |       | CB |
| timely completion   |     minimum of 4.5/5.**          | **p.  | ** |
| of transactions,    |                                  | 1     |    |
| (2) vendor and      | -   **External customer          | 23)** | ** |
| customer            |     satisfaction rated at        |       | CB |
| satisfaction, and   |     4.5/5.**                     | **    | ** |
| (3) efficient and   |                                  | Rated |    |
| effective use of    | -   **Customer satisfaction      | 4.6** | ** |
| resources.**        |     survey results assessed and  |       | CB |
|                     |     used to implement changes to | *     | ** |
|                     |     improve and enhance          | *Met. |    |
|                     |     services.**                  | (De   |    |
|                     |                                  | tails |    |
|                     | **99**                           | p.    |    |
|                     |                                  | 1     |    |
|                     | -   **Internal customer          | 23)** |    |
|                     |     satisfaction rated at        |       |    |
|                     |     minimum of 4.5/5.**          | **    |    |
|                     |                                  | 4.4** |    |
|                     | -   **External customer          |       |    |
|                     |     satisfaction rated at 4/5.** | **    |    |
|                     |                                  | 4.0** |    |
|                     | -   **Customer satisfaction      |       |    |
|                     |     survey results assessed and  | *     |    |
|                     |     used to implement changes to | *Met. |    |
|                     |     improve and enhance          | (De   |    |
|                     |     services.**                  | tails |    |
|                     |                                  | p.    |    |
|                     |                                  | 85)** |    |
+---------------------+----------------------------------+-------+----+
| **Goal 7            | **01 DISCONTINUED**              | *     | ** |
| continued:**        |                                  | *3.2/ | CB |
|                     | **00**                           | 5.0** | ** |
| **00-99 Objective   |                                  |       |    |
| 7.3. Continued      | -   **Customer satisfaction      | *     | ** |
| enhancement and     |     rated at minimum of 3.5/4.** | *Met. | CB |
| expansion of Agency |                                  | (De   | ** |
| Intranet site to    | -   **Demonstration through      | tails |    |
| ensure staff have   |     customer satisfaction        | pp.   | ** |
| immediate access to |     surveys that the daily work  | 123-1 | CB |
| all current         |     of staff has been            | 24)** | ** |
| information.**      |     facilitated by the           |       |    |
|                     |     Intranet.**                  | *     |    |
|                     |                                  | *Met. |    |
|                     | -   **Assessment of customer     | (De   |    |
|                     |     satisfaction surveys and use | tails |    |
|                     |     of such surveys to implement | p.    |    |
|                     |     changes to improve and       | 1     |    |
|                     |     enhance services as          | 24)** |    |
|                     |     necessary.**                 |       |    |
|                     |                                  | **3.  |    |
|                     | **99**                           | 1/4** |    |
|                     |                                  |       |    |
|                     | -   **Customer satisfaction      | **    |    |
|                     |     rated at minimum of 3.5/4.** | Met** |    |
|                     |                                  |       |    |
|                     | -   **Customer satisfaction      |       |    |
|                     |     surveys assessed and used to |       |    |
|                     |     implement changes to improve |       |    |
|                     |     and enhance services as      |       |    |
|                     |     necessary.**                 |       |    |
+---------------------+----------------------------------+-------+----+

+---------------------+----------------------------------+-------+----+
| **Goal 7            | **02**                           | **Co  | ** |
| continued:**        |                                  | mplet | CB |
|                     | -   **Perform initial tests,     | ed.** | ** |
| **01-02 Objective   |     (periodically, beginning in  |       |    |
| 7.4. Establish and  |     2^nd^ quarter of FY 2002) to | **Co  | ** |
| maintain a secure   |     evaluate the preliminary     | mplet | CB |
| Agency computer     |     policies and procedures.**   | ed.** | ** |
| network             |                                  |       |    |
| infrastructure.**   | **01**                           | **Co  |    |
|                     |                                  | mplet |    |
|                     | -   **Preliminary policies and   | ed.** |    |
|                     |     procedures for reducing      |       |    |
|                     |     security risks will be       | **Co  |    |
|                     |     developed by the end of FY   | mplet |    |
|                     |     2001.**                      | ed.** |    |
|                     |                                  |       |    |
|                     | -   **Initial criteria for       | **Co  |    |
|                     |     reporting security incidents | mplet |    |
|                     |     will be established by the   | ed.** |    |
|                     |     end of CY 2001.**            |       |    |
|                     |                                  | **Co  |    |
|                     | -   **Initial procedures for     | mplet |    |
|                     |     responding to security       | ed.** |    |
|                     |     incidents will be            |       |    |
|                     |     established by the end of CY |       |    |
|                     |     2001.**                      |       |    |
|                     |                                  |       |    |
|                     | -   **Implementation of a Secure |       |    |
|                     |     Phase 1 LAN for analysis of  |       |    |
|                     |     intramural research and      |       |    |
|                     |     survey data will be          |       |    |
|                     |     completed by end of FY-01.** |       |    |
|                     |                                  |       |    |
|                     | -   **Implementation of a Phase  |       |    |
|                     |     1 firewall, intrusion        |       |    |
|                     |     detection and virus control  |       |    |
|                     |     system will be in place by   |       |    |
|                     |     end of CY 2001.**            |       |    |
|                     |                                  |       |    |
|                     | -   **Initial security awareness |       |    |
|                     |     training will begin by end   |       |    |
|                     |     of CY 2001.**                |       |    |
+---------------------+----------------------------------+-------+----+

# Brief List of Patient Safety Awards

  ------------------------------------------ -------------- ------------ ------------------
  **Supporting Demonstration Projects to                                 
  Report Medical Errors Data**                                           

  **Institution**                            **City**       **State**    **Award**

  Georgia Hospital Association               Marietta       GA           \$1,636,373

  University of Mississippi Medical Center   Jackson        MS           \$993,212

  University of Colorado Health Center       Denver         CO           \$956,183

  Massachusetts Department of Public Health  Boston         MA           \$1,295,658

  Abt Associates, Inc.                       Cambridge      MA           \$213,765

  Indiana University                         Indianapolis   IN           \$503,005

  Medical College of Wisconsin               Milwaukee      WI           \$670,000

  Center for Health Care Policy and          Minneapolis    MN           \$190,118
  Evaluation/United Health Group                                         

  Marshfield Medical Research Foundation     Marshfield     WI           \$137,564

  Research Triangle Institute                Research       NC           \$200,000
                                             Triangle\                   
                                             Park                        

  Denver Health                              Denver         CO           \$154,097

  Johns Hopkins University                   Baltimore      MD           \$1,519,620

  Harvard University                         Boston         MA           \$1,380,666

  Health Research, Inc./New York State       Albany         NY           \$1,734,580
  Department of Health                                                   

  Utah Department of Health                  Salt Lake City UT           \$1,500,933

  Health Partners Research Foundation        Minneapolis    MN           \$401,802

  University of Minnesota/Sponsored Projects Minneapolis    MN           \$294,759
  Administration                                                         

  American College of Surgeons               Chicago        IL           \$1,633,423

  Columbia University                        New York       NY           \$2,322,914

  Washington University                      St. Louis      MO           \$1,935,551

  University of Pittsburgh                   Pittsburgh     PA           \$1,600,514

  Harvard Pilgrim Health Care                Brookline      MA           \$2,838,449

  Denver Health                              Denver         CO           \$244,760

  Research Triangle Institute                Research       NC           \$299,999
                                             Triangle\                   
                                             Park                        

                                                            **Total:**   **\$24,657,945**
  ------------------------------------------ -------------- ------------ ------------------

  ---------------------------------------------- -------------- ------------ -----------------
  **Using Computers and Information Technology                               
  to Prevent Medical Errors**                                                

  **Institution**                                **City**       **State**    **Award**

  DoctorQuality                                  Philadelphia   PA           \$99,354

  Advance Medical Electronics Corporation        Fridley        MN           \$750,000

  Protolex, LLC                                  Lanham         MD           \$97,578

  Abacus Management                              Cranston       RI           \$99,957

  Montefiore Medical Center                      New York       NY           \$486,805

  Johns Hopkins University                       Baltimore      MD           \$358,139

  Creighton University                           Omaha          NE           \$345,286

  Intellegent Automation, Inc.                   Rockville      MD           \$100,000

  Compliance Laboratories, LLC                   Lenexa         KS           \$99,935

  Aptima, Inc.                                   Woburn         MA           \$100,000

  University of Alabama                          Birmingham     AL           \$255,289

  University of California, Davis                Davis          CA           \$332,244

  University of Chicago                          Chicago        IL           \$275,179

  Columbia University                            New York       NY           \$356,099

  Modus Operandi, Inc.                           Indialantic    FL           \$100,000

  Prediction Sciences                            San Diego      CA           \$99,829

  Simulation Technologies, Inc.                  Huntsville     AL           \$99,969

  Brown University                               Providence     RI           \$349,938

  Computing Solutions, Inc.                      Cedar Rapids   IA           \$99,831

  University of Washington                       Seattle        WA           \$303,069

  Foundation for Long Term Care                  Albany         NY           \$419,328

  Pharmacon International, Inc.                  New York       NY           \$99,985

                                                                **Total:**   **\$5,327,814**
  ---------------------------------------------- -------------- ------------ -----------------

  ---------------------------------------------- ------------ ------------ -----------------
  **Understanding the Impact of Working                                    
  Conditions on Patient Safety**                                           

  **Institution**                                **City**     **State**    **Award**

  Oregon Health & Science University             Portland     OR           \$283,436

  Brigham and Women\'s Hospital/Harvard Medical  Boston       MA           \$499,689
  School                                                                   

  University of California, San Francisco School San          CA           \$394,068
  of Nursing                                     Francisco                 

  University of Michigan                         Ann Arbor    MI           \$31,116

  University of Colorado Health Sciences Center  Denver       CO           \$361,100
  School of Nursing                                                        

  Massachusetts General Hospital                 Boston       MA           \$493,733

  American College of Surgeons                   Chicago      IL           \$461,607

  National Academy of Sciences                   Washington   DC           \$500,000

                                                              **Total:**   **\$3,024,749**
  ---------------------------------------------- ------------ ------------ -----------------

  ---------------------------------------------- -------------- ------------ -----------------
  **Developing Innovative Approaches to                                      
  Improving Patient Safety**                                                 

  **Institution**                                **City**       **State**    **Award**

  Hektoen Institute                              Chicago        IL           \$199,978

  University of Washington                       Seattle        WA           \$199,986

  RAND                                           Santa Monica   CA           \$50,000

  University of Vermont                          Burlington     VT           \$200,000

  University of Florida                          Gainesville    FL           \$191,617

  Johns Hopkins University                       Baltimore      MD           \$125,000

  University of Chicago                          Chicago        IL           \$168,371

  University of Wisconsin                        Madison        WI           \$200,000

  University of California, San Diego            La Jolla       CA           \$190,616

  University of Maryland Medical System          Baltimore      MD           \$200,000

  Emory University                               Atlanta        GA           \$184,361

  Brigham and Women\'s Hospital                  Boston         MA           \$981,773

  University of Pennsylvania                     Philadelphia   PA           \$1,310,903

  Oregon Health & Science University             Portland       OR           \$198,979

  Boston University                              Boston         MA           \$199,999

  New York University                            New York       NY           \$199,919

  Kaiser Foundation Research Institute           Oakland        CA           \$331,946

  University of California, Los Angeles          Los Angeles    CA           \$199,984

  University of South Florida                    Tampa          FL           \$199,354

  University of Michigan                         Ann Arbor      MI           \$200,000

  American Academy of Family Physicians          Leawood        KS           \$196,344

  Vanderbilt University                          Nashville      TN           \$199,992

  University of Texas Health Science Center      Houston        TX           \$1,418,949

                                                                **Total:**   **\$7,998,071**
  ---------------------------------------------- -------------- ------------ -----------------

  --------------------------------------------- -------------- ------------ -----------------
  **Disseminating Research Results**                                        

  **Institution**                               **City**       **State**    **Award**

  University of California, San Francisco       San Francisco  CA           \$747,239

  Health Research and Educational Trust         Chicago        IL           \$352,079

  American College of Surgeons                  Chicago        IL           \$193,659

  National Patient Safety Foundation            Chicago        IL           \$242,479

  American College of Physicians-American       Philadelphia   PA           \$256,537
  Society of Internal Medicine                                              

  Harvard Pilgrim Healthcare                    Brookline      MA           \$240,051

  Stanford University                           Stanford       CA           \$330,566

                                                               **Total:**   **\$2,362,610**
  --------------------------------------------- -------------- ------------ -----------------

  ----------------------------------------------- ------------ ------------ -----------------
  **Additional Patient Safety Research                                      
  Initiatives**                                                             

  **Institution**                                 **City**     **State**    **Award**

  University of California, San Francisco         San          CA           \$438,209
                                                  Francisco                 

  National Academy of Sciences                    Washington   DC           \$1,000,000

  University of California, San Francisco         San          CA           \$560,312
                                                  Francisco                 

  Research Triangle Institute                     Research     NC           \$238,027
                                                  Triangle\                 
                                                  Park                      

  Johns Hopkins University, School of Medicine    Baltimore    MD           \$500,000

  MEDSTAT Group                                   Washington   DC           \$898,285

  Barents Group                                   McLean       VA           \$499,889

  Westat                                          Rockville    MD           \$1,124,496

  Veterans Medical Research Foundation            San Diego    CA           \$96,844

  User Liaison Program                            Rockville    MD           \$1,000,000

                                                               **Total:**   **\$6,356,062**
  ----------------------------------------------- ------------ ------------ -----------------

[^1]:
    > Eisenberg JM. Health Services Research in a Market-Oriented Health
    > Care System. *Health Affairs*, Vol. 17, No. 1:98-108, 1998.