Clinical Practice

Guidelines

Primer

Management Decision and Research Center

Health Services Research and Development Service

Office of Research and Development

Department of Veterans Affairs

In collaboration with

Association for Health Services Research
Clinical Practice Guidelines.

Boston, MA: Management Decision and Research Center; Washington, DC: VA Health Services 

Research and Development Service in collaboration with Association for Health Services 

Research, 1998.

p. cm.

Includes bibliographical references. Primer.

DNLM: 1. Hospitals, Veterans -organization and administration. 2. Practice Guidelines. 

3. United States, Department of Veterans Affairs, Management Decision and Research 

Center. 4. Association for Health Services Research.

W84.AA1 C641 1998

The Health Services Research and Development Service (HSR&D) is a program within the Veterans Health Administration’s Office of Research and

Development. HSR&D provides expertise in health services research, a

field that examines the effects of organization, financing and management

on a wide range of problems in health care delivery – quality of care,

access, cost and patient outcomes. Its programs span the continuum of

health care research and delivery, from basic research to the dissemination

of research results, and ultimately to the application of these findings to

clinical, managerial and policy decisions.
Clinical Practice 

Guidelines

Purpose of primer series: to help bridge the gap among

health services researchers, policy makers, managers and clinicians

in an effort to improve the quality and cost effectiveness of health

care for veterans. The primer series is part of a larger set of dissemination initiatives developed by VHA’s Office of Research and Development through its Management Decision and Research Center and

in collaboration with the Association for Health Services Research.

Purpose of Clinical Practice Guidelines: to provide

an introduction to clinical practice guidelines as a tool to support

clinical decision making and quality health care delivery in the Veterans Health Administration. The primer provides a basic framework for understanding clinical practice guidelines and discusses a

number of issues regarding implementing guidelines. More in depth

readings and other resources are listed in the appendices.

Suggested audience: professionals involved in health care

delivery and decision making, including managers working in clinical

care, quality management, administration and strategic planning at

VA Headquarters, Veterans Integrated Service Networks and within

VA facilities.

Suggested uses: individual study, orientation for health care

providers, management training programs in Veterans Integrated

Service Networks and medical centers, as a resource for guideline

implementation planning, continuing medical education courses and

other medical and health professional training programs.

November 1998

Boston, Massachusetts 
Table of Contents

Contributors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .i

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

What are clinical practice guidelines? Why do we need 

evidence-based clinical practice guidelines, and how do they 

contrast with consensus-based guidelines?. . . . . . . . . . . . . . . . . . . . . . 3

Are clinical practice guidelines the same as algorithms? 

As clinical pathways?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Why are guidelines important to VA managers and clinicians?. . . . . .4

How do guidelines relate to quality improvement?. . . . . . . . . . . . . . . . 4

What are the limitations of clinical practice guidelines?. . . . . . . . . . . .5

What do we know about issues related to guideline 

implementation?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

What are the different types of implementation strategies?. . . . . . . . . .6

What does research tell us about successful implementation 

of guidelines?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

What do managers need to know about implementation 

of guidelines?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

What are the implications of guideline implementation for 

health care professionals and patients?. . . . . . . . . . . . . . . . . . . . . . . . 10

How is VA implementing clinical practice guidelines?. . . . . . . . . . . . 11

What do we still need to learn about successful implementation 

of clinical practice guidelines?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Concluding Remarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Appendix A: Where can VA managers turn for additional information

about successful guideline implementation strategies?. . . . . . . . . . . .15

Appendix B: What reading materials are available to provide more 

in-depth information on the implementation of clinical practice 

guidelines?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

Appendix C: What research is VA conducting on guideline 

implementation strategies?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Appendix D: Ongoing research by the Cochrane Collaboration. . . . . 27

Appendix E: How is VA Measuring Guideline Implementation?. . . . 29
Contributors

The MDRC and AHSR staff thank the many managers and

researchers within and outside of VA who participated in the development of this primer. Mary Darby, editorial consultant, wrote the

primer with input from a number of researchers and others with

expertise in clinical practice guidelines. Special thanks go to the following individuals for their assistance: Margaret Baumann, M.D., VA

Medical Center, Hines, IL; Dan Berlowitz, M.D., M.P.H., VA HSR&D

Center of Excellence, Bedford, MA; Jennifer Daley, M.D., M.P.H., VA

Medical Center, West Roxbury, MA; Susan Edgerton, House Committee on Veterans Affairs, Washington, D.C.; Ronald Gebhart, M.D.,

Office of Patient Care Services, VA Headquarters, Washington, DC;

Mary Goldstein, M.D., VA HSR&D Center of Excellence, Palo Alto, CA;

Susan Hedrick, Ph.D., VA HSR&D Center of Excellence, Seattle, WA;

Christine LaGana, Ph.D., VA Maryland Health Care System, Baltimore, MD; Richard McCormick, Ph.D., VA Medical Center, Cleveland,

OH; Brian Mittman, Ph.D., VA HSR&D Center of Excellence, Sepulveda, CA; Gordon Mosser, M.D., Institute for Clinical System Integration, Bloomington, MN; Cynthia Mulrow, M.D., M.Sc., VA HSR&D

Center of Excellence, San Antonio, TX; Richard Owen, M.D., VA

HSR&D Center of Excellence, North Little Rock, AR; Robert (Randy)

Petzel, M.D., Upper Midwest Network, Minneapolis, MN; Jacqueline

Pugh, M.D., VA HSR&D Center of Excellence, San Antonio, TX;

Donna Schoonover, R.N., M.S.N., Employee Education System, St.

Louis, MO; Scott Sherman, M.D., VA HSR&D Center of Excellence,

Sepulveda, CA; William Tierney, M.D., VA Medical Center, Indianapolis, IN; James Tuchschmidt, M.D., VA Medical Center, Portland, OR;

Debby Walder, R.N., M.S.N., Office of Performance and Quality, VA

Headquarters, Washington, DC; Scott Weingarten, M.D., Cedars-Sinai

Medical Center, Beverly Hills, CA.

Acknowledgments:

We also extend our appreciation to colleagues throughout VA for their

thoughful comments in reviewing an earlier draft of the primer: 

Martin Charns, D.B.A., VA HSR&D Management Decision and

Research Center, Boston, MA; John G. Demakis, M.D., VA Health Services Research and Development Service, VA Headquarters, Washington, DC; Sheila Gelman, M.D., VA Healthcare System of Ohio,

Cincinnati, OH; Alan Goss, VA Medical Center, Alexandria, LA; Terry

Gottlieb, R.N., VA Medical Center, Bronx, NY; Jeffrey Kuch, M.D., VA

Medical Center, Bay Pines, FL; David Nardone, M.D., VA Medical Center, Portland, OR; Ellen Kingston, R.N., VA Medical Center, Omaha,

NE; Claire Maklan, Ph.D., VA Health Services Research and Development Service, VA Headquarters, Washington, DC; Carter Meecher,

M.D., Atlanta Network, Atlanta, GA; Shirley Meehan, Ph.D., M.B.A.,

VA Health Services Research and Development Service, VA Headquarters, Washington, DC; Karen Robinson, Ph.D., R.N., VA Medical Center, Fargo, ND; Michael Stafford, Association for Health Services

Research, Washington, DC; Gary Young, Ph.D., J.D., VA HSR&D 

Management Decision and Research Center, Boston, MA.

Primer development and editing:

Laurel Long, M.S., VA HSR&D Management Decision and Research

Center, Boston, MA; Geraldine McGlynn, M.Ed., VA HSR&D Management Decision and Research Center, Boston, MA; Diane Alexander

Meyer, M.A., Association for Health Services Research, Washington, DC.

i
Introduction

1

During the 1970s and 1980s, health services researchers uncovered evidence of substantial practice variations, suggesting that medicine was often more art than science. Those findings, combined

with rapidly rising health care costs, supplied the impetus for developing practice guidelines. By synthesizing the best available

research evidence on various conditions and procedures, good practice guidelines supply health care professionals with needed support.

Used properly, guidelines can improve health care quality and

encourage more efficient use of limited health care resources.

Over time, practice guidelines have gained wide acceptance. Hospitals, health plans, medical specialty groups, the federal government,

and others have used various approaches to develop thousands of

guidelines. But, as health care organizations throughout the country

have learned, developing guidelines is relatively easy compared to

implementing them. Numerous questions remain about how best to

implement practice guidelines and whether different guidelines,

treatment settings, and patient populations require different implementation strategies. Unless guidelines are implemented effectively,

they may not live up to their potential. 

VA is committed to implementing nationally developed, evidencebased practice guidelines that can improve health care outcomes and

efficiency for our patient population. Ultimately, VA hopes to see

increasingly more of the care it provides supported by guidelines, and

is funding research to improve implementation at the national, network, and facility levels. In addition, guidelines form a core component of VA’s evolving performance measurement system for Veterans

Integrated Service Networks (VISNs). Thus, evidence-based practice

guidelines figure prominently in VA’s goals to promote excellence

and accountability in health care. “Practice guidelines are clinical

decision-making tools that can help VA improve the quality of care it

provides,” notes VA Under Secretary for Health Kenneth W. Kizer,

M.D., M.P.H. “Although they are only one element in VHA’s overall

quality management strategy, they are an especially important element.”1

This primer was developed to help a broad audience, including

clinicians, health care managers, and policy makers – both within

and outside VA – use guidelines more effectively by answering some

basic questions about the challenges of guideline implementation and

the lessons gleaned from current research. A question-and-answer

format is used for easier reading and accessibility. Because of VA’s

considerable work in developing, evaluating, and implementing

guidelines, several VA research products are highlighted. Resources

for additional information, including appendices of resources in and

outside VA and references for more in-depth reading, are provided as

well. 

As health care organizations throughout the

country have learned,

developing guidelines is

relatively easy compared

to implementing them.
What are clinical practice guidelines? Why do we need

evidence-based clinical practice guidelines, and how do

they contrast with consensus-based guidelines?

Evidence-based guidelines represent a systematic rational approach

to medicine because they

are more rigorously

grounded in science.

VAdefines clinical practice guidelines as “recommendations for

the performance or exclusion of specific procedures or services

derived through a rigorous methodological approach that includes

the following:

■ determination of appropriate criteria, such as effectiveness, 

efficacy, population benefit, or patient satisfaction; and 

■ literature review to determine the strength of the evidence 

(based in part on study design) in relation to these criteria.”2

VA has been active in the development and implementation of evidence-based guidelines, which make explicit links between practice

recommendations and the quality of supporting evidence. Guidelines help optimize management of a given condition, problem, or

patient population by identifying best care practices. Evidencebased guidelines represent a systematic, rational approach to medicine because they are more rigorously grounded in science – not

expert opinion or anecdotal evidence. 

Most national guidelines are developed through a consensus

process that involves a group of experts who are convened for the

specific purpose of constructing a guideline on a given condition or

procedure. Such a group may formally consider scientific evidence

or it may rely on position statements or the expertise of its members. 

So, although many evidence-based guidelines are developed

through a consensus process, many consensus guidelines are not evidence-based. The difference is that evidence-based guidelines are

produced through systematic reviews that assess the quality of relevant clinical trials and studies and weight them accordingly (e.g.,

randomized versus nonrandomized, controlled versus uncontrolled,

blinded versus unblinded, etc).3

To ensure scientific rigor in its guideline activities, VA is developing a weighting system that will be used to evaluate the quality of

evidence behind VA practice guidelines.

A

Are clinical practice guidelines the same as algorithms?

As clinical pathways?

n algorithm is a set of rules for solving a problem or accomplishing a task. Clinical algorithms provide guidance for a specific patient

care problem. Typically, a clinical algorithm diagrams a practice

guideline into a step-by-step flow chart, which provides a visual display of the algorithm’s branching logic.4

A clinical pathway organizes, specifies, and sequences the major

patient care activities and interventions of an entire interdisciplinary team for a particular diagnosis or procedure. It is a management plan that defines key processes and events in the day-to-day

delivery of care. The difference between guidelines and pathways is

that pathways focus on the quality and efficiency of care after deci3
sions have been made about what procedures or services to perform.

Thus, a guideline may be viewed as a decision support tool, whereas

a pathway may be viewed as a quality assurance tool.

Under VA policy, VA facilities use nationally developed, evidencebased practice guidelines. However, clinical pathways should be

developed or customized locally. 

Why are guidelines important to VA managers 

and clinicians?

VA Under Secretary for Health Kenneth W. Kizer, M.D., M.P.H.,

notes that, “no matter how [guidelines] are used, the goal is the

same: To improve patient care.”5 Better patient outcomes often go

hand in hand with improvements in efficiency and cost-effectiveness,

“but achieving savings is not the primary motivation for implementing practice guidelines,” Dr. Kizer states.

At VA, clinical quality is “critically dependent on organizational

systems and structures that minimize the chances for mistakes to

occur, improve efficiency, promote accountability, and encourage continuous improvement,” Dr. Kizer writes in Prescription for Change.

6

Practice guidelines can help VA put those systems and structures

into place by identifying and implementing best practices that ultimately will benefit VA patients. 

In addition, guideline implementation has been incorporated into

performance agreements between network directors and the Under

Secretary for Health. Guidelines or specific elements of guidelines

supported by strong or conclusive evidence have been used to develop national performance measures for conditions like ischemic heart

disease, major depressive disorder, hypertension, smoking cessation,

and diabetes. The list of mandated guidelines and performance measures will be updated annually. (For more information on mandated

guidelines and performance measures, see Appendix E.)

How do guidelines relate to quality improvement?

Better patient care means better patient outcomes. This is what

clinical quality is all about. At VA, quality improvement focuses on

“prospectively building into the delivery system those processes and

techniques that optimize access, coordination of services, use of best

practices, and patient participation in clinical decision-making,” Dr.

Kizer writes.7 Guidelines are one of several tactics for implementing

this strategy. 

As noted, VA is incorporating guidelines into quality improvement

and performance measurement activities at the national and network levels. A bold new program of VA’s Health Services Research

and Development Service, the Quality Enhancement Research Initiative (QUERI), will create and implement a national system to

translate the latest information on evidence-based medicine and best

practices into patient care in several ways, including implementation

of guidelines. VA is already using practice guidelines to improve care

4

“No matter how [guidelines] are used, the goal

is the same: To improve

patient care.”

Better patient care

means better patient

outcomes. This is what

clinical quality is all

about.
at the local level. For example, the use of practice guidelines on

pressure ulcer care has led to a 25 percent reduction in pressure

ulcer development among long-term care patients studied.8 In

another local VA project, locally adapted national practice guidelines

for the treatment of chronic obstructive pulmonary disease, asthma,

and sexually transmitted diseases had positive impacts on local practice.9 The study also provided a valuable model for changing health

care practice that is being applied in other initiatives. (See Appendix

C for more details.)

What are the limitations of clinical practice guidelines?

There are several limitations to practice guidelines. First, guidelines are only as good as the evidence on which they are based.

Unfortunately, science does not always provide clear answers to

questions surrounding clinical care. In addition, studies used to support guideline recommendations may be poorly designed or of limited applicability. These factors explain why it is so important to take

an evidence-based approach to guideline development. It is wise to

keep in mind that guidelines are dynamic and evolving; they must

change to keep pace with new scientific knowledge and technologies. 

Second, guidelines should be viewed as a tool for health care decision-making – not a dictate. It is not reasonable to expect 100 percent compliance with any guideline, since there may be perfectly

valid reasons for not complying with a guideline in a given clinical

situation. These reasons could include patient comorbidities, age, or

other complicating factors. Quality, as reflected in patient outcomes

– not rigid compliance – should be the basis for professional

accountability. In addition, there must always be room for clinical

judgment and patient preference in medical decision-making. Thus,

health care professionals should view guidelines as a resource – not

as pronouncements from on high.

Third, guidelines are useless unless they are implemented properly, so that they make a positive impact on clinical practice patterns

and patient outcomes.

R

What do we know about issues related to guideline 

implementation?

esearch on guideline implementation has tended to focus on

issues related to changing health care practice. These issues include

lack of awareness, lack of agreement or “buy-in” regarding a guideline, inability to implement a guideline, and inability to remember it.

Recently, attention has started to shift to system barriers to guideline implementation, such as inadequate staffing, lack of technology,

facility design, and conflicting financial incentives. However, evidence on the impact of these barriers – and strategies for overcoming

them — is not in yet.

5

Guidelines are dynamic

and evolving; they must

change to keep pace

with new scientific

knowledge and technologies.
Many types of health care professionals may be involved in guideline implementation, including physicians, nurses, medical directors,

case managers, quality directors, patient educators and others.

Implementation issues related to changing professional practice may

be classified into four types:

■ Knowledge: The provider must be aware of and understand the

guideline.

■ Attitude: The provider must agree with the guideline’s recommendations.

■ Behavior: The provider must be able – and encouraged – to use

the guideline. Relevant factors include time and resources, financial

incentives, appropriate skill mix, guideline complexity, patient acceptance, etc. Some of these factors reflect system barriers.

■ Maintenance: Without reinforcement, practice changes tend to

decay over time.10 Again, system issues often play a role in this area.

System characteristics can have a significant impact on guideline

implementation. For example, VA facilities may benefit from being

part of a large organization that has its own culture and a clearly

stated mission. However, it is often difficult to implement guidelines

within large, complex organizations like VA. This is especially true

during times of great change when organizational goals and strategies are being realigned. Other challenges stem from the complexity

of VA patients, who tend to be older and sicker, with multiple diagnoses.11 VA is working to install state-of-the-art computer technology that will support guideline implementation.

What are the different types of implementation strategies?

No single implementation strategy works across the board. Targeted dissemination is the critical first step. However, one thing is

clear: Dissemination alone makes little impact on clinical practice.

Most current implementation strategies are geared toward changing

the practice of health care professionals. A multi-faceted approach

that combines several methods – such as education, opinion leaders

and clinical audit and feedback – in a package, appears to be most

effective in this regard. Types of strategies for changing professional

practice include:

■ Knowledge-based: systematic reviews, evidence summaries, continuing medical education, dissemination

■ Attitude-based: local adaptation of guidelines, opinion leaders,

academic detailing, specialty society endorsements, peer review

■ Behavior-based: time and scheduling, skill-building, equipment

and personnel, total quality management interventions, decision

support tools, patient compliance

■ Maintenance-based: computerized reminder systems, standing

orders, audits, feedback, follow-up programs12

6

A generic approach to

guideline implementation won’t change clinical practice.
Experience shows that a generic approach to guideline implementation won’t change clinical practice. Each case is unique and

depends on the type of guideline and the type of facility or setting

where it is being implemented. While there may be cross-cutting

issues, VA managers and clinicians need to identify unique challenges to implementing each guideline that they undertake and

develop a strategy or a set of strategies for addressing those problems. 

R

What does research tell us about successful 

implementation of guidelines?

esearch on guideline implementation is sparse, especially when it

comes to identifying which strategies work in specific practice settings, such as ambulatory care centers or nursing homes. In addition, studies are often small, use inadequate analysis techniques, or

fail to evaluate sustainability of effects. VA’s newly established Center of Excellence, called the Veterans Evidence-Based Research, Dissemination and Implementation Center (VERDICT), compiled the

synopsis of findings on guideline implementation strategies presented in the table on the next page. The most effective strategies

appear at the top of the table. Multi-faceted approaches appear to

hold the most promise. “Simple Dissemination” and “Traditional

Continuing Education” appear less effective. It should be noted that

these strategies were specifically designed to change professional

practice; they do not explicitly address system barriers to guideline

implementation. Hopefully, further research will provide evidence

on how those types of barriers may be overcome. (For more information on VERDICT, see Appendix A.) The Cochrane Collaboration, an

international organization that produces and disseminates systematic reviews on the effects of health care interventions, is doing some

work in this area. (See Appendix D for more information.)

7
8

What the Research Says About Guideline Implementation 

Strategies for Health Care Professionals

Multi-Faceted Approaches While results were mixed, 31 of 39 studies of multi-faceted interventions

Combining strategies to change (e.g., using three or more educational strategies) were found to have positive

provider behavior. Especially using effects. In addition to appearing more effective, multi-faceted approaches

more than one strategy proven to demonstrated positive changes in health care outcomes. While it wasn’t

impact behavior at least modestly. clear which combinations of approaches worked best, and in which settings,

multi-faceted approaches appear, at present, to hold the most promise.

Davis, Journal of the American Medical Association, 274 (9): 700-05.

Local Opinion Leaders Five studies showed positive effects on some process-of-care outcomes. Only

Use of providers who are highly three described patient outcomes; one of these was positive.

respected by their colleagues to pro- Thomson. Cochrane Library. September 1997.

mote guidelines and educate their

colleagues on them.

Outreach Visits Outreach worked, particularly in changing drug prescription practices, but

A trained person meets with health evidence was insufficient to identify the key characteristics of successful outcare professionals in their practice reach. Social marketing techniques that target and identify barriers to

settings to provide information (also change hold great potential.

known as detailing). Davis, Journal of the American Medical Association, 274 (9): 700-05.

Thomson. Cochrane Library. September 1997.

Audit and Feedback Provider prescribing and test ordering behavior improved modestly. It was

Retrospective summaries of clinical not clear whether complicated audit methods improved general medical

performance of health care that may management.

include recommendations for clinical Thomson (Part 1). Cochrane Library. March 1998.

action.

Computerized Tools Reminders aimed at triggering periodic preventive tests such as mammoPrompts/reminders, computerized grams, sigmoidoscopies, and vaccinations changed both professional and

algorithms, provider feedback, com- patient behavior. Computerized drug algorithms were effective in helping

puterized medical record access, com- physicians improve drug-prescribing patterns.

puter-assisted diagnosis. Balas, Archives of Family Medicine, 1996, 5 (5): 271-8.

Shea, Journal of the American Medical Informatics Association, 1996, 3: 399-409.

Training Training had a measurable impact on professional performance (e.g., offerInteractive conferences that involve ing counseling, setting smoking cessation dates). Training in history-taking

health care professionals in discus- and verbal skills reduced patients’ emotional distress.

sion, role-playing, or skill-building. Silagy, Quality in Health Care, 1994, 3: 193-8.

Stewart, Canadian Medical Association Journal, 1995, 152 (9): 1432-33.

Smith, Journal of General Internal Medicine, 1994, 9: 390-6.

Traditional Continuing No positive effects were seen on professional practice outcomes. Small

Education effects that surfaced after followup ended may have been missed.

Participation of health care profes- Davis, Journal of the American Medical Association, 274 (9): 700-05

sionals in lecture-style conferences of

a day or less.

Simple Dissemination No statistically significant effects were shown in changing either provider

Distribution via journals or other behavior or patient outcomes. Combination strategies increased the success

publications, hand delivery, mail, or of print materials.

electronic media.

The information in this table was compiled by the Veterans Evidence-Based Research Dissemination and Implementation Center,

an HSR&D Center of Excellence based in San Antonio and Charleston. Verdict Brief, Biannual Newsletter, Spring 1998, pp. 2-3.

Strategy/Description Results
What do managers need to know about implementation

of guidelines?

Following is a compilation of advice from investigators at the

Cochrane Collaboration and other experts.13 (For more information

on the Cochrane Collaboration, see Appendices A and D.) Again,

most of these recommendations are geared toward changing provider

practice – not changing or removing system barriers to guideline

implementation, for which there is scant evidence at this point. The

most important advice for managers is to remember that guideline

implementation is hard work. It takes time and commitment. It

can be frustrating – but when it succeeds, it can also be very rewarding.

■ Focus on what’s important. Choose your guidelines carefully

and make sure that they address areas of clinical importance to

health care professionals. For example, VA is focusing its guideline

activities on those conditions that most commonly affect veterans,

such as heart disease, hypertension, diabetes and depression.

■ Show them the evidence. To increase acceptance, establish the

credibility of your guidelines. For example, at Cedars-Sinai Health

System in Los Angeles, guidelines are reviewed and rated according

to the scientific evidence on which they are based. Randomized clinical trials get the highest grades and studies that involve samples of

more than 200 patients are valued more highly than those with

patient samples of less than 200. 

■ Consider what type of behavior you are trying to change.

To make changes in simple, periodic behavior, such as test ordering

or immunization, computerized reminders work well. But to change

behavior in a more complex process, such as disease management,

reminders alone won’t work. Try breaking the process into components and develop strategies that work well for each component. For

example, failure to provide smoking cessation counseling often stems

from physicians’ perceptions that they lack necessary skills. In that

case, skill-building or increasing staff support may be appropriate

implementation strategies. 

■ Be willing to change the system. Implementation strategies

need to be supported by appropriate system changes and enhancements if they are to succeed. Among the factors to be considered:

nursing support, lab turnaround, computerized information systems,

equipment maintenance, and organizational structures.

■ Try different approaches. Three strategies showed good

results for changing prescription patterns: reminders, outreach visits, and audit and feedback.

■ Prevent mistakes. Use decision aids, such as computer algorithms or pre-printed prescription forms for medication selection

and dosing situations to prevent common errors.

■ Support decisions at the point of care. Feedback and computerized reminders are particularly effective when they are readily

available to physicians at the point of care.

■ Choose strategies carefully. Don’t rely solely on relatively

unproven expensive strategies to optimize practice.

9

Guideline implementation takes time and commitment. It can be

frustrating – but when it

succeeds, it can also be

very rewarding.
■ Update continuing education. Incorporate approaches that

show positive impacts in changing behavior (e.g., outreach visits and

intensive workshops) into continuing professional development programs.

■ Keep it simple. Break the guideline down to its simplest and

most important components; key in on decision points that affect

patient outcomes.

■ Narrow your target audience. Try to involve only those people necessary to put a guideline into practice (e.g., pharmacists

might be a good audience for hypertension management guidelines).

■ Keep your guidelines up to date. This helps maintain credibility. VA plans to routinely review guidelines to maintain their scientific base.

H

What are the implications of guideline implementation

for health care professionals and patients?

ealth care professionals and patients need to know that guidelines are a tool for enhancing medical decision-making and achieving

optimal outcomes. They are not rigid dictates from on high; nor are

they intended to usurp professional judgment or patient preference.

As the Institute of Medicine notes, guidelines are developed to

“assist practitioner and patient decisions about appropriate health

care for specific clinical circumstances.”14 To that end, guidelines

should inform discussions between health care professionals and

their patients and even increase patient participation in decisionmaking. It must also be clear to health care professionals and

patients alike that the primary objective to implementing guidelines

is improving patient outcomes – not controlling or reducing health

care costs.

Health care professionals are most interested in knowing the evidence on which a guideline is based. They want to be assured that

using a guideline will have a positive impact on their patients’ outcomes. In other words, they need to know that using a particular

guideline is the right thing to do for their patients. Beyond that,

v- they need to know that the system in which they are working is set

up to support the guideline. There must be adequate time, equipment, skilled personnel, and coordination across departments to

make the guideline work properly.

Patients also want to know that health care professionals are

doing the right thing for them. They want to be assured that their

providers are being guided by the weight of scientific evidence and

their own professional judgment – not pressured to reduce costs. To

that end, educational materials that explain patients’ conditions and

the options available to them, along with their relative benefits and

risks, should be distributed to patients, so that they can discuss

these issues in an informed way with their providers. This type of

informed, shared decision-making between health care professionals

and patients promotes more patient-centered care. In other words,

it reflects patients’ preferences and values. In addition, it can lead

to better outcomes and greater patient satisfaction.

It must be clear to

providers and patients

alike that the primary

objective to implementing guidelines is impro

ing patient outcomes.

10
How is VA implementing clinical practice guidelines?

Under VA policy, VA facilities are implementing nationally developed, evidence-based guidelines. The National Clinical Guideline

Advisory Council for the Adaptation, Development, Implementation

and Evaluation of Clinical Practice Guidelines provides the overarching framework for these activities. VA’s philosophy for implementing practice guidelines is a flexible one allowing for discovery of new

knowledge about guideline implementation strategies. A subcommittee on education and implementation is considering issues like

the role of information technology in guideline implementation, the

effectiveness of traditional education models, and the link between

process and structure in changing clinician behavior.

Different VISNs are taking different approaches to guideline

implementation. For example, in VISN 12, a task force for implementing clinical guidelines identified four basic steps for successful

implementation:

■ Develop an efficient and effective communication process.

■ Determine operational requirements.

■ Design and recommend a framework for guideline monitoring.

■ Identify barriers to implementation and recommend actions to

remove them.

After identifying various implementation activities at each VISN

facility and achieving consensus on definitions of guidelines, algorithms, and pathways, the task force focused on common barriers to

implementation and recommended actions to remove them. At each

medical center in VISN 12, a local steering committee provides direct

oversight of guideline implementation, monitoring the various program guideline teams and providing feedback to management and

front-line providers.

In VISN 5, the VA Maryland Health Care System is moving ahead

with a model for implementing behavioral health care guidelines

based on three levels of implementation: the network level, the facility level, and the treatment team level.15 At each level, clinical managers and guideline implementation committees work together on

such details as data collection and analysis, goal-setting, program

evaluation, and professional education. 

The treatment team level is the most important level of implementation – it’s where the rubber meets the road. At this level, education must be combined with implementation strategies that

recognize the realities of everyday practice. The treatment environment must be evaluated to determine what components of the system need to be adjusted, replaced, enhanced, re-engineered, or

removed in order to make the guideline work. For example, it may

be necessary to train staff in new skills, establish new lines of communication within the treatment team, develop new standing order

forms or referral protocols, or purchase equipment. 

“It’s been hard work,” says Christine M. LaGana, Ph.D., deputy

director of the Maryland Health Care System’s Mental Health Clinical Center. But the system is already seeing improvement: In just

one month, documentation of depression screens in primary settings

jumped from 0 to 60 percent, according to Dr. LaGana.

The treatment team

level is the most important level of implementation – it’s where the

rubber meets the road.

11
What do we still need to learn about successful 

implementation of clinical practice guidelines?

Much remains to be learned about guideline implementation.

Very little is known about what really works. Specifically, we need to

better understand how to target implementation strategies to different care settings, types of behavior and barriers. The effectiveness of

continuous quality improvement programs that use practice guidelines also needs to be studied in greater depth. In addition,

researchers are just beginning to scratch the surface of another critical area: system barriers to implementation and how they can be

overcome.

VA is committed to supporting research that will advance guideline implementation within VA. Four Health Services Research &

Development Service (HSR&D) Centers of Excellence focus research

on guideline implementation (see Appendix A for a listing of these

Centers). In addition to the broad-based Quality Enhancement

Research Initiative, HSR&D also has a solicitation inviting

researchers to evaluate strategies for implementing evidence-based

practice guidelines within VA and to identify approaches that may be

replicated systemwide. 

For example, the Bedford VAMC is evaluating different strategies

for implementing pressure ulcer care guidelines in long-term care

settings. The results are being used to develop quality improvement

interventions. Another study underway at the Cleveland VAMC is

investigating the nuances of implementing guidelines for patients

with dual diagnoses — in this case, patients who have both serious

psychosis and substance abuse addiction. And, in a groundbreaking

study coordinated by the Hines Cooperative Studies Program Coordinating Center involving 12 VA hospitals, researchers are investigating the effectiveness of computerized reminders in increasing

resident physicians’ compliance with ambulatory care guidelines.

The results, which are undergoing final analysis, could have national

implications. A more detailed description of some of these funded

projects may be found in Appendix C.

In addition, the Cochrane Collaboration, in which the VA participates through the San Antonio VA Cochrane Center at the Audie L.

Murphy Memorial Veterans Hospital, is conducting more reviews of

research on guideline implementation, including issues related to

system barriers. (See Appendix D for a description of the Cochrane

Collaboration’s research projects.)

We need to better understand how to target

implementation 

strategies. . . 

12
Concluding remarks

13

Evidence-based practice guidelines promote rational, informed

health care decision-making that reflects the best available scientific

knowledge and insights. VA recognizes the value of guidelines in

improving health care quality, increasing efficiency, measuring

provider performance, and encouraging more productive dialogue

between health care professionals and patients. Accordingly, VA has

embarked on a policy of adopting nationally developed, evidencebased practice guidelines to help manage the health care services

provided to VA patients.

Developing and adopting good guidelines is not enough. In order

for guidelines to make a positive impact on patient care, they must

be implemented. A guideline may be painstakingly researched and

rigorously crafted, but if is not incorporated effectively into everyday

clinical practice, where it can make a difference in patient outcomes,

all the work lavished on development will have been for nothing.

Because guidelines are such valuable tools for improving health care

quality and efficiency, VA will continue to investigate and identify

effective strategies for putting guidelines into action and using them

to their maximum advantage to improve patient health outcomes.

This is the ultimate test of clinical guidelines.

In order for guidelines

to make positive impact

on patient care, they

must be implemented.
References

1 Kizer KW. “Clinical Practice Guidelines.” Federal Practitioner, July

1998, page 58.

2 VHA Directive 96-053, August 29, 1996, pages 1-2.

3 MRDC Technology Assessment Program. “Stereotactic Radiosurgery for Metastases to the Brain,” December 1997, page 4.

4 Workgroup to Develop Methods for Deriving Medical Review Criteria. Standards of Quality and Performance Measures from Guidelines. Using Clinical Practice Guidelines to Evaluate Quality of Care.

Vol. 2. Agency for Health Care Policy and Research, March 1995,

pages 109-110. 

5 Kizer KW. “Clinical Practice Guidelines.” Federal Practitioner,

July 1998, page 53.

6 Kizer KW. Prescription for Change. page 8. Washington DC: Veterans Health Administration: 1996.

7 Kizer KW. “Clinical Practice Guidelines.” Federal Practitioner, page 52.

8 Berlowitz DR, Halpern J. Evaluating and improving pressure ulcer

care: the VA experience using administrative data. Joint Commission Journal on Quality Improvement, 23: 424-433, 1997.

9 Shekelle, PG. HSR&D Funded Study IIR 91-035. “Do Medical

Guidelines Improve the Practice of Medicine.” West Los Angeles

VAMC, Los Angeles, CA. 

10 Woolf, SR. Clinical Practice Guidelines and Evidence-based Medicine: Implementation Theory and Practice. Presented at the Annual

Meeting of the Association for Health Services Research; Washington

DC. June 23, 1998.

11 Kazis LE, Miller D, Clark J, Skinner K, et al. Health related quality of life in patients served by the Department of Veterans Affairs:

results from the Veterans Health Study. Archives of Internal Medicine. 158: 626-632, March 1998.

12 Woolf SR. Clinical Practice Guidelines and Evidence-based Medicine: Implementation Theory and Practice. Presented at the Annual

Meeting of the Association for Health Services Research; Washington

DC, June 23, 1998.

13 Veterans Evidence-based Research, Dissemination and Implementation Center (VERDICT), San Antonio and Charleston. “Strategies

To Keep Providers Informed, Keep Practice Current and Appropriate.” Verdict in Brief, 1998: Spring, pages 2-4. 

14 Field MJ, and Lohr KN, Eds. Institute of Medicine Committee on

Clinical Practice Guidelines. Guidelines for Clinical Practice: From

Development to Use. Washington, DC: National Academy Press; 1992.

15 LaGana CM. Implementation of Behavioral Guidelines. Undated

white paper. Washington, D.C., Baltimore, MD. VA Headquarters,

VA Maryland Health Care System. 

14
Appendix A: Where can VA managers turn for 

additional information about successful guideline

implementation strategies?

15

Listed below are some organizations within and outside VA that

may provide useful information about guideline implementation.

This list is not exhaustive.

Within VA

■ Office of Research and Development

John R. Feussner, M.D., Chief Research and Development Officer 

VA Headquarters

Phone and FTS: 202/273-8284

Fax: 202/273-6526

The Office of Research and Development oversees the full range of

health services research, medical research, multi-site cooperative

studies, and rehabilitation research in VA. A number of VA research

efforts focus on identifying and evaluating successful strategies for

implementing evidence-based clinical practice guidelines. Some of

those efforts are described in the following section, in the text of the

primer, and in Appendix C.

Health Services Research and Development Service (HSR&D)

John G. Demakis, M.D., Director

VA Headquarters

Phone and FTS: 202/273-8287

Fax: 202/273-9007

Within the Office of Research and Development, the Health Services

Research and Development Service provides expertise in health services research, a field that examines the effects of health care organization, financing and management on a wide range of delivery issues

including quality of care, access, cost and patient outcomes. In addition to the HSR&D research activities relating to implementation

strategies described elsewhere in the document (see the discussion of

the Quality Enhancement Research Initiative on page 4 and the studies of implementation strategies in Appendix C), four of HSR&D’s

eleven Centers of Excellence focus on guideline implementation. 

Center for the Study of Healthcare Provider Behavior

Lisa V. Rubenstein, M.D., M.S.P.H.

Sepulveda Campus/Southern California System of Clinics, 

Sepulveda, California 

Telephone: 818/895-9449 or 700/996-9449

Fax: 818/895-5838

E-mail: lisar@rand.org

Researchers specifically focus on provider behavior and practice

patterns, health care quality and outcomes, quality improvement,

clinical practice guideline implementation, and primary care/managed care evaluation. Providing technical assistance as well as

training and education opportunities are also high priorities.
16

Veterans Evidence-Based Research, Dissemination and Implementation Center (VERDICT)

Jacqueline Pugh, M.D.

VA South Texas Health Care System, San Antonio, Texas

Telephone: 210/617-5314

Fax: 210/567-4423

E-mail: PUGH.JACQUELINE_A+@San-Antonio.va.gov

Research efforts of this Center aim to link research evidence with

clinical practice by summarizing and translating the evidence into

a useful document for various consumers, such as providers,

patients, managers and policymakers. 

Center for Health Quality, Outcomes, & Economic Research

(CHQOER)

Mark Prashker, M.D., M.P.H.

Edith Nourse Rogers Memorial Veterans Hospital, Bedford, 

Massachusetts 

Telephone: COM & FTS 781/687-3250

Fax: 781/687-3106

E-mail: Prashker.Mark@Bedford.va.gov

Investigators here focus on research to improve both the quality of

care that veterans receive and the efficiency with which the care is

provided. Primary research areas include: health economics, productivity measurement, quality assessment in ambulatory and

long term care, health outcomes measurement, case-mix measurement, cost effectiveness analysis, and decision analysis.

Center for Practice Management and Outcomes Research

Rodney A. Hayward, M.D.

VA Medical Center, Ann Arbor, Michigan 

Telephone: COM & FTS 734/930-5100 

Fax: 734/930-5159

E-mail: rhayward@umich.edu

Researchers at this Center conduct research that promotes optimal

management of resource-intensive care for America’s veterans.

Specific research priority areas include: quality monitoring and

improvement, outcome evaluation of alternative treatments, practice and outcomes variations, resource allocation, and shared decision-making.

■ VHA Headquarters Offices of Patient Care Services and Performance and Quality are jointly responsible for recommending the

adoption of non-VHA guidelines and for the adapting of guidelines

for use within the VHA. Recommendations from these two Offices

will be based on the advice of the Advisory Council for Adoption,

Development and Implementation of Clinical Practice Guidelines.

Office of Patient Care Services (112)

Ronald Gebhart, M.D., Chief Consultant, Primary Care

VA Headquarters

Phone: 202/273-8558

Fax: 202/273-9148

E-mail: gebhart.ron@mail.va.gov
Office of Performance and Quality (105A)

Debby Walder, R.N., M.S.N., Performance Management Facilitator

VA Headquarters

Phone: 202/273-8336

Email: debby.walder@mail.va.gov.

Advisory Council for Adoption, Development and Implementation of Clinical Practice Guidelines

Margaret Baumann, M.D. 

Associate Chief of Staff for Geriatrics and Extended Care (181)

Hines VA Medical Center

Telephone: 708/343-7200 ext. 2592

Fax: 708/216-2163

E-mail: Margaret.Baumann@med.va.gov

Adoption and Adaptation Work Group

Chairperson: Everett Jones, M.D.

VAMC, Salem

Telephone: 540/983-1037

Fax: 540/983-1080

E-mail: Everett.Jones@med.va.gov

Revision Work Group

Chairperson: Peter Almenoff, M.D.

VAMC, Kansas City

Telephone: 816/922-2475

Fax: 816/922-3323

E-mail: Peter.Almenoff@med.va.gov

Implementation and Education

Chairperson: Mike Davies, M.D.

Black Hills VAMC 

Telephone: 605/347-7172

Fax: 605/347-7171

E-mail: Michael.Davies@med.va.gov

Measurement

Chairperson: Reginald Penniston, M.D.

VAMC, Tampa

Telephone: 813/972-2000 ext. 6536

Information Management

Chairperson: Greg Larson, M.D.

VAMC, Portland

Telephone: 503/220/8262 ext. 55632

Fax: 503/273-5366

E-mail: Greg.Larsen@med.va.gov

Research and Evaluation

Chairperson: Jacqueline Pugh, M.D.

VA South Texas Health Care System, San Antonio, Texas

Telephone: 210/617-5314

Fax: 210/567-4423

E-mail: PUGH.JACQUELINE_A+@San-Antonio.va.gov

17
■ Employee Education System (EES)

Donna Schoonover, R.N., M.S.N., Program Manager

Phone: 314/894-5735

Fax: 314/894-6506

E-mail: schoonoverdon@lrn.va.gov

The mission of the Employee Education System is to provide educational services which are customer focused, accessible, performance

based, cost effective and which lead to the accomplishment of VHA’s

organizational goals and objectives. The EES plays a key role in

VHA’s efforts to implement clinical practice guidelines. Among EES

activities to support the implementation of clinical practice guidelines is the Clinical Pathways InfoNet which can be accessed on the

VHA intranet at http://vaww.lrn.va.gov/ees/ClinicalPathways/

SourcesOfClinPractGuide.asp

Mentor Visit Program The Mentor Visit Program provides an

opportunity for VISNs and facilities to have consultation/peer visit

by content experts and implementers of specific VHA-developed

guidelines. Based on needs identified by the requesting facility,

mentors can provide content review of specific guidelines and

assistance in integration of the guidelines with local practices.

They can address issues for successful implementation of the

guidelines. Mentors can also discuss outcomes and process measures to evaluate implementation of the guideline and practice.

Web-based information resources on the VHA Intranet The

following links were accurate at the time of publication. However,

the information available on the intranet changes frequently, so

resources listed below may no longer be available. For new intranet

resources since the publication of this primer, try using the intranet

search tool to search for some relevant phrases (e.g., “guideline

implementation,” “practice guideline,” “clinical guideline”).

Employee Education System: ClinPath InfoNet

http://vaww.lrn.va.gov/ees/ClinicalPathways/SourcesOfClinPractGuide.asp

Office of Patient Care Services

http://vaww.va.gov/med/patientcare

Office of Performance and Quality 

http://vaww.va.gov/stratinit/105a/clin/clinprac.htm

Pharmacy Benefits Management Strategic Health Group:

Treatment Guidelines

http://www.dppm.med.va.gov/#guidelines

VHA Intranet Hot Issues: Clinical Guidelines

http://vaww.va.gov/med/hotissues/clinical.asp

VHA Virtual Learning Center: Links

http://vaww.va.gov/med/osp/cgi-bin/links.asp

18
Outside VA

The following organizations outside VA are active in work related

to guideline implementation – sometimes in collaboration with VA

centers. This list is not exhaustive.

■ Agency for Health Care Policy and Research (AHCPR)

Douglas B. Kamerow, M.D., M.P.H.

Director, Practice and Technology Assessment

6000 Executive Blvd.

Rockville, MD 20720

Phone: 301/594-4015

Fax: 301/594-4027

E-mail: dkamerow@ahcpr.gov

World Wide Web: http://www.ahcpr.gov/

AHCPR, a long-standing leader in the development and promotion of

clinical practice guidelines has published 19 guidelines on a variety

of conditions and procedures. In 1996, AHCPR got out of the business of guideline development to focus on research in support of evidence-based medicine. Evidence-based practice centers supported by

AHCPR produce comprehensive reviews and rigorous analyses that

can be used to develop guidelines, performance measures, educational materials, and implementation strategies. In addition, AHCPR,

the American Medical Association, and the American Association of

Health Plans are jointly developing a National Guideline Clearinghouse to provide online access to a wide array of clinical guidelines

and syntheses of guidelines. The Clearinghouse is expected to be

operational by the fall of 1998 at the following Web site:

http://www.guidelines.gov.

■ The Cochrane Collaboration

Cochrane Collaboration Secretariat

PO Box 726

Oxford, OX2 7UX

United Kingdom

Phone: 44/1865-310138

Fax: 44/1865-516311

E-mail: secretariat@cochrane.co.uk

World Wide Web: http://www.cochrane.co.uk

The Cochrane Collaboration is an international organization that

produces and disseminates systematic reviews on the effects of

health care interventions. The Cochrane Library provides rapid

access to regularly updated reviews, structured abstracts on quality,

previously published reviews, references to controlled clinical trials

and to articles on the science of research review, and other sources of

information. It is published quarterly on CD-Rom and on the Internet.

■ Center for Clinical Effectiveness

Henry Ford Health System

Jennifer Elston Lafata, Ph.D.

Acting Director

1 Ford Place – 3C

Detroit, MI 48202

Phone: 313/874-1882

World Wide Web: http://hfhs-cce.org/

The Center for Clinical Effectiveness serves to coordinate and

improve the research, development, and implementation of clinical

19
policies and guidelines for the health care community at large and

for Henry Ford Health System in particular. Methods under evaluation through its clinical policy implementation trials include: application of CQI techniques, data feedback to physicians, information

technology, patient education, and practice models that use physician

extenders or nurses for service delivery.

■ Clinical Practice Enhancement Project (CPEP)

Robert Haywood, M.D., M.P.H.

Co-principal Investigator

Health Information Research Unit

McMaster University Medical Centre

1200 Main Street West

HSC 3H7C

Hamilton, Ontario Canada

L8N 3Z5

Phone: 905/525-9140, ext. 22060

Fax: 905/546-0401

E-mail: haywardr@fhs.mcmaster.ca

World Wide Web: http://hiru.mcmaster.ca/cpep/default.htm

CPEP promotes the implementation of clinical practice guidelines

through the development, evaluation, and dissemination of clinical

information tools, with emphasis on the collection, analysis, and

application of patient-reported health data. It is directed by investigators at McMaster University, the University of Chicago, and Johns

Hopkins University.

■ Institute for Clinical System Integration (ICSI)

Gordon Mosser, M.D.

Executive Director

8009 34th Ave., Suite 1200

Bloomington, MN 55425

Phone: 612/883-7991

Fax: 612/858-9675

ICSI is a clinical quality improvement organization that develops,

disseminates, and implements clinical guidelines for common conditions with its member medical groups. Created in 1992 by Park

Nicollet Clinic HealthSystem Minnesota, Mayo Clinic, and Health

Partners, ICSI provides a mechanism for health care providers and

purchasers to work together on clinical quality improvement.

Guideline implementation is a key component of ICSI’s activities.

■ Cedars-Sinai Medical Center

Department of Health Services Research

Scott Weingarten, M.D.

200 North Robertson Boulevard

Beverly Hills, CA 90211

Phone: 310/724-6386

Fax: 310/724-0746

E-mail: iweingarten@csmc.edu

The Department of Health Services Research at Cedars-Sinai develops and implements clinical practice guidelines that are used to support disease management programs to improve the care of patients

with chronic illnesses. Dr. Weingarten has conducted extensive

research on the implementation of practice guidelines, particularly

on the use of computerized reminders to support guideline implementation.

20
Collaboration of VA with External Organizations

■ Agency for Health Care Policy and Research — Evidence-based

Practice Centers

Douglas B. Kamerow, M.D., M.P.H.

Director, Practice and Technology Assessment

(see Outside VA Sources, above, for contact information) 

VA is collaborating with the Agency for Health Care Policy and

Research (AHCPR) in a national effort to gather scientific evidence

for local guideline development efforts. Evidence-based practice centers supported by AHCPR produce comprehensive reviews and rigorous analyses that can be used to develop guidelines, performance

measures, educational materials, and implementation strategies.

The following VA facilities are participating in AHCPR’s evidencebased practice initiative:

The Philadelphia VAMC is collaborating with MetaWorks, Inc.,

and the Leonard Davis Institute to develop an evidence report on

the diagnosis of sleep apnea.

The Northwest Network and the Portland VAMC are collaborating

with Oregon Health Sciences University and Northwest Kaiser

Permanente to review the evidence on rehabilitation of persons

with traumatic brain injury.

The VA Cochrane Center at San Antonio is collaborating with the

University of Texas at San Antonio, The San Francisco Cochrane

Center, and the American College of Physicians on a project to

evaluate the treatment of depression with new drugs.

VAMCs in California are conducting an evidence review on the

prevention and management of urinary complications in paralyzed

persons, with Rand, the University of California at Los Angeles

and San Diego, the University of Southern California, CedarsSinai Health System, and Value Health Sciences.

The San Francisco, Palo Alto, and Menlo Park VAMCs are working with the University of California at San Francisco, Stanford

University, the San Francisco Cochrane Center, and Kaiser Permanente to review evidence on the management of stable angina.

■ VA Cochrane Center at San Antonio

Cynthia Mulrow, M.D.

Director

Phone: 210/617-5190

Fax: 210/617-5234

E-mail: cochrane@merece.uthscsa.edu

VA participates in the Cochrane Collaboration, an international

organization that produces and disseminates systematic reviews on

the effects of health care interventions, through the San Antonio VA

Cochrane Center at the Audie L. Murphy Memorial Veterans Hospital. 

21
Appendix B: What reading materials are available

to provide more in-depth information on the implementation of clinical practice guidelines?

Conroy, M., and Shannon, W. “Clinical Guidelines: Their Implementation in General Practice.” British Journal of General Practice, 45

(396): 371-5, July 1995.

Davis, D.A., Thomson, M.A., et al. “Changing Physician Performance. A Systematic Review of the Effect of Continuing Medical

Education Strategies.” Journal of the American Medical

Association, 274 (9): 700-05, Sept. 6, 1995.

Davis, D.A., and Taylor-Vaisey, A. “Translating Guidelines into Practice: A Systematic Review of Theoretic Concepts, Practical Experience and Research Evidence in the Adoption of Clinical Practice

Guidelines.” Canadian Medical Association Journal, 157 (4): 408-

16, Aug. 15, 1997.

Ellrodt, A.G., Conner, L., et al. “Measuring and Improving Physician

Compliance with Clinical Practice Guidelines, A Controlled Intervention Trial.” Annals of Internal Medicine, 122 (4): 277-82, Feb. 15,

1995.

Greco, P.J., and Eisenberg, J.M. “Changing Physicians’ Practices.”

New England Journal of Medicine, 329 (17): 1271-3, Oct. 21, 1993.

Grilli, R., and Lomas, J. “Evaluating the Message: The Relationship

between Compliance Rate and the Subject of a Practice Guideline.”

Medical Care, 32 (3): 202-13, March 1994.

Grimshaw, J., Eccles, M., and Russell, I. “Developing Clinically Valid

Practice Guidelines.” Journal of Evaluation in Clinical Practice, 

1(1), 37-48, Sept. 1995.

Grimshaw, J.M., and Russell I.T. “Effect of Clinical Guidelines on

Medical Practice: A Systematic Review of Rigorous Evaluations.”

Lancet, 342 (8883): 1317-22, Nov. 27, 1993.

Grol, R. “Personal Paper. Beliefs and Evidence in Changing Clinical Practice.” British Medical Journal, 315 (7105): 418-21, Aug. 16,

1997.

Hutchinson, A. “The Philosophy of Clinical Practice Guidelines:

Purposes, Problems, Practicality and Implementation.” Journal of

Quality in Clinical Practice, 18 (1): 63-73, March 1998.

Mittman, B.S., Tonesk, X., and Jacobson, P.D. “Implementing Clinical Practice Guidelines: Social Influence Strategies and Practitioner

Behavior Change.” Quality Review Bulletin, 18 (12): 413-23, Dec.

1992.

22
Mosser, G. “Clinical Process Improvement: Engage First, Measure

Later.” Quality Management in Health Care, 4 (4): 11-20, 1996.

Oxman, A.D., Thomson, M.A., et al. “No Magic Bullets: A Systematic Review of 102 Trials of Interventions to Improve Professional

Practice.” Canadian Medical Association Journal, 153 (10): 1423-

31, Nov. 15, 1995.

Soumerai, S.B., McLaughlin, T.J., et al. “Effect of Local Medical

Opinion Leaders on Quality of Care for Acute Myocardial Infarction:

A Randomized Controlled Trial.” Journal of the American Medical

Association, 279 (17): 1358-63 , May 6, 1998.

Thomson, R., Lavender, M., and Madhok, R. “How to Ensure that

Guidelines are Effective.” British Medical Journal, 311 (6999): 237-

42, July 25, 1995.

Thomson, R., McElroy, H., and Sudlow, M. “Guidelines on Anticoagulant Treatment in Atrial Fibrillation in Great Britain: Variation in

Content and Implications for Treatment.” British Medical Journal,

316 (7130): 509-13, Feb. 14, 1998.

Weingarten, S., and Ellrodt, A.G. “The Case for Intensive Dissemination: Adoption of Practice Guidelines in the Coronary Care Unit.”

Quality Review Bulletin, 18 (12): 449-55, Dec. 1992.

Wise, C.G., and Billi, J.E. “A Model for Practice Guideline Adaptation and Implementation: Empowerment of the Physician.” Journal on Quality Improvement, 21 (9): 465-76. Sept. 1995.

23
Appendix C: What research is VA conducting on

guideline implementation strategies?

VA’s Health Services Research & Development Service

(HSR&D) has long recognized the importance of investigating

promising strategies for guideline implementation. More recently, 

a solicitation invited researchers to evaluate strategies for implementing evidence-based practice guidelines within VA and to identify

approaches that may be replicated systemwide. Listed below is a

sampling of guideline implementation projects within VA, with contact information and a brief description of relevant activities. This

list is not exhaustive.

Implementing AHCPR’s Smoking Cessation Guideline

Ann Joseph, M.D., M.P.H.

Minneapolis VAMC 

Phone: 612/725-2158

Fax: 612/725-2118

National guidelines for smoking cessation urge identification of all

smokers in the patient record, a minimum of brief counseling for all

smokers, and liberal use of nicotine replacement therapy. However,

data suggest a low rate of compliance with these recommendations

among VA facilities. This study is exploring whether an organizational support strategy that provides training and follow-up consultation for key personnel on smoking cessation interventions can

improve provider adherence to the guidelines and increase smoking

cessation rates.

Drug Therapy for Hypertension

Brian Hoffman, M.D.

Palo Alto VAMC

Phone: 650/493-5000, ext. 64575

Fax: 650/855-9437

E-mail: Bhoffmann@leland.stanford.edu

Evidence-based clinical practice guidelines recommend the use of

thiazide diuretics and beta receptor antagonists to reduce hypertension, but newer, heavily marketed, and more expensive drugs, such

as angiotensin converting enzyme inhibitors and calcium channel

antagonists, are gaining favor among physicians. In this study,

investigators are exploring how an automated reminder system can

optimize cost-effective drug selection for hypertension in a manner

that is sensitive to individual patient outcomes. This new system,

called ATHENA, will be evaluated to determine its impact on clinician adherence to practice guidelines and on blood pressure control,

quality of life, and drug costs for antihypertensive agents. The longterm objective of this project is to help VA respond flexibly to evolving medical knowledge by establishing a system for implementing

guidelines that can be used throughout VA.

Using Expert Consultants to Implement Smoking Cessation

Guidelines

Scott Sherman, M.D.

Sepulveda VAMC

Phone: 818/891-7711, ext. 9909

Fax: 818/895-9509

E-mail: sherman.scott@sepulveda.va.gov

24
This study, which involves the primary care practices of 20 VAMCs,

is testing the effectiveness of evidence-based quality improvement

(QI) as an implementation strategy for national guidelines on smoking cessation. Using survey, administrative, and medical record

data, the investigators will determine whether evidence-based QI

methods improve provider compliance, increase smoking cessation

rates, and achieve better patient outcomes. The long-term goal of

this study is to develop and disseminate effective strategies for

implementing practice guidelines and improving the quality and outcomes of VA services related to smoking behavior and other health

care problems.

Educating Providers on Substance Abuse Treatment 

Guidelines

Richard McCormick, Ph.D.

Cleveland VAMC

Phone: 216/526-3030, ext. 6904

Fax: 440/546-2762

E-Mail: McCormick.Richard@cleveland.va.gov

Up to 52 percent of the seriously mentally ill have comorbid substance abuse problems that complicate treatment and produce negative outcomes. General mental health specialists who treat the

dually diagnosed psychotic often have limited training in substance

abuse treatment. This study is investigating the impact of provider

education on compliance with newly developed practice guidelines

for dually diagnosed patients. Investigators are also measuring the

guidelines’ effects on patient outcomes, resource utilization, treatment compliance, and patient satisfaction.

Pressure Ulcer Care in Nursing Homes

Dan Berlowitz, M.D., M.P.H.

Bedford VAMC

Phone: 617/687-3250

Fax: 617/687-3106

E-mail: berlowitz.d@bedford.va.gov

Little is known about successful strategies for implementing practice

guidelines in long-term care settings. Project investigators believe

that there are distinct organizational features, including culture,

quality improvement practices, employee control systems, and hospital policies that may promote successful implementation. This study

is using data from 36 VA nursing homes to identify relationships

between these features and adherence to pressure ulcer care guidelines and risk-adjusted patient outcomes. The results will be used to

develop specific strategies for improving guideline implementation

and patient outcomes. These findings may have implications for

guidelines on other conditions that are common among nursing

home patients.

Clinical Effects of Enhanced Computerized Guidelines

William Tierney, M.D.

Indianapolis VAMC

Phone: 317/630-7660

Fax: 317/630-6611

E-mail: btierney@vax1.iupui.edu

In this project, investigators are using national guidelines for treating heart failure to develop a computerized reminder system that

can be triggered both by administrative data and by subjective data

collected directly from patients on symptoms, history, and behavior.

Without these data, the investigators believe, computerized guideline

25
reminders cannot support critical care decisions. This study will

determine whether the reminder system being developed is valid, so

that it may be tested further in a randomized, controlled trial. 

The results of this study have system-wide implications for VA.

Strategies for Implementing Schizophrenia Guidelines

Richard Owen, M.D.

Little Rock VAMC

Phone: 501/688-1622

Fax: 501/688-1621

E-mail: rrowen@hsrd.uams.edu

Antipsychotic medication is an essential component of treatment for

nearly all patients with schizophrenia – yet data indicate that it frequently is not prescribed. This project is examining the effectiveness

of two strategies for implementing guidelines on schizophrenia: (1)

mentor visits to educate VISN and facility leaders, and (2) identification and training of opinion leaders at VAMCs to promote implementation. The study, which involves eight VA sites, will also evaluate

the impact of the implementation strategies on patient outcomes,

service utilization, and providers’ knowledge of and attitude toward

the guidelines. The researchers hope to apply what they learn from

this study to the implementation of guidelines for other psychiatric

disorders as well.

Implementing Guidelines in the Emergency Department

Paul G. Shekelle, M.D.

West Los Angeles VAMC

Phone: 310/268-3254

Fax: 310/268-4933

E-mail: shekelle.Paul@West-LA.va.gov

In this project, investigators are using a computerized expert charting system to implement guidelines for the treatment of urethritis

and obstructive airway disease in the emergency department of a VA

teaching hospital. So far, they have identified significant deficiencies

in care provided to these patients, compared with optimal care as

defined in the guidelines. They are using health status measures to

evaluate the impact of guideline implementation on patients.

Computerized Reminders for Ambulatory Care Guidelines

Charles Beauchamp, M.D., Ph.D.

Durham VAMC

Phone: 919/286-6963

Fax: 919/416-5881

E-mail: Beauchamp@Durham.va.gov

John G. Demakis, MD

Hines VAMC

Phone: 202/273-8287

Fax: 202/273-9007

E-mail: john.demakis@mail.va.gov

The primary objective of this study is to determine whether computer-generated reminders can improve resident physicians’ compliance

with guidelines for ambulatory care. Investigators evaluated the

effectiveness of computerized reminders in a prospective, randomized trial involving 275 residents and 13 well-accepted sets of guidelines. Improvements in adherence resulting from the computer

prompts will have a direct and important impact on the quality of

care provided to patients at VA ambulatory care clinics.

26
T

Appendix D: Ongoing Research by the Cochrane 

Collaboration

he Cochrane Effective Practice and Organization of Care Review

Group (EPOC) is conducting a series of evidence reviews, continuing

its examination of strategies for improving health care practice and

outcomes. Some of these reviews will shed light on questions surrounding system barriers to implementing practice guidelines and

changing health care practice.

To contact EPOC:

Graham Mowatt

Review Group Coordinator

Health Services Research Unit

University of Aberdeen

Polwarth Building

Foresterhill

Aberdeen, UK AB25 2ZD

Phone: +44 1224-403082

Fax: +44 1224-663087

E-mail: g.mowatt@abdn.ac.uk

Lisa Bero (Editorial Team)

Institute for Health Policy Studies

Clinical Pharmacy Department

University of California, San Francisco

1388 Sutter St., 11th Floor

San Francisco, CA 94109

Phone: 415/476-1067

Fax: 415/476-0705

E-mail: bero@cardio.ucsf.edu

Following is a description of some of EPOCs ongoing reviews.

The effectiveness of reminders in improving health care practice and

patient outcomes. Reminders can help health care professionals

recall important information from practice guidelines at the time

that it’s needed. Several studies have found that reminders can promote change in professional practice across a variety of clinical conditions and settings. This series of reviews examines three types of

reminders – paper, computer-generated, and on-screen — and considers a number of other factors likely to have an impact on their

effectiveness.

Interventions for applying prevention in primary care. The primary

care setting offers a unique opportunity to delivery preventive services, but that opportunity may be missed if health care professionals lack a “preventive attitude.” This review critically analyzes the

effectiveness of different methods of improving professional preventive behavior in primary care.

The impact of educational meetings, workshops, and preceptorships

on health care practice. Although recent evidence suggests that some

continuing education interventions may improve health care practice, traditional activities such as workshops or seminars may not be

the most effective. This review will determine the effectiveness of

27
educational meetings, workshops, lectures, symposia, courses,

traineeships, and other educational interventions in improving

health care practice and outcomes.

Computerized advice on drug dosage. Monitoring drug therapy to

optimize effects and minimize adverse events can be very time-consuming and require meticulous attention to detail. Computers, however, are very good at collecting information and performing

repetitive calculations; there are a number of drugs for which computer advice might be helpful. This review is examining the effectiveness of computerized drug dosage advice in improving

prescribing practices.

Guidelines in professions allied to medicine. Most practice guidelines are geared toward physicians, even though many types of

health care professionals are involved in the delivery of care. To

date, there has been no evaluation of the effectiveness of implementation strategies for professionals allied to medicine. The objective

of this review is to identify rigorous evaluations of clinical practice

guidelines in the following professions: nursing (including midwifery and health visiting), chiropody, speech and language therapy,

physiotherapy, occupational therapy, dietetics, clinical psychology,

pharmacy, and radiography.

Interventions to improve immunization rates. A number of interventions – including audit and feedback, reminders, financial incentives,

and practice policy changes – have been devised to improve the delivery of immunizations. Even though immunization is a cost-effective

and widely accepted means of preventing disease worldwide, the percentage of children and adults immunized falls below targeted rates.

The objective of this review is to assess the effectiveness of interventions designed to improve immunization delivery.

28
VA 

Appendix E: How is the VA Measuring Guideline

Implementation?

has incorporated guideline implementation into performance

agreements between network directors and the Under Secretary for

Health. Following is a list of the 1998 mandated performance measures, which were developed from practice guidelines. (These measures are in addition to measures included in VA’s Chronic Disease

Care Index, Prevention Index, and Palliative Care Index. Some of

those measures are also derived from practice guidelines.) The list of

guidelines and performance measures will be updated annually. Recommended guidelines for FY 1999 include: chronic heart failure,

chronic obstructive pulmonary disease, and psychoses medication

use for schizophrenia and bipolar disorder.

For more information on mandated guidelines and performance measures, contact:

Debby Walder, R.N., M.S.N.

Office of Performance and Quality (105A)

Performance Management Facilitator

810 Vermont Avenue, NW

Washington, DC 20420

Phone: 202/273-8336

E-mail: debby.walder@mail.va.gov. 

Guideline FY 98 Measure Additional Data Collection

Ischemic Heart Disease

Modules 1 & 5. Patients ER ASA in 24 hours

with non-traumatic chest pain.

Module 2. Early treatment. All patients with AMI consid- Door-to-needle time of 

ered for thrombolytic therapy administration of 

or PTCA thromobolytic therapy

Module 8. Ambulatory care Use of aspirin Percentage of patients with left

follow-up. ventricular dysfunction treated

Plan for reduction of cardiac with ACE inhibitors

risk factors: lifestyle changes,

exercise, nutritional modifica- Percentage of patients who 

tions, smoking cessation, currently smoke

cholesterol management as

appropriate Percentage of patients with

LDL>130

Continued on next page

29
Guideline FY 98 Measure Additional Data Collection

Diabetes Mellitus

Module G. Glycemic control. Annual HBA1c

Percentage of patients with 

HBA1c greater than 10

Module F. Foot care. Annual foot inspection Percentage of patients with 

at-risk foot referred to foot

Check of pedal pulses care specialist

Foot sensory exam Amputation rates

Module E. Eye care. Annual retinal exam

Module R. Renal. Annual serum creatinine and 

urine protein (if urine protein 

is negative or trace, micro 

albuminuria test)

Module H. Hypertension. BP checked at each visit Distribution of hypertension 

grade, i.e., 1, 2, 3, 4

Module L. Lipid Control. Annual lipid profile Percentage of patients with 

LDL>130

Hypertension

BP checked at two of three visits Distribution of hypertension 

grade, i.e., 1, 2, 3, 4

Major Depressive Disorder

Module A. Patients seen in general Percentage screened for alcohol

medicine, primary care, and the use using a standardized

women’s clinic screened for MDD instrument

Percentage screened for PTSD

Module C. Inpatient Mental Patients with acute psychiatric 

Health. admission have one GAF rating

Smoking Cessation

Percentage of patients screened Percentage of patients who

for tobacco use currently smoke

If current smoker, advised of 

health risks of smoking and 

offered smoking cessation 

counseling. If counseling not 

offered, reason must be 

documented.

30
Fax us your comments!

To: MDRC 

Fax: 617/278-4438

How will you use the clinical practice guidelines primer? (check all that apply)

___ for my own education/information

___ to work with other staff members to increase understanding of clinical practice guidelines

___ as a meeting/conference/inservice training handout

___ other (please specify)__________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

Please rate the amount of information provided (circle one)

(1=not enough 5=just right) 1 2 3 4 5

What is your overall rating of the primer? (circle one)

(1=not helpful 5=very helpful) 1 2 3 4 5

General comments _____________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

Suggestions for future primer topics___________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

From: _________________________________________

_________________________________________

_________________________________________

_________________________________________

_________________________________________ 

name

title

address/facility
Clinical Practice Guidelines is available in

electronic and printed formats. Additional copies may

be obtained from the sources listed below.

Electronic copies (PDF format) can be downloaded

from the VA home page on the world wide web.

Point your browser to 

http://www.va.gov/resdev/prt

Faxed copies:

From a telephone, call

617/278-4492

Follow the voice menu system instructions to order the

clinical practice guidelines primer. Requesters are advised 

that fax transmission can take up to 30 minutes.

Printed copies:

contact Special Projects Office (152)

VA Medical Center

Perry Point, MD 21902

phone: 410/642-1092

FTS: 700/956-5442

fax: 410/642-1095

e-mail: laurel.long@med.va.gov

Other primers in the series include:

Primary Care in VA

Health Care Technology Assessment in VA

Using Outcomes to Improve Healthcare Decision Making

Program Evaluation for Managers

Risk Adjustment: A Tool for Leveling the Playing Field