Technical Review 

Number 9 

Closing the Quality Gap: 

A Critical Analysis of Quality Improvement Strategies 

Volume 4—Antibiotic Prescribing Behavior 

Prepared for: 

Agency for Healthcare Research and Quality 

U.S. Department of Health and Human Services 

540 Gaither Road 

Rockville, MD 20850 

www.ahrq.gov 

Contract No. 290-02-0017 

Prepared by: 

Stanford-UCSF Evidence-based Practice Center 

Series Editors 

Kaveh G. Shojania, M.D., University of California, San Francisco 

Kathryn M. McDonald, M.M., Stanford University

Robert M. Wachter, M.D., University of California, San Francisco

Douglas K. Owens, M.D., M.S., VA Palo Alto Health Care System, Palo Alto, California 

Stanford University

Managing Editor

Amy J. Markowitz, J.D. 

Investigators

Sumant R. Ranji, M.D. 

Michael A. Steinman, M.D. 

Kaveh G. Shojania, M.D. 

Vandana Sundaram, M.P.H. 

Robyn Lewis, M.A. 

Sandra Arnold, M.D., M.S. 

Ralph Gonzales, M.D., M.S.P.H. 

AHRQ Publication No. 04(06)-0051-4 

January 2006
This report is based on research conducted by the Stanford-UCSF Evidence-based Practice 

Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), 

Rockville, MD (Contract No. 290-02-0017). The findings and conclusions in this document 

are those of the author(s), who are responsible for its content, and do not necessarily 

represent the views of AHRQ. No statement in this report should be construed as an official 

position of AHRQ or of the U.S. Department of Health and Human Services. 

The information in this report is intended to help clinicians, employers, policymakers, and 

others make informed decisions about the provision of health care services. This report is

intended as a reference and not as a substitute for clinical judgment. 

This report may be used, in whole or in part, as the basis for the development of clinical 

practice guidelines and other quality enhancement tools, or as a basis for reimbursement and 

coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of 

such derivative products may not be stated or implied. 
ii 

This document is in the public domain and may be used and reprinted without permission except 

those copyrighted materials noted for which further reproduction is prohibited without the 

specific permission of copyright holders. 

Suggested Citation: 

Ranji SR, Steinman MA, Shojania KG, Sundaram V, Lewis R, Arnold S, Gonzales R. 

Antibiotic Prescribing Behavior. Vol. 4 of: Shojania KG, McDonald KM, Wachter RM, Owens 

DK, editors. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies. 

Technical Review 9 (Prepared by the Stanford University-UCSF Evidence-based Practice Center 

under Contract No. 290-02-0017). AHRQ Publication No. 04(06)-0051-4. Rockville, MD: 

Agency for Healthcare Research and Quality. January 2006. 

None of the investigators has any affiliations or financial 

involvement that conflicts with the material presented in 

this report. 
iii 

Preface 

 The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-Based 

Practice Centers (EPCs), sponsors the development of evidence reports and technology 

assessments to assist public- and private-sector organizations in their efforts to improve the 

quality of health care in the United States. The reports and assessments provide organizations 

with comprehensive, science-based information on common, costly medical conditions and new 

health care technologies. The EPCs systematically review the relevant scientific literature on 

topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to 

developing their reports and assessments. 

 To bring the broadest range of experts into the development of evidence reports and health 

technology assessments, AHRQ encourages the EPCs to form partnerships and enter into 

collaborations with other medical and research organizations. The EPCs work with these partner 

organizations to ensure that the evidence reports and technology assessments they produce will 

become building blocks for health care quality improvement projects throughout the Nation. The 

reports undergo peer review prior to their release. 

 AHRQ expects that the EPC evidence reports and technology assessments will inform 

individual health plans, providers, and purchasers as well as the health care system as a whole by 

providing important information to help improve health care quality. 

 We welcome written comments on this evidence report. They may be sent by mail to the 

Task Order Officer named below at: Agency for Healthcare Research and Quality, 540 Gaither 

Road, Rockville, MD 20850 or by email to epc@ahrq.gov. 

Carolyn M. Clancy, M.D. Jean R. Slutsky, P.A., M.S.P.H. 

Director Director, Center for Outcomes 

Agency for Healthcare Research and Evidence 

 and Quality Agency for Healthcare Research 

 and Quality 

Kenneth S. Fink, M.D., M.G.A., M.P.H. Marian D. James, Ph.D., M.A. 

Director, EPC Program EPC Program Task Order Officer 

Agency for Healthcare Research Agency for Healthcare Research 

 and Quality and Quality 

 
iv 

Acknowledgments 

We are grateful to our many technical experts and content reviewers for their uniformly 

thoughtful comments. 

We give special thanks to our panel of technical advisors: Jeremy Grimshaw, MBChB, PhD, 

University of Ottawa, Cochrane Collaboration Effective Practice and Organisation of Care 

Group (EPOC); Andrew Oxman, MD, MSc, Department of Health Services Research, 

Norwegian Directorate for Health and Social Welfare, EPOC; Russell Glasgow, PhD, Kaiser 

Permanente, Colorado; Martin Eccles, MD, Centre for Health Services Research, University of 

Newcastle upon Tyne, EPOC; Harmon Jordan, ScD, previously with the New England Medical 

Center Evidence-based Practice Center and currently at Abt Associates; Val Lawrence, MD, 

MSc, The University of Texas Health Science Center at San Antonio and South Texas Veterans 

Health Care System; James Zazzali, PhD, RAND Institute; and Charles Homer, MD, MPH, 

National Institute for Children’s Health Care Quality; whose attentive guidance to the entire 

Closing the Quality Gap series and continued assistance on this report was critical to the project. 

We are grateful to our panel of peer reviewers: Richard Besser, MD, Centers for Disease Control 

and Prevention; Edward Belongia, MD, Marshfield Clinic; Jonathon Finkelstein, MD, MPH, 

Department of Ambulatory Care and Prevention, Harvard Medical School; Kitty Corbett, PhD, 

Denver University; Randall Stafford, MD, PhD, Stanford University; Brian Strom, MD, MPH, 

CERT Program, University of Pennsylvania; and Theoklis Zaoutis, MD, Department of 

Pediatrics, University of Pennsylvania School of Medicine; who read early drafts of the chapters 

and provided timely and detailed comments. 

We also extend our thanks to David Atkins, MD, MPH, Marian James, PhD, MA, and Susan 

Norris, MD, MPH, from the Agency for Healthcare Research and Quality for their assistance 

with this report.
v 

Structured Abstract

Objective: Unnecessary prescribing of antibiotics is a major problem in the US and worldwide, 

contributing to the problem of antimicrobial resistance (AMR). This review examines the effects 

of quality improvement strategies on reducing inappropriate prescribing of antibiotics, targeting 

both prescribing of antibiotics for non-bacterial illnesses (“the antibiotic treatment decision”) and 

prescribing of broad-spectrum antibiotics when narrow-spectrum agents are indicated (“the 

antibiotic selection decision”). 

Search Strategy and Inclusion Criteria: We evaluated studies examining the effectiveness of 

quality improvement (QI) strategies targeting outpatient antibiotic prescribing for acute illnesses. 

Studies were identified by searching the Cochrane Collaboration’s Effective Practice and 

Organisation of Care registry and MEDLINE®. We included randomized and quasi-randomized 

controlled trials, controlled before-after studies, and interrupted time series that reported 

measures of antimicrobial use. QI strategies were classified as clinician education, patient 

education, provision of delayed prescriptions, audit and feedback, clinician reminders, and 

financial or regulatory incentives. Our primary outcomes were the percentage of patients 

prescribed an antibiotic (for antibiotic treatment studies); or the percentage of patients prescribed 

a recommended antibiotic or guideline-concordant antibiotic therapy (for antibiotic selection 

studies). Secondary outcomes included effects on antimicrobial resistance, intervention safety 

(disease outcomes and adverse events), prescribing costs, and patient satisfaction. 

Data Collection and Analysis: Two reviewers abstracted data on the components of the QI 

intervention, study population, targets, and outcomes. We compared the effects of QI strategies 

in terms of the median effect achieved for the primary outcomes, using nonparametric tests; 

studies not eligible for median effects analysis were summarized qualitatively. 

Main Results: Fifty-four studies reporting a total of 74 trials met the inclusion criteria; 34 

studies (reporting 41 trials) addressed the treatment decision, and 26 studies (reporting 33 trials) 

addressed the selection decision. Six studies evaluated both decisions. Study methodologic 

quality was generally fair. Nearly all studies took place in outpatient primary care clinics. 

Studies addressing the antibiotic treatment decision: Most studies addressed prescribing for 

acute respiratory infections (ARIs). Interventions were effective at reducing prescribing, with a 

median absolute effect of -8.9% [interquartile range (IQR) -12.4% to -6.7%]. No individual QI 

strategy (or combination of strategies) was more effective at reducing prescribing. Within 

clinician education, active educational strategies appeared more effective than passive strategies. 

When extrapolated to a population level, strategies targeting general antibiotic prescribing 

appeared to reduce antibiotic prescribing more than strategies targeting prescribing for a single 

condition. Few studies addressed secondary endpoints; patient satisfaction was not worsened by 

QI interventions, but effects on AMR or costs could not be assessed. 

Studies addressing the antibiotic selection decision: Interventions targeted prescribing for ARIs 

or urinary tract infections (UTIs). Interventions were effective, with a median absolute 

improvement in prescribing of recommended antibiotics of 10.6% (IQR 3.4% to 18.2%). 

Clinician education alone appeared more effective than education in combination with audit and 
vi 

feedback, but this finding likely represents confounding. Very few studies addressed secondary 

outcomes. 

Conclusion: Quality improvement efforts appear generally effective at reducing both 

inappropriate treatment with antibiotics and inappropriate selection of antibiotics. While no 

single QI strategy was more effective than others, active clinician education may be more 

effective than passive education, particularly for addressing the antibiotic treatment decision. 

Greater reductions in overall prescribing may be achieved through efforts targeting prescribing 

for all acute respiratory infections, rather than targeting single conditions. The available 

evidence is of only fair quality, and further research on the cost-effectiveness and potential 

harms of these interventions is needed. 
vii 

Contents

Executive Summary.........................................................................................................................1 

Chapter 1. Introduction ...................................................................................................................7 

Rationale for Reducing Excess Antibiotic Use in Ambulatory Practice ...................................7 

Current Antibiotic Prescribing Practice Patterns in US Ambulatory Practices .........................8 

The Purpose of This Evidence-based Practice Center Report ...................................................9 

An Explanatory Model of Antibiotic Prescribing Behavior ....................................................10 

Clinician Factors ................................................................................................................11 

Public/Patient Factors ........................................................................................................12 

Health Care Delivery System Factors................................................................................12 

Chapter 2. Methods.......................................................................................................................13 

Scope........................................................................................................................................13 

Definitions of Quality Improvement Terms Used in This Report ...........................................13 

Classification of Interventions .................................................................................................13 

Classification of Quality Improvement Strategies...................................................................14 

Inclusion and Exclusion Criteria..............................................................................................15 

Literature Search and Review Process.....................................................................................16 

Outcome Measures...................................................................................................................17 

Assessment of Study Quality...................................................................................................17 

An Explanatory Model for Evaluating the Different QI Strategies To Improve 

Antibiotic Prescribing Behavior ........................................................................................18 

Statistical Analysis...................................................................................................................19 

Calculation of Effective Sample Sizes.....................................................................................21 

Calculation of the Population Effect Size................................................................................21 

Particularly Salient Studies......................................................................................................22 

Chapter 3. Results .........................................................................................................................23 

Studies To Improve the Antibiotic Treatment Decision..........................................................23 

Studies Meeting Inclusion Criteria ....................................................................................23 

Settings, Goals, and Target Population of Studies.............................................................23 

Intervention Characteristics ...............................................................................................25 

Quality of Included Studies ...............................................................................................26 

Effectiveness of Specific QI Strategies and Combinations of Strategies ..........................27 

Delayed Prescription Studies .............................................................................................29 

Studies Not Eligible for Median Effect Size Analysis.......................................................29 

Population Effect Size........................................................................................................30 

Within-Study Comparisons................................................................................................32 

Special Populations and Secondary Endpoints..................................................................33 

Studies To Improve the Antibiotic Selection Decision............................................................34 

Studies Meeting Inclusion Criteria ....................................................................................34 

Setting, Goals, and Target Population of Studies ..............................................................34 

QI Strategy Characteristics ................................................................................................35 

Quality of Included Studies ...............................................................................................35 
viii 

Effect of Interventions on Antibiotic Selection .................................................................36 

Studies of Duration of Antibiotic Therapy ........................................................................38 

Studies Not Eligible for Median Effect Size Analysis.......................................................39 

Comparisons Within Studies..............................................................................................40 

Special Populations and Secondary Endpoints..................................................................40 

Chapter 4. Discussion ...................................................................................................................43 

Summary..................................................................................................................................43 

Limitations ...............................................................................................................................43 

Conclusions..............................................................................................................................45 

References......................................................................................................................................49 

Listing of Excluded Studies...........................................................................................................59 

Figures 

Figure 1. Factors affecting the decision to prescribe antibiotics ..................................................11 

Figure 2. Evaluation of QI strategies for confounders and moderators........................................19 

Figure 3. Ambulatory antibiotic prescriptions for various ARIs, by age group ...........................22 

Figure 4. Search strategy and article triage...................................................................................24 

Figure 5. Net reduction in antibiotic prescribing rate for individual studies 

(antibiotic treatment studies)....................................................................................................28 

Figure 6. Population-level effect of studies targeting the antibiotic treatment decision ..............31 

Figure 7. Projected cost savings from antibiotic treatment interventions.....................................32 

Figure 8. Effect sizes for each included study (antibiotic selection studies)................................37 

Tables 

Table 1A. QI strategies and outcomes for antibiotic treatment studies ........................................71 

Table 1B. QI strategies and outcomes for antibiotic selection studies .........................................81 

Table 1C. Antibiotic treatment studies: target condition and population .....................................88 

Table 1D. Antibiotic selection studies: target condition and population......................................89 

Table 2. Antibiotic treatment studies: number of QI strategies utilized.......................................90 

Table 3. Median effect size of QI strategies on antibiotic treatment decision..............................91 

Table 4. Antibiotic treatment studies: QI strategies and study modifiers.....................................92 

Table 5. Results of Wilcoxon rank-sum tests for differences between groups.............................94 

Table 6. Delayed prescription studies...........................................................................................96 

Table 7. Antibiotic selection studies: number of QI strategies utilized........................................97 

Table 8. Median effect size of QI strategies on antibiotic selection decision...............................98 

Table 9. Selection studies: QI strategies and study modifiers ......................................................99 

Table References..........................................................................................................................103 
ix 

Appendixes 

Appendix A. Summary of key studies 

Appendix B. Summaries of included studies, grouped by setting and measured 

population 

Appendix C. EPOC and MEDLINE® search for antibiotic QI articles 

Appendix D. Antibiotic abstraction forms for screening and full-text review 

Appendix E. Random effects meta-analysis of included trials 

Appendix F. Technical expert panel and peer reviewers 

Appendixes and Tables for this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 

1 

Executive Summary 

The introduction of antibiotic therapy into medical practice has yielded substantial benefits 

for patients over the past 6 decades. However, the benefits of antibiotic use to individual patients 

come at a societal cost: the emergence of antimicrobial resistance (AMR) among bacterial 

pathogens. Once confined to the inpatient setting, resistant bacteria are now common 

community-acquired infections as well. Antibiotic use encourages the development and spread of 

antibiotic-resistant bacteria by at least two mechanisms: (1) by applying selective pressure, 

encouraging development of new strains of antibiotic-resistant bacteria, and (2) by eliminating 

normal bacterial flora in human hosts, which promotes colonization and spread of existing 

antibiotic-resistant strains. The increasing prevalence of antibiotic resistance has led to the use of 

more expensive and broad-spectrum antibiotics for empiric treatment of common outpatient 

infections, and increased morbidity and mortality among patients hospitalized with serious 

community-acquired infections. 

Reducing inappropriate use of antibiotics is a critical step in slowing the progression of 

current levels of resistance, and in preventing the emergence of new strains of antibiotic-resistant 

bacteria. Accomplishing this requires a two-part approach. First, the use of antibiotics in 

conditions for which these drugs provide little or no benefit must be reduced. Second, antibiotics 

prescribed for patients who do require antimicrobial therapy must be appropriately targeted, and 

inappropriately lengthy treatment courses should be shortened. 

In this fourth volume of the Closing the Quality Gap series, we critically analyze quality 

improvement strategies to reduce inappropriate antibiotic prescribing. We focus on interventions 

targeting antibiotic prescribing for acute illnesses in the outpatient setting, primarily acute 

respiratory infections (ARIs). Prescribing for acute conditions accounts for the majority of 

antibiotics dispensed in the US, and is thus likely to have the greatest influence on AMR 

patterns. We examined the effect of quality improvement strategies on antibiotic treatment (the 

decision to prescribe antibiotics for illnesses for conditions generally not requiring antibiotic 

therapy) and antibiotic selection (the choice of one antibiotic over another for illnesses requiring 

antibiotic treatment). Our review includes studies on the effect of prescribing-focused QI 

strategies on AMR, clinical outcomes, costs of prescribing, and patient satisfaction. 

We structured the review to address the following key threshold questions: 

1. Are quality improvement strategies to improve outpatient antibiotic use effective? 

2. What are the critical components of effective intervention strategies? 

3. Which patients and conditions should be targeted in order to exert the maximal impact on 

antibiotic prescribing? 

4. What are the limitations of current research in this field, and which areas require further 

study? 

As in previous reviews in this series, we performed a rigorous search of the published 

literature using the Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) 

database, supplemented by targeted MEDLINE® searches. We classified QI interventions 

according to a modification of a taxonomy used in previous volumes of this series. The QI 

strategies were classified as follows: 

1. Clinician education 

2. Patient education 
2 

3. Provision of delayed prescriptions 

4. Audit and feedback 

5. Clinician reminder systems 

6. Financial or regulatory incentives for patients 

7. Financial or regulatory incentives for clinicians 

Educational strategies were subdivided into active or passive strategies, based on whether or 

not the learner was actively engaged in the learning process.1

 We analyzed the QI strategies for 

their effects on antimicrobial prescribing, defined as the change in the percentage of patient visits 

at which an antibiotic was prescribed (for treatment decision studies) or the change in the 

percentage of visits at which a recommended antibiotic was prescribed (for selection decision 

studies). We quantitatively synthesized the studies by determining the median effect of studies 

using a particular QI strategy (or combination of strategies), and rigorously evaluated for the 

presence of potential confounders or effect modifiers. Formal meta-analysis was not possible 

due to heterogeneity among the included studies. For details on the statistical methodology used, 

please refer to Closing the Quality Gap, Volume 1—Series Overview and Methodology (AHRQ 

publication No. 04-0051-1). 

From a sample of 521 potentially relevant articles, we reviewed the full text of 147 articles. 

Of these, a total of 54 articles, reporting a total of 74 comparisons, met our inclusion criteria and 

were reviewed. The treatment decision was addressed in 34 articles (41 separate comparisons), 

and the selection decision was addressed in 26 articles (totaling 33 comparisons); six articles 

evaluated both the treatment and selection decision. Among these studies, 24 comparisons for the 

treatment decision presented data amenable to median effects analysis, and 22 comparisons for 

the selection decision were similarly amenable to quantitative analysis. 

Based on our findings, we reached the following conclusions: 

1. Quality improvement strategies are moderately effective at reducing the inappropriate 

prescribing of antibiotics and improving the appropriate selection of antibiotics. 

Overall, interventions targeting the antibiotic treatment decision were effective at reducing 

prescribing, with a median effect of -8.9% (interquartile range (IQR) -12.4% to -6.7%); 

indicating an absolute reduction in antibiotic prescribing rates of 8.9% in intervention groups 

compared with comparison groups. Similar effects were also apparent in studies not meeting 

criteria for quantitative analysis. We did not find evidence for confounding or effect 

modification by study design or other moderating factors. 

Antimicrobial resistance was measured in only two studies, neither of which demonstrated a 

reduction in resistance despite a reduction in prescribing rates; however, the limited duration of 

followup (6 months) was likely insufficient to detect potential effects on resistance rates. 

Strategies to reduce antibiotic prescribing were not associated with increased use of health 

services, increased duration of illness symptoms, or decreased patient satisfaction. Costs were 

reduced in the two studies measuring this outcome. 

 

1

 Active clinician education strategies included academic detailing (educational outreach), consensus-building sessions, and 

educational workshops. Active patient education strategies included one-on-one or group educational meetings. Passive 

educational strategies (for both clinicians and patients) included distribution of educational materials (e.g., waiting room 

pamphlets for patients) and lectures (e.g., traditional CME for clinicians). 
3 

Interventions targeting the antibiotic selection decision were also effective, with a median 

absolute improvement of 10.6% (IQR 3.4% to 18.2%) in prescribing of recommended antibiotics 

in the intervention groups compared with the comparison groups. Most studies used either 

clinician education alone or clinician education combined with audit and feedback. Interventions 

targeted both ARIs and urinary tract infections (UTIs), with some interventions targeting general 

prescribing; no significant differences were found for QI strategies targeting different disease 

processes or patient populations. In four studies, duration of antibiotic therapy for urinary tract 

infections was assessed, with effects ranging from no benefit to a reduction in mean antibiotic 

duration of approximately 2 days. 

No studies assessed the effect of interventions targeting antibiotic selection on resistance, 

health services utilization, or disease outcomes. Three studies measuring prescribing costs 

showed 20-30% relative reductions attributable to the intervention. 

We conclude that QI strategies are moderately effective at improving prescribing behavior, 

with regard to both antibiotic treatment and antibiotic selection decisions. Organizations should 

compare studies performed in similar settings, and with similar patient and provider populations, 

in order to identify specific QI strategies that are most likely to be effective in their own setting. 

Several particularly salient studies (representative of the types of interventions and settings 

encountered in our review) are summarized in Appendix A*

, to provide a starting point for 

considering important implementation factors. Appendix B contains summaries of all included 

studies organized by setting and patient population in order to facilitate identification of relevant 

studies. 

2. Although no single quality improvement strategy is clearly superior, active clinician 

education may be more effective in certain settings. 

Studies predominantly used clinician education strategies or clinician education in 

combination with patient education; smaller numbers of studies used audit and feedback, or other 

combinations of strategies. We did not find definitive evidence for superiority of one strategy 

(or combination of strategies) over another, within either the antibiotic treatment studies or 

antibiotic selection studies. Delayed prescribing interventions achieved large absolute reductions 

in ultimate antibiotic consumption, but these studies operated under the assumption that the 

“default” practice was to uniformly administer antibiotics (even when they might not be 

indicated, as in acute cough illness). In the treatment studies, the trend suggested that active 

educational strategies were more effective (p=0.11), an effect also seen in studies not eligible for 

median effects analysis. Among antibiotic selection studies, the addition of audit and feedback 

conferred significantly less benefit than clinician education alone (3.4% vs. 13.9%, P=0.03). 

This finding may in part be explained by confounding: clinician education-alone studies were 

more likely to have a small sample size, and smaller studies were associated with larger median 

effects. This effect may be mediated through publication bias, a higher level of engagement 

between the study directors and target physicians and/or greater intensity of the intervention. In 

contrast to interventions aimed at the treatment decision, interventions using active educational 

strategies to improve antibiotic selection were not significantly associated with larger median 

effects. However, in each of the five studies where active and passive educational strategies 

were compared head-to-head, the active strategies were superior. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
4 

3. Interventions targeting prescribing for all acute respiratory tract infections may exert a 

greater effect on overall prescribing than interventions targeting specific types of acute 

respiratory infections. 

Most treatment studies targeted prescribing for ARIs; there were no significant differences in 

effectiveness for interventions targeting specific ARIs vs. general ARIs, or for interventions 

targeting children vs. adults. Among antibiotic selection studies, there was no difference in 

median effects among interventions targeting general ARIs, specific ARIs, or UTIs. In an effort 

to maximize the relevance of interventions targeting the antibiotic treatment decision at the 

health system level, we conducted a separate analysis extrapolating study effect sizes to the 

population level, determining the number of antibiotics per 1000 person-years that could be 

saved as a result of the intervention. Interventions targeting prescribing for all ARIs have more 

impressive effects when their impact is extrapolated to the population level (35-85 antibiotic 

prescriptions saved per 1000 person-years), even though the individual study effect sizes are 

more modest. In contrast, condition-specific interventions (e.g., those targeting prescribing for 

pharyngitis in children), reporting large effects within the study population, have smaller effects 

on prescribing at the population level (e.g., 5-15 antibiotics saved per 1000 person-years). The 

greater population effects of strategies targeting prescribing for all ARIs (as opposed to focusing 

on a single condition or patient age range) could result in relatively higher cost savings as well, 

depending on the nature and intensity of the intervention. 

4. Study design and quality should be improved. Studies that formally evaluate the cost 

effectiveness of interventions to improve antibiotic treatment and selection are needed, and 

studies should evaluate the potential harms of such interventions. 

As noted in previous reviews of diabetes and hypertension QI strategies in this series,1, 2 the 

methodological quality of included trials was only fair. In both treatment and selection studies, 

approximately half the trials were non-randomized; most failed to document the rationale for 

selection of the comparison group. Other basic quality problems such as inadequate concealment 

of allocation and unit-of-analysis errors appeared frequently. Moreover, studies consistently 

failed to describe the theoretical basis for their intervention or the rationale for choice of the QI 

strategies used, and did not document the reach of the intervention (i.e., the extent to which the 

target population actually received the intervention). These differences may affect the internal 

and external validity of the studies. In addition, it is also likely that our results were confounded 

by substantial inter-study variations in patient population, disease target, and clinical settings, as 

well as many unmeasured confounding variables such as local factors, the culture of medical 

practice, and the health care system structure. 

Future studies should aim for higher methodological standards, and should maximize the 

applicability of their interventions by clearly defining the characteristics of the intervention, the 

setting, and the participants. Few studies evaluated potential harms to patients that could result 

from reducing antibiotic use, such as adverse clinical consequences or increased use of health 

services. In addition, our conclusions are limited to fairly short term intervention effects. Most 

importantly, very few studies reported any information on the implementation costs, and no 

studies performed a formal cost effectiveness analysis. Documentation that QI strategies recover 

their implementation costs through savings in antibiotic costs would greatly enhance the appeal 

of such programs to health systems. 
Technical Review

7 

Chapter 1. Introduction 

Rationale for Reducing Excess Antibiotic Use 

in Ambulatory Practice 

The introduction of antibiotic therapy into medical practice has yielded substantial benefits 

for patients over the past 6 decades. Recently, however, there has been a growing awareness that 

the benefits of antibiotic use to individual patients come at a societal cost: the emergence of 

antimicrobial resistance (AMR) among bacterial pathogens.3, 4 Though initially restricted to 

hospital settings, antibiotic resistance is now growing among important community-acquired 

bacterial pathogens, particularly Streptococcus pneumoniae (SP),5-7 Staphylococcus aureus,8, 9

and Escherichia coli.

10-12 

Although the problem of antibiotic resistance can be viewed from many perspectives, a 

pragmatic viewpoint approaches it as a problem of antibiotic prescribing practices. Antibiotic 

use promotes the development and spread of antibiotic-resistant bacteria by at least two 

mechanisms: (1) selective pressure that enhances the proliferation and sustainability of bacterial 

strains that contain mutations conferring resistance to a specific antibiotic (or class of 

antibiotics), and (2) elimination of an individual's normal flora, thus conferring a survival 

advantage to antibiotic-resistant bacteria should exposure to antibiotics occur (i.e., creating a 

susceptible host). While development of antibiotic resistance primarily occurs via the first 

mechanism, the rapid rise in prevalence of antibiotic-resistant bacteria is primarily a function of 

the second mechanism. Carriage, transmission, and infection with antibiotic-resistant bacteria 

are strongly associated with prior antibiotic use.13-16 

Community-acquired antibiotic-resistant infections affect morbidity, mortality, and health 

care costs. Treatment failures have been reported in patients with pneumococcal meningitis after 

treatment with penicillin, chloramphenicol, clarithromycin, ceftriaxone, and cefotaxime.17-22 The 

management of children with recurrent or persistent otitis media or sinusitis has become 

significantly more difficult as a result of the emergence of high rates of drug-resistant 

Streptococcus pneumoniae (DRSP) in this subgroup.23-25 One US study reported an association 

between high-level DRSP and excess mortality among patients surviving more than 4 days in the 

hospital, after controlling for age and comorbidity,26 and another demonstrated an increased risk 

of suppurative complications for patients with bacteremic pneumonia due to DRSP.27 

The emergence of drug-resistant bacteria has implications not only for patients with 

documented resistance, but for all patients who might have such infections. Currently, the 

specter of DRSP and other resistant bacteria has led to recommended empiric treatment regimens 

for a variety of infections (such as community-acquired pneumonia28, 29) that include antibiotics 

previously reserved for life-threatening infections; antibiotic resistance to these powerful agents-

-often our last lines of defense--will certainly accelerate as a result. Only 6 years after the 

introduction of extended-spectrum macrolides, macrolide resistance among S. pneumoniae

increased from 10% to 20%.30 In Canada, where ambulatory use of levofloxacin (the agent of 

choice for treatment of DRSP infections) has been very high, investigators have already detected 

the emergence of levofloxacin resistance.7

 Previous optimistic predictions about “the end of the 

era of infectious diseases” have increasingly been replaced by dire predictions of a new “postantibiotic era.”3, 31, 32 
8 

Reducing inappropriate use of antibiotics is a critical step in preventing or slowing the 

progression of resistance. Progress will require a two-part approach. First, it is critical to reduce 

the use of antibiotics in conditions for which these drugs provide little or no benefit. Second, it 

is important to modify the choice of drug given to patients who do require antimicrobial therapy, 

and in some cases to shorten inappropriately lengthy treatment courses. Based on the best 

evidence and mathematical modeling to date, reductions in antibiotic consumption may not lead 

to major reductions in existing levels of antibiotic resistance among community-acquired 

bacterial infections,33-35 although studies have demonstrated reduced levels of resistance to 

specific antibiotics.36 However, ecological studies do support the notion that the amount of 

antibiotic consumption in a community directly influences how rapidly new resistance emerges 

or rises.7, 37 

Current Antibiotic Prescribing Practice Patterns in US 

Ambulatory Practices 

In the US, the majority of outpatient antibiotic prescriptions are for acute respiratory tract 

infections (ARIs)2

; in 1998, an estimated 76 million ambulatory office visits for ARIs resulted in 

41 million antibiotic prescriptions.38 Based on a comparison of bacterial prevalence estimates to 

prescribing rates, it appears that 55% of total antibiotics prescribed for ARIs in 1998 (n=22.6 

million prescriptions) were unlikely to be treating a bacterial infection, resulting in a total cost 

for excess antibiotic prescriptions of approximately $726 million. 

In response to efforts to publicize overuse of antibiotics,39, 40 initial progress to reduce 

inappropriate antibiotic prescribing was made. Between 1990 and 2000, antibiotic prescription 

rates for adults and children with ARIs declined significantly. Unfortunately, prescribing rates 

stabilized in 1999-2000.40 

In addition to the general inappropriate use of antibiotics in conditions not requiring 

antimicrobial therapy, increasing attention has been paid to the choice of antibiotic for patients 

receiving such therapy. The use of broad-spectrum antibiotics (quinolones, second- and thirdgeneration cephalosporins, newer-generation macrolides, and amoxicillin-clavulanate) in 

community-based settings rose from 24% of total adult antibiotic prescriptions in 1991-1992 to 

48% 7 years later.41 A similar rise (23% to 40%) was observed in children.

A variety of national and international organizations have recommended action to improve 

antibiotic treatment and selection.3

 In 1996, a task force convened by the Centers for Disease 

Control and Prevention (CDC) outlined strategies for addressing this epidemic, highlighting the 

importance of promoting judicious antibiotic use in ambulatory practice.42 The issue was further 

underscored by a subsequent Institute of Medicine (IOM) report on antimicrobial resistance,43 as 

well as in the recommendations from a US federal multi-agency task force that created the 

"Public Health Action Plan to Combat Antimicrobial Resistance."44 In collaboration with the 

CDC, the National Committee for Quality Assurance and the Council for Affordable Quality 

Healthcare are creating quality measures for appropriate antibiotic use for inclusion in the Health 

 

2

 The terms acute respiratory infection (ARI) and upper respiratory infection (URI) are frequently used interchangeably in the 

literature. For this report, we use ARI to refer to these conditions. 

3

 Key initiatives include the CDC’s “Get Smart” campaign (http://www.cdc.gov/drugresistance/community/), the WHO’s Global 

Strategy for Containment of Antimicrobial Resistance 

(http://www.who.int/csr/resources/publications/drugresist/WHO_CDS_CSR_DRS_2001_2_EN/en/), and the Association for 

Prudent Use of Antimicrobials (http://www.tufts.edu/med/apua/). 
9 

Plan Employer Data and Information Set (HEDIS)45. Other countries have also developed their 

own campaigns, some with striking success.36 Almost 30 countries in Europe have combined 

their efforts to monitor trends in antibiotic use.46 Finally, in the seminal IOM report "Priority 

Areas for National Action: Transforming Health Care Quality,"47 antibiotic overuse (as a major 

part of medication management) was identified as one of 20 priority areas for improving health 

care quality -- a designation reflecting not only the severity of the problem, but also the 

substantial potential to narrow the gap between actual and best practices. 

The Purpose of This Evidence-based Practice 

Center Report 

In this report, we systematically review published research articles evaluating interventions 

to improve antibiotic prescribing practices. Our intent is to identify the most effective strategies 

for improving the use and selection of antibiotics. We focus our review on acute infectious 

illnesses in the outpatient setting, and separately review (1) the antibiotic treatment decision (i.e., 

the use of antibiotics for conditions generally not requiring antibacterial therapy), and (2) the 

antibiotic selection decision (i.e., the choice of one antibiotic over another, for conditions 

requiring antibacterial therapy). In particular, our research questions are: 

1. Which QI strategies are most effective in reducing unnecessary antibiotic treatment of acute 

illnesses? 

A. Are particular QI strategies more effective for certain target conditions? 

B. Are particular QI strategies more effective in specific patient populations? 

C. Do QI strategies to reduce unnecessary antibiotic treatment affect clinical 

 outcomes? 

 1. Do they reduce adverse drug events? 

2. Do they improve clinical outcomes? 

3. Do they increase return visits or illness-related hospitalizations? 

D. Do QI strategies to reduce unnecessary antibiotic treatment have downstream 

 patient- and system-level consequences? 

 1. Do they reduce antimicrobial resistance? 

2. Do they reduce antibiotic/pharmacy costs? 

3. Do they decrease patient satisfaction? 

2. Which QI strategies are most effective in improving antibiotic selection for the treatment of 

acute illnesses? 

A. Are particular QI strategies more effective when the goal is (i) to reduce use of 

 broad-spectrum agents, or (ii) to increase use of broad-spectrum agents (i.e., due 

 to local resistance patterns)? 

B. Are particular QI strategies more effective at improving optimal dosing and 

 duration of a selected antibiotic therapy? 

C. Are particular QI strategies more effective for certain target conditions? 

D. Are specific QI strategies more effective in specific patient populations? 

E. Do QI strategies to improve antibiotic selection affect clinical outcomes? 

 1. Do they reduce adverse drug events? 
10 

2. Do they improve clinical outcomes? 

3. Do they increase return visits or illness-related hospitalizations? 

F. Do QI strategies to improve antibiotic selection have downstream patient- and 

 system-level consequences? 

 1. Do they reduce antimicrobial resistance? 

2. Do they reduce antibiotic/pharmacy costs? 

3. Do they decrease patient satisfaction? 

In order to maximize the relevance of our findings for organizations that are considering 

implementing these interventions, we summarize the existing evidence by attempting to answer 

the following questions: 

1. Are quality improvement strategies to improve outpatient antibiotic use effective? 

2. What are the critical components of effective intervention strategies? 

3. Which patients and conditions should be targeted in order to exert the maximal impact 

 on antibiotic prescribing? 

4. What are the limitations of current research in this field, and what areas require further 

 study? 

An Explanatory Model of Antibiotic Prescribing Behavior

The decision to prescribe antibiotics is the result of complex interactions among patient, 

physician, and system factors, and effective strategies to improve antibiotic prescribing behavior 

may need to address each of these domains. Previous research on the influence of these factors 

on office-based antibiotic prescribing is summarized below. Most of this research has focused 

on the antibiotic treatment decision, although several studies have explored factors that influence 

antibiotic selection. 

Past research has identified the characteristics of successful QI programs across a variety of 

disease states and clinical settings.48-51 While this work provides an important framework for 

examining QI strategies, factors that drive antibiotic prescribing decisions draw on a differently 

prioritized set of inputs from those that impact physician behavior in other clinical settings. 

Specifically, the management of common outpatient infections and the decision to prescribe 

antibiotics, particularly when they are not indicated, can be viewed within a sociocultural and 

social ecological context in addition to the traditional biomedical perspective. The various inputs 

to this decision are presented in the adaptation of Kleinman and colleagues' model52 for clinician 

decisionmaking in Figure 1. 
11 

Figure 1. Factors affecting the decision to prescribe antibiotics. 

Clinician Factors 

Patient and parent expectations are consistently identified as the primary factor influencing 

unnecessary prescribing of antibiotics.53-58 Nonetheless, internal clinician factors such as 

knowledge, experience, and training also play a role, suggesting that intervention strategies 

include efforts to improve clinician education. 

Despite strong evidence that antibiotics are ineffective for acute bronchitis, 50 – 80% of 

adults with bronchitis are prescribed them.38, 59, 60 Practice-based studies repeatedly show that 

there are few or no predictors of antibiotic treatment for acute bronchitis, suggesting that the 

diagnosis of acute bronchitis is often viewed as an indication for antibiotic treatment. Antibiotic 

prescribing for uncomplicated ARIs is strongly associated with the presence of purulent 

manifestations (purulent nasal discharge, purulent phlegm production, and tonsillar exudate).61 

These findings suggest that clinicians appear to use a heuristic (or "rule of thumb") for deciding 

which patients with ARIs to treat with antibiotics, one that pays particular heed to the presence 

of purulence. However, purulence does not predict bacterial infection or antibiotic 

responsiveness among patients with ARIs.62-66

Other studies have found that clinician specialty and level of training are also associated with 

antibiotic use for these conditions. High prescribers of antibiotics for colds, ARIs, and bronchitis 

are more likely to be older,67 and to practice in rural locations.38 In a national study of antibiotic 

treatment of ARIs in emergency departments, antibiotics were prescribed less often by house 

staff than by staff or other physicians, and more often to adults than children, regardless of 

specific ARI diagnosis.68 In several national studies evaluating antibiotic selection, physician 

specialty was one of the strongest independent predictors of antibiotic selection, suggesting 

possible differences in knowledge or the “culture of prescribing.”69-71 

Clinician Factors

-sociodemographics 

-training/specialty 

-knowledge 

-judgment and heuristics 

-perceived patient expectations

Patient Factors

-sociodemographics 

-past experiences 

-expressed expectations 

-reported symptoms 

-illness severity

System Factors

-practice setting 

-health plan features 

-visit and pharmacy co-pay 

-patient enabling systems 

-formularies/restrictions 

-pharmaceutical detailing 

Clinician's Decision to 

Prescribe Antibiotics 
12 

Public/Patient Factors

Multiple studies demonstrate that patients who seek care for ARIs expect to receive 

antibiotics,69-73 and that patients or parents who expect antibiotics receive them more 

frequently.54, 55, 57, 58 These expectations are strongly associated with the patient’s previous 

experiences of receiving antibiotic treatment for these illnesses,74 and appear to arise from 

misperceptions about antibiotic efficacy for viral illnesses.75 Illness labeling may play an 

important role in conveying antibiotic treatment necessity. For example, referring to an acute 

cough illness as a "chest cold" was associated with much lower beliefs that antibiotics are 

necessary treatment than when the same illness was called "bronchitis."76

Although patients and parents frequently expect antibiotics for ARIs, most studies find that 

satisfaction appears more closely related to how much time the doctor spent explaining the 

illness rather than whether the patient received an antibiotic.55, 57, 77, 78 In a setting where 

antibiotic prescribing for acute bronchitis had been reduced by 50%, patient satisfaction with 

care and number of return office visits did not change.77 

Patient sociodemographic factors are also associated with excess antibiotic use for ARIs, and 

may in part be related to varying knowledge, attitudes, and expectations in different population 

groups as well as clinician attitudes about these groups. Antibiotic prescription rates for ARIs 

appear to be lower among blacks than whites,38, 79 which may account for the lower relative risk 

of carriage or infection with antibiotic-resistant bacteria among black populations.14, 38, 79-81 

Similarly, black patients receiving an antibiotic for ARI were less likely to be given a broadspectrum agent than white counterparts.69 Age also plays an important role. The frequency of 

antibiotic treatment for colds, ARIs, and bronchitis is greatest among the very young (age < 5 

years) and lowest among the elderly (age > 64 years).38, 60 The mechanisms by which these 

sociodemographic factors influence clinician decisionmaking are not known, but are probably 

related to varying patient expectations for antibiotics and/or clinician judgments about the ability 

to adequately discuss treatment choices with patients whose backgrounds differ from theirs, or 

who have low health literacy.82 

Health Care Delivery System Factors

Health plan and physician practice characteristics may act as barriers to, or facilitators of 

antibiotic prescribing for ARIs. Health plans can influence their members’ propensity to seek 

care and expectations for care by instituting visit and pharmacy co-payments,83 requiring prior 

authorization for urgent care or same-day clinician visits, and making telephone advice service 

available.84 Lack of insurance also exerts an effect.69 Health plans can influence clinician 

behavior by restricting formularies. Such restrictions influence both prescription antibiotic 

selection and patients’ access to alternatives to antibiotic treatment, including non-prescription 

agents. 69 Practice characteristics such as location, clinician panel size, protocols for mid-level 

clinicians, exposure to pharmaceutical sales representatives, and availability of pharmaceutical 

samples also affect access and treatment decisions. For example, clinicians with greater patient 

workloads prescribe antibiotics for ARIs more frequently.38, 85, 86 Lack of patient continuity and 

inadequate followup care mechanisms may provide additional challenges to antibiotic 

improvement programs based in acute care settings such as urgent care clinics and emergency 

departments. 
13 

Chapter 2. Methods 

Scope 

This report focuses on QI strategies to reduce unnecessary or inappropriate antibiotic 

treatment in the outpatient setting. Changes in outpatient antibiotic use are likely to have the 

greatest impact on efforts to arrest the development and spread of antibiotic-resistant bacteria 

that cause community-acquired infections. We restricted our focus to antibiotic treatment of 

acute illnesses, excluding studies of prophylactic antibiotic use and antibiotic therapy for chronic 

illnesses, both to minimize the conceptual heterogeneity of our approach and because chronic 

and prophylactic antibiotic use account for a small fraction of total outpatient antibiotic use. 

As noted earlier, we reviewed strategies to affect antibiotic use in two realms: the antibiotic 

treatment decision, and the antibiotic selection decision. Studies in the “treatment decision" 

group addressed the dichotomous decision of whether or not to treat with antibiotics at all. These 

studies evaluated QI strategies to reduce the prescribing of unwarranted antibiotics, either overall 

or for a specific condition or patient population. Studies in the “antibiotic selection” group 

evaluated QI strategies to increase adherence to recommended choices of antibiotics in situations 

where antibiotic treatment is warranted. 

Definitions of Quality Improvement Terms Used 

in This Report 

For the purposes of this report, we used quality improvement terminology in accordance with 

the Institute of Medicine report described in previous volumes of the Closing the Quality Gap

series1 as follows: 

• Quality gap: the difference between health care processes or outcomes observed in 

practice, and those potentially achievable on the basis of current professional knowledge. 

The difference must be attributable in whole or in part to a deficiency that could be 

addressed by the health care system. 

• Quality improvement strategy: any intervention strategy aimed at reducing the quality 

gap for a group of patients representative of those seen in routine practice. 

• Quality improvement target: the outcome, process or structure that the QI strategy 

targets, with the goal of reducing the quality gap. For this report, the main QI target was 

clinician prescribing behavior, either in terms of the decision to prescribe an antibiotic or 

the selection of an antibiotic. 

Classification of Interventions 

A single study may include different study arms (groups of subjects), and each arm may 

receive a different QI intervention. For the purposes of this report, each intervention type was 

abstracted separately and evaluated as a separate comparison. For example, a single study could 
14 

contain three arms: a control or comparison group (depending on whether the study was an RCT 

or not), a group receiving clinician and patient education, and a group receiving clinician 

education only. We would consider such a study to contain two trials (two separate comparisons 

against the comparison group). 

The intervention(s) used in a study sometimes included more than one QI strategy. Each 

type of QI strategy was abstracted separately, although considered part of the same intervention. 

Interventions containing two or more different QI strategies (as defined by the categorization 

listed below) were considered multifaceted interventions. For example, an intervention using (a) 

audit and feedback and (b) clinician education was defined as a multifaceted intervention, using 

two QI strategies. Finally, a study could contribute to both "treatment" and "selection" groups if 

the intervention clearly attempted to influence both the decision to prescribe and the selection of 

antibiotic, and reported these outcomes appropriately. 

Classification of Quality Improvement Strategies 

We classified QI strategies targeted at improving antibiotic prescribing in the following 

manner: 

QUALITY IMPROVEMENT STRATEGIES4

Clinician education 

Interventions designed to increase understanding of clinical care principles, or awareness of 

specific practice recommendations. Specific subtypes of clinician education included passive 

strategies, where clinicians are not actively engaged in the learning process (e.g., distribution of 

educational materials, lectures, and meetings such as traditional CME); and active strategies, 

where clinicians interact and actively participate in their education in one-on-one or small 

group settings (e.g., workshops, consensus-building sessions, and educational outreach visits). 

In active strategies, the teacher can actively assess comprehension, and tailor information 

appropriately; whereas, in passive strategies there is little or no real-time monitoring of whether 

the information was received and understood by the learner. 

Patient education

Interventions designed to promote increased understanding of target conditions or to teach 

specific prevention or treatment strategies. As with clinician education, patient education can 

also be subdivided into passive strategies (distribution of educational materials) and active 

strategies (one-on-one or group educational workshops). Mass media efforts (e.g., television 

announcements or billboards) are a type of passive education usually targeted at patients, but 

potentially exerting influence on clinicians as well. 

 

4

 Other commonly used QI strategies include patient reminders (efforts to improve adherence to self-care or keeping 

appointments), facilitated relay of clinical data to providers (provision of data outside of normal chart-based means), promotion 

of patient self-management (distribution of materials or access to a resource that enhances self-care), and organizational change 

(changes in the structure or delivery of care to improve efficiency or breadth of care). These strategies are commonly applied in 

chronic disease management, but are less applicable to an acute illness model, and were only featured in one included study.87 
15 

Provision of delayed prescriptions 

In this strategy, patients are given an antibiotic prescription, but instructed to fill the 

prescription only if their condition does not improve with supportive measures after a specified 

period of time. We considered delayed prescriptions to be a unique category. This strategy 

does not reduce antibiotic prescribing by the clinician per se, and it assumes that the “default” 

practice is to uniformly administer antibiotics (even when they might not be indicated, as in 

acute cough illness). Delayed prescriptions incorporate elements of patient self-management 

and patient incentives (e.g., guarantee of a “back-up” medication—antibiotics) to attempt to 

reduce actual antibiotic use.

Audit and feedback 

Summary of clinical performance of health care delivery over a specified period of time, 

provided to the target clinicians by an individual or organization (e.g., the proportion of a 

clinician’s patients with acute bronchitis for whom the clinician prescribed an antibiotic). 

Profiling, in which the performance of an individual, clinic or health system is compared with 

peers, and benchmarking, when the reference point is some established quality standard, would 

also be considered audit and feedback.

Clinician reminders and decision support systems 

A paper-based or electronic system, intended to help a clinician incorporate specific 

information into a clinical decision. Decision support systems combine reminders with a 

prompt to follow a recommended pathway of clinical care. 

Financial and regulatory incentives or disincentives 

Inducements or disincentives that influence individuals to take a desired course of action. We 

considered incentives for patients and clinicians separately. Common types of financial 

incentives to patients to reduce unnecessary health care services include office-visit and 

prescription drug co-payments; capitated insurance plans are an example of financial incentives 

to clinicians to promote efficient care. A common regulatory disincentive targeted at clinicians 

is the requirement to submit extra documentation to receive approval to prescribe a restricted 

drug.

Inclusion and Exclusion Criteria

The inclusion criteria were adapted from previous volumes of the Closing the Quality Gap

series.1, 2 Included studies were required to: 

• Evaluate an intervention incorporating one of the QI strategies defined above to improve 

the quality of outpatient antibiotic prescribing for acute illnesses in the outpatient setting 

(clinic, urgent care, or emergency department). 

• Use either (1) an experimental design with a comparison group assigned by the 

investigators, including patient- or cluster-randomized controlled trials (RCT), quasi-
16 

RCT5

 design, and controlled before-after (CBA)6 studies; or (2) when a comparison 

group was not employed, use of an interrupted time series (ITS) with a clearly defined 

intervention time period, and at least three measurements before and after the 

intervention. 

• Report at least one measure of antimicrobial agent use. 

Trials that reported outcomes related to antimicrobial use, such as antimicrobial resistance 

rates, prescribing costs, health services utilization, or satisfaction with care, were included only if 

they also measured antimicrobial prescribing. For example, a study of antimicrobial costs to a 

health care system would be excluded if costs were the only outcome measured; however, if the 

study also measured antimicrobial prescribing (and met the other methodologic criteria above) 

the study would be included. We also included studies that assessed the effect of QI strategies 

on the dose or duration of antimicrobial therapy, where these outcomes were targets of the 

intervention. 

We anticipated that some interventions would give rise to multiple publications. In these 

situations, we regarded the first publication as the primary study, but included additional 

information on the intervention or results from the other publications as warranted. In cases 

where results of an intervention were described at multiple time points in multiple studies, we 

abstracted data from the short-term followup article, judging that longer-term followup might 

address the sustainability of an intervention rather than its effectiveness, and in an attempt to 

maintain consistency of intervention time periods to maximize our ability to pool results across 

studies. 

Literature Search and Review Process

To identify studies for potential inclusion, we searched the electronic database of the 

Cochrane Registry Effective Practice and Organisation of Care (EPOC) group (Appendix C*

). 

The EPOC database catalogs studies that attempt to “improve professional practice and the 

delivery of effective health care services.” The database includes the results of extensive periodic 

searches of MEDLINE® (from 1966-present), EMBASE (1980-present), and CINAHL® (1982-

present), as well as hand searches of specific journals and article bibliographies.88 The strategy 

for identification of appropriate studies has a sensitivity of 92.4%.88 The main EPOC registry 

primarily includes studies of clinician and system-targeted interventions, identified from a larger 

database of quality-improvement related studies compiled by Cochrane search experts. To 

maximize our search yield, we searched the main registry as well as the larger database of QI 

articles identified for possible inclusion but not meeting criteria for entry in the main registry. 

We searched for studies relating to antimicrobial prescribing, using terms specific to 

antimicrobials and prescribing practices as well as specific target conditions (e.g., pharyngitis, 

urinary tract infection) (Appendix C). We searched the EPOC registry through October 2004. To 

 

5

 “Quasi-RCT” refers to studies described as randomized where randomization was altered by investigators, or 

where a method of patient assignment was used that is not truly random (e.g. assignment by even/odd medical 

record number or date of clinic visit.) 

6

 Controlled before-after studies are non-randomized trials with a contemporaneous comparison group. *

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
17 

identify more recent studies, we performed a targeted search of MEDLINE® from June 2004 to 

November 2004 using similar search terms (Appendix C). In addition, we hand-searched the 

reference lists of each article that met all inclusion criteria. We included only English-language 

studies. 

A trained research assistant and a physician-investigator (SRR) screened titles and abstracts 

of the retrieved citations for relevance. If reviewers could not make a decision based on the title 

and abstract alone, the article automatically moved to full-text review. All full-text reviews were 

performed by a trained research assistant and at least one core investigator (SRR, MAS, RG). In 

that stage, reviewers abstracted detailed information on study characteristics, study design, 

measures of study quality, and outcomes, which were recorded on standardized abstraction forms 

(Appendix D*

). Disagreements on the abstracted fields were resolved by consensus, occasionally 

involving discussion with another core investigator. 

Outcome Measures 

For studies evaluating the antibiotic treatment decision, the principal outcome measure was 

the proportion of office visits in which a clinician prescribed an antibiotic. For studies 

evaluating the antibiotic selection decision, the primary outcome measures were proportion or 

volume of total antibiotic prescriptions written for a recommended antibiotic, or written in 

compliance with a specific guideline. Other forms of antibiotic utilization outcomes (e.g., 

antimicrobial prescriptions per person-year, antimicrobial prescriptions per clinician-year) were 

abstracted as well, including studies that did not report data in the form of our primary outcome 

measures. 

Secondary outcome measures (in articles that also reported a measurement of antimicrobial 

prescribing) included clinical and health system effects of antibiotic prescribing, including 

intervention effects on adverse drug events, clinical outcomes, return visits or illness-related 

hospitalizations, antimicrobial resistance, costs of prescribing, patient satisfaction, and duration 

of office visits.

Assessment of Study Quality 

In order to assess the overall quality of the literature, we assessed studies based on key 

elements that increase internal validity and translatability. However, once studies met our 

inclusion criteria, we did not exclude any on the basis of study quality, nor did we weight 

statistical analyses by study quality, as this process may unduly influence the results of metaanalysis.89 Study factors that influence internal validity were chosen based on the methodology 

of the Cochrane Collaboration, and factors influencing translatability were chosen based on prior 

literature in the field. They were: 

• Factors affecting study internal validity 

• Method of treatment assignment 

o Were study subjects randomized, and if so, was the randomization process 

described? 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
18 

o For non-randomized studies, was the rationale for selection of the 

comparison group explained, and a baseline observation period included 

(to assess selection bias)? 

• Blinding 

o Were the outcome assessors blinded to treatment group assignment? 

• Statistical analyses 

o Was a unit-of-analysis error present? If so, were appropriate statistical 

methods used for correction? 

• Factors affecting study translatability 

• Study design 

o Did the study document a quality gap in antibiotic prescribing in the study 

population? 

o Did the study address prescribing for a specific condition (as opposed to 

prescribing in general)? 

o Did the comparison group receive usual care (no intervention) or a lowintensity intervention? 

o Did the study measure prescribing by chart review, or through use of 

administrative claims data? (Chart review is a more accurate reflection of 

the clinician's prescribing patterns, since administrative claims data 

captures only when a patient fills a prescription. Prior studies suggest that 

about 10 – 15% of antibiotic prescriptions identified on chart review are 

missed on searches of administrative claims.90)

An Explanatory Model for Evaluating the Different QI 

Strategies To Improve Antibiotic Prescribing Behavior 

A comparison of effects for QI strategies with different targets must also take into account 

the possibility that contextual factors, such as intervention delivery, setting, and population 

factors, could confound (or be responsible for) the observed associations between QI strategies 

and behavior change. We have constructed the following model for guiding evaluation of QI 

strategies in light of these possible confounding and moderating factors (Figure 2). Some of 

these factors may bias the results of an individual study—for example, sample size or the type of 

intervention delivered to the comparison group. They may also act as effect modifiers or 

confounders of the observed association between a QI strategy and median effect size. We also 

examined certain intervention and population factors, which are not typically considered 

“confounders” at the individual study level (e.g., study design), but which could influence the 

overall association between QI strategy and median effect size at the summary level. 
19 

Figure 2. Evaluation of QI strategies for confounders and moderators. 

Statistical Analysis 

We expected that the identified studies would exhibit significant heterogeneity, due to 

variations in study populations (e.g., by age or condition), methodologic features, and 

characteristics of the interventions (e.g., intensity) or the context in which they were delivered 

(e.g., magnitude of quality problem, alignment with attitudes of patients and clinicians). Our 

conventional random-effects meta-analysis confirms the existence of marked heterogeneity, 

which persisted even after stratifying studies by various design and intervention features 

(Appendix E*

). 

Therefore, as our primary approach to the analysis, we chose a framework based on the 

median effect size, comparing groups of studies that shared specific features of interest. This 

approach was first developed in a large review of strategies to foster the implementation of 

clinical practice guidelines91 and subsequently applied to the reviews of QI strategies for diabetes 

and hypertension care in previous volumes in this series.1, 2

To calculate the median effect size, we calculated the net effect size for each study reporting 

dichotomous outcomes by subtracting the difference between post-intervention and pre-

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 

Moderating Factors

-Quality Gap 

 -baseline Rx rate 

-Intervention Intensity 

 -single vs. repetitive 

-Method of Delivery 

 -active vs. passive 

-Social/Cultural Norms 

 -US vs. non-US 

-Control Group 

Intervention 

 -low-intensity vs. none 

QI Targets

Clinician 

System 

 Disease 

Behavior 

Change 

(Effect size) 

QI Improvement 

Strategy

Study Biases

-Publication 

Bias 

-Study Design 

-Sample Size 
20 

intervention rates in the comparison group from the difference between post-intervention and 

pre-intervention rates in the intervention group. Thus, in the treatment group, negative effect 

sizes indicate a reduction in antibiotic prescribing (lower rate of prescribing in the intervention 

group, post-intervention); in the selection group, positive effect sizes indicate an increase in 

prescribing of a recommended antibiotic in the intervention arm compared with the comparison 

arm. 

The median effect for a group of studies is simply the median of the individual effect sizes of 

individual studies. Using the median effect approach allows preservation of the natural study 

units, and allows comparisons across different methodologic features or effect modifiers (e.g., 

baseline adherence). 

In order to minimize multiple comparisons and spurious associations, we followed a 

structured approach to the evaluation of each research question based on our model above. In 

the first step, we assessed whether the crude median effect size varied between different QI 

strategies using non-parametric rank-sum tests (Wilcoxon or Kruskal-Wallis, as appropriate). 

 We next sought to determine if other study characteristics could potentially act as effect 

modifiers or confounders of the observed relationship between QI strategy and median effect 

size. To do this, we established two criteria that had to be met for a variable to be considered a 

confounder or effect modifier: (1) the distribution of QI studies differed across strata of the 

study characteristic, and (2) the median effect size of studies varied across the study 

characteristic. For the first step, we analyzed contingency tables comparing the distribution of 

the number of studies for each QI strategy across dichotomized levels of each study factor (study 

factors were dichotomized due to the limited number of studies in our sample). The variables 

analyzed in this fashion included those we used to assess study translatability, along with other 

intervention characteristics and study population factors. For this purpose, each group of studies 

(treatment and selection), was examined for the following factors: country of study (US vs. 

other), effective sample size (above vs. below median sample size), baseline prescription or 

compliance rate (above vs. below median prescribing rate), intervention characteristics (repeated 

vs. one-time intervention, multifaceted vs. single QI strategy, use of passive vs. active QI 

strategies), type of comparison group (no intervention vs. low-intensity intervention), and target 

populations (specific disease target vs. no disease target; children targeted vs. children not 

targeted). To preserve power, we only included QI strategies represented by more than three 

interventions. A p-value of < 0.20 (by Fisher’s exact test) was used as the threshold to consider 

the variable a potential effect modifier or confounder. We chose this liberal threshold to reduce 

the chance of missing potential confounders simply due to relatively small numbers of studies. 

When a factor was identified as a potential effect modifier or confounder through its association 

with the QI strategy, for the second step we then tested whether the factor was associated with 

the outcome (median effect size) using the non-parametric rank-sum tests noted above. 

In addition to this analysis, we also performed stratified analyses to assess for differences in 

median effect sizes based on presence or absence of our prespecified quality criteria for internal 

validity, as these factors could bias individual study results. All statistical analyses were 

performed using STATA version 8.2 statistical software (Stata Corp., College Station, TX). 
21 

Calculation of Effective Sample Sizes 

Cluster randomized trials, in which treatment allocation occurs at the level of clinicians or 

groups (e.g., randomization by clinic), are an increasingly encountered form of evaluation in QI 

research.92 Allocating treatment in this manner is sometimes done out of convenience, but 

sometimes reflects the investigator’s desire to avoid contamination. If treatment were assigned 

randomly at the patient level, clinicians would have some patients in each group and might 

therefore change their behavior for control and intervention patients.93, 94 In cluster-randomized 

trials, some of the care that patients within each group receive is similar, and thus the two groups 

cannot be considered truly independent. Correction for this non-independence results in a 

smaller effective patient sample size for each study.95-99 The effective sample size N is equal to: 

N Effective = (k*m) / (1 + (m-1)*ICC) where ‘k’ represents the number of clusters, ‘m’ denotes the 

number of observations per cluster, and ‘ICC’ equals the intra-cluster correlation coefficient. 

We used ICCs calculated from an existing database of prescriptions and clinician characteristics 

collected as part of the Minimizing Antibiotic Resistance in Colorado Project (AHRQ R01 

HS13001). This database consists of all office visits and antibiotic prescription claims related to 

ARIs in Denver and Colorado Springs Metropolitan Statistical Areas from four large managed 

care organizations. This analysis showed an ICC of 0.055 for clustering at the clinician level, 

and 0.033 for clustering at the clinic level. The effective sample sizes calculated in this fashion 

were used for stratified analyses, as described above. These numbers fall within the range of 

values for ICCs found in quality improvement trials in primary care settings.98

Calculation of the Population Effect Size 

Because certain patient populations and conditions are much more common than others, the 

median effects approach does not identify which interventions would have the greatest effect on 

total antibiotic use at the population level. Thus, we took a payor and public health perspective 

in standardizing changes in antibiotic prescriptions to the community level. An additional 

advantage of this approach is that it also allows inclusion of studies that were not eligible for the 

median effects analysis due to differences in how outcomes were measured. In order to 

standardize the observed median effects at the community level, we used data from the 2002 

National Ambulatory Medical Care Survey (NAMCS)100 (Figure 3) to extrapolate the relative 

reductions in antibiotic prescribing for specific conditions and age groups to absolute reductions 

in antibiotic prescribing in the general population, expressing results across studies as antibiotic 

prescriptions saved per 1000 person-years. This model assumes that the observed effect in each 

study lasted for 1 year. Using US Census estimates and NAMCS for 2002, we estimated that the 

US population of 283.1 million persons yielded 83.17 million ARI office visits in 2002. When 

interventions did not target a specific condition or age group, we multiplied total number of ARI 

office visits by the absolute net change in proportion of visits at which antibiotics were 

prescribed as a result of the intervention, then divided by the US population to estimate 

antibiotics saved per person-year. When studies focused on specific conditions and age groups, 

we used data from NAMCS to estimate the number of condition-specific office visits for the 

relevant age group(s), and divided by 283.1 million to standardize this effect to the general 

population. For studies that expressed results as antibiotic prescriptions per person-year, we 
22 

multiplied the absolute change in prescriptions per person-year by the number of (conditionspecific) office visits to arrive at the population effect. 

To estimate the cost of antibiotic prescriptions, we used the average wholesale price for a 

7-day supply (except for azithromycin, which was 5 days) published by Red Book in January 

2002. Because we did not have data on specific antibiotic prescriptions, we calculated the 

weighted average cost for a single prescription within each antibiotic class based on the 

distribution of specific antibiotic prescriptions within each class derived from the National 

Ambulatory Medical Care Survey in 2002. Based on this methodology, the weighted average 

costs for a single prescription within each antibiotic class in 2002 was the following: 

tetracyclines ($5.24), cephalosporins ($46.94), macrolides ($41.91), penicillins ($28.36), 

quinolones ($52.97), other ($37.18). These costs do not include the cost of dispensing the 

prescription. 

Figure 3. Ambulatory antibiotic prescriptions for various ARIs, by age group. 

Ambulatory Antibiotic Prescriptions (in millions) 

0-4 years 5-14 years 15-44 years 45-64 years > 65 years 

bronchitis 2.10 1.10 3.76 3.10 2.69 

otitis media 7.88 3.45 1.55 0.88 0.33 

pharyngitis 2.90 7.32 4.70 1.26 0.29 

pneumonia 0.62 0.70 0.47 0.79 0.64 

sinusitis 1.41 2.19 6.55 5.17 1.08 

URI not otherwise 

specified 

6.84 4.27 4.98 2.57 1.59 

Total ARIs 21.74 19.03 22.01 13.76 6.63 

Figure 3 Legend 

Calculated number of ambulatory antibiotic prescriptions, stratified by type of acute respiratory infection (ARI) and agegroup. 

Data compiled from 2002 National Ambulatory Care Medical Survey.100

Particularly Salient Studies 

 Although the median effects analysis and systematic review outlined above provides the 

most comprehensive summary of the body of evidence, we recognize the limitations of such 

analyses in providing concrete examples for stakeholders. We have thus identified three 

particularly salient studies that illustrate the following three virtues: first, studies that met a high 

number of our prespecified quality criteria; second, studies that would be easily translatable to 

other settings; third, studies incorporating representative types of quality improvement 

strategies. The final studies chosen for highlighting were identified by discussion among the 

core investigators. These studies are summarized in Appendix A*

. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
23 

Chapter 3. Results 

Our search strategy (Figure 4) yielded a total of 521 citations (509 from the EPOC database 

and 12 additional studies identified from MEDLINE®). Of these, we excluded 374 after title 

and abstract review, the majority of which (N=222) were outside our scope due to a focus on 

inpatient antimicrobial use, antimicrobial use in chronic diseases, or prescribing of medications 

other than antimicrobials. We reviewed the full text of the remaining 147 articles. Many of 

these (N=35) were excluded due to ineligible study design (e.g., single-group or before-after 

studies). Other reasons for exclusion included lack of eligible study outcomes (N=9), 

inappropriate focus (N=13), and unavailable full-text article (N=57

). Our final sample included 

54 articles, six of which reported the effects of an intervention on both the antibiotic treatment 

decision and the antibiotic selection decision (Tables 1A and 1B). 

Studies To Improve the Antibiotic Treatment Decision 

Studies Meeting Inclusion Criteria 

Our final sample included 34 studies reporting a total of 41 trials (i.e., 41 separate 

comparisons against comparison groups). Of these, 24 trials (from 17 studies87, 101-116) reported 

data as the percentage of visits where an antimicrobial was prescribed before and after the 

intervention, and thus were suitable for median effect size analysis. Three other studies117-119

reported this outcome but did not report complete pre- and post-intervention data; these were 

excluded from median effects analyses and instead were summarized qualitatively. 

The other trials included nine studies120-128 that reported a variety of continuous outcomes, 

such as antibiotic prescriptions per person-year120, 122, 123 or antibiotic prescriptions per 

provider121, 123-125; these studies were analyzed separately in a qualitative fashion. Five studies of 

delayed prescribing129-133 were also analyzed separately, as pre-intervention prescribing data 

were not provided. 

Settings, Goals, and Target Population of Studies 

Appendix B* contains summaries of each included study. These tables organize studies by 

the setting and patient population in which the outcome was measured. We chose to group 

studies by the measured population, rather than the targeted population, in order to help 

stakeholders identify the studies whose results are most applicable to their settings. These tables 

also contain the QI strategies used, and specific details of the interventions themselves, as well as 

the individual study results. 

Nearly all of the studies took place in outpatient primary care clinics (N=32), with two 

conducted in an urgent care or walk-in clinic.102, 108 Most studies took place outside the US 

 

7

 These included three studies published only as abstracts on the International Network for Rational Use of Drugs Web site 

(www.inrud.org) and two studies for which the journal could not be located. *

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
24 

Figure 4. Search strategy and article triage. 

Stage 1: title & abstract review by two independent reviewers (at 

least one core investigator)

N = 147

articles 

93 

Exclusions 

R1/R2: 6 R5: 9 

R3: 14 R6: 6 

R4: 37 R7: 21 

Stage 2: full text review by two independent reviewers (at least 

one core investigator) 

Included articles 

N=54 

articles 

(74 comparisons)* 

**Articles answering 

research question 1 

N=28 articles 

 (reporting 35 comparisons) 

**Articles answering 

research question 2 

N= 20 articles 

(reporting 27 comparisons) 

Articles answering both research 

questions 1 and 2 

N=6 

(reporting 12 comparisons) 

521 

Citations 

(509 EPOC articles, 

12 articles from MEDLINE 

search) 

374 

Exclusions 

R1: 86 R3: 222 R6: 4 

R2: 49 R4: 13 
25 

Figure 4 Legend

Reasons for Exclusion 

R1 = Not an evaluation of an intervention 

R2 = Not a QI strategy 

R3 = Ineligible topic (focused on inpatient care, antimicrobial use in chronic disease, not related to antimicrobial use) 

R4 = Ineligible study design 

R5 = No eligible outcomes 

R6 = Foreign language article 

R7 = Other (outcomes data not available for entire population studied, e.g., only presented in subgroups, article not available, 

duplicate article) 

** Research question 1: Which quality improvement strategies reduce prescribing of antibiotics for nonbacterial illnesses? 

** Research question 2: Which quality improvement strategies improve selection of the appropriate antibiotic for treatment of 

bacterial illnesses? 

The core investigators (who were responsible for full-text article abstraction and data analysis) were SRR, MAS, and RG

(N=24), including 11 from Europe,87, 101, 102, 104, 106, 112, 114, 127, 129, 130, 133 three from Australia,114, 

125, 128 two from Canada,118, 119 two from Africa,105, 116 and the remainder from other countries. 

Of the 28 trials which reported the period during which the study was conducted, 22 took place 

in the last decade.87, 101, 102, 105-109, 111-113, 118, 119, 121, 122, 125-128, 131-133 

Most included studies sought to reduce the prescribing of antibiotics for targeted conditions 

in which antibiotics are generally not recommended (Table 1C). Six studies did not identify a 

specific target condition, instead focusing on reducing overall antibiotic prescribing in a broader 

population.115, 116, 120-122, 125 Among the 28 studies with a specific disease focus, nearly all 

(N=26) addressed antibiotic prescribing for acute respiratory infections; two other studies103, 117

addressed prescribing for acute diarrhea. Antibiotic prescribing for children was targeted in 13 

studies102, 110, 113, 115, 117, 119, 122, 123, 126-129, 132; however, no studies specifically addressed 

prescribing for elderly patients or patients of lower socioeconomic status. Five studies107, 111, 117, 

122, 126 selected patients with a specific type of insurance (e.g., managed care or Medicaid 

enrollees). The median baseline prescribing rate was 31.0% in interventions targeting general 

ARIs (N=9 studies supplying data, IQR 21.4% to 37.0%), but, among interventions targeting a 

specific ARI (N=5 studies supplying data), prescribing rate ranged as high as 78% in a study 

examining acute bronchitis111 and 95% in a study examining pediatric otitis media.102 

Intervention Characteristics 

Interventions utilized a variety of combinations of QI strategies targeted at both clinicians 

and patients. Educational strategies were most commonly used, with 27 trials (from 22 studies87, 

102-106, 108-117, 121, 122, 124-126, 128) using some form of clinician education and 18 (from 15 studies87, 

101, 102, 107-111, 113, 122, 126-128, 132, 133) using patient education; 12 trials (from 10 studies87, 102, 108-111, 

113, 122, 126, 128) provided education to both clinicians and patients. Audit and feedback to 

individual clinicians or groups of clinicians was used in 12 trials (from 11 studies101, 106-108, 111, 114, 

117, 121, 124-126), frequently in combination with clinician education (N=10 trials106, 108, 101, 111, 114, 117, 

121, 124-126). Delayed prescribing was used in five studies,129-133 clinician reminders in two,118, 119

with patient self-management and financial and regulatory incentives used in one study each87, 128
26 

(both in combination with other strategies). One87 trial used organizational change. No trials 

used patient reminders or facilitated relay of clinical data. The median number of QI strategies 

used per trial was two (range one to six), with the majority of trials (N=22) using more than one 

strategy (Table 2). 

Within the broad categories of patient and clinician education, interventions incorporated a 

mix of passive and active educational elements. Active educational strategies for clinicians 

included educational outreach (“academic detailing”), workshops, and consensus-building 

sessions; active strategies for patients consisted of one-on-one or group interactive educational 

sessions. More than half of trials (N=17, from 16 studies103-106, 108-110, 112-114, 117, 121, 122, 124, 126, 128) 

using clinician education used at least one active strategy, including educational outreach visits 

(academic detailing) in 10 trials, workshops in eight trials, and consensus-building sessions in 

four trials (some trials used more than one of these strategies.) Most clinician education efforts 

used more than one sub-strategy, such as combining educational outreach visits with distribution 

of written material. In contrast, patient education approaches were nearly exclusively passive 

(N=15 trials87, 101, 102, 107-111, 113, 122, 126-128, 132, 133), using only distribution of written materials. 

Educational efforts most commonly took place in the clinician’s office or another clinical setting, 

but two studies109, 122 used mass media efforts to attempt to educate the entire population of a 

geographic area. 

The proportion of visits at which an antibiotic was prescribed was the primary outcome in 32 

trials (from 25 studies),87, 101-119, 129-133 with the remaining studies reporting the total number of 

antibiotic prescriptions (expressed as either prescriptions per patient or prescriptions per 

clinician). Scant information was provided on secondary outcomes. The safety of the 

intervention was addressed in only six trials (from five studies87, 108, 111, 131, 133) which reported the 

effects of the intervention on the rate of return visits; four trials addressed patient satisfaction,101, 

129-131 and four assessed the clinical outcome of the disease being studied. 112, 129-131 No study 

reported the rate of adverse drug events. Only two studies109, 113 reported the effect of an 

intervention on antimicrobial resistance, and two other studies112, 121 reported information on the 

costs of prescribing. 

Quality of Included Studies 

The methodologic quality of included studies was generally fair. Most trials were 

randomized controlled trials (N=24, from 20 studies) 87, 103, 105, 107, 112, 114, 115, 118-120, 123-126, 128-133; 

five trials (from three studies)101, 110, 116 used a quasi-randomized design, with allocation altered 

by the investigators to assure equivalent demographic characteristics between control and 

intervention groups (e.g., using alternating patients, or even/odd identifying numbers). For 

analytic purposes, these trials were grouped with the remaining 12 trials, which used a 

nonrandomized CBA design. Among these 12 trials, only 5 (from four studies108, 111, 117, 122) 

explicitly explained the criteria used for selecting the comparison group. The comparison groups 

received a low-intensity intervention in seven trials118, 119, 121, 125, 130, 132, 133 and no intervention in 

34 trials. 

Outcome assessors were clearly defined as being blinded to treatment group assignment in 

two trials. The unit of analysis and unit of allocation differed in 22 trials; 14 of these trials (in 11 

studies103, 104, 106-110, 117, 122, 124, 127) failed to statistically correct for this difference. Prescribing 

behavior was measured directly by chart review in 18 trials, with the remainder using pharmacy 

or other administrative databases. 
27 

As has been documented in previous reviews of QI studies, included trials frequently failed 

to describe the theoretical basis for their intervention, and failed to document key intervention 

and study characteristics.51, 134 Most of the trials (N=28) did not document the extent of the 

antibiotic prescribing quality gap in their patient population. Few trials explained the rationale 

for selecting the QI strategies in their intervention, and almost all trials failed to document the 

reach of the intervention (e.g., the percentage of clinicians who actually attended the educational 

session). The median length of followup was 6 months (in the 39 trials reporting this 

information); eight trials had followup of 1 year or greater. 

Effectiveness of Specific QI Strategies and Combinations 

of Strategies 

Table 3 summarizes the median effect size of net changes in antibiotic prescribing rates for 

the 24 trials that provided sufficient information to calculate effect size, with individual study 

effect sizes shown in Figure 5. (The five delayed prescribing trials are summarized separately in 

Table 6.) Overall, the calculated median effect size across all QI interventions was -8.9% 

(IQR -12.4% to -6.7%). The smallest median effect size was seen in the two studies that used 

patient education only (-4.9%, IQR -9.9% to -0.2%), with slightly greater effects seen in studies 

using clinician education alone (-8.1%, IQR -13.7% to -7.0%). No single strategy appeared to be 

more effective than the others (p=0.848 by Kruskal-Wallis test). The two most common 

intervention types were clinician education only (N=9) or clinician education combined with 

patient education (N=6). There was no difference in median effects between these two strategies 

(p=0.478). Twelve studies used multiple QI strategies (six with clinician and patient education, 

two with clinician education and audit and feedback, one with patient education and audit and 

feedback, and three with patient education, clinician education, and audit and feedback). Studies 

using multiple QI strategies were not more effective than those using single strategies (p=0.82 by 

Wilcoxon rank-sum test). 

Independent of QI strategy, we examined whether any of the selected study characteristics 

was associated with effect size. To maximize power, we restricted these analyses to the two 

dominant QI strategies (clinician education vs. clinician education combined with patient 

education). None of these factors appeared to have a strong influence on the overall effect size 

(Table 4). With regard to intervention characteristics, trials using active educational strategies 

appear to have slightly larger median effects (-13.7% compared with -7.0%,) but this relationship 

was not statistically significant (p=0.11 by Wilcoxon rank-sum test for studies with the two 

dominant QI strategies (Table 5); p=0.33 for all studies). Repeated interventions also did not 

exert stronger effects on prescribing behavior. Similarly, non-US studies had greater median 

effects than studies conducted in the US (median effect -8.9% compared with -3.0%), but this 

relationship was also not statistically significant (p=0.15 for studies with the two dominant QI 

strategies). Interventions targeting a specific disease were also not significantly more likely to 

reduce prescribing than interventions targeting general prescribing (p=0.66). We could not 

evaluate for the effect of the type of comparison group, as all 15 trials had a no-intervention 

comparison group. 
28 

Figure 5. Net reduction in antibiotic prescribing rate for individual studies (antibiotic treatment studies). 

-6.5%

-3.0% -3.0%

-29.5%

-9.6%

0.2%

-12.0%

-9.9%

-7.2%

-2.0%

-28.6%

-23.3%

-13.7%

-12.0%

-15.0%

1.5%

-13.9%

-7.3% -6.7%

-9.7%

-12.0% -8.1%

-24.0%

-35.0%

-30.0%

-25.0%

-20.0%

-15.0%

-10.0%

-5.0%

0.0%

5.0%

Agunawela

Belongia

Bexell

Coenen

Flottorp

Gonzales

Gonzales (2)

Gonzales-Ochoa

Gonzales-Ochoa (2)

Gonzales-Ochoa (3)

Hennessy

Hickman

Mainous (1)

Mainous (2)

Mainous (3)

Melander

Meyer

Molstad

Santoso

Santoso (2)

Smabrekke

Welschen

Zwar

Study

Net Change in Prescribing Rate

Figure 5 Legend 

Effect size for each study (N=24) that supplied sufficient information for analysis. Percentages show the net change in the 

percentage of visits at which an antibiotic was prescribed, calculated as: Net ∆ in prescribing = (Post-intervention prescribing rate 

– Pre-intervention prescribing rate)intervention group - (Post-intervention prescribing rate – Pre-intervention prescribing rate)control group. 

We further stratified the studies by study design and sample size to assess if these factors 

correlated with median effect size. No statistically significant relationship was found for either 

variable (p=0.77 and 0.75 respectively). We also hypothesized that trials with higher baseline 

rates of prescribing antibiotics might be more likely to demonstrate significant effects (the “lowhanging fruit” effect); this also was not borne out (p=0.82). 

As we did not observe a difference in median effects among QI strategies, our assessment of 

potential confounding by moderating factors and study biases amounts to evaluating for potential 

negative confounding, which would result in a spurious finding of no difference among groups 

(Table 5). We identified study design (RCT vs. CBA) and study location (US vs. other) as 

meeting the threshold of association (P<0.20) with the dominant QI strategies (clinician 

education vs. clinician and patient education). However, neither of these factors was 

significantly associated with overall median effects (P=0.52 and 0.15, respectively), eliminating 

them as confounders. We did not find evidence that the presence or absence of our prespecified 

quality criteria affected study results, as there was no significant difference in median effects for 

studies with or without any of the internal validity or translatability elements. For example, 

studies where prescribing was measured by chart review showed no difference in median effects 
29 

than studies in which prescribing was measured by other means (median effect -9.6% and -7.3%, 

respectively; p=0.47.); and as noted above, no difference was found in median effects between 

randomized and nonrandomized studies. 

Delayed Prescription Studies 

Five studies used delayed prescriptions, where patients were asked to fill their prescription 

after a specified period of time if their symptoms did not improve, as the primary strategy to 

reduce antibiotic use, with two of these studies132, 133 also incorporating a patient educational 

intervention (Table 6). All trials evaluated ARIs, with two studies enrolling children with otitis 

media,129, 132 and one each evaluating adults with the “common cold,”130 acute cough,131 and 

acute bronchitis.133 These trials enrolled and randomized patients judged to not need antibiotics 

by the treating physician. Notably, these trials were methodologically sound: all trials were 

RCTs (randomized by patient) and had prescribing measured by chart review. Antibiotic use was 

measured by patient report in three studies130, 132, 133; in two, patients in the intervention group 

were required to return to the office to collect their prescription.129, 131 All had sample sizes 

below the median for all treatment studies. 

For these studies, we calculated the post-intervention difference between antibiotic use in the 

intervention group and the control group. The median difference was -46.7% (IQR -55.0% to -

26.7%), indicating a marked reduction in antibiotic use in the intervention groups; however, the 

post-intervention median rate of antibiotic use in the control groups was still 87.0% (IQR 63.0% 

to 98.5%). This raises the possibility that delayed prescribing strategies may appear more 

effective only because of artificially high rates of antibiotic use in the comparison group (since 

the protocols called for near-universal antibiotic treatment in the control group), which under 

non-study conditions might have had a lower antibiotic prescription rate.

Studies Not Eligible for Median Effect Size Analysis 

Twelve studies117-128 met all of our inclusion criteria, but did not report outcomes in a 

fashion compatible with those included in the median effects analysis. Six of these studies120-122, 

125, 126, 128 targeted overall antibiotic prescribing, while five targeted upper respiratory 

infections,118, 119, 123, 124, 127 and one117 acute diarrhea. Four studies122, 123, 126, 127 exclusively 

addressed prescribing of antibiotics to children. The overall methodologic quality of these trials 

was variable. Eight118-120, 123-126, 128 of the 12 studies were RCTs, and all four nonrandomized 

trials clearly explained the rationale for selection of the comparison group. However, only one 

study120 documented blinding of outcome assessors. The baseline quality gap was documented 

in only three studies122, 124, 126, and only two studies118, 119 measured prescribing rates by chart 

review, with the others all using administrative claims data. 

Five of the trials addressing general prescribing used multi-faceted strategies involving 

combinations of patient education, clinician education, and audit and feedback to address 

antibiotic use in large patient and clinician populations. Two of these121, 125 delivered a 

combined clinician education and audit and feedback intervention to all clinicians in a 

geographic area, either via passive mailings125 or through interactive workshops121; neither 

documented a significant reduction in overall antibiotic use. Combinations of patient and 

clinician education were used in two studies. A reduction of 1.13 prescriptions per 100 visits 

was achieved in an Australian study128 that used consensus-building sessions and a clinical 
30 

guideline as well as patient education. Perz et al122 delivered an educational intervention to 

patients and clinicians, with use of educational outreach and mass media efforts, and achieved an 

11% intervention-attributable reduction in antibiotic prescription rates (a decrease of about 0.16 

antibiotic courses per Medicaid managed care child [0-15 years of age] per year). Finkelstein et 

al126 combined active clinician education, audit and feedback, and patient education to achieve 

reductions of about 0.06 antibiotic courses per commercial managed care enrolled child (0-5 

years of age) per year. 

Clinician reminders were used in three studies.118, 119, 123 Christakis et al123 used a computerdelivered reminder for prescribing in pediatric otitis media; there was no significant reduction in 

overall antibiotic use, but the intervention did lead to a significant improvement in prescribing 

for the appropriate duration of therapy. McIsaac et al118, 119 conducted two separate studies using 

a chart-based reminder to use a validated scoring system for antibiotic prescribing in sore throat 

cases, but both failed to significantly reduce prescribing. 

Active clinician education and audit and feedback interventions were used for two conditionspecific studies. One124 achieved a reduction of 6.4 prescriptions per clinician per 6 months for 

ARIs. Another,117 targeting prescribing behavior in acute diarrhea, demonstrated absolute 

reductions of 15.2% for antibiotic prescribing and 20.2% for antiparasitics in the intervention 

group, but comparable comparison group values were not provided. Pediatric otitis media was 

addressed in another study using delayed prescribing127, which achieved a significant reduction 

of 17 prescriptions per clinic per month. Finally, one study120 (the Rand Health Experiment) 

allocated patients to either a free or cost-sharing insurance plan. Antibiotic use decreased by 

0.43 antibiotic courses per insured person (non-elderly families) per year in the cost-sharing 

group. 

Population Effect Size 

In this analysis, we found that interventions targeting all ARIs in adults (Figure 6) translate 

into the largest effects at the general population level, with savings of 35-85 antibiotics per 1000 

person-years. In the median effects analysis, these studies appear to have only modest effect 

sizes, but the larger population to which they apply outweighs the modest effect. Conversely, 

highly effective condition-specific interventions (or interventions targeting children) may not 

project to large reductions in prescribing at the population level. Two studies not included in the 

median effects analysis can also be compared in this manner. At the population level, Perz et 

al122 extrapolate to 23 antibiotic courses saved per 1000 per person-year, and Finkelstein et al126

to five antibiotic courses saved per 1000 per person-year (Figure 6). 
31 

Figure 6. Population-level effect of studies targeting the antibiotic treatment decision. 

-28.6%

-23.3%

-15.0%

-12.0% -12.0%

*

-2.0%

-7.3% -7.2%

-9.9%

-29.5%

0.2%

-9.7%

-24.0%

-6.5% -3.0%

-3.0%

-12.0% * -9.6%

1.5%

-5

5

15

25

35

45

55

65

75

85

95

Smabrekke

Flottorp

Coenen

Gonzales

(2)

Gonzales

(1)

HickmanBelongia

Finkelstein

GonzalesOchoa (3)GonzalesOchoa (2)GonzalesOchoa (1)

Mainous

(1)

Mainous

(2)

Mainous

(3)

Melander

Perz

Hennessy

Welschen

Zwar

Molstad

Meyer

Otitis

Media

Children

Sore

throat

Age >

3

Acute

cough

in

adults

Acute bronchitis in

adults

ARI in children ARI in adults

Target Disease and Patient Population

Antibiotics Saved per 1000 Patient-Years

Figure 6 Legend 

Bars show the potential antibiotic prescriptions saved per 1000 person-years of observation for each individual study. Studies are 

grouped by target condition and target population. These results were extrapolated from individual study effect sizes based on

visit rates from the 2002 NAMCS survey and census data. Percentages show the individual study effect size, expressed as net 

change in the percentage of visits at which an antibiotic was prescribed. 

*Studies by Perz et al and Finkelstein et al reported continuous data (antibiotics prescribed per person-year) so effect sizes are not 

provided. 

For a 100,000-member health maintenance organization (HMO), these studies suggest that a 

QI strategy targeting all ARIs for patients of all age groups would result in a savings of 

approximately 3000 to 8000 antibiotic prescriptions per year. Figure 7 shows the range of 

antibiotic cost savings that could be obtained from reducing antibiotic prescribing across a range 

of population-level reductions in prescribing and antibiotic cost estimates. These cost estimates 

do not take into account the implementation cost of the intervention itself. Assuming average 

costs of $30 - $50 per antibiotic prescription, the organization would save between $90,000 and 

$400,000 in antibiotic costs (Figure 7). 
32 

Figure 7. Projected cost savings from antibiotic treatment interventions. 

Figure 7 Legend 

This figure represents the potential savings in antibiotic costs that could be obtained through reductions in antibiotic prescribing, 

for a 100,000 member Health Maintenance Organization (HMO). The reduction in antibiotic prescribing is expressed as number 

of antibiotic prescriptions saved per 1000 person-years of observation. Costs are expressed as thousands of dollars saved.

Within-Study Comparisons 

 In five studies, investigators examined more than one type of intervention. These five 

studies103, 107, 110, 111, 115 evaluated a total of nine comparisons of intervention and comparison 

groups. In two studies103, 115 the interventions consisted of either passive or active clinician 

education; in both cases, the active education intervention resulted in a significantly larger effect 

size. Another study107 evaluated three strategies: passive patient education alone, audit and 

feedback alone, or both strategies. The absolute rate of antibiotic prescribing increased in all 

intervention groups, although the rate of increase was significantly less in the groups receiving 

patient education. Another study evaluated three comparisons110: intervention groups received 

either active clinician education, passive patient education, or a combination of both strategies. 

0

50

100

150

200

250

5 15 25 35 45

Population Effect Size, per 1000 person-years

Antibiotic Savings, $

$30 $40 $50 

Average cost per 

prescription 
33 

The largest absolute reduction in prescribing was found in the combined strategy group, with no 

overall reduction found in the patient education only group. Finally, a study examining acute 

bronchitis in adults111 carried out a “limited intervention” consisting of passively distributed 

educational materials targeted at patients and clinicians, and a “full intervention” with active 

clinician education, audit and feedback, and patient education. The full intervention resulted in a 

significant reduction in prescribing, while the limited intervention had no overall effect. 

Special Populations and Secondary Endpoints 

The median effect size of interventions targeted at children was not significantly different 

from interventions as a whole (median effect size -8.5% for 10 trials targeting children, -8.9% in 

the 14 trials not targeting children). The QI strategies used in these 10 trials were similar to the 

overall distribution, with two trials107, 110 using patient education alone (targeted at parents of 

children), three using patient education combined with clinician education,102, 110, 113 three using 

clinician education alone,103, 110 and one trial each (from the same study107) using audit and 

feedback alone and patient education combined with audit and feedback. Eight117, 122, 123, 126-129, 

132 studies not used in our median effects analyses targeted children, including three studies127, 

129, 132 using delayed prescribing strategies for childhood otitis media; as discussed earlier, all 

showed much lower relative rates of antibiotic use. The other five studies included three 

community-based studies using multifaceted strategies discussed above122, 126, 128, all of which 

significantly reduced antibiotic prescribing; one condition-specific study using reminders that 

failed to reduce prescribing123, and one condition-specific study using a multifaceted information 

which reported only intervention group data.117

Two studies109, 113 reported rates of antimicrobial resistance. Both trials used combined 

patient and clinician educational strategies to target antibiotic use at the community level. 

Although both exerted positive effects on overall antimicrobial use, the percentage of drugresistant organisms (penicillin-resistant S. pneumoniae) did not significantly change in either 

case. The length of followup was 6 months in both studies, which may have been too short to 

detect significant changes. 

Two studies assessed costs of antibiotic prescribing. One study,121 which failed to document 

an overall reduction in antibiotic prescribing at the community level did show a reduction in total 

prescribing costs of 18.1% in the intervention group relative to control. An increase in the use of 

narrow-spectrum antibiotics may have been responsible for this savings. A Belgian study102

using an active clinician educational intervention reduced antibiotic prescribing for acute cough 

and achieved a reduction in prescribing costs of 6.97 euros (approximately $9) per patient. The 

calculated antibiotic cost reductions in both these studies did not take into account the costs of 

the interventions themselves. 

The safety of the intervention was addressed in seven trials (from six studies)87, 108, 111, 112, 131, 

133 that measured post-intervention use of health services. Five of these (from four studies)111, 112, 

131, 133 measured the rate of return office visits, and none found more return visits in the 

intervention group. One study87 found no difference in the need for telephone consultation 

between groups. The only study that found a possible increase in health services utilization108

documented a slight increase in the prescribing of antibiotics at a return visit if the patient did not 

receive antibiotics originally. The rate of return visits was not different, however. Four 

studies,112, 129-131 three of which used delayed prescribing, measured the time to symptom 

resolution through patient interviews or diaries; none found a greater overall duration of patient 
34 

symptoms if patients did not receive antibiotics, although one study129 of delayed prescribing in 

children with otitis media found a greater duration of distress in the delayed group. Four studies 

measured patient satisfaction101, 129-131; fewer patients were “very satisfied” with treatment in one 

study131 of delayed prescribing, but no difference was present in the other studies. Adverse drug 

events were measured in only one study,129 which found significantly less diarrhea in patients not 

receiving antibiotics, but no difference in the incidence of rash. 

Studies To Improve the Antibiotic Selection Decision 

Studies Meeting Inclusion Criteria 

Twenty-six studies102, 104, 106, 112, 114, 121, 135-154 on antibiotic selection, reporting data on 33 

trials, met inclusion criteria for this review (Table 1B). Of these 33 trials, 22 (from 20 

studies)102, 104, 106, 112, 114, 121, 137, 138, 140-145, 147, 149-151, 153, 154 reported outcome data that allowed 

inclusion in the median effects analyses, including 19 that reported data on patient sample size. 

The remaining three trials114, 140, 143 listed information on cluster sizes, from which an 

approximate sample size could be estimated; each of these three sample sizes were above the 

median sample size calculated from the other 19 trials. The 11 trials not suitable for median 

effects analysis were from six studies.135, 136, 139, 146, 148, 152 Four of these studies136, 139, 146, 148 did 

not report complete pre-intervention and post-intervention data, reporting only the percentage 

change. Of the other two, one study reported count data152 and the other continuous data 

(antibiotic prescriptions per provider),135 neither of which could be combined with dichotomous 

data. 

Setting, Goals, and Target Population of Studies

Appendix B* contains summaries of each included study, grouped by measured population. 

The study intervention and results are summarized in these tables as well. 

Most studies (21) were conducted outside of the United States, including 12 from Europe,102, 

104, 106, 112, 121, 136, 141, 146, 147, 151, 153, 154 four from Australia,114, 138, 144, 149 three from Canada,139, 143, 

148 and two from other countries.142, 150 Nearly all (22) were conducted in primary care settings. 

Other intervention sites included emergency rooms (one study), urgent care or walk-in clinics 

(one study), and other settings (two studies)141, 148; no interventions targeted specialists’ offices. 

All but four studies reported the study dates; midpoints ranged from 1975 to late 2000, with 13 

studies occurring over the past decade. 

All studies focused on reducing the use of broad-spectrum or costly antibiotics or improving 

the selection of certain recommended antibiotics over others; no interventions attempted to 

increase the use of broad-spectrum or expensive agents in the face of increased antibiotic 

resistance. Four studies attempted to shorten the duration of prescriptions for acute illnesses, 

including one for which this was the exclusive antibiotic utilization outcome measured. No 

studies attempted to increase the duration of prescriptions. No studies focused on improving 

antibiotic dosing. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
35 

The 26 eligible studies examined a variety of populations and conditions (Table 1D). Several 

studies focused on special patient populations, including elderly patients (two),143, 148 children 

(three)102, 137, 144 (including one137 specifically on children in an HMO), and a Medicaid 

population (one).135 Respiratory tract infections were the most commonly targeted condition (13 

studies),102, 104, 106, 112, 114, 137-141, 144, 150, 153 followed by urinary tract infections (eight),136, 144-147, 149, 

151, 154 and sexually transmitted diseases (one)142; one study examined both ARI and UTI 

prescribing.144 Six studies did not examine a specific condition.121, 135, 141, 143, 148, 152 Among 22 

trials included in the median effects analysis, baseline rates of prescribing of recommended 

antibiotics in the comparison group ranged from 20% to 90% with a median of 57%. 

QI Strategy Characteristics 

Nearly all trials (30, from 23 studies) were targeted exclusively at clinicians, including 18 

trials (from 13 studies104, 106, 112, 135, 137-140, 144, 145, 149, 150, 152) that provided clinician education 

only, nine trials114, 121, 136, 141, 143, 146, 147, 153, 154 that combined clinician education with audit and 

feedback, and two trials (from one study136) that provided both of the above as well as a 

regulatory intervention directed at providers. In addition, one trial provided audit and feedback 

alone.151 The remaining three trials provided both patient and clinician interventions, including a 

combination of clinician and patient education (two trials)102, 142 and a combination of financial 

and regulatory interventions directed at clinician and patients (one trial).148 No trials targeted 

patients exclusively, and none targeted the community. 

The median number of QI strategies per trial was one (Table 7); 12 trials used two QI 

strategies and two trials employed three distinct QI strategies. 

Among the 31 trials involving clinician education, 25 involved distribution of educational 

materials to clinicians, and six used meetings in which physicians were passive participants. In 

addition, at least one “active” educational strategy was employed in 20 trials (from 13 studies112, 

114, 136-140, 142, 144, 146, 147, 149, 152) involving clinician education, including educational outreach (13 

trials from 11 studies112, 114, 136, 138-140, 142, 144, 149, 150, 152), educational workshops (seven trials121, 140, 

142, 146, 147, 153, 154), and consensus-building sessions (three trials104, 137, 146). Among trials providing 

education, approximately two-thirds (20) provided the education at the clinician’s office or 

workplace; three provided off-site education, and the remainder did not clearly specify the site. 

Several different antibiotic utilization outcomes were reported, often in the same study. 

These include the percentage of patients receiving a recommended antibiotic (14 trials) or being 

in compliance with a clinical guideline for prescribing (10 trials), antibiotic dose or prescription 

duration (four trials), and other outcomes (six trials). Outcomes other than antibiotic utilization 

were reported infrequently, including antibiotic cost data in four studies121, 140, 143, 144 and 

clinician satisfaction in one trial.140 No studies provided data on clinical outcomes, health 

services utilization, or antimicrobial resistance. 

Quality of Included Studies 

Many of the studies did not meet most of the prespecified quality criteria. Randomized 

controlled designs were used in 13 trials (from 12 studies),112, 114, 135, 138, 139, 142-144, 146, 147, 153, 154 19 

used CBA designs(from 13 studies),102, 104, 106, 121, 136, 137, 140, 141, 145, 149-152 and one employed an 

ITS design.148 Among 19 trials using a non-randomized comparison group, nine described the 

rationale for selection of the comparison population. 
36 

Treatment assignment was frequently allocated at levels (i.e., clinician, clinic, community) 

that differed from the unit of analysis (i.e., patient; N=16 studies). In nine of these studies104, 137, 

140-143, 145, 151, 152 investigators failed to statistically correct for differences between the unit of 

analysis and unit of allocation. Antibiotic prescribing outcomes were assessed using chart 

review data in nine trials; another 21 used administrative data, and three used other methods. 

Nine trials reported empiric evidence for a quality gap in the population under study. Usual 

care was provided to the control group in 24 trials, whereas in nine the comparison group 

received some form of low-intensity intervention. 

The duration of study followup (either following or concurrent with the study) was less than 

1 year in 21 trials. The remaining eight trials that provided this data reported followup periods 

ranging from 1 – 1.5 years. 

Effect of Interventions on Antibiotic Selection

Table 8 summarizes the median effect of each type of intervention on selection of 

recommended antibiotics in the 22 interventions that reported results in a manner suitable for this 

approach. (Individual study effect sizes are shown in Figure 8.) The median absolute effect was a 

10.6% net improvement (IQR 3.4% - 18.2%) in the rate of recommended antibiotic prescribing 

in intervention groups compared with comparison groups. Clinician education combined with 

audit and feedback had the smallest median effect (3.4%), while the combination of clinician and 

patient education had the largest effect (22.8%). These differences in median effect sizes were 

not statistically significant (P=0.18). However, the efficiency of this statistical test is impaired 

by the presence of two groups (clinician education combined with patient education, and audit 

and feedback alone) with low sample sizes (two and one, respectively) in each group. To limit 

the effect of intervention types with small sample sizes on the tests of significance, we compared 

the two QI strategies for which more than three trials contributed data (the “dominant” QI 

strategies); in this subset of studies, clinician education alone was more effective than clinician 

education combined with audit and feedback (13.9% vs. 3.4%, P=0.03). 
37 

Figure 8. Effect sizes for each included study (antibiotic selection studies) 

22.7%

35.6%

13.9%

8.4% 7.2%

5.1%

3.1%

43.1%

4.3%

14.9%

48.0%

17.0%

2.5%

-7.0%

20.4%

9.0%

12.5%

13.9%

2.4% 2.0% 

-10.0%

0.0%

10.0%20.0%30.0%40.0%50.0%60.0%

Bush (1) Bush(2) Coenen De Santis Farris Friis (1) Friis (2) HarrisonHuxIlett Klein Lundborg McNulty Melander Molstad PetersonRaz Rokstad Smabrekke Sondergaard Ve

Study

Net Change in Prescribing Rate

Figure 8 Legend Effect size of each study (N=22) that supplied sufficient information for analysis. Percentages show the net change in percentage of patients prescribe

calculated as: Net ∆ in prescribing = (Post-intervention prescribing rate – Pre-intervention prescribing rate)intervention group - (Post-intervention prescribin

prescribing rate)control group.
38 

Table 9 shows the distribution of potential confounders of the association between QI 

strategy and effect on prescribing for the two dominant intervention types. Among these 19 

trials, the absolute difference in effect between each confounder stratum ranged from 2.3% for 

RCTs vs. CBA trials, to 20.8% for trials conducted inside vs. outside the US. In our assessment 

of potential confounders, three study characteristics were not evenly distributed across our 

“dominant” QI strategies: study design (RCT vs. CBA, p=0.18 by Fisher’s exact test), sample 

size (above vs. below the median, p=0.15), and target disease (specific disease targeted vs. 

general prescribing targeted, p=0.16). 

Table 5 shows the effect of these potential confounders on the median effect size. Although 

randomized and non-randomized interventions were unevenly distributed across the two 

dominant QI strategies, this uneven distribution did not appear to impact the median effect size 

of these QI strategies (p=0.62 for difference in median effect size). The median effects of 

interventions targeting prescribing in general was also not significantly different from 

interventions targeting a specific disease (p=0.19). However, interventions with below-median 

sample size showed a greater median effect size than those with above-median sample size 

(P=0.06 for difference). As almost all clinician education combined with audit and feedback 

interventions were above the median sample size (compared with about half of the clinician 

education-alone interventions), the presence of positive confounding must be considered (i.e., the 

larger median effect size for clinician-education alone interventions compared with clinician 

education and audit and feedback can be partially explained by the greater number of abovemedian sample size studies in the latter group). 

Study country and type of comparison group were also significantly associated with median 

effect size, with trials conducted inside the US and trials employing a “usual care” comparison 

group showing greater median effect sizes than their counterparts (P<0.05 for comparison with 

non-US interventions and interventions using a “limited intervention” control group, 

respectively; Table 5). However, because these study characteristics were evenly distributed 

among interventions employing the dominant QI strategies, they did not meet our criteria as a 

confounder of the relationship between QI strategy and outcome effect. Other potential 

confounders that did not have a significant impact on median effect size included baseline 

prescribing rate, use of active vs. passive educational strategies, and single vs. repeated 

interventions (P>0.2 for all). Only one study137 targeted children, and the two trials comprising 

this study reported larger effects than studies not targeting children. With regard to our 

prespecified quality criteria, no factors potentially affecting internal validity had a significant 

effect on median effects, nor did factors potentially affecting translatability. 

No studies attempted to increase the use of broad-spectrum antibiotics (e.g., in the face of 

resistant organisms), so we were unable to compare the efficacy of such interventions with those 

that aimed to decrease broad-spectrum antibiotic use. 

Studies of Duration of Antibiotic Therapy 

Four studies evaluated the effect of interventions on reducing the duration146, 147, 151, 154 of 

antibiotic prescriptions, all for women with UTIs. Three of these interventions used active forms 

of clinician education (workshops, supplemented in one study by consensus-building sessions) 

combined with audit and feedback. The results were mixed. One intervention146 was associated 

with a 13% increase in the percentage of antibiotic prescriptions for short-course regimens in the 

intervention group compared with the comparison group; the second study147 found that the 
39 

intervention had little effect on antibiotic duration (relative increase of antibiotic duration 0.06 

days), while the third154 documented a relative decrease in defined daily doses equivalent to a 

reduction of 1.89 days in antibiotic duration in the intervention compared with comparison 

group. Using a similar calculation of defined daily doses per prescription, an intervention using 

only audit and feedback found a relative decrease in antibiotic duration of 0.55 days in the 

intervention group compared with the comparison group.151 

Studies Not Eligible for Median Effect Size Analysis 

Eleven trials in six studies135, 136, 139, 146, 148, 152 met inclusion criteria for this review but did 

not present data amenable for inclusion in the median effects analyses. The majority of these 

(seven trials from three studies)135, 148, 152 targeted antibiotic selection in general, while three 

trials (from two studies)136, 146 focused on UTIs and one on ARIs.139 The methodologic quality 

of these interventions was generally similar to those that were included in the median effects 

analysis: among 11 trials, four (from three studies)135, 139, 146 used randomized controlled 

designs, but all non-randomized trials used a comparable comparison group and adequately 

described the rationale for its selection. However, none of the trials measured prescribing by 

chart review. 

The majority of these interventions evaluated the effect of clinician educational outreach 

alone or in combination with other QI strategies. One study152 evaluated the effects of outreach 

performed by a physician or pharmacist on prescribing of contraindicated antibiotics (antibiotics 

deemed unsafe for routine office use) and of oral cephalosporins. Both types of interventions 

were compared to a comparison group that received either no intervention or a passive mailed 

intervention (which had no effect relative to the no intervention group). Outreach performed by 

a physician achieved relative reductions of 85% and 50% in the number of prescriptions per 

patient over a 1-year period for oral cephalosporins and contraindicated antibiotics, respectively, 

compared with 67% and 35% reductions by the pharmacist-educator and 41% and 33% 

reductions in the comparison group. (Data on absolute reductions were not presented.) 

Another study comprising two interventions found that educational outreach reduced nonrecommended antibiotic prescribing 31% relative to the comparison group, compared with a 

relative reduction of 10% in physicians receiving only printed materials. Both of these relative 

reductions were in part attributable to a 12% increase in non-recommended antibiotic use in the 

comparison group over the study period.135 

Other interventions employed variations on the classic approach to educational outreach. 

One study using pharmaceutical sales representatives to promote amoxicillin use for children 

with otitis media documented a non-significant 1.4% increase in adjusted market share for this 

drug in regions receiving outreach compared with control regions.139 Finally, another study 

evaluated two multifaceted interventions that included feedback delivered via distribution of a 

workbook or educational outreach. These investigators found a 0% and 12% improvement in the 

workbook and educational outreach groups (respectively) compared with the comparison group, 

based on an outcome of mean percent change in recommended antibiotic prescribing per 1000 

patients over a 3-month period.136

One study used an interrupted time series design to evaluate financial and regulatory 

incentives designed to reduce fluoroquinolone prescribing to elderly patients in a Canadian 

province. After implementation, use of this drug class fell from 20.2% to 4.2% of all antibiotic 

drug claims registered with the province’s prescription drug insurance plan.148 
40 

Comparisons Within Studies 

Five studies evaluated two or more separate interventions; three of these studies135, 136, 152 are 

described in the section above, and the remainder are described below. One study documented a 

5.1% improvement in recommended antibiotic prescribing by physicians invited to lectures 

sponsored by a local department of clinical microbiology, compared with a 1.1% improvement in 

physicians invited to similar lectures sponsored by a pharmaceutical company; the control group 

and both intervention groups all received written materials.141 The other study compared 

antibiotic prescribing for pediatric otitis media among three small sites of a health maintenance 

organization, where one site developed and implemented a consensus guideline for treatment of 

this condition, the second site implemented this guideline through meetings and written 

materials, and the third site served as control.137 Guideline-concordant prescribing improved 

13.0% at the development site and 0.1% at the implementation site, and fell 22.6% at the control 

site. Of note, baseline concordance was substantially lower at the development site (64.3%) than 

at the other two sites (87.8% and 89.6%). 

As noted above, nine trials (from five studies)121, 138, 141, 152, 153 provided some form of lowerintensity activity in a comparison group that was considered in the study to be a control arm; five 

of these (from four studies)121, 138, 141, 153 presented data suitable for median effects analysis. 

These five trials were a heterogeneous group, including one intervention comparing educational 

outreach to a comparison group receiving low-intensity feedback (with 8.4% relative 

improvement in the intervention arm relative to control),138 one comparing antibiotic workshops 

to a comparison group receiving microbiology tutorials (2.5% relative improvement in the 

intervention arm),121 one comparing mailed audit and feedback to distribution of written 

materials (2% relative superiority in the intervention arm),153 and two interventions from one 

study comparing meetings to distribution of written materials (5.1% and 1.1% relative 

improvement in the intervention arms).141, 153 Overall, the median effect size of interventions 

whose efficacy was compared with that of a low-intensity control was lower than those whose 

comparison group received no form of activity (median effect 3.1% vs 14.4%, P=0.04; Table 5). 

Among these studies, there were nine comparisons of active vs. passive provider education 

(either as distinct intervention arms, or as an intervention arm and low-intensity intervention 

control arm). In all nine, the active form of education resulted in greater improvement than the 

passive one.121, 135-138, 152

Special Populations and Secondary Endpoints 

The median effect of trials focusing on ARIs and UTIs was identical (12%), with similar 

median effects across different types of interventions. Three studies enrolled children102, 137, 139

(one of which137 focused specifically on children enrolled in an HMO); the median effect of 

interventions in these studies was similar to the overall effect of non-age-targeted studies. Two 

studies enrolled the elderly143, 148; the first achieved a 4.3% improvement over control using a 

combination of passive clinician education with audit and feedback (similar to the median effect 

for all studies in this category),143 while the second (an interrupted time series) documented a 

16% absolute reduction in fluoroquinolone use after implementation of financial and regulatory 

incentives.148

No studies looked at clinical outcomes, including adverse drug reactions, clinical status, or 

health services utilization. Similarly, no studies evaluated the effect of interventions on 
41 

antimicrobial resistance, or visit duration. A single study140 that evaluated clinician attitudes 

documented a mean response of 4.0 (on a 5-point Likert scale, 5=”strongly agree” and 

1=”strongly disagree”) regarding clinicians’ opinions of the intervention as an appropriate 

mechanism to optimize medication use. 

 Four studies looked at cost outcomes,121, 140, 143, 144 of which three presented data in usable 

form. Each of these three interventions decreased cost by approximately 20-30%. One study 

using clinician education and audit and feedback documented a median $3.37 (31%) increase in 

the mean cost of an antibiotic prescription in the comparison group, compared with no change in 

costs in the intervention group.143 Two studies evaluated the total cost of antimicrobials in a 

health care system, with one144 finding an 18% relative decrease in total antimicrobial costs 

attributable to non-recommended antibiotics in the intervention group compared to the 

comparison group, and the other a 9% rise in total antibiotic costs in the comparison group, 

compared with a 10% decrease in the intervention arm.121

43 

Chapter 4. Discussion 

Summary 

The published medical literature supports the benefits of QI interventions to reduce 

unnecessary antibiotic prescribing and improve antibiotic selection in primary care practices. 

Overall, efforts to reduce prescribing of antibiotics for non-bacterial acute illnesses reduced 

prescription rates by an absolute value of 8.9% (IQR -12.4% to -6.7%). Interventions to improve 

antibiotic selection resulted in a 10.6% absolute increase in the rate of recommended antibiotic 

prescribing (IQR 3.4% - 18.2%). Similar effects were observed in studies not eligible for median 

effects analysis. The quality of included studies was generally fair, with similar problems to 

those seen in the prior reports on hypertension and diabetes in the Closing the Quality Gap

series. 

We did not find definitive evidence for the superiority of individual QI strategies or 

combinations of strategies. Active educational strategies appeared to be more effective than 

passive education, though this comparison did not achieve statistical significance in either 

antibiotic treatment or antibiotic selection studies. However, we also found evidence for the 

increased effectiveness of active educational strategies in studies not eligible for median effects 

analysis and in within-study comparisons in both groups (treatment and selection). In the 

selection studies, the combination of clinician education and audit and feedback appeared less 

effective than clinician education alone; this finding is likely due to confounding by sample size. 

Study results were consistent across patient populations and disease processes. Interventions 

reported from outside the US appeared less effective than those based in America particularly in 

the selection group, but very few US-based studies were eligible for quantitative analysis. Very 

few included studies presented data on antimicrobial resistance, clinical outcomes, or costs, and 

no firm conclusions can be reached regarding these outcomes. Limited data does indicate that 

patient satisfaction is not impaired by interventions to reduce antibiotic use. 

Limitations 

As noted in the Methods section, we were not able to perform meta-analysis or metaregression due to the limited number of eligible studies and significant heterogeneity. Our 

alternative quantitative analysis approach consisted of calculating median effect sizes stratified 

by presence or absence of study design and intervention characteristics, then comparing these 

results using non-parametric tests. This approach does allow for quantitative comparisons 

among groups, but is limited in its ability to control for important confounders, and does not 

directly incorporate measures of study quality into the analysis of summary effects. As well, 

even the few statistically significant results we found have not been corrected for multiple 

comparisons. 

In our analysis, we attempted to account for as many measurable potential confounders and 

effect modifiers as possible. However, we were limited by the description of the intervention 

and study setting provided in each article. Undoubtedly, many potential moderating factors go 

unmeasured simply due to lack of adequate description in the literature. These can include 

factors that can increase the likelihood of intervention success (e.g., a high degree of support 
44 

from top management) and factors that decrease the likelihood (e.g., lack of resources to ensure 

adequate intervention reach).51 We attempted to be as specific as possible in measuring the 

intensity of the interventions. However, intervention intensity may largely reflect the 

characteristics of individuals providing the intervention, including their own investment in the 

process and their relationships with the target population, neither of which was directly 

measured.155 We attempted to abstract information on all facets of the intervention; despite these 

efforts, unmeasured confounders may yet have influenced our results. The combination of 

limited statistical power and the likelihood of unmeasured confounders cautions against using 

strict interpretations of our quantitative findings. 

Although we did find greater effects in studies with smaller sample size in the selection 

studies, we did not find the same relationship in interventions targeting the treatment decision. 

One explanation for significant differences in effect sizes when stratified by sample size or other 

methodologic features is publication bias, the preferential publication of positive studies. 

Publication bias occurs more frequently for small sample size studies, resulting from the 

propensity for smaller studies and trials with less rigorous designs to be published if they report 

large improvements.156 In two previous reviews1, 2 of the QI literature, we identified significant 

inverse relationships between study sample size and the magnitude of reported effects (i.e., 

smaller studies reported larger effect sizes). Sample size may also be correlated with other 

important methodologic features, such as blinding. Other biases may influence individual study 

results. Studies of prescribing behavior targeting clinicians may be spuriously influenced by the 

Hawthorne effect157 (in which the knowledge that one’s behavior is under observation changes 

behavior). Another potential concern is that clinicians may engage in “code shifting,” listing a 

patient’s diagnosis as one usually requiring antibiotics (e.g., pneumonia) instead of one not 

warranting antibiotics (e.g., bronchitis). However, this concern has not been borne out in the 

literature.111, 158, 159 

Evidence indicates that a clinician’s decision to prescribe an antibiotic depends on a variety 

of factors relating to the health care system and patient beliefs and expectations, and is not solely 

dependent on the clinician’s subjective beliefs or knowledge of evidence-based practice. In this 

light, it is reasonable to hypothesize that effective strategies to reduce inappropriate antibiotic 

prescribing should target multiple domains (clinician, patient, and health system). However, 

while nearly all included strategies targeted clinicians and many targeted patients, very few 

studies examined the effect of health system factors such as formulary restrictions or drug copayments. Also, few interventions targeting clinicians specifically addressed the physicianpatient interaction; helping clinicians understand and manage patient expectations for antibiotics 

could theoretically be more efficacious. Despite these omissions, we did find that strategies 

targeting patients or clinicians can positively influence prescribing rates. Further research on the 

effectiveness of health system-level interventions, as well as assessment of the interaction 

between the health system and clinician and patient-level interventions, will add greatly to our 

ability to design effective QI programs. 

Finally, our results are limited to the short observation periods, as most followup periods 

were less than 1 year. Intervention designs that lead to sustained changes in antibiotic 

prescribing over multiple years might be different, with repeated public education/awareness and 

health system modifications playing a more dominant role. 

Thus, many of the findings in this review raise more questions than they answer, and should 

be construed as hypothesis-generating. The interactions between each of the components of an 

intervention and the population it targets (as outlined in the conceptual framework in the 
45 

introduction) are complex, and the limited number of studies available for analysis allow only a 

relatively rudimentary evaluation. 

Conclusions 

Despite the above caveats, the available studies do illustrate several distinct and effective 

approaches to improving prescribing behavior in a range of settings. While firm conclusions 

cannot be drawn, our data are consistent with several concepts, as described below. In the 

discussion, we will address the questions most relevant to stakeholders considering undertaking 

quality improvement efforts to reduce the inappropriate use of antibiotics. Organizations should 

examine their specific clinical settings as well as their clinician and patient populations, and 

compare studies performed in similar settings to identify specific QI strategies that are most 

likely to be effective in their own setting. To assist with this process, Appendix A*

 provides 

examples of key studies, and Appendix B provides details of each included study organized by 

setting and measured population. 

1. Are quality improvement strategies to improve outpatient antibiotic use effective? 

Our review found that the vast majority of published studies reported clinically significant 

improvements in antibiotic treatment and selection. The magnitude of these effects compares 

favorably to those achieved by quality improvement efforts in other settings. These findings 

were consistent across diverse patient populations and clinical settings. More than half the 

included studies were performed in the last decade; the concomitant decline in antibiotic 

prescribing in US ambulatory practices suggests that efforts to promote judicious antibiotic use 

are succeeding. However, inappropriate prescribing rates remain high, and inappropriate 

selection of antibiotics presents a continuing challenge. 

In addition to benefiting patients and the community by reducing the adverse consequences 

of inappropriate antibiotic use, effective interventions may result in cost savings, although 

definitive evidence is lacking. 

2. What are the critical components of effective intervention strategies to improve 

outpatient antibiotic use?

Antibiotic treatment studies. Within the antibiotic treatment studies, we were able to compare 

studies using clinician education alone with those using clinician and patient education. The 

addition of patient education to clinician education did not result in greater reductions in 

antibiotic prescribing, a finding that withstood evaluation for potential negative confounding. 

No distinct differences emerged among strategies, with the exception of a possible trend toward 

greater effectiveness in studies using active educational strategies. This finding was supported 

by systematic evaluation of studies that were not incorporated in the median effects analysis.122, 

126, 128 

Potential negative confounding by unmeasured variables and limited power may affect our 

finding of an apparent lack of additional benefit of patient education over clinician education 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/medigap/medigap.pdf 
46 

alone. Although studies included in median effects analyses did not show an additional benefit, 

two large population-based studies122, 126 not incorporated in median effects analyses did 

demonstrate reductions in antimicrobial prescribing with a combined clinician and patient 

educational intervention. Of note, these studies were conducted in the U.S., but most included 

studies were from outside the U.S. It is conceivable that different practice styles, levels of 

baseline prescribing rates, and patient expectation of antibiotic treatment may make patient 

education more important in the U.S. than outside the country. 

Antibiotic selection studies. Most selection trials eligible for median effects analysis employed 

one of two main QI strategies: clinician education alone, or clinician education in combination 

with audit and feedback. The absolute difference in median effect between these strategies was 

substantial; interventions adding audit and feedback to clinician education were less effective 

than interventions employing clinician education alone. The effectiveness of other types of QI 

strategies is difficult to systematically assess, since there were only three studies for all other 

categories of QI strategies. 

The surprising finding that adding audit and feedback to clinician education results in smaller 

effects on prescribing may be explained in part by confounding. Interventions employing 

clinician education alone were substantially more likely to have below-median sample size; these 

smaller studies had overall larger effect sizes. While this may reflect publication bias, it is also 

possible that low sample size is acting as a proxy for studies that use local relationships and 

leadership to achieve greater clinician “buy-in” than studies spread over many sites. In addition 

to publication bias and sample size already noted, there may be other subtle confounders. For 

example, interventions that employed audit and feedback may have used less intensive methods 

to implement the clinician education component, spreading their energies among several 

intervention strategies rather than focusing on one. These possibilities are intriguing, but not 

directly testable in this analysis. A prudent conclusion may be that we are unable to definitively 

assess the relative efficacy of clinician education alone vs. clinician education combined with 

audit and feedback, but that we found no evidence to suggest that the combined approach is 

superior to clinician education alone. 

Unlike the antibiotic treatment studies, there was no prominent association between the 

presence of active vs. passive types of educational interventions and antibiotic prescribing 

outcomes. It is difficult to assess whether this lack of association is true, or is merely negatively 

confounded by other observed or unobserved variables, compounded by limited power to detect 

differences due to low sample sizes. Other findings suggest that active educational interventions 

may in fact produce better outcomes. First, the differences in median effect size between active 

and passive techniques were in the expected direction, and of similar magnitude to differences 

observed in treatment studies, in which a trend (P=0.11) toward an association was observed, 

suggesting insufficient power. In addition, studies that compare two or more interventions to one 

another can be viewed as controlling for the many measured and unmeasured confounders that 

make inter-study comparisons so challenging.160 In each of these cases, the active group 

outperformed the passive group.135, 152 

Our overall results thus supply cautious support for including active clinician education in 

quality improvement efforts for improving antibiotic use. Combining other strategies with 

clinician education does not necessarily improve outcomes, but our ability to detect such 

differences was limited. 
47 

3. Which patients and conditions should be targeted in order to exert the maximal impact 

on antibiotic prescribing?

We did not find any evidence that targeting specific patient populations resulted in 

significant differences in study effects, in either antibiotic treatment or antibiotic selection 

studies. 

When selecting an intervention target, stakeholders and policymakers may wish to consider 

the potential population-level intervention effect, rather than simply the target-specific effect. 

Interventions that have highly significant effects on antibiotic prescribing for single conditions or 

limited patient populations may not necessarily exert large effects on overall antibiotic 

prescribing rates. At the population level, targeting all ARIs appears to translate into larger 

reductions in antibiotic consumption than focusing on a single condition or patient age range, 

and this might be an important factor in the willingness of purchasers/payors of health care to 

invest in appropriate antibiotic use interventions. 

Interventions targeting antibiotic selection appeared to be equally effective across different 

disease processes and patient populations. Thus, stakeholders should determine the quality gap 

in their unique patient population, and target interventions appropriately. 

4. What are the limitations of current research in this field and what areas require further 

study? 

Included studies rarely reported important measures of the potential harms of the 

intervention. These include the potential for increased use of health services (e.g., return visits 

due to persistent symptoms) and adverse clinical consequences (e.g., increasing rates of serious 

infections due to undertreatment). Patient satisfaction does not appear to be affected in the 

limited number of studies that did report this outcome. 

 More importantly, very few of the included studies (and none of the US-based treatment 

studies) documented the resources required for completion of the intervention and 

measurements. Only four102, 121, 143, 144 studies reported the cost savings resulting from changing 

prescribing behavior. Thus, although an active clinician education intervention may be more 

effective than a passive intervention, we are unable to make any statement regarding the costeffectiveness of such an intervention. Given the apparent benefit of other, potentially less 

resource-intensive interventions, further research should include formal reporting of both costs of 

and cost savings from QI interventions. Further studies should also clearly document 

intervention intensity and reach, the baseline quality gap, and any other local factors that could 

have affected the intervention or outcomes measurement. 

Changes in antibiotic resistance rates should be monitored not only for declines in resistance 

rates, but also for changes in the rate of rise in existing or new resistance patterns. This will 

require longer-term followup than measured in most studies. Based on the best evidence and 

mathematical modeling to date, it is unlikely that reductions in antibiotic consumption will lead 

to major reductions in levels of antibiotic resistance among community-acquired bacterial 

infections.33-35 However, ecological studies do support the notion that the amount of antibiotic 

consumption in a community directly influences how rapidly new resistance emerges or rises.7, 37 

As the benefits from preventing antimicrobial resistance will be seen largely by the community 

at large, health care organizations may be reluctant to invest heavily in programs to reduce 

antimicrobial use without a clear business case. A demonstration that these types of 
48 

interventions can recover their implementation costs through savings in antibiotic costs would be 

helpful. Our crude analysis in Figure 7 indicates that many of these interventions have the 

potential to be cost-neutral or cost-saving, depending on the cost of the intervention, the 

reduction in antibiotic prescriptions and the cost of antibiotics. 

Study design and quality should be improved. Studies that formally evaluate the cost 

effectiveness of interventions to improve antibiotic treatment and selection are needed, and 

studies should evaluate the potential harms of such interventions. 
49 

References 

1. Shojania K, Ranji S, Shaw LK, Charo LN, Lai JC, Rushakoff RJ, McDonald KM, Owens DK. Closing The 

Quality Gap: A Critical Analysis Of Quality Improvement Strategies, Volume 2 - Diabetes Mellitus Care. 

AHRQ Publication No. 04-0051-2. Rockville, MD: Agency for Healthcare Research and Quality. 2004. 

2. Walsh J, McDonald KM, Shojania KG, Sundaram V, Lewis R, Nayak S, Mechanic J, Owens DK, Goldstein 

MK. Closing The Quality Gap: A Critical Analysis Of Quality Improvement Strategies, Volume 3 - 

Hypertension Care. AHRQ Publication No. 04-0051-3. Rockville, MD: Agency for Healthcare Research and 

Quality. 2005. 

3. Cohen ML. Epidemiology of drug resistance: implications for a post-antimicrobial era. Science. 

1992;257(5073):1050-1055. 

4. Levy SB. Confronting multidrug resistance. A role for each of us. Jama. 1993;269(14):1840-1842. 

5. Whitney CG, Farley MM, Hadler J, et al. Increasing prevalence of multidrug-resistant Streptococcus 

pneumoniae in the United States. N Engl J Med. 2000;343(26):1917-1924. 

6. Doern GV, Brueggemann AB, Huynh H, Wingert E. Antimicrobial resistance with Streptococcus 

pneumoniae in the United States, 1997 98. Emerg Infect Dis. 1999;5(6):757-765. 

7. Chen DK, McGeer A, de Azavedo JC, Low DE. Decreased susceptibility of Streptococcus pneumoniae to 

fluoroquinolones in Canada. Canadian Bacterial Surveillance Network. N Engl J Med. 1999;341(4):233-239. 

8. Herold BC, Immergluck LC, Maranan MC, et al. Community-acquired methicillin-resistant Staphylococcus 

aureus in children with no identified predisposing risk. Jama. 1998;279(8):593-598. 

9. Akram J, Glatt AE. True community-acquired methicillin-resistant Staphylococcus aureus bacteremia. Infect 

Control Hosp Epidemiol. 1998;19(2):106-107. 

10. Gupta K, Hooton TM, Wobbe CL, Stamm WE. The prevalence of antimicrobial resistance among 

uropathogens causing acute uncomplicated cystitis in young women. Int J Antimicrob Agents. 1999;11(3-

4):305-308. 

11. Marco CA, Parker K. Antimicrobial resistance among organisms causing urinary tract infections. Acad Emerg 

Med. 1997;4(2):159-160. 

12. Wright SW, Wrenn KD, Haynes ML. Trimethoprim-sulfamethoxazole resistance among urinary coliform 

isolates. J Gen Intern Med. 1999;14(10):606-609. 

13. Doone JL, Klespies SL, Sabella C. Risk factors for penicillin-resistant systemic pneumococcal infections in 

children. Clin Pediatr (Phila). 1997;36(4):187-191. 

14. Hofmann J, Cetron MS, Farley MM, et al. The prevalence of drug-resistant Streptococcus pneumoniae in 

Atlanta. N Engl J Med. 1995;333(8):481-486. 

15. Moreno F, Crisp C, Jorgensen JH, Patterson JE. The clinical and molecular epidemiology of bacteremias at a 

university hospital caused by pneumococci not susceptible to penicillin. J Infect Dis. 1995;172(2):427-432. 

16. Nava JM, Bella F, Garau J, et al. Predictive factors for invasive disease due to penicillin-resistant 

Streptococcus pneumoniae: a population-based study. Clin Infect Dis. 1994;19(5):884-890. 
50 

17. Newby D, A. Fryer J. L. Henry D. A. Effect of computerised prescribing on use of antibiotics. comment. 

Medical Journal of Australia. 2003;178(5):210. 

18. Chesney PJ, Davis Y, English BK, Wang WC. Occurrence of Streptococcus pneumoniae meningitis during 

vancomycin and cefotaxime therapy of septicemia in a patient with sickle cell disease. Pediatr Infect Dis J. 

1995;14(11):1013-1015. 

19. Friedland IR, Klugman KP. Failure of chloramphenicol therapy in penicillin-resistant pneumococcal 

meningitis. Lancet. 1992;339(8790):405-408. 

20. Lonks JR, Durkin MR, Meyerhoff AN, Medeiros AA. Meningitis due to ceftriaxone-resistant Streptococcus 

pneumoniae. N Engl J Med. 1995;332(13):893-894. 

21. Radetsky MS, Istre GR, Johansen TL, et al. Multiply resistant pneumococcus causing meningitis: its 

epidemiology within a day-care centre. Lancet. 1981;2(8250):771-773. 

22. Reid R, Jr., Bradley JS, Hindler J. Pneumococcal meningitis during therapy of otitis media with 

clarithromycin. Pediatr Infect Dis J. 1995;14(12):1104-1105. 

23. Pallares R, Viladrich PF, Linares J, Cabellos C, Gudiol F. Impact of antibiotic resistance on chemotherapy for 

pneumococcal infections. Microb Drug Resist. 1998;4(4):339-347. 

24. Poole MD. Otitis media complications and treatment failures: implications of pneumococcal resistance. 

Pediatr Infect Dis J. 1995;14(4 Suppl):S23-26. 

25. Bradley JS, Kaplan SL, Klugman KP, Leggiadro RJ. Consensus: management of infections in children caused 

by Streptococcus pneumoniae with decreased susceptibility to penicillin. Pediatr Infect Dis J. 

1995;14(12):1037-1041. 

26. Feikin DR, Schuchat A, Kolczak M, et al. Mortality from invasive pneumococcal pneumonia in the era of 

antibiotic resistance, 1995-1997. Am J Public Health. 2000;90(2):223-229. 

27. Metlay JP, Hofmann J, Cetron MS, et al. Impact of penicillin susceptibility on medical outcomes for adult 

patients with bacteremic pneumococcal pneumonia. Clin Infect Dis. 2000;30(3):520-528. 

28. Bartlett JG, Dowell SF, Mandell LA, File Jr TM, Musher DM, Fine MJ. Practice guidelines for the 

management of community-acquired pneumonia in adults. Infectious Diseases Society of America. Clin 

Infect Dis. 2000;31(2):347-382. 

29. Niederman MS, Mandell LA, Anzueto A, et al. Guidelines for the management of adults with communityacquired pneumonia. Diagnosis, assessment of severity, antimicrobial therapy, and prevention. Am J Respir 

Crit Care Med. 2001;163(7):1730-1754. 

30. Hyde TB, Gay K, Stephens DS, et al. Macrolide resistance among invasive Streptococcus pneumoniae 

isolates. Jama. 2001;286(15):1857-1862. 

31. Kunin CM. Resistance to antimicrobial drugs--a worldwide calamity. Ann Intern Med. 1993;118(7):557-561. 

32. Neu HC. The crisis in antibiotic resistance. Science. 1992;257(5073):1064-1073. 

33. McCormick AW, Whitney CG, Farley MM, et al. Geographic diversity and temporal trends of antimicrobial 

resistance in Streptococcus pneumoniae in the United States. Nat Med. 2003;9(4):424-430. 

34. Lipsitch M, Samore MH. Antimicrobial use and antimicrobial resistance: a population perspective. Emerg 

Infect Dis. 2002;8(4):347-354. 
51 

35. Lipsitch M. The rise and fall of antimicrobial resistance. Trends Microbiol. 2001;9(9):438-444. 

36. Seppälä H, Klaukka T, Vuopio-Varkila J, et al. The Effect of Changes in the Consumption of Macrolide 

Antibiotics on Erythromycin Resistance in Group A Streptococci in Finland. N Engl J Med. 1997;337:441-

446. 

37. Doern GV, Heilmann KP, Huynh HK, Rhomberg PR, Coffman SL, Brueggemann AB. Antimicrobial 

resistance among clinical isolates of Streptococcus pneumoniae in the United States during 1999--2000, 

including a comparison of resistance rates since 1994--1995. Antimicrob Agents Chemother. 

2001;45(6):1721-1729. 

38. Gonzales R, Steiner JF, Sande MA. Antibiotic prescribing for adults with colds, upper respiratory tract 

infections, and bronchitis by ambulatory care physicians. Jama. 1997;278(11):901-904. 

39. Mainous AG, 3rd, Hueston WJ, Davis MP, Pearson WS. Trends in antimicrobial prescribing for bronchitis 

and upper respiratory infections among adults and children. Am J Public Health. 2003;93(11):1910-1914. 

40. McCaig LF, Besser RE, Hughes JM. Antimicrobial drug prescription in ambulatory care settings, United 

States, 1992-2000. Emerg Infect Dis. 2003;9(4):432-437. 

41. Steinman MA, Gonzales R, Linder JA, Landefeld CS. Changing use of antibiotics in community-based 

outpatient practice, 1991-1999. Ann Intern Med. 2003;138(7):525-533. 

42. Jernigan DB, Cetron MS, Breiman RF. Minimizing the impact of drug-resistant Streptococcus pneumoniae 

(DRSP). A strategy from the DRSP Working Group. Jama. 1996;275(3):206-209. 

43. Institute of Medicine. Antimicrobial Resistance: Issues and Options. Harrison PF and Lederberg J, eds. 

National Academy Press. 1998. 

44. Centers for Disease Control. Public Health Action Plan to Combat Antimicrobial Resistance. Available 

online at: http://www.cdc.gov/drugresistance/actionplan/html/index.htm. Accessed March 10, 2005. 

45. National Committee for Quality Assurance Health Plan and Employer Data Information Set (HEDIS). 

www.ncqa.org, accessed 7.1.2005. 

46. Vander Stichele RH, Elseviers MM, Ferech M, Blot S, Goossens H. European surveillance of antimicrobial 

consumption (ESAC): data collection performance and methodological approach. Br J Clin Pharmacol. 

2004;58(4):419-428. 

47. Institute of Medicine. Priority Areas for National Action: Transforming Health Care Quality. Adams K and 

Corrigan JM, eds. National Academy Press. 2003. 

48. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research 

and practice: an overview of systematic reviews of interventions to promote the implementation of research 

findings. The Cochrane Effective Practice and Organization of Care Review Group. Bmj. 

1998;317(7156):465-468. 

49. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of 

interventions. Med Care. 2001;39(8 Suppl 2):II2-45. 

50. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of 

interventions to improve professional practice. Cmaj. 1995;153(10):1423-1431. 

51. Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff 

(Millwood). 2005;24(1):138-150. 
52 

52. Kleinman A, Eisenberg L, Good B. Culture, illness, and care: clinical lessons from anthropologic and crosscultural research. Ann Intern Med. 1978;88(2):251-258. 

53. Vinson DC, Lutz LJ. The effect of parental expectations on treatment of children with a cough: a report from 

ASPN. J Fam Pract. 1993;37(1):23-27. 

54. Bauchner H, Pelton SI, Klein JO. Parents, physicians, and antibiotic use. Pediatrics. 1999;103(2):395-401. 

55. Hamm RM, Hicks RJ, Bemben DA. Antibiotics and respiratory infections: are patients more satisfied when 

expectations are met? J Fam Pract. 1996;43(1):56-62. 

56. Cockburn J, Pit S. Prescribing behaviour in clinical practice: patients' expectations and doctors' perceptions of 

patients' expectations--a questionnaire study. Bmj. 1997;315(7107):520-523. 

57. Mangione-Smith R, McGlynn EA, Elliott MN, Krogstad P, Brook RH. The relationship between perceived 

parental expectations and pediatrician antimicrobial prescribing behavior. Pediatrics. 1999;103(4 Pt 1):711-

718. 

58. Macfarlane J, Holmes W, Macfarlane R, Britten N. Influence of patients' expectations on antibiotic 

management of acute lower respiratory tract illness in general practice: questionnaire study. Bmj. 

1997;315(7117):1211-1214. 

59. Gonzales R, Corbett KK, Leeman-Castillo BA, et al. The "Minimizing Antibiotic Resistance in Colorado" 

Project: Impact of Patient Education in Improving Antibiotic Use in Private Office Practices. Health Serv 

Res. 2005;40(1):101-116. 

60. Nyquist AC, Gonzales R, Steiner JF, Sande MA. Antibiotic prescribing for children with colds, upper 

respiratory tract infections, and bronchitis. Jama. 1998;279(11):875-877. 

61. Gonzales R, Barrett PH, Jr., Steiner JF. The relation between purulent manifestations and antibiotic treatment 

of upper respiratory tract infections. J Gen Intern Med. 1999;14(3):151-156. 

62. Heald A, Auckenthaler R, Borst F, et al. Adult bacterial nasopharyngitis: a clinical entity? J Gen Intern Med. 

1993;8(12):667-673. 

63. Kaiser L, Lew D, Hirschel B, et al. Effects of antibiotic treatment in the subset of common-cold patients who 

have bacteria in nasopharyngeal secretions. Lancet. 1996;347(9014):1507-1510. 

64. Mainous AG, 3rd, Hueston WJ, Eberlein C. Colour of respiratory discharge and antibiotic use. Lancet. 

1997;350(9084):1077. 

65. Mainous AG, 3rd, Zoorob RJ, Oler MJ, Haynes DM. Patient knowledge of upper respiratory infections: 

implications for antibiotic expectations and unnecessary utilization. J Fam Pract. 1997;45(1):75-83. 

66. Robertson AJ. Green sputum. Lancet. 1952;1(1):12-15. 

67. Mainous AG, 3rd, Hueston WJ, Love MM. Antibiotics for colds in children: who are the high prescribers? 

Arch Pediatr Adolesc Med. 1998;152(4):349-352. 

68. Stone S, Gonzales R, Maselli J, Lowenstein SR. Antibiotic prescribing for patients with colds, upper 

respiratory tract infections, and bronchitis: A national study of hospital-based emergency departments. Ann 

Emerg Med. 2000;36(4):320-327. 

69. Steinman MA, Landefeld CS, Gonzales R. Predictors of broad-spectrum antibiotic prescribing for acute 

respiratory tract infections in adult primary care. Jama. 2003;289(6):719-725. 
53 

70. Linder JA, Stafford RS. Antibiotic treatment of adults with sore throat by community primary care 

physicians: a national survey, 1989-1999. Jama. 2001;286(10):1181-1186. 

71. Huang ES, Stafford RS. National patterns in the treatment of urinary tract infections in women by ambulatory 

care physicians. Arch Intern Med. 2002;162(1):41-47. 

72. Gillam SJ. Sociocultural differences in patients' expectations at consultations for upper respiratory tract 

infection. J R Coll Gen Pract. 1987;37(298):205-206. 

73. Sanchez-Menegay C, Hudes ES, Cummings SR. Patient expectations and satisfaction with medical care for 

upper respiratory infections. J Gen Intern Med. 1992;7(4):432-434. 

74. Wilson AA, Crane LA, Barrett PH, Gonzales R. Public beliefs and use of antibiotics for acute respiratory 

illness. J Gen Intern Med. 1999;14(11):658-662. 

75. Gershman K, Leff M, Brand D et al. A statewide survey of antibiotic beliefs and practices for coughs and 

colds in Colorado. 1st International Conference on Emerging Infectious Diseases. 1998. 

76. Gonzales R, Wilson A, Crane LA, Barrett PH, Jr. What's in a name? Public knowledge, attitudes, and 

experiences with antibiotic use for acute bronchitis. Am J Med. 2000;108(1):83-85. 

77. Gonzales R, Steiner JF, Maselli J, Lum A, Barrett PH, Jr. Impact of reducing antibiotic prescribing for acute 

bronchitis on patient satisfaction. see comments. Effective Clinical Practice. 2001;4(3):105. 

78. Cowan PF. Patient satisfaction with an office visit for the common cold. J Fam Pract. 1987;24(4):412-413. 

79. Melnick SL, Sprafka JM, Laitinen DL, Bostick RM, Flack JM, Burke GL. Antibiotic use in urban whites and 

blacks: the Minnesota Heart Survey. Ann Pharmacother. 1992;26(10):1292-1295. 

80. Arnold KE, Leggiadro RJ, Breiman RF, et al. Risk factors for carriage of drug-resistant Streptococcus 

pneumoniae among children in Memphis, Tennessee. J Pediatr. 1996;128(6):757-764. 

81. Butler JC, Hofmann J, Cetron MS, Elliott JA, Facklam RR, Breiman RF. The continued emergence of drugresistant Streptococcus pneumoniae in the United States: an update from the Centers for Disease Control and 

Prevention's Pneumococcal Sentinel Surveillance System. J Infect Dis. 1996;174(5):986-993. 

82. Schillinger D, Bindman A, Wang F, Stewart A, Piette J. Functional health literacy and the quality of 

physician-patient communication among diabetes patients. Patient Educ Couns. 2004;52(3):315-323. 

83. Shapiro MF, Ware JE, Jr., Sherbourne CD. Effects of cost sharing on seeking care for serious and minor 

symptoms. Results of a randomized controlled trial. Ann Intern Med. 1986;104(2):246-251. 

84. Stirewalt CF, Linn MW, Godoy G, Knopka F, Linn BS. Effectiveness of an ambulatory care telephone 

service in reducing drop-in visits and improving satisfaction with care. Med Care. 1982;20(7):739-748. 

85. Arnold SR, Allen UD, Al-Zahrani M, Tan DH, Wang EE. Antibiotic prescribing by pediatricians for 

respiratory tract infection in children. Clin Infect Dis. 1999;29(2):312-317. 

86. Hutchinson J, Foley RN. Influence of nonmedical factors on antibiotic prescription rates. 37th Interscience 

Conference on Antimicrobial Agents and Chemotherapy. 1997. 

87. Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored 

interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 

2002;325:7360. 
54 

88. Cochrane Collaboration Effective Practice and Organisation of Care group. Available online at: 

http://www.epoc.uottawa.ca/register.htm. Accessed March 10, 2005. 

89. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. 

Jama. 1999;282(11):1054-1060. 

90. Maselli JH, Gonzales R. Measuring antibiotic prescribing practices among ambulatory physicians: accuracy 

of administrative claims data. J Clin Epidemiol. 2001;54(2):196-201. 

91. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and 

implementation strategies. Health Technol Assess. 2004;8(6):iii-iv, 1-72. 

92. Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. Bmj. 

2004;328(7441):702-708. 

93. Grimshaw J, McAuley LM, Bero LA, et al. Systematic reviews of the effectiveness of quality improvement 

strategies and programmes. Qual Saf Health Care. 2003;12(4):298-303. 

94. Torgerson DJ. Contamination in trials: is cluster randomisation the answer? Bmj. 2001;322(7282):355-357. 

95. Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an 

overview. Ann Intern Med. 2001;135(2):112-123. 

96. Kerry SM, Bland JM. Sample size in cluster randomisation. Bmj. 1998;316(7130):549. 

97. Donner A, Piaggio G, Villar J. Statistical methods for the meta-analysis of cluster randomization trials. Stat 

Methods Med Res. 2001;10(5):325-338. 

98. Campbell M, Grimshaw J, Steen N. Sample size calculations for cluster randomised trials. Changing 

Professional Practice in Europe Group (EU BIOMED II Concerted Action). J Health Serv Res Policy. 

2000;5(1):12-16. 

99. Campbell MK, Mollison J, Grimshaw JM. Cluster trials in implementation research: estimation of intracluster 

correlation coefficients and sample size. Stat Med. 2001;20(3):391-399. 

100. Woodwell DA, Cherry DK. National Ambulatory Medical Care Survey: 2002 summary. Adv Data. 

2004(346):1-44. 

101. Welschen I, Kuyvenhoven MM, Hoes AW, Verheij TJ. Effectiveness of a multiple intervention to reduce 

antibiotic prescribing for respiratory tract symptoms in primary care: randomised controlled trial. Bmj. 

2004;329(7463):431. 

102. Smabrekke L, Berild D, Giaever A, et al. Educational intervention for parents and healthcare providers leads 

to reduced antibiotic use in acute otitis media. Scand J Infect Dis. 2002;34(9):657-659. 

103. Santoso B, Suryawati S, Prawaitasari JE. Small group intervention vs formal seminar for improving 

appropriate drug use published erratum appears in Soc Sci Med 1996 Jul;43(1):I. Social Science & Medicine. 

1996;42(8):1163. 

104. Molstad S, Hovelius. Reduction in antibiotic usage following an educational programme. Family Practice. 

1989;6(1):33-37. 

105. Meyer JC, Summers RS, Moller H. Randomized, controlled trial of prescribing training in a South African 

province. Med Educ. 2001;35(9):833-840. 
55 

106. Melander E, Bjorgell A, Bjorgell P, Ovhed I, Molstad S. Medical audit changes physicians' prescribing of 

antibiotics for respiratory tract infections. Scandinavian Journal of Primary Health Care. 1999;17(3):180. 

107. Mainous AG, 3rd, Hueston WJ, Love MM, Evans ME, Finger R. An evaluation of statewide strategies to 

reduce antibiotic overuse. Family Medicine. 2000;32(1):22. 

108. Hickman DE, Stebbins MR, Hanak JR, Guglielmo BJ. Pharmacy-based intervention to reduce antibiotic use 

for acute bronchitis. Annals of Pharmacotherapy. 2003;37(2):187. 

109. Hennessy TW, Petersen KL, Bruden D, et al. Changes in Antibiotic-Prescribing Practices and Carriage of 

Penicillin-Resistant Streptococcus pneumoniae: A Controlled Intervention Trial in Rural Alaska. Clin Infect 

Dis. 2002;34:1543-1550. 

110. Gonzalez Ochoa E, Armas Perez L, Bravo Gonzalez JR, Cabrales Escobar J, Rosales Corrales R, Abreu 

Suarez G. Prescription of antibiotics for mild acute respiratory infections in children. Bulletin of the Pan 

American Health Organization. 1996;30(2):106. 

111. Gonzales R, Steiner JF, Lum A, Barrett PHJ. Decreasing antibiotic use in ambulatory practice: impact of a 

multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 

1999;281(16):1512-1519. 

112. Coenen S, Van Royen P, Michiels B, Denekens J. Optimizing antibiotic prescribing for acute cough in 

general practice: a cluster-randomized controlled trial. J Antimicrob Chemother. 2004;54(3):661-672. 

113. Belongia EA, Sullivan BJ, Chyou PH, Madagame E, Reed KD, Schwartz B. A community intervention trial 

to promote judicious antibiotic use and reduce penicillin-resistant Streptococcus pneumoniae carriage in 

children. Pediatrics. 2001;108(3):575. 

114. Zwar N, Wolk J, Gordon J, Sanson-Fisher R, Kehoe L. Influencing antibiotic prescribing in general practice: 

A trial of prescriber feedback and management guidelines. Family Practice. 1999;16(5):495. 

115. Angunawela I, Diwan VK, Tomson G. Experimental evaluation of the effects of drug information on 

antibiotic prescribing: a study in outpatient care in an area of Sri lanka. International Journal of 

Epidemiology. 1991;20(2):558. 

116. Bexell A, Lwando E, von Hofsten B, Tembo S, Eriksson B, Diwan VK. Improving drug use through 

continuing education: a randomized controlled trial in Zambia. Journal of Clinical Epidemiology. 

1996;49:355. 

117. Gutierrez G, Guiscafre H, Bronfman M, Walsh J, Martinez H, Munoz O. Changing physician prescribing 

patterns: evaluation of an educational strategy for acute diarrhea in Mexico City. Medical Care. 

1994;32(5):436. 

118. McIsaac WJ, Goel V. Effect of an explicit decision-support tool on decisions to prescribe antibiotics for sore 

throat. Medical Decision Making. 1998;18(2):220. 

119. McIsaac WJ, Goel V, To T, Permaul JA, Low DE. Effect on antibiotic prescribing of repeated clinical 

prompts to use a sore throat score: lessons from a failed community intervention study. Journal of Family 

Practice. 2002;51(4):339. 

120. Foxman B, Valdez RB, Lohr KN, Goldberg GA, Newhouse JP, Brook RH. The effect of cost sharing on the 

use of antibiotics in ambulatory care: results from a population-based randomized controlled trial. Journal of 

Chronic Diseases. 1987;40(5):429. 

121. McNulty CA, Kane A, Foy CJ, Sykes J, Saunders P, Cartwright KA. Primary care workshops can reduce and 

rationalize antibiotic prescribing. Journal of Antimicrobial Chemotherapy. 2000;46(3):493. 
56 

122. Perz JF, Craig AS, Coffey CS, et al. Changes in antibiotic prescribing for children after a community-wide 

campaign. JAMA. 2002;287(23):3103. 

123. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled 

trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. 

Pediatrics. 2001;107(2). 

124. McConnell TS, Cushing AH, Bankhurst AD, Healy JL, McIlvenna PA, Skipper BJ. Physician behavior 

modification using claims data: tetracycline for upper respiratory infection. West J Med. 1982;137(5):448-

450. 

125. O'Connell DL, Henry D, Tomlins R. Randomised controlled trial of effect of feedback on general 

practitioners' prescribing in Australia. BMJ. 1999;318(7182):507-511. 

126. Finkelstein JA, Davis RL, Dowell SF, et al. Reducing antibiotic use in children: a randomized trial in 12 

practices. Pediatr. 2001;108:1-7. 

127. Cates C. An evidence based approach to reducing antibiotic use in children with acute otitis media: controlled 

before and after study. Bmj. 1999;318(7185):715-716. 

128. Wilson EJ, Nasrin D, Dear KB, Douglas RM. Changing GPs' antibiotic prescribing: a randomised controlled 

trial. Commun Dis Intell. 2003;27 Suppl:S32-38. 

129. Little P, Gould C, Williamson I, Moore M, Warner G, Dunleavey J. Pragmatic randomised controlled trial of 

two prescribing strategies for childhood acute otitis media. BMJ. 2001;322(7282):336-342. 

130. Arroll B, Kenealy T, Kerse N. Do Delayed Prescriptions Reduce the Use of Antibiotics for the Common 

Cold? J Fam Pract. 2002;51(4):324-328. 

131. Dowell J, Pitkethly M, Bain J, Martin S. A randomised controlled trial of delayed antibiotic prescribing as a 

strategy for managing uncomplicated respiratory tract infection in primary care. Br J Gen Pract. 

2001;51(3):200-205. 

132. Pshetizky Y, Naimer S, Shvartzman P. Acute otitis media--a brief explanation to parents and antibiotic use. 

Fam Pract. 2003;20(4):417-419. 

133. Macfarlane J, Holmes W, Gard P, et al. Reducing antibiotic use for acute bronchitis in primary care: blinded, 

randomised controlled trial of patient information leaflet * Commentary: More self reliance in patients and 

fewer antibiotics: still room for improvement. BMJ. 2002;324(7329):91-. 

134. Hulscher ME, Laurant MG, Grol RP. Process evaluation on quality improvement interventions. Qual Saf 

Health Care. 2003;12(1):40-46. 

135. Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach. A randomized 

controlled trial of academically based "detailing". New England Journal of Medicine. 1983;308(24):1457-

1463. 

136. Braybrook S, Walker R. Influencing prescribing in primary care: a comparison of two different prescribing 

feedback methods. Journal of Clinical Pharmacy & Therapeutics. 1996;21(4):247. 

137. Bush PJ, Rabin DL, Spector KK. Evaluation of a drug therapy protocol in an HMO. Medical Care. 

1979;17(6):566. 

138. De Santis G, Harvey KJ, Howard D, Mashford ML, Moulds RF. Improving the quality of antibiotic 

prescription patterns in general practice. The role of educational intervention. Medical Journal of Australia. 

1994;160(8):502. 
57 

139. Dolovich L, Levine M, Tarajos R, Duku E. Promoting optimal antibiotic therapy for otitis media using 

commercially sponsored evidence-based detailing: A prospective controlled trial. Drug Information Journal. 

1999;33(4):1067. 

140. Farris KB, Kirking DM, Shimp LA, Opdycke RA. Design and results of a group counter-detailing DUR 

educational program. Pharmaceutical Research. 1996;13(10):1445. 

141. Friis H, Bro F, Mabeck CE, Vejlsgaard R. Changes in prescription of antibiotics in general practice in relation 

to different strategies for drug information. Danish Medical Bulletin. 1991;38(4):380. 

142. Harrison A, Karim SA, Floyd K, et al. Syndrome packets and health worker training improve sexually 

transmitted disease case management in rural South Africa: randomized controlled trial. AIDS. 

2000;14(17):2769. 

143. Hux JE, Melady MP, DeBoer D. Confidential prescriber feedback and education to improve antibiotic use in 

primary care: a controlled trial. CMAJ. 1999;161(4):388. 

144. Ilett KF, Johnson S, Greenhill G, et al. Modification of general practitioner prescribing of antibiotics by use 

of a therapeutics adviser (academic detailer). British Journal of Clinical Pharmacology. 2000;49(2):168. 

145. Klein LE, Charache P, Johannes RS. Effect of physician tutorials on prescribing patterns of graduate 

physicians. Journal of Medical Education. 1981;56(6):504. 

146. Lagerlov P, Loeb M, Andrew M, Hjortdahl P. Improving doctors' prescribing behaviour through reflection on 

guidelines and prescription feedback: a randomised controlled study. Quality in Health Care. 2000;9(3):159. 

147. Lundborg CS, Wahlstrom R, Oke T, Tomson G, Diwan VK. Influencing prescribing for urinary tract 

infection and asthma in primary care in Sweden: a randomized controlled trial of an interactive educational 

intervention. Journal of Clinical Epidemiology. 1999;52(8):801. 

148. MacCara ME, Sketris IS, Comeau DG, Weerasinghe SD. Impact of a limited fluoroquinolone reimbursement 

policy on antimicrobial prescription claims. Annals of Pharmacotherapy. 2001;35(7-8):852. 

149. Peterson GM, Stanton LA, Bergin JK, Chapman GA. Improving the prescribing of antibiotics for urinary 

tract infection. Journal of Clinical Pharmacy & Therapeutics. 1997;22(2):147. 

150. Raz R, Porat V, Ephros M. Can an educational program improve the diagnosis and treatment of 

pharyngotonsillitis in the ambulatory care setting? Israel Journal of Medical Sciences. 1995;31(7):432. 

151. Rokstad K, Straand J, Fugelli P. Can drug treatment be improved by feedback on prescribing profiles 

combined with therapeutic recommendations? A prospective, controlled trial in general practice. Journal of 

Clinical Epidemiology. 1995;48(8):1061-1068. 

152. Schaffner W, Ray WA, Federspiel CF, Miller WO. Improving antibiotic prescribing in office practice. A 

controlled trial of three educational methods. JAMA. 1983;250(13):1728. 

153. Sondergaard J, Andersen M, Stovring H, Kragstrup J. Mailed prescriber feedback in addition to a clinical 

guideline has no impact: a randomised, controlled trial. Scandinavian Journal of Primary Health Care. 

2003;21(1):47. 

154. Veninga CC, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Improving drug treatment in general practice. 

Journal of Clinical Epidemiology. 2000;53(7):762. 

155. Flottorp S, Havelsrud K, Oxman AD. Process evaluation of a cluster randomized trial of tailored 

interventions to implement guidelines in primary care--why is it so hard to change practice? Family Practice. 

2003;20(3):333. 
58 

156. Dickersin K. The existence of publication bias and risk factors for its occurrence. Jama. 1990;263(10):1385-

1389. 

157. Mangione-Smith R, Elliott MN, McDonald L, McGlynn EA. An observational study of antibiotic prescribing 

behavior and the Hawthorne effect. Health Serv Res. 2002;37(6):1603-1623. 

158. Hueston WJ, Slott K. Improving quality or shifting diagnoses? What happens when antibiotic prescribing is 

reduced for acute bronchitis? Archives of Family Medicine. 2000;9(9):933. 

159. Harris RH, MacKenzie TD, Leeman-Castillo B, et al. Optimizing antibiotic prescribing for acute respiratory 

tract infections in an urban urgent care clinic. J Gen Intern Med. 2003;18(5):326-334. 

160. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for 

decision making in clinical and health policy. Jama. 2003;290(12):1624-1632.
59 

Listing of Excluded Studies 

Citation Reason for exclusion 

1 Using guidelines to change clinical practice: three case studies. HMO 

Practice 1996; 10(1): 30-36

Study design did not 

meet criteria for RCT, 

CBA, or ITS

2 Anokbonggo WW, Ogwal-Okeng J. W. Obua C. Aupont O. RossDegnan D. Impact of decentralization on health services in Uganda: a 

look at facility utilization, prescribing and availability of essential drugs. 

East African Medical Journal 2004:S2 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

3 Anonymous. North of England study of standards and performance in 

general practice. An overview of the study. Report no 50. Centre for 

Health Services Research, Ambulatory Care Programme 1991. 

Overlaps with or 

duplicates another article 

that was included 

4 Anonymous. Medical audit in general practice. II: Effects on health of 

patients with common childhood conditions. North of England Study of 

Standards and Performance in General Practice. BMJ 1992; 304:1484. 

Other 

5 Anonymous. Changing prescribing practices. Medicine Today 2001; 

2:14. 

Not an evaluation of a 

QI intervention 

6 Barwitz HJ. Sore throat consultation--what is the value of a treatment 

guideline? German Original Beratungsanlass Halsschmerzen--was nutzt 

eine Handlungsleitlinie? MMW Fortschritte der Medizin 1999; 141:32. 

Foreign language article 

article 

7 Barwitz HJK. Common cold - Trial to rationalize management in 

general practice by recommendation. Zeitschrift fur Allgemeinmedizin 

1999; 71(21-22):932 

Foreign language article 

article 

8 Batchelor B, Crook DW, Jones T, Bowler IC. Impact of guidelines for 

the diagnosis of urinary tract infection on trimethoprim susceptibility of 

Escherichia coli. Journal of Antimicrobial Chemotherapy 2002; 49:223. 

Not an evaluation of a 

QI intervention 

9 Beardon PH, Brown SV, Mowat DA, et al. Introducing a drug formulary 

to general practice--effects on practice prescribing costs. Journal of the 

Royal College of General Practitioners 1987; 37:305. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

10 Bowler IC, Atkins BL, Batchelor BI, et al. Impact of guidelines for the 

diagnosis of UTI letter; see comments. British Journal of General 

Practice 1998; 48:1790. 

No eligible outcomes 

11 Brook RH, Williams KN. Effect of medical care review on the use of 

injections: a study of the New Mexico Experimental Medical Care 

Review Organization. Annals of Internal Medicine 1976; 85:509. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 
60 

12 Burr A. Impact of fundholding on general practice prescribing patterns. 

Pharmaceutical Journal 1992; 249 R8 

No eligible outcomes 

13 Chalker J. Improving antibiotic prescribing in Hai Phong Province, Viet 

Nam: the "antibiotic-dose" indicator. Bulletin of the World Health 

Organization 2001; 79:313. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

14 Chiefari DM. Effect of a closed formulary on average prescription cost 

in a community health center. Drug Benefit Trends 2001; 13:45 - 52. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

15 Chowdhury AKA, et al. Effect of standard treatment guidelines with or 

without audit on prescribing for acute respiratory infections in 

government health facilities in Bangladesh. International Network for 

Rational Use of Drugs 1996. 

Article not available 

16 Cohen R, Allaert FA, Callens A, et al. Medico-economic evaluation of 

an educational intervention to optimize children uncomplicated 

nasopharyngitis treatment in ambulatory care. Medecine et Maladies 

Infectieuses 2000; 30:691. 

Foreign language article 

17 de Silva MI, Mize GN, Rissing JP. Peer review of antibiotic use: 

positive impact on physician prescribing patterns. Qrb 1985; 

QualityReviewBulle:302. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

18 Dickey FF, Mattar ME, Chudzik GM. Pharmacist counsling increases 

drug regimen compliance. Hospitals 1988; 49:85-86. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

19 Doyne EO, Alfaro MP, Siegel RM, et al. A randomized controlled trial 

to change antibiotic prescribing patterns in a community. Arch Pediatr 

Adolesc Med 2004; 158:577-83. 

Other 

20 Du B, Chen D, Liu D, Long Y, Shi Y, Wang H, Rui X, Cui N. 

Restriction of third-generation cephalosporin use decreases infectionrelated mortality. A controlled trial of a critical pathway for treating 

community-acquired pneumonia: the CAPITAL study. CommunityAcquired Pneumonia Intervention Trial Assessing Levofloxacin. Critical 

Care Medicine 2003; 31:1088. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

21 Ekedahl A, Andersson SI, Hovelius B, Molstad S, Liedholm H, 

Melander A. Drug prescription attitudes and behaviour of general 

practitioners. Effects of a problem-oriented educational programme. 

European Journal of Clinical Pharmacology; 1995: 47(5): 381 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

22 Evans RS, Classen DC, Pestotnik SL, et al. Improving empiric antibiotic 

selection using computer decision support. Archives of Internal 

Medicine 1994; 154:878. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 
61 

23 Finney JW, Friman PC, Rapoff MA, et al. Improving compliance with 

antibiotic regimens for otitis media. Randomized clinical trial in a 

pediatric clinic. American Journal of Diseases of Children 1985; 139:89-

95. 

No eligible outcomes 

24 Font M, Madridejos R, Catalan A, et al. Improving drug prescription in 

primary care: a controlled and randomized study of an educational 

method . Spanish. Medicina Clinica 1991; 96:201. 

Foreign language article 

25 Friis H, Bro F, Mabeck CE, et al. An information campaign--an 

important measure in controlling the use of antibiotics. Journal of 

Antimicrobial Chemotherapy 1989; 24:993. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

26 Friis H, Bro F, Eriksen NR, et al. The effect of reimbursement on the use 

of antibiotics. Scandinavian Journal of Primary Health Care 1993; 

11:247. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

27 Garcia Lirola MA, Cabeza Barrera J, Lirola Garcia E. Intervention to 

improve the quality of antibacterial drug prescribing practices in primary 

care. Farmacia Hospitalaria 1999; 23:42. 

Foreign language article 

28 Gonzales R, Steiner JF, Maselli J, et al. Impact of reducing antibiotic 

prescribing for acute bronchitis on patient satisfaction. see comments. 

Effective Clinical Practice 2001; 4:105. 

No eligible outcomes 

29 Gonzales R, Sauaia A, Corbett KK, et al. Antibiotic treatment of acute 

respiratory tract infections in the elderly: effect of a multidimensional 

educational intervention. Journal of the American Geriatrics Society 

2004; 52:39. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

30 Goode CJ, Tanaka DJ, Krugman M, et al. Outcomes from use of an 

evidence-based practice guideline. Nursing Economics. 2000; 18:202. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

31 Gordis L. Evaluation of the effectiveness of comprehensive and 

continuous pediatric care. Pediatrics 1971; 48:766. 

Not an evaluation of a 

QI intervention 

32 Greenfield S, Friedland G, Scifers S, et al. Protocol management of 

dysuria, urinary frequency, and vaginal discharge. Annals of Internal 

Medicine 1974; 81:452. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

33 Gregory C, Cifaldi M, Tanner LA. Targeted intervention programs: 

creating a customized practice model to improve the treatment of 

allergic rhinitis in a managed care population. American Journal of 

Managed Care 1999; 5:485. 

Other 
62 

34 Guiscafre H, Martinez H, Reyes H, et al. From research to public health 

interventions. I. Impact of an educational strategy for physicians to 

improve treatment practices of common diseases. Archives of Medical 

Research 1995; 26:S31. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

35 Gupta D, Mishra S, Chaturvedi P. ARI Control Programme: standard 

case management guidelines vs conventional treatment--an open study. 

Indian Pediatrics 1996; 33:41-43. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

36 Gurwitz JH, McLaughlin TJ, Fish LS. The effect of an Rx-to-OTC 

switch on medication prescribing patterns and utilization of physician 

services: the case of vaginal antifungal products. Health Services 

Research 1995; 30:672. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

37 Harris CM, Jarman B, Woodman E, et al. Prescribing--a suitable case for 

treatment. J R Coll Gen Pract 1984; 24:1-39. 

Other 

38 Harris CM, Fry J, Jarman B, et al. Prescribing--a case for prolonged 

treatment. J R Coll Gen Pract 1985; 35:284-7. 

Other 

39 Harris RH, MacKenzie TD, Leeman-Castillo B, et al. Optimizing 

antibiotic prescribing for acute respiratory tract infections in an urban 

urgent care clinic. J Gen Intern Med 2003; 18:326-34. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

40 Hastings JE, Mott FD, Barclay A, et al. Prepaid group practice in Sault 

Ste. Marie, Ontario. I. Analysis of utilization records. Medical Care 

1973; 11:91. 

Not an evaluation of a 

QI intervention 

41 Hemeryck L, McGettigan P, Feely J. Hospital antibiotic prescribing and 

feedback [letter; comment]. British Journal of Clinical Pharmacology 

1997; 43:449-450. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

42 Hueston WJ, Mainous AG, 3rd, Brauer N, et al. Evaluation and 

treatment of respiratory infections: does managed care make a 

difference? Journal of Family Practice 1997; 44:572. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

43 Ives T, et al. Effect of an educational intervention on oral cephalosporin 

use in primary care. Arch Internal Med 1987; 147:44-47. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

44 Jacobs B, Kambugu FS, Whitworth JA, Ochwo M, Pool R, Lwanga A, 

Tifft S, Lule J, Cutler JR. Social marketing of pre-packaged treatment 

for men with urethral discharge (Clear Seven) in Uganda. International 

Journal of STD & AIDS 2003; 14:216. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

45 Juncosa S, Porta M. Effects of primary health care reform on the 

prescription of antibiotics: A longitudinal study in a Spanish county. 

European Journal of Public Health 1997; 7:54. 

Other 
63 

46 Kafle KK. Intervention test of training supervision on prescribing 

practices. 1995. 

Article not available 

47 Kinney WC. Rhinosinusitis treatment protocol: changing provider habits 

in primary care. Otolaryngology Head & Neck Surgery 2002; 126:614. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

48 Leach RH, Wakeman A. An evaluation of the effectiveness of 

community pharmacists working with GPs to increase the costeffectiveness of prescribing. Pharmaceutical Journal 1999; 263:206. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

49 Lundborg CS, Tomson G, Wahlstrom R, et al. GPs' knowledge and 

attitudes regarding treatment of UTI and asthma in Sweden: A 

randomised controlled educational trial on guideline implementation. 

European Journal of Public Health 2000; 10:246. 

No eligible outcomes 

50 Margolis CZ, Warshawsky SS, Goldman L, et al. Computerized 

algorithms and pediatricians' management of common problems in a 

community clinic. Academic Medicine 1992; 67:282. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

51 Mather JL, Bayliff CD, Reider MJ, et al. The impact of formulary 

reservations on drug utilization: a controlled trial. Canadian Journal of 

Hospital Pharmacy. 1994; 47:111-116. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

52 Molstad S, Ekedahl. Antibiotics prescription in primary care: A 5-year 

follow-up of an educational programme. Family Practice 1994; 11:282-

286. 

Other 

53 Mtango F, Neuvians D. Acute respiratory infections in children under 

five years. Control project in Bagamoyo District, Tanzania. Trans Roy 

Soc Trop Med Hyg 1986; 80:851. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

54 Munck AP, Gahrn-Hansen B, Sogaard P, et al. Long-lasting 

improvement in general practitioners' prescribing of antibiotics by means 

of medical audit. Scandinavian Journal of Primary Health Care 1999; 

17:185-190. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

55 Naivalulevu L. Training for rational drug use (prescriptions of 

antibiotics for coughs and colds in Fiji has been greatly reduced since 

health staff participated in training workshops). ARI News 1990; 19. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

56 Needham A. Introduction and audit of a general practice antibiotic 

formulary. J Royal College of General Practitioners 1988; 38:166. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

57 Newby D, A. Fryer J. L. Henry D. A. Effect of computerised prescribing 

on use of antibiotics. comment. Medical Journal of Australia 2003; 

178:210. 

Not an evaluation of a 

QI intervention 
64 

58 Obua C, Ogwal-Okeng JW, Waako P, Aupont O, Ross-Degnan D. 

Impact of an educational intervention to improve prescribing by private 

physicians in Uganda. East African Medical Journal 2004:S17. 

Other 

59 Ornstein SM, MacFarlane LL, Jenkins RG, et al. Medication cost 

information in a computer-based patient record system. Impact on 

prescribing in a family medicine clinical practice. Archives of Family 

Medicine 1999; 8:118-121. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

60 Palmer NA, Dailey YM, Martin MV. Can audit improve antibiotic 

prescribing in general dental practice? British Dental Journal 2001; 

191:253. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

61 Paredes SP, et al. Intervention trial to decrease the inappropriate use of 

drugs during childhood diarrhoea. 2003. 

Article not available 

62 Perez Rodriguez MT, Catalan Ramos A, Parellada Esquius N. Criteria 

for the development of a formulary of drugs in a basic health area and 

impact on prescription (published erratum appears in Aten Primaria 

1995 Mar 15;15(4):270). Spanish. Atencion Primaria 1994; 14:1128. 

Foreign language article 

63 Perez-Cuevas R, Guiscafre H, Munoz O, et al. Improving physician 

prescribing patterns to treat rhinopharyngitis. Intervention strategies in 

two health systems of Mexico. Social Science & Medicine 1996; 

42:1185. 

Other 

64 Poses RM, Cebul RD, Wigton RS. You can lead a horse to water--

improving physicians' knowledge of probabilities may not affect their 

decisions published erratum appears in Med Decis Making 1995 AprJun;15(2):179. Medical Decision Making 1995; 15:65. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

65 Putnam W, Curry L. Patient care appraisal in the ambulatory setting: 

effectiveness as a continuing medical education tool. Annual Conference 

on Research in Medical Education 1980; 19:207. 

No eligible outcomes 

66 Qazi SA, Rehman GN, Khan MA. Standard management of acute 

respiratory infections in a children's hospital in Pakistan: impact on 

antibiotic use and case fatality. Bulletin World Health Organization 

1996; 74(5):501-7 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

67 Ray WA, Schaffner W, Federspiel CF. Persistence of improvement in 

antibiotic prescribing in office practice. JAMA 1985; 253:1774. 

Overlaps with or 

duplicates another article 

that was included 

68 Ray WA, Fink R, Schaffner W, et al. Improving antibiotic prescribing in 

outpatient practice. Nonassociation of outcome with prescriber 

characteristics and measures of receptivity. Medical Care 1985; 23:1307. 

Other 
65 

69 Rosser W, Dunn L, Pilla J, et al. Using academic detailing to 

disseminate clinical practice guidelines [abstract]. Annual Meeting of 

International Society of Technology Assessment in Health Care 1997; 

13:93. 

Abstract; article not 

available 

70 Saint S, Scholes D, Fihn SD, et al. The effectiveness of a clinical 

practice guideline for the management of presumed uncomplicated 

urinary tract infection in women. American Journal of Medicine 1999; 

106:636-641. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

71 Seppälä H, Klaukka T, Vuopio-Varkila J, et al. The Effect of Changes in 

the Consumption of Macrolide Antibiotics on Erythromycin Resistance 

in Group A Streptococci in Finland. N Engl J Med 1997; 337:441-446. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

72 Sever CM, MacKinnon III GE. Evaluation of a criteria-based antiinfective formulary. P & T 1993; 18:770-776. Study design did not 

meet criteria for RCT, 

CBA, or ITS 

73 Shalansky SJ. Antibiotic interchange through educational interventions 

in a community hospital. American Journal of Hospital Pharmacy 1991; 

48:2655-2657. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

74 Simon JL, Smith DB. Change in location of a student health service: a 

quasi-experimental evaluation of the effects of distance on utilization. 

Medical Care 1973; 11:59. 

Not an evaluation of a 

QI intervention 

75 Soumerai S. Economic and policy analysis of university-based drug 

"detailing". Medical Care 1986; 24:313. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

76 Soumerai S. Predictors of physician prescribing change in an educational 

experiment to improve medication use. Medical Care 1987; 25:210. 

Overlaps with or 

duplicates another article 

that was included 

77 Soumerai SB, Avorn J, Taylor WC, et al. Improving choice of 

prescribed antibiotics through concurrent reminders in an educational 

order form. Medical Care 1993; 31:552. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

78 South M, Royle J, Starr M. A simple intervention to improve hospital 

antibiotic prescribing. comment. Medical Journal of Australia 2003; 

178:207. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

79 Stewart J, Pilla J, Dunn L. Pilot study for appropriate anti-infective 

community therapy. Effect of a guideline-based strategy to optimize use 

of antibiotics. Canadian Family Physician 2000; 46:851. 

Other 
66 

80 Stuart ME, Macuiba J, Heidrich F, et al. Successful implementation of 

an evidence-based clinical practice guideline: acute dysuria/urgency in 

adult women. HMO Practice 1997; 11:150. 

No eligible outcomes 

81 Suchyta MR, Dean NC, Narus S, et al. Effects of a practice guideline for 

community-acquired pneumonia in an outpatient setting. American 

Journal of Medicine 2001; 110:306. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

82 Temte JL, Shult PA, Kirk CJ, et al. Effects of viral respiratory disease 

education and surveillance on antibiotic prescribing. Family Medicine 

1999; 31:101. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

83 Thamer M, Ray NF, Henderson SC, et al. Influence of the NIH 

Consensus Conference on Helicobacter pylori on physician prescribing 

among a Medicaid population. Medical Care 1998; 36:646. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

84 Tierney W, Hui SL, McDonald CJ. Delayed feedback of physician 

performance versus immediate reminders to perform preventive care. 

Effects on physician compliance. Medical Care 1986; 24:659. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

85 Tseng R. An audit of antibiotic prescribing in general practice using sore 

throats as a tracer for quality control. Public Health 1985; 99:177. 

Study design did not 

meet criteria for RCT, 

CBA, or ITS 

86 Tumwikirize WA, Ekwaru PJ, Mohammed K, et al. Impact of a face-toface educational intervention on improving the management of acute 

respiratory infections in private pharmacies and drug shops in Uganda. 

East Afr Med J 2004; Suppl:S25-32. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

87 Wahlstrom R, Kounnavong S. Sisounthone B. Phanyanouvong A. 

Southammavong T. Eriksson B. Tomson G. Effectiveness of feedback 

for improving case management of malaria, diarrhoea and pneumonia--a 

randomized controlled trial at provincial hospitals in Lao PDR. Tropical 

Medicine & International Health 2003; 8:901. 

No eligible outcomes 

88 Walley T, Murphy M, Codd M, et al. Effects of a monetary incentive on 

primary care prescribing in Ireland: changes in prescribing patterns in 

one health board 1990-1995. Pharmacoepidemiology & Drug Safety 

2000; 9:591. 

No eligible outcomes 

89 Williams M. Evaluation of patient counselling on the use of antibiotics 

in community pharmacies. 

Article not available 

90 Wilson RG, Bojke C, O'Neill S, et al. Designing, specifying and 

evaluating a new repeat prescribing process for UK general practice. 

Studies in Health Technology & Informatics 2000; 77:219. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 
67 

91 Wilson SD, Dahl BB, Wells RD. An evidence-based clinical pathway 

for bronchiolitis safely reduces antibiotic overuse. American Journal of 

Medical Quality 2002; 17:195. 

Excluded topic (focused 

on inpatient care or 

antimicrobial use in 

chronic care) 

92 Zwar NA, Gordon JJ, Sanson-Fisher RW. Evaluation of an educational 

program in rational prescribing for GP trainees. Australian Family 

Physician 1995; 24:833. 

Other 

93 Zwar N, Henderson J, Britt H, et al. Influencing antibiotic prescribing by 

prescriber feedback and management guidelines: a 5-year follow-up. 

Family Practice 2002; 19:12. 

Other 

Summary Tables 

71 

Table 1A. QI strategies and outcomes for antibiotic treatment studies 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & 

Fdbck 

Org 

Change 

Finkelstein J1 United 

States 

12 practices 

affiliated with 

2 managed 

care 

organizations 

Children ARI RCT 

(--) 

1997-

1998 

(12) 

3 3 3

 

Angunawela 

I2

Sri Lanka 

General 

practitioners 

at rural 

primary care 

clinics 

All patients Not 

specified 

RCT 

(1048) 

1988 

(2) 

3

 

Angunawela 

I3

Sri Lanka 

General 

practitioners 

at rural 

primary care 

clinics 

All patients Not 

specified 

RCT 

(1348) 

1988 

(2) 

3

 
72 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Arroll, B4 New Zealand 

General 

practitioners 

All patients ARI 

(Common 

cold) 

RCT 

(129) 

-- 

(10 days) 

3

 

Belongia, E5 United States 

Multiple 

counties in 

Northern 

Wisconsin 

Children ARI 

(Common 

cold, 

Acute 

sinusitis, 

Pharyngitis, 

Bronchitis, 

Otitis 

media) 

CBA 

(370) 

1997 

(4) 

3 3

 

Bexell, A6 Zambia 

General 

Practices 

All patients Not 

specified 

QRCT 

(1113) 

1991 

(5) 

3

 

Christakis, 

D7

United 

States 

Academic 

pediatric 

primary care 

clinic 

Children ARI 

(Otitis 

media) 

RCT 

(--) 

-- 

(8) 

3

 
73 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Dowell, J8 UK 

General 

practitioners 

in Scotland 

Adults ARI 

(Cough) 

RCT 

(194) 

1997-

1998 

(--) 

3

 

Flottorp, S9 Norway 

142 General 

practices 

All patients ARI 

(Sore 

throat) 

RCT 

(1739) 

2000-

2001 

(9) 

3 3 3 3 3 

Foxman, B10 United States 

6 cities: 

Dayton, OH; 

Seattle, WA; 

Fitchburg, 

MA; Franklin 

County, MA; 

Charleston, 

SC; 

Georgetown 

County, SC 

All patients Not 

specified 

RCT 

(--) 

1975 

(12) 

3
74 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Gonzales, 

R11

United 

States 

4 HMO 

primary care 

practices 

Adults ARI 

(Acute 

bronchitis) 

CBA 

 (1213) 

1997-

1998 

(4) 

3 3 3

 

Gonzales, 

R12

United 

States 

4 HMO 

primary care 

practices 

Adults ARI 

(Acute 

bronchitis) 

CBA 

(1197) 

1997-

1998 

(4) 

3 3

 

GonzalesOchoa, E13 Cuba General 

practitioners 

in 4 areas of 

Havana 

Children ARI QRCT 

(122) 

1991 

(12) 

3 3

 

GonzalesOchoa, E14 Cuba General 

practitioners 

in 4 areas of 

Havana 

Children ARI QRCT 

(100) 

1991 

(12) 

3

 

GonzalesOchoa, E15 Cuba General 

practitioners 

in 4 areas of 

Havana 

Children ARI QRCT 

(100) 

1991 

(12) 

3

 
75 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Gutierrez, 

G16

Mexico 

General 

practitioners 

at health 

care clinics 

of the 

Mexican 

Social 

Security 

Institute 

All patients Diarrhea CBA 

(152) 

-- 

(18) 

3 3

 

Hennessy, 

T17

United 

States 

13 rural 

Alaskan 

villages 

All patients ARI CBA 

(2163) 

1998-

1999 

(6) 

3 3

 

Hickman, D18 United States 

Suburban 

communitybased 

physician 

group 

All patients ARI 

(Acute 

bronchitis) 

CBA 

(1943) 

1998 

(4) 

3 3 3

 

Little, P19 UK 

General 

practices in 

southwest 

England 

Children ARI 

(Otitis 

media) 

RCT 

(285) 

-- 

(1 week) 

3

 
76 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Macfarlane, 

J20

UK 

General 

practices 

Adults ARI 

(Acute 

bronchitis) 

RCT 

(205) 

1999-

2000 

(11) 

3 3

 

Mainous, A21 United States 

8 Medicaid 

administrativ

e regions in 

Kentucky 

Children ARI RCT 

(2564) 

1997 

(1) 

3

 

Mainous, A22 United States 

8 Medicaid 

administrativ

e regions in 

Kentucky 

Children ARI RCT 

(2740) 

1997 

(1) 

3

 

Mainous, A23 United States 

8 Medicaid 

administrativ

e regions in 

Kentucky 

Children ARI RCT 

(2297) 

1997 

(1) 

3 3

 
77 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

McConnell, 

T24

United 

States 

New Mexico 

Medicaid 

program 

All patients ARI RCT 

(--) 

1997-

1998 

(?) 

3 3

 

McIsaac, W25 Canada 

Family 

physicians in 

Ontario 

All patients ARI 

(Sore 

throat) 

RCT 

(396) 

1996 

(--) 

3

 

McIsaac, W26 Canada 

Family 

physicians in 

Ontario 

All patients ARI 

(Sore 

throat) 

RCT 

(99) 

1998 

(--) 

3

 

Melander, 

E27

Sweden 

General 

practitioners 

All patients ARI CBA 

(857) 

1995 

(--) 

3 3

 

Meyer, J28 South Africa 

Nurses in 

primary 

health clinics 

All patients ARI 

Diarrhea 

RCT 

(383) 

1997 

(6) 

3

 

O’Connell, 

D29

Australia 

General 

practitioners 

in rural area 

All patients Not 

specified 

RCT 

(--) 

1995 

(7) 

3 3

 
78 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Perz, J30 United States 

4 counties in Tennessee 

Children Not 

specified 

CBA 

(--) 

1997-

1998 

(12) 

3 3

 

Santoso, B31 Indonesia 

General 

practitioners 

in 

Yogyakarta 

and Java 

provinces 

Children Diarrhea RCT 

(1638) 

-- 

3

 

Santoso, B32 Indonesia 

General 

practitioners 

in 

Yogyakarta 

and Java 

provinces 

Children Diarrhea RCT 

(1638) 

-- 

3

 

Zwar, N33 Australia 

General 

practitioner 

trainees in 

New South 

Wales 

All patients ARI RCT 

(--) 

-- 

3 3

 
79 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

McNulty, C34 UK 

General 

practices in 

Gloucestershire 

All patients Not 

specified 

CBA 

(--) 

1997 

(7 weeks) 

3 3

 

Welschen, I35 The Netherlands 

General 

practitioners 

in Utrecht 

All patients ARI QRCT 

(1074) 

2000-

2001 

(9) 

3 3 3

 

Molstad, S36 Sweden 

General 

practitioners 

at community 

health 

centers 

All patients ARI 

(Rhinitis, 

Pharyngiti

s, Acute 

bronchitis) 

CBA 

(372) 

1985 

(2) 

3

 

Cates, C37 UK 

General 

practitioners 

in outpatient 

clinics in 

Hertfordshire 

Children ARI 

(Otitis 

media) 

CBA 

(--) 

1997-

1998 

(12) 

3 3

 
80 

Table 1A. QI strategies and outcomes for antibiotic treatment studies (continued) 

Author Setting and 

Target 

Patient 

Population 

Disease 

Target 

Study 

Design 

(Effective 

sample 

size) 

Study 

Period 

(Duration/ 

months) QI Strategies Employed 

Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change Pt Ince

ntive

Pshetizky, 

Y38

Israel 

General 

practitioners 

Children ARI 

(Otitis 

media) 

RCT 

(81) 

1998-

1999 3 3

 

Smabrekke, 

L39 Norway Physicians 

and nurses 

at urgent 

care clinic 

Children ARI 

(Otitis 

media) 

CBA 

(329) 

1998-

1999 

(4) 

3 3

Wilson, E40 Australia 

General 

practitioners 

in Canberra 

Children ARI RCT 

(--) 

1998 

(4) 3 3 3

 

Coenen, S41 Belgium 

General 

practitioners 

Adults ARI RCT 

(238) 

2001 

(1) 3 

Abbreviations: ARI = acute respiratory infection; UTI = urinary tract infection; ABX = antibiotics, CBA = controlled before-after study; RCT = rando

QRCT = quasi randomized controlled trial; Pt Ed = patient education; Delayed = provision of delayed prescriptions; Clin Remind = Clinician Remind

education; Audit & Fdbck = clinician audit and feedback; Org Change = Organizational change; Pt Incentives = patient-directed financial or regulator= clinician-directed financial or regulatory incentives 
81 

Table 1B. QI strategies and outcomes for antibiotic selection studies 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Avorn, J42 United States 

Physicians in 4 

state Medicaid 

programs 

(Arkansas, New 

Hampshire, 

Vermont, and the 

District of 

Columbia) 

Medicaid 

patients 

Not 

specified 

RCT 

(--) 

1980-1981 

(6) 3

 

Avorn, J43 United States 

Physicians in 4 

state Medicaid 

programs 

(Arkansas, New 

Hampshire, 

Vermont, and the 

District of 

Columbia) 

Medicaid 

patients 

Not 

specified 

RCT 

(--) 

1980-1981 

(6) 3

 

Braybrook, 

S44

UK 

66 family health 

practices in 

Wales 

All patients UTI CBA 

(--) 

1992-1993 

(6) 

3 3 

Braybrook, 

S45

UK 

66 family health 

practices in 

Wales 

All patients UTI CBA 

(--) 

1992-1993 

(6) 

3 3 
82 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Bush, P46

United States 

Pediatricians at 3 

HMO sites in 

Washington, DC 

Children 

enrolled in 

HMO 

ARI 

(Otitis 

Media) 

CBA 

(348) 

1975-1976 

(--) 

3

 

Bush, P47

United States 

Pediatricians at 3 

HMO sites in 

Washington, DC 

Children 

enrolled in 

HMO 

ARI 

(Otitis 

Media) 

CBA 

(247) 

1975-1976 

(--) 

3

 

De Santis, G48

Australia 

General 

practitioners in 8 

rural and 

metropolitan 

Victorian Health 

Department 

regions 

All patients ARI 

(Tonsillitis) 

RCT 

(788) 

-- 

(3) 

3

 

Dolovich, L49

Canada 

Family 

physicians in 

Southern Ontario 

Children ARI 

(Otitis 

Media) 

RCT 

(--) 

-- 

3

 
83 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Farris, KB50

United States 

Family 

physicians in 

HMO in 

southeastern 

Michigan 

HMO 

patients 

ARI 

(Otitis 

media, 

Sinusitis, 

Bronchitis, 

Communit

y acquired 

pneumoni

a) 

CBA 

(--) 

1991-1992 

(--) 

3 3

 

Friis, H51

Denmark 

General 

practitioners in 5 

counties 

All patients ARI CBA 

(2600) 

-- 

(3) 

3

 

Friis, H52

Denmark 

General 

practitioners in 5 

counties) 

All patients ARI CBA 

(2046) 

-- 

(3) 

3

 

Hux, J53

Canada 

Family 

physicians in 

Ontario 

Adults ARI 

UTI 

RCT 

(--) 

1996-1997 

(6) 

3 3

 

Ilett, K54

Australia 

General 

practitioners in 

the Perth 

metropolitan 

area 

All patients ARI 

UTI 

RCT 

(2368) 

1997 

(2) 

3

 
84 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Klein, L55

United States 

Residents in 

academic 

emergency room and primary care 

center 

Adults UTI CBA 

(44) 

1978-1979 

(--) 

3

 

Lagerlov, P56 Norway 

General 

practitioners 

Adults UTI RCT 

(--)

1995 

(--) 

3 3

 

Lundborg, 

C57

Sweden 

36 General practitioner 

groups 

Adult women UTI RCT 

(1734) 

1995 

(6) 

3 3

 

MacCara, 

M58

Canada 

General 

practitioners 

Adults Not 

specified 

ITS 

(--) 

1997 

(12) 

McNulty, C59

UK 

84 general 

practices in 

Gloucestershire 

All patients ARI 

UTI 

CBA 

(121,85

7) 

1997 

(7) 

3 3

 

Peterson, 

G60

Australia 

General 

practitioners in 

Southern 

Tasmania 

Adults UTI CBA 

(26,968) 

1995 

(5) 

3

 
85 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Raz, R61 Israel 

Physicians and 

nurses at 3 

community 

outpatient clinics 

All patients ARI 

(Pharyngitis) 

CBA 

(3841) 

1988 

(1) 

3

 

Rokstad, K62 Norway 

General 

practitioners in 2 

counties 

Adults UTI CBA 

(210) 

1989 

(--) 

3

 

Schaffner, 

W63

United States 

Physicians 

participating in 

Tennessee 

Medicaid 

program 

Medicaid 

patients 

Not 

specified 

CBA 

(--) 

1980-1981 

(16) 

3

 

Schaffner, 

W64

United States 

Physicians 

participating in 

Tennessee 

Medicaid 

program 

Medicaid 

patients 

Not 

specified 

CBA 

(--) 

1980-1981 

(16) 

3

 

Schaffner, 

W65

United States 

Physicians 

participating in 

Tennessee 

Medicaid 

program 

Medicaid 

patients 

Not 

specified 

CBA 

(--) 

1980-1981 

(16) 

3

 
86 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Schaffner, 

W66

United States 

Physicians 

participating in 

Tennessee 

Medicaid 

program 

Medicaid 

patients 

Not 

specified 

CBA 

(--) 

1980-1981 

(16) 

3

 

Sondergaard

, J67

Denmark 

181 general 

practices 

All patients ARI RCT 

 (6763) 

1988-1989 

(4) 

3 3

Veninga, C68

Netherlands 

General 

practitioners 

Adult women UTI RCT 

(3433) 

1996 

(5) 

3 3

 

Melander, 

E69

Sweden 

General 

practices in 3 

communities 

All patients ARI CBA 

(68) 

1995 

(--) 3 3

Harrison, A70

South Africa 

Practitioners in 

10 primary care 

clinics in rural 

KwaZulu - Natal) 

Adults STD RCT 

(77) 

1996-1997 

(15) 

3 3

 

Zwar, N71

Australia 

General 

practitioner 

trainees in New 

South Whales 

All patients ARI 

(tonsillitis, 

pharyngiti

s) 

RCT 

(--) 

-- 

3 3

 
87 

Table 1B. QI strategies and outcomes for antibiotic selection studies (continued) 

Author Setting Patient 

Population 

Target 

Disease 

Target 

Study 

Design 

(Sample 

Size) 

Study 

Period 

(Duration/ 

months)

QI Strategies Employed 

 Pt 

Ed 

Delayed Clin Ed Clin Remind Audit & Fdbck Org Change 

Molstad, S72

Sweden 

General 

practitioners at 3 

community 

health centers 

All patients ARI 

(Rhinitis, 

pharyngiti

s, acute 

bronchitis) 

CBA 

(209) 

1985 

(2) 

3

Smabrekke, 

L73

Norway 

Physicians at an 

acute care center 

Children ARI 

(Otitis 

media) 

CBA 

(260) 

1998-1999 

(4) 3 3

 

Coenen, S74 Belgium 

General 

practitioners 

All patients ARI 

(Acute 

cough) 

RCT 

(67) 

2001 

(1) 3 

Abbreviations: ARI = acute respiratory infection; UTI = urinary tract infection; ABX = antibiotics, CBA = controlled before-after study; RCT = rando

interrupted time series; Pt Ed = patient education; Delayed = provision of delayed prescriptions; Clin Ed = clinician education; Clin Remind = clinicia

clinician audit and feedback; Org Change = organizational change; Pt Incentives = patient-directed financial or regulatory incentives; Clin Incentives = or regulatory incentive
88 

Table 1C. Antibiotic treatment studies: target condition and population 

 Studies targeting children Studies targeting adults Studies targeting entire 

population 

Total 

 Specific 

insurance 

No specific 

insurance 

Specific 

insurance 

No specific 

insurance 

Specific 

insurance 

No specific 

insurance 

General ARI 2 

[1,21] 

5 

[5,7,13,26,40] 

0 1 

[8] 

0 7 

[18,24,27,28, 

33,35,36] 

15 

Specific ARI 0 4 

[19,37-39] 

1 

[11] 

2 

[20,25] 

1 

[18] 

3 

[4,9,41] 

11 

Acute diarrhea 1 

[16] 

1 

[31] 

0 0 0 0 2 

No specific 

disease target 

1 

[30] 

0 0 0 0 5 

[2,6,10,29,34] 

6 

Total 4 10 1 3 1 15 34 

Antibiotic treatment studies classified according to target disease and population. “Specific insurance” refers to patients within a 

health maintenance organization (HMO), other managed care organization, Medicare, or Medicaid.
89 

Table 1D. Antibiotic selection studies: target condition and population 

 Studies targeting children Studies targeting adults Studies targeting entire 

population 

Total 

 Specific 

insurance 

No specific 

insurance 

Specific 

insurance 

No specific 

insurance 

Specific 

insurance 

No specific 

insurance 

General ARI 0 0 0 0 1 

[50] 

5 

[36,54,67,69, 

71] 

6 

Specific ARI 1 

[46] 

2 

[39,49] 

0 0 0 3 

[41,48,61] 

6 

Urinary tract 

infection 

0 0 0 6 

[55-57,60, 

62,68] 

0 2 

[44,45] 

8 

Other disease 0 0 0 1 

[70] 

0 0 1 

No disease 

target 

0 0 2* 

[53,58] 

2 

[42,63] 

2 

[34,51] 

6 

Total 1 2 0 9 3 12 26 

Antibiotic selection studies classified according to target disease and population. “Specific insurance” refers to patients within a 

health maintenance organization (HMO), other managed care organization, Medicare or Medicaid. 

*--These studies both targeted elderly patients.
90 

Table 2. Antibiotic treatment studies: number of QI strategies utilized 

Number of QI strategies Randomized 

Controlled Trial 

Quasi-RCT Controlled BeforeAfter Study 

Total 

One 15 

[

2-4, 7, 8, 10, 19, 21, 22, 25, 26, 28, 

31, 32, 41] 

3 

[

6, 14, 15] 

1 

[

36] 

19 

Two 6 

[

20, 23, 24, 29, 33, 38] 

1 

[

13] 

9 

 [5, 11, 16, 17, 27, 30, 34, 37, 39] 

16 

Three 2 

[

1, 40] 

1 

[

35] 

2 

[

11, 18] 

5 

Four 0 0 0 0 

Five 0 0 0 0 

Six 1 

[

9

] 

0 0 1 

More than six 0 0 0 0 

Total 24 5 12 

“Quasi-RCT” refers to studies described as randomized where randomization was altered by investigators, or where a method of 

patient assignment was used that is not truly random (e.g. assignment by even/odd medical record number or date of clinic visit.) 

“Controlled Before-After Study” refers to non-randomized controlled trials with a contemporaneous comparison group. QuasiRCT’s and Controlled Before-After studies were grouped together in statistical analyses. 
91 

Table 3. Median effect size of QI strategies on antibiotic treatment decision 

 

Median effect with QI strategy 

(Interquartile range)

All comparisons 

(N=24) 

-8.9% 

(-12.4% to -6.7%) 

Clinician Education alone 

(N=9) 

-8.1% 

(-13.7% to -7.0) 

Clinician Education combined with 

Patient Education* 

(N=6) 

-7.5% 

(-12.0% to -3.0%) 

Clinician Education combined with 

Audit and Feedback 

(N=2) 

-8.5% 

(-15.0% to -2.0%) 

Clinician Education, Patient 

Education, and Audit and Feedback 

(N=3) 

-12.0% 

(-18.0% to -10.9%) 

Patient Education alone 

(N=2)* 

-4.9% 

(-9.9% to -0.2%) 

Audit and Feedback alone 

(N=1) 

-7.3% 

Patient Education combined with 

Audit and Feedback 

(N=1) 

-7.2% 

P=0.848 for comparison across 

QI strategies 

P=0.478 for comparison of 

clinician education vs. clinician 

education and patient education 

Percentages demonstrate the net reduction in prescribing of antibiotics. Values represent the median effect across studies with 

each QI strategy (or combination of strategies) with interquartile ranges. For categories with N=2, values in parenthesis represent 

the effects of the two studies in question. 

The net change in antibiotic prescribing for each study was calculated as follows: 

Net ∆ in prescribing = (Post-intervention proportion prescribed antibiotics – Pre-intervention proportion prescribed 

antibiotics)intervention group - (Post-intervention proportion prescribed antibiotics – Pre-intervention proportion prescribed 

antibiotics)control group. 

The median value was then obtained from the values of the net ∆ in prescribing for each subset of studies. Median effects for 

each QI strategy were compared using the nonparametric Kruskal-Wallis test, which tests the equality of each median effect. 

*--Includes one study10 with four additional QI strategies (Clinician Reminders, Organizational Change, Financial incentives for 

Patients, Regulatory Incentives for Clinicians.) No other study utilized any of these QI strategies.
92 

Table 4. Antibiotic treatment studies: QI strategies and study modifiers 

Median effect overall 

(Interquartile range) 

Provider education 

alone 

(N=9) 

Provider education 

and Patient 

Education 

(N=6) 

All comparisons 

(N=15) 

-8.1% 

(-6.5% to -13.7%) 

-8.1% 

(-13.7% to -7.0) 

-7.5% 

(-12.0% to -3.0%) 

Randomized Controlled Trials 

N=7 

-7.0% 

(-13.7% to -6.7%) 

-7.0% 

(-12.0% to -6.8%) 

N=6 

-3.0% 

N=1 

Controlled Before-After Trials 

N=8 

-10.8% 

(-17.7% to -6.8%) 

-9.6% 

(-16.5% to -8.9%) 

N=3 

-12.0% 

(-12.0% to -3.0%) 

N=5 

US-based Studies 

N=3 

-3.0% 

(-12.0% to1.5%) 

N=0 -3.0% 

(-7.5% to -0.3%) 

N=3 

Non-US-based Studies 

N=12 

-8.9% 

(-16.5% to -6.9%) 

-8.1% 

(-13.7% to -7.0) 

N=9 

-12.0% 

(-20.8% to -7.5%) 

N=3 

Effective sample size below 

median 

N=12 

-10.8% 

(-26.0% to -6.8%) 

-9.6% 

(-23.3% to -7.0%) 

N=5 

-12.0% 

(-20.8% to -5.3%) 

N=3 

Effective sample size above 

median 

N=7 

-7.0% 

(-12.0% to -3.0%) 

-7.6% 

(-9.5% to -6.9%) 

N=4 

-3.0% 

(-7.5% to -0.3%) 

N=3 

Baseline prescription rate below 

median 

N=10 

-7.0% 

(-9.6% to -7.0%) 

-10.9% 

(-20.9% to -7.1%) 

N=6 

-7.5% 

(-12.0% to -3.0%) 

N=4 
93 

Table 4. Antibiotic treatment studies: QI strategies and study modifiers (continued) 

Baseline prescription rate above 

median 

N=5 

-10.5% 

(-13.7% to -6.5%) 

-7.0% 

(-8.3% to -7.0%) 

N=3 

-14.0% 

(-29.5% to 1.5%) 

N=2 

Repeated intervention 

N=5 

-8.1% 

(-12.0% to -7.1%) 

-7.0% 

(-7.6% to -7.0%) 

N=3 

12.0% 

(-12.0%, -12.0%) 

N=2 

One-time intervention 

N=10 

8.2% 

(-23.3% to -3.0%) 

-11.7% 

(-20.9% to -7.4%) 

N=6 

-3.0% 

(-9.6% to -1.9%) 

N=4 

Active educational strategy used 

N=7 

-13.7% 

(-28.6% to -6.5%) 

-18.6% 

(-24.5% to -11.9%) 

N=4 

-12.0% 

(-20.8% to -5.3%) 

N=3 

Only passive educational 

strategies used 

N=8 

-7.0% 

(-8.9% to -4.9%) 

-7.1% 

(-8.1% to -7.0%) 

N=5 

-3.0% 

(-7.5% to -0.3%) 

N=3 

Specific disease targeted 

(N=12)

-10.8% 

(-4.5% to -18.4% 

-11.7% 

(-20.9% to -7.4%) 

N=6 

-7.5% 

(-12.0% to -3.0% 

N=6 

No specific disease targeted 

(N=3)

-7.0% 

(-8.1% to -7.0%) 

-7.0% 

(-8.1% to -7.0%) 

N=3 

N=0 

Children targeted 

(N=6)

-10.8% 

(-13.7% to -6.7%) 

-9.6% 

(-11.7% to -8.2%) 

N=3 

-12.0% 

(-20.8% to -5.3%) 

N=3 

Children not targeted 

(N=9)

-7.0% 

(-12.0% to -6.5%) 

-7.6% 

(-19.5% to -6.9%) 

N=6 

-3.0% 

(-7.5% to -3.0% 

(N=3) 

Table 4 includes only the QI strategies used in >3 trials. Trials designated as quasi-Randomized Controlled Trials in Table 2 are 

grouped with Controlled Before-After studies in these analyses. 
94 

Table 5. Results of Wilcoxon rank-sum tests for differences between groups 

Treatment studies 

(N=15)

Selection studies 

(N=19) 

Median effect (%) 

[Interquartile 

range] 

P value for 

comparison 

Median effect (%) 

[Interquartile 

range] 

P value for 

comparison 

RCT 

(n = 7) 

-7.0% 

(-13.7% to -6.7%) RCT 

(n = 8) 

11.1% 

(3.2% to 16.0%) Study design 

Non-RCT 

(n = 8) 

-10.8% 

(-17.7% to -6.8%) 

0.52 

Non-RCT 

(n = 11) 

8.8% 

(3.1% to 22.7%) 

0.62 

Below 

median 

(n = 8) 

-10.8% 

(-26.0% to -6.8%) 

Below 

median 

(n = 7) 

20.4% 

(8.4% to 35.6%) 

Sample Size 

Above 

median 

(n = 7) 

-7.0% 

(-12.0% to -3.0%) 

0.30 

Above 

median 

(n = 12) 

6.2% 

(2.8% to 13.7%) 

0.063 

US 

(n = 3) 

-3.0% 

(-12.0% to1.5%) 

US 

(n = 4) 29.2% 

(14.9% to 41.8%) Country 

Non-US 

(n = 12) 

-8.9% 

(-16.5% to -6.9%) 

0.15 

Non-US 

(n = 15) 

 

8.4% 

(2.5% to 14.9%) 

0.028 

Usual 

care 

(n=15) 

Not performed 

Usual care 

(n=14) 

14.4% 

(7.2% to 20.4%) 

Type of control 

group Limited 

intervention 

(n = 0) 

Not performed 

N/A Limited 

intervention 

(n = 5) 

3.1% 

(2.5% to 7.1%) 

0.041 

Below 

median 

(n=5) 

-7.0% 

(-9.6% to -7.0%) 

Below 

median 

(n=9) 

13.9% 

(3.1% to 17.0%) Baseline 

prescribing 

rate Above 

median 

(n=10) 

-10.5% 

(-13.7% to -6.5%) 

0.82 

Above 

median 

(n=10) 

8.6% 

(4.3% to 20.4%) 

0.87 

Yes 

(n=7) 

-13.7% 

(-28.6% to -6.5%) Yes 

(n=12) 

11.1% 

Use of active (4.9% to 17.1%) 

educational 

strategies No 

(n=8) 

-7.0% 

(-8.9% to -4.9%) 

0.11 

No 

(n=7) 

5.1% 

(3.1% to 22.7%) 

0.87 

Yes 

(n=5) -8.1% 

(-12.0% to -7.1%) 

Yes 

(n=8) 

6.8% 

(3.7% to 14.8%) 

Repeated 

intervention No 

(n=16) 8.2% 

(-23.3% to -3.0%) 

0.81 

No 

(n=11) 13.9% 

(2.5% to 22.7%) 

0.37 
95 

Table 5. Results of Wilcoxon rank-sum tests for differences between groups (continued) 

Yes 

(N=12) -10.8% 

(-4.5% to -18.4% 

Yes 

(N=17) 12.5% 

Specific (3.1% to 18.6%) 

disease 

targeted 

No 

(N=3) -7.0% 

(-8.1% to -7.0%) 

0.66 

No 

(N=2) 3.4% 

(2.5% to 4.3%) 

0.19 

Yes 

(N=6) 

-10.8% 

(-13.7% to -6.7%) 

Yes 

(N=2) 29.2% 

(22.7% to 35.6%) Children 

targeted No 

N=9) 

-7.0% 

(-12.0% to -6.5%) 

0.60 

No 

(N=17) 

8.4% 

(3.1% to 13.9% 

0.05 

Results of nonparametric Wilcoxon rank-sum tests the hypothesis that the median effects in the two groups being compared are 

equal. This analysis is restricted to the subgroup of studies in the two dominant QI strategies: for treatment studies, clinician 

education and clinician education plus patient education; for selection studies, clinician education and clinician education plus 

audit and feedback. Trials designated as quasi-Randomized Controlled Trials in Table 2 are grouped with Controlled BeforeAfter studies in these analyses. 
96 

Table 6. Delayed prescription studies 

Studies using delayed prescriptions are those in which the antibiotic prescription is provided to the patient with instructions to use 

the prescription only if their condition did not improve with supportive measures within a set period of time. These studies did 

not report pre-intervention prescribing rates. In all studies, comparison group patients received antibiotic prescriptions with 

instructions to begin taking antibiotics immediately; the post-intervention control group percentages refer to the percent of 

patients who actually filled the prescription. In Dowell et al, the percentage of control group patients who used the antibiotic 

prescription was not stated. The studies by Macfarlane and Pshetizky also incorporated a brief educational intervention for 

patients. 

Study Target condition Rate of 

antibiotic use 

(comparison 

group) 

Rate of antibiotic 

use (intervention 

group) 

Reduction in 

antibiotic use 

(Interventionpost – 

Controlpost) 

Arroll et al4 Common cold 87.0% 40.3% 46.7% 

Dowell et al8 Uncomplicated ARI 100.0% 45.0% 55.0% 

Little et al19 Pediatric otitis media 98.5% 24.0% 74.5% 

MacFarlane et al20 Acute bronchitis 62.0% 47.0% 15.0% 

Pshetizky et al38 Pediatric otitis media 63.0% 37.0% 26.0% 
97 

Table 7. Antibiotic selection studies: number of QI strategies utilized 

Number of QI strategies Randomized 

Controlled Trial 

Interrupted Time 

Series 

Controlled BeforeAfter Study Total 

One 6 

[

41-43, 48, 49, 54] 

0 13 

[

36, 46, 47, 51, 52, 55, 60-66] 

19 

[

36, 41-43, 46, 47, 48, 49, 51, 

52, 54, 55, 60-66] 

Two 7 

[

53, 56, 57, 67, 68, 70, 71] 

1 

[

58] 

4 

[

34, 50, 69, 73] 

12 

[

34, 50, 53, 56-58, 67-71, 73] 

Three 0 0 2 

[

44, 45] 

2 

[

44, 45] 

Four 0 0 0 0 

Five 0 0 0 0 

Six 0 0 0 0 

More than six 0 0 0 0 

Total 13 

[

41-43, 48, 49, 53, 54, 56, 57, 67, 

68, 70, 71] 

1 

[

58] 

19 

[

34, 36, 44-47, 50-52, 55, 60-66, 69, 

73] 

There were no quasi-RCT’s among the antibiotic selection studies. Interrupted Time Series studies were required to report a 

clearly defined intervention time period, at least three data points before and after the intervention, and use appropriate statistical 

methodology. “Controlled Before-After Study” refers to non-randomized controlled trials with a contemporaneous comparison 

group. 
98 

Table 8. Median effect size of QI strategies on antibiotic selection decision 

 

Median effect with QI strategy 

(Interquartile range)

All comparisons 

(N=22) 

10.6% 

(3.4% to 18.2%) 

Clinician Education alone 

(N=11) 

13.9% 

(8.6% to 21.6%) 

Clinician Education combined 

with Audit and Feedback 

(N=8) 

3.4% 

(1.8% to 9.7%) 

Clinician Education combined 

with Patient Education 

(N=2)* 

22.8% 

(2.4% to 43.1%) 

Audit and Feedback alone 

(N=1) 

13.9% 

p=0.182 for comparison across 

QI strategies 

p=0.028 for comparison of 

clinician education alone with 

clinician education and audit and 

feedback 

Percentages demonstrate the net improvement in selection of the appropriate antibiotic. Values represent the median effect 

across studies with each QI strategy (or combination of strategies) with interquartile ranges. (For clinician education combined 

with audit and feedback, values in parenthesis represent the effects of the two studies in question.) Median effects were 

compared using the nonparametric Kruskal-Wallis test, which evaluates the null hypothesis that the observed effects are from the 

same study population. Individual study effect sizes calculated as: 

Net ∆ in prescribing recommended antibiotic = (Post-intervention recommended prescribing percentage – Pre-intervention 

recommended prescribing percentage)intervention group - (Post-intervention recommended prescribing percentage – Pre-intervention 

recommended prescribing percentage)control group. 
99 

Table 9. Selection studies: QI strategies and study modifiers 

Median effect 

overall 

(Interquartile range) 

Provider 

Education alone 

(N=11) 

Provider Education 

and Audit and 

Feedback 

(N=8) 

All comparisons 

(N=19) 

8.8% 

(3.1% to 18.6%) 

13.9% 

(8.4% to 22.7%) 

3.4% 

(1.5% to 12.1%) 

Randomized Controlled Trials 

N=8 

11.1% 

(3.2% to 16.0%) 

13.9% 

(11.2% to 14.4%) 

N=3 

4.3% 

(2.0% to 17.0%) 

N=5 

Controlled Before-After Trials 

N=11 

8.8% 

(3.1% to 22.7%) 

16.5% 

(7.8% to 25.9%) 

N=8 

2.5% 

(2.5% to 4.7%) 

N=3 

US-based studies 

N=4 

29.2% 

(15.0% to 41.8%) 

35.6% 

(29.2% to 41.8%) 

N=3 

7.2% 

N=1 

Non-US-based studies 

N=15 

8.4% 

(2.5% to 14.9%) 

10.6% 

(7.6% to 14.2%) 

N=8 

4.9% 

(1.4% to 14.6%) 

N=7 

Effective sample size below 

median 

N=7 

20.4% 

(8.4% to 35.6%) 

21.6% 

(15.5% to 32.4%) 

N=6 

-7.8% 

N=1 

Effective sample size above 

median 

N=12 

6.2% 

(2.8% to 13.7%) 

8.8% 

(5.1% to 12.5%) 

N=5 

4.3% 

(2.1% to 12.1%) 

N=7 

Baseline compliance below median 

N=9 

13.9% 

(3.1% to 17.0%) 

13.9% 

(5.1% to 14.9%) 

N=5 

9.8% 

2.4% to 17.4% 

N=4 
100 

Table 9. Selection studies: QI strategies and study modifiers (continued)

Baseline compliance above 

median 

N=10 

8.6% 

(4.3% to 20.4%) 

16.5% 

(9.1% to 22.5%) 

N=6 

2.7% 

(-1.2% to 5.0%) 

N=4 

Repeated intervention 

N=8 

6.8% 

(3.7% to 14.8%) 

6.8% 

(4.6% to 9.4%) 

N=4 

10.7% 

(3.5% to 17.0%) 

N=4 

One-time intervention 

N=11 

13.9% 

(2.5% to 22.7%) 

20.4% 

(14.4% to 29.2%) 

N=7 

2.3% 

(-0.4% to 3.7%) 

N=4 

Active provider educational 

strategy used 

N=12 

11.4% 

(4.8% to 17.8%) 

14.4% 

(10.0% to 19.0%) 

N=6 

4.9% 

(1.4% to 14.6%) 

N=6 

Only passive provider educational 

strategies used 

N=7 

5.1% 

(3.1% to 22.7%) 

12.5% 

(5.1% to 22.7%) 

N=5 

3.2% 

(0.2% to 4.3%) 

N=2 

Control group received no 

intervention 

N=14 

14.4% 

(7.2% to 20.4% 

17.7% 

(3.6% to 25.9%) 

N=8 

5.8% 

(1.8% to 14.6%) 

N=6 

Control group received lowintensity intervention 

N=5 

3.1% 

(2.1% to 5.1%) 

5.1% 

(4.1% to 6.8%) 

N=3 

2.3% 

(2.0% to 2.5%) 

N=2 

Studies targeting a specific 

disease 

N=17 

12.5% 

(3.1% to 18.6%) 

13.9% 

(8.6% to 21.6%) 

N=11 

4.3% 

(1.3% to 14.6%) 

N=6 

Studies not targeting a specific 

disease 

N=2 

3.4% 

(2.5% to 4.3%) 

N=0 3.4% 

(3.0% to 3.9%) 

N=2 

Studies targeting children 

N=2 

29.2% 

(22.7% to 35.6%) 

29.2% 

(22.7% to 35.6%) 

N=2 

N=0 
101 

Table 9. Selection studies: QI strategies and study modifiers (continued) 

Studies not targeting children 

N=17 

8.4% 

(3.1% to 13.9% 

12.5% 

(8.4% to 14.9%) 

N=9 

3.4% 

(1.8% to 9.7%) 

N=8 

Table includes only the QI strategies used in >3 trials. 

103 

Table References 

1. Finkelstein JA, Davis RL, Dowell SF, et al. Reducing antibiotic use in children: a randomized trial in 12 

practices. Pediatr. 2001;108:1-7. 

2. Angunawela I, Diwan VK, Tomson G. Experimental evaluation of the effects of drug information on 

antibiotic prescribing: a study in outpatient care in an area of Sri lanka. International Journal of 

Epidemiology. 1991;20(2):558. 

3. Angunawela I, Diwan VK, Tomson G. [Additional comparison 1] Experimental evaluation of the effects of 

drug information on antibiotic prescribing: a study in outpatient care in an area of Sri lanka. International 

Journal of Epidemiology. 1991;20(2):558. 

4. Arroll B, Kenealy T, Kerse N. Do Delayed Prescriptions Reduce the Use of Antibiotics for the Common 

Cold? J Fam Pract. 2002;51(4):324-328. 

5. Belongia EA, Sullivan BJ, Chyou PH, Madagame E, Reed KD, Schwartz B. A community intervention trial 

to promote judicious antibiotic use and reduce penicillin-resistant Streptococcus pneumoniae carriage in 

children. Pediatrics. 2001;108(3):575. 

6. Bexell A, Lwando E, von Hofsten B, Tembo S, Eriksson B, Diwan VK. Improving drug use through 

continuing education: a randomized controlled trial in Zambia. Journal of Clinical Epidemiology. 

1996;49:355. 

7. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled 

trial of point-of-care evidence to improve the antibiotic prescribing practices for otitis media in children. 

Pediatrics. 2001;107(2). 

8. Dowell J, Pitkethly M, Bain J, Martin S. A randomised controlled trial of delayed antibiotic prescribing as a 

strategy for managing uncomplicated respiratory tract infection in primary care. Br J Gen Pract. 

2001;51(3):200-205. 

9. Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored 

interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 

2002;325:7360. 

10. Foxman B, Valdez RB, Lohr KN, Goldberg GA, Newhouse JP, Brook RH. The effect of cost sharing on the 

use of antibiotics in ambulatory care: results from a population-based randomized controlled trial. Journal of 

Chronic Diseases. 1987;40(5):429. 

11. Gonzales R, Steiner JF, Lum A, Barrett PHJ. Decreasing antibiotic use in ambulatory practice: impact of a 

multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 

1999;281(16):1512-1519. 

12. Gonzales R, Steiner JF, Lum A, Barrett PHJ. [Additional comparison 1] Decreasing antibiotic use in 

ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute 

bronchitis in adults. JAMA. 1999;281(16):1512-1519. 

13. Gonzalez Ochoa E, Armas Perez L, Bravo Gonzalez JR, Cabrales Escobar J, Rosales Corrales R, Abreu 

Suarez G. Prescription of antibiotics for mild acute respiratory infections in children. Bulletin of the Pan 

American Health Organization. 1996;30(2):106. 
104 

14. Gonzalez Ochoa E, Armas Perez L, Bravo Gonzalez JR, Cabrales Escobar J, Rosales Corrales R, Abreu 

Suarez G. [Additional comparison 1] Prescription of antibiotics for mild acute respiratory infections in 

children. Bulletin of the Pan American Health Organization. 1996;30(2):106. 

15. Gonzalez Ochoa E, Armas Perez L, Bravo Gonzalez JR, Cabrales Escobar J, Rosales Corrales R, Abreu 

Suarez G. [Additional comparison 2] Prescription of antibiotics for mild acute respiratory infections in 

children. Bulletin of the Pan American Health Organization. 1996;30(2):106. 

16. Gutierrez G, Guiscafre H, Bronfman M, Walsh J, Martinez H, Munoz O. Changing physician prescribing 

patterns: evaluation of an educational strategy for acute diarrhea in Mexico City. Medical Care. 

1994;32(5):436. 

17. Hennessy TW, Petersen KL, Bruden D, et al. Changes in Antibiotic-Prescribing Practices and Carriage of 

Penicillin-Resistant Streptococcus pneumoniae: A Controlled Intervention Trial in Rural Alaska. Clin Infect 

Dis. 2002;34:1543-1550. 

18. Hickman DE, Stebbins MR, Hanak JR, Guglielmo BJ. Pharmacy-based intervention to reduce antibiotic use 

for acute bronchitis. Annals of Pharmacotherapy. 2003;37(2):187. 

19. Little P, Gould C, Williamson I, Moore M, Warner G, Dunleavey J. Pragmatic randomised controlled trial of 

two prescribing strategies for childhood acute otitis media. BMJ. 2001;322(7282):336-342. 

20. Macfarlane J, Holmes W, Gard P, et al. Reducing antibiotic use for acute bronchitis in primary care: blinded, 

randomised controlled trial of patient information leaflet * Commentary: More self reliance in patients and 

fewer antibiotics: still room for improvement. BMJ. 2002;324(7329):91-. 

21. Mainous AG, 3rd, Hueston WJ, Love MM, Evans ME, Finger R. An evaluation of statewide strategies to 

reduce antibiotic overuse. Family Medicine. 2000;32(1):22. 

22. Mainous AG, 3rd, Hueston WJ, Love MM, Evans ME, Finger R. [Additional comparison 1] An evaluation of 

statewide strategies to reduce antibiotic overuse. Family Medicine. 2000;32(1):22. 

23. Mainous AG, 3rd, Hueston WJ, Love MM, Evans ME, Finger R. [Additional comparison 2] An evaluation of 

statewide strategies to reduce antibiotic overuse. Family Medicine. 2000;32(1):22. 

24. McConnell TS, Cushing AH, Bankhurst AD, Healy JL, McIlvenna PA, Skipper BJ. Physician behavior 

modification using claims data: tetracycline for upper respiratory infection. West J Med. 1982;137(5):448-

450. 

25. McIsaac WJ, Goel V. Effect of an explicit decision-support tool on decisions to prescribe antibiotics for sore 

throat. Medical Decision Making. 1998;18(2):220. 

26. McIsaac WJ, Goel V, To T, Permaul JA, Low DE. Effect on antibiotic prescribing of repeated clinical 

prompts to use a sore throat score: lessons from a failed community intervention study. Journal of Family 

Practice. 2002;51(4):339. 

27. Melander E, Bjorgell A, Bjorgell P, Ovhed I, Molstad S. Medical audit changes physicians' prescribing of 

antibiotics for respiratory tract infections. Scandinavian Journal of Primary Health Care. 1999;17(3):180. 

28. Meyer JC, Summers RS, Moller H. Randomized, controlled trial of prescribing training in a South African 

province. Med Educ. 2001;35(9):833-840. 

29. O'Connell DL, Henry D, Tomlins R. Randomised controlled trial of effect of feedback on general 

practitioners' prescribing in Australia. BMJ. 1999;318(7182):507-511. 
105 

30. Perz JF, Craig AS, Coffey CS, et al. Changes in antibiotic prescribing for children after a community-wide 

campaign. JAMA. 2002;287(23):3103. 

31. Santoso B, Suryawati S, Prawaitasari JE. Small group intervention vs formal seminar for improving 

appropriate drug use published erratum appears in Soc Sci Med 1996 Jul;43(1):I. Social Science & Medicine. 

1996;42(8):1163. 

32. Santoso B, Suryawati S, Prawaitasari JE. [Additional comparison 1] Small group intervention vs formal 

seminar for improving appropriate drug use published erratum appears in Soc Sci Med 1996 Jul;43(1):I. 

Social Science & Medicine. 1996;42(8):1163. 

33. Zwar N, Wolk J, Gordon J, Sanson-Fisher R, Kehoe L. Influencing antibiotic prescribing in general practice: 

A trial of prescriber feedback and management guidelines. Family Practice. 1999;16(5):495. 

34. McNulty CA, Kane A, Foy CJ, Sykes J, Saunders P, Cartwright KA. Primary care workshops can reduce and 

rationalize antibiotic prescribing. Journal of Antimicrobial Chemotherapy. 2000;46(3):493. 

35. Welschen I, Kuyvenhoven MM, Hoes AW, Verheij TJ. Effectiveness of a multiple intervention to reduce 

antibiotic prescribing for respiratory tract symptoms in primary care: randomised controlled trial. Bmj. 

2004;329(7463):431. 

36. Molstad S, Hovelius. Reduction in antibiotic usage following an educational programme. Family Practice. 

1989;6(1):33-37. 

37. Cates C. An evidence based approach to reducing antibiotic use in children with acute otitis media: controlled 

before and after study. Bmj. 1999;318(7185):715-716. 

38. Pshetizky Y, Naimer S, Shvartzman P. Acute otitis media--a brief explanation to parents and antibiotic use. 

Fam Pract. 2003;20(4):417-419. 

39. Smabrekke L, Berild D, Giaever A, et al. Educational intervention for parents and healthcare providers leads 

to reduced antibiotic use in acute otitis media. Scand J Infect Dis. 2002;34(9):657-659. 

40. Wilson EJ, Nasrin D, Dear KB, Douglas RM. Changing GPs' antibiotic prescribing: a randomised controlled 

trial. Commun Dis Intell. 2003;27 Suppl:S32-38. 

41. Coenen S, Van Royen P, Michiels B, Denekens J. Optimizing antibiotic prescribing for acute cough in 

general practice: a cluster-randomized controlled trial. J Antimicrob Chemother. 2004;54(3):661-672. 

42. Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach. A randomized 

controlled trial of academically based "detailing". New England Journal of Medicine. 1983;308(24):1457-

1463. 

43. Avorn J, Soumerai SB. [Additional comparison 1] Improving drug-therapy decisions through educational 

outreach. A randomized controlled trial of academically based "detailing". New England Journal of Medicine. 

1983;308(24):1457-1463. 

44. Braybrook S, Walker R. Influencing prescribing in primary care: a comparison of two different prescribing 

feedback methods. Journal of Clinical Pharmacy & Therapeutics. 1996;21(4):247. 

45. Braybrook S, Walker R. [Additional comparison 1] Influencing prescribing in primary care: a comparison of 

two different prescribing feedback methods. Journal of Clinical Pharmacy & Therapeutics. 1996;21(4):247. 

46. Bush PJ, Rabin DL, Spector KK. Evaluation of a drug therapy protocol in an HMO. Medical Care. 

1979;17(6):566. 
106 

47. Bush PJ, Rabin DL, Spector KK. [Additional comparison 1] Evaluation of a drug therapy protocol in an 

HMO. Medical Care. 1979;17(6):566. 

48. De Santis G, Harvey KJ, Howard D, Mashford ML, Moulds RF. Improving the quality of antibiotic 

prescription patterns in general practice. The role of educational intervention. Medical Journal of Australia. 

1994;160(8):502. 

49. Dolovich L, Levine M, Tarajos R, Duku E. Promoting optimal antibiotic therapy for otitis media using 

commercially sponsored evidence-based detailing: A prospective controlled trial. Drug Information Journal. 

1999;33(4):1067. 

50. Farris KB, Kirking DM, Shimp LA, Opdycke RA. Design and results of a group counter-detailing DUR 

educational program. Pharmaceutical Research. 1996;13(10):1445. 

51. Friis H, Bro F, Mabeck CE, Vejlsgaard R. Changes in prescription of antibiotics in general practice in relation 

to different strategies for drug information. Danish Medical Bulletin. 1991;38(4):380. 

52. Friis H, Bro F, Mabeck CE, Vejlsgaard R. [Additional comparison 1] Changes in prescription of antibiotics in 

general practice in relation to different strategies for drug information. Danish Medical Bulletin. 

1991;38(4):380. 

53. Hux JE, Melady MP, DeBoer D. Confidential prescriber feedback and education to improve antibiotic use in 

primary care: a controlled trial. CMAJ. 1999;161(4):388. 

54. Ilett KF, Johnson S, Greenhill G, et al. Modification of general practitioner prescribing of antibiotics by use 

of a therapeutics adviser (academic detailer). British Journal of Clinical Pharmacology. 2000;49(2):168. 

55. Klein LE, Charache P, Johannes RS. Effect of physician tutorials on prescribing patterns of graduate 

physicians. Journal of Medical Education. 1981;56(6):504. 

56. Lagerlov P, Loeb M, Andrew M, Hjortdahl P. Improving doctors' prescribing behaviour through reflection on 

guidelines and prescription feedback: a randomised controlled study. Quality in Health Care. 2000;9(3):159. 

57. Lundborg CS, Wahlstrom R, Oke T, Tomson G, Diwan VK. Influencing prescribing for urinary tract 

infection and asthma in primary care in Sweden: a randomized controlled trial of an interactive educational 

intervention. Journal of Clinical Epidemiology. 1999;52(8):801. 

58. MacCara ME, Sketris IS, Comeau DG, Weerasinghe SD. Impact of a limited fluoroquinolone reimbursement 

policy on antimicrobial prescription claims. Annals of Pharmacotherapy. 2001;35(7-8):852. 

59. McNulty CA, Kane A, Foy CJ, Sykes J, Saunders P, Cartwright KA. [Additional comparison 1] Primary care 

workshops can reduce and rationalize antibiotic prescribing. Journal of Antimicrobial Chemotherapy. 

2000;46(3):493. 

60. Peterson GM, Stanton LA, Bergin JK, Chapman GA. Improving the prescribing of antibiotics for urinary tract 

infection. Journal of Clinical Pharmacy & Therapeutics. 1997;22(2):147. 

61. Raz R, Porat V, Ephros M. Can an educational program improve the diagnosis and treatment of 

pharyngotonsillitis in the ambulatory care setting? Israel Journal of Medical Sciences. 1995;31(7):432. 

62. Rokstad K, Straand J, Fugelli P. Can drug treatment be improved by feedback on prescribing profiles 

combined with therapeutic recommendations? A prospective, controlled trial in general practice. Journal of 

Clinical Epidemiology. 1995;48(8):1061-1068. 

63. Schaffner W, Ray WA, Federspiel CF, Miller WO. Improving antibiotic prescribing in office practice. A 

controlled trial of three educational methods. JAMA. 1983;250(13):1728. 
107 

64. Schaffner W, Ray WA, Federspiel CF, Miller WO. [Additional comparison 1] Improving antibiotic 

prescribing in office practice. A controlled trial of three educational methods. JAMA. 1983;250(13):1728. 

65. Schaffner W, Ray WA, Federspiel CF, Miller WO. [Additional comparison 2] Improving antibiotic 

prescribing in office practice. A controlled trial of three educational methods. JAMA. 1983;250(13):1728. 

66. Schaffner W, Ray WA, Federspiel CF, Miller WO. [Additional comparison 3] Improving antibiotic 

prescribing in office practice. A controlled trial of three educational methods. JAMA. 1983;250(13):1728. 

67. Sondergaard J, Andersen M, Stovring H, Kragstrup J. Mailed prescriber feedback in addition to a clinical 

guideline has no impact: a randomised, controlled trial. Scandinavian Journal of Primary Health Care. 

2003;21(1):47. 

68. Veninga CC, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Improving drug treatment in general practice. 

Journal of Clinical Epidemiology. 2000;53(7):762. 

69. Melander E, Bjorgell A, Bjorgell P, Ovhed I, Molstad SAc. Medical audit changes physicians' prescribing of 

antibiotics for respiratory tract infections. Scandinavian Journal of Primary Health Care. 1999;17(3):180. 

70. Harrison A, Karim SA, Floyd K, et al. Syndrome packets and health worker training improve sexually 

transmitted disease case management in rural South Africa: randomized controlled trial. AIDS. 

2000;14(17):2769. 

71. Zwar N, Wolk J, Gordon J, Sanson-Fisher R, Kehoe LAc. Influencing antibiotic prescribing in general 

practice: A trial of prescriber feedback and management guidelines. Family Practice. 1999;16(5):495. 

72. Molstad S, Hovelius. [Additional comparison 1] Reduction in antibiotic usage following an educational 

programme. Family Practice. 1989;6(1):33-37. 

73. Smabrekke L, Berild D, Giaever A, et al. [Additional comparison 1] Educational intervention for parents and 

healthcare providers leads to reduced antibiotic use in acute otitis media. Scand J Infect Dis. 2002;34(9):657-

659. 

74. Coenen S, Van Royen P, Michiels B, Denekens J. [Additional comparison 1] Optimizing antibiotic 

prescribing for acute cough in general practice: a cluster-randomized controlled trial. J Antimicrob 

Chemother. 2004;54(3):661-672.
APPENDIXES 

to 

“Closing the Quality Gap: A Critical Analysis 

of Quality Improvement Strategies 

Volume 4—Antibiotic Prescribing Behavior” 

Prepared by the Stanford-UCSF Evidence-based Practice Center 

(Contract #290-02-0017)
A-1 

Appendix A. Summary of Key Studies 

Antibiotic Treatment Decision Studies

Finkelstein et al: Reducing antibiotic use in children: a randomized trial in 12 

practices (Pediatrics 2001; 108(1): 1-7) 

This quasi-randomized controlled trial, conduced in 12 Managed Care Organizations 

(MCOs) in Massachusetts and Washington state, demonstrates the effectiveness of 

repeated, active clinician education in combination with parent education in reducing 

antibiotic prescribing to children. 

In this trial, practices were randomly assigned to intervention or comparison groups 

(after stratification by size and baseline prescribing rate). The clinician intervention 

consisted of academic detailing sessions, led by a practicing pediatrician “peer 

leader”; at these interactive 90-minute small-group sessions, the groups engaged in 

discussions on potential ways to prevent overuse of antibiotics. Approximately 4 

months later, the peer leaders conducted another educational session at which the 

groups received feedback on their practice- and clinician-specific prescribing 

practices. In the parent intervention (conducted contemporaneously), each family in 

the MCO was mailed CDC-produced pamphlets,44 reinforced by pamphlets and 

posters in waiting rooms. The control groups did not receive either clinician or 

parent education. 

During this trial, which was conducted in the late 1990s, there was a substantial 

secular trend toward reduced antibiotic prescribing; in the comparison groups, 

antibiotic prescribing declined by 0.17 – 0.33 prescriptions per person-year (for 

children 36 - 72 months old and 3 – 36 months old, respectively). Despite this, the 

intervention still achieved a 12 – 16% relative reduction in prescribing rates in the 

two groups. At the practice level, this resulted in absolute reductions of 0.23 fewer 

antibiotic courses per person-year for children aged 3 - 36 months, and 0.13 fewer 

antibiotic courses per person-year for children aged 36-72 months. To arrive at 

these results, the investigators carried out statistical analyses accounting for 

clustering of patients within clinics. 

This study used an intensive clinician and patient education intervention to achieve 

significant reductions in antibiotic prescribing in a managed care setting. This trial 

clearly demonstrates the value of active, repeated clinician education. It also 

demonstrates that a quality improvement intervention can successfully reduce 

inappropriate prescribing even while prescribing rates are declining overall. 

Gonzales et al: Decreasing antibiotic use in ambulatory practice: Impact of a 

multidimensional intervention on the treatment of uncomplicated acute 

bronchitis in adults (JAMA 1999; 281: 1512-1519) 

This study provides an example of a rigorously conducted, comprehensive 

intervention targeting clinicians and patients in a health maintenance organization 

(HMO). Conducted in Kaiser Permanente clinics in Denver, Colorado, this non-
A-2 

randomized controlled trial targeted antibiotic prescribing for acute bronchitis in 

adults. 

The study contained three sites: a “full intervention” site, a “limited intervention” site, 

and a control site. Before the trial, the investigators conducted preliminary studies to 

identify why clinicians prescribed antibiotics and why patients expected antibiotic 

treatment for bronchitis. The intervention was then targeted to specifically address 

these factors. In both intervention sites, each enrolled patient household received 

mailed educational materials (such as refrigerator magnets and pamphlets). Officebased educational materials directed at clinicians and patients included colorful 

posters and pamphlets placed in waiting rooms and examination rooms. The 

importance of reducing inappropriate antibiotic prescribing was emphasized through 

a letter to all patients from the medical director of the organization. The “limited 

intervention” site received only the office-based and household educational 

materials; the “full intervention” site received the office and household educational 

materials as well as a clinician education session, a one-time educational 

intervention conducted at regularly scheduled staff meetings. This intervention 

consisted of education on the evidence-based management of acute bronchitis, 

advice on communication skills to deal with patient expectations, and practicespecific feedback on baseline prescribing rates. This intervention was conducted by 

the medical director of the clinic. 

The study was conducted in a methodologically rigorous fashion; in addition to 

conducting preliminary studies to identify a specific quality gap and target the 

intervention, the study used a comparable comparison group with contemporaneous 

measurements, with outcomes assessors blinded to treatment group assignment. 

The statistical analysis took into account potential clustering of clinician prescribing 

practices by site. 

The full intervention significantly reduced antibiotic prescribing. At baseline, 74% of 

patients received antibiotics for acute bronchitis; this fell to 48% after the 

intervention. Little change in prescribing rates was seen in both the limited 

intervention site and control site (prescribing rates went from 78% to 76% and 82% 

to 77%, respectively). In addition to measuring prescribing rates, the investigators 

also assessed the safety of the intervention by measuring the percentage of patients 

diagnosed with pneumonia at a return visit (there was no difference between 

groups). They also addressed potential confounders such as “code shifting.” 

(Theoretically, an intervention to reduce antibiotic prescribing for bronchitis could 

have resulted in clinicians shifting diagnoses to conditions more likely to warrant 

antibiotics, such as acute sinusitis.) They found no evidence of it.

This study highlights several key intervention characteristics. First, the use of 

preliminary studies to identify reasons for inappropriate prescribing allowed the 

investigators to tailor their intervention appropriately. While the purely passive 

limited intervention had no effect, when the more active clinician education session 

was added (the full intervention), the results were striking. Pairing the focused 

clinician intervention with high-visibility strategies such as exam room posters likely 

helped reinforce the message to clinicians. The involvement of high-level 

management may also have helped the success of the intervention. Finally, the 

methodological rigor of the study as well as the attention paid to documenting the 
A-3 

absence of potentially harmful consequences of the intervention reinforces the 

conclusions. 

Perz et al: Changes in antibiotic prescribing for children after a communitywide campaign (JAMA 2002; 287(23): 3103-3109) 

This study demonstrates the effectiveness of a community-wide campaign to reduce 

inappropriate antibiotic prescribing to children for respiratory infections. The 

intervention in this non-randomized controlled trial took place in Knox County, 

Tennessee, with a control group consisting of three other Tennessee counties. The 

study measured the effect on antibiotic prescriptions for children under 15 years of 

age enrolled in the Tennessee Medicaid Managed Care Program (TennCare). 

The intervention targeted clinicians, parents, and the general public through a 

multifaceted approach. The main goals of the intervention were to reduce antibiotic 

prescribing for non-bacterial illnesses, particularly acute respiratory infections (ARIs); 

and to increase the prescribing of narrow-spectrum antibiotics for bacterial illnesses. 

Clinicians received an intensive educational intervention consisting of lectures and 

presentations in a variety of settings (hospital staff meetings, grand rounds, CME 

seminars, and resident conferences). Prescribing guidelines for ARIs were 

distributed to clinicians, and articles on the campaign were mailed to all physicians in 

Knox County. Parent education pamphlets (developed by the CDC44 were 

distributed to parents of all children in day care and grades K-3, as well as to parents 

of all newborns. Clinicians also received patient education materials to distribute in 

their offices. An effort to reach the entire population was made by distributing over 

100,000 pamphlets to hospitals, clinics, dental offices, and pharmacies; the 

campaign was further publicized through mass media efforts such as television, 

radio, and newspaper public service announcements. The control counties engaged 

in no organized efforts to reduce or rationalize antibiotic prescribing during this time. 

Investigators used negative binomial regression models to determine the effect of 

the intervention on adjusted prescription rates, with a random-effect model used to 

account for heterogeneity within the counties. The visit rates for respiratory illnesses 

was measured as well, to account for the possibility that changes in prescribing 

could be confounded by changes in visit rates. The rates of antimicrobial resistance 

among cases of invasive Streptococcus pneumoniae infection were also measured. 

After the year-long intervention, an 11% intervention-attributable decline in total 

antibiotic prescriptions for children was found in the intervention county. This 

translates to an approximate savings of 23 antibiotic prescriptions per 1000 children 

per year. No change in respiratory illness visit rates was found, indicating that 

physicians were truly prescribing antibiotics to a smaller proportion of patients. 

Antimicrobial resistance rates were high at baseline (>50% for penicillin) and did not 

change over the 3-year study period. 

The investigators multifaceted approach, targeting the general public, parents, and 

clinicians, achieved impressive reductions in antibiotic prescribing. This 

demonstrates the viability of a community-based campaign to reduce general 

antibiotic prescribing in an area where baseline prescribing rates were high. This 

study was one of the few to use mass media advertisements to educate the public 
A-4 

on the problem of inappropriate prescribing. The study was methodologically sound, 

using appropriate models to determine the effect of the intervention. 

Antibiotic selection studies

Hux et al: Confidential prescriber feedback and education to improve 

antibiotic use in primary care: a controlled trial (Canadian Medical Association 

Journal 1999; 161: 388-92) 

Conducted in Ontario in the mid-to-late 1990s, this randomized controlled trial 

provides a case example of the effectiveness of passively-delivered clinician 

feedback and education. 

Study investigators provided mailed packages of feedback on antibiotic prescribing 

practices (based on claims data obtained from the province’s prescription drug 

claims database), accompanied by brief educational bulletins to primary care 

physicians. These packages were sent every 2 months over a 6-month period. The 

goal of the intervention was to encourage use of first-line antibiotics as suggested by 

guidelines written by a provincial panel. The comparison group was consented but 

was provided no materials during the study period, although it was promised 

feedback information after completion of the study. 

Among the 250 participating physicians, the proportion of visits in which a first-line 

antibiotic was prescribed was 67.2% in the intervention group and 68.5% in the 

comparison group at baseline. After implementation of the intervention, the 

proportion of visits involving first-line antibiotic prescriptions rose by 2.6% in the 

intervention group, compared with a 1.7% decline in the comparison group, a 

statistically significant difference. 

This study highlights the general finding that passively-delivered feedback and 

education can result in statistically significant improvements in antibiotic selection, 

but that the magnitude of the change tends to be small. 

MacCara et al: Impact of a limited fluoroquinolone reimbursement policy on 

antimicrobial prescription claims (Annals of Pharmacotherapy 2001; 35: 852-8) 

This study is a case example of the potential power of regulatory interventions. In 

this interrupted time series analysis conducted in the mid-to-late 1990s in Nova 

Scotia, the investigators evaluated the impact of a change in provincial guidelines 

governing the reimbursement of fluoroquinolones for elders aged 65 and older 

enrolled in the province’s pharmaceutical care program. 

In early 1997, new guidelines were instituted that required completion of additional 

paperwork certifying that the drug was being prescribed for one of several guidelineapproved uses, which was delivered with the prescription to the pharmacy. Data 

was obtained from the province’s drug plan claims database. 

In the 12 months before and after institution of the policy, fluoroquinolone use fell 

from 20.2% to 4.2% of antibiotic prescriptions for elders, a relative reduction of 80%. 
A-5 

This reduction was immediate. Use of other antibiotics rose, such that overall 

antibiotic use remained stable. 

This study demonstrates the power of regulatory interventions, which in this case 

resulted in an immediate and dramatic decline in fluoroquinolone use. Like other 

research in this area, this study did not report the impact of the intervention on 

quality indicators or on potential patient harms, for example the number of patients 

whose clinical situation merited a fluoroquinolone but who did not receive the drug. 

Schaffner et al: Improving antibiotic prescribing in office practice: a 

controlled trial of three educational methods. (JAMA 1983; 250: 1728-1732) 

Conducted in the early 1980s in Tennessee, this non-randomized controlled study 

provides important insight into the relative value of different forms of educational 

outreach and mailed materials. 

The investigators delivered three separate interventions to 300 physicians in 

geographically distinct areas of the state: (1) a mailed brochure; (2) visits by a 

pharmacist who was trained as a “drug educator,” and (3) visits by physicians who 

were trained as “physician counselors.” Each type of visit lasted less than 15 

minutes. A fourth area of the state served as a comparison group. The goal of the 

interventions was to reduce use of non-recommended antibiotics, and to reduce use 

of the oral cephalosporins (as a cost-saving measure). Data on antibiotic utilization 

were measured using Medicaid claims data for the year before and after the 

intervention. 

The prescribing practices of physicians receiving the mailed brochure were very 

similar to the comparison group, and the two groups were combined for analyses. In 

this combined comparison group, use of both types of medications declined 

substantially (41% relative reduction in prescriptions for non-recommended 

antibiotics, and 33% for oral cephalosporins). In the intervention arms, the use of 

physician counselors was the most effective intervention, resulting in an 85% relative 

reduction in non-recommended antibiotic use, and a 50% reduction in oral 

cephalosporin use. Pharmacist drug educators were less effective in reducing the 

use of non-recommended antibiotics (67% relative reduction); they were no more 

effective than mailed brochures in reducing the use of oral cephalosporins (35%). 

This study highlights several important points. First, it exemplifies the general finding 

that active educational outreach is more effective than passive forms of education 

(such as mailed brochures). Second, it highlights that not all forms of outreach are 

equally effective; what is unclear is whether the differences were due to the relative 

effectiveness of a physician vs. a pharmacist in general, or due to individual 

characteristics of the persons conducting the outreach. Third, this study highlights 

the importance of controlling for secular trends in antibiotic use, which may be 

substantial. Finally, it provides a model for examining different types of interventions 

head-to-head, thereby addressing the problems intrinsic to inter-study comparisons. 
B-1 

Appendix B. Summaries of Included Studies, Grouped by SettinMeasured Population Table 1. Antibiotic treatment studies conducted in the U.S. Studies targeting prescribing for children 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Finkelstein JA, Pediatrics, 2001 

United States 12 practices affiliated with 2 managed care organizations (located in northwest Washington state and eastern Massachusetts) 

Children in Managed Care Organizations with respiratory infections. 

Clinician education (distribution of materials, 

education meetings), Patient education (distribution of 

materials), Audit and feedbackThe intervention consisted of small group provider education sessions with a peer leader and a reinforcing visit four months after the start of the intervention. Providers also received feedback on practitioner and practice-level antibiotic prescribing rates. Parent education consisted of a mailed CDC-produced brochure with a cover letter signed by their pediatrician. Educational pamphlets and posters were also present in the waiting room and examination rooms of the intervention practices. 

Overall antibioby 0.08 coursechildren aged per child per ymonths. 

Belongia EA, Pediatrics, 2001 

United States Multiple counties in Northern Wisconsin 

Children in managed care and fee-forservice practices with respiratory infections. 

Clinician education (distribution of materials, educational 

meetings, educational outreach), Patient education(distribution of materials) This study was a community-based trial with patient and provider education targeting antibiotic prescribing for ARIs. Patient education involved distribution of educational materials to clinics, daycare facilities, and schools. Providers received academic detailing in small groups, led by study authors and lasting 30-60 minutes. At the meetings, clinical practice guidelines, educational materials from CDC (on appropriate antibiotic use), and parent education pamphlets were distributed. Project nurses visited each practice to educate office staff as well. 

Absolute effecThe percentagantibiotics for intervention rethe prescribing Although the preceiving antibwas not statistthe total numb(for both solid decreased signregion compar
B-2 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Christakis DA, Pediatrics, 2001 

United States Academic pediatric primary care clinic 

Children receiving care at academic primary care clinic with otits media. 

Clinician reminder Providers received an electronic evidence-based reminder to avoid treating otitis media with antibiotics, and if using antibiotics to treat for less than 10 days. Reminder was electronic in nature, incorporated into existing electronic prescription program. When providers selected an antibiotic to prescribe, a pop-up screen displayed information on appropriate use and appropriate dosage. The pop-up window contained options for receiving more information.

There was no percentage of either the interstatistically sigprescribing at therapy was fo

Mainous AG, Family Medicine, 2000 [comparison 1]

United States 8 Medicaid administrative regions in Kentucky 

Children in Medicaid program with respiratory infections 

Patient education (distribution of materials) The intervention consisted of a statewide program that used direct patient education to reduce antibiotic prescribing for pediatric respiratory infections. Each physician received 25 pamphlets that they were to distribute themselves to patients with upper respiratory infections. 

Absolute effecIn the intervenincreased from study period. from 31.0% to not statistically 

Mainous AG, Family Medicine, 2000 [comparison 2]

United States 8 Medicaid administrative regions in Kentucky 

Children in Medicaid program with respiratory infections 

Audit and feedback The intervention consisted of a statewide program that used physician feedback. Physicians received a prescribing profile, consisting of the percent of patients with pediatric respiratory conditions to whom they prescribed antibiotics and information on total and relative costs. It also contained their percentile ranking for antibiotics prescribing compared to peers.

Absolute effecIn the intervenincreased from study period. from 31.0% to not statistically 
B-3 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Mainous AG, Family Medicine, 2000 [comparison 3]

United States 8 Medicaid administrative regions in Kentucky 

Children in Medicaid program with respiratory infections 

Patient education (distribution of materials), Audit and 

feedback The intervention was a combination of physician feedback and patient education. Physicians received a prescribing profile, consisting of the percent of patients with pediatric respiratory conditions to whom they prescribed antibiotics and information on total and relative costs. It also contained their percentile ranking for antibiotic prescribing compared to peers. Physicians also received 25 patient education leaflets that they were to distribute themselves to patients with upper respiratory infections. 

Absolute effecIn the intervenincreased from group increasedifference was 

Perz JF, JAMA, 2002 

United States 4 counties in Tennessee 

Children in Medicaid program 

Clinician education (distribution of materials, educational 

meetings, educational outreach), Patient education(distribution of materials, mass media educational campaign) A year long intervention targeting providers, parents, and the general public was performed. Provider education consisted of lectures from an expert, presentations at conferences, distribution of guidelines, and newsletters. Parent education consisted of distribution of pamphlets by mail, via the providers, to all parents of newborns. Public education consisted of mass media advertisements and distribution of pamphlets at hospitals, clinics, dental offices, and pharmacies. 

The interventiabout 0.16 antyear, an 11% irelative reductThis difference 
B-4 

Studies targeting prescribing for adults 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Gonzales R, JAMA, 1999 [comparison 1] 

United States 4 HMO primary care practices 

Kaiser Permanente patients with acute bronchitis 

Patient education (distribution of materials, other) Clinician education (distribution of materials, educational meetings), Audit and feedback

The study consisted of a multifaceted educational intervention provided to Kaiser patients and providers with the goal of reducing antibiotic use for acute bronchitis. Patients received mailed educational materials (refrigerator magnets, pamphlets, letters), and educational materials in the provider's office (posters). Providers received site-specific antibiotic prescription rates, education on evidence-based management of bronchitis. Provider education was carried out by the medical director of the facility in a one-time, 30-minute meeting. 

Absolute effecIn the intervenvisits at which for acute broncbefore the inteintervention. Iprescribing ratThis difference 

Gonzales R, JAMA, 1999 [comparison 2]

United States 4 HMO primary care practices 

Kaiser Permanente patients with acute bronchitis 

Patient education (distribution of materials), Clinician education (other) In this study arm, the intervention consisted of office-based patient educational materials only. These were colorful posters placed on the walls of each room, accompanied by information on the ineffectiveness of antibiotic for bronchitis, as well as information on antimicrobial resistance. 

Absolute effecIn the intervenvisits at which for acute bronc78%. In the corate declined fdifference was 
B-5 

Studies targeting prescribing for patients of all ages 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Hickman DE, Annals of Pharmacotherapy, 2003 

United States Suburban communitybased physician group 

Patients enrolled in nonprofit medical group (approximately 50% in capitated health plans) with acute bronchitis 

Patient education (distribution of materials), Clinician education (distribution of materials, educational meetings, educational outreach), Audit and feedbackThe intervention consisted of as patient education delivered via a newsletter and posters and pamphlets in the waiting room. Providers received pharmacist-led seminars, written materials, and provider-specific prescription profiling. This took place during a 20 minute presentation by a clinical pharmacist at regular staff meetings. Feedback was provided confidentially in a folder, which also contained educational materials. 

Absolute effecIn the intervendeclined from the interventioprescribing ratbefore and aftedifference was 

McConnell TS, Western Journal of Medicine, 1982 

United States United States New Mexico Medicaid program 

Medicaid patients with respiratory infections 

Clinician education (distribution of materials, educational 

outreach), Audit and feedback

Physicians who were found on audit to be inappropriately prescribing tetracycline for acute respiratory infections (ARIs) received educational visits. Visits took about 30 

minutes and occurred a single time. Information on their prescribing was presented, as well as educational materials. Visits were carried out by physician consultants in the physician’s office. 

The interventioprescriptions fper 6-month peprescribing waThe investigatdifference was 

Foxman B, Journal of Chronic Diseases, 1987 

United States 6 cities: Dayton, OH; Seattle, WA; Fitchburg, MA; Franklin County, MA; Charleston, SC; Georgetown County, SC 

Comparison of cost-sharing insurance plans (intervention) versus fee-forservice insurance (control) 

Financial or regulatory incentives for patients This study was the RAND health insurance experiment, where patients were randomized to insurance plans varying by level of cost-sharing. How antibiotic use varies by insurance plan, diagnosis and health status, geographic area, and demographic characteristics were also examined. 

Antibiotic use insured family group comparedifference was 
B-6 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND 

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Hennessy TW, Clinical Infectious Diseases, 2002 

United States 13 rural Alaskan villages 

Patients receiving care through Indian Health Service with respiratory infections 

Patient education (distribution of materials, mass media educational campaign), Clinician education (educational 

outreach, educational workshops) The investigators described the intervention as a community-based educational intervention provided with the goal of reducing inappropriate antibiotic use for acute respiratory infections. Physicians and community health aides received workshops on antibiotic use. Residents received information at village-wide meetings, community 

fairs, and high school classrooms. Residents also received a total of four informational mailings. 

Absolute effecIn the intervenprescribing deARIs to 47%. prescribing deinvestigators dwas statisticall

* The measured population refers to the patient population in which prescribing data was measured. This popula

which the study results are most generalizable. 

** The Absolute effect size is provided for studies that were used in the median effects analysis. These studies re

percentage of visits at which patients received an antibiotic prescription. The individual study Absolute effect si[(Intervention)post – (Intervention)pre] – [(Control)post – (Control)pre]. A negative effect size means an overall redu

antibiotic prescribing. The % sign refers to the percent of patients with a particular diagnosis. 
B-7 

Appendix B Table 2: Antibiotic treatment studies conducted outside the U.S. Studies targeting prescribing for children 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Smabrekke L, Scandinavian Journal of Infectious Diseases, 2002 

Norway Physicians and nurses at urgent care clinic 

Children with otitis media 

Patient education (distribution of materials), Clinician 

education (distribution of materials, educational meetings) The intervention consisted of a symposium on evidencebased management of otitis media for physicians and nurses at an acute care clinic. Guidelines on appropriate treatment were distributed. The guideline recommended narrowspectrum antibiotic use when necessary, and suggested use of delayed prescriptions. Pamphlets for patients were made available in exam rooms; the pamphlets stressed that most patients recover without antibiotic and indiscriminate use of antibiotic leads to resistance. This information was also provided to patients who called the clinic. 

Absolute effect sizThe percentage of pdecreased 16.0% (fintervention to 74.0the prescribing rate 91.0%). This diffe

Cates C, BMJ, 1999 

UK General practitioners in outpatient clinics in Hertfordshire 

Children with otitis media 

Patient education (distribution of materials), Delayed 

prescriptionsParents of children with acute otitis media who were "not particularly ill" were given a handout summarizing the limited benefit of antibiotics. Parents also received a delayed prescription for antibiotics with instructions to fill the script if the child did not improve in the next "day or two." The comparison was another nearby clinic that followed usual care.

In the intervention prescriptions per m32% relative reducnumber of amoxici72 to 66 per month difference was stati

Little P, BMJ, 2001 UK 

General practices in southwest England 

Children with otitis media 

Delayed prescriptions Children with otitis media were randomized to receive either immediate antibiotics or delayed treatment. Patients in the delayed treatment group were to receive antibiotics after 72 hours if their symptoms did not improve. 

In the intervention antibiotics, comparcontrol group. Thisignificant.
B-8 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Gonzalez Ochoa E, Bulletin of the Pan American Health Organization, 1996 [comparison 1] 

Cuba General practitioners in 4 areas of Havana 

Children with respiratory infections 

Patient education (group educational sessions, distribution of materials), Clinician education (distribution of 

materials, educational meetings, educational outreach visits, educational workshops) The intervention consisted of education of health care providers and community education components. For education of health care providers, there was a special refresher course for family physicians which included videos and slide presentations on treatment of acute respiratory infections in children. There were also opportunities for provider group discussions. The community education program consisted of group discussions in the participants' homes and talks given to mothers in the clinics. 

Absolute effect sizThe percentage of pdecreased 27.8% (fintervention to 7.8% prescribing rate inc20.3%). 

Gonzalez Ochoa E, Bulletin of the Pan American Health Organization, 1996 [comparison 2] 

Cuba General practitioners in 4 areas of Havana 

Children with respiratory infections 

Clinician education (distribution of materials, educational meeting) This study arm consisted of health care provider education. For education of health care providers, there was a special refresher course for family physicians which included videos and slide presentations on treatment of acute respiratory infections in children. There were also opportunities for provider group discussions.

Absolute effect sizIn the intervention 8.9% (from 20.6% after the interventioprescribing rate inc20.3%). The invesdifference was stati

Gonzalez Ochoa E, Bulletin of the Pan American Health Organization, 1996 [comparison 3] 

Cuba General practitioners in 4 areas of Havana 

Children with respiratory infections 

Patient education (individual educational sessions, group educational sessions, distribution of materials) This study arm consisted of a community education program. The community education program consisted of group discussions in the participants' homes and talks given to mothers in the clinics. 

Absolute effect sizIn the intervention (from 11.4% before In the control group0.7% (from 19.6% not report if this difsignificant. 
B-9 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Pshetizky Y, Family Practice, 2003 

Israel General practitioners 

Children with otitis media 

Patient education (individual educational sessions), Delayed prescriptions

The intervention targeted parents of children with acute otitis media. All patients were given an antibiotic prescription. Parents in the intervention group received 

instruction to only use antibiotics if the child's condition did not improve in 24- 48 hours. The intervention group also received a brief explanation from the physician, stressing that most patients recover regardless of whether antibiotics are used, and that serious complications of acute otitis media are not necessarily prevented by antibiotics. 

In the intervention antibiotics after the group, 63.0% of padifference was stati

Wilson EJ, Communicable Diseases Intelligence, 2003 

Australia General practitioners in Canberra 

Children with respiratory infections 

Clinician education (distribution of materials, consensus building sessions), Patient education (distribution of 

materials), Self-managementPhysicians participated in focus groups with parents of young children and workshops with other GP’s; these were used to develop a practice guideline for antibiotic prescribing in upper respiratory infection. All intervention general practitioners then received a packet containing the guideline, information sheets on otitis media and sore throat for patient education, prescription pads for nonprescription medicines, patient self-management advice, and a poster advocating the guideline.

In the intervention 0.78 prescriptions pgroup, prescribing per 100 visits. Thisignificant. 

Santoso B, Social Science & Medicine, 1996 [comparison 1] 

Indonesia General practitioners in Yogyakarta and Java provinces 

Children with diarrhea 

Clinician education (distribution of materials, educational outreach, educational workshops) The study consisted of a small group intervention designed to reduce antibiotic prescribing in cases of acute diarrhea. The small group meetings consisted of face-to-face, 

interactive discussions involving 8-12 providers at a health center. Discussions were led by a trained physician moderator, assisted by a pharmacologist or pediatrician. Each lasted approximately two hours. A booklet on appropriate management of diarrhea was also provided.

Absolute effect sizIn the intervention by 17.0% (from 7760.4% after the inteprescribing rates debefore the intervenintervention.) 
B-10 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Santoso B, Social Science & Medicine, 1996 [comparison 2] 

Indonesia General practitioners in Yogyakarta and Java provinces 

Children with diarrhea 

Clinician education (distribution of materials, educational meetings) The study consisted of a two hour seminar on appropriate prescribing for acute diarrhea targeted at physicians and paramedical personnel. The goal was to reduce antibiotic use and encourage use of oral rehydration solution. The presentation was by two physicians from the district office and the university. Question-and-answer sessions followed the seminar. Written materials on management of diarrhea were also provided. 

Absolute effect sizIn the intervention by 10.0% (from 82group, prescribing 82.6% before the inintervention.) 
B-11 

Studies targeting prescribing for adults 

STUDY SETTING

AND

TARGET

MEASURED 

POPULATION

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

McIsaac WJ, Medical Decision Making 1998 

Canada Family physicians in Ontario 

Adults with sore throat 

Clinician reminder Physicians received a mailing containing both a modified scoring system to help guide antibiotic prescribing and management recommendations. The intervention group also received an encounter form for a specific patient visit with the scoring system and recommendations for testing or treating based on the score. The control group received a similar encounter form without the scoring system or treatment recommendations. 

In the intervention antibiotics, compargroup. This differesignificant. 

Coenen SJ, Antimicrobial Agents and Chemotherapy, 2004 

Belgium General practitioners 

Adults with respiratory infections 

Clinician education (distribution of materials, educational outreach) Physicians received a mailed guideline for treatment of patients with acute cough, which was followed by a reminder phone call. Provides also received an educational outreach visit from a pharmacist and "former medical representative", which focused on guideline adherence and dialogue with the provider to overcome barriers to appropriate prescribing. Following this, the providers received a mailing with the key messages from the guideline. 

Absolute effect sizThe percentage of pdecreased 15.6% (fintervention to 27.4the prescribing rate 28.7%). This differ

Macfarlane J, BMJ, 2002 

UK General practices 

Adults with acute bronchitis 

Patient education (distribution of materials), Delayed 

prescriptions Patients with acute bronchitis not requiring immediate treatment received a delayed prescription for antibiotics. They also were given a patient information leaflet about symptoms and the advantages/disadvantages of antibiotic 

use. The leaflet described when to fill the prescription and methods of self-treatment. 

Patients in the inter47.0% of cases, comcomparison group. significant. 
B-12 

Studies targeting prescribing for patients of all ages 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION 

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

McNulty CA, Journal of Antimicrobial Chemotherapy, 2000 

UK General practices in Gloucestershire 

All patients Clinician education (educational workshops), Audit and 

feedback General practitioners received twelve antibiotic educational workshops, which were delivered over a seven week period. Workshops focused on promoting antibiotic guidelines, reducing antibiotic use for unnecessary conditions and making better prescribing decisions.

In the intervention prescriptions per pr395.9 before the inintervention. In the declined from 356.This difference was 

Melander E, Scandinavian Journal of Primary Health Care, 1999 [comparison 1] 

Sweden General practitioners 

All patients with respiratory infections 

Clinician education (educational workshops), Audit and 

feedback The intervention consisted of an audit of physician's antibiotic prescribing behavior. Based on the results of this audit, group discussions on appropriate management of ARIs were held, and an educational program was developed using local opinion leaders. Prescribing data was fed back to the individual physicians. 

Absolute effect sizThe percentage of pARIs decreased 4.0intervention to 9.0% prescribing rate dec35.0%). The investdifference was stati

Molstad S, Family Practice, 1989 

Sweden General practitioners at community health centers 

All patients with respiratory infections 

Clinician education (consensus building sessions) Physicians designed and administered an educational program consisting of developing indications for antibiotic treatment and reviewing regional resistance patterns. 

Physicians were encouraged to use rapid streptococcal assays when warranted. 

Absolute effect sizThe percentage of pdecreased 23.7% (fintervention to 43.9the prescribing rate 71.1%). The invesdifference was stati

Zwar N, Family Practice, 1999 

Australia General practitioner trainees in New South Wales 

All patients with respiratory infections 

Clinician education (distribution of materials, educational outreach, other), Audit and feedback

The intervention consisted of a mailed packet of individualized feedback to providers, management guidelines, and patient handouts. Providers who exceeded thresholds for inappropriate prescriptions based on feedback data also received an academic detailing session, in person or over the telephone. 

Absolute effect sizThe percentage of pdecreased 5.3% (frto 19.7% after). In rate increased 9.7% difference was stati
B-13 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION 

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Angunawela I, International Journal of Epidemiology, 1991 [comparison 1] 

Sri Lanka General practitioners at rural primary care clinics 

All patients Clinician education (distribution of materials) 

The intervention consisted of newsletters mailed fortnightly to providers, which covered topics about treatment of common infections, encouraged use of penicillin, discouraged use of antibiotics in viral infections and tetracycline to children. 

Absolute effect sizThe percentage of pdecreased 7.4% (frto 24.1% after). In rate decreased 0.4% difference was not 

Angunawela I, International Journal of Epidemiology, 1991 [comparison 2] 

Sri Lanka General practitioners at rural primary care clinics 

All patients Clinician education (distribution of materials, educational 

meetings) In addition to the newsletters described above, providers participated in a seminar conducted by two clinical 

pharmacologists, one microbiologist, and one general physician. The seminar reinforced the information in the 

newsletter. 

Absolute effect sizThe percentage of pdecreased 7.3% (frto 31.5% after). In rate decreased 0.4% difference was not 

Arroll B, Journal of Family Practice, 2002 

New Zealand General practitioners 

All patients with common cold 

Delayed prescriptions Patients with the common cold who requested antibiotics were given a prescription with instructions to fill it only if their symptoms did not improve after 3 days. The comparison group was given antibiotics immediately. 

In the intervention antibiotics; in the cused them. This disignificant. 

Bexell A, Journal of Clinical Epidemiology, 1996 

Zambia General practices in 

All patients with diarrhea and respiratory infections 

Clinician education (educational meetings) The intervention consisted of three 2-day seminars for providers on rational drug prescribing. Educational content consisted of standardized guidelines for managing "common conditions seen in primary care." Seminars were led by pharmacists, clinical officers, medical officers, and a nurse. Specific diseases discussed included diarrheal diseases and acute respiratory infections. 

Absolute effect sizThe percentage of pdecreased 7.0% (frto 34.2% after). In rate increased 1.1% difference was stati
B-14 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION 

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Flottorp S, BMJ, 2002 

Norway 142 General practices 

All patients with sore throat 

Patient education (distribution of materials, other), Clinician education (distribution of materials, other), Clinician Reminders, Organizational change, Regulatory changes for providers, Financial or regulatory incentives for patients The investigators conducted a multifaceted intervention to improve management of sore throat. Providers received guidelines on sore throat management, in electronic and poster format. Computer based decision support, reminders, and interactive courses were offered to providers. Patients received electronic and paper-based educational materials. Patients had to pay an increased fee for a telephone consult (but not for office visit.) Only practices using an electronic medical records system were eligible. 

Absolute effect sizThe percentage of pdecreased 4.3% (frto 43.8% after). In rate decreased 1.3% difference was stati

Gutierrez G, Medical Care, 1994 

Mexico General practitioners at health care clinics of the Mexican Social Security Institute 

All patients with diarrhea 

Clinician education (distribution of materials, educational workshops, consensus building session), Audit and feedback Physicians received an educational workshop on treatment of acute diarrhea for physicians. They received literature and had group discussions to develop a treatment algorithm and discuss management of simulated cases. Five months after the workshops, physicians' practice habits were audited and reviewed by a committee including study physicians.

After the interventiintervention group before the intervengroup values were 

McIsaac WJ, Journal of Family Practice, 2002 

Canada Family physicians in Ontario 

All patients with sore throat 

Clinician reminder Physicians received a sample encounter form for patients with sore throat, which explicitly described a scoring system (to help guide antibiotic prescribing) and provided management recommendations. The form also had space to record the score. The control group did not receive the sticker, and the encounter form did not have the prompt to record the score. 

The prescribing rat28.1%, compared tThis difference was 
B-15 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION 

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Meyer JC, Medical Education, 2001 

South Africa Nurses at primary health clinics 

All patients with diarrhea 

Clinician education (educational workshops) Nurses in primary health care centers received training focused on teaching rational drug prescribing, conducted during a 4-day workshop. A World Health Organization manual was used as the framework for the training course. The target was reducing antibiotic prescribing for ARIs. 

Absolute effect sizThe percentage of pdecreased 36.9% (fintervention to 29.4the prescribing rate 45.6%). This differ

O'Connell DL, BMJ, 1999 

Australia General practitioners in rural area 

All patients Clinician education (distribution of materials), Audit and 

feedback Physicians received a graphical display of their level of prescribing of five different drug groups (including oral antibiotics) over the past eight quarters. These data were provided as graphs displaying the individual physician's prescribing in comparison to all physicians. This intervention was repeated twice, six months apart. The first intervention was accompanied by a general prescribing educational letter. The second was accompanied by a letter with specific antibiotic prescribing information. 

In the intervention prescribing rates ch100 patients before prescriptions per 10Corresponding comprescriptions per 10and 10.1 after the innot statistically sign
B-16 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION 

AND

CONDITION

*

INTERVENTION DESCRIPTION

AND

QUALITY IMPROVEMENT STRATEGIES

Welschen I, BMJ, 2004 

The Netherlands General practitioners in Utrecht 

All patients with respiratory infections 

Patient education (distribution of materials), Clinician 

education (distribution of materials, educational meetings, educational workshops, consensus building sessions), Audit 

and feedback Physicians attended a group educational meeting, at which feedback data on antibiotic prescribing was provided, evidence on the effectiveness of antibiotic for upper 

respiratory infections was presented, and used to develop consensus on indications for antibiotic use and choice. These were formalized into guidelines that were mailed to participations. Six months later physicians received repeat feedback on their prescribing. Physician assistants and pharmacists also received a 2 hour session on antibiotic prescribing guidelines. Physicians, assistants, and pharmacists also received communication skills training. Patient education materials, consisting of brochures and posters, were provided in offices, pharmacies, and municipal health services. 

Absolute effect sizThe percentage of pdecreased 4.0% (frto 23.0% after). In rate increased 8.0% 

* The measured population refers to the patient population in which prescribing data was measured. This popula

which the study results are most generalizable. 

** The Absolute effect size is provided for studies that were used in the median effects analysis. These studies re

percentage of visits at which patients received an antibiotic prescription. The individual study Absolute effect si[(Intervention)post – (Intervention)pre] – [(Control)post – (Control)pre]. A negative effect size means an overall redu

antibiotic prescribing. The % sign refers to the percent of patients with a particular diagnosis. Appendix B Table 3: Antibiotic selection studies conducted in the U.S. 
B-17 

Studies targeting prescribing for children 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION

AND 

CONDITION* 

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

R

Bush PJ, Medical Care, 1979 [comparison 1] 

United States Pediatricians at 3 HMO sites in Washington, DC 

Children enrolled in HMO with otitis media 

Clinician education (distribution of materials, consensus building session) The intervention consisted of a drug therapy protocol for otitis media, which specified drugs, doses, and days of therapy by patient age. The protocol was developed and implemented by a physician group at one site. The intervention encouraged use of ampicillin or penicillin. The control group received no intervention.

Absolute effect size: 2In the intervention grourecommended antibioticbefore the intervention group, the prescribing rdecreased 22.6% (from investigators did not repgroups was statistically 

Bush PJ, Medical Care, 1979 [comparison 2] 

United States Pediatricians at 3 HMO sites in Washington, DC 

Children enrolled in HMO with otitis media 

Clinician education (distribution of materials) In this study, the drug therapy protocol (above) was implemented at a separate site from the one that created the protocol.

Absolute effect size: 3In the intervention grourecommended antibioticbefore the intervention group, the prescribing r89.6% to 67.0%). The the difference between significant. 
B-18 

Studies targeting prescribing for adults 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

R

Klein LE, Journal of Medical Education, 1981 

United States Residents in academic emergency room and primary care center 

Adults with urinary tract infections 

Clinician education (educational meetings) The intervention consisted of an educational tutorial that addressed four misconceptions about the treatment of urinary tract infection. This tutorial aimed to encourage use of sulfisoxazole or tetracycline, and discourage use of ampicillin or trimethoprimsulfamethoxazole. The tutorial was presented to each participant for 15 minutes. In addition to the tutorial, the authors gave a ten minute talk just before the postintervention data collection period on options for treating urinary tract infection. 

Absolute effect size: 4In the intervention grourecommended antibioticbefore the intervention group, the prescribing rto 39.0%). This differesignificant.
B-19 

Studies targeting prescribing for patients of all ages

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION

AND

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

R

Avorn J, New England Journal of Medicine, 1983 [comparison 1] 

United States Physicians in 4 state Medicaid programs (Arkansas, New Hampshire, Vermont, and the District of Columbia) 

Medicaid patients 

Clinician education (distribution of materials) Physicians received an office-based education program consisting of a visit from a clinical pharmacist to discuss ways to reduce prescribing of targeted drugs. Physicians also received mailed educational materials, including drug information letters and “unadvertisements” (specially designed brochures with colorful illustrations on appropriate prescribing practices.) These materials were distributed eight times over a period of four months. The objective was to reduce the prescribing of oral cephalosporins. 

In the intervention grourecommended antibiotic 31% relative to the contstatistically significant. 

Avorn J, New England Journal of Medicine, 1983 [comparison 2] 

United States Physicians in 4 state Medicaid programs (Arkansas, New Hampshire, Vermont, and the District of Columbia) 

Medicaid patients 

Clinician education (distribution of materials) In this study, physicians received only the mailed drug 

information letters and “unadvertisements” described above. 

In the intervention grourecommended antibiotic 10% relative to the contnot statistically signific
B-20 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION

AND

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

R

Farris KB, Pharmaceutical Research, 1996 

United States Family physicians in HMO in southeastern Michigan 

HMO patients with respiratory infection 

Clinician education (distribution of materials, educational meetings, educational outreach, educational workshops), Audit and feedback

The intervention consisted of face to face group detailing sessions involving opinion leaders with written presentation summaries placed in physicians clinic mailboxes 3-6 weeks after the detailing session. Physicians also received personal and peer comparison feedback on prescribing trends for target medications, periodic posted reminders, summary poster displays, and a report of program results at the end of the intervention. 

Absolute effect size: 7In the intervention grourecommended antibioticbefore the intervention 

group, the prescribing rto 59.7%). The investidifference between grou

Schaffner W, JAMA, 1983 [comparison 1] 

United States Physicians participating in Tennessee Medicaid program 

Medicaid patients 

Clinician education (distribution of materials, educational outreach) Physicians received an educational visit from a drug educator, who went over a brochure that discussed the reasons not to use contraindicated antibiotics (chloramphenicol, clindamycin, and tetracycline for children under age 8) and provided reasonable alternatives. The drug educator was a trained clinical pharmacist. 

In the intervention groufor contraindicated antibrelative to the pre-intervgroup, prescribing was pre-intervention period. statistically significant.

Schaffner W, JAMA, 1983 [comparison 2] 

United States Physicians participating in Tennessee Medicaid program 

Medicaid patients 

Clinician education (distribution of materials, educational outreach) Physicians received an educational visit from a physician counselor, who went over a brochure that discussed the reasons not to use the contraindicated antibiotics (chloramphenicol, clindamycin, and tetracycline for children under age 8) and provided reasonable alternatives. The physician counselors were three senior physicians in part-time practice recruited by study advisory committee. 

In the intervention groufor contraindicated antibrelative to before the intgroup, prescribing was pre-intervention period. statistically significant. 
B-21 

STUDY SETTING

AND 

TARGET

MEASURED 

POPULATION

AND

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

R

Schaffner W, JAMA, 1983 [comparison 3] 

United States Physicians participating in Tennessee Medicaid program 

Medicaid patients 

Clinician education (distribution of materials, educational outreach) Physicians received an educational visit from a drug educator who went over a brochure that discussed the reasons not to use the oral cephalosporins and provided reasonable alternatives. The drug educator was a trained clinical pharmacist. 

In the intervention groufor cephalosporins was pre-intervention period. prescribing was reduced intervention period. Thstatistically significant. 

Schaffner W, JAMA, 1983 [comparison 4] 

Physicians participating in Tennessee Medicaid program 

Medicaid patients 

Clinician education (distribution of materials, educational outreach) Physicians received an educational visit from a physician counselor, who went over a brochure that discussed the reasons not to use oral cephalosporins counselors were three senior physicians in part-time practice recruited by study advisory committee. 

Prescribing of cephalosthe intervention group ito before the interventioprescribing was reduced intervention period. Thsignificant. 

* The measured population refers to the patient population in which prescribing data was measured. This popula

which the study results are most generalizable. 

** The Absolute effect size is provided for studies that were used in the median effects analysis. These studies re

percentage of visits at which patients received an antibiotic prescription. The individual study Absolute effect si[(Intervention)post – (Intervention)pre] – [(Control)post – (Control)pre]. A negative effect size means an overall redu

percentage of patients receiving a recommended antibiotic. 
B-22 

Appendix B Table 4: Antibiotic selection studies conducted outside the U.S. Studies targeting prescribing for children

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Dolovich L, Drug Information Journal, 1999 

Canada Family physicians in Southern Ontario 

Children with otitis media 

Clinician education (distribution of materials, 

educational outreach) The intervention consisted of verbal and written evidence-based academic detailing on prescribing for acute otitis media. The academic detailers were pharmaceutical sales representatives that were asked to promote the message that physicians use amoxicillin as the first line drug. (The sales representatives worked for a company that sold a different second-line agent, pivampicillin.) Commission-based financial incentives tied to the volume of products sold were eliminated in the intervention region. Each physician had at least two follow up visits during the six month intervention period. 

The primary outcomshare” for amoxicilall antibiotic prescrAmoxicillin’s markcompared to the coThis difference was 

Smabrekke L, Scandinavian Journal of Infectious Diseases, 2002 

Norway Physicians at an acute care center 

Children with otitis media 

Patient education (distribution of materials), Clinician education (distribution of materials, educational meetings) The intervention was a symposium on evidence-based management of otitis media for physicians and nurses at an acute care clinic. Guidelines on appropriate treatment were distributed. The guideline recommended Penicillin V use when antibiotics were necessary. Pamphlets for patients were made available in exam rooms; the pamphlets stressed that most patients recover without antibiotic and indiscriminate use of antibiotic leads to resistance. This information was also provided to patients who called the clinic. 

Absolute effect sizIn the intervention recommended antib72.3% before the incontrol group, the pantibiotics increase78%). This differensignificant.
B-23 

Studies targeting prescribing for adults

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Coenen SJ, Antimicrobial Agents and Chemotherapy, 2004 

Belgium General practitioners 

Adults with acute cough 

Clinician education (distribution of materials, educational outreach) The intervention consisted of a mailed guideline for treatment of patients with acute cough, which was delivered to physicians by mail and followed by a reminder phone call. Provides also received an educational outreach visit from a pharmacist and "former medical representative", which focused on guideline adherence and dialogue with the provider to overcome barriers to appropriate prescribing. Following this, the providers received a mailing with the key messages from the guideline. The recommended antibiotics were 

amoxicillin and doxycycline. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 37.3% aftersignificant.

Dowell J, British Journal of General Practice 

UK General practitioners in Scotland 

Adults with cough 

Delayed prescriptions Patients presenting with acute cough were given an antibiotic prescription, but asked to wait one week before using the prescription.

In the intervention groantibiotics after the ingroup, 100% of patienwas statistically signi

Harrison A, AIDS, 2000 

South Africa Practitioners in 10 primary care clinics in rural KwaZulu - Natal 

Adults with sexually transmitted diseases 

Clinician education (distribution of materials, educational outreach, educational workshops), Patient education (distribution of materials) The intervention was a training and supervision program for health care workers. The training was an initial two day workshop attended by a senior primary health care nurse from each clinic. Three follow up sessions were held in each clinic. There were also monthly follow up visits to each clinic by a member of the district STD team. The intervention also included the development and implementation of syndrome packets, including 

recommended drug treatment information, condoms, partner cards and patient information leaflet. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 50.0% aftersignificant.
B-24 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Veninga CC, Journal of Clinical Epidemiology, 2000 

Netherlands General practitioners 

Adults with urinary tract infection 

Clinician education (educational workshops, other), Audit and feedbackThe intervention was a series of meetings constituting a self-learning auditing program for providers. These meetings took place two times over a four month period. 

The first consisted of reviewing a guideline for urinary tract infection treatment and going over case vignettes. Decision strategy feedback was given on the provider's responses to the vignettes. At the second meeting, providers received individualized feedback on their 

prescribing behavior over the previous six months. The goals of the intervention were to increase prescription of recommended antibiotics (trimethoprim, nitrofurantoin, and sulfamethoxazole), and to encourage short courses of treatment. 

Absolute effect size: In the intervention grorecommended antibio(from 89.0% before ththe control group, the (from 86.0% before to not statistically signif

Lagerlov P, Quality in Health Care, 2000 

Norway General practitioners 

Adults with urinary tract infections 

Clinician education (distribution of materials, educational workshops, consensus building sessions), Audit and feedbackThe intervention consisted of two group meetings in which the doctors (with the help of guidelines, and review of their own prescribing habits) established their own criteria for inappropriate and appropriate prescribing for urinary tract infection. The goal was to encourage 

treatment of urinary tract infection with 4 or fewer days of antibiotics. 

The intervention resuproportion of UTI trethe intervention groupsignificant. 

Lundborg CS, Journal of Clinical Epidemiology, 1999 

Sweden 36 General practitioner groups 

Adults with urinary tract infections 

Clinician education (distribution of materials, educational workshops), Audit and feedback

The intervention was a two-part educational session based on national treatment guidelines for urinary tract infection, which recommended the use of first-line antibiotics with a treatment duration of 3-7 days. Sessions, which were run by an external GP-pharmacist team, involved feedback on simulated written cases and on actual physician-level prescriptions. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 58.0% aftersignificant.
B-25 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Rokstad K, Journal of Clinical Epidemiology, 1995 

Norway General practitioners in 2 counties 

Adults with urinary tract infection 

Audit and feedback The intervention consisted of a one time report sent by mail that included a prescription profile for treatment of urinary tract infection, for both the individual physician and the total sample of general practitioners in the study. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventio

group, the prescribing before to 14.4% afterdifference was statisti

MacCara ME, Annals of Pharmacotherapy, 2001 

Canada General practitioners 

Adults over age 65 enrolled in Nova Scotia Seniors’ Pharmacare Program 

Regulatory changes for providers, Financial and regulatory incentives for patients A policy change which limited reimbursement of the fluoroquinolones and other antimicrobials was instituted. In order to prescribe fluoroquinolones, the physician filled out a special form and a pharmacist determined whether the request was within the established criteria. Flouroquinolones use was limited to treatment of infections due to Pseudomonas and infections resistant to other drugs. 

After instituting the pfrom 20.2% of all antprescriptions. This ch

Hux JE, CMAJ, 1999 

Canada Family physicians in Ontario 

Adults with respiratory infections or urinary tract infections 

Clinician education (distribution of materials), Audit and feedback The intervention consisted of mailings with prescribing feedback and targeted educational bulletins stressing the use of less expensive first-line antibiotics, which took place every 2 months over 6 month period. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 66.8% aftersignificant. 
B-26 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Peterson GM, Journal of Clinical Pharmacy & Therapeutics, 1997 

Australia General practitioners in Southern Tasmania 

Adults with urinary tract infection 

Clinician education (distribution of materials, educational outreach) The investigators described the intervention as educational mailings sent to general practitioners. The materials were designed to encourage rational prescribing of antibiotics in the management of acute, uncomplicated urinary tract infection. After two weeks, each general practitioner was contacted by telephone to confirm receipt of materials and to set up an appointment with a 

pharmacist to discuss materials. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 83.3% aftersignificant.
B-27 

Studies targeting prescribing for patients of all ages

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Braybrook S, Journal of Clinical Pharmacy & Therapeutics, 1996 [comparison 1] 

UK 66 family health practices in Wales 

All patients with urinary tract infections 

Clinician education (educational outreach), Audit and feedback, Regulatory changes for providersIntervention practices received visits from a pharmaceutical prescribing adviser to discuss rational antibiotic use and to give feedback on current antibiotic use. This information was presented in a graphical computer slide show. Participating physicians received postgraduate education credit

Prescribing of generic antibiotic) increased bcontrol group. In the generic amoxicillin inpatients. The study dstatistically significan

Braybrook S, Journal of Clinical Pharmacy & Therapeutics, 1996 [comparison 2] 

UK 66 family health practices in Wales 

All patients with urinary tract infections 

Clinician education (distribution of materials), Audit and feedback, Regulatory changes for providers

The intervention consisted of practice-specific feedback on prescribing practices (identical in layout to the information given in comparison 1 above) contained within a workbook and accompanied with a series of discussion questions. One general practitioner within each practice received the workbook and organized discussion sessions on rational prescribing. Participating physicians received postgraduate education credit.

Prescribing of generic antibiotic) increased bboth the control and i

De Santis G, Medical Journal of Australia, 1994 

Australia General practitioners in 8 rural and metropolitan Victorian Health Department regions 

All patients with tonsillitis 

Clinician education (distribution of materials, educational outreach)The intervention consisted of a 3-month educational campaign directed at providers. The campaign was initiated with a mailed brochure, which contained the recommended guidelines. These guidelines encouraged 

use of penicillin V or erythromycin to treat tonsillitis. The brochure was used as a focus of discussion by an academic detailer (project pharmacist) who visited 

doctors to discuss the campaign messages. Providers received five additional mailings, reiterating key messages. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 71.7% aftersignificant.
B-28 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Friis H, Danish Medical Bulletin, 1991 [comparison 1] 

Denmark General practitioners in 5 counties 

All patients with respiratory infections or skin or soft tissue infections 

Clinician education (distribution of materials, educational meetings) An informational campaign stressed the importance of reducing ampicillin and co-trimoxazole, while increasing penicillin usage. Written materials were distributed and a series of ten lectures were held. In this comparison, the meetings were arranged by the department of clinical microbiology. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventio

group, the prescribing before to 72.0% aftersignificant

Friis H, Danish Medical Bulletin, 1991 [comparison 2]

Denmark General practitioners in 5 counties 

All patients with respiratory infections or skin or soft tissue infections 

Clinician education (distribution of materials, educational meetings) An informational campaign stressed the importance of reducing ampicillin and co-trimoxazole, while increasing penicillin usage. Written materials were distributed and a series of ten lectures were held. In this comparison, the meetings were sponsored by a pharmaceutical company and held free of charge.

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 72.0% afterstatistically significan

Ilett KF, British Journal of Clinical Pharmacology, 2000 

Australia General practitioners in the Perth metropolitan area 

All patients with respiratory infections or urinary tract infections 

Clinician education (distribution of materials, educational outreach) The investigators described the intervention as a one time visit by an academic detailer to providers. The visit consisted of a brief discussion of best practice guidelines and included the distribution of a laminated copy of a chart summarizing prescribing guidelines. The recommended antibiotics were amoxicillin and doxycycline for ARIs, and penicillin for UTIs. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventio

group, the prescribing before to 33.7% afterif this difference was 

McNulty CA, Journal of Antimicrobial Chemotherapy, 2000 

UK 84 general practices in Gloucestershire 

All patients with respiratory infections or urinary tract infections 

Clinician education (educational workshops), Audit and 

feedback The intervention consisted of twelve antibiotic educational workshops, which were delivered to providers over a seven week period. Workshops focused on promoting antibiotic guidelines, reducing antibiotic use for unnecessary conditions and making better prescribing decisions. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 17.9% afterif this difference was 
B-29 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Melander E, Scandinavian Journal of Primary Health Care, 1999 [comparison 2] 

Sweden General practices in 3 communities 

All patients with respiratory infections 

Clinician education (educational workshops), Audit and 

feedback Physician’s antibiotic prescribing behavior was audited and fed back to the individual physicians. Group 

discussions and an educational program were based on the results of these data. Physicians were encouraged to use penicillin V as the first choice drug. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 49.9% afterstatistically significan

Molstad S, Family Practice, 1989 

Sweden General practitioners at 3 community health centers 

All patients with respiratory infections 

Clinician education (consensus building sessions) Clinicians received an educational program consisting of reviewing regional resistance patterns and indications for antibiotic treatment. Phenoxypenicillin was agreed upon as the first-choice antibiotic, with amoxicillin or pivampicillin for treatment failures and erythromycin for penicillin-allergic patients. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventio

group, the prescribing before to 53.1% afterif this difference was 

Raz R, Israel Journal of Medical Sciences, 1995 

Israel Physicians and nurses at 3 community outpatient clinics 

All patients with pharyngitis 

Clinician education (distribution of materials, educational meetings) Two 1-hour educational sessions on antibiotic prescribing for pharyngitis were given by the study physicians to the entire medical staff (physicians, nurses and pharmacists.) Written handouts were distributed recommending treatment with penicillin V for streptococcal pharyngitis.

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 43.4% aftersignificant. 
B-30 

STUDY SETTING 

AND 

TARGET

MEASURED 

POPULATION 

AND 

CONDITION*

INTERVENTION DESCRIPTION

AND 

QUALITY IMPROVEMENT STRATEGIES

Sondergaard J, Scandinavian Journal of Primary Health Care, 2003 

Denmark 181 general practices 

All patients with respiratory infections 

Clinician education (distribution of materials), Audit and feedback The intervention consisted of mailed information to general practitioners on their prescribing practices combined with a clinical guideline on the diagnosis and 

treatment of respiratory tract infections. The feedback 

provided some aggregate data so the general practitioner could compare the prescription rates of their own practice to other local practices. The clinical guidelines recommended restricting antibiotics used for respiratory tract infections to the lowest justifiable level, and that narrow-spectrum antibiotics (e.g. penicillin V) should be preferred to broad-spectrum antibiotics. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventio

group, the prescribing before to 43.0% afterstatistically significan

Zwar N, Family Practice, 1999 

Australia General practitioner trainees in New South Whales 

All patients with pharyngitis 

Clinician education (distribution of materials, educational outreach, other), Audit and feedbackThe intervention consisted of a mailed packet of individualized feedback to providers, management guidelines, and patient handouts. Providers also received an academic detailing session, in person or over phone, for doctors who exceeded thresholds for inappropriate 

prescriptions based on feedback data. Providers were encouraged to use narrow-spectrum penicillin or erythromycin for streptococcal pharyngitis. 

Absolute effect size: In the intervention grorecommended antibiobefore the interventiogroup, the prescribing before to 58.5% aftersignificant.

* The measured population refers to the patient population in which prescribing data was measured. This popula

which the study results are most generalizable. 

** The Absolute effect size is provided for studies that were used in the median effects analysis. These studies re

percentage of visits at which patients received an antibiotic prescription. The individual study Absolute effect si[(Intervention)post – (Intervention)pre] – [(Control)post – (Control)pre]. A negative effect size means an overall redu

percentage of patients receiving a recommended antibiotic. 
C-1 

Appendix C. EPOC and Medline Search for Antibiotic 

QI Articles 

We conducted two searches to identify relevant articles. 

1. EPOC Databases Search: The EPOC, Medline and HealthSTAR search strategy (see below) was combined with 

the antiobiotic terms search strategy (see below) to identify relevant articles from the EPOC databases upto October 

2004.

2. Medline Search: To identify more recent articles that may not have been indexed in the EPOC databases, we 

searched Medline from June 2004 to November 2004 using the search below. 

1. EPOC Databases Search 

EPOC MEDLINE and HealthSTAR * Search Strategy

1. exp *education,continuing/ 

2. (education$ adj2 (program$ or intervention? or meeting? or session? or strateg$ or workshop? or visit?)).tw. 

3. (behavio?r$ adj2 intervention?).tw. 

4. *pamphlets/ 

5. (leaflet? or booklet? or poster or posters).tw. 

6. ((written or printed or oral) adj information).tw. 

7. (information$ adj2 campaign).tw. 

8. (education$ adj1 (method? or material?)).tw. 

9. *advance directives/ 

10. outreach.tw. 

11. ((opinion or education$ or influential) adj1 leader?).tw. 

12. facilitator?.tw. 

13. academic detailing.tw. 

14. consensus conference?.tw. 

15. *guideline adherence/ 

16. practice guideline?.tw. 

17. (guideline? adj2 (introduc$ or issu$ or impact or effect? or disseminat$ or distribut$)).tw. 

18. ((effect? or impact or evaluat$ or introduc$ or compar$) adj2 training program$).tw. 

19. *reminder systems/ 

20. reminder?.tw. 

21. (recall adj2 system$).tw. 

22. (prompter? or prompting).tw. 

23. algorithm?.tw. 

24. *feedback/ or feedback.tw. 

25. (feedback adj1 (loop? or control? or regula$ or mechanism? or inhib$ or system? or circuit? or sensory or visual 

or audio$ or auditory)).tw. 

26. 23 not 24 

27. chart review$.tw. 

28. ((effect? or impact or records or chart?) adj2 audit).tw. 

29. compliance.tw. 

30. marketing.tw. 

31. or/1-22,26-30 

32. exp *reimbursement mechanisms/ 

33. fee for service.tw. 

34. *capitation fee/ 

35. *"deductibles and coinsurance"/ 

36. cost shar$.tw. 

37. (copayment? or co payment?).tw. 

38. (prepay$ or prepaid or prospective payment?).tw. 

39. *hospital charges/ 

40. formular$.tw. 

41. fundhold$.tw. 

42. *medicaid/ 
C-2 

43. *medicare/ 

44. blue cross.tw. 

45. or/32-44 

46. *nurse clinicians/ 

47. *nurse midwives/ 

48. *nurse practitioners/ 

49. (nurse adj (rehabilitator? or clinician? or practitioner? or midwi$)).tw. 

50. *pharmacists/ 

51. clinical pharmacist?.tw. 

52. paramedic?.tw. 

53. *patient care team/ 

54. exp *patient care planning/ 

55. (team? adj2 (care or treatment or assessment or consultation)).tw. 

56. (integrat$ adj2 (care or service?)).tw. 

57. (care adj2 (coordinat$ or program$ or continuity)).tw. 

58. (case adj1 management).tw. 

59. exp *ambulatory care facilities/ 

60. *ambulatory care/ 

61. or/46-60 

62. *home care services/ 

63. *hospices/ 

64. *nursing homes/ 

65. *office visits/ 

66. *house calls/ 

67. *day care/ 

68. *aftercare/ 

69. *community health nursing/ 

70. (chang$ adj1 location?).tw. 

71. domiciliary.tw. 

72. (home adj1 treat$).tw. 

73. day surgery.tw. 

74. *medical records/ 

75. *medical records systems, computerized/ 

76. (information adj2 (management or system?)).tw. 

77. *peer review/ 

78. *utilization review/ 

79. exp *health services misuse/ 

80. or/62-79 

81. *physician's practice patterns/ 

82. quality assurance.tw. 

83. *process assessment/ [health care] 

84. *program evaluation/ 

85. *length of stay/ 

86. (early adj1 discharg$).tw. 

87. discharge planning.tw. 

88. offset.tw. 

89. triage.tw. 

90. exp *referral/ and consultation/ 

91. *drug therapy,computer assisted/ 

92. near patient testing.tw. 

93. *medical history taking/ 

94. *telephone/ 

95. (physician patient adj (interaction? or relationship?)).tw. 

96. *health maintenance organizations/ 

97. managed care.tw. 

98. (hospital? adj1 merg$).tw. 

99. or/81-98 

100. ((standard or usual or routine or regular or traditional or conventional or pattern) adj2 care).tw. 

101. (program$ adj2 (reduc$ or increas$ or decreas$ or chang$ or improv$ or modify$ or monitor$ or care)).tw. 

102. (program$ adj1 (health or care or intervention?)).tw. 

103. ((effect? or impact or evaluat$ or introduc$ or compar$) adj2 treatment program$).tw. 

104. ((effect? or impact or evaluat$ or introduc$ or compar$) adj2 care program$).tw. 
C-3 

105. ((effect? or impact or evaluat$ or introduc$ or compar$) adj2 screening program$).tw. 

106. ((effect? or impact or evaluat$ or introduc$ or compara$) adj2 prevent$ program$).tw. 

107. (computer$ adj2 (dosage or dosing or diagnosis or therapy or decision?)).tw. 

108. ((introduc$ or impact or effect? or implement$ or computer$) adj2 protocol?).tw. 

109. ((effect? or impact or introduc$) adj2 (legislation or regulations or policy)).tw. 

110. or/100-109 

111. 31 or 45 or 61 or 80 or 99 or 110 

112. randomized controlled trial.pt. 

113. controlled clinical trial.pt. 

114. intervention studies/ 

115. experiment$.tw. 

116. (time adj series).tw. 

117. (pre test or pretest or (posttest or post test)).tw. 

118. random allocation/ 

119. impact.tw. 

120. intervention?.tw. 

121. chang$.tw. 

122. evaluation studies/ 

123. evaluat$.tw. 

124. effect?.tw. 

125. comparative studies/ 

126. animal/ 

127. human/ 

128. 126 not 127 

129. or/112-125 

130. 129 not 128 

131. 111 and 130 

*insert : 

(randomised or randomized).tw. 

(random$ adj1 (allocat$ or assign$)).tw.

Antibiotics Terms Search Strategy

antibiotic*

Amoxicillin

Azithromycin

Cephalosporin*

Cephalexin

Ciprofloxacin

Clarithromycin

Clindamycin

Doxycycline

Erythromycin

Flouroquinolone*

Fluconazole

Levofloxacin

Macrolide*

Metronidazole

Nitrofurantoin

Penicillin

Tetracycline

Trimethoprim
C-4 

2. Medline Search 

#1 

Targets clinical 

conditions 

"Community-Acquired Infections"[MeSH] OR "Respiratory Tract 

Infections"[MeSH] OR "Bronchitis"[MeSH] OR "Pneumonia, 

Bacterial"[MeSH] OR Sinusitis[MeSH] OR Common Cold[MeSH] OR 

Influenza[MeSH] OR "Urinary Tract Infections"[MeSH] OR Sexually 

Transmitted Diseases[MeSH] OR Vaginitis[MeSH] OR Candidiasis, 

Vulvovaginal[MeSH] OR Pelvic Inflammatory Disease[MeSH] 

365549

#2 

Targets antibiotics and 

prescribing 

Anti-Bacterial Agents[MeSH] OR Antibiotic[ti] OR Anti-Infective 

Agents, Urinary[MeSH] OR Antibiotics[ti] OR Prescriptions, 

Drug[MeSH] OR Drug Utilization[MeSH] OR Physician's Practice 

Patterns[MeSH] OR Attitude of Health Personnel [MeSH] OR 

Prescribing[ti] OR Prescription[ti] 

255351 

#3 

Targets QI strategies 

that tend to be multifactorial using relevant 

MeSH terms and title 

words 

Disease Management [mh] OR Patient Care Planning [mh] OR 

Patient-Centered Care [mh] OR Primary Health Care [mh] OR 

Progressive Patient Care [mh] OR Critical Pathways [mh] OR 

Delivery of Health Care, Integrated [mh] OR Health Services 

Accessibility [mh] OR Managed Care Programs [mh] OR Product 

Line Management [mh] OR Patient Care Team [mh] OR PatientCentered Care [mh] OR Behavior Control [mh] OR Counseling [mh] 

OR Health Promotion [mh] OR Patient Compliance [mh] OR AfterHours Care [mh] OR ((coordination [ti] OR coordinated [ti] OR 

Multifactorial [ti] OR Multi-factorial [ti] OR Multicomponent [ti] OR 

Multi-component [ti] OR multidisciplinary [ti] OR multi-disciplinary [ti] 

OR interdisciplinary [ti] OR inter-disciplinary [ti] OR integrated [ti] OR 

community-based [ti] OR organized [ti]) AND (care [ti] OR approach 

[ti] OR intervention [ti] OR strategy [ti] OR strategies [ti] OR 

management [ti] OR managing [ti] OR center* [ti] OR clinic*[ti])) OR 

Organization and Administration [mh] 

735481 

#4 

Targets TQM and CQI Total Quality Management [mh] OR Quality control [mh] OR TQM [ti] 

OR CQI [ti] OR (quality [ti] AND (continuous [ti] OR total [ti]) AND 

(management [ti] OR improvement [ti])) 

31249 

#5 

Targets provider 

education 

Education, Continuing [mh] OR (Education [ti] AND Continuing [ti] 

AND (medical [ti] OR professional* [ti] OR nursing [ti] OR physician* 

[ti] OR nurse* [ti])) OR (outreach [ti] AND (visit*[ti] OR educational [ti]) 

OR (academic [ti] AND detailing [ti])) 

36,929 

#6 

Targets diffusion of 

innovation 

Diffusion of Innovation [mh] OR (Diffusion [ti] AND (Innovation [ti] OR 

technology [ti])) 

5519 

#7 

Targets audit and 

feedback, reminder 

systems, and financial 

incentives 

Medical audit [mh] OR ((Audit [ti] OR feedback [ti] OR compliance [ti] 

OR adherence [ti] OR training [ti]) AND (improvement* [ti] OR 

improving [ti] OR improves [ti] OR improve [ti] OR guideline* [ti] OR 

practice* [ti] OR medical [ti] OR provider* [ti] OR physician* [ti] OR 

nurse* [ti] OR clinician* [ti] OR practice guidelines [mh] OR academic 

[ti] OR visit* [ti])) OR Reminder Systems [mh] OR Reminder* [ti] OR 

((financial [ti] OR economic [ti] OR physician* [ti] OR patient*) AND 

incentive* [ti]) OR Reimbursement Mechanisms [mh]

40,152 

#8 

Targets informatics and 

telemedicine 

Medical Informatics [mh] OR computer [ti] OR (decision [ti] AND 

support [ti]) OR Telemedicine[mh] OR Telemedicine [ti] OR 

telecommunication* [ti] OR Internet [mh] OR web [ti] OR modem [ti] 

OR telephone* [ti] OR telephone [mh] 

339608 

#9 

Overall set of articles 

relating to QI 

#3 OR #4 OR #5 OR #6 OR #7 OR #8 1072468 
C-5 

#10 

Combines QI strategies 

search terms with ABX 

search 

#9 AND (#1 OR #2) 75,957

#11 

Systematic review 

search string 

(meta-analysis [pt] OR meta-analysis [tw] OR metanalysis [tw]) OR 

((review [pt] OR guideline [pt] OR consensus [ti] OR guideline* [ti] OR 

literature [ti] OR overview [ti] OR review [ti] OR Decision Support 

Techniques [mh]) AND ((Cochrane [tw] OR Medline [tw] OR CINAHL 

[tw] OR (National [tw] AND Library [tw])) OR (handsearch* [tw] OR 

search* [tw] OR searching [tw]) AND (hand [tw] OR manual [tw] OR 

electronic [tw] OR bibliographi* [tw] OR database* OR (Cochrane [tw] 

OR Medline [tw] OR CINAHL [tw] OR (National [tw] AND Library 

[tw]))))) OR ((synthesis [ti] OR overview [ti] OR review [ti] OR survey 

[ti]) AND (systematic [ti] OR critical [ti] OR methodologic [ti] OR 

quantitative [ti] OR qualitative [ti] OR literature [ti] OR evidence [ti] OR 

evidence-based [ti]))) BUTNOT (case reports [pt] OR case* [ti] OR 

report [ti] OR editorial [pt] OR comment [pt] OR letter [pt] 

46,865 

#12 

Original research 

search string 

Randomised [ti] OR Randomized [ti] OR Controlled [ti] OR 

intervention [ti] OR evaluation [ti] OR impact [ti] OR effectiveness [ti] 

OR Evaluation [ti] OR Studies [ti] OR study [ti] Comparative [ti] OR 

Feasibility [ti] OR Program [ti] OR Design [ti] OR Clinical Trial [pt] OR 

Randomized Controlled Trial [pt] OR Epidemiologic Studies [mh] OR 

Evaluation Studies [mh] OR Comparative Study [mh] OR Feasibility 

Studies [mh] OR Intervention Studies [mh] OR Program Evaluation 

[mh] OR Epidemiologic Research Design [mh] 

2394801 

#13 

Subset of #10 likely to 

include original 

research or systematic 

reviews 

#10 AND (#11 OR #12) 21,936

#14 

Supplementary search 

results 

[See supplement below] 25,569 

#15 #13 OR #14 42317 

#16 #15 BUTNOT (editorial [pt] OR comment [pt] OR letter [pt] OR news 

[pt] OR newspaper article [pt] OR case reports[pt]) 

36850 

#17 #14 limit to Human and English 31200 

#18 #15 limit to June 2004 – November 2004 846 
C-6 

Supplemental search: 

#1 (Drug Therapy [mh] OR Drug Utilization [mh] OR Drug Utilization 

Review [mh] OR Drug Therapy, Combination [mh] OR Decision 

Support Techniques [mh] OR Drug Information Services [mh] OR 

Decision Making [mh] OR Attitude of Health Personnel [mh] OR 

Attitude to Health [mh] OR Clinical Competence [mh] OR Education, 

Continuing [mh] OR Education, Medical, Continuing [mh] OR 

Guideline Adherence [mh] OR Practice Guidelines [mh] OR guideline 

[ti] OR Health Education [mh] OR “Health Knowledge, Attitudes, 

Practice” [mh] OR Health Personnel [mh] OR Health Promotion [mh] 

OR Health Services Misuse [mh] OR Medical Audit [mh] OR Models, 

Educational [mh] OR Counseling [mh] OR Patient Education [mh] OR 

Patient Participation [mh] OR Patient Satisfaction [mh] OR PhysicianPatient Relations [mh] OR Physician's Practice Patterns [mh] OR 

Point-of-Care Systems [mh] OR Prescribing [ti] OR Prescription* [ti] 

OR Prescriptions, Drug [mh]) 

887970 

#2 (Anti-Infective Agents [mh] OR Drug Resistance, Microbial [mh] OR 

Disease Transmission [mh] OR Infection/ drug therapy [mh] OR 

antibiotic* [ab] OR antimicrobial* [ab] OR anti-infective [ab] OR 

Cephalexin [ab] OR Cephalosporin* [ab] OR Chloramphenicol [ab] 

OR Clindamycin [ab] OR Cephalexin [mh] OR Cephalosporin* [mh] 

OR Chloramphenicol [mh] OR Clindamycin [mh] Common Cold/ drug 

therapy [mh] OR Community-Acquired Infections/drug therapy [mh] 

OR Cough/drug therapy [mh] OR Cystitis/drug therapy [mh] OR 

Diarrhea/drug therapy [mh] OR Diarrhea/chemically induced [mh] 

OR Erythromycin [ab] OR Erythromycin [mh] OR Fluoroquinolone* 

[ab] OR Fluoroquinolone* [mh] OR Otitis Media/ drug therapy [mh] 

OR Penicillin* [mh] OR penicillin* [ab] OR Pharyngitis [mh] OR 

Pneumococcal Infections [mh] OR Respiratory Tract Infections [mh] 

OR Rhinitis [mh] OR Streptococcal Infections [mh] OR Streptococcus 

[ab] OR Tetracycline [ab] OR Tonsillitis [mh] OR Trimethoprim [ab] 

OR Tetracycline [mh] OR Trimethoprim [mh] OR Urinary Tract 

Infections [mh] OR Virus Diseases [mh] OR Anti-Infective [ti] OR 

Antiinfective [ti] OR antibiotic* [ti] OR antimicrobial* [ti] OR 

Cephalosporin* [ti] OR Chloramphenicol [ti] OR Clindamycin [ti] OR 

infection* [ti] OR Cystitis [ti] OR Diarrhea [ti] OR Erythromycin [ti] OR 

macrolide* [ti] OR Fluoroquinolone* [ti] OR “Otitis Media” [ti] OR 

Pharyngitis [ti] OR throat [ti] OR cough [ti] OR Pneumococcal [ti] OR 

“Respiratory Tract” [ti] OR Streptococcal [ti] OR Tetracycline [ti] OR 

Tonsillitis [ti] OR Trimethoprim [ti]) 

948535 

#3 #1 AND #2 73845 

#4 (systematic [sb] OR Randomised [ti] OR Randomized [ti] OR 

Controlled [ti] OR intervention [ti] OR evaluation [ti] OR impact [ti] OR 

effectiveness [ti] OR Evaluation [ti] OR Studies [ti] OR study [ti] 

Comparative [ti] OR Feasibility [ti] OR Program [ti] OR Design [ti] OR 

Clinical Trial [pt] OR Randomized Controlled Trial [pt] OR 

Epidemiologic Studies [mh] OR Evaluation Studies [mh] OR 

Comparative Study [mh] OR Feasibility Studies [mh] OR Intervention 

Studies [mh] OR Program Evaluation [mh] OR Epidemiologic 

Research Design [mh]) 

2394833 

#5 #3 AND #4 25,569 
D-1 

Appendix D. Antibiotic Abstraction Forms for 

Screening and Full-Text Review 

Stage 1 

1. Does this article report or evaluate the results of an intervention (whether performed by the investigators or not)? 

o Yes 

o No [exclusion]

o Can't tell 

 

2. Does the article involve quality improvement or a QI strategy? 

o Yes - involves quality improvement or a QI strategy 

o Yes - systematic review of evaluations of a QI strategy 

o No [exclusion]

o Can't tell 

 

***Only answer questions 3-5 if questions 1-2 were answered "Yes"*** 

 

3. Should this article proceed to full text review for this topic? 

o Yes - evaluates a QI strategy involving microbial use 

o No - ineligible topic (focussed on inpatient care, antimicrobial use in chronic disease, not related to 

antimicrobial use) [exclusion]

o No - not an evaluation or not QI [exclusion]

o Can't tell - need article 

o No - foreign language [exclusion]

 

4. What type of study design was used? 

o RCT or quasi-RCT 

o CBA* or ITS** 

o Cohort study; before-after or time series not meeting CBA* or ITS** definitions [exclusion]

o Observational (e.g.,cross-section, case-control) [exclusion]

o Systematic review or meta-analysis 

o Economic or decision analysis, modeling [exclusion]

o Non-research (commentary, review, news) [exclusion]

o Qualitative research (e.g.,focus groups) [exclusion]

o Guideline or consensus statement [exclusion]

o Can't tell (need article) 

 

* CBA: Controlled Before After requires contemporaneous observation periods for control and intervention groups 

AND judgement that control represents a comparable group or setting. 

** ITS: Interrupted Time Series requires statement of well-defined time period for intervention implementation AND 

data measurement for at least three time points before and after intervention. 

NOTE: At this stage of triage, err on the side of inclusion if there is a reasonable chance the study is an RCT, CBA or 

ITS. Similarly, if there is a reasonable chance that the article is a systematic review, designate it as such so the 

article can be pulled. 

 

5. What category of study question is addressed by the article? 

o Can antimicrobial prescribing for illnesses not requiring antimicrobial therapy be reduced? 

o Can the selection of antimicrobials for illnesses requiring antimicrobials be improved? 

o Can prescribing antimicrobials at the appropriate dose or duration of treatment be improved? 

o Unclear 

o Other (explain) 

o None of the above 
D-2 

Stage 2 

1. Does this article merit full text abstraction? 

o Yes 

o No – not QI or not an evaluation of a QI strategy [exclusion]

o No – ineligible study design (i.e., not RCT, CBA, or ITS) [exclusion]

o No - excluded topic (focused only on inpatient care, antimicrobial use in chronic disease) 

 [exclusion]

o No – no eligible outcomes* [exclusion]

o No- other [exclusion]

 

*Eligible outcomes include measures of antimicrobial prescribing, antimicrobial resistance, or health services 

utilization (as a marker of prescription use). 

 

2. Does this article present data overlapping with another article? 

o Exclude this article as a duplicate publication (identify included citation being duplicated) 

 [exclusion]

o Include this article, but obtain listed citation to help with abstraction (e.g., separate methods paper; identify 

required citation ) 

o No or N/A 

3. Does abstraction of this study require information from methods or results reported in other citations? 

o Yes (specify) 

o No 

 

4. Does the article report data for more than one comparison (i.e., should it be abstracted as more than one study)? 

o Yes (specify which comparison is being abstracted here and which others will be abstracted elsewhere) 

o No 

5. What category of study question is addressed by the article? 

o Can antimicrobial prescribing for illnesses not requiring antimicrobial therapy be reduced? 

o Can the selection of antimicrobials for illnesses requiring antimicrobials be improved? 

o Can prescribing antimicrobials at the appropriate dose or duration of treatment be improved? 

o Can antimicrobial resistance be reduced? 

o Can adverse effects of antimicrobial prescribing (i.e., adverse drug events or clinical consequences) be 

reduced? 

o Can the costs associated with antimicrobial prescribing be reduced? 

o Can health services utilization be reduced via improving the antimicrobial prescribing process? 

o Other 

6. Describe the QI strategy used and its salient features. [text box]

A) Study Setting and Participants 

 

7. In what country did the study take place? 

o US 

o non-US (specify) 

 

 

8. When did the study take place? 

o If supplied, give exact dates of study period (beginning to end of intervention period) 

o Not reported 

 

9. Who or what was targeted by the intervention? (check all that apply) 

o Patients 

o Providers (i.e., individual clinicians) 

o Entire population of a geographic area 

o Other (describe) 

 

10. If the intervention targeted providers, in what setting did they practice? (check all that apply) 

o Outpatient primary care clinic (specify type of clinic, e.g.,academic, VA, HMO) 

o Urgent care or walk-in clinic 
D-3 

o Emergency department 

o Other (describe) 

o Not stated or not clear 

12. Which specific disease process was the target of the antimicrobial intervention? (check all that apply) 

o Respiratory tract infection (specify) 

o Urinary tract infection 

o Sexually transmitted disease (e.g.,gonorrhea/chlamydia, syphilis) 

o Gynecologic infection (e.g.,vaginitis) 

o Other infection(s) (describe) 

o No specific disease targeted 

13. Were patients in the study selected on the basis of specific demographic or clinical characteristics? (check all 

that apply) 

o Children (specify age groups) 

o Elderly (specify age groups) 

o Immunocompromised patients (specify) 

o Specific type of insurance (i.e., patients within a particular HMO) (describe) 

o Other demographic or clinical characteristic (describe) 

o No specific patient population targeted 

 14. What type of intervention was provided to the control population? 

o No intervention or usual care 

o Some form of low intensity intervention (describe) 

o No true control - just two or more different types of intervention (discuss with other reviewers; study may 

need to be excluded) 

 

B) Study Design 

 

15. What was the study design? 

o Randomized trial (state method of randomization if described) 

o Quasi-randomized trial (state basis for treatment allocation, e.g.,alternating patients, calendar date, even or 

odd identification numbers) 

o Controlled before-after study* 

o Interrupted time series** 

 

*Controlled Before After (CBA) requires contemporaneous observation periods for control and intervention groups 

AND judgment that control represents a comparable group or setting 

 

** Interrupted time series (ITS) requires statement of well-defined time period for intervention implementation AND 

measurement of data at three or more time points both before and after intervention 

16. What was the unit of randomization or treatment allocation? 

o Patient 

o Episode of care 

o Clinic day 

o Provider 

o Clinic or practice 

o Firm (describe) 

o Institution 

o Community 

o Other 

o Not applicable—ITS study (skip to question 27) 

 

17. For the unit of treatment allocation above, state sample size in each group (If sample size differs for 

outcomes, detail differences in "Not stated or not clear" text box): 

o control group 

o intervention group 

o Not stated or not clear (explain) 

 
D-4 

18. If unit of analysis differed from unit of treatment allocation (e.g., providers randomized, but patient outcomes 

analyzed), state sample size in each group: (Use text box for "Not applicable" if sample size for any outcomes 

reported is different-give details) 

o control group 

o intervention group 

o Not stated or not clear 

o Not applicable (unit of analysis same as unit of treatment allocation above) 

 

19. If unit of analysis differed from unit of treatment allocation, did authors acknowledge this issue and/or make 

appropriate adjustments? 

o Yes (describe) 

o No 

o Not applicable (unit of analysis did not differ from unit of treatment allocation) 

 

20. Was there adequate concealment of treatment allocation? 

o Yes (unit of allocation was institution, team or professional and randomization process explicity described, 

OR unit of allocation was patient or episode of care and some form of centralized randomization scheme or 

sealed envelopes used) 

o Not clear (only partially meets above criteria) or not stated - specify which 

o No inadequate concealment (enrollment of patients in alternation or through use of even/odd identifying 

numbers OR unit of allocation was patient or episode of care and reported use of any allocation process that 

is entirely transparent before assignment (e.g., open list of random numbers) OR allocation was altered by 

investigators, professionals or patients) 

21. Was informed consent obtained? (check all that apply) 

o Obtained from patients 

o Obtained from providers 

o Not obtained or not stated (specify) 

22. Was IRB approval obtained by investigators? 

o Yes 

o No or not stated (specify) 

Design criteria for randomized and quasi-randomized trials 

(If study is a CBA, skip to question 27; if ITS, skip to question 33) 

23. Did the study have a cross over design? (Patients randomized to a sequence of interventions such as treatment 

A followed by treatment B in one group and treatment B followed by treatment A in the other group). 

o Yes (describe) 

o No 

o Not sure - clarify with other reviewers before proceeding 

 

24. Were patients blind to intervention/treatment allocation? 

o Yes 

o No 

o Not sure (explain) 

o Not applicable (patients not actively involved in study - e.g., provider-focused intervention with patient level 

data obtained retrospectively from charts) 

25. Were providers blind to intervention/treatment allocation? 

o Yes 

o No 

o Not sure (explain) 

o Not applicable (explain) 

26. Were outcomes assessors blinded to intervention/treatment allocation? 

o Yes 

o No 

o Not sure (explain) 

o Not applicable (explain) 
D-5 

Design criteria for CBA trials 

27. Were measurements in the control group performed at the same time as the intervention group? 

o Yes 

o No 

o Unclear 

28. Were the criteria used for selecting the control site explained? 

o Yes (describe) 

o No 

29. Was the control site comparable (in both patients and providers)? 

o Yes 

o No (explain why not) 

o Unclear (describe) 

Design criteria for ITS trials 

30. Was the intervention performed independent of other quality improvement efforts or other changes? 

o Yes 

o No 

o Unclear 

31. Was the intervention unlikely to affect data collection? 

o Yes 

o No 

o Unclear 

32. Was the data analyzed using a formal test for trend (time series ANOVA or regression)? 

o Yes 

o No 

o Unclear 

 

33. (For all studies) Do any methodologic aspects of the study design not captured above seriously undermine 

appropriateness of inclusion? 

o Yes (explain) 

o No (use text box to document any non-fatal, but still noteworthy methodological features) 

 

C) Quality Improvement Attributes of Intervention 

34. Did the investigators identify a specific quality gap (a difference between optimal and actual care) in the study 

population? 

o Yes (describe) 

o No 

35. Did the investigators cite previous literature to describe the evidence base for their proposed intervention? 

o Yes 

o No 

36. Did the QI strategy involve PATIENT EDUCATION? 

o Yes 

o No patient education (skip to question 36) 

37. Which of the following educational strategies was used? (check all that apply) 

o One-on-one session, in person or via telephone 

o Group session (e.g.,classes) 

o Distribution of printed or audiovisual materials (e.g.,pamphlets or poster in waiting room) 

o Community-wide mass media efforts (e.g.,television advertisements or billboards) 

o Interactive computer-based learning 

o Provision of clinical data to patient (e.g.,test results) 

o Not sure or other (describe) 
D-6 

38. In what setting was the educational content delivered? (check all that apply) 

o Clinical setting (e.g.,office or emergency department) 

o Home (in person, by phone, mail or internet) 

o School or workplace 

o Other community setting (e.g.,church, community center) (describe) 

o Mass media exposure 

o Other or unclear (describe) 

39. Who was responsible for delivery of the educational content? (check all that apply) 

o Physician 

o Nurse or nurse practitioner 

o Pharmacist 

o Other ancillary health provider (describe) 

o Health educator 

o No specific delivery person (e.g.,entirely mailed, computer-based, or passively distributed content) 

o Other (describe) 

40. Did the intervention include access to a resource that promoted PATIENT SELF- MANAGEMENT? 

o Delayed prescriptions or other therapy based on patient self-monitoring of symptoms (describe) 

o Decision support for patient to use before seeing health care provider (e.g.,access to a call center) 

(describe) 

o Other type of decision support (describe) 

o No 

 

41. Did the QI strategy involve PROVIDER EDUCATION? 

o Yes 

o No (skip to question 42) 

 

42. Which of the following educational strategies was used? (check all that apply) 

o Distribution of educational materials (published or printed recommendations for clinical care, including 

clinical practice guidelines, audio-visual materials and electronic publications) 

o Meetings or lectures (e.g.,traditional CME) 

o Educational outreach visits (e.g.,“academic detailing”—a trained person who met with providers in their 

practice settings to give information with the intent of changing the provider's practice) 

o Interactive in-person education (e.g.,workshops or role-playing) 

o Computer- or internet-based interactive tutorials 

o Consensus-building sessions (e.g.,for development of guideline) 

o Not sure or other (describe) 

43. In what setting was the educational content delivered? (check all that apply) 

o Office, emergency department, or other clinical setting 

o Off-site meeting (e.g.,CME) 

o Other (describe) 

44. Who was responsible for delivery of the educational content? (check all that apply) 

o Expert opinion leader (describe how selected) 

o Other physician (including colleagues) 

o Pharmacist 

o Pharmaceutical sales representative 

o Other (describe) 

o Not clear or not specified 

45. Did the QI strategy involve a PROVIDER REMINDER system*? 

o Chart based decision support or reminder system* 

o Computer based reminder* or decision support* 

o Not sure 

o No or N/A 

 

* Patient or provider encounter specific information, provided verbally, on paper or on a computer screen, which is 

intended to prompt provider to recall information at the time of the patient encounter (e.g., reminder that antibiotics 

are ineffective for the common cold) 
D-7 

46. Did the QI strategy involve provider AUDIT AND FEEDBACK*? (check all that apply) 

o feedback to individual provider (state if confidential) 

o feedback about clinic or practice performance only 

o Public reporting of performance data (state if individual data or data for a group or institution) 

o Benchmarking** 

o Not sure or other 

o No or N/A 

 

*Any summary of clinical performance of health care over a specified period of time. E.g., the number of times over 

the last year a provider prescribed antimicrobials for bronchitis. 

**Benchmarking refers to the provision of performance data from institutions or providers regarded as "leaders in the 

field." These data provide targets for other providers and institutions to emulate. 

 

47. Did the QI strategy involve ORGANIZATIONAL CHANGE? 

o Adding new members to team (e.g., adding a clinical pharmacist to clinic, or creation of a call center for 

patients) or creating multidisciplinary teams for patient care 

o Revision of professional roles among health professionals (e.g., nurse practitioner or pharmacist given 

prescribing authority) 

o Increased staffing without changes in roles (e.g., adding more nurses) 

o TQM/CQI - cycles of measurement of quality problems, design of interventions, implementation and remeasurement 

o Changes in medical records systems -- e.g.,changing from paper to computerized records, patient tracking 

systems 

o Communication and case discussion between distant health professionals (e.g., telemedicine) 

o Not sure or other (describe) 

o No or N/A 

 

48. If the intervention involved changes to medical record systems, what type of change was instituted? 

o Change from paper to computerized records 

o Implementation of computerized provider order entry (CPOE) 

o New patient tracking system 

o Other (describe) 

o Not applicable - No change to medical record system 

 

49. Did the intervention involve FINANCIAL INCENTIVES DIRECTED AT PROVIDERS? 

o Financial incentives for achievement of performance goals 

o Change in reimbursement system (i.e., capitation) 

o Other (describe) 

o No component of provider-directed financial incentives 

50. Did the intervention involve REGULATORY CHANGES DIRECTED AT PROVIDERS? 

o Restriction of formulary to cover only certain antimicrobials 

o Authorization from another physician required to prescribe antimicrobial 

o Authorization from health plan required to prescribe antimicrobial 

o Restriction of access to pharmaceutical sales representatives 

o Other (describe) 

o No component of provider-directed regulatory changes 

51. Did the intervention involve FINANCIAL OR REGULATORY INCENTIVES DIRECTED AT PATIENTS? 

o Additional charge (copayment) for specific antimicrobials 

o Additional charge (copayment) for visits or phone calls 

o Change in health insurance premiums 

o Other (describe) 

o No component of patient-directed financial or regulatory incentives 

52. Did the intervention involve FINANCIAL OR REGULATORY INCENTIVES DIRECTED AT A PRACTICE OR 

HEALTH SYSTEM? 

o Yes (describe) 

o No component of health-system-directed financial or regulatory incentives 
D-8 

53. Did a clinical information system play a role in design or implementation of the intervention? 

o Identification and/or group allocation of eligible patients or providers 

o Reminders generated by existing clinical information system 

o Decision support at point of care (e.g., for provider order entry) 

o Facilitated communication between providers (e.g., generated emails between members of care team) 

o Audit data gathered from clinical information system to design QI strategy (e.g., audit and feedback, TQM, 

provider education, financial incentives) 

o Other 

o No role for a clinical information system 

54. Use textbox to state any important study features or concerns not captured above. 

 

D) Results 

 

55. For unit of treatment allocation (e.g., clinics, providers, patients), were results reported for at least 80% of 

participants? 

o Yes (state %) 

o No (state %) 

o Not stated 

 

56. If unit of analysis differed from unit of treatment allocation (e.g., providers randomized, but patient level outcomes 

analyzed), were results reported for at least 80% of participants? 

o Yes (state %) 

o No (state %) 

o Not stated or not clear 

o Not applicable (unit of analysis same as unit of treatment allocation) 

57. What was the length of the study follow-up period? (describe) 

 

58. What were the outcome types? (check all that apply) 

o Measure of antimicrobial use 

o Measure of antimicrobial resistance 

o Measure of clinical outcome of disease 

o Health services utilization (when used as a marker of antimicrobial use) 

o Adverse drug events 

o Cost 

o Patient or provider satisfaction with plan of care 

o Not sure or other 

 

Measures of antimicrobial use 

59. For studies reporting measures of antimicrobial use, what specific outcomes were measured? (check all that 

apply) 

o Total prescriptions for antimicrobials (overall and/or a specific antimicrobial or class) by an individual 

provider, practice or health care system 

o Total volume of antimicrobials used by a provider, practice, or health care system (e.g.,number of defined 

daily doses of an antibiotic prescribed by a physician) 

o Percentage of patient visits resulting in an antimicrobial prescription (overall and/or use of a specific 

antimicrobial or class) 

o Rate of compliance to clinical guideline for antimicrobial prescription (including decision to prescribe 

antimicrobial, choice of antimicrobial, or dose/duration of therapy) 

o Other (describe) 

o No measurement of antimicrobial use (skip to question 69) 

60. For studies reporting data in the form of total prescriptions for antimicrobial use, provide the following data for the 

CONTROL group; if data is missing, record “NR” 

o Exact units of measurement 

o Number of prescriptions prior to intervention 

o Number of prescriptions after intervention 

o Percentage change in number of prescriptions 
D-9 

61. For studies reporting data in the form of total prescriptions for antimicrobial use, provide the following data for the 

INTERVENTION group; if data is missing, record “NR” 

o Exact units of measurement 

o Number of prescriptions prior to intervention 

o Number of prescriptions after intervention 

o Percentage change in number of prescriptions 

62. For studies reporting data in the form of total volume of antimicrobial prescribed, provide the following data for 

the CONTROL group; if data is missing, record “NR” 

o Exact units of measurement 

o Volume of antibiotic used prior to intervention 

o Volume of antibiotic used after intervention 

o Percentage change in volume of antibiotic used 

63. For studies reporting data in the form of total volume of antimicrobial prescribed, provide the following data for 

the INTERVENTION group; if data is missing, record “NR” 

o Exact units of measurement 

o Volume of antibiotic used prior to intervention 

o Volume of antibiotic used after intervention 

o Percentage change in volume of antibiotic used 

64. For studies reporting data in the form of percentage of visits resulting in an antimicrobial prescription, provide the 

following data for the CONTROL group; if data is missing, record “NR” 

o Percentage of visits resulting in an antimicrobial prescription prior to intervention 

o Percentage of visits resulting in an antimicrobial prescription after intervention 

o Absolute change in percentage (after – prior) 

65. For studies reporting data in the form of percentage of visits resulting in an antimicrobial prescription, provide the 

following data for the INTERVENTION group; if data is missing, record “NR” 

o Percentage of visits resulting in an antimicrobial prescription prior to intervention 

o Percentage of visits resulting in an antimicrobial prescription after intervention 

o Absolute change in percentage (after – prior) 

66. For studies reporting data in the form of adherence to a guideline for antimicrobial prescribing, provide the 

following data for the CONTROL group; if data is missing, record “NR” 

o Percent adherence prior to intervention 

o Percent adherence after intervention 

o Absolute change in percentage (after – prior) 

67. For studies reporting data in the form of adherence to a guideline for antimicrobial prescribing, provide the 

following data for the INTERVENTION group; if data is missing, record “NR” 

o Percent adherence prior to intervention 

o Percent adherence after intervention 

o Absolute change in percentage (after – prior) 

68. Provide the following data for other types of measurements of antibiotic use not abstracted above: 

o Type of measurement and units of measurement 

o Value in CONTROL group before intervention 

o Value in CONTROL group after intervention 

o Value in INTERVENTION group before intervention 

o Value in INTERVENTION group after intervention 

Measures of antimicrobial resistance 

 

69. For studies reporting measures of antimicrobial resistance, what specific outcomes were 

measured? (check all that apply) 

o Percentage of isolates of an organism resistant to an antimicrobial or class of antimicrobials 

o Percentage of patients infected with a resistant organism 

o Rate of treatment failure for a specific antimicrobial used to treat a specific condition 

o Other (describe) 

o No measurement of antimicrobial resistance (skip to question 79) 
D-10 

70. For studies reporting the percentage of isolates resistant to an antimicrobial, record the following data; if data is 

missing, record “NR” 

o Type(s) of organism 

o Active infection or colonizer 

o Antimicrobial (or class of antimicrobial) for which resistance was assessed 

71. For studies reporting the percentage of isolates resistant to an antimicrobial, record the following data for the 

CONTROL group; if data is missing, record “NR” 

o Number of isolates before intervention 

o Percentage of isolates resistant to antimicrobial BEFORE intervention 

o Number of isolates after intervention 

o Percentage of isolates resistant to antimicrobial AFTER intervention 

o Change in percentage of resistant isolates (after – before) 

72. For studies reporting the percentage of isolates resistant to an antimicrobial, record the following data for the 

INTERVENTION group; if data is missing, record “NR” 

o Number of isolates before intervention 

o Percentage of isolates resistant to antimicrobial BEFORE intervention 

o Number of isolates after intervention 

o Percentage of isolates resistant to antimicrobial AFTER intervention 

o Change in percentage of resistant isolates (after – before) 

73. For studies reporting the percentage of patients infected with a resistant organism, record the following data for 

the CONTROL group; if data is missing, record “NR” 

o Number of patients evaluated before intervention 

o Percentage of patients with resistant organism BEFORE intervention 

o Number of patients evaluated after intervention 

o Percentage of patients with resistant organism AFTER intervention 

o Change in percentage of resistant isolates (after – before) 

74. For studies reporting the percentage patients infected with a resistant organism, record the following data for the 

INTERVENTION group; if data is missing, record “NR” 

o Number of patients evaluated before intervention 

o Percentage of patients with resistant organism BEFORE intervention 

o Number of patients evaluated after inervention 

o Percentage of patients with resistant organism AFTER intervention 

o Change in percentage of resistant isolates (after – before) 

75. For studies reporting the rate of treatment failure for a specific antimicrobial used to treat a specific condition, 

record the following data: 

o Type of condition 

o Antimicrobial or class of antimicrobial 

76. For studies reporting the rate of treatment failure for a specific antimicrobial used to treat a specific condition, 

record the following data for the CONTROL group; 

o Number of treated patients before intervention 

o Rate of treatment failure (as a percentage) before intervention 

o Number of patients after intervention 

o Rate of treatment failure (as a percentage) after intervention 

o Change in rate of treatment failure (after – before) 

77. For studies reporting the rate of treatment failure for a specific antimicrobial used to treat a specific condition, 

record the following data for the INTERVENTION group; 

o Number of treated patients before intervention 

o Rate of treatment failure (as a percentage) before intervention 

o Number of treated patients after intervention 

o Rate of treatment failure (as a percentage) after intervention 

o Change in rate of treatment failure (after – before) 

78. Provide the following data for other types of measurements of antimicrobial resistance not abstracted above: 

o Type of measurement and units of measurement 

o Value in CONTROL group before intervention 
D-11 

o Value in CONTROL group after intervention 

o Value in INTERVENTION group before intervention 

o Value in INTERVENTION group after intervention 

Measures of disease outcomes 

Note: measures of disease outcomes should be abstracted only if the study also has usable data for one of the 

primary outcomes (antimicrobial usage or antimicrobial resistance). 

79. For studies reporting clinical disease outcomes, what specific outcomes were measured? 

o Subjective measure of disease severity (e.g.,time to symptom resolution) 

o Objective measure of disease severity (e.g.,symptom score) 

o Other (describe) 

o No measurement of disease outcomes (skip to question 82) 

 

80. For studies reporting disease outcomes as subjective measures of disease severity, record the following data: 

o Definition of clinical outcome as supplied in article, including unit of measurement 

o Value for CONTROL group 

o Value for INTERVENTION group 

81. For studies reporting disease outcomes as objective measures of disease severity, record the following data: 

o Definition of clinical outcome as supplied in article 

o Value for CONTROL group 

o Value for INTERVENTION group 

Measures of health services utilization 

Note: measurements of health services utilization should be abstracted only if the study also has usable data for one 

of the primary outcomes (antimicrobial usage or antimicrobial resistance). 

82. For studies reporting on health services utilization, what specific outcomes were measured? 

o Percentage of patients requiring an initial visit with a health care provider 

o Percentage of patients requiring a follow-up visit with a health care provider (may include return visit to clinic 

or emergency department, or hospitalization) 

o Other (describe) 

o No measurement of health services utilization (skip to question 86) 

83. For studies reporting the percentage of patients requiring an INITIAL visit with a health care provider, record the 

following data: 

o Percentage in CONTROL group 

o Percentage in INTERVENTION group 

o Absolute difference (intervention – control) 

84. For studies reporting the percentage of patients requiring a FOLLOW-UP visit with a health care provider, record 

the following data: 

o Type of health care setting(s) at follow-up visit 

o Percentage in CONTROL group 

o Percentage in INTERVENTION group 

o Absolute difference (intervention – control) 

85. For studies measuring health services utilization not captured above, record the measurement and results: 

Measures of adverse events 

Note: adverse events should be abstracted only if the study also has usable data for one of the primary outcomes 

(antimicrobial usage or antimicrobial resistance). 

86. For studies reporting measures of adverse events related to antimicrobial use, how were the adverse events 

assessed? 

o Rate of adverse drug events (i.e., clostridium dificile colitis, rash or other allergic reaction) 

o Incidence of severe infections possibly due to undertreatment (e.g.,bacteremia) 

o Other (describe) 

o No measurement of adverse events (skip to question 90) 
D-12 

87. For studies reporting the rate of adverse drug events, record the following data: 

o Specific type of ADE measured 

o Rate in CONTROL group 

o Rate in INTERVENTION group 

o Absolute difference (intervention – control) 

88. For studies reporting the rate of severe infections possibly due to undertreatment, record the following data: 

o Specific type of infection measured 

o Rate in CONTROL group 

o Rate in INTERVENTION group 

o Absolute difference (intervention – control) 

89. For studies reporting measurement of adverse events not captured above, record the measurement and results: 

Measures of costs 

Note: cost outcomes should be abstracted only if the study also has usable data for one of the primary outcomes 

(antimicrobial usage or antimicrobial resistance). 

90. For studies reporting cost outcomes, what specific measures were used? 

o Total cost of antimicrobials to a practice, hospital, or health system 

o Cost of health services utilization 

o Other (describe) 

o No measurement of costs (skip to question 94) 

91. For studies reporting the total cost of antimicrobials, record the following data: 

o Exact units of measurement 

o Total costs in CONTROL group before intervention 

o Total costs in CONTROL group after intervention 

o Total costs in INTERVENTION group before intervention 

o Total costs in INTERVENTION group after intervention 

o Cost difference before intervention (intervention – control) 

o Cost difference after intervention (intervention – control) 

o Net change in costs attributable to intervention (cost difference after – cost difference before) 

92. For studies reporting the total cost of health services utilization, record the following data: 

o Exact units of measurement 

o Total costs in CONTROL group before intervention 

o Total costs in CONTROL group after intervention 

o Total costs in INTERVENTION group before intervention 

o Total costs in INTERVENTION group after intervention 

o Cost difference before intervention (intervention – control) 

o Cost difference after intervention (intervention – control) 

o Net change in costs attributable to intervention (cost difference after – cost difference before) 

93. For studies reporting cost measurements not abstracted above, record the units of measurement and results: 

Patient or provider satisfaction with care 

Note: patient/provider satisfaction should be abstracted only if the study also has usable data for one of the primary 

outcomes (antimicrobial usage or antimicrobial resistance). 

94. For studies reporting data on PATIENT satisfaction with care, provide the following data; if data is missing, 

record “NR” 

o No measurement of patient satisfaction 

o Percent satisfaction in CONTROL group after intervention 

o Percent satisfaction in INTERVENTION group after intervention 

o Absolute difference (intervention – control) 
D-13 

95. For studies reporting data on PROVIDER satisfaction with care, provide the following data; if data is missing, 

record “NR” 

o No measure of provider satisfaction 

o Percent satisfaction in CONTROL group after intervention 

o Percent satisfaction in INTERVENTION group after intervention 

o Absolute difference (intervention – control) 

96. Use textbox to state any important results or study concerns not captured above. 
E-1 

Appendix E. Random Effects Meta-Analysis of 

Included Trials 

A. Methods 

For our initial analytic approach, we identified studies that supplied information on the 

post-intervention rate of antibiotic use in both intervention and control groups. The 

primary outcomes were the same as used in the main report (for treatment articles, the 

percentage of visits at which an antibiotic was prescribed; for selection articles, the 

percentage of patients receiving a recommended antibiotic or the percentage 

compliance with a clinical guideline for antibiotic prescribing). We used the pooled risk 

difference as the summary statistic; the risk difference for each individual study was 

equal to (interventionpost - controlpost). Thus, for the treatment articles, the risk difference 

indicates the reduction in antibiotic prescribing in the intervention group compared to 

control. In the selection articles, the risk difference indicates the improvement in 

prescribing of a recommended antibiotic in the intervention group compared to control. 

We performed meta-analysis using the random effects model of DerSimonian and Laird 

on these data. In order to be suitable for meta-analysis, included studies were required 

to report the number of patients in each group (intervention and control). Analyses were 

performed using Stata version 8.2 (Stata Corp. College Station, TX). 

B. Results 

Antibiotic treatment decision studies. Twenty-one articles reporting a total of 26 

comparisons met the inclusion criteria. The pooled risk difference across all studies 

was -13.8% (95% CI -17.7% to -9.9%,) indicating a significant reduction in antibiotic 

prescribing, as illustrated in the Figure below. However, significant heterogeneity was 

present, with the chi-squared test of heterogeneity significant at p < 0.001. The I2

 value, 

which indicates the percentage of the variance in the risk difference attributable to 

heterogeneity, was 96.2%. 
E-2 

We performed stratified analyses to investigate potential sources of heterogeneity. 

Having identified delayed prescribing studies as a unique subset of interventions, we 

repeated the analyses without these studies. The results continued to show a 

statistically significant reduction in prescribing (pooled RD -9.6%, 95% CI -12.6% to -

6.9%,) but the tests of heterogeneity remained significant (p<0.001 for chi-squared test, 

I

2

 92.9%). We found consistent evidence for high levels of heterogeneity when 

analyses were further restricted to randomized controlled trials (p<0.001, I2 86.4%), 

interventions targeting ARIs only (p<0.001, I2 93.8%), or studies with baseline 

prescribing rates above the median (p<0.001, I2 92.3%) or below the median (p<0.001, 

I

2

 97.1%). Heterogeneity was present for studies using each specific QI strategy as 

well, with p<0.001 for all individual QI strategies and combinations of strategies. 

Heterogeneity may always be present in a meta-analysis, and finding heterogeneity 

generally warrants a search for potential sources of heterogeneity via stratified 

analyses. However, we recognize that pooling studies of quality improvement 

strategies of necessity involves combining studies across a broad range of target 

populations, target conditions, baseline care practices, and interventions. Although we 

selected articles reporting a common outcome, the outcome itself may have been 

measured in different ways (e.g., direct measurement of prescribing by chart review, 

versus measurement of actual antibiotic use by patient interview or pharmacy records). 

Antibiotic selection decision studies. For the selection decision studies, an 

additional potential complication was induced by combining articles with an explicit 

clinical guideline for prescribing and those with recommendations but no defined 

guideline. Thus, we did not feel that stratified analyses alone could identify any factors 

responsible for heterogeneity, and this was borne out by the results in stratified 

analyses of the treatment studies. 

Based on these results, we did not feel that performing random-effects meta-analysis 

would be appropriate for either the treatment studies or the selection studies, and opted 

to use the median effects approach as described in the Methods section.
E-3 

Figure. Forest plot showing risk differences for treatment studies by random effects meta-analysis. 

 Risk difference

 -.81650 0 .816505

 Study Risk difference (95% CI) % Weight

 46 -0.08 (-0.13,-0.03) 4.3 

 53 -0.47 (-0.61,-0.32) 2.8 

 69 0.04 (-0.03, 0.11) 4.0 

 104 -0.09 (-0.22, 0.03) 3.1 

 118 -0.06 (-0.08,-0.03) 4.6 

 132 -0.28 (-0.33,-0.23) 4.3 

 149 0.07 ( 0.01, 0.12) 4.3 

 174 -0.75 (-0.82,-0.67) 4.0 

 180 -0.15 (-0.29,-0.02) 3.0 

 182 -0.09 (-0.10,-0.08) 4.6 

 192 -0.08 (-0.17, 0.01) 3.7 

 193 0.00 (-0.07, 0.07) 4.0 

 197 -0.26 (-0.31,-0.21) 4.3 

 200 -0.16 (-0.23,-0.09) 4.0 

 233 -0.19 (-0.23,-0.15) 4.4 

 278 -0.08 (-0.12,-0.04) 4.4 

 284 -0.07 (-0.11,-0.03) 4.4 

 285 -0.14 (-0.18,-0.10) 4.4 

 458 -0.27 (-0.37,-0.18) 3.7 

 581 0.01 (-0.04, 0.06) 4.4 

 587 -0.10 (-0.11,-0.08) 4.6 

 588 -0.04 (-0.05,-0.02) 4.6 

 594 -0.26 (-0.47,-0.05) 2.0 

 595 -0.17 (-0.25,-0.10) 3.9 

 601 -0.01 (-0.08, 0.05) 4.1 

 Overall -0.14 (-0.18,-0.10) 100.0 

 Reduction in antibiotic prescribing Increase in antibiotic prescribing 

Figure Legend 

Forest plot for the risk difference for antibiotic prescribing in intervention groups post-intervention as compared with comparison 

groups post-intervention. Negative numbers indicate reductions in antibiotic use.
F-1 

Appendix F. Technical Expert Panel and Peer 

Reviewers 

Technical Expert Panel 

Jeremy Grimshaw, MBChB, PhD, University of Ottawa, Cochrane Collaboration 

Effective Practice and Organisation of Care Group (EPOC) 

Andrew Oxman, MD, MSc, Department of Health Services Research, Norwegian 

Directorate for Health and Social Welfare, EPOC 

Russell Glasgow, PhD, Kaiser Permanente, Colorado 

Martin Eccles, MD, Centre for Health Services Research, University of Newcastle upon 

Tyne, EPOC 

Harmon Jordan, ScD, previously with New England Medical Center Evidence-based 

Practice Center, currently at Abt Associates 

Val Lawrence, MD, MSc, The University of Texas Health Science Center at San Antonio 

and South Texas Veterans Health Care System 

James Zazzali, PhD, RAND 

Charles Homer, MD, MPH, National Institute for Children's Health Care Quality 

Peer Reviewers 

Richard Besser, MD, Medical Epidemiologist, Centers for Disease Control and 

Prevention 

Edward Belongia, MD, Epidemiology, Marshfield Clinic 

Jonathan Finkelstein, MD, MPH, Department of Ambulatory Care and Prevention, 

Harvard Medical School 

Kitty Corbett, PhD, Anthropologist, Denver University 

Randall Stafford, MD, PhD, Stanford University 

Brian Strom, MD, MPH, CERT Program, University of Pennsylvania 

Theoklis Zaoutis, MD, Department of Pediatrics, University of Pennsylvania School of 

Medicine