Randomized-controlled Trial to Assess Whether Feedback on a New Stewardship Metric Can Improve Antibiotic-prescribing for Acute Respiratory Tract Infections in Urgent Care Clinics
Phase: NA
Conditions: Antibacterial Agents, Ambulatory Care
Sponsor: Daniel Livorsi

1 
 
Randomized-controlled trial to assess whether feedback on a new stewardship metric 
can improve antibiotic-prescribing for acute respiratory tract infections 
 
Protocol date January 4, 2024 
Version 1 
 
Principal investigator: Daniel Livorsi, MD, MSc 
 
Data analyst/statistician: Amy O’Shea, PhD 
 
 

2 
 
BACKGROUND 
 
Urgent care clinics are a rapidly expanding provider of same-day outpatient care in the United 
States because they offer easier access and lower out-of-pocket costs than Emergency 
Departments. Antibiotics are frequently prescribed in Urgent Care settings, and a large 
proportion of these antibiotics are unnecessary. Unnecessary antibiotics provide no clinical 
benefit to the patient but increase the risk of an antibiotic-related adverse event and reinforce 
patients’ antibiotic-seeking behavior. 
 
Within the University of Iowa Hospitals and Clinics (UIHC) system, antibiotics were prescribed to 
30% of all visits to the Urgent and QuickCare clinics during 2018-2022. One out of every two 
antibiotics prescribed in these settings was for a respiratory tract illness. 
 
Our team has used an established HEDIS (Healthcare Effectiveness Data and Information Set) 
metric to evaluate clinicians on their antibiotic use for respiratory tract diagnoses (RTDs). This 
RTD metric excludes visits that are more complicated, such as repeat visits for the same 
complaint, visits with concomitant non-respiratory infections, and visits for patients with 
advanced comorbidities, such as immunosuppression. During 2018-2022, the mean frequency 
of antibiotic-prescribing at the clinician level for RTDs was 37.9% (standard deviation 5.4%) 
within the UIHC Urgent/QuickCare system. We estimate that the optimal antibiotic-prescribing 
frequency for RTDs should be around 28%, or roughly 25% lower than it is right now.  
 
This protocol describes our plan to evaluate the effect of giving clinicians individualized 
feedback on the RTD metric. The proposed project will involve 3 Urgent Care and 4 QuickCare 
clinics within the UIHC system. If additional clinics come on-line during the trial, clinicians at 
those sites will be invited to join. Currently, clinics are staffed by approximately 40 clinicians and 
provide 80,000 patient-visits annually.  
 
Objectives: To evaluate whether providing individualized feedback to clinicians on the RTD 
metric can safely reduce antibiotic use for qualifying respiratory tract visits across Urgent and 
QuickCare settings within the UIHC system. 
 
METHODS 
 
Trial design:  
• 
We will perform a matched-pair randomized controlled trial to assess the effectiveness of 
giving individualized feedback to clinicians on the RTD metric.  
• 
Initially we will balance allocation between the two study arms in terms of two factors: 
o Only providers who have had a qualifying RTD visit in the past 6 months will be 
included in this initial randomization. 
o Each provider’s frequency of antibiotic-prescribing for RTD visits during the year 
prior to the feedback intervention starting; 
o Each provider’s number of RTD visits over the year prior to the feedback 
intervention starting. 
• 
Providers newly hired to UIHC after study start will also be randomized. For new hires, 
we anticipate 1-year historical prescribing information will not be available. We will thus 
randomize pairs of providers in 1:1 ratio to the two study arms.  
 
 
 

3 
 
Participants:  
 
We will seek to enroll all active clinicians in Urgent and QuickCare settings who do not opt-out 
of the trial.  
 
Inclusion criteria: Active clinician in UIHC’s Urgent and QuickCare system 
 
Exclusion criteria: None 
 
Method of recruitment: All Urgent Care and QuickCare clinicians will be notified during both a 
monthly staff meeting and by e-mail of the planned randomization process outlined below 
approximately two weeks prior to the randomization happening. Clinicians will be given two 
weeks to opt-out of the study by e-mail. If clinicians do not opt-out, they will be enrolled in the 
randomization process. Once the intervention starts, clinic leadership will notify the study team 
about any new hires on a monthly basis. New hires will notified about the study by e-mail during 
the month prior to the next feedback point; these providers will be given two weeks to opt-out 
before randomization occurs. 
 
Interventions: 
 
Baseline activities:  
 
At baseline, UIHC’s antibiotic stewardship team has implemented antibiotic-prescribing order 
sets and a monthly feedback process for all Urgent/QuickCare clinicians. This feedback process 
involves a never-event metric, which quantifies the number of times a clinician prescribes an 
antibiotic for a respiratory condition that is exclusively viral in etiology. Feedback on this never-
event metric will continue during the trial in both the control and experimental arms.  
 
Intervention activities: 
 
After the randomization is performed by the biostatistician, all clinicians who did not opt-out will 
be notified by e-mail of their assignment to either the control or experimental arm. 
 
From that point forward, an e-mail will be sent to the clinicians in the experimental arm every 
two months describing their performance on the RTD metric. If a clinician had more than 20 
qualifying RTD visits within the prior two months, his/her performance will be compared to 
peers, both in words and graphically. The e-mail will also direct clinicians to visit a dashboard in 
the electronic medical record where they can review the patient visits captured by the metric in 
greater depth. 
 
Any clinician assigned to the control arm will not receive these e-mails but will continue to be 
informed of their never-event metric, as outlined above. 
 
Antibiotic-prescribing data for both experimental and control clinicians will be collected both 
retrospectively (beginning in January 2022) and prospectively through the end of the 18-month 
trial. Data on all clinicians will be identifiable but will be kept strictly confidential.  
 
Outcomes:  
 
We will use the RE-AIM framework to assess our pilot trial, as outlined on page 5.  
 

4 
 
Primary outcome: The primary outcome of effectiveness will be the frequency at which antibiotics 
are prescribed for qualifying RTD visits. 
 
Secondary outcomes: To ensure that this intervention is safe, we will measure the following 
outcomes: antibiotic prescriptions written after but within 30-days of the index clinic visit, 30-day 
return visits to an Urgent/QuickCare clinic or the Emergency Department, 30-day hospital 
admission rate, and 30-day C. difficile testing and infection rate. We will also track the 
proportion of monthly Urgent/QuickCare visits within tiers 1, 2 and 3 to ensure that diagnostic 
shifting does not occur over time. Finally, we will track total antibiotic prescriptions per 1,000 
patient visits. Because clinicians may stop using RTD-related diagnostic codes once they know 
they are being tracked on this metric, quantifying total antibiotic use will provide another way for 
us to determine whether changes in antibiotic-prescribing have occurred. 
 
Process measures and qualitative outcomes: During the trial, we will track dashboard visits. A 
qualitative researcher (K. Dukes) will design a post-trial electronic survey or semi-structured 
interview guide that we will use to evaluate clinician perceptions of the RTD metric and the 
dashboard after this pilot study ends.  
Sample Size:  Our power calculation was based on the following assumptions: 25,000 RTD 
visits per year across 38 clinicians, a baseline antibiotic prescription rate of 35%, and 18-months 
of data for the baseline and intervention periods. Using an interrupted time series model within 
indicators for implementation, time, and the interaction of the two and a type I error rate of 5%, 
we calculated more than 90% power to detect a 4% absolute difference in antibiotic prescribing 
between the experimental and control arms. 
Blinding: None 
 
Unit of Analysis: Each participating clinician  
 
Statistical Methods: After the 18-month intervention period, we will perform an intention to 
treat analysis. We will use logistic regression to perform a stratified interrupted time series (ITS) 
analysis to assess the level and slope change for antibiotic use between the intervention and 
baseline periods within the experimental and control arms. We will use a general linear mixed 
model with log link and Poisson distribution accounting for clinician clustering to assess 
differences between study arms before and after implementation of the intervention. This 
analysis will be performed in two ways: 1) all providers randomized, and 2) only the initial set of 
providers randomized. We will perform similar statistical analyses for each secondary outcome. 
 
 

5 
 
Applying the RE-AIM implementation framework to our evaluation of this pilot study’s 
findings 
Dimension 
Definition1 
Measurement 
Reach 
The absolute number, proportion and 
representativeness of individuals who are 
willing to participate in a given initiative or 
program, and reasons why they do or do not 
participate.  
Who actually participates or is exposed to the 
initiative? 
Record the # of clinicians 
who choose not to opt-out 
of the trial. 
Monitor the number of 
clinicians in the 
experimental arm who 
acknowledge receipt of 
their feedback e-mail. 
Effectiveness 
The effect of an intervention on important 
individual outcomes, such as clinical events, 
quality of life,  and economic outcomes 
 
See primary and 
secondary outcomes, as 
listed above 
Adoption 
The absolute number, proportion and 
representativeness of settings and agents 
who are willing to start a program and why 
they are willing to start it. 
Where is the program applied and who 
applied it? 
Describe the location and 
complexity of the clinic 
sites. Explain why clinic 
leadership wanted to 
participate in this project.  
Implementation 
Fidelity to the key elements of an evidence-
base practice, including consistency of 
delivery as intended and the time and cost of 
the program. 
 
How consistently was the program delivered? 
How was it adapted to each setting? How 
much did it cost? Why did it achieve the 
observed result? 
Measure fidelity by 
tracking the number of 
audits performed and 
feedback reports 
distributed.  
 
Measure acceptability 
with a post-intervention 
survey/interview of 
frontline prescribers. 
  
Maintenance 
The extent to which a program or policy 
becomes institutionalized or part of the routine 
organizational practices and policies. 
How long are the results of the program or 
policy sustained? 
Assess whether clinic 
leadership wants to 
maintain the dashboard 
and, if so, how it is 
sustained. 
1. All definitions are taken from https://www.re-aim.org