Implementing Prescriber-Pharmacist Collaborative Care for Evidence-based Anticoagulant Use
Phase: NA
Conditions: Pulmonary Embolism, Venous Thromboembolism, Atrial Fibrillation
Sponsor: University of Michigan

Title:  Implementing Prescriber-Pharmacist Collaborative Care 
for Evidence-based Anticoagulant Use 
NCT: NCT05351749
Protocol IRB Approval Date: 8/11/2022

Implementing Prescriber-Pharmacist Collaborative Care 
for Evidence-based Anticoagulant Use 
National Clinical Trial (NCT) Identified Number: NCT05351749 
Principal Investigators: Geoffrey Barnes, MD, MSc & 
Shawna Smith, PhD  
Funded by: Agency for Healthcare Research and Quality 
Version Number: 1.0 DRAFT 
August 2022 
Summary of Changes from Previous Version: 
Affected Section(s) 
Summary of Revisions Made 
Rationale 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
2 
 
Table of Contents 
Statement of Compliance 
5 
1 Protocol Summary 
6 
1.1 Synopsis 
6 
1.2 Schema 
8 
1.3 Schedule of Activities (SoA) 
10 
2 Introduction 
11 
2.1 Background and Study Rationale 
11 
2.2 Risk/Benefit Assessment 
13 
2.2.1 Known Potential Risks 
13 
2.2.2 Known Potential Benefits 
13 
2.2.3 Assessment of Potential Risks and Benefits 
13 
3 Objectives and Endpoints 
14 
4 Study Design 
15 
4.1 Overall Design and Scientific Rationale 
15 
4.2 End of Study Definition 
16 
5 Study Population 
17 
5.1 Inclusion Criteria 
17 
5.2 Exclusion Criteria 
17 
5.3 Screen Failures 
17 
5.4 Strategies for Recruitment and Retention 
17 
6 Study Intervention 
18 
6.1 Study Intervention(s) Administration 
18 
6.1.1 Study Intervention Description 
18 
6.1.2 Administration 
18 
6.2  Acquisition and Accountability 
19 
6.3 Measures to Minimize Bias: Randomization and Blinding 
19 
6.4 Study Intervention Compliance 
19 
7 Study Intervention Discontinuation and Participant Discontinuation/Withdrawal 
20 
7.1 Discontinuation of Study Intervention 
20 
7.2 Participant Discontinuation/Withdrawal from the Study 
20 
7.3 Lost to Follow-Up 
20 
8 Study Assessments and Procedures 
20 
8.1 Efficacy Assessments 
20 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
3 
 
8.2 Implementation outcomes 
21 
8.3 Safety and Other Assessments 
22 
8.4 Adverse Events and Serious Adverse Events 
23 
8.4.1 Definition of Adverse Events (AE) 
23 
8.4.2 Classification of an Adverse Event 
23 
8.4.2.1 Severity of Event 
23 
8.4.2.2 Relationship to Study intervention 
23 
8.4.3 Adverse Event Reporting 
25 
8.4.4 Reporting Events to Participants 
26 
8.5 Unanticipated Problems 
26 
8.5.1 Definition of Unanticipated Problems (UP) 
26 
8.5.2 Unanticipated Problem Reporting 
26 
8.5.3 Reporting Unanticipated Problems to Participants 
27 
9 Statistical Considerations 
27 
9.1 Sample Size Determination 
27 
9.2 Populations for Analyses 
28 
9.3 Statistical Analyses 
28 
9.3.1 General Approach 
28 
9.3.2 Analyses Of Primary Outcomes 
30 
9.3.3 Analyses Of Secondary Outcome 
30 
9.3.4  Analyses Of Implementation 
30 
9.3.5 exploratory analysis 
32 
9.3.6 Safety Analyses 
33 
10 Supporting Documentation and Operational Considerations 
33 
10.1 Regulatory, Ethical, and Study Oversight Considerations 
33 
10.1.1 Informed Consent Process 
33 
10.1.1.1 Consent/assent and Other Informational Documents Provided to participants 33 
10.1.1.2 Consent Procedures and Documentation 
34 
10.1.2 Study Discontinuation and Closure 
34 
10.1.3 Confidentiality and Privacy 
34 
10.1.4 Future Use of Stored Specimens and Data 
35 
10.1.5 Key Roles and Study Governance 
35 
10.1.6 Safety Oversight 
35 
10.1.7 Data Handling and Record Keeping 
36 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
4 
 
10.1.7.1 Data Collection and Management Responsibilities 
36 
10.1.7.2 Study Records Retention 
36 
10.1.8 Protocol Deviations 
37 
10.1.9 Publication and Data Sharing Policy 
37 
10.1.10 Conflict of Interest Policy 
37 
10.2 Additional Considerations 
37 
10.3 Abbreviations 
37 
10.4 Protocol Amendment History 
38 
11 References 
39 
References 
39 
12 Appendices 
40 
 
 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
5 
 
STATEMENT OF COMPLIANCE 
 
The trial will be carried out in accordance with International Council on Harmonisation Good Clinical 
Practice (ICH GCP) and the following: 
• 
United States (US) Code of Federal Regulations (CFR) applicable to clinical studies (45 CFR Part 
46, 21 CFR Part 50, 21 CFR Part 56, 21 CFR Part 312, and/or 21 CFR Part 812). 
Agency for Healthcare Quality and Innovation (AHRQ)-funded investigators and clinical trial site staff 
who are responsible for the conduct, management, or oversight of AHRQ-funded clinical trials have 
completed Human Subjects Protection and ICH GCP Training. 
 
The protocol, informed consent form(s), recruitment materials, and all participant materials will be 
submitted to the IRB for review and approval. Approval of both the protocol and the any relevant 
consent form(s) must be obtained before any participant is consented. Any amendment to the protocol 
will require review and approval by the IRB before the changes are implemented to the study. All 
changes to the consent form(s) will be IRB approved; a determination will be made regarding whether a 
new consent needs to be obtained from participants who provided consent, using a previously approved 
consent form. 
 
 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
6 
 
1 PROTOCOL SUMMARY 
1.1 SYNOPSIS 
Title: 
Implementing Prescriber-Pharmacist Collaborative Care for Evidence-based 
Anticoagulant Use 
Study Description: 
Direct oral anticoagulant medications are commonly used to treat or prevent 
thrombotic conditions, such as pulmonary embolism, deep vein thrombosis, 
and atrial fibrillation. However, up to 10-15% of patients receiving these 
medications get unsafe doses based on a patient’s kidney or liver function, 
potential interactions with other medications, and indication for taking the 
medication. Medication alerts and notifications directed to prescribers may 
be beneficial for improving evidence-based prescribing, but can be 
burdensome. This study will improve upon the existing systems by testing 
augmentations that encourage collaboration between prescribing health care 
providers (e.g., physicians, nurse practitioners) and expert pharmacists 
working in anticoagulation clinics and incorporating dynamic long-term 
monitoring of patient needs, as well as updates to evidence-based guidelines. 
The target participant for this trial is the DOAC prescriber (physician, 
physician assistant, nurse practitioners). We will specifically test two different 
clinical situations: (1) a medication alert at the time an inappropriate new 
direct oral anticoagulation (DOAC) prescription is written, and (2) a 
medication notification that is sent anytime a new prescribing error develops 
after the initial DOAC prescription is written. This project will  implement and 
evaluate an Electronic Health Record (EHR) medication alert and notification 
system to improve safe DOAC prescribing through prescriber-pharmacist 
collaboration. Findings from this project will establish a framework for 
implementing prescriber-pharmacist collaboration for high risk medications, 
including anticoagulants. 
 
Objectives: 
Primary Objective: 
Implement and evaluate EHR medication alert and 
notification systems to improve safe DOAC prescribing 
through prescriber-pharmacist collaboration.  
 
 
Endpoints: 
 
Primary Endpoint: 
The number (proportion) of notifications (in the 
existing-prescription notification conditions) that are 
addressed within 7 days. 
 
Secondary & 
Exploratory 
Endpoints: 
The number (proportion) of alerts (in the newly 
prescribed DOAC alert conditions) that are addressed 
within 7 days. 
 
Change in magnitude of improvement for the existing-
prescription notification over time, reported on at the 
institution level (not individual level). 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
7 
 
 
Change in magnitude of improvement for the new-
prescription alert over time, reported on at the 
institution level (not individual level). 
 
Study Population: 
300 (minimum) prescribers of DOACs 
Phase: 
NA 
 
Description of Sites 
Enrolling 
Participants: 
 
This single center clinical trial will be conducted at the University of Michigan. 
Participants (DOAC prescribers) will be recruited from Michigan Medicine 
prescribers.  
Description of Study 
Intervention: 
The study examines the effects of medication alerts and notifications in the 
EHR regarding problems with the DOAC medications Apixaban and 
Rivaroxaban. Alerts and notifications vary only in the degree to which 
pharmacists are involved. For alerts (for new prescriptions), prescribers will 
be randomly assigned to receive either an alert at the time of prescribing that 
includes an explicit option to consult with a pharmacist or one that does not 
include an explicit option to consult with a pharmacist. (Pharmacy consults 
are always available.) For notifications (related to existing prescriptions), a 
notification will be initially routed to either a pharmacist (who can consult the 
prescriber) or to the prescriber (who can consult a pharmacisit).  
 
Study Duration: 
18 months 
Participant 
Duration: 
Up to 18 months 
  
Source of funding: 
AHRQ grant: R18HS028562  
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
8 
 
1.2 SCHEMA 
Schema for enrollment and participation 
 
 
Note that prescribers can be enrolled in just one or both the alert and notifications arms of the 
randomized controlled trial if they are eligible. 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
9 
 
Representation of alerts and notifications in protocol 
 
Note: Reference to the “dashboard” relate to a current tracking system used by anticoagulation clinic 
pharmacists to identify when potentially inappropriate DOAC prescriptions exist. These will be 
embargoed for 15 days to measure the effect of an alert and/or notification on prescriber behavior 
before reverting to the usual care pathway (display alert on dashboard). 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
10 
 
1.3 SCHEDULE OF ACTIVITIES (SOA) 
Procedure   
Screening and 
Enrollment  
 
At time of first 
triggered alert 
or notification 
 
At time of 
subsequent 
alerts or 
notifications 
6 & 12 
months of 
trial 
18 months 
of trial 
Informed Consent (waived 
documentation per IRB) 
X 
 
  
  
 
Inclusion/exclusion 
 
X 
  
  
 
Randomization 
 
X 
  
  
 
Collection of EHR alert data 
 
X 
X  
  
 
Conclusion of data collection 
 
 
 
 
X 
Event review and DSMB 
● Adverse events (See 
appendix A) 
● Appropriateness of Rx 
 
 
 
X 
X 
Final Status 
 
 
  
 
X 
 Key: EHR = electronic health record 
 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
11 
 
2 INTRODUCTION 
2.1 BACKGROUND AND STUDY RATIONALE 
Direct Oral Anticoagulant (DOAC) use is rapidly expanding for six million Americans with common 
thrombotic conditions. Atrial fibrillation (AF) and venous thromboembolism (VTE) are the leading 
indications for chronic anticoagulation therapy. Since 2010, four DOAC medications (apixaban, 
dabigatran, edoxaban, and rivaroxaban) have rapidly overtaken warfarin as first-line therapy for AF and 
VTE.1–3 DOAC therapy is critical to prevent life-threatening stroke and thromboembolic complications. 
 
Unsafe DOAC prescribing is 
common and dangerous. While 
initially heralded as easier to 
prescribe than warfarin, in real-
world settings 10-20% of DOAC 
prescriptions do not follow the 
Federal Drug Administration 
(FDA) evidence-based package 
label instructions.4–6 The most 
common reasons for unsafe 
DOAC use include inappropriate 
dosing based on renal or liver function, overlooked drug-drug interactions, and dosing based on the 
wrong indication (e.g., using AF dosing for VTE). These prescribing errors occur both at the time of initial 
prescription and in the months to years of follow up (Figure 1). Data from both large national 
registries4,5 and our local studies7 confirm a higher rate of bleeding, hospitalization, and death for 
patients with unsafe DOAC use as compared to evidence-based prescribing. Despite these common 
prescribing errors, most health systems only provide alerts for potential drug-drug interactions, not for 
other (more common) causes of inappropriate DOAC prescribing. 
 
Prescriber-pharmacist collaboration is an underutilized resource. Prescribers have long collaborated 
with more than 3000 anticoagulation clinics nation-wide to manage warfarin. The majority of these 
clinics rely on pharmacists with anticoagulation expertise to manage complex patients and their high-
risk medications safely. Our 2017 survey of these clinics found that most anticoagulation clinics offer 
collaborative DOAC care.15 In this collaborative mode, the pharmacists review prescriptions for 
appropriateness and for potential drug-drug interactions and recommend appropriate drug/dose 
selection. However, despite their high volume, DOAC-treated patients accounted for a small fraction 
(~10%) of the overall anticoagulation clinic volume, indicating prescriber underutilization. One exception 
is the Veterans Health Affairs system, where pharmacist collaboration is common for DOAC prescribing 
and rates of unsafe DOAC prescribing are significantly lower than outside the Veterans Health Affairs 
system.16  
 
Most medication alerts do not follow key design 
principles and fail to improve patient care. Studies 
have shown that well-designed medication alerts in 
the EHR can reduce adverse drug events by up to 
50%.17 However, these promising results are too 
often not realized. In fact, poorly implemented 
Table 1. Medication Alert Design Principles18,19 
• Improve signal-to-noise ratio by incorporating 
clinical context into alert logic 
• Support collaborative work, including pharmacists 
• Fit within prescriber workflow and mental model, 
which includes non-interruptive alerts 
• Display relevant data on why alert occurred 
• Ensure system transparency 
• Include actionable tools within the alert 
Figure 1 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
12 
 
medication alerts frequently lead to clinician dissatisfaction from alert fatigue and habitual override.17,18 
Currently available DOAC medication alerts suffer from three fundamental design flaws, which together 
lead to incorrect DOAC dosing both at the time of the initial prescription and at subsequent moments in 
time where clinical changes should lead to modification of the DOAC medication or dose: (1) alerts 
intrude or interrupt prescriber work flow with low-yield information without actionable tools, often 
leading to alert dismissal without action; (2) alerts occur only at the time of prescribing, ignoring 
changes to the clinical scenario that may occur after the initial prescription is written; and (3) alerts do 
not promote collaboration between prescribers and pharmacists. As a result, many patients receive 
unsafe prescriptions that can cause significant harm. Medical informatics experts have proposed key 
design principles to address these issues (Table 1).18,19 Key among these is supporting collaborative work 
between prescribers and pharmacists. 
Many prescribers have multiple patients with unsafe DOAC prescriptions. DOAC prescribing has rapidly 
accelerated between 2012 and 2020, to more than 19,000 unique patients at Michigan Medicine alone 
with active DOAC use. A single-day (Dec 21, 2020) cross-sectional analysis at Michigan Medicine found 
9,325 patients had DOAC use documented by 1,002 primary care, cardiology, hematology, or surgery 
prescribers (median 23 patients/prescriber, interquartile range 7-47). Of these, 670 (7.2%) patients 
(among 250 unique prescribers) clearly did not follow evidence-based guidelines, a median of 2 (IQR 1-
5) unsafe DOAC prescriptions per prescriber observed on that single day snapshot.  
 
Prescribers change when alerted. During a recent pilot study that we conducted, when prescribers were 
contacted about unsafe DOAC use they made meaningful changes in 51/81 (63%) cases.20 Actions taken 
included changing the medication, changing the dose, or discontinuing an interacting medication. When 
changes were not made, a thoughtful discussion usually occurred between the prescriber and an 
anticoagulation expert focusing on unique patient factors. 
 
Figure 2. Sample Medication Alert with Pharmacist Referral Option and notification (right), and 
notification (left) 
     
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
13 
 
Proposed Intervention. We propose two different interventions to improve DOAC prescribing.  The first 
intervention is an automated EHR alert that occurs at the time a DOAC medication is prescribed but 
some potential error exists (e.g., drug-drug interaction, wrong dose for given renal function). Prescribers 
will be randomized to receive a detailed alert, but half will include a referral link for optional pharmacist 
review and the other half will receive an alert that does not include the pharmacist review referral link.  
 
The second intervention is an EHR notification that is sent to the inbox of either the DOAC prescriber or 
an anticoagulation pharmacist anytime after a DOAC medication is prescribed when a new potential 
issue develops (e.g., worsened renal function that impacts dosing, new drug-drug interactions). 
 
2.2 RISK/BENEFIT ASSESSMENT 
2.2.1 KNOWN POTENTIAL RISKS 
There are minimal risks to the subjects (prescribers). 
• 
Confidentiality  
• 
Risk to reputation if “errors” are revealed to Michigan Medicine administrators 
• 
Alert fatigue 
• 
The alert and notifications may recommend a medication change that is harmful for the patient 
2.2.2 KNOWN POTENTIAL BENEFITS 
Participation in this study will help prescribers of anticoagulants perform their job better by providing 
them with clinical decision support that they do not currently have. There are no current DOAC alerts or 
notifications in the EHR relative to prescribing other than for drug-drug interactions. 
The current EHR drug-drug alerts will be improved to provide better rationale for why the drug-drug 
interactions exists and recommendations on how to address the problem (e.g., dose change, alterantive 
medication) 
Patients (who are not the primary participants in the protocol) may benefit from reduced probability of 
having a prescription error related to DOAC medications.  
 
2.2.3 ASSESSMENT OF POTENTIAL RISKS AND BENEFITS 
We assess the potential risks to be minimal. Our preliminary data shows that an overwhelming majority 
of prescribers are happy to receive feedback and suggestions on how to correct inappropriate DOAC 
prescriptions. 
We assess the potential benefits to be moderate-large. Prescribers are likely to improve the evidence-
based use of DOAC medications and gain knowledge about how to safely prescribe DOACs. Patients are 
likely to experience the benefit of evidence-based DOAC use while minimizing the risk of harm (e.g., 
bleeding from excessive anticoagulant effect). 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
14 
 
2.3.4. Risk Mitigation Strategies 
The likelihood of breach of confidentiality is rare. Every possible effort will be made to keep the research 
information in the strictest confidence. We will de-itendify all records as soon as possible. This includes 
the use of a dummy “ID” variable in the primary data set that requires a separate linking database to 
gather medical record numbers. All data will be stored on secured servers at the University of Michigan 
and will be password protected.  Any paper records will be destroyed using University of Michigan 
secure processes. 
Materials informing prescribers about the study will include reassurances that data will be de-identified 
prior to analyses, and any results shared through publications, presentations, or with administrators will 
only use aggregated information. They will also be reminded that existing systems used by Michigan 
Medicine already monitor inappropriate prescriptions for medications such as DOACs, thus the only new 
information being generated relates to their receipt and engagement with the alerts.  
All alerts and notifications have undergone a rigorous, multi-stage user-centered design process to 
improve readability, actionability, and acceptability by the end users. This should minimize any alert 
fatigue issues. 
All alert and notifications are based on FDA-approved dosing recommendations and nationally trusted 
sources (Michigan Anticoagulation Quality Improvement Initiative and Anticoagulation Forum). All logic 
has been reviewed by at least two clinical experts (anticoagulation physician and pharmacist) to ensure 
accuracy. 
3 OBJECTIVES AND ENDPOINTS 
 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATIONS 
Primary 
 
Determine the effect of 
notifications regarding existing 
DOAC prescriptions on 
prescriber behavior 
The number (proportion) of 
notifications (in the existing-
prescription notification conditions) 
that result in any prescription change 
within 7 days. 
The goal of alerts and notifications is to 
bring about corrections in prescription 
errors. Given that “correct” is not 
always an absolute assessment, our 
primary outcome measure will capture 
any change to the DOAC prescription 
that results from the notification. 
 Secondary 
 
Determine the effect of alerts 
regarding newly prescribed 
DOACs on prescriber behavior 
The number (proportion) of alerts (in 
the newly prescribed DOAC alert 
conditions) that result in any 
prescription change within 7 days. 
 
The goal of alerts and notifications is to 
bring about corrections in prescription 
errors. Given that “correct” is not 
always an absolute assessment, our 
primary outcome measure will capture 
any change to the DOAC prescription 
that results from the notification. 
 
Change in magnitude of effect for the 
existing-prescription notification over 
time. This is reported on at the 
institution level (not individual level). 
The effectiveness of notifications should 
be sustained over time. This measure 
examines longitudinal trends in 
effectiveness.  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
15 
 
(This outcome measure analysis is 
based on the primary outcome) 
 
Change in magnitude of effect for the 
new-prescription alert over time, 
reported on at the institution level (not 
individual level).   
The effectiveness of alerts should be 
sustained over time. This measure 
examines longitudinal trends in 
effectiveness.  
Tertiary/Exploratory 
 
Determine the effectiveness of 
promoting pharmacist-
prescriber collaboration in new 
DOAC prescriptions through 
the design of CDS alerts 
The difference in outcome measures 
(primary and secondary outcomes) 
between Notification A and B 
conditions 
While alerts are expected to reduce 
prescription errors, variations in alert 
and design to encourage pharmacy 
collaboration may provide added 
benefit 
Determine the effectiveness of 
promoting pharmacist-
prescriber collaboration in 
existing prescriptions through 
routing of DOAC prescription 
error notification 
The difference in outcome measures 
(primary and secondary outcomes) 
between Alert 1 and 2 conditions 
While notifications are expected to 
reduce prescription errors, 
automatically routing notifications to 
pharmacy may provide added benefit 
 
4 STUDY DESIGN 
4.1 OVERALL DESIGN AND SCIENTIFIC RATIONALE 
This prospective randomized clinical trial will determine the effectiveness of alerts (for newly prescribed 
DOAC) and notifications (for existing prescriptions) in reducing prescription errors. We have two central 
hypotenses: (1) alerts and notifications will reduce prescription errors; and (2) involving pharmacists in 
the prescribing process through alerts and notifications will increase the efficacy of the alerts and 
notifications. These two hypotheses will be evaluated by the proportion of prescriptions that are 
changed following an alert (for a new prescription) or notification (for an existing prescription). 
 
This study is 18-month prospective, randomized, controlled clinical trial of medication alerts and 
notifications in the ambulatory settings.  See section 1.2 for an overview of the schema. Prescribers will 
be randomized into one of four conditions upon triggering a DOAC alert or notification (see figure 4.1-1, 
below).  
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
16 
 
 
Figure 4.1-1: Timeline of trial protocol. 
 
Alert styles 1 and 2 (for newly prescribed DOACs) will differ only in the inclusion or exclusion of a 
pharmacy consult (i.e. there will or will not be an option to click to automatically receive a consult). 
Notifications (for existing DOAC prescriptions) will differ in that the notification will either be routed 
directly to the pharmacist, who will then consult the prescriber if need be, or be routed to the prescriber 
directly. 
 
 
4.2 END OF STUDY DEFINITION 
The study will end after 18 months of the alerts and notifications being active in the EHR. The duration 
of any individual prescriber will vary, as an individual’s participation is dependent on triggering a DOAC 
medication alert or notification. 
  
We will perform a one-time interim analysis after 12 months. This will asses only for safety outcomes. 
We will not assess for effectiveness outcomes to avoid overestimation of the effect size for 
effectiveness.  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
17 
 
5 STUDY POPULATION 
5.1 INCLUSION CRITERIA 
 
The overarching study will include all ambulatory DOAC prescriptions by a primary care clinician, 
cardiologist, hematology, or vascular surgeron at the University of Michigan.  Within that overarching 
study, a randomized trial will be conducted to explore the impact of medication alerts and notifications 
on prescriber behavior. This observational dataset will be used for secondary analyses only to better 
describe the context of DOAC use within the institution. 
 
For the randomized trial, we will enroll prescribers who meet all the following criteria 
1. Michigan Medicine providers with prescribing privileges (including attending physicians, house 
officers, nurse practitioners, and physician assistants)  
2. Prescribing DOAC to a patient in an ambulatory setting 
3. Trigger a DOAC alert or notification for an ambulatory patient  
5.2 EXCLUSION CRITERIA  
1. Member of the study team 
2. Prescriptions written in Emergency Department or hospital setting (including upon discharge) 
3. Prescriptions written in a skilled nursing facility or other institutionalized setting.  
The reason for the exclusions for #2 and #3 is because these prescribers typically do not follow 
patients longitudinally and therefore are not eligible to receive notifications when new medication 
prescribing issues develop after the initial prescription is written.  
5.3 SCREEN FAILURES 
N/A  
5.4 STRATEGIES FOR RECRUITMENT AND RETENTION 
All prescribers using MiChart will be included in the observational data portion of the study.  For the 
randomized trial, any prescriber of a DOAC medication will be considered for recruitment. Prescribers, 
not patients, are the primary subjects of this study. 
Prescribers will be recruited through a message sent to MiChart users (prescribers) at least 1 week 
before being enrolled, as part of the regularly occurring messages to MiChart users regarding important 
changes and updates to MiCHart.  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
18 
 
The message will describe the protocol, and include a link to the IRB-approved consent form. It will also 
inform prescribers about how to "opt-out" of the protocol by replying to the email. Any prescriber who 
does not “opt-out” will be included in the study through a waiver of documented informed consent as 
approved by the IRB. All potential participants will be contacted. Only those who prescribe an 
anticoagulant in a manner that triggers a medication alert will be "enrolled." 
Prescribers’ participation will consist of receiving medication error alerts through MiChart as 
appropriate. The specific alert they receive will be randomly assigned. Note: all alerts will contain the 
same valid and clinically relevant warning about a potential prescribing error, and the alert will vary only 
as to whether or not there is a link/button to automatically consult a pharmacist. Similarly, notifications 
of an existing prescription (longitudinal) that triggers a medication alert will be routed initially to a 
pharmacist or initially to the prescriber's MiChart inbox.. 
6 STUDY INTERVENTION 
6.1 STUDY INTERVENTION(S) ADMINISTRATION 
6.1.1 STUDY INTERVENTION DESCRIPTION 
The intervention consists of an EHR alert that is presented to prescribers at the time that they are 
prescribing a DOAC, OR a notification in the EHR that indicates that an existing prescription for a DOAC 
does not meet the prescribing guidelines (a potential medication error). Existing prescription 
notifications are triggered once daily through an EHR registry that is updated continuously. 
The alerts were designed through a user-centered design process to ensure that they are clear and 
usable. There will be two versions of the alerts that are identical other than except for one aspect: One 
alert will  include a ‘button’ to click to automatically get a pharmacy consult; one will not include such a 
button. 
The notifications were designed through a user-centered design process to ensure that they are clear 
and usable. Alerts will be identical, but will be routed one of two ways:  
1. They will be routed initially to a pharmacist, who may then contact the prescriber to 
request/recommend a prescription change. 
2. They will be routed directly to the prescriber, who may change the prescription, contact a 
pharmacist, or opt to keep the prescription unchanged. 
6.1.2 ADMINISTRATION 
 
Alerts and notificaitons will fire through automated mechanisms using best practice advisory logic and 
registries in the EHR, and therefore will not require intervention from the research coordinator to 
function properly.  
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
19 
 
The PIs will monitor announcements from the FDA regarding the drugs apixaban and rivaroxaban for 
announcements and label changes to ensure that alerts and notifications adhere to up-to-date guidance. 
Members of the study team and steering committee also help to lead the Michigan Medicine 
Anticoagulation Pharmacy and Therapeutics subcommittee on anticoagulation and will ensure 
harmonization between institutional prescribing protocols and those used in this study. 
 
Opt-out requests will be processed by the research coordinator and relevant study team members 
within 7 days of the request.  Upon receipt, the requesting prescriber will be entered into the EHR 
database such that all study-related alerts and notifications will be suppressed. Usual care activities will 
not be impacted if a prescriber opts out of the study alerts and notifications. 
 
Other than changes in participation (opt out) and changes in FDA guidance regarding the relevant 
medication, administration of this research will be conducted through the EHR (MiChart).  No direct 
contact between the study team and the participants is anticipated.  
 
 
 
6.2  ACQUISITION AND ACCOUNTABILITY 
N/A – No devices will be acquired. 
6.3 MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING 
Participants will be randomized by computer based solely on the sampling algorithm, so unintended 
selection bias by the study team will not be a factor. The randomization algorithm will stratify by: 
1. Trainee vs non-trainee specialist vs non-trainee pcp  
2. Alert vs notification  
  
This clinical trial is not fully blinded. Participants randomized to the different alert and notification 
conditions and will be blinded to the specifics of their assignment. However, they may be able to discern 
which condition they have been assigned to owing to the specifics of the alerts and notifications they 
receive. During the data analysis, study team members will be blinded to the alert and notification 
condition of the participants.  
6.4 STUDY INTERVENTION COMPLIANCE 
There are no discrete participant-initiated activities associated with participation in this protocol, so 
compliance is not an issue. Participants will receive alerts or notifications generated by the EHR, based 
on their routine clinical prescribing activities. Receipt of the notification constitutes participation and 
compliance.  Prescribers’ responses to the alerts and notifications are measured as an outcome, not as 
compliance. 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
20 
 
7 STUDY INTERVENTION DISCONTINUATION AND PARTICIPANT DISCONTINUATION/WITHDRAWAL 
7.1 DISCONTINUATION OF STUDY INTERVENTION 
Discontinuation of a particular alerts or notification does not mean discontinuation from the study, and 
remaining study procedures (e.g., adverse event review) should be completed as indicated by the study 
protocol.  If a clinically significant finding is identified after enrollment, the investigator or qualified 
designee will determine if any change in participant management is needed.   
7.2 PARTICIPANT DISCONTINUATION/WITHDRAWAL FROM THE STUDY 
Participants are free to withdraw from the study at any time. Data collected prior to the withdrawal may 
be used for analysis and publications. The study coordinator will ensure that the participant is removed 
from the EHR’s categorization to receive alerts and notification (i.e. will no longer receive alerts and 
notifications) within 7 business days. 
 
An investigator may discontinue or withdraw a participant from the study at his/her discretion.  The 
reason for participant discontinuation or withdrawal from the study will be recorded on the study 
team’s logs. Adverse events of special interest will be collected from the time of discharge through study 
completion or study withdrawal/discontinuation.  
 
Participants who are randomized and subsequently withdraw, or are withdrawn or discontinued from 
the study, will not be replaced. 
7.3 LOST TO FOLLOW-UP 
● 
Participants who leave the institution or are otherwise no longer eligible to prescribe DOAC 
medications will be considered “lost to follow-up.” 
● 
No actions will be taken by the study team to prevent their loss to the study. 
● 
At the end of the trial period, the study team will compare the list of enrolled prescribers against 
a list of potentially eligible prescribers to determine if any enrolled prescribers have left the 
institution.  These providers status will be noted in the data. Analysis of their prescribing 
patterns will be included in analysis as appropriate. 
8 STUDY ASSESSMENTS AND PROCEDURES 
8.1 EFFICACY ASSESSMENTS 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
21 
 
Outcome Measurements: 
Our primary outcome measure (adoption) is the proportion of DOAC notifications that lead to a change 
in the prescription within seven days. The denominator is the total number of DOAC prescription 
notifications that are sent to either the prescriber or pharmacist (per randomization). The numerator is 
the number of DOAC notifications that result in any change to the DOAC prescription within seven days 
after the notification has been sent.  
 
Secondary  
Our secondary outcome measure (adoption) is the proportion of DOAC alerts that lead to a change in 
the prescription within seven days. The denominator is the total number of DOAC prescription alerts 
with or without a visible option for pharmacist sonsultation (per randomization). The numerator is the 
number of DOAC alerts that result in any change to the DOAC prescription within seven days after the 
alert has occurred. 
 
Effectiveness: We will evaluate the effectiveness of our DOAC alert system components by examining 
patient level 30-day clinical adverse event rates (bleeding and thrombotic). This time period was 
selected to minimize potential contamination from the safety review mechanism described above, and 
we recognize that rates for the adverse events will be low. Supplementary analyses will also examine the 
effect of the alert system overall on adverse events.  
 
Clinical adverse events assessed will include major21 and clinically-relevant non-major bleeding 
(CRNMB)22 events, as defined by the International Society on Thrombosis and Haemostasis (ISTH), new 
or recurrent VTE events, and stroke or systemic arterial embolic events. Each of these events will be 
captured using health informatics tools (described below) and independently adjudicated by two expert 
clinicians (one pharmacist, one prescriber) who meet once in Y1 and twice in Y2 to adjudicate any 
potential adverse events. We will use two Michigan Medicine-developed health informatics tools, 
DataDirect and EMERSE,23 to identify adverse events and capture clinical data (e.g., notes, labs, imaging, 
procedure reports) for adjudication. These tools allow for rapid identification of populations based on 
granular clinical details (e.g., demographics, diagnosis, medication use) and for textbased searches of 
the medical record for key terms (e.g., “bleeding”, “stroke”) across pre-defined patient populations. The 
adjudicators will be blinded as to which alert the patient’s prescribing clinician was randomized. Using 
these EHR search tools, we can also capture all ICD10 diagnosis and procedure codes for longitudinal 
system-level analyses.  [See Appendix A for current list of ICD10 codes used.] 
 
 
8.2 IMPLEMENTATION OUTCOMES 
Our exploratory analysis of outcomes will follow the RE-AIM implementation evaluation framework. 
 
Reach: All alerts for unsafe DOAC use will be categorized based on the prescriber specialty and patient 
characteristics (e.g., demographics, comorbidities). These will be described both independently for the 
both the alerts and the notifications separately, as well as in aggregate. We will also examine the 
proportion of patients on DOACs that had alerts or notifications fire, as well as the propotion of 
prescribers that received alerts and/or notifications. Finally, we will examine the frequency of unsafe 
DOAC prescribing for prescribers who opt out of the alert system.  
 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
22 
 
Adoption: In addition to our primary outcome measure of prescriber-level adoption of evidence-based 
DOAC prescription, secondary adoption measures will examine the frequency and clinical predictors of 
prescribers who dismiss the new-prescription alert or ignore the existing-prescription notification 
(defined as no action within seven calendar days). We will also identify the frequency and predictors of 
prescribers who use anticoagulation clinic pharmacist referrals in new-prescription alerts.  
 
Implementation (including fidelity): To address how various EHR alerts impact implementation of 
evidence-based DOAC prescribing, we examine process measures related to prescriber and pharmacist 
alert and notification receipt.  
 
For prescribers, we will examine how often they order (or do not order) the recommended medication 
(and whether they provide a reason when they do not order) for both new-prescription alerts and 
existing prescription notifications. For pharmacists, we will measure how often they respond to referrals 
at the time of initial prescribing or to existing-prescription notifications, as well as the time from 
referral/alert until the pharmacist documents a recommendation in the EHR. For existing-prescription 
notifications, we also will measure the time from notification until the notification is read as well as time 
until any DOAC prescription change or related EHR documentation occurs (automated chart abstraction 
using EMERSE, validated by random manual chart review) as well as the number of times a existing-
prescription notification is read before being acted on. 
 
Maintenance: To measure maintenance and sustainment, we will assess all “reach” and “adoption” 
outcomes after 6 and 12 months for all patients, and after 18 months for the subset of prescribers 
enrolled prior to the second quarter of Year 2. This will also allow us to investigate whether treatment 
effects for prescribers are maintained after the initial adoption. 
8.3 SAFETY AND OTHER ASSESSMENTS 
Participants (prescribers) will not undergo any procedures. Prescribers’ patients will be monitored for 
adverse events  through the DSMB review. 
DSMB will monitor for adverse events every 6 months. Adverse events will be measured up to 30 days 
after the alert or notification occurred. The safety outcomes being monitored by the DSMB include 
bleeding and thromboembolic events (including stroke and venous thromboembolism). These will be 
identified through diagnostic code searches within the EHR (see appendix A) and will undergo 
adjudication as described below. 
Investigators will report serious adverse events in accordance with IRBMED requirements. The 
investigator will follow usual clinical practices at UM for reporting to regulatory authorities serious, 
unexpected events related to standard of care medications and devices.  
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
23 
 
8.4 ADVERSE EVENTS AND SERIOUS ADVERSE EVENTS 
8.4.1 DEFINITION OF ADVERSE EVENTS (AE) 
An adverse event (AE) is any untoward or unfavorable medical occurrence associated with the subject’s 
participation in the research, whether or not considered related to the subject’s participation in the 
research (modified from the definition of adverse events in the 1996 International Conference on 
Harmonization E-6 Guidelines for Good Clinical Practice) (from OHRP "Guidance on Reviewing and 
Reporting Unanticipated Problems Involving Risks to Subjects or Others and Adverse Events 
(1/15/07)"http://www.hhs.gov/ohrp/policy/advevntguid.html#Q2). 
8.4.2 CLASSIFICATION OF AN ADVERSE EVENT 
AEs will be graded for severity and relationship to the intervention by the investigator according to the 
guidelines to be specified below. 
 
8.4.2.1 SEVERITY OF EVENT 
The following guidelines will be used to grade severity of AEs: 
Bleeding related events, as defined by the International Society on Thrombosis and Haemostasis 
(ISTH): 
● 
Major,  
● 
clinically relevant, non major 
● 
minor 
Thrombotic (stroke and venous thromboembolism): 
● 
There will be no severity grading of thrombotic events. 
Any bleeding or new thrombotic event that occurs within 30 days of a study alert or notification will be 
included as an adverse event. 
Serious adverse events include any death or life threatening outcome. This includes ISTH major bleeding 
event and any new thrombotic event that occurs within 30 days of a study alert or notification.  
8.4.2.2 RELATIONSHIP TO STUDY INTERVENTION 
Our protocol is intended to increase adherence to evidence based prescribing. Adverse outcomes will be 
assessed in relation to the degree to which prescribers adhered to evidence-based prescribing.  
 
Use of evidence based prescribing (Appendix B) will be evaluated by blinded adjudicators (one 
pharmacist, one prescribing clinician) at the time of any adverse event as well as randomly selected 
cases. The following categories will be used: 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
24 
 
Initially, each adverse event will be examined with regard to medication management. Cases 
will be classified into one of the following 5 categories:   
1. Definitely appropriate medication management – There is clear evidence to suggest that 
prescribers considered the appropriate clinical factors and prescribed medication in accord with 
evidence based prescribing. (Example: Rivaroxaban dose changed from 15->20mg daily in a 
patient with atrial fibrillation, creatinine clearance 75 ml/min, and no drug drug interactions) 
2. Probably appropriate medication management – There is some evidence to suggest that 
prescribers considered the appropriate clinical factors and prescribed medication in accord to 
evidence based prescribing. (Example: Rivaroxaban dose changed from 15->20mg daily in a 
patient with atrial fibrillation, creatinine clearance of 51ml/min [previously 48 ml/min], and no 
drug-drug interactions) 
3. Probably inappropriate medication management – There is some evidence to suggest that the 
prescriber may have deviated from evidence based prescribing.  (Example: Rivaroxaban dose 
changed from 15->20mg daily in a patient with atrial fibrillation, creatinine clearance of 48 
ml/min [previously 55 ml/min], and no drug-drug interactions) 
4. Definitely inappropriate medication management – There is clear evidence to suggest that the 
prescriber deviated from evidence based prescribing. (Example: Rivaroxaban dose changed from 
15->20mg daily in a patient with atrial fibrillation, creatinine clearance of 35 ml/min, and no 
drug-drug interactions) 
5. Unable to assess appropriateness of medication management: Due to lack of information in 
the medical record, it is not possible to assess the appropriateness of the medication 
management.  
 
Adverse events that are classified in the Medication Management categories 3, 4 or 5 above will then be 
reviewed with regard to whether the adverse event is related to the prescription. The following 5 
categories will be used:  
A. Definitely not related–The clinical event, despite occurring in a plausible time relationship to 
study intervention administration, can be explained by concurrent disease or mechanism, or 
other other drugs or chemicals, or is a non-preventable medication adverse event.   
B. Probably not related –The clinical event, despite occurring in a plausible time relationship to 
study intervention administration, is more likely than not explained by concurrent disease or 
mechanism, or other other drugs or chemicals, or is a non-preventable medication adverse 
event.  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
25 
 
C. Probably related –  It is more likely than not that a causal relationship exists between the 
adverse event and the prescription. However, other factors may have contributed to the event 
(e.g., the participant's clinical condition, other concomitant events).  
D. Definitely related –There is no significant evidence that suggests other drugs or chemicals or 
underlying disease provides plausible explanations (e.g., the participant's clinical condition, 
other concomitant treatments). 
E. Unable to assess relatedness of medication management: Due to lack of information in the 
medical record, it is not possible to assess the relationship between the adverse event and the 
prescription.  
Adverse events that are classified into categories of C, D, or E (as well as categories 3, 4, and 5) will be 
reviewed by the DSMB. This includes all cases in which the medication management was more likely 
than not inappropriate medication management, and adverse events which are more likely than not 
related to the medication. It also includes all cases where the medication management and the 
relationship of the medication to the adverse event is unknown.  
All adverse events (serious and non-serious) will further be assessted based on the time from 
medication error until the adverse event occurred. This will be done to determine if the 15 day embargo 
on the existing dashboard may have contributed to patient harm. Specifically, we will look to see if 
medication alerts or notifications were not addressed through the study interventions that may have 
been addressed via the dashboard if no embargo were in place. 
 
8.4.3 ADVERSE EVENT REPORTING 
All reporting comply with Michigan Medicine IRB guidelines 
(https://az.research.umich.edu/medschool/guidance/adverse-event-reporting).  
 
Any serious adverse event that are adjudicated as being definitely or possibly related to the study 
intervention will be reported to the IRB within 7 calendar days of the study team becoming aware of the 
event. 
 
All non-serious adverse events that are adjudicated as being definitely or possibly related to the study 
intervention will be reported to the IRB in aggregate at least annually. 
 
All serious adverse events that are adjudicated as being definitely or possibly unrelated to the study 
intervention will be reported to the IRB in aggregate at least annually. 
 
All non-serious adverse events that are adjudicated as being definitely or possibly unrelated to the study 
intervention will be collected for analysis but not reported to the IRB. 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
26 
 
All adverse events in which the adjudication team is unable to assess the relatedness to the study 
intervention will be reported to the IRB in aggregate at least annually. 
 
8.4.4 REPORTING EVENTS TO PARTICIPANTS 
Not applicable 
8.5 UNANTICIPATED PROBLEMS 
8.5.1 DEFINITION OF UNANTICIPATED PROBLEMS (UP) 
The Office for Human Research Protections (OHRP) considers unanticipated problems involving risks to 
participants or others to include, in general, any incident, experience, or outcome that meets all of the 
following criteria: 
● 
Unexpected in terms of nature, severity, or frequency given (a) the research procedures that are 
described in the protocol-related documents, such as the Institutional Review Board (IRB)-
approved research protocol and informed consent document; and (b) the characteristics of the 
participant population being studied; 
● 
Related or possibly related to participation in the research (“possibly related” means there is a 
reasonable possibility that the incident, experience, or outcome may have been caused by the 
procedures involved in the research); and 
● 
Suggests that the research places participants or others at a greater risk of harm (including 
physical, psychological, economic, or social harm) than was previously known or recognized. 
8.5.2 UNANTICIPATED PROBLEM REPORTING 
The investigator will report unanticipated problems (UPs) to the reviewing institutional review board 
(IRB) and to the DSMB. The UP report will include the following information: 
● 
Protocol identifying information: protocol title and number, PI's name, and the IRB project 
number; 
● 
A detailed description of the event, incident, experience, or outcome; 
● 
An explanation of the basis for determining that the event, incident, experience, or outcome 
represents an UP; 
● 
A description of any changes to the protocol or other corrective actions that have been taken or 
are proposed in response to the UP. 
To satisfy the requirement for prompt reporting, UPs will be reported using the following timeline:  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
27 
 
● 
UPs that are adverse events will be reported to the IRB and to the study sponsor within 24 hours 
of the investigator becoming aware of the event. 
● 
Any other UP (e.g., study protocol deviations) will be reported to the IRB and to the study 
sponsor within 2 weeks or 10 business days of the investigator becoming aware of the problem. 
8.5.3 REPORTING UNANTICIPATED PROBLEMS TO PARTICIPANTS 
Not applicable.   
9 STATISTICAL CONSIDERATIONS 
9.1 SAMPLE SIZE DETERMINATION 
This study is powered for our primary comparison, which is the comparison of our two different 
notitifications. We will use mixed-effects binary logistic regression to model the proportion of 
inappropriate DOACs that are changed within 7 days following an alert.  
 
We anticipate randomizing 300 prescribers to different notifications over the course of the study, 
allowing for a main effect comparison of 150 prescribers randomized to each of Notification A vs. B. This 
estimate for prescribers is pulled from 2019 preliminary data indicating Michigan Medicine had more 
than 1000 unique DOAC prescribers, and a one-day pull in February 2020 that indicated more than 250 
prescribers on that day had one or more patients currently receiving an inappropriate DOAC, thus 
eligible for randomization to a longitudinal notificaiton. For our secondary aim, examining the main 
effect of alerts, we also anticipate randomzing 300 prescribers to initial prescription alerts over the 
study period, or 150 to each of Alert 1 and Alert 2, respectively.  
 
For our patient-level main effects analyses, we estimate that prescribers will average 2 patients that 
trigger an inappropriate DOAC alert over the 12-month trial period. As of June 14, 2022, Michigan 
Medicine currently has 11,590 patients receiving a DOAC. Pilot data indicates that about 40% of 
prescribers changed prescriptions following an notication that did not involve a pharmacist; notificaitons 
that did involve pharmacists resulted in change approximately 60% of the time. Using a base rate of 40% 
and assuming a patient n=600, alpha of 0.05 and prescriber-level ICC of 0.1, we will have 80% power to 
detect a difference in proportions of 0.12 (or 40% vs. 52%).Table 3 shows power for several other risk 
ratios, baseline rates, and number of patients per prescriber. All estimates were computed in Stata 17.  
 
Table 3. Power Calculations (Prescriber N=300)     

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
28 
 
 
 
For our secondary prescriber-level outcomes, we will have 94% power to detect a difference in 
proportions of 0.4 to 0.6. For longitudinal outcomes (e.g., maintenance), this calculation is conservative 
as it does not account for repeated measures. For our clinical effectiveness secondary outcome, we will 
have 63% power to detect a difference of 1% vs. 0.5% in adverse event rates for each factor, again 
assuming 300 prescribers with an average of 2 patients each, and assuming a prescriber-level ICC of 0.03 
(for clinical adverse events). A less conservative estimate of 3 patients per prescriber increases this 
power to 79%  
 
For our exploratory moderation analyses, described below in Section 9.3.5, again assuming 300 
prescribers with 2 patients each and an ICC of 0.1, we will have 80% power to detect a minimum effect 
size of 0.34 for our binary prescriber-level moderators, using a two-sided test with [alpha] = 0.05. If our 
average number of patients per prescriber is 3, this minimum detectable effect size decreases to 0.29 
under the same assumptions. For our binary cross-level (patient-level) moderators, assuming a non-
randomly varying slope, with an average of 2 patients per prescriber, we will have 80% power to detect 
a minimum effect size of 0.34; assuming 3 patients per prescriber decreases this minimum detectable 
effect size to 0.27. Minimum detectable effect sizes for cross-level moderators that also include a 
random slope are slightly larger at 0.36 and 0.28 respectively  
 
9.2 POPULATIONS FOR ANALYSES 
Our population for analysis is prescribers of DOACs for ambulatory patients. All eligible participants will 
be randomized.  (Randomization by prescriber, not by patient.)  All randomized participants who 
received an intervention will be included in safety analyses. All analyses, primary or secondary, will be 
reported by sex/gender and race/ethnicity as reported in the EHR. 
9.3 STATISTICAL ANALYSES 
9.3.1 GENERAL APPROACH 
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
29 
 
The main effect analyses will allow us to determine the impact of different alert- and notification types 
on evidence-based DOAC prescribing in both newly prescribed DOAC prescriptions and existing DOAC 
prescriptions in order to understand under what contexts prescriber-pharmacist collaboration is an 
effective implementation strategy. In both our primary and secondary outcomes, our analyses will 
compare the main effects of each intervention, i.e., new prescription alerts (Alerts 1 vs 2) and existing-
prescription notifications (Notifications A vs B), to determine which are most effective. For both 
interventions, the outcome of interest will be whether the patient whose prescription  triggered the 
alert/notification (i.e., is receiving a non-evidence based DOAC) has their prescription changed following 
the alert or notification.  
For the primary intervention outcome (existing-prescription notifications), main effect analyses will 
compare the proportion of patients receiving an evidence-based DOAC prescription within seven 
business days following the notification between prescribers randomized to receive the notifications 
themselves vs. prescribers for whom notifications were sent directly to anticoagulation clinic 
pharmacists.  
For the secondary intervention outcome (new DOAC prescription alerts), main effect analyses will 
compare the proportion of patients receiving an evidence-based DOAC seven business days following 
the notification between prescribers randomized to receive enhanced alerts vs. enhanced alerts plus 
pharmacist referral.  
The treatment interaction will allow evaluation of whether certain alerts and notifications work better 
when provided in tandem.  
The main outcome analysis will use mixed-effects logistic regression models to account for the binary 
patient-level outcome variable of evidence-based DOAC receipt after an alert. The two treatment 
factors and their interaction will be included as fixed effects, as well as any stratification variables and 
baseline patient- and/or prescriber-level characteristics that are unbalanced. A prescriber-level random 
effect will account for patient clustering within prescribers, and an unstructured covariance matrix will 
be used for residual errors. Similar mixed-effects regression analyses will be conducted for secondary 
outcomes, adapted for different outcome measures or levels of analysis.   
 
Missing Data. Patients will contribute primary outcome data through EHRs as long as they stay with 
Michigan Medicine. Patient attrition due to leaving Michigan Medicine also discontinues the ability for 
their Michigan Medicine prescribers or pharmacists to affect their medication prescription, so does not 
induce missing data under intent-to-treat principles. Similarly, data on prescribers (e.g., secondary 
outcome/process data) will continue to be captured via EHR as long as they are employed at Michigan 
Medicine. Prescribers that leave Michigan Medicine during the study will be lost to follow-up; however, 
the Michigan Medicine prescriber population (attending physicians, nurse practitioners, and physician 
assistants) is generally quite stable and, given the nature of the intervention, we expect attrition to be 
entirely unrelated to treatment assignment. House officers, while less stable in longevity, do have 
predictable start and stop dates for patient care, such that missingness is predictable and not related to 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
30 
 
treatment. Proposed mixed-effects analyses produce valid inferences for data that is missing at random, 
and multiple imputation will be used as appropriate. If multiple imputation is employed, all analyses will 
be performed with and without imputed data. Further sensitivity analyses will evaluate robustness of 
results under informative missingness scenarios, as appropriate.     
 
9.3.2 ANALYSES OF PRIMARY OUTCOMES 
The primary outcome will be the number (proportion) of prescriptions triggering notifications that were 
changed within 7 days, among all prescriptions that triggered a notification. 
 
As noted in the general approach, for the primary factor (existing-prescription notifications), main effect 
analyses will compare the proportion of DOAC prescriptions that were changed within seven business 
days following the notification between prescribers randomized to receive the notifications themselves 
vs. prescribers for whom notifications were sent directly to anticoagulation clinic pharmacists.  
 
9.3.3 ANALYSES OF SECONDARY OUTCOME   
Secondary outcomes will be the number (proportion) of new DOAC prescriptions triggering alerts that 
were changed within 7 days, among all prescriptions that triggered an alert. 
 
As noted in the general approach, for the secondary outcome factor (new DOAC prescription alerts), 
main effect analyses will compare the proportion of DOAC prescriptions that were changed within seven 
business days following the notification between prescribers randomized to receive enhanced alerts vs. 
enhanced alerts plus pharmacist referral.  
 
9.3.4  ANALYSES OF IMPLEMENTATION   
 
Reach: To assess reach, we will examine the proportion of patients on DOACs that had alerts or 
notifications fire, as well as the propotion of prescribers that received alerts and/or notifications. To 
futher understand reach, we will also report the proportion and clinical predictors of prescribers who 
dismissed alerts or notififcations (defined as no action within seven calendar days). We will also identify 
the proportion and predictors of prescribers who used anticoagulation clinic pharmacist referrals in 
new-prescription alerts. We will also categorize all DOAC alerts based on prescriber and patient 
characteristics (e.g., demographics, comorbidities).  
 
Effectiveness:  Analysis of Effectiveness will be the focus of the primary and secondary outcome 
measures (as noted above). We will evaluate the effectiveness of our DOAC alert system components by 
examining patient-level clinical adverse event rates (bleeding and thrombotic) within 30 days of 
triggering the alert. This time period was selected to minimize potential contamination from the safety 
review mechanism described above, and we recognize that rates for the adverse events will be low. 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
31 
 
However, as a T4 implementation study, the primary goal is implementing the evidence-based practice 
for which efficacy is already established. Supplementary analyses will also examine the effect of the 
alert system overall on adverse events.   
 
Clinical adverse events assessed will include major21 and clinically-relevant non-major bleeding 
(CRNMB)22 events, as defined by the International Society on Thrombosis and Haemostasis (ISTH), new 
or recurrent VTE events, and stroke or systemic arterial embolic events objectively identified by imaging 
and identified as a new event by their primary clinical provider. Transient Ischemic Attacks (TIAs) that do 
not result in findings on imaging (such as CT or MRI) will not be included. Each of these events will be 
captured using health informatics tools (described below) and adjudicated every six months. 
Adjudication will occur at least one month prior to six month DSMB meetings to allow for the DSMB to 
review findings in advance of the meeting. We will use Michigan Medicine-developed health informatics 
tools to create a report that identifies all patients whose prescribers have received an alert or 
notification and any adverse event diagnosis codes within 30 days thereafter. These cases will then be 
reviewed by the adjudicators to determine if an adverse event occurred and if it was associated with 
medication management, as defined in Section 8.4.2.2. The adjudicators will have access to all 
laboratory and medication data as well as clinical notes, but will be blinded to group allocation. The 
report, created by Michigan Medicine health informatics research team, will undergo a validation 
process for accuracy by the study team leads, including a cardiovascular physician and pharmacist.  
 
Implementation (including fidelity): To address how various EHR alerts impact implementation of 
evidence-based DOAC prescribing, we examine process measures related to prescriber and pharmacist 
alert receipt. For prescribers, we will examine how often they order (or do not order) the medication 
recommended by the alerts (and, for the latter, whether they provide a reason) for both new-
prescription alerts and existing-prescription notifications. For pharmacists, we will measure how often 
they respond to referrals at the time of initial prescribing or to existing prescription notifications, as well 
as the time from referral/alert until the pharmacist documents a recommendation in the EHR. For 
existing prescription notifications, we also will measure the time from notification until the notification 
is read as well as time until any DOAC prescription change or related EHR documentation occurs as well 
as the number of times a existing-prescription notification is read before being acted on.    
 
Maintenance: To measure maintenance and sustainment, we will assess all “reach” and “adoption” 
outcomes after 6 and 12 months for all patients, and after 18 months for the subset of prescribers 
enrolled prior to the second quarter of Year 2 (approximately N=225, or 75% of the total prescriber 
enrollment). This will also allow us to investigate whether treatment effects for prescribers are 
maintained after the initial adoption.  
 
Other implemention outcomes: We will also estimate the resource utilization by the pharmacists using 
a combination of EHR tool tracking (measured in hours) and self-reported weekly hours dedicated to 
study activities using a study provided tracking sheet similar to those used in our prior implementation 
trials (R01MH099898; R01MH114023).    
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
32 
 
 
9.3.5 EXPLORATORY ANALYSIS 
Interrupted time-series analyses (comparison to usual care). 
While our trial and analyses are designed to determine the best approach for improving evidence-based 
DOAC prescribing through EHR alerts via comparative effectiveness analyses, observational analyses can 
also evaluate the impact of the introduction of the DOAC EHR alert system overall on patient receipt of 
evidence-based DOACs, relative to prior to the trial. Using EHR data, we will compute the proportion of 
patients that were receiving an inappropriate DOAC for each quarter, starting 24 months prior to 
randomization and continuing 18 months after randomization. Interrupted time series analyses (ITSA) 
will then be used to compute the change in inappropriate DOAC prescriptions, pre- and post-system 
introduction, to determine whether the system implemented via the trial resulted in a significant decline 
in patients on inappropriate DOACs. We will also assess for system-level changes in adverse event rates, 
as defined in the Measures section. As a further check on changes in DOAC prescribing behaviors in 
response to the alerts and/or increased prescriber-pharmacist collaboration, we will also use ITSA 
models to assess for changes in the rate of new-prescription alerts (per total number of DOACs 
prescribed) over the 18-month study period, relative to the 24-month period prior to study start.  
 
Moderator analyses.  
Moderator analyses will be used to help pinpoint the prescriber- and patient-characteristics that most 
benefit from alerts designed to facilitate prescriber-pharmacist collaboration for both the new-
prescription alerts and existing-prescription notifications. We hypothesize that for both new-
prescription alerts and existing-prescription notifications, alerts and notifications that engage 
pharmacists will be more effective at improving patient receipt of evidence-based DOACs for prescribers 
that (1) are based in primary care vs. medical specialists and/or (2) have prescribed fewer DOACs in the 
six months preceding randomization; and for patients that (1) are aged 70+; (2) have a VTE (as opposed 
to AF) diagnosis, as the former has more complex dosing that many clinicians are less familiar with; (3) 
have 5+ concurrent medication prescriptions; and/or (4) have moderate or worse renal function 
(creatinine clearance ≤ 60 ml/min). Moderators will be assessed by adding interaction effects between 
the treatments and the moderator(s) of interest to the primary analysis model described above. All 
tested moderators will be reported and those that show both clinical and statistical significance will be 
considered for use in tailoring alert delivery.  
 
Sensitivity analyses.   
 
 Carryover effects for new-prescription alerts: Primary analyses will be conducted as intent-to-treat 
analyses among prescriber-patient dyads for whom an alert was triggered. This allows investigation of 
the best types of alerts given the need for an alert. However, for new-prescription alerts, it is also 
possible that alerts that include the option for pharmacist referral vs. those that don’t also differentially 
affect how many future DOAC prescribing errors are made. Specifically, prescriber-pharmacist 
collaboration may provide the prescriber with new knowledge about evidence-based DOAC prescribing 
that could be applied to future DOAC prescriptions, avoiding the occurrence of an alert altogether. To 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
33 
 
explore these effects, we will also evaluate the main effect of the two new-prescription alerts on the 
proportion of DOAC patients that receive a new prescription for an evidence-based DOAC using the 
model described above, but including all patients to whom they prescribed a DOAC from the point they 
received their first new-prescription alert through the end of the 18-month study period. This will allow 
us to evaluate whether correct DOAC prescribing improved overall by type of new-prescription alert.   
 
 Treatment-as-received analyses: In line with intent-to-treat clinical best practices, prescribers will be 
randomized to both types of alerts at the time of their initial receipt of either type of alert. This 
approach was chosen for both good clinical practice and pragmatic reasons, but also with the support of 
our preliminary data, which showed that on one day, more than 1/4 of Michigan Medicine prescribers 
had one or more DOAC patients receiving a non-evidence-based prescription (see Power and Sample 
Size, above). There is a possibility that during trial period, some prescribers will only ever receive one 
type of alert, never exposed to their second randomized treatment—i.e., they made an initial 
prescribing error but never had a DOACreceiving patient need a change in prescription, or vice versa. 
While we believe that this will be the case for only a minority of enrolled prescribers, sensitivity analyses 
will examine treatment-as-received effects and will re-assess main effects for the two factors including 
only patients for prescribers who received that type of alert.   
 
 Changes in patient population: While unlikely, it is also possible that a prescriber’s patient panels may 
change directly or indirectly as a result of their assigned treatment. To account for this, sensitivity 
analyses will also rerun models including only patients that were affiliated with the prescriber prior to 
randomization.   
 
9.3.6 SAFETY ANALYSES 
The number of and proportion of participants with SAEs and non-serious AE’swill be presented, overall, 
and by randomization group.  These presentations will be descriptive, with no formal inferential 
methods used.  
 
If the number of adverse events reaches a statistically significant level at p<0.01 as compared to other 
arms of the study, the trial may be halted or otherwise modified as appropriate at the time of DSMB 
review. 
10 SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS 
10.1 REGULATORY, ETHICAL, AND STUDY OVERSIGHT CONSIDERATIONS 
10.1.1 INFORMED CONSENT PROCESS 
10.1.1.1 CONSENT/ASSENT AND OTHER INFORMATIONAL DOCUMENTS PROVIDED TO PARTICIPANTS 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
34 
 
IRB approved information sheets describing in detail the study intervention, study procedures, and risks 
are made available for the participant.  Written documentation of informed consent is not required 
prior to starting the study intervention (waiver of documentation of consent approved by IRB).  
10.1.1.2 CONSENT PROCEDURES AND DOCUMENTATION 
The IRB has approved a detailed information sheet which will be provided to all eligible prescribers 
simultaneously through an electornic mail notification (weekly Epic notice sent to all Michigan Medicine 
clinicians). Participants will have the opportunity to carefully review the information sheet (available 
through a link on the weekly Epic electronic mail notification) and ask questions for at least 7 days 
before potentially being enrolled. For the randomized trial component of the study, enrollment occurs 
when prescriptions written by the individual trigger a medication alert or notification in the EHR. 
Participants are informed in writing that participation is voluntary and that they may withdraw from the 
study at any time, without prejudice.  
 
If the protocol is amended and requires a change in the consent form, determination of whether a 
participant is re-consented will be based on IRB discretion.  
10.1.2 STUDY DISCONTINUATION AND CLOSURE 
This study may be temporarily suspended or prematurely terminated if there is sufficient reasonable 
cause. Written notification, documenting the reason for study suspension or termination, will be 
provided by the suspending or terminating party to study participants, investigators, funding agency, 
and regulatory authorities.  If the study is prematurely terminated or suspended, the PI will promptly 
inform study participants, the IRB, and sponsor and will provide the reason(s) for the termination or 
suspension.  Study participants will be contacted, as applicable, and be informed of changes to study 
visit schedule. 
  
Circumstances that may warrant termination or suspension include, but are not limited to: 
● 
Determination of unexpected, significant, or unacceptable risk to participants 
● 
Insufficient compliance to protocol requirements 
● 
Data that are not sufficiently complete and/or evaluable 
Study may resume once concerns about safety, protocol compliance, and data quality are addressed, 
and satisfy the sponsor and/or the IRB. 
 
The role of the DSMB to advise suspension, resumption, and termination of the study is detailed below.  
10.1.3 CONFIDENTIALITY AND PRIVACY 
Participant confidentiality and privacy is strictly held in trust by the participating investigators, their 
staff, and the sponsor 
 
All research activities will be conducted in as private a setting as possible. 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
35 
 
 
The study monitor, other authorized representatives of the sponsor, DSMB, representatives of the IRB, 
and regulatory agencies may inspect all documents and records required to be maintained by the 
investigator, including but not limited to, medical records (office, clinic, or hospital) and pharmacy 
records for the participants in this study. The clinical study site will permit access to such records. 
 
The study participants’ contact information will be securely stored for internal use during the study in a 
password protected file on a secure University of Michigan server. At the end of the study, all records 
will continue to be kept on a secure University of Michigan server for at least 3 years beyond the 
completion of the study. 
  
Study participant research data, which is for purposes of statistical analysis and scientific reporting, will 
be stored on secure servers at the University of Michigan.  
 
10.1.4 FUTURE USE OF STORED SPECIMENS AND DATA 
Data collected for this study will be analyzed and stored on a secure server at the University of 
Michigan. After the study is completed, the data will be de-identified and archived for a minimum of 6 
years.  Details shared with prescribers about data retention and use is included in the IRB-approved 
information sheet.  
10.1.5 KEY ROLES AND STUDY GOVERNANCE 
Principal Investigator 
Principal Investigator 
Geoffrey Barnes, MD, MSc 
Shawna Smith, PhD 
University of Michigan 
University of Michigan 
Frankel Cardiovascular Center and Institute for 
Healthcare Policy and Innovation  
2800 Plymouth Rd, B14 G214  
Ann Arbor, MI 48109-2800, USA  
Department of Health Management and Policy 
School of Public Health 
University of Michigan 
1415 Washington Heights 
Ann Arbor 48109 
gbarnes@med.umich.edu 
shawnana@umich.edu 
  
10.1.6 SAFETY OVERSIGHT 
Safety oversight will be under the direction of an internal Data and Safety Monitoring Board (DSMB) 
composed of individuals with the appropriate expertise, including members with expertise in cardiology, 
statistics, pharmacy, and/or research ethics. Members of the DSMB will be independent from the study 
conduct and free of conflict of interest, or measures should be in place to minimize perceived conflict of 
interest. The DSMB will meet twice a year to assess safety and efficacy data on each arm of the study. 
The DMSB will operate under the rules of an approved charter that will be written and reviewed at the 
organizational meeting of the DSMB. The DSMB will provide its input to the sponsor (AHRQ), the PIs, and 
the governing IRB at the University of Michigan through a report issued after each meeting.  

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
36 
 
 
The DSMB will act in an advisory capacity to the study team, the IRB, and the sponsor (AHRQ) to monitor 
patient safety and data quality. They will review the study protocol, monitor all aspects of the study 
(e.g., recruitment, adverse events, protocol adherence, data quality, attrition, demographic and baseline 
characteristics) and recommend protocol modifications, including early study termination. All proposed 
changes to the study protocol will be reviewed by the DSMB.  
 
In the event that the DSMB believes the study should be suspended or terminated due to safety or data 
management concerns, the DSMB will make a specific and written recommendation to the study 
investigators, sponsor (AHRQ), and IRB. In the case of a temporary suspension, the DSMB will 
subsequently make a written recommendation to the study investigators, sponsor, and IRB prior to 
resuming study activities. 
 
Reports will be prepared by the PIs prior to each meeting of the DSMB. These will minimally include 
recruitment, adherence, attrition, adverse events, data quality, and descriptive characteristics of the 
study sample both in aggregate and according to randomization assignment (unblinded if specifically 
requested by the DSMB). The DSMB will meet at least twice a year via teleconference/webinar to review 
the cumulative data. A recording secretary will be available for all DSMB meetings. The sponsor or the 
chairman of the DSMB in consultation with the sponsor may convene additional meetings. Based on the 
data presented, the DSMB will make recommendations including continuation or termination of the 
study. 
 
A report containing the recommendations for continuation or modification of the study will be 
prepared. The draft report will be sent to the DSMB members for review and approval no later than 
three weeks after the meeting. It is the responsibility of the Principal Investigator to distribute the DSMB 
recommendation to all co-investigators and to ensure that copies are submitted to the IRB that 
reviewed and approved the study documents. 
10.1.7 DATA HANDLING AND RECORD KEEPING 
10.1.7.1 DATA COLLECTION AND MANAGEMENT RESPONSIBILITIES 
The investigator is responsible for ensuring the accuracy, completeness, and timeliness of the data 
reported. Clinical data (including adverse events (AEs), concomitant medications, and specific 
prescribing data) and clinical laboratory data will be stored on HIPAA compliant servers provided by the 
Michigan Clinical Outcomes Research and Reporting Program (MCORRP) using password-protected CSV 
files.  
10.1.7.2 STUDY RECORDS RETENTION 
Paper and electronic data for the study will be retained for at least 3 years after the study ends. 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
37 
 
10.1.8 PROTOCOL DEVIATIONS 
All protocol deviations (e.g., data breach, receipt of incorrect alert or notification) that significantly 
impact the integrity of the study or subject safety must be reported within 7 days through IRBMED. Any 
event that requires immediate action will be brought to the attention of the IRB Chair. Events that meet 
the criteria of serious and/or continuing noncompliance will be communicated to IRBMED.  
10.1.9 PUBLICATION AND DATA SHARING POLICY 
AHRQ policy requires that unique research data be made available to the scientific community after the 
conclusion of a study. In accordance with NIH NOT-OD-16-149 and DHHS rule 81 FR 64922, our trial is 
registered with clinicaltrials.gov [ClinicalTrials.gov Identifier: NCT05351749]. We report our results to 
clinicaltrials.gov as well as present our findings through conventional dissemination routes like 
publications and presentations at specialty societies. 
 
Resource sharing may include sharing of study documents (e.g., recruitment materials, protocols, SOPs, 
and informed consents that may be useful as starting points for other studies).  
10.1.10 CONFLICT OF INTEREST POLICY 
The independence of this study from any actual or perceived influence is critical.  Therefore, any actual 
conflict of interest of persons who have a role in the design, conduct, analysis, publication, or any aspect 
of this trial will be disclosed and managed. Furthermore, persons who have a perceived conflict of 
interest will be required to have such conflicts managed in a way that is appropriate to their 
participation in the design and conduct of this trial.  
10.2 ADDITIONAL CONSIDERATIONS 
None 
 
10.3 ABBREVIATIONS 
AE 
Adverse Event 
AF 
Atrial Fibrillation 
AESI 
Adverse Event of Special Interest 
AHRQ 
Agency for Healthcare Research and Quality 
CDS 
Clinical Decision Support 
CFIR 
Consolidated Framework for Implementation Research 
CFR 
Code of Federal Regulations 
DOAC 
Direct Oral Anticoagulant 
DSMB 
Data and Safety Monitoring Board 
EHR 
Electronic Health Record 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
38 
 
ERIC 
Expert Recommendations for Implementing Change 
FDA 
Food and Drug Administration 
GCP 
Good Clinical Practice 
ICH 
International Conference on Harmonization 
IQR 
Interquartile Range 
IRB 
Institutional Review Board 
IRBMED 
Institutional Review Board for Michigan Medicine 
ISTH 
International Society on Thrombosis and Haemostasis 
MAQI2 
Michigan Anticoagulation Quality Improvement Initiative 
MiChart 
EHR used in Michigan Medicine 
MOP 
Manual of Procedures 
NIH 
National Institutes of Health 
NHLBI 
National Heart Lung and Blood Institute 
OHRP 
Office for Human Research Protections 
PHI 
Patient Health Information  
PI 
Principal Investigator 
QC 
Quality Control 
SAE 
Serious Adverse Event 
SAP 
Statistical Analysis Plan 
SBP 
Systolic Blood pressure 
SOP 
Standard Operating Procedure 
UP 
Unanticipated Problem 
US 
United States 
VTE 
Venous thromboembolism 
 
 
10.4 PROTOCOL AMENDMENT HISTORY 
Version 
Date 
Description of Change  
Brief Rationale 
 
 
 
 
  
  
  
  
  
  
  
  
 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
39 
 
11 REFERENCES 
REFERENCES 
1.  
Kearon C, Akl EA, Ornelas J, et al. Antithrombotic Therapy for VTE Disease: CHEST Guideline and Expert Panel 
Report. Chest. 2016;149(2):315-352. doi:10.1016/j.chest.2015.11.026 
2.  
January CT, Wann LS, Calkins H, et al. 2019 AHA/ACC/HRS Focused Update of the 2014 AHA/ACC/HRS 
Guideline for the Management of Patients With Atrial Fibrillation: A Report of the American College of 
Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart R. 
Circulation. 2019;140(2):e125-e151. doi:10.1161/CIR.0000000000000665 
3.  
Barnes GD, Lucas E, Alexander GC, Goldberger ZD. National trends in ambulatory oral anticoagulant use. 
American Journal of Medicine. 2015;128(12). doi:10.1016/j.amjmed.2015.05.044 
4.  
Steinberg BA, Shrader P, Thomas L, et al. Off-Label Dosing of Non-Vitamin K Antagonist Oral Anticoagulants 
and Adverse Outcomes: The ORBIT-AF II Registry. J Am Coll Cardiol. 2016;68(24):2597-2604. 
doi:10.1016/j.jacc.2016.09.966 
5.  
Arbel R, Sergienko R, Hammerman A, et al. Effectiveness and Safety of Off-Label Dose-Reduced Direct Oral 
Anticoagulants in Atrial Fibrillation. Am J Med. 2019;132(7):847-855 e3. doi:10.1016/j.amjmed.2019.01.025 
6.  
Sanghai S, Wong C, Wang Z, et al. Rates of Potentially Inappropriate Dosing of Direct-Acting Oral 
Anticoagulants and Associations With Geriatric Conditions Among Older Patients With Atrial Fibrillation: The 
SAGE-AF Study. J Am Heart Assoc. 2020;9(6):e014108. doi:10.1161/JAHA.119.014108 
7.  
Hanigan S, Das J, Pogue K, Barnes GD, Dorsch MP. The real world use of combined P-glycoprotein and 
moderate CYP3A4 inhibitors with rivaroxaban or apixaban increases bleeding. J Thromb Thrombolysis. 
Published online 2020. doi:10.1007/s11239-020-02037-3 
8.  
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health 
services research findings into practice: a consolidated framework for advancing implementation science. 
Implement Sci. 2009;4:50. doi:10.1186/1748-5908-4-50 
9.  
Amroze A, Mazor K, Crawford S, O’Day K, McManus DD, Kapoor A. Survey of confidence in use of stroke and 
bleeding risk calculators, knowledge of anticoagulants, and comfort with prescription of anticoagulation in 
challenging scenarios: SUPPORT-AF II study. Journal of Thrombosis and Thrombolysis. 2019;48(4):629-637. 
doi:10.1007/s11239-019-01950-6 
10.  Murphy A, Kirby A, Bradley C. Knowledge is power: General practitioners prescribing of new oral 
anticoagulants in Ireland. BMC Research Notes. 2018;11(1):4-9. doi:10.1186/s13104-018-3597-x 
11.  Komwong D, Greenfield G, Zaman H, Majeed A, Hayhoe B. Clinical pharmacists in primary care: a safe 
solution to the workforce crisis? Journal of the Royal Society of Medicine. 2018;111(4):120-124. 
doi:10.1177/0141076818756618 
12.  Hwang AY, Gums TH, Gums JG, Uni- HP. The benefits of physician- pharmacist collaboration. 2017;66(12):1-9. 
13.  Quinn M, Forman J, Harrod M, et al. Electronic health records, communication, and data sharing: challenges 
and opportunities for improving the diagnostic process. Diagnosis. 2018;6(3):241-248. doi:10.1515/dx-2018-
0036 
14.  Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the 
Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. 
doi:10.1186/s13012-015-0209-1 
15.  Barnes GD, Kline-Rogers E, Graves C, et al. Structure and function of anticoagulation clinics in the United 
States: an AC forum membership survey. J Thromb Thrombolysis. 2018;46(1):7-11. doi:10.1007/s11239-018-
1652-z 
16.  Leef GC, Perino AC, Askari M, et al. Appropriateness of Direct Oral Anticoagulant Dosing in Patients With 
Atrial Fibrillation: Insights From the Veterans Health Administration. Journal of Pharmacy Practice. Published 
online 2019:897190019828270. doi:10.1177/0897190019828270 
17.  Tolley CL, Slight SP, Husband AK, Watson N, Bates DW. Improving medication-related clinical decision 
support. Am J Health Syst Pharm. 2018;75(4):239-246. doi:10.2146/ajhp160830 

DOAC CDS Alert AHRQ R18 
 Version 1.0 
June 1, 2022 
40 
 
18.  Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and 
clinical role tailoring: a systematic review. J Am Med Inform Assoc. 2019;26(10):1141-1149. 
doi:10.1093/jamia/ocz095 
19.  Marcilly R, Ammenwerth E, Roehrer E, Nies J, Beuscart-Zephir MC. Evidence-based usability design principles 
for medication alerting systems. BMC Med Inform Decis Mak. 2018;18(1):69. doi:10.1186/s12911-018-0615-9 
20.  Dawson T, DeCamillo D, Kong X, et al. Correcting Inappropriate Prescribing of Direct Oral Anticoagulants: A 
Population Health Approach. Journal of the American Heart Association. 2020;9(22):e016949. 
doi:10.1161/JAHA.120.016949 
21.  Schulman S, Kearon C. Definition of major bleeding in clinical investigations of antihemostatic medicinal 
products in non-surgical patients. J Thromb Haemost [Internet] 2005;3(4):692–4. Available from: 
http://www.ncbi.nlm.nih.gov/pubmed/15842354 
22. Kaatz S, Ahmad D, Spyropoulos AC, Schulman S, Subcommittee on Control of A. Definition of clinically 
relevant non-major bleeding in studies of anticoagulants in atrial fibrillation and venous thromboembolic 
disease in non-surgical patients: communication from the SSC of the ISTH. J Thromb Haemost [Internet] 
2015;13(11):2119–26. Available from: https://www.ncbi.nlm.nih.gov/pubmed/26764429 
23. Hanauer DA, Mei Q, Law J, Khanna R, Zheng K. Supporting information retrieval from electronic health 
records: A report of University of Michigan’s nine-year experience in developing and using the Electronic 
Medical Record Search Engine (EMERSE). J Biomed Inf [Internet] 2015;55:290–300. Available from: 
https://www.ncbi.nlm.nih.gov/pubmed/25979153 
 
12 APPENDICES 
 
Appendix A:  < ICD10 diagnosis and procedure codes for longitudinal system-level analyses > 
[This appendix will be included and updated in coordination with or data analyst, as needed.] 
 
Appendix B: Appropriate DOAC prescribing guidelines