Evaluating Strategies to Improve Guideline Directed Medical Therapy: The GDMT Research, Education & Assist Trial for Heart Failure Care (GREAT-HF Care)
Phase: NA
Conditions: Heart Failure, Heart Failure With Reduced Ejection Fraction
Sponsor: Geisinger Clinic

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 1 of 26 
STUDY PROTOCOL 
 
Study Information 
Title 
Evaluating Strategies to Improve Guideline 
Directed Medical Therapy: The GDMT 
Research Education & Assist Trial for Heart 
Failure Care (GREAT-HF Care) 
Protocol version identifier 
Version 1.0 
Date 
September 4, 2024 
Research objective 
The objective of this study is to implement 
and evaluate a multifaceted, interdisciplinary 
intervention to improve guideline-directed 
medical therapies (GDMT) use, reduce 
mortality, and reduce future heart failure 
events in patients with HFrEF. 
NCT Number 
NCT05990296 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 2 of 26 
TABLE OF CONTENTS 
1. LIST OF ABBREVIATIONS ................................................................................................3 
2. RESPONSIBLE PARTIES ....................................................................................................4 
3. ABSTRACT ...........................................................................................................................5 
4. BACKGROUND AND SIGNIFICANCE .............................................................................6 
5. OBJECTIVE AND HYPOTHESIS .......................................................................................7 
5.1 Objective ....................................................................................................................7 
5.2 Hypothesis ..................................................................................................................7 
6. STUDY METHODS ..............................................................................................................7 
6.1 Study Design ..............................................................................................................7 
6.2 Randomization details ................................................................................................8 
6.3 Setting .........................................................................................................................8 
6.4 Targeted Clinician Inclusion ......................................................................................9 
6.5 Inclusion Criteria ........................................................................................................9 
6.6 Exclusion Criteria .......................................................................................................9 
6.7 Interventions ...............................................................................................................9 
6.8 Outcome Measures ...................................................................................................17 
7. STATISTICAL ANALYSIS PLAN ....................................................................................19 
7.1 Power analysis ..........................................................................................................19 
7.2 Analysis details ........................................................................................................19 
7.2.1 Descriptive statistics ....................................................................................19 
7.2.2 Approach for comparisons across intervention arms ...................................19 
7.2.3 Regression Specifications ............................................................................20 
7.3 Variables ...................................................................................................................21 
7.4 Data Sources .............................................................................................................21 
7.5 Data Management ....................................................................................................21 
7.6 Record Retention ......................................................................................................21 
8. PROTECTION OF HUMAN SUBJECTS ..........................................................................22 
8.1 Patient Information ...................................................................................................22 
9. PLANS FOR DISSEMINATING STUDY RESULTS .......................................................22 
10. PROJECT STATUS...........................................................................................................22 
11. REFERENCES ..................................................................................................................23 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 3 of 26 
1. LIST OF ABBREVIATIONS 
Abbreviation 
Definition 
ACEi 
Angiotensin converting enzyme inhibitors 
ARB 
Angiotensin receptor blockers 
ARNI 
Angiotensin-neprilysin inhibitors 
BPA 
Best practice alert 
CDS 
Clinical decision support 
CE 
Cost Effectiveness 
CKD 
Chronic kidney disease 
CME 
Continuing medical education 
CPA 
Collaborative practice agreement 
EBBB 
Evidence-based beta-blockers 
ED 
Emergency department 
EHR 
Electronic health records 
GDMT 
Guideline-directed medical therapies 
GHP 
Geisinger Health Plan 
HF 
Heart failure 
HFrEF 
Heart failure with reduced ejection fraction 
ICER 
Incremental cost effectiveness ratios 
ITT 
Intention-to-treat 
LVEF 
Left ventricular ejection fraction 
MTDM 
Medication Therapy Disease Management 
MRA 
Mineralocorticoid receptor antagonists 
MRN 
Medical record number 
OLS 
Ordinary least squares 
PACDC 
Phenomic Data Analytics and Clinical Data Core 
PHI 
Protected health information 
ROI 
Return on Investment 
SGLT2i 
Sodium-glucose cotransporter 2 inhibitors  
WTP 
Willingness-to-pay 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 4 of 26 
2. RESPONSIBLE PARTIES 
 
Principal Investigator(s) of the Protocol 
Name 
Job Title 
Affiliation 
Voyce, Stephen 
Physician Cardiologist and 
Director of Clinical 
Cardiology Research 
Geisinger Community 
Medical Center 
Wright, Eric 
Professor and System 
Director 
Geisinger Center for 
Pharmacy Innovation and 
Outcomes 
Goren, Amir 
Program Director 
Behavioral Insights Team 
Geisinger Steele Institute for 
Health Innovation 
 
Project Management 
Name 
Job Title 
Affiliation 
Hayduk, Vanessa 
Project Manager I 
Geisinger Center for 
Pharmacy Innovation and 
Outcomes 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 5 of 26 
3. ABSTRACT 
GREAT- HF Care 
 
• Background and Rationale - Heart failure with reduced Ejection Fraction (HFrEF) is 
associated with high mortality and adverse events (hospitalization or urgent outpatient 
visits for heart failure (HF)), along with diminished quality of life. Despite convincing 
data that evidenced-based, guideline-directed medical therapies (GDMT) improve 
mortality and heart failure-related events, there remains insufficient utilization of these 
life-saving drugs (evidence-based beta-blockers (EBBB), angiotensin-neprilysin 
inhibitors (ARNI)/ angiotensin converting enzyme inhibitors (ACEi)/ angiotensin 
receptor blockers (ARB), mineralocorticoid receptor antagonists (MRA) and sodium-
glucose cotransporter 2 inhibitors (SGLT2i)) in patients with HFrEF. 
• Objectives - The primary objective of this study is to implement and evaluate a 
multifaceted, interdisciplinary intervention to improve GDMT use, reduce mortality, and 
reduce future heart failure events in patients with HFrEF. 
• Study Design – We will conduct a cluster-randomized controlled trial at the level of the 
clinician comparing two multi-faceted interventions to usual care with or without 
systemic education within a population of patients with HFrEF seen at a Geisinger 
cardiology clinic who are not on complete GDMT.  
• Population - Patients at least 18 years of age with documented HFrEF or Left Ventricular 
Ejection Fraction (LVEF) < 40 (within two years prior to the triggering visit) who have 
an outpatient encounter with the Geisinger Cardiology department (office visit or 
telemedicine). Patients must also not currently be on optimal GDMT according to their 
electronic health records (EHR), as reflected in use of medications in fewer than four of 
the recommended classes or in all four classes but not ARNI. And patients must not 
currently be in hospice or palliative care or allergic to all non-prescribed GDMT. The 
clinician target population comprises clinicians practicing at one or more Geisinger 
outpatient cardiology clinics who manage care for patients with HFrEF and include 
cardiologists (MD/DO), nurse practitioners, and physician assistants. 
• Interventions – We will implement the following interventions: 1) focused education 
through a series of focused in-person or virtual sessions (randomized to receive early or 
late education); 2) multi-pronged clinical decision support (CDS) inclusive of a patient 
portal message, an interruptive advisory upon chart entry notifying clinicians to consider 
GDMT, and a Best Practice Advisory (BPA) that opens a GDMT order set; and 3) the 
same CDS as #2 but replacing the GDMT order set with a referral to integrated clinical 
pharmacist co-management.  
• Control – Patients seen by clinicians randomized to usual care will be labeled as control 
patients and receive care as per usual standard practice, which may also include referral 
to Medication Therapy Disease Management (MTDM). 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 6 of 26 
• Outcomes – The primary outcome will be whether or not improvements in GDMT 
prescribing occur within 30 days of index date. Additional outcomes will include the 
primary outcome at 60 and 90 days, GDMT changes at 30, 60 and 90 days, healthcare 
resource utilization, and total costs of care. 
• Data analysis – Analyses are described in detail below. Primary analyses will assess 
whether outcomes improve between each interventional arm versus control.  
 
4. BACKGROUND AND SIGNIFICANCE 
HFrEF is associated with high mortality and events (hospitalization or urgent outpatient visit 
for HF), along with poorer quality of life.1 Newer agents (ARNI, SGLT2i) have been 
demonstrated to improve outcomes in multiple trials2-4 and are strongly advocated by 
ACC/AHA Guidelines1 Despite evidenced-based, guideline-directed medical therapies 
(GDMT) including evidence-based beta-blockers (EBBB), angiotensin-neprilysin inhibitors 
(ARNI) /Angiotensin converting enzyme inhibitors (ACEi)/ angiotensin receptor blockers 
(ARB), mineralocorticoid receptor antagonists (MRA) and sodium-glucose cotransporter 2 
inhibitors (SGLT2i), there remains insufficient utilization of these life-saving drugs in 
patients with HFrEF.1  
Some background research taught us the following:  
- 
A review of Geisinger data on 7,144 patients with HFrEF from 2018-2021 
demonstrated that ~80% of patients were on EBBB; ~50% on ARNI/ACEi/ARB; 
(3.2-9.7% ARNI), 16-22% on an MRA and 1.3-4.6% on SGLT2i.5 
- 
Machine learning algorithms derived from Geisinger data suggest prioritization of 
patients and closure of treatment gaps will result in lower patient mortality.6 
- 
Interviews with 20 Geisinger cardiologists identified “familiarity with newer 
medications” and “recollection at time of prescribing” as barriers to prescribing 
GDMT and “assistance by a clinical pharmacist” as a facilitator.7 
 
Electronic health record functionality and clinical pharmacy services may help to address 
underlying barriers that presently exist to GDMT optimization. Although clinical decision 
support (CDS) and best practice alerts (BPAs) are ubiquitous in the EPIC system and may 
assist with broad efforts to improve GDMT prescribing, many barriers exist to their 
effectiveness in practice. 
- 
HF BPAs do not lead to action: A recent analysis identified 454 BPAs fired for HF 
for 257 unique patients. There were zero actions taken and 66 overrides of the BPA.8 
- 
HF BPAs are not comprehensive: Present BPAs firing for HF at Geisinger are limited 
to single drug recommendations, do not provide sufficient context for adequate 
decision-making, and do not include newer agents (i.e., ARNI or SGLT2i) with 
strong evidence for use. 
- 
HF BPAs are not designed with best evidence principles: Alerts can be ineffective 
due to multiple design factors that affect behavior, such as inappropriate workflow 
placement, passive alerting, and alert fatigue.9 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 7 of 26 
Having identified the problems above, we have investigated solutions to help guide us to 
improve GDMT prescribing. Solutions include: 
- 
Education and engagement of the clinical staff (physicians, nurse practitioners, 
physician assistants, clinical pharmacists, nurses). 
- 
Electronic facilitation to overcome clinical inertia. 
o Recent evidence from the PROMPT-HF study suggests a more tailored 
advisory presented to prescribers increases GDMT.10 
o Additional evidence from the BETTER CARE-HF study suggests that alerting 
informed by evidence-based design principles can substantially increase MRA 
prescribing in HFrEF.11  
o Principles guiding lightweight interventions (behavioral “nudges”), from the 
fields of psychology and economics—including loss framing, social norms, 
optimal defaults, and other means of changing choice architecture—can be 
implemented using tools such as EHR-based CDS to increase the likelihood 
that clinicians and patients engage more consistently in optimal behaviors.12,13 
- 
Embedded pharmacists with established collaborative practice agreements (CPA) 
enabling medication assessment, initiation, titration, monitoring and management. 
 
5. OBJECTIVE AND HYPOTHESIS 
5.1 Objective 
The primary objective of this study is to implement and evaluate a multifaceted, 
interdisciplinary intervention to improve GDMT use, reduce mortality, and reduce future 
heart failure events in patients with HFrEF. 
5.2 Hypothesis 
H0: The implementation of a multifaceted, interdisciplinary intervention will not improve 
GDMT for patients with HFrEF. 
HA1: The implementation of a multifaceted, interdisciplinary intervention incorporating an 
order set will improve GDMT for patients with HFrEF more than usual care. 
HA2: The implementation of a multifaceted, interdisciplinary intervention incorporating a 
pharmacist referral will improve GDMT for patients with HFrEF more than usual care.  
 
6. STUDY METHODS  
6.1 Study Design  
This is a cluster randomized study designed to evaluate the effectiveness of interventions 
aimed at improving GDMT in patients with HFrEF. Clinicians were stratified based on 
practice specialty, location, and pharmacist referral habits (see details below) and were 
permuted block randomized to achieve 45%/45%/10% proportional distribution across the 
following arms respectively: (1) usual care (Control arm); (2) multi-pronged CDS inclusive 
of a patient portal message about GDMT, an interruptive advisory upon chart entry advising 
clinicians of the need to optimize treatment, and a BPA that links to a GDMT order set 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 8 of 26 
(CDS-OS arm); and (3) multi-pronged CDS as in #2 but replacing GDMT order set with 
referral to integrated clinical pharmacist co-management (CDS-MTDM arm). Secondarily, 
each of the 5 clinical practice sites of roughly equal HFrEF patient loads were assigned to 
receive either an early or delayed education rollout.  
6.2 Randomization details 
Permuted blocked randomization was completed once at the beginning of the study.  
 
Randomization was stratified as described below given concerns about imbalances in GDMT 
prescribing practices across clinics, certain roles, and current MTDM referral practices. 
 
Clinicians were divided into HF specialists and non-HF specialists given differences in 
practicing patterns and propensity for increased GDMT prescribing patterns among 
specialists. 
 
Within Non-HF specialists, randomization was stratified on the following variables: 
• Role: electrophysiology, interventional, non-invasive  
• MTDM referral history: none/0%, 1%-100% 
• Primary practice location: Northeast (GCMC, GWV, Orwigsburg), Central (GMC, 
Shamokin, Lewisburg), and West (GLH, Gray's Woods)  
 
Within HF specialists, randomization was stratified on MTDM referral history. They were 
NOT stratified on practice location because there was not hypothesized to be a significant 
difference in practice as a function of location in this group. 
 
We also generated permuted block randomized sequences for allocating any new clinicians 
who enter the system during the study. The study team collected information about role and 
primary practice location for any new clinician joining the system monthly. However, all 
new clinicians were assigned to the 0% MTDM referral history stratum, because new 
clinicians do not have an MTDM referral history in the system. Based on the pre-randomized 
sequence generated, the clinician would be assigned to an interventional arm. In generating 
these sequences, we assumed turnover rates would be equal across strata. 
 
One hundred and fifty-five clinicians (103 current and 52 hypothetical) were randomized into 
control (45%), CDS-OS (45%) and CDS-MTDM arms (10%). 
 
6.3 Setting 
This study is being implemented at Geisinger, an integrated health delivery network in 
Central and Northeastern Pennsylvania, that serves more than 1 million patients per year. The 
system includes 10 hospital campuses, a health plan with more than half a million members, 
two research centers, an addiction treatment center, and the Geisinger Commonwealth 
School of Medicine. Geisinger’s Cardiology Department includes 35 outpatient clinics with 
over 120 providers engaged in care delivery. 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 9 of 26 
6.4 Targeted Clinician Inclusion 
Clinicians practicing within a Geisinger cardiology clinic and caring for patients with HFrEF 
were included. These include Cardiologist Physicians (MD, DO), Nurse Practitioners, and 
Physician Assistants. The lead Principal Investigator on this study was excluded from 
participation. 
 
6.5 Inclusion Criteria  
Patients are included if: 
1. Patients are aged 18 years or older  
AND  
2. Completed visit at included Geisinger cardiology outpatient clinics (office visit or 
telemedicine)  
AND  
3. Patients’ clinicians are on a list of currently active Geisinger clinicians in outpatient 
cardiology clinics who can prescribe heart failure medications  
AND  
4a. Active problem list diagnosis of HFrEF at time of Cardiology clinic encounter,  
OR  
4b. Left Ventricular Ejection Fraction (LVEF) < 40: most recent to the cardiology clinic 
encounter within 2 years of the visit.   
 
6.6 Exclusion Criteria 
Currently in hospice or palliative care. (ICD 10 code: Z51.5)  
Patient has been prescribed medications from all four categories of GDMT, including ARNI 
specifically. 
Patient has a documented allergic reaction to each class of GDMT not presently prescribed.  
 
6.7 Interventions 
The study includes the following three intervention arms, crossed with the presence or 
absence of focused education: 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 10 of 26 
1. Control: Clinicians in this arm do not receive CDS and instead experience usual care. 
To index targeted patients, we initiated a BPA during eligible encounters for patients 
meeting inclusion/exclusion criteria that is not seen by clinicians but is recorded with 
date and time for the study team. No additional interventions are provided (i.e., no 
order sets or referral notices), although pharmacist services are available to clinicians 
that are allocated to this group. 
2. Multiprong CDS with GDMT order set (CDS-OS), including: 
a. A patient-facing nudge to improve GDMT prescribing is delivered to patients 
14 days prior to their scheduled cardiology encounters. The questionnaire 
includes a statement about their cardiac mortality risk and a prompt inquiring 
if the patient would like to discuss GDMT options with their provider. For 
those without portal access, an in-clinic questionnaire delivered at the time of 
rooming with the nurse is available. See Figure 1 below.  
Figure 1. Patient-facing questionnaire presented through patient portal. 
 
b. A clinician-facing “Heads-Up” BPA that fires upon initial launch of the 
encounter in Epic. The BPA notifies the clinician of the patient’s diagnosis of 
HFrEF and encourages assessment of present GDMT. Patient response to the 
pre-visit questionnaire is provided in alternate color font if completed in the 
affirmative by the patient. See Figures 2 and 3 below. 
 
 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 11 of 26 
Figure 2. Provider-facing EHR alert upon entry into patient chart (without portal 
information). 
 
Figure 3. Provider-facing EHR alert upon entry into patient chart (with portal information). 
 
c. A BPA during the encounter that supplies a multitude of relevant heart failure-
related metrics (e.g. Left Ventricular Ejection Fraction, heart rate, glomerular 
filtration rate, etc.) and indicates present use or non-use of GDMT along with 
the medication information. If a class of GDMT is not active on the EHR 
medication list, alternative text is conveyed to the provider, in bolded and 
alternate color font, the mortality reductions expected with drug class 
initiation, as observed in published studies. The preferred action within the 
BPA is to open the GDMT order set, which has been informed by prior 
studies and by behavioral science nudge principles. Classes least likely to be 
prescribed (i.e., ARNI, SGLT2i, and MRA) are listed before the more 
commonly prescribed EBBBs. One-click ordering options are available within 
the order set that link to a medication order if selected. If this option is not 
selected, the clinician must document their reason for not selecting a GDMT 
change in the acknowledgement section of the BPA. This section includes one 
option for patient declining treatment (where a written reason is not required), 
one for a contraindication to prescription (with required open text response), 
and an “other” category (with required open text response). See figure 4 
below for one example of how a BPA may appear. 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 12 of 26 
Figure 4. Provider-facing EHR alert upon order entry for GDMT prescribing. 
 
3. Multiprong CDS with referral to pharmacist co-management (CDS-MTDM), 
including: 
a. A patient-directed questionnaire administered as above in the multi-prong 
CDS intervention.   
b. A clinician-facing “Heads-Up” BPA that fires upon initial launch of the 
encounter in Epic. The BPA notifies the clinician of the patient’s diagnosis of 
HFrEF and encourages referral to pharmacy for co-management. Patient 
responses to the pre-visit questionnaire are provided in alternate color font if 
completed as affirmative by the patient. See Figures 5 and 6 below. 
 
 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 13 of 26 
Figure 5. Provider-facing EHR alert upon entry into patient chart (without portal 
information). 
 
Figure 6. Provider-facing EHR alert upon entry into patient chart (without portal 
information). 
 
c. A BPA during the encounter that presents information exactly as in the 
GDMT order set version described above. The preferred action within the 
BPA is to refer patients to embedded pharmacist co-management, which is 
already highlighted for the clinician with an urgent referral sent to pharmacy. 
Pharmacists are expected to schedule patients within 3 days and meet with 
patients and optimize GDMT through a CPA. If this option is not selected, the 
clinician must document their reason for not selecting referral in the 
acknowledgement section of the BPA. This section included one option for 
patient declining treatment (will manage Rx on their own), and an “other” 
category (with required open text response). See Figure 7 below. 
 
 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 14 of 26 
Figure 7. Provider-facing EHR alert upon order entry for pharmacist referral and co-
management. 
 
 
The main interruptive BPA is located in the Best Practice Advisory section of the 
patient charts. If no action is taken, the BPA fires at the time of first order or when the 
clinician tries to close out of the chart. Once an action has been taken, the BPA is 
suppressed for a period of 30 days for that patient (except for contraindications, 
which result in a 90-day lockout period), to avoid repetitive firing and resultant alert 
fatigue. This suppression logic is applicable both to the order set and the referral-
specific BPAs. A silent BPA is triggered for clinicians assigned to the no-BPA 
control arm and for all included clinicians and sites between the time when clinicians 
are randomized across the interventional and control BPAs and in the 2 months prior, 
when the initial round of education has been rolled out. This is intended to assist with 
tracking prescribing behavior accurately.  

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 15 of 26 
In the pharmacist co-management arm, pharmacists follow a protocol for managing 
patients with HFrEF as developed by Cardiology and pharmacy. The procedures are 
listed in Tables 1 and 2 below. 
 
Table 1. Cardiologist and pharmacist HFrEF treatment intervals and actions. 
 
 
 
 
 
 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 16 of 26 
Table 2. GDMT treatment options and monitoring parameters. 
 
The mechanism to refer patients to MTDM who were eligible was agreed upon by the 
study team and clinical leadership in advance of intervention implementation. Since 
individual clinician referral rates to MTDM varied prior to study initiation, those with 
none or few MTDM referrals were educated on the role of the MTDM pharmacist and 
referral process. To assist with implementation, feedback on referral rates among 
eligible patients is provided to pharmacy leadership and clinical pharmacists 
throughout the study. In some cases, 1:1 discussions with clinicians on referring 
patients into MTDM care are made to address barriers to placing MTDM referrals 
within the CDS. 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 17 of 26 
4. Focused Education: A series of focused, interactive education sessions were 
developed by study team members and clinical personnel to educate cardiology 
clinicians on why, when, and how to prescribe GDMT to patients. The objectives of 
the sessions are to: address previously identified barriers to GDMT such as clinician 
knowledge and comfort with use of GDMT, encourage clinician prescribing of 
GDMT, and improve attention to MTDM to work cohesively to improve GDMT 
prescribing. 
 
The target audience for this education program are Cardiology clinicians (physicians 
(MD, DO) and advanced practitioners (PA, CRNP)) at Geisinger. Three clinics were 
selected for early education (i.e. before additional CDS and pharmacist referral 
interventions) and two clinics were reserved for late education (timed to occur after 
sufficient time has elapsed to assess the impact of the early education on its own). 
The early education programs were delivered over one month to each clinic with in-
person attendance being encouraged.  
 
Topics delivered in the early education sessions included: 
1. Sacubitril / Valsartan (ARNI) in CHF 
2. SGLT2 inhibitors in CHF  
a. SGLT2i in Cardiovascular Disease & HFrEF 
b. SGLT2i in HFrEF with Diabetes: What the Cardiologist should know  
3. Management of HFrEF and chronic kidney disease (CKD) 
4. Geisinger Pharmacist role in HFrEF 
 
The early education sessions were delivered live, at each clinic location separately on 
a given date (either in-person or virtually depending on clinic preference). 
Participants were encouraged by the lead PI to attend the live sessions if they were 
being offered at their clinic location. Continuing medical education (CME) credits 
were provided to attendees where applicable and lunch was provided during the live 
sessions. Presentations lasted about 30 minutes with a question-and-answer session to 
follow. An RSVP request was asked of those attending. Those missing their sessions 
were invited to a virtual session being held to try to increase attendance rates. The 
virtual sessions were recorded. The same approach will be used for delayed 
education. 
 
6.8 Outcome Measures 
Primary effectiveness outcome: HF GDMT prescription increased (new class added, dose 
titrated upward on existing medication, or switch to ARNI from ACEi/ARB), within 30 days 
of index visit.  
 
Secondary effectiveness outcomes: 
• HF GDMT prescription increased (new class added, dose titrated upward on existing 
medication, or switch to ARNI from ACEi/ARB), within 60 and 90 days of index visit. 
• Addition of SGLT2i or ARNI (including switch from ACEi/ARB to ARNI), within 30, 
60, and 90 days of index visit.  This will be analyzed primarily using the entire eligible 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 18 of 26 
sample and secondarily only among those who have not already been prescribed SGLT2i 
or ARNI prior to the intervention. 
 
Other pre-specified effectiveness outcomes: 
• All-cause mortality or emergency visit or hospitalization for heart failure, within 365 
days of index visit. This will be assessed as patient death (yes/no), patient with 
emergency visit with a primary diagnosis of heart failure (yes/no), or admission for 
inpatient hospitalization with primary diagnosis of heart failure (yes/no). 
• Optimal GDMT dosing (at least 50% of target dose) on all classes (with or without 
ARNI), within 180 days of index visit. 
• Increased dosing or initiation for each individual class, within 180 days of index visit. 
• Decreased dosing or removal for each individual class, within 180 days of index visit. 
• Timing of increases in GDMT over 180 days since index visit. 
• Economic outcomes such as (1) return on investment (ROI) and (2) cost effectiveness 
(CE) of CDS and MTDM interventions compared with standard of care (using modeling 
approaches specified in the ROI and CE analysis section above). 
 
Implementation Outcomes 
• 
We used the RE-AIM framework to evaluate Reach, Adoption, Implementation and 
Maintenance for each interventional strategy (effectiveness is measured as noted above). 
A summary of metrics is in Table 3. 
Table 3 Implementation Outcomes 
Variable 
Reach 
Adoption 
Implementation 
Maintenance 
Focused 
Education  
% of providers 
completing education 
% of clinics 
receiving 
education 
• 
Feedback 
received from 
continuing 
education forms 
• 
Adaptations 
made from 
original 
education plan 
Change in 
GDMT over 
time post 
education vs. 
control 
CDS-OS 
• % of eligible 
patients where BPA 
action taken by 
provider 
• % of eligible 
patients where the 
BPA SmartSet was 
opened 
% of eligible 
clinicians 
participating in the 
CDS intervention 
Acknowledgement 
response assessment 
 
Changes in 
%BPA action 
taken by 
providers over 
time. 
CDS-MTDM  
• % of eligible 
patients where 
clinician referred to 
MTDM 
• % of eligible 
patients seen by 
MTDM 
% of eligible 
pharmacists 
participating in the 
MTDM 
intervention 
Fidelity to HFrEF 
protocol (e.g. % 
scheduled within 3 
days) 
 
Return on 
Investment of 
CDS-MTDM  

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 19 of 26 
 
7. STATISTICAL ANALYSIS PLAN  
7.1 Power analysis 
Data will be collected from 4,300 patients assigned to the clinicians in the study. With CDS-
OS and CDS-MTDM (versus control), we expect to have 80% power to detect approximately 
5 and 9 percentage-point increases, respectively, in prescribing GDMT within 30 days, at 
two-tailed p < .05 with robust standard errors clustered at the clinician level. With education 
versus delayed education, we expect to have 80% power to detect an approximately 5.5 
percentage-point increase in prescribing GDMT, with robust standard errors clustered at the 
clinician level. Power analyses were conducted using DeclareDesign.14 
 
7.2 Analysis details 
Analyses described below represent our planned analyses assuming statistical assumptions 
are met. If assumptions are not met, we will run alternative tests appropriate for the outcome 
distribution (e.g., non-parametric tests). 
 
The analyses will be conducted based on the principle of intention-to-treat (ITT). All patients 
who met the inclusion criteria will be included in the analyses in the arms to which they were 
assigned. Likewise, all clinicians will be included in analysis in their assigned education 
arms, regardless of whether they attended their education sessions. Because exposure to 
pharmacists is dependent upon action by a clinician to refer, we will also conduct a per 
protocol sensitivity analysis for patients who were consulted to pharmacist care and those 
patients seen by the pharmacist.   
 
Two-tailed p-values < 0.05 will be used to determine statistical significance in all analyses. 
 
7.2.1 Descriptive statistics 
Descriptive statistics will be computed for baseline socioeconomic and demographic 
characteristics (i.e., sex, age, race, ethnicity, insurance, etc.), including broken out across 
study arms, with differences compared using the appropriate statistical tests: two-tailed 
Student’s t-tests will be used for continuous data, column proportion z tests or χ2 for 
categorical data, and Mann-Whitney for non-normal quantitative data. 
 
7.2.2 Approach for comparisons across intervention arms 
CDS analyses: We will test whether each outcome differs as a function of each CDS active 
arm (CDS-OS and CDS-MTDM) relative to no-CDS control. Regressions will include as 
covariates education arm and interactions between CDS and education. We will retain both 
terms if the interaction is significant and retain the main effect if only it is significant. The 
education variable in CDS analyses will be coded as delayed education (baseline), early 
education, other (for clinicians who were not in the system early enough to be included in an 
education arm, to avoid either diluting or biasing the education effects).  
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 20 of 26 
As an exploratory analysis, we will compare CDS-MTDM against CDS-OS to see which is 
most effective.  
 
Additionally, covariates will be added to exploratory versions of the final models as 
sensitivity analyses, to adjust for measured differences across clinicians, such as role, number 
of patients seen, and baseline rates of prescribing each of the GDMT classes. We have no a 
priori hypotheses about the moderating effects of these covariates. 
 
Education analyses: We will test whether each outcome differs as a function of early 
education relative to delayed education. Regressions will include as covariates CDS arm and 
interactions between CDS and education. We will retain both terms if the interaction is 
significant and retain the main effect if only it is significant. Assignment to early versus 
delayed education is not random; therefore, regression analysis comparing education to no-
education controls will adjust for as many relevant covariates as possible, including baseline 
prescribing rates. 
 
In these analyses, we will exclude clinicians who were not present throughout the data 
collection period, beginning at assignment to early education. That is, we only intend to 
include clinicians in early and delayed education arms who theoretically could have been 
exposed to the education and who then had time to treat patients based on what they learned.  
 
7.2.3 Regression Specifications 
For all outcomes except timing of GDMT increases and ROI outcomes, ordinary least 
squares (OLS) regressions will be run15 with heteroskedasticity-robust standard errors 
clustered at the clinician level (to account for clustering of patients within clinicians). We 
will also explore the effect of clustering standard errors at the clinic level.  
 
We will run a survival analysis to assess timing of GDMT increases.  
 
Cost and outcomes estimate for the cost-effectiveness model will be obtained using EHR 
data, claims data, and literature-based sources. These data will be used to develop a decision 
tree analytical model with a state transition Markov cohort simulation to compute 
incremental cost effectiveness ratios (ICER), cost-effectiveness acceptance curves will be 
generated for all cost-effectiveness measurements. The effects of uncertainties from the 
model parameters will be evaluated by deterministic and stochastic sensitivity analyses. 
Following the World Health Organization’s guidelines,16,17 we will define the willingness-to-
pay (WTP) threshold as 1 to 3 times the local gross domestic product per capita.16,18 
Interventions with an ICER below the threshold will be considered cost-effective. We will 
also compare the total dollar amount spent on implementing this program against the cost 
savings, if any, attributable to the interventions (CDS-OS + CDS-MTDM) to calculate the 
corresponding ROI.  
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 21 of 26 
7.3 Variables 
We will explore demographic information, past medical history, clinical procedures, 
medications, encounters (ambulatory, ED, and hospital), and administrative claims (inclusive 
of costs of care) among the HF population. Examples of variables of interest include age, sex, 
BMI, comorbid conditions, medication use, healthcare utilization, and costs of care.  
 
7.4 Data Sources  
A data broker in Geisinger’s Phenomic Data Analytics and Clinical Data Core (PACDC) will 
extract EHR data, Geisinger Health Plan (GHP) medical and prescription claims, and patient 
reported data necessary for the study. Approved study personnel may need to consult 
individual patient medical records for manual chart review and abstraction if further 
investigation/validation of the data is needed. 
 
Protected Health Information (PHI) will be limited to medical record numbers (MRNs), 
dates, GHP insurance numbers, and department and clinician IDs. Approved study personnel 
conducting the manual chart reviews will be exposed to other elements of PHI, but these 
elements will not be collected as part of chart abstraction. In addition to PHI, we will also 
collect data including but not limited to the following: 
• Demographics (e.g., age, sex) 
• Vitals (e.g., body mass index, blood pressure) 
• Encounter and Problem list diagnosis 
• Patient-reported data 
• Health behaviors (e.g., smoking status) 
• Medications (ordered and dispensed) 
• Procedures (e.g., surgery) 
• Laboratory values 
• Utilization of health services (e.g., inpatient, outpatient, emergency room) 
• Cost of care  
• Social determinants of health data (when available) 
 
7.5 Data Management  
Members of the Geisinger approved study team will coordinate with the data broker to pull 
the necessary data for the study. Members of the Geisinger approved study team who have 
been trained in chart abstraction will conduct chart reviews as needed. Study data will only 
be directly accessible to Geisinger approved study team members and stored on password 
protected computers and/or locked in filing cabinets. Any data shared externally will be de-
identified and/or aggregated.  
 
7.6 Record Retention 
Records of data generated during the study may be retained indefinitely. 
 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 22 of 26 
8. PROTECTION OF HUMAN SUBJECTS 
Anticipated risk to participants will be minimal. This study will not affect patient care or 
access to care. While PHI will be requested, it will only be available to approved study team 
members to limit the risk of breaching confidentiality. All electronic study data will be kept 
in password-protected locations and any hard copy data will be locked in filing cabinets and 
only accessible to approved study team members on an as needed basis. Only group-level 
information without personal identifiers will be included when presenting results or 
submitting manuscripts for publication. We intend to permanently and securely archive a 
fully deidentified dataset at a research repository such as Open Science Framework (OSF) in 
order to be consistent with the best practices for open and reproducible science. 
 
8.1 Patient Information  
All parties will comply with all applicable laws, including laws regarding the implementation 
of organizational and technical measures to ensure protection of patient personal data. Such 
measures will include omitting patient names or other directly identifiable data in any 
reports, publications, or other disclosures, except where required by applicable laws.  
 
The personal data will be stored at the study site in electronic and/or paper form and will be 
password-protected and/or secured in a locked filing cabinet to ensure that only authorized 
study staff have access.  The study site will implement appropriate technical and 
organizational measures to ensure that the personal data can be recovered in the event of 
disaster. In the event of a potential personal data breach, the study site shall be responsible 
for determining whether a personal data breach has in fact occurred and, if so, providing 
breach notifications as required by law. 
 
9. PLANS FOR DISSEMINATING STUDY RESULTS 
We plan to disseminate our findings as abstracts and presentations at national meeting(s) and 
in peer-reviewed journal(s). 
10. PROJECT STATUS 
Early education was launched on 8/1/2023 and data collection is ongoing. CDS and MTDM 
arms launched on 10/9/2023. Outcome data have not yet been extracted or examined as a 
function of study arms, except in a very limited way for the initial BPA investigation 
described below.  
  
We removed non-telemedicine telephonic encounters as an inclusion criterion partway into 
data collection (as of 11/4/2023), based on immediate post-launch investigations of BPAs 
that were intended to help ensure BPAs were functioning correctly and firing appropriately in 
terms of patients, clinicians, and encounters. These investigations included chart reviews to 
examine acknowledgement reasons and prescribing patterns in the referral and order set 
arms. The results suggested that telephonic encounters were inappropriate occasions in which 
to expect clinicians to make prescription adjustments, given other pressing reasons for a 
majority of these calls (e.g., review of lab results, scheduling appointments) and very low 
rates of prescribing during these encounters. Prescribing rates were not compared between 
the no-BPA control arm and the CDS arms, and no other outcomes were examined. 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 23 of 26 
Telemedicine telephonic encounters remain a valid inclusion criterion, as those are scheduled 
appointments appropriate for prescribing.  
  
Within the referral arm only, data on referral rates across clinicians were monitored and chart 
reviews conducted to better understand where and why referrals to MTDM were not 
occurring for certain clinicians. These data were used to inform follow-up by MTDM 
pharmacists and potential escalation by clinical leadership to overcome lack of awareness 
and educate clinicians on the purpose and appropriate use of the referral. For example, if 
referral rate to MTDM for a given clinician was below 50% in February 2024, the study PI 
contacted the clinicians to educate or remind them about the benefits of pharmacy co-
management. 
 
Delayed education sessions began in early August 2024. This was anticipated to occur after 
final data were collected, but enrollment was still incomplete at the time. We will limit our 
primary education analysis so it only includes encounters before the delayed education 
sessions began (i.e. the period where the delayed education group had not yet experienced 
education), so the delayed education encounters can be used as controls. However, follow-up 
exploratory analyses will include encounters after delayed education so we can examine main 
effects of delayed education, or interactions between delayed education and ongoing CDS 
interventions. 
 
 
11. REFERENCES 
1. Heidenreich PA, Bozkurt B, Aguilar D, et al. 2022 AHA/ACC/HFSA guideline for the 
management of heart failure: A report of the american college of cardiology/american heart 
association joint committee on clinical practice guidelines. Journal of the American College 
of Cardiology. 2022;79(17):e263-e421. https://www.ncbi.nlm.nih.gov/pubmed/35379503. 
doi: 10.1016/j.jacc.2021.12.012. 
2. McMurray JJV, Packer M, Desai AS, et al. Angiotensin–Neprilysin inhibition versus 
enalapril in heart failure. The New England journal of medicine. 2014;371(11):993-1004. 
https://nejm.org/doi/full/10.1056/NEJMoa1409077. doi: 10.1056/NEJMoa1409077. 
3. McMurray JJV, Solomon SD, Inzucchi SE, et al. Dapagliflozin in patients with heart 
failure and reduced ejection fraction. The New England journal of medicine. 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 24 of 26 
2019;381(21):1995-2008. https://nejm.org/doi/full/10.1056/NEJMoa1911303. doi: 
10.1056/NEJMoa1911303. 
4. Packer M, Anker SD, Butler J, et al. Cardiovascular and renal outcomes with 
empagliflozin in heart failure. The New England journal of medicine. 2020;383(15):1413-
1424. https://nejm.org/doi/full/10.1056/NEJMoa2022190. doi: 10.1056/NEJMoa2022190. 
5. Graham J. Personal communication. prep to research. 2022. 
6. Jing L, Ulloa Cerna AE, Good CW, et al. A machine learning approach to management of 
Heart Failure populations. JACC. Heart failure. 2020;8(7):578-587. 
https://dx.doi.org/10.1016/j.jchf.2020.01.012. doi: 10.1016/j.jchf.2020.01.012. 
7. Paterson K, Nguyen M, Pradhan A, Lussier M, Wright E, Voyce S. Understanding barriers 
and facilitators to prescribing sodium-glucose cotransporter inhibitors (SGLT2i) in patients 
with heart failure with reduced ejection fraction (HFrEF) . ACC Submitted Abstract. 2022. 
8. Tavares S. Personal communication. 
9. Maddox TM. Clinical decision support in statin Prescription—What we can learn from A 
negative outcome. JAMA Cardiology. 2020;6(1):48-49. 
http://dx.doi.org/10.1001/jamacardio.2020.4756. doi: 10.1001/jamacardio.2020.4756. 
10. Ghazi L, Yamamoto Y, Riello RJ, et al. Electronic alerts to improve Heart Failure 
therapy in outpatient practice: A Cluster Randomized trial. Journal of the American College 
of Cardiology. 2022;79(22):2203-2213. https://www.ncbi.nlm.nih.gov/pubmed/35385798. 
doi: 10.1016/j.jacc.2022.03.338. 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 25 of 26 
11. Mukhopadhyay A, Reynolds HR, Phillips LM, et al. Cluster-randomized trial comparing 
ambulatory decision support tools to improve Heart Failure care. Journal of the American 
College of Cardiology. 2023;81(14):1303-1316. 
https://dx.doi.org/10.1016/j.jacc.2023.02.005. doi: 10.1016/j.jacc.2023.02.005. 
12. Adusumalli S, Aragam G, Patel M. A nudge towards cardiovascular health: Applications 
of behavioral economics for primary and secondary cardiovascular prevention. Curr Treat 
Options Cardio Med. 2020;22(9). https://link.springer.com/article/10.1007/s11936-020-
00824-y. doi: 10.1007/s11936-020-00824-y. 
13. Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. The 
New England journal of medicine. 2018;378(3):214-216. 
https://nejm.org/doi/full/10.1056/NEJMp1712984. doi: 10.1056/NEJMp1712984. 
14. Blair G, Coppock A, Humphreys M. Research Design in the Social Sciences: 
Declaration, Diagnosis, and Redesign. 2023. Princeton University Press.  
15. Gomila R. Logistic or linear? estimating causal effects of experimental treatments on 
binary outcomes using regression analysis. Journal of experimental psychology. General. 
2021;150(4):700-709. https://www.ncbi.nlm.nih.gov/pubmed/32969684. doi: 
10.1037/xge0000920. 
16. Yu DS, Li PW, Li SX, Smith RD, Yue SC, Yan BPY. Effectiveness and cost-
effectiveness of an empowerment-based self-care education program on health outcomes 
among patients with heart failure: A randomized clinical trial. JAMA network open. 

Geisinger – GREAT-HF Care Study Protocol 
Version 1, September 4, 2024 
Page 26 of 26 
2022;5(4):e225982. https://www.ncbi.nlm.nih.gov/pubmed/35380643. doi: 
10.1001/jamanetworkopen.2022.5982. 
17. TAN-TORRES EDEJER T, BALTUSSEN R, ADAM T, et al. Making choices in health : 
WHO guide to cost effectiveness analysis. Genève: OMS; 2003. 
18. The World Bank. GDP per capta (current US$) – hong kong SAR, china. 
https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?locations=HK Web site. Accessed 
January 9, 2024.