Pragmatic Trial of Messaging to Providers About Treatment of Heart Failure
Phase: NA
Conditions: Heart Failure With Reduced Ejection Fraction
Sponsor: Yale University

HIC#2000027014 
1 
 
 
 
 
 
 
 
 
Pragmatic Trial of Messaging to Providers about Treatment of Heart Failure (PROMPT HF) 
NCT 04514458 
Protocol and Statistical Analysis Plan 
Document Date: 1/31/2022 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

HIC#2000027014 
2 
 
 
 
 
                    
                                    
HRP-503B – BIOMEDICAL RESEARCH PROTOCOL (2020) 
 
Protocol Title: PRagmatic Trial Of Messaging to Providers about Treatment of Heart Failure 
(PROMPT-HF) 
 
Principal Investigator: Tariq Ahmad, MD, MPH, Yale University School of Medicine, Yale 
New Haven Health System 
Co-Investigators: Nihar Desai, MD, MPH, Ralph Riello, PharmD; F. Perry Wilson, MD, MHS, 
Yale University School of Medicine, Yale New Haven Health System 
 
Version Date: 01/31/2022 
 
Clinicaltrials.gov Registration #: NCT04514458 
 
SECTION I: RESEARCH PLAN 
 
Statement of Purpose: Heart failure (HF) is the major cause of healthcare expenditure, 
morbidity, and mortality in the United States.1 Data from clinical trials suggest that 
pharmacological therapies prescribed at appropriate doses will lead to dramatic improvements 
in survival and hospitalization rates in patients with heart failure with reduced ejection fraction 
(HFrEF). Consequently, major cardiovascular societies assign the highest level of 
recommendation to use these therapies in all eligible patients.2 However, data from several 
registries over the last three decades has failed to see use of these evidence based therapies 
at levels noted in clinical trials, despite aggressive guideline recommendations and promotion 
by thought leaders in the field.3-5 In fact, our pilot data across the Yale New Haven Health 
System shows almost identical use of evidence based heart failure therapies as noted in the 
most current and comprehensive registry—CHAMP-HF.  
 

HIC#2000027014 
3 
 
It remains unclear as to why many patients with HFrEF are not on evidence-based therapies, 
and why the percentages are consistent 
across national registries over time. One 
explanation might be that providers know 
the data regarding evidence-based 
therapies, but the therapies only benefit a 
narrow population. Another factor might be 
a lack of knowledge among providers about 
the appropriate management of HFrEF 
patients. A simple way to test this 
hypothesis is to examine whether electronic 
health record (EHR) based "best practice 
advisories" (BPAs) can increase use of 
evidence based therapies.6 If found to be 
effective, these low cost interventions can 
be rapidly applied across large healthcare 
systems.7  
We therefore propose a randomized controlled trial across outpatient clinics at four teaching 
hospitals within the Yale New Haven Health System comparing the  effectiveness of an EHR 
BPA system that informs practitioners (Physicians, Nurse Practitioners (NPs), Physician 
Assistants (PAs)) about what medications they can prescribe that are evidence based for 
HFrEF versus usual care (no alert). The primary outcome for the trial will be the proportion of 
patients with HFrEF with an increase in evidence based medical therapies for HFrEF(beta-
blockers, ACE-I/ARB/ARNI, MRA, SGLT2i (currently at 10%)). Secondary outcomes will 
include the proportion of patients prescribed each class of evidence-based medication, the 
percent of filled prescriptions as assessed by Sure Scripts, medication doses, 30-day hospital 
admission rates, 30-day all-cause ED visits, one year all-cause mortality, and 6 month total 
healthcare cost.  We will also collect other data available in the EHR that may pertain to 
potential side-effects of the therapies.    
 
Probable Duration of Project: Approximately 3000 unique patient encounters with a 
diagnosis of HFrEF are generated annually at the proposed sites, with a planned enrollment of 
100 unique providers and 1310 patients to start soon after IRB approval, we conservatively 
estimate that the study will take 2 years to complete.  
 
Background: Heart failure is a complex and heterogenous disease with mortality and 
morbidity that rivals most cancers.1, 8 There is robust data from clinical trials in support of the 
premise that pharmacological therapies prescribed at appropriate doses lead to dramatic 
decreases in mortality and hospitalization rates in patients with HFrEF. As a result, all major 
cardiovascular societies give the highest level of recommendation to use of these therapies in 
eligible patients.2 However, data from registries and real world analyses shows that use of 
evidence based therapies falls far short of what is seen in clinical trials. This might be due to 
the lack of applicability of these practices in the real-world setting. Whether the gap between 
clinical trial use and real-world practice is due to a lack of knowledge or providers making 
individualized decisions about their patients is unclear. It is for this reason that we plan to 

HIC#2000027014 
4 
 
perform the PROMPT-HF Trial across outpatient clinics at four teaching hospitals within the 
Yale New Haven Health System.  
 
Research Plan: We plan to conduct a randomized, single-blind intervention trial to test the 
comparative effectiveness of an EHR BPA system that informs practitioners (physicians, NPs, 
and PAs) about what medications they can prescribe that are evidence based and FDA 
approved for HFrEF versus usual care (no alert and how things are done currently). This will 
be done across outpatient internal medicine and cardiology clinics of the four teaching 
hospitals within the Yale New Haven Health System, including Yale New Haven Hospital, St. 
Raphael’s Campus, Greenwich Hospital, and Bridgeport Hospital. 
 
Study Population: One hundred outpatient cardiology and internal medicine providers (to 
include physicians, NPs, and PAs) practicing at either Yale New Haven Hospital, St. Raphael’s 
Campus, Greenwich Hospital, or Bridgeport Hospital will be enrolled and undergo 
randomization to either an intervention (alert) group or a control (usual care) group. Those in 
the intervention group will receive an informational alert for their eligible adult outpatients.  
Those in the control group will not receive any alerts and will continue to care for patients as 
usual. 
Adult outpatients of randomized providers within internal medicine and cardiology clinics at one 
of the eligible study sites who have HFrEF with a left ventricular ejection fraction of less than or 
equal to 40% and are registered in the Yale Heart Failure Registry (NCT04237701) will be 
automatically enrolled into the study.  Patient subjects will either be placed into the alert or 
usual care group based on the randomization status of their provider at the time of the clinic 
visit in which they meet eligibility criteria. 
 
Our study flow is the following: 
 

HIC#2000027014 
5 
 
 
Subject Recruitment 
One hundred outpatient providers (to include physicians, PAs, and NPs) of Cardiology and 
Internal Medicine at the four teaching hospitals within the Yale health system will be selected 
for potential participation in the study.  Providers will be selected based on the frequency with 
which they see patients who meet the inclusion criteria (see below), as identified by 
retrospective review of outpatient clinical encounters within Epic. Upon identification, all 
providers will be approached for consent to participate in the study.   
 
Eligible patient subjects will be identified through our best practice alert build within Epic using 
similar methodology that has been used in other ongoing studies by the PIs. Upon opening the 
patient’s medical chart and placing orders, the build will examine the patient record, and all 
patients who meet inclusion and exclusion criteria will automatically be identified and enrolled 
into the study. The inclusion criteria will be all adults ≥18 years who are seen in an internal 
medicine or cardiology clinic of one of our study sites and who have a left ventricular ejection 
fraction of ≤40% and membership in the Yale Heart Failure Registry (NCT04237701). Our 
target sample size is 655 subjects per study arm, for a total of 1310 patients. A patient’s study 
arm (whether a patient generates an alert or not) will be determined by the randomization 
status of the provider at the time of their first eligible outpatient visit. 
 
As this is an intervention assessing the influence of an alert system on provider behaviors 
surrounding heart failure prognosis, we cannot inform patients of their participation in the study 
at the time of enrollment, as this would contaminate the randomized exposure. As this study 
presents minimal risk to patients, we will be requesting a waiver of informed consent at the 
patient level. 
 
Randomization 
Randomization in this study will occur at the level of the provider. All consented physicians will 
be randomized within Epic to either the intervention group or the control (usual care) group via 
a permutated block randomization scheme to ensure an equal number of providers in each 
study arm. Logic checks within the alerts ensure that once a provider is assigned to an arm, 
they remain in that arm for the remainder of the study. This will create 100 clusters (providers) 
to which eligible patient participants will be assigned upon their outpatient visit.   
 
Because it is likely that providers may internalize the best practice metrics once repeatedly 
exposed to the alert, cluster randomization of patients at the level of the provider will ensure 
that providers consistently receive an alert or not, which will reduce contamination between 
study arms. It will also minimize the chance that providers get so accustomed to seeing an 
alert that they will become less attentive to the condition in patients not generating an alert.  
We will ask the providers to not discuss the details of the trial and/or alerts amongst each other 
so as to limit the contamination across study groups.  Additional pre- and post- intervention 
analyses in the control group, leveraging the three-month baseline period, can assess the level 
of contamination and the effect of secular trends towards better care over time.  
 
Intervention 

HIC#2000027014 
6 
 
Providers randomized to the intervention arm will have a best practice alert appear for each of 
their eligible patients upon opening of the order entry screen in the patient’s medical record.  
This alert informs the provider to the presence of HFrEF, notes the patient’s current LVEF and 
current evidence based medications, and gives access to an order set with recommended 
evidence- based and FDA- approved therapies for patients with HfrEF. Providers will also have 
access to a link to best available guideline recommended information regarding treatment of 
heart failure. With the alerts, we will also make sure to note that monitoring and follow-up are 
required for prescriptions. 
 
 
The alert will appear as follows: 
 
 
                              
Below is the order set that will be linked in the alert.  It contains evidence-based therapy for 
HfrEF along with each FDA indication9: 

HIC#2000027014 
7 
 
 

HIC#2000027014 
8 
 
Alerts will fire only once per provider, during the first encounter in which a patient is eligible 
and only when the chart is open to enter orders. This ensures that the alert is displayed at the 
most relevant places in a provider’s workflow and will minimize alert burden.  Once a patient is 
seen by an enrolled provider, they will not generate further alerts, which will ensure that a 
patient who sees two providers in different arms of the study will not be enrolled twice. 
 
Should a patient have a documented allergy or contraindication to a particular medication 
class, this class will not be displayed in the order set for that patient.  We will also provide the 
clinician with the opportunity to dismiss the alert and indicate the reason, including if the 
patient had any contraindications.   Any patient seen by a provider that is randomized to the 
control study arm will not generate an alert for their provider, but a “silent alert” that registers 
the patient into the study.  Patients will receive care as usual. 
 
Endpoints: The primary outcome for the trial will be the proportion of patients with HfrEF who  
have an increase in the number of prescribed  evidence-based therapies for HfrEF (beta-
blockers, ACE-I/ARB/ARNI, MRA, SGLT2i) 30 days post randomization . This endpoint will be 
based on the prescription of medication rather than filling of the prescription by the patient. 
 
Secondary outcomes will include: percentage of patients prescribed each medication class (i.e. 
percentage on beta-blockers, percentage on ACE-I/ARB/ARNIs, percentage on SGLT2i, and 
percentage on MRAs), the percent of filled prescriptions (as assessed by Sure Scripts), 
medication doses, one-year all-cause mortality, 30-day all-cause hospital admission rate, all-
cause 30-day ED visits, and 6 month total healthcare costs.Other data that is present in the 
EHR that might be related to adverse events such as falls will also be collected. Outcomes will 
be determined by review of medical, hospital, and billing records. 
 
Secondary analyses will look at outcomes based on insurance coverage, specialty of the 
provider, and according to prescriptions filled by patients.  We will also stratify analyses by 
provider type (MD vs PA vs NP).  Finally, per request of our DSMB, we will collect laboratory 
values and vital signs as categorical variables to enable simpler detection of safety signals as 
the data accrues, as well as secondary endpoints at 60 days (to include ED visits, 
hospitalizations, and all-cause mortality). 
 
Engaging Providers: We will approach all eligible providers for consent, during which we will 
inform them of the nature of the study and review evidence-based guidelines for the treatment 
of HFrEF. We will also perform periodic outreach to all consented clinicians to address any 
concerns or questions that may arise throughout the study. We will additionally inform them 
that limited data is being collected regarding provider behavior. However, we will also make it 
clear that this data will not be linked to individual clinicians or outcomes. All such data will only 
be analyzed in aggregate. 
 
Additionally, we will collect provider education and experience metrics.  We will assess the 
percentage of providers who visit the guideline site via the link provided within the alert and 
who open the order set within the alert. Further, all providers will be asked to complete both a 
pre- and post- study survey that will assess knowledge and comfort levels with the evidence-
based guidelines for heart failure treatment. The post survey will also include questions that 

HIC#2000027014 
9 
 
will allow us to assess provider approval and overall acceptance of the alert, assessing their 
opinions on user friendliness, usefulness versus disruptiveness, and overall user experience.  
Analysis of responses will allow us to assess which aspects of the alert were most helpful and 
which could be improved in possible future iterations of this alert and others. 
 
Consent and Survey distribution: Provide consent will be collected in one of several ways, and 
we are requesting a waiver of documentation of consent for this study as the study poses 
minimal risk with no collection of PHI for providers.  The first method for provider outreach and 
consent will be done through email in which a link to the consent and provider survey will be 
provided.  In this method, the consent and survey will be administered electronically via 
REDCap (Research Electronic Data Capture) and an electronic signature will be captured. 
This link of the consent will be sent to all eligible providers that will direct them to our online 
consent form in REDCap for review and signature. If the provider consents, we will follow up 
and send a survey link that will direct them to the survey form in REDCap. 
 
Should provider outreach via email prove difficult, we will use other means to contact eligible 
clinicians.  The first will be via an Epic in basket message, where the informed consent will be 
copied directly into the body of an Epic message that is then sent to the eligible provider.  The 
provider can then read through the consent and reply back with “Yes I do not consent” or “No I 
do not consent”, without requiring a signature.  This response will be documented, and 
consented providers will then be sent instructions and a link for the REDCap survey. 
 
We will also content via telephone.  Study coordinators will contact eligible providers via phone 
to describe the study, answer questions, and receive and document verbal consent.  
Consented providers will then be sent instructions and a link for the REDCap survey. 
 
If/when feasible, study coordinators will also conduct in-person signed consent. 
 
Data for this study (consent and survey only) will be collected, recorded and stored using 
REDCap. REDCap is a secure, web application designed to support data capture for research 
studies. It includes features for HIPAA compliance including real-time data entry validation 
(e.g. for data types and range checks), a full audit trail, user-based privileges, de-identified 
data export mechanism to statistical packages (SPSS, SAS, Stata and R), and integration with 
the institutional Active Directory. Access to study data in REDCap will be restricted to the 
members of the study team with authentication through University NetID credentials. The 
REDCap@Yale database and web server are housed on secure platforms that are backed up 
daily. REDCap@Yale meets the security standards for use with high risk data as set forth by 
the Yale Information Security Office.Electronic data will be kept in password-protected files 
located within REDCap™. Access to the study's data in REDCap™ will be restricted to the 
members of the study team by username and password. 
Prior and Concomitant Therapy: All therapies are permissible within this protocol.   
 
Blinding of Intervention: Patients will not be informed of their randomization status or 
participation in this trial as the trial could not be feasibly performed if subjects were told they 
were enrolled. Provider subjects will, obviously, not be blinded to the intervention as they are 
receiving the alert and will be consenting to participate in the study. We will engage in both 

HIC#2000027014 
10 
 
pre-trial and periodic teaching and discussion with all participating care providers to inform 
clinicians about the nature of the study and to discuss specific factors that are being 
measured. The study team will be blinded to the treatment assignment until the end of the trial 
period. 
 
Intervention Duration: The prognosis alert will be displayed to the relevant provider upon 
opening the order entry screen in the patient’s chart.  The alert will only be displayed once per 
provider, at the patient’s first eligible outpatient visit, such that it will appear at the most 
relevant place in a provider’s workflow and also minimize alert burden.  
 
 
Genetic Testing    N/A ☒ No biological material will be collected on patients.  
 
1. Subject Population: We will aim to recruit and consent 100 providers (MDs, PA, and NPs) 
who practice in cardiology and internal medicine outpatient clinics at Yale New Haven 
Hospital, St. Raphael’s Campus, Bridgeport Hospital and Greenwich Hospital. Providers will 
be selected via retrospective review of outpatient records within Epic and chosen based on the 
frequency with which they see eligible patients. There will be a target sample size of 
approximately 1310 patients.   Subjects who will be enrolled are those with a diagnosis of 
HFrEF (LVEF ≤ 40%) who are seen in an outpatient internal medicine or cardiology clinic 
within the four teaching hospitals of the YNHS.All patients will be in the ongoing Yale Heart 
Failure Registry (NCT04237701).  
 
2. Subject classification: N/A 
☐Children 
 
 
☐ Healthy  
 
 
☐Fetal material, placenta, or 
dead fetus 
☐Non-English Speaking 
 
☐ Prisoners  
 
☐Economically 
disadvantaged persons 
☐ Decisionally Impaired 
 
☐ Employees 
 
 
☐Pregnant women 
and/or fetuses 
☐Yale Students  
 
 
☐ Females of childbearing potential 
 
NOTE: Is this research proposal designed to enroll children who are wards of the state 
as potential subjects? 
No ☒  
 
Inclusion/Exclusion Criteria:  
For patient subjects: 
 Age>18 Years 
 Seen in internal medicine or cardiology clinic 
 Left Ventricular Ejection Fraction ≤40% 
 In the Yale Heart Failure Registry (NCT04237701) 
 
 
Exclusion Criteria: 
 Opted out of EHR-based research 
 Hospice care 

HIC#2000027014 
11 
 
 Already receiving each of the four classes of evidence-based HFrEF medical 
therapy 
 
Providers must be practicing at an outpatient cardiology and internal medicine practices within 
one of the four teaching hospitals within the Yale New Haven Health System and will be 
selected for study participation based on the frequency with which they see patients who meet 
the above criteria, based on a retrospective chart review. 
 
How will eligibility be determined, and by whom?  
Eligibility of patients will be assessed automatically by the best practice alert framework 
directly within Epic and will not involve any human intervention. When a provider opens the 
order entry screen of the patient’s medical record, this build will examine the record for all 
inclusion and exclusion criteria. Those who meet criteria will be immediately and automatically 
enrolled and placed into a randomization group based on the randomization status of their 
current provider at the time of enrollment.  Providers will be selected as described above via 
retrospective chart review by study investigators. 
 
 
 
Protection of Human Subjects: Protection of Human Subjects: Modifications Made in 
Response to Findings from the Electronic Alerts for the recently completed Acute Kidney Injury 
Amelioration (ELAIA-1) trial: The recently completed ELAIA-1 trial was negative. Alerts about 
acute kidney injury (AKI) did not reduce progression, dialysis initiation or death rates among 
hospitalized patients with AKI. However, there was a small signal for harm at one non-teaching 
hospital (Lawrence and Memorial Hospital). Despite heart failure being an entirely different 
disease state and the intervention being tested in PROMPT-HF is guideline based whereas 
ELAIA was not, we are making the following modifications to the study out of an abundance of 
caution. These have been made in partnership with the leadership at both Yale School of 
Medicine/Yale Center for Clinical Investigation (YCCI) and Yale New Haven Hospital: Drs. 
Brian Smith (YCCI), Teisha Johnson (YCCI), Allen Hsiao (YNHS), and Nitu Kashyap (YNHS) 
who believe that studies of decision support tools continue to pose minimal risk to patients. 
The changes are as follows: 
 We will restrict the study only to teaching hospitals within the YNHS: Yale New Haven, 
Greenwich, Bridgeport, Hospital of Saint Raphael 
 Our Data Safety Monitoring committee will perform an interim analysis of the data at 
25% and50% enrollment   
 
Human subjects’ involvement, characteristics and design: The studies outlined in this proposal 
depend on the enrollment of individuals with heart failure. No vulnerable populations are being 
specifically targeted. We are limiting enrollment to individuals above age 18 years as the 
etiology and practices surrounding heart failure in pediatrics populations differ significantly 
from those in adults. All data is transmitted in encrypted and secure fashion, stored on servers 
with "triple-lock" certification, and is available only to members of the study team, IRB, and any 
state or federal agencies with auditing power. 
Sources of Materials: No biological materials will be obtained or stored as part of these 
studies.  Only data, as collected during routine medical care will be obtained. Data includes 

HIC#2000027014 
12 
 
medical record elements such as demographics, medication usage and dosing, laboratory 
values, and administrative codes. All data will be stored in the absence of PHI, though we will 
retain a linking dataset to be able to re-link individual data to actual patients for future studies 
(for example, studies examining longer term outcomes of patients, which may require linking to 
national datasets). Access to individually identifiable information will be limited to the executive 
committee of the study, and only then via a linking file as aforementioned. All data used for 
analysis and dissemination to other investigators will be de-identified.   
 
Over or under treatment: If alerts affect physician behavior, then patients randomized to an 
alert arm may be more likely to be started on evidence based medical therapies. These 
interventions fall within the standard-of-care and may benefit patients, but it is also possible 
that additional interventions may not benefit patients and could incur additional costs. 
However, this is what we are testing as part of this pragmatic trial.  
 
Alert Fatigue: We have modified the protocol in response to findings from the ELAIA-1 trial and 
ongoing REVEAL-HF study at Yale. The following are the interventions we have made: 
A. Alert is both sensitive and specific to heart failure 
B. Alert will only include patients in the Yale Heart Failure Registry (NCT04237701) 
C. Alert only fires when clinician enters chart to enter an order  
Even with these changes, it is possible that alert fatigue might result. While a potential risk, this 
is also a major motivation of this line of research, as only via randomized trials can truly 
effective alerts be discovered. Should no effect be found in these studies, alerts will not be 
continued at the institution. Also, this will be the only alert for heart failure patients in the 
outpatient setting, minimizing risk of fatigue.  
The alert will be removed after all patients have been enrolled.  
 
Adequacy of Protection Against Risks: The study is being conducted under a waiver of 
informed consent. This research is minimal risk, as outlined above, and cannot be practicably 
performed in the absence of a waiver of informed consent as it would terminally contaminate 
the study. There is no infringement of rights or welfare of patients, as the alert has no direct 
effect on the patient, short of via the actions of their providers. Data abstracted from the 
medical record will be de-identified, as described in the confidentiality section above. Another 
defense against these risks is the alert language itself. Each alert calls attention to the fact that 
alerts are a part of a research study and the same alert will not fire on every patient.  
 
Potential benefits of the proposed research to the subjects and others: Subjects in this study 
may benefit from their provider being given information on how to improve the use of evidence-
based therapies in heart failure. Regardless of the outcome for participants, the results of 
these studies may lead to significant societal benefit, as even a negative study would lead to 
less enthusiastic adoption of ineffective alerting. The risk/benefit ratio, given the minimal risk to 
study subjects, is more than acceptable in this series of studies. 
 
 
 
Data and Safety Monitoring Plan:  

HIC#2000027014 
13 
 
 What is the investigator’s assessment of the overall risk level for subjects participating 
in this study? We believe that this poses minimal risk to the patients 
 
 If children are involved, what is the investigator’s assessment of the overall risk level for 
the children participating in this study? No children will be involved. 
 
Include an appropriate Data and Safety Monitoring Plan. Examples of DSMPs are   
available here http://your.yale.edu/policies-procedures/forms/420-fr-01-data-
and-safety-monitoring-plans-templates for 
i. Minimal risk 
ii. Greater than minimal 
The principal investigators (PI) is responsible for monitoring the data, assuring protocol 
compliance, and conducting the safety reviews at the specified frequency regularly. During the 
review process the PIs will evaluate whether the study should continue unchanged, require 
modification/amendment, or close to enrollment. The PIs or the Institutional Review Board 
(IRB) has the authority to stop or suspend the study or require modifications.  
 
Despite this being a minimal risk study, we reviewed results of the ELAIA-1 trial study that was 
recently completed that showed a small signal of harm at a non-teaching hospital within the 
YNHS. We realize that heart failure is an entirely different disease state and the intervention 
being tested in PROMPT-HF is strictly guideline based. Out of an abundance of caution, we 
decided to include an independent DSMB that will review the study results at 25% and 50% 
enrollment. These have been made in partnership with the leadership at both Yale School of 
Medicine/Yale Center for Clinical Investigation (YCCI) and Yale New Haven Hospital: Drs. 
Brian Smith (YCCI), Teisha Johnson (YCCI), Allen Hsaio (YNHS), and Nitu Kashyap (YNHS) 
who believe that studies of decision support tools continue to pose minimal risk to patients. Of 
note, we are also restricting the study only to teaching hospitals within the YNHS: Yale New 
Haven, Greenwich, Bridgeport, and Saint Raphael. Within the alerts, we will also make sure to 
note that monitoring and follow-up are required for these prescriptions. 
 
This protocol presents minimal risks to the subjects and Unanticipated Problems Involving 
Risks to Subjects or Others (UPIRSOs), including adverse events, are not anticipated. In the 
unlikely event that such events occur, Reportable Events (which are events that are serious or 
life-threatening and unanticipated (or anticipated but occurring with a greater frequency than 
expected) and possibly, probably, or definitely related) or Unanticipated Problems Involving 
Risks to Subjects or Others that may require a temporary or permanent interruption of study 
activities will be reported immediately (if possible), followed by a written report within 5 
calendar days of the Principal Investigator becoming aware of the event to the IRB (using the 
appropriate forms from the website) and any appropriate funding and regulatory agencies. The 
investigator will apprise fellow investigators and study personnel of all UPIRSOs and adverse 
events that occur during the conduct of this research project through regular study meetings 
and via email as they are reviewed by the principal investigator. 
d. For multi-site studies for which the Yale PI serves as the lead investigator: this is a 
single-center study 
i. How will adverse events and unanticipated problems involving risks to subjects or 
others be reported, reviewed and managed? Write here 

HIC#2000027014 
14 
 
ii. What provisions are in place for management of interim results? Write here 
iii. What will the multi-site process be for protocol modifications? Write here 
 
Statistical Considerations 
Sample Size Determination:  
The primary outcome is the percentage of HFrEF patients on a higher number of evidence-
based therapies at 30-days after randomization.  Our preliminary data suggests that the baseline 
rate for increasing evidence-based therapy use after a visit is up to 20%. A clinically meaningful 
increase from that baseline would be 10%. In a fixed design with no interim analysis, sample 
sizes of 650 in group 1 and 650 in group 2, which were obtained by sampling 50 clusters with 
13 subjects each in group 1 and 50 clusters with 13 subjects each in group 2, achieve 91.207% 
power to detect a difference between the group proportions of 0.1. After accounting for the 2 
interim analyses, the final sample size is 1310.  The proportion in group 1 (the treatment group) 
is assumed to be 0.2 under the null hypothesis and 0.3 under the alternative hypothesis. The 
proportion in group 2 (the control group) is 0.2. The test statistic used is the two-sided Z-Test 
(Unpooled). The intracluster correlation is a conservative 0.05, and the significance level of the 
test is 0.048. 
 
 
Interim Analysis:  
We plan to have interim analyses when 25% (n=328), and 50% (n=655)of patients have been 
enrolled.  The interim analyses will allow us to alter the sample size or stop the trial earlier for 
ethical considerations, unexpected adverse events, or high efficacy. Earlier stopping will be 
considered for the reason of safety and efficacy. This is a novel intervention and we are 
obligated to ensure the safety of the alert and that it is minimal risk. The trial will stop for 
declaring efficacy if the effect size is large.  We will use the O’Brien and Fleming stopping rule 
to stop the trial earlier if the P value for the primary outcome is 0.00007 for efficacy at 25% 
enrollment, and 0.005 at 50% enrollment.  
 
 
Statistical Analysis:  
The primary analysis will utilize the intention to treat principle. A generalized linear model will be 
used to assess the relationship between randomization status and the primary outcome.  The 
model will use a log-link and independent correlation structure to allow for calculation of adjusted 
risk ratios.   Statistical significance will be based on a P value of <0.048 to account for two interim 
analyses.  For categorical secondary outcomes, a similar generalized linear modelling approach 
will be used.  For continuous secondary outcomes, a generalized linear model will be used with 
an identity-link to describe adjusted absolute differences between outcomes. 
 
Assessment of contamination: The cluster-randomized nature should minimize contamination 
across the study arms. Providers may “learn” to treat heart failure as consistent with evidence-
based therapies over time – this will be assessed in a supplementary analysis. There is the 
possibility that a provider randomized to the intervention group may share knowledge with one 
randomized to the control group. We will discourage providers from discussing the alerts among 
themselves until the study is over. Furthermore, among providers assigned to the control group, 

HIC#2000027014 
15 
 
we will identify the subset who share office space with a provider assigned to the intervention 
group, allowing a direct assessment of contamination. 
 
 
SECTION II: RESEARCH INVOLVING DRUGS, BIOLOGICS, RADIOTRACERS, PLACEBOS AND DEVICES 
 
 
If this section (or one of its parts, A or B) is not applicable, check off N/A and delete the rest of 
the section. 
 
A.  RADIOTRACERS 
☒N/A 
 
B.  DRUGS/BIOLOGICS    ☒N/A 
 
C.  DEVICES ☒N/A 
 
 
SECTION III: RECRUITMENT/CONSENT AND ASSENT PROCEDURES  
1. Targeted Enrollment: Give the number of subjects:  
a. Targeted for enrollment at Yale for this protocol: 100 providers; 1,310 participants 
b. If this is a multi-site study, give the total number of subjects targeted across all sites: NA 
 
2. Indicate recruitment methods below.  Attach copies of any recruitment materials that will 
be used. 
☐ Flyers 
☐ Internet/web postings 
☐ Radio 
☐ Posters 
☒ Mass email solicitation (providers 
only)
☒ Telephone 
(providers only)
☐ Letter 
☐ Departmental/Center website 
☐ Television 
☒ Medical record review*  
☐ Departmental/Center research 
boards
☐ Newspaper 
☐ Departmental/Center 
newsletters
☐ Web-based clinical trial registries 
☐ Clinicaltrails.gov 
☐ YCCI Recruitment database 
☐ Social Media (Twitter/Facebook):  
 
 
* Requests for medical records should be made through JDAT as described at 
http://medicine.yale.edu/ycci/oncore/availableservices/datarequests/datarequests.aspx 
 
3.  Recruitment Procedures:  
 
Describe how potential subjects will be identified: Patient subjects will be identified 
based on the following inclusion/exclusion criteria using our best practice alert build in Epic:   
 Age>18 Years 
 Seen in internal medicine or cardiology clinic 
 Left Ventricular Ejection Fraction ≤40% 
 Yale Heart Failure Registry (NCT04237701) 

HIC#2000027014 
16 
 
Providers will be identified based on retrospective chart review by study investigators to 
determine the frequency with which they see eligible patients.  
 
Describe how potential subjects are contacted. Eligible providers will be sent an email 
that includes the REDCap link which directs the providers to the electronic consent 
form.The email will be sent by one of the study team members.  Should email contact be 
unsuccessful, we will attempt to consent providers via in-person signed consent, or via a 
waiver of documentation of consent via Epic inbasket messages or verbal telephone 
consent.   
 
We will engage in pre-trial education (during the consent process) and periodic outreach to 
all participating clinicians, informing them of the nature of the study, the fact that it is a 
randomized trial, and describing the alert and evidence-based therapies for HFrEF. We will 
additionally inform them that limited data is being collected regarding provider behavior. 
However, we will also make it clear that this data will not be linked to individual clinicians or 
outcomes. All such data will only be analyzed in aggregate. Patient subjects will not be 
contacted, and we will be requesting a waiver of informed consent. 
 
 
Who is recruiting potential subjects? For patients, this will be done electronically via the 
electronic health record. Patients will be identified if they are seen in outpatient general 
medicine or cardiology clinic and meet the inclusion and exclusion criteria stated above.  
Providers will be recruited and contacted by study coordinators and the principle 
investigator as described above.  
  
  
4. Assessment of Current Health Provider Relationship for HIPAA Consideration: 
Does the Investigator or any member of the research team have a direct existing clinical 
relationship with any potential subject?  
☐Yes, all subjects 
☒Yes, some of the subjects 
☐No 
 
If yes, describe the nature of this relationship. Some members of the research team 
might have previously treated a subject. We do not foresee this impacting the care of any 
patient.  
 
5. Request for waiver of HIPAA authorization: We are requesting HIPAA Waiver 
Choose one:  
☒ For entire study  
☐ For recruitment/screening purposes only 
 
☐ For inclusion of non-English speaking subject if short form is being used and there is 
no translated HIPAA research authorization form available on the University’s HIPAA 
website at hipaa.yale.edu. 
 

HIC#2000027014 
17 
 
Describe why it would be impracticable to obtain the subject’s authorization for 
use/disclosure of this data: Patient subjects will not be informed of their randomization 
status or participation in this trial as the trial could not be feasibly performed if subjects were 
told they were enrolled. All investigators will be blinded to treatment assignment until the 
end of the trial period. Care providers will, obviously, not be blinded to the intervention as 
they are receiving the alert and will consent to take part in the study.. We will engage in both 
pre-trial and periodic teaching and discussion with all consented care providers  to inform 
clinicians about the nature of the study. A letter will be sent to the providers.  
 
ii. If requesting a waiver of signed authorization, describe why it would be 
impracticable to obtain the subject’s signed authorization for use/disclosure of this 
data: The study is being conducted under a waiver of informed consent. This research is 
minimal risk, as outlined above, and cannot be practicably performed in the absence of a 
waiver of informed consent (as patients randomized to the usual care arm would be required 
not to reveal their HF diagnosis to their providers). There is no infringement of rights or 
welfare of patients, as the alert has no direct effect on the patient, short of via the actions of 
their providers. Data abstracted from the medical record will be de-identified, as described in 
the confidentiality section above. Another defense against these risks is the alert language 
itself. Each alert calls attention to the fact that alerts are not issued for all patients with heart 
failure.   
 
 
The investigator assures that the protected health information for which a Waiver of 
Authorization has been requested will not be reused or disclosed to any person or 
entity other than those listed in this application, except as required by law, for 
authorized oversight of this research study, or as specifically approved for use in 
another study by an IRB. 
 
Researchers are reminded that unauthorized disclosures of PHI to individuals outside of the 
Yale HIPAA-Covered entity must be accounted for in the “accounting for disclosures log”, by 
subject name, purpose, date, recipients, and a description of information provided.  Logs are to 
be forwarded to the Deputy HIPAA Privacy Officer. 
 
6. Process of Consent/Assent: Eligible providers will approached for consent in one of the 
methods described above.  In cases where signed consent can occur, providers will be 
provided with a link that includes the REDCap link which directs the providers to the 
electronic consent form. The email will be sent by one of the study team members. The 
provider consent to participation by typing in their first and last name, include date/time of 
consent, and by signing the electronic consent. It will also have providers give their 
address, telephone number, and date of birth for purposes of payment through the OnCore 
system.  Following that, the providers click on the “submit” button. Following that, the study 
team will then send a copy of the signed form to the provider along with a link to the 
REDCap survey.  
 
Patient subjects will not be informed of their randomization status or participation in this trial 
as the trial could not be feasibly performed if subjects were told they were enrolled. Also, 

HIC#2000027014 
18 
 
there are no guideline-based specific recommendations based on a risk score assessment, 
or any other prognostic assessment for that matter. All investigators will be blinded to 
treatment assignment until the end of the trial period. Care providers will, obviously, not be 
blinded to the intervention as they are receiving the alert. We will engage in both pre-trial 
and periodic teaching and discussion with all consented care providers to inform clinicians 
about the nature of the study, evidence based guidelines and the specific factors that are 
being measured. 
 
7. Evaluation of Subject(s) Capacity to Provide Informed Consent/Assent: N/A (see 
above).  
 
8. Non-English-Speaking Subjects: Since the focus of this study is provider behavior, the 
language status of the subject would not matter.  
 
Note* If more than 2 study participants are enrolled using a short form translated into the same 
language, then the full consent form should be translated into that language for use the next 
time a subject speaking that language is to be enrolled. 
 
Several translated short form templates are available on the HRPP website (yale.edu/hrpp) 
and translated HIPAA Research Authorization Forms are available on the HIPAA website 
(hipaa.yale.edu). If the translation of the short form is not available on our website, then the 
translated short form needs to be submitted to the IRB office for approval via modification prior 
to enrolling the subject.   Please review the guidance and presentation on use of the short 
form available on the HRPP website. 
 
If using a short form without a translated HIPAA Research Authorization Form, please 
request a HIPAA waiver in the section above.  
 
9. 
Consent Waiver: In certain circumstances, the HIC may grant a waiver of signed 
consent, or a full waiver of consent, depending on the study. If you will request either a 
waiver of consent, or a waiver of signed consent for this study, complete the appropriate 
section below.   
 
☐Not Requesting any consent waivers  
☒Requesting a waiver of signed consent: (for provider subjects only) 
☐ Recruitment/Screening only (if for recruitment, the questions in the box below will 
apply to recruitment activities only) 
☒ Entire Study (Note that an information sheet may be required.) 
 

HIC#2000027014 
19 
 
 
  
  ☒ Requesting a waiver of consent: (for patient subjects only)☐ Recruitment/Screening only (if 
for recruitment, the questions in the box below will apply to recruitment activities only) 
☒ Entire Study  
 
 
 
 
 
 
 
 
 
SECTION IV: PROTECTION OF RESEARCH SUBJECTS 
 
    Confidentiality & Security of Data: 
 
1. What protected health information (medical information along with the HIPAA 
identifiers) about subjects will be collected and used for the research? Age, gender, 
date of birth and several categories of health information (provider encounters, notes, 
comorbidity, medication lists, problem lists, family history, allergies, laboratory findings, 
procedures, immunizations, vital signs, and medical record numbers) and relevant clinical 
For a waiver of signed consent, address the following: 
 Would the signed consent form be the only record linking the subject and the research?  YES 
☒  NO ☐  
 Does a breach of confidentiality constitute the principal risk to subjects? YES ☒  NO ☐ 
 OR 
 Does the research pose greater than minimal risk? YES ☐    NO☒  
 Does the research include any activities that would require signed consent in a non-research 
context? YES ☐   NO ☒ 
For a full waiver of consent, please address all of the following: 
 
Does the research pose greater than minimal risk to subjects?   
☐ Yes If you answered yes, stop. A waiver cannot be granted.   
☒ No 
 
Will the waiver adversely affect subjects’ rights and welfare? YES ☐    NO☒ 
 
Why would the research be impracticable to conduct without the waiver? The research 
proposed is of minimal risk to subjects. No procedures or tests are being performed.  No 
additional studies are being requested. The sole intervention is an electronic alert that will be 
sent to relevant members of the subjects’ clinical care team. The risk of the alert itself is minimal. 
Subjects will not be informed of their randomization status or participation in this trial as the trial 
could not be feasibly performed if subjects were told they were enrolled. All investigators will be 
blinded to treatment assignment until the end of the trial period. Care providers will, obviously, 
not be blinded to the intervention as they are receiving the alert. We will engage in both pre-trial 
and periodic teaching and discussion with all care providers (administered through short 
presentations at divisional conferences) to inform clinicians about the nature of the study, the 
fact that the same alerts do not fire for all patients with heart failure and to discuss specific 
factors that are being measured. 
 
Where appropriate, how will pertinent information be returned to, or shared with subjects 
at a later date? We expect that there will be considerable academic output that results from 
these efforts. This will be shared with the providers who participated in the study and any 
publication that result will be made available to the general public. 

HIC#2000027014 
20 
 
outcomes will be collected on subjects. This data will be deidentified. Individual provider data 
will not be collected. 
 
2. How will the research data be collected, recorded and stored? This data will initially be 
transferred toClinical and Translational Research Accelerator (CTRA)) to a data analyst 
associated with this project for cleaning. All YNHS patient data will then be shared using Yale 
ITS managed file transfer. This platform is a web-based application used to share data 
packages over the internet through a secure channel. When being shared, the data will be 
encrypted using https secure protocol. Access to this data will only by available to study 
personnel who are a part of the study team. 
 
3. How will the digital data be stored? ☐CD  ☐DVD  ☐Flash Drive  ☐Portable Hard Drive   
☒Secured Server  ☐Laptop Computer  ☐Desktop Computer  ☐Other 
 
4. What methods and procedures will be used to safeguard the confidentiality and 
security of the identifiable study data and the storage media indicated above during 
and after the subject’s participation in the study? All data used at Yale will be transferred 
over secure channels and encrypted during sending. Data will be stored on 3-lock compliant 
servers within Yale or on secure, Yale-issued IronKey devices. Only study personnel directly 
involved in data analysis with a need to access PHI will have access to these data.  
 
All portable devices must contain encryption software, per University Policy 5100.  If there is a 
technical reason a device cannot be encrypted please submit an exception request to the 
Information Security, Policy and Compliance Office by clicking on url http://its.yale.edu/egrc or 
email it.compliance@yale.edu 
 
5. What will be done with the data when the research is completed? Are there plans to 
destroy the identifiable data? If yes, describe how, by whom and when identifiers will 
be destroyed. If no, describe how the data and/or identifiers will be secured. Data will 
be maintained on secure, encrypted servers at the Program of Applied Translational 
Research (PATR) after completion of the research study for a minimum of 5 years after 
publication of our findings in a peer-reviewed journal (in such case as there is a need to 
return to the original data source to validate a finding or respond to a question).  
 
6. If appropriate, has a Certificate of Confidentiality been obtained?  
Since the data obtained are from broad health characteristics from personal health records 
that do not target any particular sensitive research areas, a CoC has not been obtained.  
SECTION V: POTENTIAL BENEFITS 
 
 
Potential Benefits: Subjects in this study may benefit from their provider being given 
information about their prognosis. Regardless of the outcome for participants, the results of 
these studies may lead to significant societal benefit, as even a negative study would lead to 
less enthusiastic adoption of ineffective alerting. The risk/benefit ratio, given the minimal risk to 
study subjects, is more than acceptable in this series of studies. 
 

HIC#2000027014 
21 
 
         SECTION VI: RESEARCH ALTERNATIVES AND ECONOMIC CONSIDERATIONS 
 
1. Alternatives: The provider can dismiss the alert if they choose.  
 
Payments for Participation (Economic Considerations): Patient subjects will not receive 
any payment.  Provider subjects will receive payment in the form of a gift card for their 
completion of the pre- and post-trial surveys assessing their knowledge of guidelines and 
their overall experience with the alert. Providers will receive $50 for completion of the pre-
trial survey and $200 for the completion of the post-trial survey. There is no payment 
directly connected to their experience with the alert itself (i.e. whether they accept or 
dismiss the alert, acknowledge or ignore the alert, etc.). All enrolled providers, regardless of 
randomization status, will partake in the study surveys and receive payment. 
 
2. Costs for Participation (Economic Considerations): None.   
 
3. In Case of Injury: Not applicable. 
 
IMPORTANT REMINDERS 
 
Will this study have a billable service? No☒ 
 
A billable service is defined as any service rendered to a study subject that, if he/she was not 
on a study, would normally generate a bill from either Yale-New Haven Hospital or Yale 
Medical Group to the patient or the patient’s insurer. The service may or may not be performed 
by the research staff on your study but may be provided by professionals within either Yale-
New Haven Hospital or Yale Medical Group (examples include x-rays, MRIs, CT scans, 
specimens sent to central labs, or specimens sent to pathology). Notes: 1. There is no 
distinction made whether the service is paid for by the subject or their insurance (Standard of 
Care) or by the study’s funding mechanism (Research Sponsored). 2. This generally includes 
new services or orders placed in EPIC for research subjects.  
 
If answered, “yes”, this study will need to be set up in OnCore, Yale’s clinical research 
management system, for Epic to appropriately route research related charges. Please 
contact oncore.support@yale.edu 
 
Are there any procedures involved in this protocol that will be performed at YNHH or one of its 
affiliated entities?  Yes ☐  
 
If Yes, please answer questions a through c and note instructions below.   
a. Does your YNHH privilege delineation currently include the specific procedure that you will 
perform? Yes ☐ 
b. Will you be using any new equipment or equipment that you have not used in the past for 
this procedure? ☐  No ☐ 
c. Will a novel approach using existing equipment be applied? No ☐ 
  

HIC#2000027014 
22 
 
If you answered “no” to question 4a, or "yes" to question 4b or c, please contact the YNHH 
Department of Physician Services (688-2615) for prior approval before commencing with your 
research protocol. 
IMPORTANT REMINDER ABOUT RESEARCH AT YNHH  
Please note that if this protocol includes Yale-New Haven Hospital patients, including patients 
at the HRU, the Principal Investigator and any co-investigators who are physicians or mid-level 
practitioners (includes PAs, APRNs, psychologists and speech pathologists) who may have 
direct patient contact with patients on YNHH premises must have medical staff appointment 
and appropriate clinical privileges at YNHH. If you are uncertain whether the study personnel 
meet the criteria, please telephone the Physician Services Department at 203-688-2615. By 
submitting this protocol as a PI, you attest that you and any co-investigator who may 
have patient contact has a medical staff appointment and appropriate clinical privileges 
at YNHH. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

HIC#2000027014 
23 
 
References: 
1. 
Braunwald E. Heart Failure. JACC: Heart Failure. 2013;1:1-20. 
2. 
Hollenberg SM, Warner Stevenson L, Ahmad T, Amin VJ, Bozkurt B, Butler J, Davis LL, 
Drazner MH, Kirkpatrick JN, Peterson PN, Reed BN, Roy CL and Storrow AB. 2019 ACC 
Expert Consensus Decision Pathway on Risk Assessment, Management, and Clinical 
Trajectory of Patients Hospitalized With Heart Failure: A Report of the American College of 
Cardiology Solution Set Oversight Committee. J Am Coll Cardiol. 2019;74:1966-2011. 
3. 
Soufer A, Riello RJ, Desai NR, Testani JM and Ahmad T. A Blueprint for the Post 
Discharge Clinic Visit after an Admission for Heart Failure. Prog Cardiovasc Dis. 2017;60:237-
248. 
4. 
Greene SJ, Butler J, Albert NM, DeVore AD, Sharma PP, Duffy CI, Hill CL, McCague K, 
Mi X, Patterson JH, Spertus JA, Thomas L, Williams FB, Hernandez AF and Fonarow GC. 
Medical Therapy for Heart Failure With Reduced Ejection Fraction: The CHAMP-HF Registry. 
J Am Coll Cardiol. 2018;72:351-366. 
5. 
Fonarow GC, Yancy CW, Hernandez AF, Peterson ED, Spertus JA and Heidenreich 
PA. Potential impact of optimal implementation of evidence-based heart failure therapies on 
mortality. Am Heart J. 2011;161:1024-30 e3. 
6. 
Lee J, Szeto L, Pasupula DK, Hussain A, Waheed A, Adhikari S, Sharbaugh M, Thoma 
F, Althouse AD, Fischer G, Lee JS and Saba S. Cluster Randomized Trial Examining the 
Impact of Automated Best Practice Alert on Rates of Implantable Defibrillator Therapy. Circ 
Cardiovasc Qual Outcomes. 2019;12:e005024. 
7. 
Tingley J, Dolansky MA and Walsh MN. Team-Based Transitions of Care in Heart 
Failure. Heart Fail Clin. 2015;11:371-8. 
8. 
Ahmad T, Desai N, Wilson F, Schulte P, Dunning A, Jacoby D, Allen L, Fiuzat M, 
Rogers J, Felker GM, O'Connor C and Patel CB. Clinical Implications of Cluster Analysis-
Based Classification of Acute Decompensated Heart Failure and Correlation with Bedside 
Hemodynamic Profiles. PLoS One. 2016;11:e0145881. 
9. 
Wilson FP and Greenberg JH. Acute Kidney Injury in Real Time: Prediction, Alerts, and 
Clinical Decision Support. Nephron. 2018;140:116-119.