Uplift Modeling to More Narrowly Target Alerts for Acute Kidney Injury
Phase: NA
Conditions: Acute Kidney Injury
Sponsor: Yale University

 
 
 
 
 
 
 
Electronic Alerts for AKI Amelioration (ELAIA-3): 
Using Uplift Modeling for Targeted AKI Alerts 
NCT02786277 
February 26, 2025 
Protocol and SAP 

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
CLINICAL STUDY PROTOCOL
Interventional Drug or Biologic
 
Electronic Alerts for AKI Amelioration 
(ELAIA-3):
Using Uplift Modeling for Targeted AKI Alerts
 Protocol Number
Protocol Number
Protocol Version
02/25/2025
3.0
PRINCIPAL INVESTIGATOR:
Name Francis P. Wilson
Department Internal Medicine
Email Address francis.p.wilson@yale.edu
Study Phase (1, 2, 3, or 4-postmarketing): 3
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
2
REVISION HISTORY:
Revision #
Version Date
1.0
06/12/2023
2.0
11/14/2023
3.0
2/25/25
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
3
Synopsis
Primary Objective
The primary objective of this study is to determine whether the use of uplift (also known as 
Conditional Average Treatment Effect – CATE) modeling to empirically identify patients 
expected to benefit the most from AKI alerting and to target AKI alerts to these patients will 
reduce the rates of AKI progression, dialysis, and mortality.
Secondary Objective (if applicable)
The secondary objective of this study is to determine whether the use of uplift modeling to 
empirically identify patients expected to benefit the most from AKI alerting and to target AKI 
alerts to these patients will improve the rates of AKI best practices by clinicians.
Study Duration
2 years
Study Design 
Multicenter, randomized, parallel-group, controlled trial testing the efficacy of a targeted AKI 
alert on a composite of AKI progression, dialysis and death.
Number of Study Sites
Four teaching hospitals within the Yale New Haven Health System and located through the 
state of CT.
Study Population
Eligible subjects are adult inpatients with Acute Kidney Injury based on KDIGO creatinine 
criteria.
Number of Participants
4,092
Primary Outcome Variables
The primary outcome will be a composite of AKI progression, inpatient dialysis, and inpatient 
death within 14 days of randomization.  
Secondary and Exploratory Outcome Variables (if applicable)
Secondary outcomes will assess individual components of the primary outcome, as well as a 
variety of AKI best practice metrics.  
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
4
Abbreviations
Abbreviation 
Explanation
AKI
     Acute Kidney Injury
EHR
     Electronic Health Record
KDIGO 
     Kidney Disease: Improving Global Outcomes
ELAIA   
 
     Electronic Alerting for AKI Amelioration
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
5
Glossary of Terms
Glossary
Explanation
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
6
Table of Contents
Synopsis ..................................................................................................................................3
Primary Objective.................................................................................................................3
Secondary Objectives (if applicable)....................................................................................3
Study Duration......................................................................................................................3
Study Design........................................................................................................................3
Study Population ..................................................................................................................3
Number of Participants.........................................................................................................3
Number of Study Sites .........................................................................................................3
Primary Outcome Variables .................................................................................................3
Secondary and Exploratory Outcome Variables (if applicable)............................................3
Visit Schedule Table (Optional)............................................................................................3
Study Flow Chart (optional)..................................................................................................3
Abbreviations ...........................................................................................................................4
Glossary of Terms....................................................................................................................5
1
Introduction.....................................................................................................................10
1.1
Introductory Statement ............................................................................................10
2
Background.....................................................................................................................11
2.1
Background/prevalence of research topic ...............................................................11
3
Rationale/Significance ....................................................................................................13
3.1
Problem Statement..................................................................................................13
3.2
Purpose of Study/Potential Impact ..........................................................................13
3.2.1
Potential Risks..................................................................................................13
3.2.2
Potential Benefits .............................................................................................13
4
Study Objectives.............................................................................................................14
4.1
Hypothesis...............................................................................................................14
4.2
Primary Objective ....................................................................................................14
4.3
Secondary Objectives (if applicable) .......................................................................14
5
Study Design ..................................................................................................................15
5.1
General Design Description.....................................................................................15
5.1.1
Study Date Range and Duration ......................................................................20
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
7
5.1.2
Number of Study Sites .....................................................................................20
5.2
Outcome Variables..................................................................................................20
5.2.1
Primary Outcome Variables .............................................................................20
5.2.2
Secondary and Exploratory Outcome Variables (if applicable)........................20
5.3
Study Population .....................................................................................................21
5.3.1
Number of Participants.....................................................................................22
5.3.2
Eligibility Criteria/Vulnerable Populations.........................................................22
6
Methods..........................................................................................................................23
6.1
Treatment ................................................................................................................23
6.1.1
Identity of Investigational Product ....................................................................23
6.1.2
Dosage, Admin, Schedule (if applicable) .........................................................23
6.1.3
Method of Assignment/Randomization (if applicable) ......................................23
6.1.4
Blinding and Procedures for Unblinding (if applicable).....................................23
6.1.5
Packaging/Labelling .........................................................................................23
6.1.6
Storage Conditions...........................................................................................23
6.1.7
Concomitant therapy ........................................................................................23
6.1.8
Restrictions.......................................................................................................23
6.2
Assessments ...........................................................................................................24
6.2.1
Efficacy.............................................................................................................24
6.2.2
Safety/Pregnancy-related policy.......................................................................24
6.2.3
Adverse Events Definition and Reporting.........................................................24
6.2.4
Pharmacokinetics (if applicable).......................................................................25
6.2.5
Biomarkers (if applicable).................................................................................25
6.3
Study Procedures....................................................................................................25
6.3.1
Study Schedule ................................................................................................25
6.3.2
Informed Consent.............................................................................................26
6.3.3
Screening .........................................................................................................26
6.3.4
Enrollment ........................................................................................................26
6.3.5
On Study Visits.................................................................................................26
6.3.6
End of Study and Follow-up .............................................................................26
6.3.7
Removal of subjects.........................................................................................27
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
8
6.4
Statistical Method ....................................................................................................27
6.4.1
Statistical Design..............................................................................................27
6.4.2
Sample Size Considerations ............................................................................27
6.5
Planned Analyses....................................................................................................28
6.5.1
Primary Objective Analyses .............................................................................28
6.5.2
Secondary Objectives Analyses.......................................................................28
6.5.3
Exploratory Objectives Analyses (if applicable) ...............................................28
6.5.4
Safety ...............................................................................................................29
6.5.5
Analysis of Subject Characteristics ..................................................................31
6.5.6
Interim Analysis (if applicable)..........................................................................31
6.5.7
Handling of Missing Data .................................................................................31
7
Trial Administration.........................................................................................................32
7.1
Ethical Considerations: Informed Consent/Assent and HIPAA Authorization .........32
7.2
Institutional Review Board (IRB) Review.................................................................32
7.3
Subject Confidentiality.............................................................................................32
7.4
Deviations/Unanticipated Problems.........................................................................33
7.5
Data Collection ........................................................................................................34
7.6
Study Records.........................................................................................................34
7.7
Access to Source Documents .................................................................................34
7.8
Data or Specimen Storage/Security ........................................................................34
7.9
Retention of Records...............................................................................................34
7.10
Study Monitoring......................................................................................................34
7.11
Data Safety Monitoring Plan....................................................................................35
7.12
Study Modification ...................................................................................................35
7.13
Study Discontinuation..............................................................................................35
7.14
Study Completion ....................................................................................................36
7.15
Conflict of Interest Policy.........................................................................................36
7.16
Funding Source .......................................................................................................36
7.17
Publication Plan.......................................................................................................36
8
Appendices.....................................................................................................................37
9
List of Tables ..................................................................................................................38
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
9
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
10
1 Introduction
1.1
Introductory Statement
This document is a protocol for a human research study. The purpose of this protocol is to 
ensure that this study is to be conducted according to ICH GCP guidelines, and according to 
CFR 21 Part 312, other applicable government regulations and Institutional research policies 
and procedures.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
11
2 Background
2.1
Background/prevalence of research topic
Acute kidney injury (AKI) carries a significant, independent risk of morbidity and 
mortality among hospitalized patients.[1,2] Recent studies have demonstrated increased 
mortality among patients with even small increases in serum creatinine concentration.[3]  
International guidelines for the treatment of AKI focus on appropriate management of drug 
dosing, avoiding nephrotoxic exposures, and careful attention to fluid and electrolyte 
balance.[2,4] Early nephrologist involvement may also improve outcomes in AKI.[5] Without 
appropriate provider recognition of AKI, however, none of these measures can be taken, and 
patient outcomes may suffer.[6–8] 
Despite its association with poor clinical outcomes, AKI is asymptomatic and 
frequently overlooked by clinicians, with fewer than half of all AKI patients with 
documentation of the syndrome in the electronic medical record, which was associated with 
decreased rates of AKI clinical best practices.[8]  Increased provider recognition of the 
syndrome in hospitalized patients may improve patient care and clinical outcomes.  While 
multiple health systems in the United States have implemented electronic AKI alerting as a 
part of routine clinical care, data on the efficacy of such alerts in the context of randomized 
controlled trials is limited.[9,10]
Our research group recently conducted a large-scale multicenter randomized 
controlled trial of electronic alerts for AKI throughout the Yale New Haven Health System 
from 2018 to 2020 (ELAIA-1).[11,12]  The trial, which enrolled 6,030 patients with AKI, as 
defined by an increase in creatinine of 0.3mg/dL over 48 hours or 50% over 7 days, 
randomized patients between usual care and an intervention group whereby providers 
received a general AKI alert informing them to the presence of AKI and the patient’s recent 
creatinine trends, and provided a link to an AKI-specific order set.  Our study showed that, 
overall, alerting physicians to the presence of AKI did not demonstrate a difference in the 
rate of our primary outcome of progression of AKI, dialysis, or death, despite the alert 
leading to some process of care changes such as measurement of creatinine and urinalysis.  
There was, however, substantial heterogeneity among the study sites.  Stratification of 
results by hospital revealed that signals of harm exist in the two non-teaching hospitals of 
our study, where a higher relative risk for our primary outcome was seen in the intervention 
group.  This increased risk was shown to be driven by an increased number of deaths in the 
alert group, however, process measures did not mediate the observed difference. 
The proliferation of alerting systems that are ineffective can lead to the phenomenon 
of alert fatigue, whereby providers tend to ignore alerts in a high-alert environment.[13,14]  
Several studies have suggested that alert responses wane over time, and that providers may 
perceive alerts as extraneous and disruptive.  The phenomenon of alert fatigue can have 
deleterious effects on patient care, which emphasizes the need for rigorous evaluation of all 
clinical alerts, even ones that are seemingly benign, for true effectiveness on patient 
outcomes.  Further, given the highly heterogenous nature of AKI, a more personalized 
approach to AKI alerting may be warranted.  Our prior study enrolled all patients who 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
12
developed AKI rather than a targeted subset of patients who may benefit.  To date, no 
rigorous assessment of AKI alerts in a targeted population has been done.
Typically, prognostic modeling (trial enrichment) is used to preferentially enroll 
patients into clinical trials based on probability of a high-risk outcome[15,16].  Uplift 
modeling, commonly used in marketing, is a novel concept in the medical field and aims to 
determine phenotypic characteristics that predict a response (benefit or harm) to a given 
intervention.[17]  In this way, patients who are predicted to benefit most from an intervention 
are identified and preferentially targeted.  Uplift modeling of alerting systems has the 
potential to both improve alert effectiveness through intelligent targeting, and reduce alert 
fatigue.
As a proof of concept, we created an uplift model using data from a previous single-
center AKI alert trial performed by our group.[18]  Three uplift models were built and 
compared using a population of 2,278 adult patients with AKI who were randomized to an 
AKI alert versus usual care.  The uplift algorithm was trained in 70% of the data and 
evaluated in the final 30%.  Overall performance was defined by the strength of the 
interaction, as characterized by the t-statistic in a linear regression model, between the uplift 
score (prediction of benefit) and the randomization status.  The outcome of interest was the 
maximum relative change in creatinine from randomization to 3 days post randomization.  A 
statistically significant interaction term (P<0.05) implied that the uplift modelling significantly 
modified the effect of the alert towards our outcome.  In other words, by targeting alerts to 
patients with higher uplift scores, we see an increase in the beneficial effect of the alert.  
Each of the three models produced a significant interaction term, and successfully stratified 
patients according to alert effect.  While for those in the group with low uplift scores, alerting 
was associated with a median increase in our outcome (change in creatinine), those in the 
group with a high uplift score had alerts that were associated with a reduction in the change 
in creatinine.  These higher uplift scores were more often seen in the elderly, women, and 
those with lower creatinine at randomization, suggesting that alerts may preferentially benefit 
those with more slowly developing, or less noticeable, AKI.
In this study, we will expand upon our prior AKI alert trial to determine prospectively 
whether the use of uplift modeling to preferentially target patients expected to benefit from 
an AKI alert will reduce the rates of AKI progression, dialysis and death among hospitalized 
patients with AKI.  Inpatients at 4 teaching hospitals within the YNHH system with AKI, 
based on the Kidney Disease: Improving Global Outcomes (KDIGO) creatinine criteria will 
be randomized to receive alerts based on uplift score (with higher scores receiving alerts 
and lower scores not receiving alerts as recommended) versus the opposite (with higher 
scores not receiving alerts and lower scores receiving alerts anti-recommended). This study 
design maximizes power to detect the benefit of uplift-targeting and, as we show below, is 
operationally indistinguishable from a traditional trial of alert versus usual care (eg ELAIA-1). 
The primary outcome will be a composite of AKI progression, dialysis, or mortality within 14 
days of randomization.  Secondary outcomes will focus on AKI-specific process measures 
defined below. 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
13
3 Rationale/Significance
3.1
Problem Statement
AKI carries significant mortality and morbidity, yet goes frequently unrecognized in 
hospitalized patients.  AKI alerts have been implemented in various hospital systems, but 
data on efficacy from rigorous randomized controlled trials is lacking, and no rigorous trials 
of targeted AKI alerts has been conducted to-date.
3.2
Purpose of Study/Potential Impact
To determine whether preferential targeting of AKI alerts to those empirically identified via 
uplift modeling to have the greatest expected benefit will reduce the rates of AKI 
progression, dialysis and death among hospitalized patients with AKI.  Targeting AKI alerts 
has the potential to both improve outcomes and reduce alert fatigue.
3.2.1
Potential Risks
Risks to subjects include loss of confidentiality, overtreatment in those randomized to the 
intervention group, undertreatment of those randomized to the control group, and alert 
fatigue to providers exposed to the alerts.
3.2.2
Potential Benefits
Subjects who participate in this study may benefit from their physicians getting an alert to 
increase their awareness of their AKI.  However, as no standard of treatment for AKI exists, 
there remains clinical equipoise on the benefits of AKI alerting.  This trial will benefit society 
at large regardless of the outcomes of the study.  Should targeted alerting improve 
outcomes, this alert may be broadly implemented to improve outcomes of those with AKI 
and serve as an example of how to use uplift modeling to target alerts to other patient 
populations.  If targeted alerts prove deleterious then our study will provide an argument for 
removing an ineffective alert to reduce provider alert fatigue and possibly associated 
negative or unintended affects.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
14
4 Study Objectives
4.1
Hypothesis
The use of uplift modeling to target AKI alerts to those expected to benefit most will reduce 
the rates of AKI progression, dialysis, and mortality among hospitalized patients with AKI.
4.2
Primary Objective
The primary objective of this study is to determine whether the use of an AKI alert system 
that targets alerts based on uplift modeling will reduce the rates of AKI progression, dialysis, 
and mortality.
4.3
Secondary Objectives (if applicable)
The secondary objective of this study is to determine whether the use of uplift modeling to 
empirically identify patients expected to have the greatest benefit from AKI alerting will 
improve the rates of AKI best practices by clinicians.
4.4
Exploratory Objectives (if applicable)
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
15
5 Study Design
5.1
General Design Description
This is a single-blind multicenter randomized trial to determine the efficacy of an uplift-
targeted AKI alert system compared to an uplift anti-targeted alert system among 
hospitalized patients with AKI in reducing a composite outcome of AKI progression, death, 
and dialysis within 14 days of randomization.
Targeted Alert Build
Data from our ELAIA-1 clinical trial was used to learn and tune our uplift modeling strategy, 
which also goes by the name of policy learning. The ELAIA-1 data set was temporally split 
into a training set (66.66%) and a validation set (33.33%). In the training set, our model uses 
adaptive lasso techniques with approximate cross-fitting to estimate a doubly-robust score 
for each patient.  The average of these scores is widely cited to produce the celebrated 
augmented inverse-propensity weighted estimator for the average treatment effect (ATE). 
Instead, we follow recent literature and perform a regression to predict these scores from 
patient covariates.  More specifically, we use a penalized weighted logistic regression, where 
the weights use a novel construction to allow us to interpret the predictions as a (linearly 
transformed) estimate of the conditional average treatment effect (CATE).   Lastly, we use 
these CATE estimates to train another penalized weighted logistic regression to estimate a 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
16
“stochastic policy” which outputs the recommendation probability for alerting using a reduced 
set of features. The corresponding “deterministic policy” that we shall implement chooses to 
send an alert when the recommendation probability for alerting is greater than 0.5.
In the validation set, we predict the recommendation probability for each patient and seek to 
evaluate how well ranked the estimates are. Rank-based evaluation provides us aggregate 
information on the performance of all policies that could be created by different thresholds 
for the given recommendation estimates. Large values of this evaluation metric support 
accurate CATE modeling, the recommendation probability’s ranking calibration, and 
robustness to the deterministic policy’s threshold choice.  Analogous to the classic AUROC, 
our proposed evaluation metric is a c-statistic, calculated between a novel score and the 
observed composite outcomes.  We choose our training model’s hyperparameters and, as a 
result, features according to this criteria’s performance in the validation set.
50 features were selected to be included in the policy.  The 10 top features selected to be 
most predictive of alert benefit/harm are:
1) Most recent diastolic blood pressure
2) Most recent systolic blood pressure
3) Prior receipt of ultrasound for deep veinous thrombosis within the current encounter
4) Prior receipt of a transthoracic echocardiogram within the current encounter
5) Prior exposure to insulin within the current encounter
6) Prior exposure to docusate within the current encounter
7) Prior exposure to miralax within the current encounter
8) Whether opiates were detected in the urine within the current encounter (binary 
variable; 1 = yes; 2 = no)
9) Whether the patient had their blood type tested with an ‘A’ result (binary variable; 1 = 
yes; 2 = no)
10) Whether the patient had a thyroid-stimulating hormone measurement within the 
current encounter (binary variable; 1 = yes; 2 = no)
Various alternative models were fit in the training set; however, the described model had the 
best c-statistic in the validation set. 
In the current study, all hospitalized patients with AKI will be automatically enrolled and 
randomized into our trial.  Prospectively, the deterministic policy will be applied to each 
enrolled patient; when combined with a randomization step, this will determine alert vs usual 
care.  We further detail this randomized process below. For those randomized to follow our 
policy, we expect 51% of the total AKI population within our study sites to be sent alerts.
Subject Eligibility
Identification of patient subjects will be performed entirely within the Epic electronic medical 
record system based on inclusion and exclusion criteria outlined below.  Eligible patients will 
be identified by our algorithm embedded into our best practice alert with the Epic electronic 
medical record.  This algorithm will identify any hospitalized patients at all participating 
hospitals who have AKI (based on KDIGO creatinine criteria).  To do this, upon opening of a 
patient chart, the algorithm will examine the most recent creatinine value against the 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
17
minimum value in the past 48 hours and 7 days.  If the current value is 0.3 mg/dl above the 
48 hour minimum, or 50% higher than the 7 day minimum, the patient meets AKI criteria.  
Randomization
Randomization will occur the moment the best practice build identifies a patient as being 
eligible. Randomization is achieved using a random number rule that is incorporated in the 
alert. This ensures that, upon meeting criteria, each patient is immediately and randomly 
assigned to an arm. Logic checks within the alerts ensure that once a patient is assigned to 
an arm, they remain on that arm for the remainder of their hospital stay.
Randomization is not to alert versus usual care (as per ELAIA-1). Rather, randomization is to 
alert “as recommended” versus “anti-recommended” (see FIGURE 1). This strategy has the 
advantage of maximizing the observable effect size of a targeting strategy.  It may, at first, 
seem intuitive that this approach differs fundamentally from ELAIA-1.  However, as we 
demonstrate in the figure – this is not the case. In fact, 50% of individuals at random will 
receive the alert and 50% of individuals at random will receive usual care under this design. 
The fundamental difference between this approach and ELAIA-1 is entirely in the analysis – 
as the primary analysis will not evaluate alert versus usual care, but whether the individual 
was randomized to “as recommended” versus “anti-recommended” – regardless of alert 
status.
Intervention
Patients in the alert group will have an alert generated within the electronic health record 
which will consist of a pop-up upon opening of the patient’s chart.  Providers who will receive 
an alert include physicians, physician assistants, nurse practitioners, advanced practice 
registered nurses, fellows, and residents upon opening of the patient’s chart, regardless of 
their relationship with the patient.  
The alert will display as follows: The alert notifies the provider of the presence of AKI, giving 
recent creatine values (most recent, lowest in the last 7 days, and highest in the last 7 days).  
There is also an option to open an AKI-specific order set and to add AKI to the patient’s 
problem list.  Any additional treatment or test that is performed as a result of the alert and 
order set will be considered standard of care for AKI patients and cost of treatment will be 
the responsibility of the patient.  The alert also states that “This alert does not fire for all 
patients.  This is part of a randomized trial” and links to our study website and to AKI best 
practices.  Finally, the alert will also state that “The clinician that sees the alert and decides 
to act on it must confirm with the attending the AKI order and subsequent SOC treatment to 
avoid overtreatment” as a risk mitigation measure. 
The provider will have the option to “accept” or “dismiss” the alert.  If the provider accepts 
the alert, the alert will be suppressed for a 24 hour period.  Our prior data suggests that 
roughly 40% of AKI cases will resolve within this time period and thus no further alerts would 
be generated. The alert will stop firing for the provider under the following conditions.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
18

The provider acknowledges the alert by "agreeing" that AKI is present and accepting 
the alert (alert will be suppressed for 48 hours)

The most recent creatinine does not meet AKI criteria

The patient receives an order for hemodialysis, continuous renal replacement 
therapy, or peritoneal dialysis

The patient is transferred to the hospice service

The patient is discharged from the hospital
The AKI order set will appear as follows:
Non-alert
Patients who are not selected to receive the alert will not generate an alert in the medical 
record, and will continue to receive standard of care.
Study Timeline
We plan to activate our targeted alert in a step-wise fashion throughout our 4 study 
hospitals, beginning in Yale New Haven Hospital, followed by St. Raphael’s Campus, 
Bridgeport Hospital and Greenwich Hospital.  
Staggered enrollment will allow for proper pre-trial education as well as adequate site visits, 
study monitoring, quality control and clinician engagement at each site as roll out begins.  
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
19
Study Outcomes
Our primary outcome will be a composite of AKI progression, dialysis (hemodialysis, 
continuous renal replacement therapy, or peritoneal dialysis), or mortality within 14 days of 
randomization.  
Secondary study endpoints will look at a variety of patient outcomes as well as best practice 
metrics, including the following:

Inpatient mortality

14-day mortality

Inpatient dialysis

14-day dialysis

Discharged on dialysis

Percent who progress to stage 2 AKI

Percent who progress to stage 3 AKI

AKI duration

30-day readmission rate

Cost of index hospitalization

AKI documentation

“Best practice” outcomes: Proportion achieved per patient during index 
hospitalization:
o
Contrast administration
o
Fluid administration
o
Aminoglycoside administration
o
NSAID administration/cessation
o
ACEi administration/cessation
o
Urinalysis order
o
Documentation of AKI
o
Monitoring of creatinine
o
Monitoring of urine output
o
Renal consults
All investigators will be blinded to the treatment assignment until the end of the trial period.
Clinician Outreach and Engagement
While the unit of randomization is the patient, clinicians may also be considered subjects of 
this research, as limited data will be collected regarding completion of AKI best practices.  
We will engage in pre-trial and periodic outreach to all clinicians who may be exposed to the 
study via departmental communications.  We will inform them of the randomized nature of 
the trial and remind them that alerts will not fire for all patients with AKI.  We will also make it 
clear that data regarding clinician behavior will be collected but analyzed only in aggregate, 
and that specific actions will not be linked to individual clinicians or specific patient 
outcomes.
Blinding
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
20
Subjects will not be informed of their randomization status or participation in this trial as the 
trial could not be feasibly performed if subjects were told they were enrolled. We do not feel 
that post-facto informing of patients randomized in this trial is appropriate for several 
reasons.  First, there is no guideline-based specific follow-up or intervention for acute kidney 
injury. Second, many patients may incorrectly assume that acute kidney injury is an 
iatrogenic condition, caused by poor medical care, when in fact it is indicative of the severity 
of the underlying medical condition.  Finally, most patients will not be familiar with "acute 
kidney injury" and informing them of the presence of the condition may engender significant 
stress or anxiety without offering a tangible benefit.
All investigators will be blinded to treatment assignment until the end of the trial period. Care 
providers will not be blinded to the intervention as they are receiving the alert. 
5.1.1
Study Date Range and Duration
We anticipate the study to last two years.  The first six months will involve development and 
refinement of the ELAIA-3 tool.  The expected timeline for patient enrollment is one year, 
followed by 6 months of data analysis and publication.
5.1.2
Number of Study Sites
There are 4 planned enrollment locations, each a teaching hospital within the Yale New 
Haven Health System:  Yale New Haven Hospital York Street, Yale New Haven Hospital St. 
Raphael’s campus, Bridgeport Hospital, and Greenwich Hospital.  We have excluded the two 
non-teaching hospital sites within the Yale New Haven Health System, as our previous alert 
study (ELAIA-1) demonstrated signals of harm with respect to our primary outcome.  While 
this exclusion reduces risk in the present study, it will reduce heterogeneity in our model and 
limit the broader generalizability of our study.
5.2
Outcome Variables
5.2.1
Primary Outcome Variables
The primary outcome will be a composite of AKI progression, inpatient dialysis, and inpatient 
death within 14 days of randomization.  
Progression of AKI is defined as the increase in KDIGO stage from the time of randomization 
to the present.  For patients who are discharged, we will impute 14-day creatinine using the 
last observation carried forward method. 
Dialysis is defined as the receipt of hemodialysis, continuous renal replacement therapy, or 
peritoneal dialysis.  Isolated ultrafiltration treatments will not be included. 
5.2.2
Secondary Outcome Variables (if applicable)
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
21
Secondary outcomes will assess individual components of the primary outcomes, as well as 
a variety of AKI best practice metrics.  Secondary outcomes include:
-
14 day mortality (proportion of patients who expire within 14 days of randomization, 
assessed from point of randomization to date of death from any cause within 14 days 
of randomization)
-
Inpatient mortality (assessed from point of randomization to date of death from any 
cause)
-
14-day dialysis (proportion of patients who receive dialysis, assessed from point of 
randomization to date of first documented dialysis order, within 14 days of 
randomization)
-
Inpatient dialysis (assess from point of randomization to date of first documented 
dialysis order during index hospitalization)
-
Discharge on dialysis (assessed as active orders for dialysis at point of discharge 
from index hospitalization)
-
Progression to stage 2 AKI (assessed as a doubling of serum creatinine from the 
date of randomization to 14 days post randomization)
-
Progression to stage 3 AKI (assessed as a tripling of serum creatinine from the date 
of randomization to 14 days post randomization)
-
Duration of AKI (defined as the time in hours between AKI onset and AKI cessation 
during index hospitalization)
-
30 day readmission rate
-
Index hospitalization cost (assessed from point of randomization to date of discharge 
from index hospitalization)
-
Chart documentation of AKI (assessed by post-discharge ICD-10 codes)
-
Proportion of best practices achieved per subject during index hospitalization, to 
include:
o
Contrast administration
o
Fluid administration
o
Aminoglycoside administration
o
NSAID administration/cessation
o
ACE inhibitor administration/cessation
o
Urinalysis order
o
Documentation of AKI
o
Monitoring of creatinine
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
22
o
Monitoring of urine output
o
Renal consult
Each metric above is binary and the outcome is reported as a composite best 
practice outcome representing the proportion of best practices achieved per 
subject.
5.3
Study Population
Participants include adult inpatients with Acute Kidney Injury as defined by the KDIGO 
creatinine criteria of a 0.3 mg/dl increase over 48 hours or a 50% increase over 7 days.
5.3.1
Number of Participants
All inpatients at each study site will be screened by our best practice alert for eligibility.  We 
will enroll 4,092 patients. Enrollment will be proportional to the number of patients meeting 
enrollment criteria at each hospital, which is to say we will not have specific enrollment goals 
by study site. 
5.3.2    Eligibility Criteria/Vulnerable Populations
In order to be eligible to participate, an individual must meet all of the following:
1.
Adults ≥ 18 years 
2.
Admitted to a participating hospital
3.
Has AKI as defined by creatinine criteria
a.
0.3 mg/dl increase in inpatient serum creatinine over 48 hours OR
b.
50% relative increase in inpatient serum creatinine over 7 days
An individual who meets any of the following criteria will be excluded from participation in 
this study:
1.
Dialysis order prior to AKI onset
2.
Initial creatinine ≥ 4.0 mg/dl
3.
Prior admission in which patient was randomized
4.
Admission to hospice service or comfort measures only order
5.
ESKD diagnosis code
6.
Kidney transplant within six months 
7.
Enrollment in another research study
8.
Opted out of electronic health record research (based on screening by our best 
practice alert developed with Yale’s Joint Data Analytics Team (JDAT))
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
23
6 Methods
6.1
Treatment 
6.1.1
Identity of Investigational Product
The intervention is a best practice alert built by our team in conjunction with our clinical 
application analyst team.  This alert consists of a “pop-up” in the electronic medical record 
when a provider opens an eligible patient’s chart.  The pop up alerts the provider to the 
patient’s AKI, gives recent creatinine values, and provides options to open an AKI-specific 
order set as well as add AKI to the patient’s problem list.
6.1.2
Dosage, Administration, Schedule 
N/A
6.1.3
Method of Assignment/Randomization 
Randomization will occur the moment the best practice build identifies a patient as being 
eligible. Randomization is achieved using a random number rule that is incorporated in the 
alert. This ensures that, upon meeting criteria, each patient is immediately and randomly 
assigned to an arm. Logic checks within the alerts ensure that once a patient is assigned to 
an arm, they remain on that arm for the remainder of their hospital stay.
6.1.4
Blinding and Procedures for Unblinding 
Subjects will not be informed of their randomization status or participation in this trial as the 
trial could not be feasibly performed if subjects were told they were enrolled. We do not feel 
that post-facto informing of patients randomized in this trial is appropriate for several 
reasons.  First, there is no guideline-based specific follow-up or intervention for acute kidney 
injury. Second, many patients may incorrectly assume that acute kidney injury is an 
iatrogenic condition, caused by poor medical care, when in fact it is indicative of the severity 
of the underlying medical condition.  Finally, most patients will not be familiar with "acute 
kidney injury" and informing them of the presence of the condition may engender significant 
stress or anxiety without offering a tangible benefit.
All investigators will be blinded to treatment assignment until the end of the trial period. Care 
providers will not be blinded to the intervention as they are receiving the alert. 
6.1.5
Packaging/Labelling
N/A
6.1.6
Storage Conditions
N/A
6.1.7
Concomitant therapy
Potential subjects will be excluded if they have a prior order for dialysis.  All other therapies 
are permissible within this protocol.
6.1.8
Restrictions
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
24
There are no restrictions.
6.2
Assessments
6.2.1
Efficacy
Efficacy of the alerts will be assessed by data analysis towards our primary and secondary 
outcomes after all participants have been enrolled.
6.2.2
Safety and Pregnancy-related policy
We will conduct three interim analyses, at 25%, 50%, and 75% enrollment as described 
below in the statistical analysis plan.
6.2.3
Adverse Events Definition and Reporting
Definitions
Adverse event (AE) means any untoward medical occurrence associated with the use of an 
intervention in humans, whether or not considered intervention-related (21 CFR 312.32 (a)).
Given that this is a low-risk intervention in a population of individuals at high risk of unrelated 
adverse events (ie hospitalized patients with acute kidney injury), the study team will monitor 
for adverse effects of randomization AND adverse effects of alerting by periodic, blinded 
review of the following outcomes. Should significant differences in these outcomes arise, the 
independent DSMB will be convened to examine the results in an unblinded analysis and 
make their recommendations as to the continuation of the trial.
These events include:

All inpatient deaths

All dialysis events

Progression to more severe AKI stages
Additionally, any comments received from providers regarding the safety or efficacy of the 
alert will be recorded and investigated to determine if an issue resulted in patient harm. 
Relationship to Investigational Product
All AEs must have their relationship to study intervention assessed by the clinician who 
examines and evaluates the participant based on temporal relationship and his/her clinical 
judgment. The degree of certainty about causality will be graded using the categories below. 
In a clinical trial, the study product must always be suspect.

Definitely Related – There is clear evidence to suggest a causal relationship, and 
other possible contributing factors can be ruled out. The clinical event, including an 
abnormal laboratory test result, occurs in a plausible time relationship to study 
intervention administration and cannot be explained by concurrent disease or other 
drugs or chemicals. The response to withdrawal of the study intervention 
(dechallenge) should be clinically plausible. The event must be pharmacologically or 
phenomenologically definitive, with use of a satisfactory rechallenge procedure if 
necessary.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
25

Probably Related – There is evidence to suggest a causal relationship, and the 
influence of other factors is unlikely. The clinical event, including an abnormal 
laboratory test result, occurs within a reasonable time after administration of the 
study intervention, is unlikely to be attributed to concurrent disease or other drugs or 
chemicals, and follows a clinically reasonable response on withdrawal (dechallenge). 
Rechallenge information is not required to fulfill this definition.

Potentially Related – There is some evidence to suggest a causal relationship (e.g., 
the event occurred within a reasonable time after administration of the trial 
medication). However, other factors may have contributed to the event (e.g., the 
participant’s clinical condition, other concomitant events). Although an AE may rate 
only as “possibly related” soon after discovery, it can be flagged as requiring more 
information and later be upgraded to “probably related” or “definitely related”, as 
appropriate.

Unlikely to be related – A clinical event, including an abnormal laboratory test result, 
whose temporal relationship to study intervention administration makes a causal 
relationship improbable (e.g., the event did not occur within a reasonable time after 
administration of the study intervention) and in which other drugs or chemicals or 
underlying disease provides plausible explanations (e.g., the participant’s clinical 
condition, other concomitant treatments).

Not Related – The AE is completely independent of study intervention administration, 
and/or evidence exists that the event is definitely related to another etiology. There 
must be an alternative, definitive etiology documented by the clinician.
Expectedness
The Principal Investigator will be responsible for determining whether an AE is expected or 
unexpected.  An AE will be considered unexpected if the nature, severity, or frequency of the 
event is not consistent with the risk information previously described for the study 
intervention.
6.2.4
Pharmacokinetics (if applicable)
N/A
6.2.5
Biomarkers (if applicable)
N/A
6.3
Study Procedures
Eligible subjects will be automatically enrolled and randomized into our study under a waiver 
of informed consent.  There will be no patient visits or procedures performed.  Data will be 
unblinded and aggregated for analysis after all patients have been enrolled.    
6.3.1
Study Schedule
N/A
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
26
6.3.2
Informed Consent
This study will operate under a waiver of informed consent.  Due to the nature of the study, it 
is not feasible to consent subjects, as it would severely contaminate the exposure of interest.  
Patients randomized to the control arm would need to be told not to inform their provider of 
their participation in the trial, which would severely compromise the therapeutic relationship. 
The only way to avoid this would be to obtain informed consent from every patient on 
admission to the hospital, but given that the incidence of AKI is approximately 15% of 
admissions, this would be impractical and would be unnecessary for the vast majority of 
patients.  Given that the research proposed presents no more than minimal risk to the 
subjects, and that no procedures are being performed that require consent outside of the 
research paradigm, we will request a waiver of consent from the Institutional Review Board.
6.3.3
Screening
Screening for eligible patients will be done entirely by the algorithm built into our best 
practice alert within the Epic electronic medical record.  This algorithm will identify any 
hospitalized patients at all participating hospitals who have AKI (based on KDIGO creatinine 
criteria).  Upon opening of a patient chart, the algorithm will examine the most recent 
creatinine value against the minimum value in the past 48 hours and 7 days.  If the current 
value is 0.3 mg/dl above the 48 hour minimum, or 50% higher than the 7 day minimum, the 
patient meets AKI criteria.  If the patient meets all other inclusion and exclusion criteria, the 
patient will be automatically enrolled into our study and randomized into the “recommended” 
or “anti-recommended” group.  Whether or not a subject generates an alert in either study 
arm will depend on the uplift score generated for each subject by the uplift algorithm.  Those 
whose uplift score represents a probability of benefit greater than 0.5 will generate an alert in 
the “recommended” study arm, but will not generate an alert in the “anti-recommended” 
study arm.  Those whole uplift score represents a probability of benefit less than 0.5 will not 
generate an alert in the “recommended” study arm and will generate an alert in the “anti-
recommended” study arm. 
6.3.4
Enrollment 
Patients are automatically and immediately enrolled into our study if our best practice alert 
confirms their eligibility as described above.
6.3.5
On Study Visits
N/A
6.3.6
End of Study and Follow-up
There will be no patient follow-up and patients will not be able to withdraw from the study.  
We do not plan to inform subjects of their participation in the trial.  We feel that the results of 
this study will not be pertinent to the subjects, as the effect of the alert is expected to be 
transient and should have no impact on their future quality of life beyond that measured by 
the outcome of the study itself.  Moreover, post-hoc disclosure of the nature of the study to 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
27
the subjects may generate undue stress and concern on the part of the subject as to the 
quality of their overall clinical care.  
We will update all clinicians of the results of this trial after trial completion. We are not 
collecting data that would tie a specific clinician to an outcome. For example, while we would 
inform clinicians that the alerts increased or decreased the rate of certain outcomes, we will 
not be able to tell them what their specific rate of use of best practices are or their individual 
impact on outcomes.
6.3.7
Removal of subjects
N/A
6.4
Statistical Method
6.4.1
Statistical Design
The primary analysis will utilize the intention to treat principle.  The primary outcome (a 
composite of progression of AKI, dialysis, and death) will be compared across the groups 
using the Mantel-Haenszel chi-square test to account for the 4 randomization strata (which 
account for the four enrolling hospitals in the study). Final statistical significance will be 
based on a p-value of <0.044 to account for the interim analysis.
In addition, it is possible that the alert may change clinician behavior over time such that best 
practices are performed at a higher rate even in the absence of the alert.  A retrospective 
examination will be performed to assess improvement in the control group before / after 
intervention as a measurement of contamination.  We expect that providers who receive AKI 
alerts on some patients may be more likely to look for AKI in other patients, potentially 
diluting the effect of the intervention – this pre/post intervention analysis will help in 
assessing that level of contamination.
We have specified several subgroups in whom the benefit of an AKI alert may differ from the 
general population.  These groups will be analyzed as secondary and exploratory analyses.  
They include:
-Surgical patients
-Subjects with baseline creatinine <1.0mg/dl
-Subjects with baseline creatinine <0.5 mg/dl
-Female participants (due to lower rate of increase in creatinine after AKI)
-Black participants (due to higher rate of increase in creatinine after AKI)
-Elderly subjects  (age > 65, age > 70, and age > 75)
-Subjects in an ICU at the time of the alert
-Subjects who enter the study based on relative vs. absolute creatinine criteria vs. 
both
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
28
Finally, as the academic year progresses, new clinicians (such as residents and interns) may 
become more facile in their ability to recognize AKI, which may attenuate the effect of this 
intervention. We will model this in exploratory analyses using an intervention-by-time of year 
interaction term in models of the primary outcome.  
6.4.2
Sample Size Considerations
Based on analysis of the ELAIA-1 trial (restricted to the sites planned for ELAIA-3) we 
estimate that the overall outcome rate in the “anti-recommended” arm will be 24%, and the 
rate in the “recommended” arm will be 20% lower, 19.2% . At a total alpha threshold of 0.05, 
we will have 90% power to detect a difference of at least this great or greater by enrolling 
3,410 individuals. We are increasing this target sample size by 20% to 4,092 to account for 
contamination across the study arms.
6.5
Planned Analyses
6.5.1
Primary Objective Analysis
For the primary analysis, and all comparisons of categorical variables between the 
intervention and control group, we will use the Mantel-Haenszel test, accounting for each 
hospital site as an individual stratum. The Mantel-Haenszel approach will be used to obtain 
the pooled relative risks across hospital strata without adjusting for other baseline factors.  
The primary analysis will examine the rate of the primary outcome in groups defined by 
whether the individual was randomized to “as recommended” vs. “anti-recommended”. The 
effect of alert vs control (which is also randomized in this framework) will be considered a 
secondary outcome.
6.5.2
Secondary Objectives Analyses
Categorical secondary outcomes will be compared across the study groups with the use of 
the Cochrane-Mantel_Haenszel test, similar to the primary analysis. Continuous outcomes 
will be compared using the VanElteren test. Our threshold for statistical significance will be 
p<0.05 for secondary analyses, but given the fact that there will be multiple such analyses 
we acknowledge that any conclusions from those analyses should be considered 
hypothesis-generating.
6.5.3
Exploratory Objectives Analyses (if applicable)
A retrospective examination will be performed to assess improvement in the control group 
before / after intervention as a measurement of contamination.  We expect that providers 
who receive AKI alerts on some patients may be more likely to look for AKI in other patients, 
potentially diluting the effect of the intervention – this pre/post intervention analysis will help 
in assessing that level of contamination.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
29
We have specified several subgroups in whom the benefit of an AKI alert may differ from the 
general population.  These groups will be analyzed as secondary and exploratory analyses.  
They include:
-Surgical patients
-Subjects with baseline creatinine <1.0mg/dl
-Subjects with baseline creatinine <0.5 mg/dl
-Females (due to lower rate of increase in creatinine after AKI)
-African Americans (due to higher rate of increase in creatinine after AKI)
-Elderly subjects  (age > 65, age > 70, and age > 75)
-Subjects in an ICU at the time of the alert
-Subjects who enter the study based on relative vs. absolute creatinine criteria vs. 
both
Finally, as the academic year progresses, new clinicians (such as residents and interns) may 
become more facile in their ability to recognize AKI, which may attenuate the effect of this 
intervention. We will model this in exploratory analyses using an intervention-by-time of year 
interaction term in models of the primary outcome.  
6.5.4
Safety 
The risks to subjects in these trials is minimal, as the studies randomize them to usual care 
versus an alert that simply synthesizes data that is already present in the medical record. 
Potential risks include:
-
Loss of confidentiality, as we will be collecting patient information and data.
o
Risk mitigation: There is limited risk to the loss of confidentiality, as only de-
identified data is being stored for analysis.  All data in this study will be stored 
on a central server within the Clinical and Translational Research Accelerator 
(CTRA). The server is only accessible from within the Yale intranet (or via 
VPN remotely) and additionally requires separate logon username and 
password. 
Data abstracted from the medical record (see Appendix A) will be de-
identified, with a linking file retained in a separate location that will allow for 
future linking of de-identified data to protected health information (PHI) for the 
purpose of potential future studies. Studies that require the use of PHI (for 
example, linking patient info to national outcomes databases) will require 
approval of both the manuscript and executive committees and a separate 
IRB approval. De-identified data will be stored on a secure server, accessed 
via “dumb” terminals, and all analyses will proceed on that server alone. De-
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
30
identified data will be transmitted to outside investigators using secure, 
encrypted channels upon approval of the manuscript committee.
The study is not causing any novel data to be gathered about any patient; it 
only gathers data from already-existing electronic health records. No PHI are 
included in the data being analyzed or published. Study participants therefore 
face no greater dangers of loss of confidentiality than they already face as 
patients.
This data is not subject to any data sharing agreements at this time. As this 
will be a large and useful data set that may be utilized in future research 
protocols, we plan to maintain both the de-identified data set and the linking 
dataset for at least the duration of IRB approval, and will seek permission to 
maintain those files on a yearly basis from the IRB if the research protocol is 
renewed.
-
Overtreatment: If alerts affect physician behavior, then patients randomized to an 
alert arm may be more likely to undergo certain tests or interventions such as fluid 
boluses. These interventions fall within the standard-of-care and may benefit 
patients, but it is also possible that additional interventions may not benefit patients 
and could incur additional costs.
o
Risk mitigation:  We will educate providers that clinical judgement should 
supersede any AKI alert.  While an order set is an optional action for 
providers, the order set was designed so as to not promote or increase the 
use of any one particular therapeutic strategy and includes only benign 
interventions that can be applied to most patients with AKI.
o
In addition, we have text on the alert that reads “The clinician that sees the 
alert and decides to act on it must confirm with the attending the AKI order 
and subsequent SOC treatment to avoid overtreatment”.
-
Inattention to AKI: It is possible that, as providers learn that alerts exist, they become 
less attentive to the presence of AKI in patients randomized to the usual care group. 
o
Risk mitigation: We limit this risk by including the language “THIS ALERT 
DOES NOT FIRE FOR ALL PATIENTS WITH AKI” in each alert.  Additionally, 
our educational efforts (which target each hospital division before the trial and 
periodically after initiation) make explicit the fact that these alerts are being 
conducted in a randomized framework.
-
Alert Fatigue: These studies represent an additional alert to which providers will be 
exposed and prior research has demonstrated that more frequent alerting may lead 
to less attention to other alerts. 
o
Risk mitigation:  We have given providers an option to stop alerts for 
particular patients by “Accepting” the alert, which will suppress the alert for a 
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
31
patient for at least 48 hours.  While alert fatigue is a potential risk, this is also 
a major motivation of this line of research, particularly to develop truly 
targeted alerts that will only display for those most excepted to benefit.  Only 
via randomized trials can truly effective alerts be discovered. Should no effect 
be found in these studies, it will be advised that AKI alerts should not be 
continued at the institutions.
-
Risk of increased signals of harm in the alert group. Our recent AKI alert trial showed 
signals of harm of our general AKI alert, as shown by an increased relative risk of the 
primary outcome, in the two non-teaching hospitals of our study.  
o
Risk mitigation:  We specifically designed this study to target those subjects 
who are expected to benefit most from alerting.  We also have purposely 
included only teaching hospitals in this trial, as they showed no signals of 
harm in our prior alert study.  We plan to perform an interim analysis at 50% 
enrollment.  
6.5.5
Analysis of Subject Characteristics
We will present descriptive statistics as median (interquartile range) for continuous variables 
and proportions for categorical variables.  We will use the Van Elteren test to compare 
continuous variables across the intervention groups, again accounting for hospital strata.   
Statistical significance will be based on a p-value of <0.05.
6.5.6
Interim Analysis (if applicable)
We plan to do three interim analyses, and guidelines will be given to the DSMB for stopping 
the trial for efficacy, harm, and futility.  Our interim analyses will be conducted as follows:
The interim analyses will be performed at 25%, 50%, and 75% enrollment. At the first interim 
analysis, the trial will stop for efficacy if the p-value is < .0016. At the second interim 
analysis, the trial will stop for efficacy or futility if the p-value is < .0048 or > .72 and  < .014 
and > .22 for the third interim analysis. The p-value threshold for the final analysis of the 
primary outcome is set at <0.044.
6.5.7
Handling of Missing Data
Patients discharged prior to 14 days without an outcome of interest will be assumed to be 
free of that outcome at 14 days. Death will be treated as a censoring event in analyses 
where death was not the outcome.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
32
7  Trial Administration
7.1
 Ethical Considerations: Informed Consent/Assent and HIPAA Authorization
This study will be conducted in accordance with the ethical principles of the Declaration of 
Helinski.
This study will operate under a waiver of informed consent.  Due to the nature of the study, it 
is not feasible to consent subjects, as it would severely contaminate the exposure of interest.  
Patients randomized to the control arm would need to be told not to inform their provider of 
their participation in the trial, which would severely compromise the therapeutic relationship. 
The only way to avoid this would be to obtain informed consent from every patient on 
admission to the hospital, but given that the incidence of AKI is approximately 15% of 
admissions, this would be impractical and would be unnecessary for the vast majority of 
patients.  Given that the research proposed presents no more than minimal risk to the 
subjects, and that no procedures are being performed that require consent outside of the 
research paradigm, we will request a waiver of consent from the Institutional Review Board.
7.2
Institutional Review Board (IRB) Review
The protocol will be submitted to the IRB for review and approval. Approval of the protocol 
must be obtained before initiating any research activity. Any change to the protocol or study 
team will require an approved IRB amendment before implementation.  The IRB will 
determine whether informed consent and HIPAA authorization are required.  
The IRB will conduct continuing review at intervals appropriate to the degree of risk, but not 
less than once per year.
A study closure report will be submitted to the IRB after all research activities have been 
completed.   
Other study events (e.g. data breaches, protocol deviations) will be submitted per Yale’s 
IRB's policies.
7.3
Subject Confidentiality
Subject confidentiality is held in strict trust by the research team.  Subject medical record 
review will be limited to the just the elements needed to complete the study.  All data, both 
patient and provider, will be collected electronically and skimmed from the EHR (see Appendix 
A for data to be extracted), processed electronically, encrypted, deidentified, and transferred 
to a secure HIPAA-compliant server with 2-factor authentication for storage and later analysis.  
Data will not be stored on personal computing devices of any kind and will only be accessible 
from within the Yale firewall or via VPN.  The primary dataset will contain no PHI.  We will 
maintain a separate “linking file” that contains all PHI and will be stored on a separate server 
to limit the risk of accidental disclosure.  The linking file is being maintained for potential future 
linking to national databases of death and dialysis.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
33
7.4
Deviations/Unanticipated Problems
If the study team becomes aware of an anticipated problem (e.g. data breach, protocol 
deviation), the event will be reported to the IRB.
A protocol deviation is any noncompliance with the study protocol. The noncompliance may 
be either on the part of the participant, the investigator, or the study site staff. As a result of 
deviations, corrective actions are to be developed by the site and implemented promptly.
It is the responsibility of the site investigator to identify and report all protocol deviations at 
Institutional Review Board (IRB) annual renewal. All deviations must be addressed in study 
source documents, reported to the study sponsor, and the reviewing IRB per their policies.
Unanticipated problems involving risks to participants or others include, in general, any 
incident, experience, or outcome that meets all of the following criteria:
•
Unexpected in terms of nature, severity, or frequency given (a) the research 
procedures that are described in the protocol-related documents, such as the Institutional 
Review Board (IRB)-approved research protocol and informed consent document; and (b) 
the characteristics of the participant population being studied;
•
Related or possibly related to participation in the research (“possibly related” means 
there is a reasonable possibility that the incident, experience, or outcome may have been 
caused by the procedures involved in the research); and
•
Suggests that the research places participants or others at a greater risk of harm 
(including physical, psychological, economic, or social harm) than was previously known or 
recognized.
The investigator will report unanticipated problems (UPs) to the reviewing Institutional 
Review Board (IRB) and to the study sponsor. The UP report will include the following 
information: 
•
Protocol identifying information: protocol title and number, PI's name, and the IRB 
project number;
•
A detailed description of the event, incident, experience, or outcome;
•
An explanation of the basis for determining that the event, incident, experience, or 
outcome represents an UP;
•
A description of any changes to the protocol or other corrective actions that have 
been taken or are proposed in response to the UP.
To satisfy the requirement for prompt reporting, UPs will be reported using the following 
timeline:
•
UPs that are serious adverse events (SAEs) will be reported to the IRB and study 
sponsor, if applicable within 5 working days in accordance with of the investigator becoming 
aware of the event.
•
Any other UP will be reported to the IRB and study sponsor within 10 working days of 
the investigator becoming aware of the problem.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
34
•
All UPs should be reported to appropriate institutional officials (as required by an 
institution's written reporting procedures), the supporting agency head (or designee), and the 
Office for Human Research Protections (OHRP) within & [insert timeline in accordance with 
policy] of the IRB's receipt of the report of the problem from the investigator.
7.5
Data Collection
All data, both patient and provider, will be collected electronically and skimmed from the 
EHR (see Appendix A for data to be extracted), processed electronically, encrypted, 
deidentified, and transferred to a secure HIPAA-compliant server with 2-factor authentication 
for storage and later analysis.  Data will not be stored on personal computing devices of any 
kind and will only be accessible from within the Yale firewall or via VPN.  The primary 
dataset will contain no PHI.  We will maintain a separate “linking file” that contains all PHI 
and will be stored on a separate server to limit the risk of accidental disclosure.  The linking 
file is being maintained for potential future linking to national databases of death and 
dialysis.
7.6
Study Records
The study protocol, subject medical records, data stored on our secure server, and all DSMB 
records will be considered study records.
7.7
Access to Source Documents
Data will be collected solely from the electronic medical record and stored on our secure 
server, accessed only by IRB-approved study personnel.
7.8
Data or Specimen Storage/Security
All data, both patient and provider, will be collected electronically and skimmed from the 
EHR, processed electronically, encrypted, deidentified, and transferred to a secure HIPAA-
compliant server with 2-factor authentication for storage and later analysis (see Appendix A).  
Data will not be stored on personal computing devices of any kind and will only be 
accessible from within the Yale firewall or via VPN.  The primary dataset will contain no PHI.  
We will maintain a separate “linking file” that contains all PHI and will be stored on a 
separate server to limit the risk of accidental disclosure.  The linking file is being maintained 
for potential future linking to national databases of death and dialysis.
7.9
Retention of Records
All data collected will be stored for at least one year after study close-out, with continued IRB 
approval for each additional year of storage.
7.10 Study Monitoring
All study monitoring will be performed by the Principal Investigator, Francis P. Wilson, on an 
as needed basis.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
35
7.11 Data Safety Monitoring Plan
The principal investigator is responsible for monitoring the data, assuring protocol 
compliance, and conducting the safety reviews biannually.  During the review process the 
principal investigator will evaluate whether the study should continue unchanged, require 
modification/amendment, or close to enrollment.  The principal investigator and the 
Institutional Review Board (IRB) have the authority to stop or suspend the study or require 
modifications.
Based on new information obtained from our recent general AKI alert study, we are taking 
necessary precautions to maximize patient safety in this trial.  While we still feel that this is a 
minimal risk trial, we plan to have a Data Safety Monitoring Board review our results at three 
separate interim analyses.  The first will be at 25% enrollment across our four teaching 
hospitals and the second and third will be at 50% and 75%. Any signal of harm will suspend 
the study to further enrollment.
The DSMB will convene at least three times for each of the interim analyses, and again 
should our periodic analysis of safety outcomes and adverse events show significant 
difference between study arms.  In each case, the DSMB will make a determination whether 
the trial should continue or be paused.
This protocol presents minimal risks to the subjects and Unanticipated Problems Involving 
Risks to Subjects or Others (UPIRSOs), including adverse events, are not anticipated. In the 
unlikely event that such events occur, Reportable Events (which are events that are serious 
or life-threatening and unanticipated (or anticipated but occurring with a greater frequency 
than expected) and possibly, probably, or definitely related) or Unanticipated Problems 
Involving Risks to Subjects or Others that may require a temporary or permanent interruption 
of study activities will be reported immediately (if possible), followed by a written report within 
5 calendar days of the Principal Investigator becoming aware of the event to the IRB (using 
the appropriate forms from the website) and any appropriate funding and regulatory 
agencies. The investigator will apprise fellow investigators and study personnel of all 
UPIRSOs and adverse events that occur during the conduct of this research project through 
regular study meetings and via email as they are reviewed by the principal investigator. The 
protocol’s research monitor(s), e.g., DSMBs, study sponsors, funding and regulatory 
agencies, and regulatory and decision-making bodies will be informed of all adverse events 
within 5 days of the event becoming known to the principal investigator.
7.12 Study Modification
Any modifications will be added as tracked changes to the current protocol and submitted for 
IRB review.  The change will be implemented into the study only after full IRB approval.
7.13 Study Discontinuation
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
36
If the study is prematurely terminated or suspended, the Principal Investigator (PI) will 
promptly inform the Institutional Review Board (IRB) and will provide the reason(s) for the 
termination or suspension.  
 Circumstances that may warrant termination or suspension include, but are not limited to:

Determination of unexpected, significant, or unacceptable risk to participants

Demonstration of efficacy that would warrant stopping   

Insufficient compliance to protocol requirements

Data that are not sufficiently complete and/or evaluable

Determination that the primary endpoint has been met

Determination of futility
7.14 Study Completion
Study completion will occur after all patients have been enrolled and data has been 
analyzed.  The IRB will be notified of study completion.
7.15 Conflict of Interest Policy
The independence of this study from any actual or perceived influence, such as by the 
pharmaceutical industry, is critical. Therefore, any actual conflict of interest of persons who 
have a role in the design, conduct, analysis, publication, or any aspect of this trial will be 
disclosed and managed. Furthermore, persons who have a perceived conflict of interest will 
be required to have such conflicts managed in a way that is appropriate to their participation 
in the trial. The study leadership in conjunction with the appropriate conflict of interest review 
committee has established policies and procedures for all study group members to disclose 
all conflicts of interest and will establish a mechanism for the management of all reported 
dualities of interest.
 All investigators will follow the applicable conflict of interest policies.
7.16 Funding Source
The study is funded by a grant from the NIDDK (NIH K23DK097201.)
7.17 Publication Plan
It is anticipated that publications and presentations will be generated from the findings of this 
study.  The Principal Investigator, Francis P. Wilson, holds primary responsibility for 
publishing results.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
37
8 Appendices
Appendix #
Title
Section
Topic
A
Data Extraction
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
38
9 List of Tables
10 References
.         Uchino S, Bellomo R, Goldsmith D, Bates S, Ronco C. An assessment of the RIFLE 
criteria for acute renal failure in hospitalized patients. Crit. Care Med. 
2006;34(7):1913-1917.
2.
Chertow GM, Soroko SH, Paganini EP, et al. Mortality after acute renal failure: 
models for prognostic stratification and risk adjustment. Kidney Int. 2006;70(6):1120-
1126.
3.
Lombardi R, Nin N, Lorente JA, et al. An assessment of the Acute Kidney Injury 
Network creatinine-based criteria in patients submitted to mechanical ventilation. 
Clin. J. Am. Soc. Nephrol. 2011;6(7):1547-1555.
4.
Nin N, Lombardi R, Frutos-Vivar F, et al. Early and small changes in serum 
creatinine concentrations are associated with mortality in mechanically ventilated 
patients. Shock. 2010;34(2):109-116.
5.
Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work 
Group. KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 
2012;2(Suppl.):1-138.
6.
Balasubramanian G, Al-Aly Z, Moiz A, et al. Early nephrologist involvement in 
hospital-acquired acute kidney injury: a pilot study. Am. J. Kidney Dis. 
2011;57(2):228-234.
7.
Bihorac A, Yavas S, Subbiah S, et al. Long-term risk of mortality and acute kidney 
injury during hospitalization after major surgery. Ann. Surg. 2009;249(5):851-858.
8.        Wilson FP, Bansal AD, Jasti SK, et al. The impact of documentation of severe acute 
kidney injury on mortality. Clin. Nephrol. 2013.
9.         Lachance P, Villeneuve PM, Rewa OG, et al. Association between e-alert 
implementation for detection of acute kidney injury and outcomes: a systematic 
review. Nephrol. Dial. Transplant. 2017;32(2):265-272.
10.
Park S, Baek SH, Ahn S, et al. Impact of Electronic Acute Kidney Injury (AKI) Alerts 
With Automated Nephrologist Consultation on Detection and Severity of AKI: A 
Quality Improvement Study. Am. J. Kidney Dis. 2018;71(1):9-19.
11.
Selby NM. Electronic alerts for acute kidney injury. Curr. Opin. Nephrol. Hypertens. 
2013;22(6):637-642.
12.       Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload 
and missed test results in electronic health record-based settings. JAMA Intern Med. 
2013;173(8):702-704.
13.
Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial 
alerts: findings from a randomized controlled study. J Am Med Inform Assoc. 
2012;19(e1):e145-148.
14.
Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended 
consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007:26-
30.
15.       Kuusisto F, Santos Costa V, Nassif H, Burnside E, Page D, Shavlik J. Support Vector 
Machines for Differential Prediction. Mach Learn Knowl Discov Databases. 
2014;8725:50-65.
16.       Kan M, Shan S, Chen X. Multi-view Deep Network for Cross-view Classification.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025

APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025
Protocol Number
Version Date and Version #
39
17.
Guelman L, Guillén M, Pérez-Marín AM. Uplift Random Forests. Cybernetics and 
Systems. 2015;46(3-4):230-248.
18.       Rzepakowski P, Jaroszewicz S. Decision trees for uplift modeling with single and 
multiple treatments. Knowledge and Information Systems. 2012;32(2):303-327.
19.
Sołtys M, Jaroszewicz S, Rzepakowski P. Ensemble methods for uplift modeling. 
Data Mining and Knowledge Discovery. 2015;29(6):1531-1559.
APPROVED BY THE YALE UNIVERSITY IRB 2/26/2025