IMPROVE AKI: A Cluster-Randomized Trial of Team-Based Coaching Interventions to IMPROVE Acute Kidney Injury
Phase: NA
Conditions: Acute Kidney Injury
Sponsor: Dartmouth-Hitchcock Medical Center

 
 
 
2020 
IMPROVE AKI Cluster-
Randomized Trial 
(IMPROVE-AKI) 
NCT03556293 
09-21-2020 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
1 | P a g e  
IMPROVE AKI: A Cluster-Randomized Trial of Team-Based Coaching 
Interventions to IMPROVE Acute Kidney Injury 
Objective: 
We propose to test the implementation of evidence-based preventive interventions 
through a Virtual Learning Collaborative (VLC) with and without the novel use 
Automated Surveillance Reporting (ASR) intervention to change clinical practice and 
improve patient safety in common diagnostic procedures. We focus this test on 
prevention of acute kidney injury (AKI) following diagnostic or interventional cardiac 
catheterization (herein referred to as “procedure”).6 Over 2 million people in the United 
States undergo this procedure each year. AKI, a patient safety metric set by the 
National Quality Forum, occurs in up to 14% of all patients following a procedure and up 
to 50% in patients with pre-existing chronic kidney disease (CKD)1 making AKI this 
procedure’s most prevalent adverse event.2,3,7 When AKI occurs, patients have an 
increased risk of cardiovascular events, prolonged hospitalization, end-stage renal 
disease, and mortality.8 We have shown the inconsistent application of standardizing 
orders, using adequate oral and IV fluids, and limiting contrast dye dose is responsible 
for a five-fold variability in the incidence of post-procedure AKI across hospitals.1,9,10 
 
Our group and others have contributed to an evidence-based of widely accepted 
interventions to prevent AKI in patients undergoing cardiac catheterization.1,3,11-13 These 
interventions, however, are rarely implemented, leaving many patients at risk.1 The 
critical research question is not what hospitals should do, but how to get them to do it. 
Our own work has demonstrated the feasibility and potential effectiveness of two 
promising implementation strategies to increase the use of AKI prevention protocols: 1. 
Virtual Learning Collaborative (VLC) to support the use of an AKI Prevention Toolkit 
showing a statistically significant 28% reduction in AKI (piloted in a 10-hospital trial: 
AHRQ grant HS018443), and 2. Automated Surveillance Reporting (ASR) providing 
near-realtime feedback to frontline care workers (VA HSRD grant IIR11292).1,7,14 
 
The proposed study asks whether supporting the use of the AKI Prevention Toolkit by 1) 
VLC coaching augmented by ASR (VLC+ASR) will lead to better patient outcomes 
compared to TA, TA + ASR, and VLC alone; and 2) VLC will be superior to TA with or 
without ASR. We will address these questions in a 2x2 factorial cluster-randomized trial 
that randomizes 16 hospitals to receive one of the following interventions for 18-months: 
A) Technical Assistance (TA); B) Technical Assistance with Automated Surveillance 
Reporting (TA+ASR); C) Virtual Learning Collaborative (VLC) with team-based 
coaching; and D) Virtual Learning Collaborative with Automated Surveillance Reporting 
(VLC+ASR). All sites have been recruited and will receive the AKI Prevention Toolkit 
(Appendix 1) that includes 3 core preventive interventions: 1. Standardized order sets; 
2. IV and oral fluids; and 3. Reduced contrast volume. The interventions were 
developed and tested in our pilot to implement AKI preventive strategies.1,3 
 
 
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
2 | P a g e  
Specific Aims 
Specific Aim 1: Compare the efficacy of a Virtual Learning Collaborative and/or 
Automated Surveillance Reporting compared to Technical Assistance to reduce 
the incidence of AKI. 
Our working hypothesis is that multi-disciplinary clinical teams in a Virtual Learning 
Collaborative (VLC) with team-based coaching in process measurement and 
implementation methods will reduce the incidence of AKI following procedures, 
compared to a Technical Assistance (TA) intervention, both with or without Automated 
Surveillance Reporting (ASR). We also hypothesize that ASR will have a bigger impact 
with VLC than with TA. 
 
Specific Aim 2: Evaluate the sustained efficacy of Virtual Learning Collaborative 
and/or Automated Surveillance Reporting to reduce the incidence of AKI 
following the intervention period. 
Our working hypothesis is that VLC coaching augmented by ASR (VLC+ASR) will be 
superior to TA, TA +ASR, and VLC alone. Secondarily, VLC will be superior to TA with 
or without ASR. Sixteen hospitals will continue to enroll patients following the 18-month 
intervention period for an additional 18-months post-intervention with the TA, ASR, and 
VLC interventions removed. We will evaluate whether the reduced incidence of AKI will 
be sustained for each of the randomized clusters during the post-intervention phase. 
 
Impact: Evidence of the efficacy of Virtual Learning Collaborative, augmented by 
Automated Surveillance Reporting, on the sustained use of preventive interventions will 
enable hospitals to improve quality of care for the over 2 million patients undergoing 
cardiac catheterization and, as a result, reduce their significant risk of acute kidney 
injury. More broadly, these findings will be relevant to supporting hospital’s 
implementation of a wide array of preventive interventions and has the potential for 
vastly improving patient care and outcomes. 
 
Specific Aim 3: Laboratory Predictors of AKI in Patients Undergoing Coronary 
Angiography 
In th is aim, our goal is to examine whether lower serum magnesium, lower hemoglobin, 
higher hemoglobin A1c, and other laboratory predictors are associated with a higher risk 
of AKI in patients undergoing coronary angiography. We are defining AKI as a doubling 
of serum creatinine (SCr) or need for RRT. We will also examine whether laboratory 
predictors are associated with an increased risk of death at 30 days. 
……………………………………………………………………………………………………… 
 
A. Background and Significance: 
Over two million diagnostic or interventional cardiac catheterization procedures are 
performed in the United States each year.6,15 Acute kidney injury (AKI) is preventable 
but occurs in up to 14% of all patients following a procedure and up to 50% in patients 
with pre-existing chronic kidney disease (CKD),1 making AKI one of the most prevalent 
adverse events.2,3,7 Reducing the prevalence AKI is a patient safety objective set forth 
by the National Quality Forum.16 When AKI occurs, patients have an increased risk of 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
3 | P a g e  
cardiovascular events, prolonged hospitalization, end-stage renal disease, all-cause 
mortality, and increased acute care costs of over $7,500 per case.8,11,17 Based on our 
preliminary evidence in our pilot demonstrating a statistically significant 28% reduction 
in AKI in patients with pre-existing CKD, we estimate that over 70,000 AKI events could 
be prevented with an annual cost savings of $525 million each year nationally using 
evidence based strategies delivered through a Virtual Learning Collaborative (VLC) in 
common diagnostic procedures using contrast dye.  
 
Scientific Premise: The premise is that a parsimonious set of two evidence-based 
preventive interventions have been shown to substantially reduce the incidence of post-
procedure AKI; however, these preventive interventions are applied inconsistently. We 
have strong preliminary evidence suggesting that more systematic application of these 
preventive interventions can dramatically reduce the complication of AKI and reduce 
costs. The scientific premise of this application rests on three assertions: 1) a 
parsimonious set of evidence-based preventive interventions are effective in preventing 
AKI, 2) these preventive interventions are applied inconsistently, among hospitals, and 
3) our preliminary data from a large pilot of our interventions suggest AKI can be 
prevented. There are two widely accepted preventive interventions to prevent AKI after 
procedures: 1) standardizing orders for intravenous (IV) volume expansion, and 2) 
limiting contrast volume. For intravenous (IV) fluid administration, three prospective 
randomized trials in over 1,000 patients have compared giving fluid to no fluid.18-20 
These trials included procedures where the urgency to restore flow to an occluded 
coronary artery preempted the ability to administer IV fluids before contrast exposure. In 
all three trials, a significant reduction in the incidence of AKI was seen in those who 
received fluid compared to no fluid.18-20  
 
With respect to the second premise, our preliminary data suggests the 
inconsistent application of AKI preventive interventions is responsible for five-fold 
variability in AKI incidence after cardiac catheterization.1 We collected this evidence 
from a sample of 10 hospitals with interviews and data collection to ascertain the rate of 
AKI and determine the presence or absence of protocols for AKI prevention. Only one in 
five hospitals had established protocols for 
preventing AKI. Hospitals without established 
protocols were found to have significantly higher 
rates of AKI.1 Since our evidence was limited to a 
regional collaborative, we validated our findings in a 
VA nationwide survey. The VA survey confirmed our 
findings, where only one in four hospitals with 
cardiac catheterization laboratories had a mandated 
IV fluid protocol for AKI prevention. Therefore, our 
objective is to increase the consistency with which 
AKI preventive approaches are applied, which was 
demonstrated by a 28% reduction in AKI in our 
preliminary data and pilot intervention described 
below.3  

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
4 | P a g e  
With respect to the third premise, we tested a VLC coaching intervention that 
employed best-practice approaches to prevent AKI (HS018443). This pilot was a pre-
post implementation design in 10 hospitals with 21,067 consecutive PCI patients 
(Figure 1), which included 4,131 patients with pre-existing CKD.1,3 We observed a 28% 
reduction in AKI for patients with pre-existing CKD (≥Stage 3).3 Strengths included 
multiple intervention hospitals, and a large sample, demonstrating a strong benefit of 
team-based coaching and bundling preventive strategies to prevent AKI. Therefore, we 
will build on our pilot to conduct a national cluster-randomized controlled trial.  
 
Aim 1. Supporting Literature. AKI Prevention. Avoiding potentially nephrotoxic 
contrast volumes during cardiac catheterization is a target for AKI prevention.11 The link 
between contrast volume and AKI has been well established.21 The consistent 
application of reducing contrast volume and avoiding volumes exceeding the safe 
threshold is an essential protocol in our proposed interventions. There have been no 
randomized contrast dosing trials limiting contrast volume, yet limiting contrast is 
supported in the guideline.13  
 
While it is known that IV volume expansion and reducing contrast dose are 
beneficial, these preventive measures for AKI are inconsistently applied.1 To date, there 
have been no empirical evaluations; a logical next step would be to empirically evaluate 
implementation methodologies to prevent AKI in cluster-randomized control trials. 
Based on our pilot, we propose VLC and ASR to be the vehicles for the consistent 
application of AKI preventive strategies.  
 
Aim 1. Preliminary Data. Virtual Learning Collaboratives (VLC). Despite widespread 
use and publication of learning collaboratives, such as the Institute for Healthcare’s (IHI) 
Breakthrough Series in health care settings,22,23 to our knowledge there are no studies 
testing the effectiveness of a plug-in automated surveillance reporting (ASR) 
intervention with or without a learning collaborative. Few randomized trials have 
evaluated learning collaboratives using quantitative outcomes.24 According to a recent 
systematic review of 23 learning collaborative studies, only five used a controlled 
design, with only one RCT measuring quantitative outcomes; none compared 
effectiveness to an active control condition.23 Research findings are also limited to 
studying face-to-face collaboratives,23,24 which entail significant costs associated  
with travel, lodging, meals, and meeting facilities. A recent refinement is the VLC, 
conducted by video, phone, and e-mail, decreasing the cost of participation and 
increasing the speed of incorporating lessons into practice. There is limited but 
encouraging data on VLC effectiveness. A 2006 IHI study of a virtual collaborative to 
improve primary-care access achieved outcomes comparable to those of a traditional 
collaborative at a reduced cost.25 A 2011 cluster-randomized trial assigned 60 hospitals 
to use either VLC (n=31) or a toolkit (n=29) for prevention of nosocomial infections.26 
The VLC outpaced the toolkit in changing care processes, though neither approach 
improved outcomes. These reports suggest VLC may be effective but requires more 
testing. 
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
5 | P a g e  
A.1. Critical need to test TA. There is a critical need to evaluate the VLC and ASR 
interventions against TA alone to determine if the null hypothesis is supported that there 
is no difference in TA compared to more intensive interventions such as the VLC with or 
without ASR. If the null hypothesis is not supported, the AKI Prevention Toolkit could be 
rolled out nationally in a more rapid dissemination design compared to that of a national 
campaign involving a more robust VLC and/or ASR.  
 
A.2. Automated Surveillance Reporting (ASR). The landmark prediction model for 
AKI was established by Mehran in 2004 on a large PCI cohort. However, the 
discrimination of the model was poor (c-statistic=0.67).27 It is well known that models 
have degrading calibration performance over time and across different practice 
settings.28,29 These prediction models have not yet been incorporated into surveillance 
systems that could provide real-time feedback to operators on both the risk of AKI and 
AKI incidence.14 ASR is important and useful in detecting and understanding changes in 
performance. We are the first to develop an ASR toolkit for medical product comparative 
effectiveness and institutional and provider care variation detection,30-33 and further 
adapted it to conduct surveillance for post-catheterization AKI in the VA health system 
(IIR11292). The toolkit uses the new VA AKI prediction model developed by the PIs14 
and provides tailored automated reports to VA operators on AKI outcomes to provide 
teams with near-real-time data on AKI outcomes and consistent adoption of protocols. 
ASR can be used to identify problems, encourage the need for improvement, and 
quantify the extent to which improvement initiatives have been successful.34 It is well 
established by that participating in an outcome monitoring program can improve patient 
outcomes, provide clinicians with targeted data for improved decision-making, and lower 
costs of care.35  
 
A.3. Expertise in Virtual Learning Collaboratives (VLC). In 2011, the VA National 
Center for Patient Safety (Zubkoff, R01MH102325) had initial success with the use of a 
VLC to reduce postoperative respiratory failure.36 Learning sessions and presentations 
were held via teleconferences, and all materials were distributed using a listserv and 
collaborative website. One unique component of this VLC was the use of coaches to 
assist teams in making improvements. Coaches provided feedback to teams on monthly 
reports and held group calls to foster sharing and communication among teams. Over 6 
months, teams attended calls and submitted monthly progress reports. Over 76% of 
team members attended all 11 calls; each team implemented an intervention; 44% 
implemented at least 4 unique evidence-based interventions, and 31% implemented at 
least 5 out of 11 interventions.36 This project observed improved outcomes among 
participating teams suggesting the VLC is a useful improvement strategy. Zubkoff and 
colleagues have continued to conduct additional VLCs in the VA to prevent falls, 
hospital-acquired conditions, and pressure ulcers. Our research team is experienced in 
national VLC (Zubkoff, R01MH102325) implementing a health promotion and physical 
activity program for people with serious mental illness in community mental health 
centers. In this study, 48 community mental health centers are randomized to a VLC or 
to TA. This is, to our knowledge, the first national study to compare the VLC 
effectiveness with or without ASR, and its methods used and lessons learned to inform 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
6 | P a g e  
the proposed work. The proposed study addresses a gap in knowledge about the 
effectiveness of VLC compared to TA, as well as the effectiveness of enhanced data 
support as part of a VLC.  
 
A.4. Expertise in Automated Surveillance Reporting (ASR). Our team has expertise 
in the development and adaptation of statistical surveillance methods,33,37 risk prediction 
models in cardiology,28,38 hierarchical modeling in medical device trials,39-41 and 
informatics surveillance and clinical decision support tools42-45 to conduct retrospective 
and prospective surveillance of electronic health record and clinical registry derived 
patient data for the purposes of detecting institutional and provider variation31,33 and for 
medical product comparative effectiveness.45 As part of research funded by NIH, NLM, 
FDA, and VA, we developed an automated surveillance engine (Data Extraction and 
Longitudinal Time Analysis [DELTA]) and used for medical device comparative 
effectiveness trials, and institutional and operator quality surveillance initiatives.30-33,42 
 
As an example of prior impact, we used the risk-adjusted sequential probability 
ratio testing and automated surveillance tools to evaluate a single center cardiac 
catheterization registry from 2002 to 2006 for operator outliers with respect to all-cause 
mortality.33 When compared with national expectations, the institution was found to be 
within expectations for the outcome, and only one of 18 operators was found to be an 
outlier. Chart review found that the operator in question experienced a high incidence of 
compassionate use (salvage procedure or not a surgical candidate). This risk factor was 
not included in the risk adjustment models, and after excluding all such patient cases, 
the operator’s risk-adjusted mortality rate was within expectation. This resulted in a local 
policy discussion and informed our work in rolling out a compassionate use variable in 
the Massachusetts mandatory registry, which improved risk adjustment model 
performance.46 
 
Most relevant to this proposal, we recently improved on the Mehran AKI risk 
score (c-statistic=0.67)27 by developing an AKI prediction model in 115,633 diagnostic 
cardiac catheterization and PCI patients in the VA (c-statistic=0.74) with external 
validation in 10 non-VA hospitals.14 We have adapted our surveillance toolkit to survey 
AKI in the VA health system in a study concluding this year (IIR11292). The surveillance 
toolkit uses the new VA AKI prediction model and sends tailored automated reports to 
VA operators on AKI outcomes. We used the newly developed risk model to measure 
risk-adjusted institutional variation for each of the VA’s catheterization laboratories, 
using the methodology for the ASR tool from our prior work.31, 32 The risk-adjusted 
sequential probability ratio test (RA-SPRT) found 8 sites that were worse than expected 
for AKI event rates and at least 2 sites that were above expectation for 4 years of the 5-
year analysis period. In the same period, 7 sites were better than expected for 2 years, 
and 2 were better than expected for 4 years of the analysis period.  
 
Aim 2. Supporting Literature. Besides our publication of the pilot intervention,1, 3 there 
is no evidence to document the consistent application of these evidence-based 
preventive measures for AKI, or evidence to document if an intervention is effective 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
7 | P a g e  
after the implementation phase. Implementation trials have not evaluated the 
sustainability of the interventions deployed. It is imperative we test the comparative 
effectiveness of our interventions and determine if they result in sustainable 
improvements after they are removed.  
 
Aim 2. Preliminary Data. In our pilot intervention 
(HS018443), we coached and trained teams in methods 
to implement AKI preventive approaches in their 
respective centers. During a 32-month intervention 
period, we observed a sustained improvement in the 
intervention (Figure 2). In Figure 2, adjusted rates of 
AKI are plotted by month, stratified by intervention (red) 
and benchmark (blue) hospitals, using interrupted time 
series analysis. The vertical dashed line marks the start 
of the pilot intervention.3 We identified cultural 
improvements in the awareness of AKI, as well as 
significant changes in the adoption of standardized IV fluid protocols. However, we did 
not test whether removing the intervention would sustain a significant reduction in AKI. 
In our proposed study, we will evaluate AKI reduction 18 months after the interventions 
are removed and determine whether or not the intervention clusters will demonstrate a 
sustained reduction in AKI or regress back to a baseline AKI incidence.  
 
A.5. Significance of the Expected Research Contributions Impact of proposed 
study: Our overall objective is to develop successful implementation strategies to 
prevent avoidable complications of common diagnostic procedures in complex patients 
such as cardiac catheterization and AKI. We will address how to implement established 
methods to prevent AKI in patients undergoing cardiac catheterization into routine 
clinical practice. Through our proposed research, we will directly inform several of the 
NIDDK priority areas, including 1) Reducing the incidence of AKI; 2) “Prevention, 
treatment and management strategies of AKI… with the goal of improving short- and 
long-term outcomes including morbidity, mortality, progression of CKD, functional 
independence and quality of life;” 3) “Build upon the emerging opportunities that are the 
fruits of past research investments.” In addition: 4) This proposal aims to deploy and 
evaluate the effectiveness of the ASR tools with or without learning collaboratives, 
which are all open source and can be rapidly deployed in other health care systems to 
evaluate the more rapid adoption and implementation of evidence-based healthcare for 
kidney disease; and 5) The successful completion of the study will provide a vehicle for 
rapid dissemination AKI preventive strategies nationally. 
……………………………………………………………………………………………………… 
 
B. INNOVATION  
It is estimated that over 70,000 events of AKI could be prevented in the US each year if 
evidence-based preventive strategies were implemented consistently in usual care for 
the common diagnostic procedures using contrast dye. Despite compelling evidence 
and guideline recommendations, few hospitals have consistently applied AKI preventive 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
8 | P a g e  
measures into routine clinical practice.1, 47 A new era of clinical trials evaluating the 
comparative effectiveness of automated surveillance reporting (ASR) with or without 
learning collaborative targeting the implementation and sustained application of 
preventive intervention for AKI is warranted. The proposed research is innovative, in our 
opinion, as follows:  
ü First national implementation trial using automated surveillance reporting (ASR) 
with or without a virtual learning collaborative (VLC) or technical assistance (TA) 
to improve patient safety endpoints.  
ü First national implementation trial testing AKI preventive interventions using ASR, 
VLC and TA approaches.  
ü Provides novel development and evaluation of sustainability and fidelity methods 
in AKI.  
 
………………………………………………………………………………………………………  
C. Research Design and Methods:  
C.1. Specific Aim 1: Compare the efficacy of a Virtual Learning Collaborative 
and/or Automated Surveillance Reporting compared to Technical Assistance to 
reduce the incidence of AKI.  
Our working hypothesis is that VLC coaching augmented by ASR (VLC+ASR) will be 
superior to TA, TA + ASR, and VLC 
alone. Secondarily, VLC will be 
superior to TA with or without ASR. 
 
C.2. Research Design. We will 
conduct a prospective cluster-
randomized 2x2 factorial design 
controlled trial of adult patients 
undergoing cardiac catheterization 
with or without intervention within 
the VA for the primary endpoint of 
AKI. A 2x2 factorial design is ideal 
to determine the effect of VLC 
compared to TA and the effect of 
ASR compared to no ASR while 
guarding against confounding by 
the other factor.48 This study design 
also allows for the interaction of the 
two factors (i.e., the extent to which the effect of one factor varies across the levels of 
the other) enabling a maximally efficient determination of the optimal intervention 
strategy. Hospitals will be randomized to receive one of the following interventions: (A) 
TA; (B) TA+ASR; (C) VLC; (D) VLC+ASR (see Figure 3). the target number of patient 
records that will be reviewed from the EHR will be approximately 10,576 (which is 
approximately 2,644 in each of the four arms of the study (TA, TA+ASR, VLC, 
VLC+ASR) with a minimum of four hospitals within each arm (average 661 patients per 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
9 | P a g e  
hospital)). The intervention phase in all arms will be for 18- months from 7/1/2018 to 
12/31/2019. Interventions will stay active during the 18-month post-intervention phase. 
 
In contrast to a one-factor study or an imbalanced two-factor study, a factorial 
design is advantageous because it allows one to determine firstly whether the factors 
interact and then simplifying the analysis if there is no interaction to more precisely infer 
the main effect of each factor all without relying on the validity of an assumed statistical 
model for the outcomes. We hypothesize that the VLC intervention may have a different 
effect size depending on whether ASR is used, and this method guards against 
confounding by the other factor and allows a robust, perhaps more precise, assessment 
of both factors separately. This study design assumes that each intervention is 
implemented independently, meaning that the TA and VLC, and ASR and no ASR will 
all be implemented separately, and the presence of one will not change the intervention 
of the other. 
 
The proposed trial is guided by the Promoting Action on Research 
Implementation in Health Services (PARIHS) framework.49-55 The PARIHS framework 
and its three dimensions (context, facilitation, and evidence) was selected for 
addressing these barriers to implementation in AKI prevention.54 
 
C.2.a. Study Population: Site Inclusion and Exclusion Criteria. All patients at the 
participating sites will be enrolled into the trial with a series of inclusion and exclusion 
criteria.  
VA Site Inclusion Criteria: All VA medical centers with a cardiac catheterization 
laboratory are eligible, and all catheterization operators at a site are included.. We 
currently have over-recruited 29 VA sites, but a minimum of 16 VA sites will be enrolled.   
 
Patient Inclusion Criteria: Among these sites, only patients aged 18 or greater 
who undergo diagnostic coronary angiography or percutaneous coronary intervention 
(PCI or angioplasty) will be enrolled in the trial. CKD is of primary interest and it will be 
determined by a pre-existing CKD diagnosis in the VA medical record, or by two or 
more estimated glomerular filtration rates <60 (ml/min/1.73 m2) at least 90-days apart 
prior to presentation. Patient Exclusion Criteria: Patients with a history of dialysis 
(hemodialysis, peritoneal dialysis) or under the age of 18 will be excluded. Cardiac 
catheterization (also referred to as coronary angiography) is defined as a procedure in 
which a catheter is inserted into the femoral or radial artery and is threaded to the 
cardiac vasculature where radio-contrast dye is administered and a series of x-rays 
obtained in order to visualize the coronary arterial anatomy. PCI is when a clinical 
intervention is then performed to address any treatable pathology that is found. 
 
C.2.b. Randomization. Randomization will occur at the hospital level in a 2x2 factorial 
design.48 We will enroll a minimum of 16 Veteran Affairs (VA) hospitals with cardiac 
catheterization laboratories from across the United States (see Letters of Support and 
Facilities) that meet the medical center inclusion/exclusion criteria above. We will use a 
random number generator and block-randomize centers by the following variables: 1) 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
10 | P a g e  
VA region (1-4); and 2) patient volume (<800, 800+), see Table 1 for 
the sites that initially met out inclusion/exclusion criteria.  
 
C.2.c. Primary Endpoint. KDIGO AKI defined by ≥0.30 (mg/dL) or 
≥50% increase in serum creatinine over baseline within 48-hours of 
the procedure or within 7-days for in-patients, or onset of dialysis 
within 7-days.13 
 
C.2.d. Data Collection and Data Coordinating Center. We have 
experience in using existing electronic health record (ViSTa, CPRS) 
and CART program quality initiative data collection to capture all 
patient and procedural characteristics, comorbidities, and outcomes 
in order to identify patients for pre-existing CKD, ascertain AKI 
outcomes, and to provide the data infrastructure to support the ASR 
intervention. In order to conduct national risk-adjusted surveillance 
for participating sites, we will collect data for the first 6 months after study approval at, in 
each of the 76 VA institutions with cardiac catheterizations laboratories.14 The CART 
data is collected through a structured data entry tool in the EHR. As part of a prior 
research project (VA HSR&D IIR 11-292), we have established a comprehensive 
automated data center with weekly feeds from CART and daily to weekly feeds from the 
data domains in the VA Corporate Database Warehouse (CDW), which extract data 
from the production EHR. For those few data sources with a delay more than a week, 
we have a supplemental automated tool that uses a VA approved interface layer to 
extract the data in near-real time from the production EHR. These data are transformed 
and merged through a validated process into an analytic data cube that can be 
analyzed to provide reports to operators and hospitals as well as providing data for 
review. A summary of the currently available data elements from the CART-CL clinical 
registry and EHR are shown in Appendix 1. Individual clinical registry elements 
conform to the American College of Cardiology National Clinical Data Repository data 
element definitions.56 The research team, including the director of CART (CART Letter), 
has a long history of utilizing these data sources for research. 
 
Due to the 2020 COVID-19 pandemic, many of our enrolled VA site cardiac 
catheterization laboratories have been temporarily closed or procedures have been 
greatly reduced. Disruption to normal laboratory procedures could impact the IMPROVE 
AKI study findings. Therefore, our research team is requesting access the COVID-19 
Shared Data Resource in VINCI, containing information relevant to COVID-19 inside 
and outside the VA.  
 
We will be using the COVID Shared data resource to identify overall rates of testing and 
COVID positivity at each site that has a cath lab to get a sense of COVID impact at that 
site.   We will also be linking each of the cath patients to COVID testing status to get a 
sense of how often they were tested, and how many were cath'ed and COVID+.   Lastly, 
we will assess whether COVID+ status had enough patients amonth cath patients to 
warrant inclusion as a risk factor, since AKI is a risk factor from covid.   We will not be 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
11 | P a g e  
analyzing any case level data from non-cath patients, but will use the COVID shared 
resource to aggregate as above. 
 
C.3. AKI Prevention Toolkit. Intervention: All sites will receive the AKI Prevention 
Toolkit (Appendix 1) including 3 core interventions: 1. Standardized order sets; 2. IV 
and oral fluids; and 3. Reduced contrast volume. The interventions adhere to the 
KDIGO guidelines13 and add interventions developed and tested in our pilot intervention 
to implement AKI preventive strategies.1,3 We will also educate the sites on the 
hemodynamic guided fluid administration for volume expansion guidelines and 
PRESERVE results when released.57,58 
 
C.4. Virtual Learning Collaborative (VLC). Intervention: The VLC will be offered to 
eight teams (minimum of 4 with ASR, minimum of 4 without ASR) and will receive the 
AKI Prevention Toolkit.1,3 Each participating site will be supported to establish a 
multidisciplinary team charged with continuously improving AKI, which will include 
interventional cardiologists, cardiac catheterization lab manager and technicians, 
nursing representatives from the intensive care unit and/or holding areas, cardiology 
administration, nephrology, and representation from the quality improvement 
department (VA Clinical Application Coordinator [CAC] and Systems Redesign). Each 
VLC site team will be assigned 2 expert coaches from White River Junction VA (VA 
affiliate of Dartmouth): AKI quality improvement specialist and VLC and improvement 
specialist. A 60-minute VLC training call will be held monthly during the trial. The VLC 
and VLC+ASR will be coached independently to avoid contamination.  All VLC sessions 
will be conducted using VA Microsoft Teams Meeting and recorded.  VLC clusters will 
have access to shared team materials on VA  including, AKI Prevention Toolkit, VLC 
session recording, uploaded homework from sites (which may include aggregated data 
on AKI and other factors over time), educational materials, and other tools or 
documents to support the VLC interventions.  
 
Each of the sites in the VLC intervention will 
follow the same structure each month (see Figure 
4. VLC monthly calendar). On the first of the month, 
teams will be required to submit homework to “VA 
VLC O365/ SPO Sharepoint  folder,” a web-based 
platform that allows multiple users to access and 
upload files to be shared among approved 
members. The monthly homework will ask teams to 
share about the changes made during the previous 
month, including a review of successes and challenges. Expert team coaches will 
review the uploaded homework and provide team-based feedback. There will be a 
monthly learning lesson call. Each learning session call will consist of several key 
components: introduction and roll call, review AKI Prevention Toolkit components, 
process changes or quality improvement methodology, team sharing and/or reports on 
their work during the previous month. Coaches will be available via phone and email to 
answer any questions from teams, if requested. See Table 2 for a description of VLC 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
12 | P a g e  
and TA call topics, which may be modified based on need. Participation will be 
measured by activity on the monthly calls, simple progress reports, and participation.  
 
 
Evaluation and Measurement: Process 
measures will be self-reported by 
participating teams each month in a 
monthly homework or report identifying 
the process changes made during the 
previous month including the bundle 
interventions implemented during the 
previous month. Process measure 
collection will be duplicated by existing 
field-defined data elements captured by 
the Data Coordinating Center for which 
our research team has extensive 
experience. The report will also ask 
teams to submit the team meeting 
agenda and minutes. Process 
measures are reported in Table 3. To 
supplement these reports, each team 
champion(s) (current directors of 
catheterization laboratories) will be 
tasked with giving a 4-5-minute report 
on their progress on each VLC call. 
Based on our pilot, the key process 
changes to be measured are the use of 
AKI prevention toolkit, clinical 
champions, empowered nurses, team 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
13 | P a g e  
champions, protected time for team meetings, and QI training.3 A survey will be 
completed monthly for each team following each call or session through VA RedCap. 
 
C.5. Technical Assistance (TA). Intervention: TA will be offered to the eight teams 
randomized to the TA condition (minimum of 4 ASR, minimum of 4 without ASR) and 
will receive the AKI Prevention Toolkit.1, 3 TA will have monthly scheduled TA calls (60 
minutes each) with each team individually to review and discuss the bundle 
interventions (as is done in the VLC group) and allow for a consultation with experts on 
the AKI bundle interventions. TA calls will be driven by an agenda with timely topics 
used to assess where sites are at with implementing the AKI Prevention Toolkit led by 
the AKI improvement specialist (Solomon, External Collaborator). All TA calls will be 
conducted with individual hospitals in each cluster (TA, TA+ASR). The goal of the TA 
consultation call is to address questions, review progress, and discuss challenges for 
the sites in implementing the AKI Prevention Toolkit. TA sites are also allowed to raise 
issues or concerns for discussion. If additional expertise is needed for specific 
questions, the TA expert will either schedule a follow-up call or respond via email. 
Teams are allowed to reach out to the TA expert at any point during the 36 months. TA 
will follow Call Topics in Table 2. Fidelity to the TA intervention and semi-structured 
interviews will be conducted as described in the VLC methods. TA will have the same 
touch points as VLC. All TA sessions will be conducted using VA Microsoft Teams 
Meeting and recorded.  A survey will be completed monthly for each team following 
each call or session through VA RedCap. 
 
Evaluation and Measurement: The process changes and primary outcomes (AKI) 
will be the same as those teams in the VLC (see Table 3. VLC and TA Evaluation and 
Measures). Fidelity measures for TA will include individual attendance at the 4 
scheduled calls and team initiated contacts (see Appendix 1). Semi-structured phone-
based interviews will also be conducted every 6 months for 18 months with TA team 
members to learn about improvement training, efforts, and implementation barriers and 
successes (see VLC methods).  
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
14 | P a g e  
C.6. Automated Surveillance Reporting (ASR). Intervention: ASR will be offered to 
eight teams (4 with VLS, 4 with TA) as noted above in block randomization.1,3 The 
surveillance tools will execute a weekly updated estimation of each site’s risk-adjusted 
performance, which generates a set of data that will populate series of tables and 
graphs for each site. There will be 3 levels of granularity (Figure 5A): 1) National Risk-
Adjusted Ranking (Figure 5B) that reports the overall performance of each site relative 
to the national expectation, 2) Site Specific Performance Summary (Figure 5C) which 
highlights whether the site is out of expectation with regards to the statistical process 
control based methods used for sequential surveillance, and 3) patient list, which 
provides a table of the patients, with columns for each of the risk factors in the risk 
adjustment model and the predicted risk for the outcome and observed outcome 
(Figure 5D).  
 
The figures and graphs will be placed inside a PDF directly along with text 
describing the methods used in the analysis, and an attached excel sheet of the 3rd 
level patient list report, which includes demographics, risk variables used in risk 
adjustment, the 
predicted risk of the 
outcome, and the 
observed outcome value 
for each patient at that 
site. The same data and 
format will be populated 
within a website with an 
identical workflow 
(Figure 5) and 
visualization to the PDF 
that users can log in and 
review the current and 
historical data of their 
site and all other 
catheterization sites 
within the VA intranet at 
any time. Only those 
sites randomized to 
receiving ASR (n=8) will 
be able to access the 
dashboard or receive the 
PDF’s vial email link. 
This type of internally 
‘public’ reporting has been shown in prior literature to potentially be more impactful than 
de-identified reporting, and we have the CART approval to operationalize the ASR 
reporting.  
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
15 | P a g e  
Within either the web or excel-embedded-in-PDF user interface, users will be 
able to review patient data from their site to see which patients experienced the AKI 
outcome and to explore the analytic data for which factors were most likely related to 
AKI. This allows users maximal flexibility to inform their site-specific quality 
improvement initiative. Lastly, should a deeper chart review be desired, patient 
identifiers are available in the Level 2 Patient List portion of the tool to allow a user to 
open the EHR to conduct a deeper review and/or conduct other activities as part of their 
QI initiative protocols. Within the analytics module, the risk prediction models for post-
procedural AKI will be developed monthly using the risk-adjusted sequential probability 
ratio test analysis method testing an odds ratio of 2.0 with a type I error of 0.05 and a 
type II error of 0.10, and a retrospective rolling 1 year window prior to the current month 
using the LASSO variable selection method applied to all candidate variables included 
in our previously published national risk model development work.59 LASSO is 
conservative in its variable selection; unlike Ridge regression which merely shrinks 
coefficients LASSO pushes coefficients of weaker predictors to 0. The rolling window 
allows for continuous risk model recalibration, which is considered best practice for the 
use of risk prediction models due to calibration drift over time. For each model, 
discrimination (measured by the AUC) and calibration (by Cox intercept and slope) will 
be calculated and reported.60, 61 
 
Evaluation and Measurement: As noted above, the primary endpoint of AKI 
following cardiac catheterization will be assessed within the 2x2 study design. A number 
of process measures will be assessed as secondary endpoints, in a similar pattern to 
those reported for the VLC intervention. A survey will be administered monthly to each 
team with access to the ASR through VA RedCap as required progress reports prior to 
each monthly session (Appendix 4): whether they felt the information provided by risk-
adjusted performance ratings was useful, whether access to the patient reviewer 
dashboard tool was helpful, and whether it influenced the implementation process 
during that month. Other characteristics will be captured through website logging and 
PDF report download usage statistics, which is tied to user authentication/direct 
download URL and will allow tracking of how often users and sites access the 
information and which parts of the report are viewed and utilized. We will analyze 
whether the use of the ASR increases or decreases over the course of the study and 
whether the use is different between the TA and VLC sites that have access to the ASR. 
To ensure that we capture the key process changes, team champions will submit 
responses to the mandatory monthly questionnaire for ASR (Appendix 4). A survey will 
be completed monthly for each team following each call or session through VA RedCap. 
 
C.7. Statistical Analysis. We will conduct a 2x2 factorial analysis using a two-level 
hierarchical model that adjusts for observed patient covariates included in our VA AKI 
pre-procedural risk model and accounts for the clustering of patients within hospitals in 
comparing the effects of the four interventions comprising VLC, TA, and ASR on AKI.14 
Adjusting for patient case-mix accounts for differences in the distribution of patient 
covariates across intervention groups, thereby removing chance confounding in the 
sample (randomization implies that the expected confounding is 0), and yields more 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
16 | P a g e  
precise estimates by accounting for the effects of patient-level factors that would 
otherwise inflate the error variance. Because ASR is a binary variable, we use a 
hierarchical logistic regression model with a two-way ANOVA at the hospital-level as the 
base model for estimating the effects of the interventions on patient outcomes. Let 
𝐴𝐾Iij, 𝑉𝐿𝐶i, 𝐴𝑆𝑅ij, and 𝑃𝑎𝑡𝑖𝑒𝑛𝑡ij denote the value of AKI, the hospital’s VLC status (1 = 
VLC, 0 = TA), the hospital’s ASR status (1 = Used, 0 = Not used), and a vector of 
observed covariates for the jth patient in hospital i. The model for this analysis is 
specified as follows: logit(Pr (𝐴𝐾𝐼ij = 1|𝜃i)) = 𝛽0 + 𝛽1 𝑉𝐿𝐶i + 𝛽2𝐴𝑆𝑅i + 𝛽3𝑉𝐿𝐶i 𝐴𝑆𝑅i + 
𝛽!
"𝑃𝑎𝑡𝑖𝑒𝑛𝑡 + 𝜃i + 𝜀ij (1) where 𝜃i ~𝑁𝑜𝑟𝑚𝑎𝑙(0, 𝜏2 ) and 𝜀 ij ~𝑁𝑜𝑟𝑚𝑎𝑙(0, 𝜎2). Under (1), the 
regression parameters 𝛽1 through 𝛽4 denote the effects of VLC (versus TA) in the 
absence of ASR, the effect of ASR in the absence of VLC (i.e., under TA), the 
difference in the effect of VLC when ASR is present compared to when it is absent, the 
patient covariates, respectively, while 𝜏2 is the unexplained between-hospital variation, 
and 𝜎2 is the unexplained between patients within hospital variation. We will first 
examine the significance of the interaction effect, 𝛽3. If this is significant we will report 
separate effects for VLC (versus TA) for when ASR is present than when it is absent – 
the effects obtained under a 4-level one-way ANOVA design (one level for each 
intervention strategy) – to enable the optimal intervention strategy to be easily seen. If 
an interaction is not present we will drop 𝑉𝐿𝐶i 𝐴𝑆𝑅i from the model in (1) and estimate 
the resulting no-interaction two-factor model to obtain more precise estimates of the 
main effects of VLC and ASR and of the optimal treatment strategy.48 Under both 
scenarios, multiple comparisons will be accounted for using Bonferroni correction 
methods and we will limit regression adjustment to the pre-specified set of patient 
predictors described in the next section to guard against selectively choosing the model 
that yields the most favorable result. An alternative to using regression adjustment to 
account for patient characteristics in (1) is to use propensity score methods to balance 
the treatment groups with respect to patient case-mix – with only a minimum of 16 sites 
being randomized it is not unreasonable to anticipate some differences in the 
distribution of patient characteristics between the 4 groups, despite our blocking the 
sites. Because propensity score matching across 4 groups will be cumbersome, we will 
use inverse probability weighting by the propensity scores. However, the a priori 
expected propensity score for the four groups is 0.25 and using weights inflates 
standard errors and lowers power, propensity score weighting will only be performed as 
a sensitivity analysis. In addition, we will evaluate the trial against non-intervention 
control groups using both a historical non-intervention control group among the enrolled 
sites in this trial and a concurrent non-intervention control group from a sample of non-
enrolled sites in the national VA.  Additional analyses on AKI from retrospective national 
catheterization procedures to support and inform the AKI interventions may be 
conducted including AKI prediction, trends overtime, trends and use of preventive 
practices (for example, IV fluid volume use, contrast volume use, etc).   
 
 
Cost-effectiveness evaluation. We will assess both the clinical effectiveness in 
AKI reduction and the two interventions’ cost-effectiveness. A cost effectiveness 
analysis will be conducted estimating the incremental cost for each incremental AKI 
prevented above the TA only baseline and stratified across the other three clusters. A 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
17 | P a g e  
separate budget impact analysis for implementing the VLC arm and stratified by ASR 
will be generated. Budget impact analyses estimate the cost to the hospital or service 
entity to implementing a new program or initiative. 
 
C.8. Qualitative Methods. We will employ a qualitative approach in assessing site 
fidelity to the randomized intervention arm (TA, TA+ASR, VLC, VLC+ASR) and 
implementation of the interventions. Building off prior quantitative and qualitative work 
conducted by our research team, we will measure fidelity to the VLC, including 
participation on monthly calls, submission of key documents (i.e., homework), and a 
general active participation in the learning community in months 12 to 18 (see 
Appendix 2). We have mapped data collection tools including fidelity and 
implementation measures to the PARIHS framework constructs (see Appendix 3) to 
ensure we gather appropriate information for the context, facilitation, and evidence 
domains in PARIHS.54 Semi-structured phone-based interviews with all team members 
together  (without the catheterization laboratory director) at each site (TA and VLC) will 
occur at baseline, month 12 and 18 post-intervention to collect qualitative data on the 
participant experiences with the improvement process (Appendix 3).  With these 
interviews, we hope to examine aspects of clinic culture that may support or hinder AKI 
prevention and the dynamics and organizational culture of the clinic. Such information 
(e.g., hierarchies and power dynamics, consensus and disagreement) will augment our 
understanding of barriers and facilitators of AKI prevention within real-world clinical 
settings.62,63 Interviews will be conducted at each site for each team member.  These 1-
hour interviews will be conducted using an interview guide to solicit in-depth information 
about the work of the staff, including the strategies to prevent AKI. Two qualitative 
researchers will lead the qualitative approach in assessing fidelity and implementation. 
Building on our prior work,3 interviews will ask about the fidelity of the appropriate 
strategies identified in our pilot work to improve AKI. We will also inquire about barriers 
and facilitators of implementing the identified strategies. Interviews, TA and VLC calls 
will be audio-recorded using a VA-approved audio recorder (VA Microsoft Teams 
Meeting) and transcribed by an approved federal contractor, Transperfect, and used for 
qualitative analysis (C.8. Qualitative Analysis).   
 
Qualitative Analysis. Audio-recorded qualitative interviews will be transcribed, 
managed and analyzed with the aid of Atlas.ti, a qualitative analytic software program. 
We will conduct qualitative content analysis, a widely-recognized strategy for the 
interpretation of the content of text through systematic coding to identify patterns and 
themes.64 We will approach our analysis through four coding levels: 1) descriptions of a 
priori themes based on the interview guide, 2) fidelity of the interventions, 3) 
unanticipated or emergent themes, and 4) barriers and facilitators to the implementation 
of strategies to improve AKI. Qualitative coding and content analysis will consist of 
identifying quotations that express a cogent theme or concept and labeling these 
quotations for categorization and summarization.65 Coding is the pivotal link between 
data collection and deriving meaning from qualitative data.66,67 In the initial stages of 
coding, we will conduct ‘open’ coding, which is the process of labeling portions of text to 
identify and formulate all ideas, themes, and issues suggested by the data.66,68 Analytic 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
18 | P a g e  
codes constructed in the context of open coding are provisional and grounded strongly 
in the data.66,69 Through ongoing immersion in the dataset, we will proceed to ‘focused 
coding’, which utilizes the prominent themes identified during open coding as the basis 
for more fine-grained analyses.69 We will compare different pieces of data for similarities 
and differences, known as a constant comparisons approach (comparing different 
pieces of data for similarities and differences). This will be done until no new categories 
or relevant themes emerge (point at which thematic saturation is reached).70  
 
Two analysts will code the qualitative data. Involving multiple analysts is a highly-
regarded check on rigor in qualitative research as it allows for multiple perspectives and 
resolution of discrepancies through consensus.69 We will review our coding approach 
and conduct regular consultations with the research team during the coding process. 
Based on previous qualitative studies, the selected sample size for the interviews is 
anticipated to achieve saturation of themes. Our analyses will yield an in-depth 
understanding of efforts and strategies to improve AKI at VA Medical Centers. We 
expect our findings will be generalizable to all VA and non-VA medical centers and 
generate hypotheses relevant to future efforts designed to improve AKI.  
 
C.9. Scientific Rigor and Sex as a Biological Variable. We will enroll all eligible 
consecutive patients in our study including women. Our model includes the patient 
characteristics of sex and age. This will allow us to assess the beta coefficients of these 
biological variables and plot their marginal effects on AKI outcomes and their influence 
on our findings. While women are a smaller proportion of patients in the VA, sex will be 
incorporated (1 = woman, 0 = men) to determine mediating effects in women versus 
men and age will be modeled as a spline function to assess curvilinear effects 
dependent on patient age.  
 
C.10. Rigor and Reproducibility. To ensure scientific rigor and reproducibility, an 
analytic program file used to produce the analyses and a de-identified dataset will be 
made available to third parties within the VA system to conduct confirmatory analyses 
for the proposed specific aims. All parties must be in compliance with VA regulations 
and access. All analytic results will be reported in supplementary appendices for all 
publications.  
 
C.11. Sample Size and Power. To be conservative, power calculations assume the 
two-factor ANOVA with interaction. The target enrollment of approximately 2,644 is 
planned in each of the four arms of the study (TA, TA+ASR, VLC, VLC+ASR) with four 
hospitals within each arm (average approximately 661 patients per hospital). Based on 
national VA catheterization data for CKD patients,14 the proportion of AKI in the VLC 
and VLC+ASR arms are assumed to be 0.2700 under the null hypothesis and 0.2025 
under the alternative hypothesis (an effect size of 25% or 0.0675). The proportion of AKI 
in TA and TA+ASR arms are projected at 0.2700 for the intervention period. We further 
assume an intra-cluster correlation coefficient (the ratio of variation between hospitals to 
the sum of variation within and between hospitals) of 0.0009 – this is implied by the 
intervention arm of the pilot study.3 We plan to conduct 0.05-level tests and desire 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
19 | P a g e  
power of at least 90%. The power for analysis to detect any difference at all across the 
four intervention studies can be approximated by assuming the distribution of the 
proportion of AKI cases under each strategy follows a normal distribution and applying 
an F-test with penalty for clustering. Under these assumptions, our power to detect any 
difference against the null of no difference is above 99%. We also compute power for 
illustrative individual contrasts of interest. The power to detect a significant effect of VLC 
within a level of ASR (i.e., either for ASR used or ASR not used) is also above 99% 
(although numerically lower than for the test of any difference across the four groups as 
sample size is halved) with a minimum detectable effect size of 0.034. If under the 
alternative hypothesis the proportion of AKI for the VLC arm was also 0.27 (so that an 
interaction effect is present), the power to detect a significant interaction between 
TA/VLC and ASR status is just above 90%. Power is lower for the interaction contrast 
due to four groups being compared as opposed to only two groups. In summary, we 
have overwhelming power to detect a difference among any of the four intervention 
strategies and between the levels of one factor within a level of the other factor, and 
acceptable power (i.e., above 90%) to detect a significant interaction between the 
effects of the factors. Finally, assuming a Bonferroni correction for multiple testing and a 
25% effect size for each difference, testing the full set of 6 pairwise differences among 
the four intervention strategies has power >97% implying we will be adequately 
powered to infer the optimal intervention strategy.  
 
C.12. Expected Outcomes. We expect to prevent AKI and improve outcomes in 
common diagnostic procedures. As depicted by the above power calculations, we 
expect to reject the null hypothesis that the interventions (TA, TA+ASR, VLC, 
VLC+ASR) will all produce the same reduced rate of AKI after the 18-month intervention 
period. Comparable to our pilot intervention, we expect the VLC with or without ASR to 
have more than a 25% reduction in AKI over the TA intervention with or without ASR, 
where we observed a 28% reduction among patients with pre-existing CKD,3 implying 
that the above power calculation likely errs on the side of being too conservative (i.e., 
we expect to have greater power). Using process measures from our fidelity tool and 
interviews, we expect to observe the VLC teams to successfully implement more AKI 
prevention strategies from our toolkit. At our 6-month semi-structured interviews of 
individual team members, we expect team members to report their teams achieving a 
hospital-wide shift in the attitudes and awareness of AKI prevention comparable to our 
pilot.3  
 
C.13. Generalizability to non-VA sites. The VA offers a strong learning laboratory with 
existing standardization across sites. The VLC intervention was piloted in 10 non-VA 
hospitals. The ASR intervention, while developed in the VA for national ASR for AKI, 
can be ported to any EMR; daily data from the EMR laboratory or catheterization 
system would need to be directed to the mounted ASR toolkit in the data warehouse. 
Coding to allow the ease of portability to non-VA EMRs will be conducted by Dr. 
Matheny’s Core during the award period. The AKI Prevention Toolkit and measures can 
be applied in any VA or non-VA site.  
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
20 | P a g e  
C.14. Potential Problems and Alternative Strategies. One potential problem is site 
enrollment. Sample size estimates required a minimum of 16 sites (minimum of 4 in 
each arm); we have over-recruited 29 sites with planned over-enrollment in each arm to 
a minimum of 4 sites per arm; the two excess sites with the lowest volume will be 
excluded from the study. Another potential threat is contamination where providers 
from one condition educate providers in the comparison condition of their 
implementation group. Indeed, this is the reason why we have chosen to randomize at 
the site as opposed to the physician level, for which contamination would be highly 
plausible. In regards to our proposed site-level cluster design, we control for potential 
contamination by constraining team communications to the 4 sites in the same study 
condition. In addition, only the ASR sites will have website access to the ASR resources 
regulated by user login requirements. In addition, while login access is restricted to 
ASR, it is possible that ASR centers could directly share surveillance data with a non-
ASR site, but beyond basic site risk-adjusted ranking, all other data will focus only on 
that site, which will limit possible contamination. First, we will evaluate the possibility of 
contamination through both quantitative measures of process improvements and 
qualitative data from calls and semi-structured interviews of team members. Second, we 
have systems in place to prevent transfer of recommendations on AKI Prevention 
Toolkit implementation from VLC sites or summary statistics on relative performance 
from the ASR sites to the TA site(s) or non-ASR site(s). Third, the TA sites will have the 
same number or calls, TA sites will not have access to the VLC structure, frequency, or 
level of team-based coaching. We have taken care in both the design and measurement 
to ensure contamination does not occur and if it does, it will inform our analysis.  
 
Site level randomization may generate patient level imbalances between sites, 
and this requires that we adjust for patient-level characteristics in the statistical analyses 
as noted above (see Statistical Analysis).  
 
Ascertainment of AKI is a potential issue for measurement bias. By focusing on 
patients with pre-existing CKD, we will limit the number of patients currently without 
measurement of serum creatinine up to 48- hours. Based on our extensive experience 
with the CART data and supplementary serum creatinine data from multiple sources 
within the VA, less than 5% of patients with pre-existing CKD do not have a post-
procedure serum creatinine measured within 48-hours. Current VA standard of care is 
to have post-procedure serum creatinine measured for all CKD patients. We will 
strongly encourage sites to implement a routine clinical practice that is in compliance 
with the current guidelines as part of our educational quality improvement framework. 
 
While setting up the study, we will be able to collect data from sites for a period 
of approximately 6-months before any interventions are unleashed. One strategy is to 
use these data to establish baseline covariates for each site, helping to account for 
unexplained variation between sites. However, with the ICC estimated to only be 
0.0009, there is likely to be little utility in this. Another use is to allow the 4 intervention 
strategies to be compared to a pure control or baseline in which no intervention is used 
by pooling these data with the data. 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
21 | P a g e  
 
C.15. Specific Aim 2: Evaluate the sustained efficacy of the Virtual Learning 
Collaborative and/or Automated Surveillance Reporting to reduce the incidence 
of AKI following the intervention period.  
Our working hypothesis is that teams coached in the Virtual Learning Collaborative with 
and without Automated Surveillance Reporting will sustain reductions in AKI following 
the intervention period. All 16 hospitals will continue to monitor patients following the 18-
month intervention period for an additional 18-month post-intervention period. We will 
evaluate whether or not the reduced incidence of AKI will be sustained following the 
removal of the TA, VLC, and ASR interventions. Although this aim is focused on the 
period 19-36 months post-intervention, randomization holds for the purpose of 
comparing the effect of the interventions on differences in patient outcomes (particularly 
AKI) during this period. In the event Aim 1 yields a null finding, Aim 2 will still be 
valuable if (1) the onset of the intervention effect might only manifest after month 18, or 
(2) the effect is constant, or nearly constant, in which case analyzing the pool of 
observations over the entire 36-month period may yield a significant finding even if the 
1-18 month analysis does not.  
 
C.16. Research Design. After 18-months, the TA, ASR, and VLC interventions (Aim 1) 
will all be removed, and all hospitals will be followed for an additional 18-months with 
continued data collection and semi-structured interviews (Aim 2). Following the Aim 1 
trial, we will conduct a prospective cohort with AKI as the primary endpoint. All sites will 
be followed for an additional 18-months with data collection and interviews; hospitals 
will be blinded to the follow-up and interviews conducted at the end of the post-
intervention 18-month period. The target number of patient records that will be reviewed 
from the EHR will be approximately 10,576 (which is approximately 2,644 in each of the 
four arms of the study (TA, TA+ASR, VLC, VLC+ASR) with four hospitals within each 
arm (average 661 patients per hospital)). During the 18-month post-intervention period, 
we will compare the sustained efficacy of the interventions using formal 2x2 factorial 
design and longitudinal interrupted time series analysis. All interventions will be actively 
continued in the post-intervention phase (months 19-36). 
 
C.16.a. Study Population: Inclusion and Exclusion Criteria. The study sample of 
interest for each of the analyses in Aim 2 will include all adult patients undergoing 
cardiac catheterization within the Veteran’s Health Administration from among hospitals 
randomized in Aim 1. The same inclusion and exclusion criteria from Aim 1 will be 
applied. The Aim 2 post-intervention enrollment period will occur from January 1, 2020, 
to June 30, 2021, for a period of 18-months.  
 
C.16.b. Primary Endpoint. KDIGO AKI defined by ≥0.30 (mg/dL) or ≥50% increase in 
serum creatinine over baseline within 48-hours of the procedure or within 7-days for in-
patients, or onset of dialysis within 7-days.13 We also constructed a Sustainability 
Measure: sites will demonstrate sustained implementation if all 3 core interventions are 
reported to be in place from Table 2 (1. Standardized order sets; 2. IV and Oral Fluids; 
and 3. Efforts to reduced contrast volume). We will ask each site to report on the 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
22 | P a g e  
sustained implementation of the 3 core 
interventions. In addition, assessment of the 
process changes will be determined by routine 
patient data collection, site audits, and be 
informed by the National Health System 
Institute for Innovation and Improvement 
Sustainability Model.71 This model will guide 
teams in planning to sustain efforts by 
measuring the effectiveness of the program and 
adaptability of the improved process, focusing 
on process, staff and organizational factors 
influencing sustainability. Measures include 10 
factors to assess the sustainability of the 
program (Table 4).71 Factor Level number is selected for each factor to describe the 
sustainability of the intervention and summed to a score ≤100 (≥55 is positive; ≤45 
indicates concern).  
 
C.16.c. Data Collection and Data Coordinating Center. The data coordinating center 
will collect all data elements from each hospital during the post-intervention 18-month 
period as reported in Aims 1. In addition to evaluating the primary population with pre 
existing CKD, we will also evaluate all cardiac catheterization patients. Data collection 
will include all cardiac catheterization procedures from 2000 through 2024 to support 
analyses and models for AKI. Patients and procedures will exceed 510,000, in addition 
to the CKD patients in the clustered randomized trial (~10,576 [C.16]).  
 
C.17. Statistical Analysis. We will compare the rate of AKI during the 18-month post-
intervention period (i.e., the period 19-36 months out from the initiation of the 
interventions) between the interventions using an identical approach to Aim 1 including 
scientific rigor and biological variables analysis.48 The reason why the approach is 
identical is that the only change is that a different cohort of patients will be analyzed. 
Although the interventions will have ceased, the study design is unaltered. Therefore, 
the model in (1) will be estimated and if appropriate simplified using the same procedure 
and will be followed by re-evaluation of the same hypotheses. Following the primary 
analyses, we will explore the longitudinal relationship between the intervention and 
post-intervention periods. The analysis will be performed by pooling the data from the 
intervention (1-18) and post-intervention (19-36) month periods. In the base 
specification, we will define a variable 𝑃𝑜𝑠𝑡ij (= 1 for the post period and 0 for the 
intervention period) and add interaction variables between 𝑃𝑜𝑠𝑡ij and each of 𝑉𝐿𝐶i, 
𝐴𝑆𝑅i, and 𝑉𝐿𝐶i𝐴𝑆𝑅i, to the model in (1). (The main effect of 𝑃𝑜𝑠𝑡ij will be added to the 
model only if time is not otherwise represented in the model.) If the effect of the 
interventions wanes over time, we expect that the 𝑃𝑜𝑠𝑡ij interaction effects will reduce 
the overall magnitude of the intervention effects. More generally, we will explore the 
time-sequence of the interventions to determine if there were statistically significant 
shifts or trends in rates of AKI over time.72 We will explore whether the effect of the 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
23 | P a g e  
interventions changes with time both within the intervention period and then with respect 
to time after the intervention period ends. 
 
C.18. Sample Size and Power. Since we expect to enroll the same number of patients 
during the 18-month period after the active interventions cease as, in the 18-month 
period of the active intervention, the power to detect the same effect sizes as 
considered for Aim 1 remains unchanged. The test for an interruption in the effect 19-
months from the start of the intervention is an exploratory analysis and so we do not 
compute power.  
 
C.19. Expected Outcomes. We expect to sustain a lower incidence of AKI and improve 
outcomes. Using process measures, we expect to observe the VLC teams during the 
post-intervention period to successfully sustain the implementation of more AKI 
prevention strategies from our toolkit. Using 6-month semi-structured interviews of 
individual team members, we expect team members to report that their team and 
hospitals are achieving a hospital-wide shift in the attitudes and awareness of AKI 
prevention comparable to our pilot.3 We expect the VLC to be sustainable using our 
measure of Sustainability.  
 
C.20. Potential Problems & Alternative Strategies. Unlike Aim 1, we will not be 
concerned with contamination in Aim 2 after all interventions are removed. However, 
comparably to our approach in Aim 1, we will still evaluate the possibility of 
contamination of the intervention arms, ensure complete ascertainment of AKI, and 
address the potential of patient imbalance in randomized clusters in our multilevel 
modeling approach described in the statistical methods (C.7). We will measure the 
possibility of contamination through quantitative measures of process improvements 
and qualitative data from calls and interviews with team members. Should Aim 1 not 
demonstrate significant differences between the 4 intervention arms, Aim 2 will continue 
to monitor the endpoints and conduct a trial autopsy from the robust quantitative and 
qualitative data including site audits to determine fidelity to the interventions, and ability 
of sites to implement improvements.  
 
C.21. Timeline. The study will be carried out over five years. VA site recruitment has 
already been completed (Letters and Facilities attached). Since our interventions and 
data coordinating center have been established from other federally-funded projects, we 
will be able to 
initiate the 
interventions 
by project year 
01, month 7. 
The cluster-
randomized 
trial period will 
run from PY01 
through the 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
24 | P a g e  
end of PY02. Post-intervention will run from PY03 to PY04. Longitudinal data, analyses, 
and reports will be completed in PY05.  
 
C.22. Future Directions. Our overall objective of this project is to increase the 
consistency with which AKI preventive approaches are applied. We expect the 
consistent application of our interventions will have a positive impact on reducing 
morbidity and mortality following cardiac catheterization in the 16 VA sites randomized 
in our study. The successful evaluation of our implementation interventions will 
determine which intervention or combination of interventions (e.g., VLC+ASR) could be 
used as a vehicle for rapid dissemination or AKI preventive strategies across the VA 
and other non-VA hospitals. We have established inexpensive and adaptable 
implementation interventions to disseminate guidelines and evidence-based practice to 
multiple hospitals and teams for the prevention of AKI. In our opinion, these strategies 
could also be used to rapidly disseminate new landmark trial findings, such as 
PRESERVE, as well as the implementation of new technologies to prevent AKI in the 
cardiac catheterization population. In the future, our implementation interventions could 
be adapted to coach teams in managing and preventing CKD, CKD progression, and 
ESRD. Our implementation interventions could also be adapted to work with dialysis 
unit teams in VLC methodology to assist in overcoming barriers and challenges in 
maintaining ideal hemoglobin, sodium, and potassium levels used as quality indicators 
for patient safety and dialysis unit performance. This study will address an important 
gap in the literature to reduce the high variability in complications of common 
procedures for complex patients and how the VLC+ASR model could be applied in other 
populations to reduce avoidable morbidity, mortality, and healthcare costs.  
 
C. 23. Specific Aim 3: Laboratory Predictors of AKI in Patients Undergoing 
Coronary Angiography 
 
Prior studies using the Veterans Health Administration coronary angiography cohort 
have developed robust, externally validated prediction models for AKI; however, it is still 
unclear if certain less well-studied laboratory markers in this context are risk factors for 
AKI. Hypomagnesemia, for example, is associated with a higher risk of AKI in patients 
receiving cisplatin, as well as in non-cancer patients. In a cohort of ~20,000 cardiac 
surgery patients, we found that hypomagnesemia (defined as a serum magnesium level 
<1.5 mg/dl) is associated with a 1.8-fold higher odds of the composite outcome of 
severe AKI, need for renal replacement therapy (RRT), or death, even after 
multivariable adjustment. We also found that lower pre-procedure hemoglobin and 
higher hemoglobin A1c are each independently associated with a higher risk of AKI 
after cardiac surgery. 
 
Study Design: In this specific aim, we will examine whether lower serum magnesium, 
lower hemoglobin, higher hemoglobin A1c, and other laboratory predictors are 
associated with a higher risk of AKI in patients undergoing coronary angiography. We 
will also examine whether laboratory predictors are associated with an increased risk of 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
25 | P a g e  
death at 30 days. The exposures will vary based on the individual analyses; however, 
the inclusion and exclusion criteria will be the same: 
 
Data Collection: The cohort of interest will be all patients undergoing cardiac 
catherization in the VA, per the parent data collection above. 
 
Inclusion criteria: VA adult patients who underwent coronary angiography during the last 
10 years (2008 to 2018) 
 
Exclusion criteria: 
-End-stage renal disease (ESRD) on dialysis 
-Missing baseline SCr 
- No post-procedure SCr available within 7 days 
-Missing a pre-procedure value of the laboratory predictor of interest (e.g., magnesium, 
HbA1c, hemoglobin) 
-AKI at the time of coronary angio, defined as a ≥2-fold increase in SCr from baseline 
value to most proximal value prior to coronary angio or any RRT within 7 days prior 
 
Primary Endpoints: Major Adverse Kidney Event within 7 days (MAKE7_1 or 
MAKE7_2), defined as ≥1.5-fold or ≥2-fold increase in SCr within 7 days, need for RRT 
within 7 days, or death within 7 days.  We are defining AKI as a doubling of serum 
creatinine (SCr) or need for RRT. 
 
Statistical Analysis: We will conduct a multivariate anlaysis for the outcome of the 
primary endpoints as noted above, adjusted for the covariates already described in the 
prior aims, names patient demographics, laboratory tests, medication administrations, 
and numerous clinical conditions defined as combinations of administrative codes. 
……………………………………………………………………………………………………… 
D. Performance Sites:  
• The Dartmouth Institute for Health Policy & Clinical Practice, Epidemiology, 
Biomedical Data Science at the Geisel School of Medicine at Dartmouth 
College– Coordinating site (Affiliate of White River Junction VA) 
• Vanderbilt University, Department of Biomedical Informatics, Vanderbilt 
University Medical Center (Affiliate of TVHS VA) 
• Tennessee Valley Healthcare Systems, Department of Veteran Affairs  
• University of Vermont, University of Vermont Medical Center 
………………………………………………………………………………………………………  
E. Study Population:  
E.1. Patients  
E.1.a. Inclusion Criteria for VA Patients  
The following conditions must be met for study eligibility:  
1) 
Adults are defined as those patients aged 18 or greater  
2) 
All patients presenting for diagnostic coronary angiography or percutaneous 
coronary intervention (PCI or angioplasty) to one of the enrolled VA hospitals will 
be enrolled in the trial. CKD is of primary interest and it will be determined by a 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
26 | P a g e  
pre-existing CKD diagnosis in the VA medical record, or by two or more estimated 
glomerular filtration rates <60 (ml/min/1.73 m2) at least 90-days apart prior to 
presentation.  
 
E.1.b. Exclusion Criteria for VA Patients  
Patients meeting any of the following criteria will not be eligible to participate in the 
study:  
1) Patients with a history of dialysis including hemodialysis, or peritoneal dialysis. 
 
E.2. Implementation Site Staff  
All staff members participating in team-based interventions will be interviewed at 
baseline, 12, and 18 months following the Virtual Learning Collaborative and Technical 
Assistance interventions. We estimate to interview 5 to 10 staff members at each of the 
16 sites participating in the study. The staff members will include the cardiac 
catheterization laboratory director, nurse managers, cardiologists, nephrologists, and 
technicians.  
 
……………………………………………………………………………………………………… 
F. Research Specific Risks:  
In this project, we proposed to compare the effectiveness of clinical team-based 
interventions implementing continuous improvements in routine care. Specifically, we 
will test four interventions: a virtual learning collaborative with automated surveillance 
reporting (VLC+ASR); a virtual learning collaborative without automated surveillance 
reporting (VLC); technical assistance with automated surveillance reporting (TA+ASR); 
and technical assistance without automated surveillance reporting (TA). Experienced 
AKI and implementation science coaches from White River Junction VA (VA affiliate of 
Dartmouth) will provide the team-based coaching intervention. Sites receiving VLC 
interventions will participate in monthly VLC calls together. Sites receiving TA will 
receive coaching on an individual basis. Sites receiving automated surveillance 
reporting (ASR) will receive bi-weekly reports tailored to each individual operator in the 
cardiac catheterization laboratory at that site. Human subjects will include patients and 
hospital staff members. Patients will provide consent to undergo the cardiac 
catheterization or percutaneous coronary intervention as routine care at the site of care. 
All data collection will be derived from routine care within the VA and remain within the 
VA health system. Teams will continuously work to improve routine care to prevent AKI 
at their respective sites. Since teams are participating in interventions to improve routine 
care, patients will not need to consent to participation in the study. Data collection and 
analysis will remain within the VA and be de-identified for analysis and automated 
surveillance reports. Clinical team members (staff) will undergo interviews throughout 
the study period including the cardiac catheterization laboratory director, nurse 
managers, cardiologists, nephrologists, and technicians. While we will not collect written 
consent from staff participants, we will inform the staff that interviewers will not collect 
their names, roles, or institution or report this information in a published report of the 
project outcomes. F.1.2. The primary risk to all human subjects (patients or staff) is the 
potential risk to confidentiality.  

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
27 | P a g e  
 
The risks involved in the participation in this study are very minimal and are outweighed 
by the potential benefits. There is no physical testing or additional laboratory testing 
involved beyond routine care. Only standard clinical practice will be implemented with 
improvements to that practice. No research interventions or tests will be conducted, only 
team-based interventions and automated reports to operators for the continuous 
improvement of routine care. Patients without a post-procedural laboratory completed 
for serum creatinine at the time of discharge will be sent home with laboratory orders 
(slips) for completion at 48 hours. Post-procedure laboratory measurement of serum 
creatinine is routine safe and effective care for patients with pre-existing CKD as the 
development of AKI is associated with a higher risk of short- and long-term mortality and 
renal failure.  
 
F.1 Recruitment and Informed Consent 
F.1.1. Recruitment Strategy  
All VA Medical Centers have been recruited for the study and agreed to randomization. 
All patients meeting eligibility criteria will be enrolled in the study. Therefore, patient 
recruitment will not be performed at each site. Data on all cardiac catheterization 
patients will be pulled to the data core. Patient eligibility will be determined both at each 
VA site and by the data core. Each site will ensure data fields are populated and follow-
up serum creatinine is measured. The data core will determine eligibility and populate 
analytic datasets for both bi-weekly automated surveillance reports and for the final 
statistical analysis of the study.  
  
Implementation site staff for team and individual organizational change evaluations will 
be selected by the Organizational Change Evaluators. (WRJ VA-Dartmouth Team). The 
cardiac catheterization laboratory director has agreed to randomization and site 
participation in the project. VA staff will be invited to participate via email prior to 
participation in the semi-structured interview.  Staff participation is completely voluntary, 
and at any point, they can withdraw from the study.  They will not be coerced or 
influenced to participate, and they will not face any consequences for withdrawal from 
the study. All staff completing interviews and questionnaires throughout the study will be 
required to provide verbal consent by the WRJ interviewer prior to initiating the 
interview. To prevent coercion and address undue influence, all semi-structured 
interviews will be handled by the WRJ VA-Dartmouth Team. 
 
F.1.2. Consent Procedures  
 
The clinical staff will test the implementation of evidence-based preventive interventions 
through a Virtual Learning Collaborative (VLC) with and without the novel use 
Automated Surveillance Reporting (ASR) intervention to change clinical practice and 
improve patient safety in common diagnostic procedures. VA Medical Center cardiac 
catheterization laboratory directors will provide informed consent to verify agreement to 
the specified inclusion criteria and to randomization as an implementation site. VA 
clinical staff participants at the randomly chosen implementation sites will be 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
28 | P a g e  
approached by WRJ VA-Dartmouth-trained interviewers and provided verbal informed 
consent to participate in the semi-structured interviews.  Invited VA staff work with a 
multidisciplinary team at each site, which includes the cardiac catheterization laboratory 
director, nurse managers, cardiologists, nephrologists and technicians.  Staff will 
provide verbal informed consent at the time of the evaluation to answer questions from 
interviewers (see semi-structured interview sheet in grant appendix).  Interviews will be 
conducted without the catheterization director present. Transcriptions from the 
interviews will not be provided back to the VA site and retained by the Dartmouth 
investigative team for coding and qualitative analysis for 1 year following the end of the 
stud.  VA patients will not need to provide consent to receive IMPROVE-AKI 
interventions continuously improving routine care taking place at the site where they are 
undergoing cardiac catheterization or PCI. Data collection is automated through the 
data core and will remain within the VA health system.  Implementation site staff 
participating in organizational change evaluations will provide verbal informed consent 
at the time of the evaluation and interviews on the Virtual Learning Collaborative. The 
other staff required for the organizational change evaluations will provide verbal 
informed consent given that their participation in the interviews is essential to evaluate 
the IMPROVE-AKI implementation and organizational change. Prior to the 
organizational change evaluations, there will be an explanation in staff verbal consents 
explaining to all staff that their position (e.g., Hospital Administrator, Clinician, Allied 
Health Professional) rather than their name will be recorded on notes of their interviews 
and that neither their names nor the name of their hospital will ever be used in any 
published report of the study. 
  
That being said, hospitals participating in the Virtual Learning Collaborative will be 
informed that their progress in implementing the IMPROVE-AKI program will be 
shared with other members of their Virtual Learning Collaborative, consistent with the 
practices of running Collaboratives. Staff will also be informed that both the 
organizational change assessments and the Virtual Learning Collaboratives will be 
audiotaped solely for the purpose of qualitative review and consensus ratings. These 
audiotapes will be maintained until they can be transcribed, with titles/positions 
substituted for any names on the tapes.  
  
F.2. Protection Against Risks  
The risks associated with sites, staff, and patients participating in this study are very 
minimal and are outweighed by the many potential benefits of improved patient safety 
and outcomes. There is, however, a slight risk in the loss of confidentiality.  
…………………………………………………………………………………………………… 
G. Confidentiality and privacy for the subjects:  
G.1.1. Protection Against Breach of Confidentiality of VA Patients  
The risk regarding the breach of confidentiality of VA patient participants will be 
controlled by retaining all data within the VA health system. Data collection will be 
automated within the VA health system to the data core operated by Dr. Michael 
Matheny. Dr. Matheny is the Associate Director of the VA Informatics and Computing 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
29 | P a g e  
Infrastructure (VINCI) and has developed a near-real-time data core pulling data from all 
the VA cardiac catheterization laboratories and VA data from the VA Corporate Data 
Warehouse (CDW). This established infrastructure will be leveraged to maintain the 
highest standards of data security and maintain confidentiality for our VA patients at all 
times. Data managers will be thoroughly trained in data safety. All data is securely held 
on locked stationary encrypted servers and password protected. All demographic 
information is coded by catheterization procedure ID numbers. Subject confidentiality 
will be carefully maintained. Patient identifiers will be stored in the master file; however, 
patient and provider identifiers will be stripped from the analysis database. The analysis 
database will be held to the same security standards and maintenance and will be used 
to conduct the statistical analysis all being retained within the VA health system. All data 
collectors, managers, and analysts will undergo human subjects’ protection and data 
information security training prior to working on the study. Subject’s identities will not be 
discussed, presented or published within this study.  
 
G.1.2. Protection Against Breach of Confidentiality of VA Staff  
Risk regarding breach of confidentiality of VA staff interview data from organizational 
assessments will be minimized by recording staff position (e.g., Hospital Administrator, 
Clinician, Allied Health Professional) rather than their name, which will be recorded on 
notes of their interviews and that neither their names nor the name of their hospital will 
ever be used in any published report of the study. The same methods will be done when 
audiotapes are transcribed. We will never use staff names or site names in any published 
report of the study. Instead, we will code the sites with consecutive numbers from 1-12. 
That being said, sites participating in the Virtual Learning Collaborative will be informed 
that their progress in implementing the IMPROVE-AKI program will be shared with other 
members of their Virtual Learning Collaborative, consistent with the practices of running 
these types of collaboratives.  
G.1.3.  
G.1.3.I. Informed Consent 
A. All VA Medical Centers have been recruited for the study and agreed to 
randomization. The cardiac catheterization laboratory director has agreed to 
randomization and participation in the project. VA staff will be invited to 
participate via email prior to participating in the semi-structured interview.  Staff 
participation is completely voluntary, and at any point, they can withdraw from the 
study.  They will not be coerced or influenced to participate, and they will not face 
any consequences for withdrawal from the study. All staff completing interviews 
and questionnaires throughout the study will be required to provide verbal 
informed consent to participate before the interview begins. To prevent coercion 
and address undue influence, all semi-structured interviews will be handled by 
the WRJ VA-Dartmouth Team. 
B. The data for all patients meeting eligibility will be collected and included in the 
analysis of the primary and secondary endpoints. 
 
G.1.3.II. Data Safety, Quality, and Management 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
30 | P a g e  
A. No patient contact will occur in this study.  The patient data obtained from 
Electronic Health Records (EHRs) will only be used. Data for this study will be 
collected prospectively through the VA electronic medical record. Routine data 
feeds from the VA electronic medical record, CART (NCDR catheterization 
procedure data, laboratory values, AKI, and outcomes including AE and SAEs 
will populate the database at the data core under the supervision of Dr. Michael 
Matheny. All data elements from the NCDR cardiac catheterization national 
database, which is sent to CART in Denver, CO, will be sent to the data core. All 
patient laboratory results will be sent to the data core. Automated surveillance 
reports will only include de-identified data and in aggregate form. The target 
enrollment of approximately 2,644 is planned in each of the four arms of the 
study (TA, TA+ASR, VLC, VLC+ASR) with four hospitals within each arm 
(average 661 patients per hospital). Based on national VA catheterization data 
for CKD patients, the proportion of AKI in the VLC and VLC+ASR arms are 
assumed to be 0.2700 under the null hypothesis and 0.2025 under the alternative 
hypothesis (an effect size of 25% or 0.0675). The proportion of AKI in TA and 
TA+ASR arms are projected at 0.2700 for the intervention period. We further 
assume an intra-cluster correlation coefficient (the ratio of variation between 
hospitals to the sum of variation within and between hospitals) of 0.0009 – this is 
implied by the intervention arm of the pilot study. We plan to conduct 0.05-level 
tests and desire power of at least 90%. Once the study is complete, a de-
identified dataset will be created for the final unblinded statistical analysis. The 
analysis will be conducted within the VA and not transported outside the VA to 
ensure data security. Although we do not foresee any risk or adverse events, 
blinded and unblended datasets will be created semi-annual review.   All data will 
be reviewed for protocol adherence, including a data verification check that the 
appropriate outcome measures are given at the appropriate time points. Interim 
and final reports will be given to VA central IRB, as well as the study sponsors. In 
addition to evaluating the primary population with preexisting CKD, we will also 
evaluate all cardiac catheterization patients. Data collection will include all 
cardiac catheterization procedures from years 2000 through 2024 to support 
analyses and models for AKI totaling more than 510,000 patients and 
procedures.   
B. All data is securely held on locked stationary encrypted servers and password 
protected. All demographic information is coded by catheterization procedure ID 
numbers. Subject confidentiality will be carefully maintained. Patient identifiers 
will be stored with the master file; however, patient and provider identifiers will be 
stripped from the analysis database. The analysis database will be held with the 
same security standards and maintenance and will be used to conduct the 
statistical analysis all being retained within the VA health system. 
C. Blinded and unblended data reports will be developed. 
D. The Data and Safety Monitoring Plan (DSMP) will adhere to the protocol approved by 
NIH. A Data and Safety Monitoring Board (DSMB) charter will be created and members 
will be appointed to provide additional oversight of the trial. The DSMB will meet and 
review data bi-annually throughout the project. The DSMB will include experts in the 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
31 | P a g e  
following scientific disciplines with expertise in cardiology, nephrology, and statistics to 
interpret the data and ensure patient safety. The DSMB will not include the PIs or 
members from the VA. The DSMB will review the protocol and data collected to date, 
and advise the Co-PIs on any potential risks as well as on any risk mitigation plans. 
 
G.1.3.III. Confidentiality 
A. Confidentiality will be maintained during all phases of the study, including 
monitoring, preparation of interim results, review, and response to monitoring 
recommendations. 
B. Retaining all data within the VA health system will control confidentiality of VA 
patient participants. Data collection will be automated within the VA health 
system to the data core operated by Dr. Michael Matheny. Dr. Matheny has 
developed a near-real-time data core pulling data from all the VA cardiac 
catheterization laboratories. This established infrastructure will provide the 
highest standards of data security and maintain confidentiality for VA patient 
participants. 
C. Recording staff position rather than their name will control confidentiality of VA 
staff. Neither VA staff names nor name of their hospital will be used in any 
published report of the study 
………………………………………………………………………………………………………  
H. Benefits:  
VA Medical Centers who are randomly assigned to the Virtual Learning Collaborative 
may benefit substantially from the opportunity to receive high-intensity team coaching to 
implement the AKI preventive strategies and learn from other teams participating in the 
Virtual Learning Collaborative as they continuously work through the process of 
implementing the AKI preventive strategies. The support provided by the coaches and 
other teams participating in the Virtual Learning Collaborative may enable the sites to 
create sustainable protocols and cultural changes to prevent the occurrence of AKI in 
patients undergoing cardiac catheterization or PCI at the site once the interventions are 
removed. Likewise, sites who are randomly assigned to the Technical Assistance 
intervention will receive intermittent team-based coaching and tools to prevent AKI at 
their site, which may enable the site to create sustainable protocols and cultural 
changes to prevent the occurrence of AKI in patients undergoing cardiac catheterization 
or PCI at the site once the Technical Assistance is removed. For sites randomly 
assigned to receive automated surveillance reporting, individual operators and teams 
may benefit from biweekly customized reports that may assist the teams in monitoring 
and identifying actionable improvement opportunities. The automated surveillance 
reporting will likely assist sites in creating the sustainable protocols and cultural 
changes to prevent the occurrence of AKI. Patients will benefit from high-quality care, 
which will likely improve routine care as the study progresses. The teams we have 
created to address the issue of AKI will provide the best evidence based medical care 
known for preventing and treating AKI following PCI. Patients may independently benefit 
from this on-going work and will benefit future patients. The benefit to risk ratio is very 
high, given the extremely low level of risk involved in participation.  
 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
32 | P a g e  
This study has the potential to substantially add to the evidence base and scientific 
knowledge on the sustainable adoption of AKI preventive strategies delivered through 
inexpensive team-based coaching programs and simple automated reports to 
catheterization laboratory operators. The successful implementation of the AKI 
preventive strategies will reduce the incidence of AKI and likely prevent progression of 
CKD, ESRD, and mortality as demonstrated in our previous research. Given the number 
of preventive strategies for AKI, it is essential to learn how to engage teams and 
hospitals to implement and sustain those preventive strategies in order to prevent AKI 
and other long-term consequences of AKI. While our study will focus on the 
implementation of AKI preventive strategies, the results of our study could be applied to 
the implementation of other patient-safety interventions in cardiac catheterization 
laboratories. By collecting implementation data, we will better understand how to 
successfully disseminate AKI preventive strategies to hospitals across the United States 
and the world.  
……………………………………………………………………………………………………… 
Cost and Compensation: There will not be any costs or compensation to subjects. 
……………………………………………………………………………………………………… 
Medical Record Review:  
Data for this study will be collected prospectively through the VA electronic medical 
record. Routine data feeds will populate the database at the data core under the 
supervision of Dr. Michael Matheny. All data elements from the NCDR cardiac 
catheterization national database, which is sent to CART in Denver, CO, will be sent to 
the data core. All patient laboratory results (in- and out-patient) will be sent to the data 
core. Automated surveillance reports will only include de-identified data and in 
aggregate form. Once the trial is complete, a de-identified dataset will be created for 
statistical analysis. The analysis will be conducted within the VA and not transported 
outside the VA to ensure data security.  
……………………………………………………………………………………………………… 
References  
1) Brown JR, McCullough PA, Splaine ME, Davies L, Ross CS, Dauerman HL, Robb 
JF, Boss R, Goldberg DJ, Fedele FA, Kellett MA, Phillips WJ, Ver Lee PN, Nelson 
EC, Mackenzie TA, O’Connor GT, Sarnak MJ, Malenka DJ. How do centres begin 
the process to prevent contrast-induced acute kidney injury: a report from a new 
regional collaborative. BMJ Qual Saf. 2012;21(1):54-62. PMID: 21890755; PMCID: 
PMC4872501 
2) Brown JR, McCullough PA. Contrast Nephropathy and Kidney Injury. In: Thompson 
CA, ed. Textbook of Cardiovascular Intervention. London: Springer-Verlag; 
2014:53-63. 
3) Brown JR, Solomon RJ, Sarnak MJ, McCullough PA, Splaine ME, Davies L, Ross 
CS, Dauerman HL, Stender JL, Conley SM, Robb JF, Chaisson K, Boss R, Lambert 
P, Goldberg DJ, Lucier D, Fedele FA, Kellett MA, Horton S, Phillips WJ, Downs C, 
Wiseman A, MacKenzie TA, Malenka DJ, for the Northern New England 
Cardiovascular Disease Study G. Reducing Contrast-Induced Acute Kidney Injury 
Using a Regional Multicenter Quality Improvement Intervention. Circulation. 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
33 | P a g e  
Cardiovascular quality and outcomes. 2014; 114:000903. PMID: 25074372; 
PMCID: PMC4869689 
4) Minsinger KD, Kassis HM, Block CA, Sidhu M, Brown JR. Meta-analysis of the 
effect of automated contrast injection devices versus manual injection and contrast 
volume on risk of contrast-induced nephropathy. The American journal of 
cardiology. 2014; 113(1):49-53. PMID: 24188890. 
5) Lambert P, Chaisson K, Horton S, Petrin C, Marshall E, Bowden S, Scott L, Conley 
S, Stender J, Kent G, Hopkins E, Smith B, Nicholson A, Roy N, Homsted B, Downs 
C, Ross CS, Brown J, Northern New England Cardiovascular Disease Study G. 
Reducing Acute Kidney Injury Due to Contrast Material: How Nurses Can Improve 
Patient Safety. Crit Care Nurse. 2017; 37(1):13-26. PMID: 28148611 
6) Benjamin EJ, Blaha MJ, Chiuve SE, Cushman M, Das SR, Deo R, de Ferranti SD, 
Floyd J, Fornage M, Gillespie C, Isasi CR, Jimenez MC, Jordan LC, Judd SE, 
Lackland D, Lichtman JH, Lisabeth L, Liu S, Longenecker CT, Mackey RH, 
Matsushita K, Mozaffarian D, Mussolino ME, Nasir K, Neumar RW, Palaniappan L, 
Pandey DK, Thiagarajan RR, Reeves MJ, Ritchey M, Rodriguez CJ, Roth GA, 
Rosamond WD, Sasson C, Towfighi A, Tsao CW, Turner MB, Virani SS, Voeks JH, 
Willey JZ, Wilkins JT, Wu JH, Alger HM, Wong SS, Muntner P, American Heart 
Association Statistics C, Stroke Statistics S. Heart Disease and Stroke Statistics-
2017 Update: A Report From the American Heart Association. Circulation. 2017; 
135(10): e146-e603. PMID: 28122885; PMCID: PMC5408160 
7) Brown JR, Malenka DJ, DeVries JT, Robb JF, Jayne JE, Friedman BJ, Hettleman 
BD, Niles NW, Kaplan AV, Schoolwerth AC, Thompson CA, Dartmouth Dynamic 
Registry I. Transient and persistent renal dysfunction are predictors of survival after 
percutaneous coronary intervention: insights from the Dartmouth Dynamic Registry. 
Catheter Cardiovasc Interv. 2008; 72(3):347-354. PMID: 18729173 
8) James MT, Samuel SM, Manning MA, Tonelli M, Ghali WA, Faris P, Knudtson ML, 
Pannu N, Hemmelgarn BR. Contrast-induced acute kidney injury and risk of 
adverse clinical outcomes after coronary angiography: a systematic review and 
meta-analysis. Circ Cardiovasc Interv. 2013; 6(1):37-43. PMID: 23322741 
9) Merten GJ, Burgess WP, Gray LV, Holleman JH, Roush TS, Kowalchuk GJ, Bersin 
RM, Van Moore A, Simonton CA, 3rd, Rittase RA, Norton HJ, Kennedy TP. 
Prevention of contrast-induced nephropathy with sodium bicarbonate: a randomized 
controlled trial. JAMA. 2004; 291(19):2328-2334. PMID: 15150204 
10) Solomon RJ, Natarajan MK, Doucet S, Sharma SK, Staniloae CS, Katholi RE, 
Gelormini JL, Labinaz M, Moreyra AE. Cardiac Angiography in Renally Impaired 
Patients (CARE) study: a randomized double- blind trial of contrast-induced 
nephropathy in patients with chronic kidney disease. Circulation. 2007; 
115(25):3189-3196. PMID: 17562951 
11) Brown JR, Robb JF, Block CA, Schoolwerth AC, Kaplan AV, O'Connor GT, 
Solomon RJ, Malenka DJ. Does safe dosing of iodinated contrast prevent contrast-
induced acute kidney injury? Circ Cardiovasc Interv. 2010; 3(4):346-350. PMID: 
20587788; PMCID: PMC4843793 
12) Gurm HS, Dixon SR, Smith DE, Share D, Lalonde T, Greenbaum A, Moscucci M, 
Registry BMC. Renal function-based contrast dosing to define safe limits of 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
34 | P a g e  
radiographic contrast media in patients undergoing percutaneous coronary 
interventions. Journal of the American College of Cardiology. 2011; 58(9):907-914. 
PMID: 21851878 
13) Lameire N, Kellum JA. Contrast-induced acute kidney injury and renal support for 
acute kidney injury: a KDIGO summary (Part 2). Critical care (London, England). 
2013; 17(1):205. PMID: 23394215; PMCID: PMC4056805 
14) Brown JR, MacKenzie TA, Maddox TM, Fly J, Tsai TT, Plomondon ME, Nielson CD, 
Siew ED, Resnic FS, Baker CR, Rumsfeld JS, Matheny ME. Acute Kidney Injury 
Risk Prediction in Patients Undergoing Coronary Angiography in a National 
Veterans Health Administration Cohort With External Validation. Journal of the 
American Heart Association. 2015; 4(12). PMID: 26656858; PMCID: PMC4845295 
15) Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Borden WB, Bravata DM, 
Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern SM, Heit JA, 
Howard VJ, Huffman MD, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, 
Lisabeth LD, Magid D, Marcus GM, Marelli A, Matchar DB, McGuire DK, Mohler 
ER, Moy CS, Mussolino ME, Nichol G, Paynter NP, Schreiner PJ, Sorlie PD, Stein 
J, Turan TN, Virani SS, Wong ND, Woo D, Turner MB, American Heart Association 
Statistics C, Stroke Statistics S. Heart disease and stroke statistics-2013 update: a 
report from the American Heart Association. Circulation. 2013; 127(1): e6-e245. 
PMID: 23239837; PMCID: PMC5408511 
16) National Quality Forum. Safe practices for better healthcare 2006 Update: a 
consensus report. Washington, DC: National Quality Forum; 2007. 17. Chertow 
GM, Burdick E, Honour M, Bonventre JV, Bates DW. Acute kidney injury, mortality, 
length of stay, and costs in hospitalized patients. J Am Soc Nephrol. 2005; 
16(11):3365-3370. PMID: 16177006 
17) Jurado-Roman A, Hernandez-Hernandez F, Garcia-Tejada J, Granda-Nistal C, 
Molina J, Velazquez M, Albarran A, Tascon J. Role of hydration in contrast-induced 
nephropathy in patients who underwent primary percutaneous coronary 
intervention. The American journal of cardiology. 2015; 115(9):1174- 1178. PMID: 
25759106 
18) Luo Y, Wang X, Ye Z, Lai, Y., Yao Y, Li J, X. L. Remedial hydration reduces the 
incidence of contrast- induced nephropathy and short-term adverse events in 
patients with ST-segment elevation myocardial infarction: a single-center, 
randomized trial. Intern Med. 2014; 53:2265-2272. PMID: 25318787 
19) Maioli M, Toso A, Leoncini M, Micheletti C, Bellandi F. Effects of hydration in 
contrast-induced acute kidney injury after primary angioplasty: a randomized, 
controlled trial. Circulation. Cardiovascular interventions. 2011; 4(5):456-462. PMID: 
21972403 
20) Cigarroa RG, Lange RA, Williams RH, Hillis LD. Dosing of contrast material to 
prevent contrast nephropathy in patients with renal disease. Am J Med. 1989; 86(6 
Pt 1):649-652. PMID: 2729314 
21) The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough 
Improvement. In: Improvement IfH, ed. Boston 2003. 23. Hulscher ME, Schouten 
LM, Grol RP, Buchan H. Determinants of success of quality improvement 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
35 | P a g e  
collaboratives: what does the literature show? BMJ Qual Saf. 2013; 22(1):19-31. 
PMID: 22879447 
22) Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for 
the impact of quality improvement collaboratives: systematic review. BMJ (Clinical 
research ed. 2008; 336(7659):1491-1494. PMID: 18577559 PMCID: PMC2440907 
23) Boushon B, Provost L, Gagnon J, Carver P. Using a virtual breakthrough series 
collaborative to improve access in primary care. Jt Comm J Qual Patient Saf. 2006; 
32(10):573-584. PMID: 17066995 
24) Speroff T, Ely EW, Greevy R, Weinger MB, Talbot TR, Wall RJ, Deshpande JK, 
France DJ, Nwosu S, Burgess H, Englebright J, Williams MV, Dittus RS. Quality 
improvement projects targeting health care- associated infections: comparing 
Virtual Collaborative and Toolkit approaches. Journal of hospital medicine. 2011; 
6(5):271-278. PMID: 21312329 
25) Mehran R, Aymong ED, Nikolsky E, Lasic Z, Iakovou I, Fahy M, Mintz GS, Lansky 
AJ, Moses JW, Stone GW, Leon MB, Dangas G. A simple risk score for prediction 
of contrast-induced nephropathy after percutaneous coronary intervention: 
development and initial validation. Journal of the American College of Cardiology. 
2004; 44(7):1393-1399. PMID: 15464318 
26) Matheny ME, Ohno-Machado L, Resnic FS. Discrimination and calibration of 
mortality risk prediction models in interventional cardiology. J Biomed Inform. 2005; 
38(5):367-375. PMID: 16198996 
27) Siregar S, Nieboer D, Vergouwe Y, Versteegh MI, Noyez L, Vonk AB, Steyerberg 
EW, Takkenberg JJ. Improved Prediction by Dynamic Modeling: An Exploratory 
Study in the Adult Cardiac Surgery Database of the Netherlands Association for 
Cardio-Thoracic Surgery. Circulation. Cardiovascular quality and outcomes. 2016; 
9(2):171-181. PMID: 26933048 
28) Kumar A, Matheny ME, Ho KK, Yeh RW, Piemonte TC, Waldman H, Shah PB, 
Cope R, Normand SL, Donnelly S, Robbins S, Resnic FS. The data extraction and 
longitudinal trend analysis network study of distributed automated postmarket 
cardiovascular device safety surveillance. Circulation. Cardiovascular quality and 
outcomes. 2015; 8(1):38-46. PMID: 25491915; PMCID: PMC5130285 
29)  Matheny ME, Normand SL, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Vidi VD, 
Donnelly S, Resnic FS. Evaluation of an automated safety surveillance system 
using risk adjusted sequential probability ratio testing. BMC Med Inform Decis Mak. 
2011; 11:75. PMID: 22168892; PMCID: PMC3262755 
30) Resnic FS, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Donnelly S, Normand SL, 
Matheny ME. Automated surveillance to detect postprocedure safety signals of 
approved cardiovascular devices. JAMA. 2010; 304(18):2019-2027. PMID: 
21063011; PMCID: PMC5130290 
31) Matheny ME, Ohno-Machado L, Resnic FS. Risk-adjusted sequential probability 
ratio test control chart methods for monitoring operator and institutional mortality 
rates in interventional cardiology. Am Heart J. 2008; 155(1):114-120. PMID: 
18082501 
32) Woodall WH, Fogel SL, Steiner SH. The Monitoring and Improvement of Surgical-
Outcome Quality. J Qual Technol. 2015; 47(4):383-399. 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
36 | P a g e  
33) Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does Surgical 
Quality Improve in the American College of Surgeons National Surgical Quality 
Improvement Program An Evaluation of All Participating Hospitals. Annals of 
Surgery. 2009; 250(3):363-376. PMID: 19644350 
34) Zubkoff L, Neily J, Mills PD, Borzecki A, Shin M, Lynn MM, Gunnar W, Rosen A. 
Using a virtual breakthrough series collaborative to reduce postoperative respiratory 
failure in 16 Veterans Health Administration hospitals. Joint Commission journal on 
quality and patient safety / Joint Commission Resources. 2014; 40(1):11-20. PMID: 
24640453 
35) Matheny ME, Morrow DA, Ohno-Machado L, Cannon CP, Sabatine MS, Resnic FS. 
Validation of an automated safety surveillance system with prospective, randomized 
trial data. Med Decis Making. 2009; 29(2):247-256. PMID: 19015285; PMCID: 
PMC2743924 
36) Matheny ME, Resnic FS, Arora N, Ohno-Machado L. Effects of SVM parameter 
optimization on discrimination and calibration for post-procedural PCI mortality. J 
Biomed Inform. 2007; 40(6):688-697. PMID: 17600771; PMCID: PMC2170520 
37) Moen EL, Fricano-Kugler CJ, Luikart BW, O'Malley AJ. Analyzing Clustered Data: 
Why and How to Account for Multiple Observations Nested within a Study 
Participant? PLoS One. 2016; 11(1): e0146721. PMID: 26766425 PMCID: 
PMC4713068 
38) O'Malley AJ, Normand SL, Kuntz RE. Sample size calculation for a historically 
controlled clinical trial with adjustment for covariates. J Biopharm Stat. 2002; 
12(2):227-247. PMID: 12413242  
39) O'Malley AJ, Normand SL, Kuntz RE. Application of models for multivariate mixed 
outcomes to medical device trials: coronary artery stenting. Stat Med. 2003; 
22(2):313-336. PMID: 12520564 
40) Matheny ME, Ohno-Machado L, Resnic FS. Monitoring device safety in 
interventional cardiology. J Am Med Inform Assoc. 2006; 13(2):180-187. PMID: 
16357356; PMCID: PMC1447549  
41) Matheny ME, Gandhi TK, Orav EJ, Ladak-Merchant Z, Bates DW, Kuperman GJ, 
Poon EG. Impact of an automated test results management system on patients' 
satisfaction about test result communication. Arch Intern Med. 2007; 167(20):2233-
2239. PMID: 17998497 
42) Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, Bates DW, 
Gandhi TK. A randomized trial of electronic clinical reminders to improve 
medication laboratory monitoring. J Am Med Inform Assoc. 2008; 15(4):424-429. 
PMID: 18436905; PMCID: PMC2442256 
43) Matheny ME, Arora N, Ohno-Machado L, Resnic FS. Rare adverse event 
monitoring of medical devices with the use of an automated surveillance tool. AMIA 
Annu Symp Proc. 2007:518-522. PMID: 18693890; PMCID: PMC2655807 
44) Resnic FS, Normand SL, Piemonte TC, Shubrooks SJ, Zelevinsky K, Lovett A, Ho 
KK. Improvement in Mortality Risk Prediction After Percutaneous Coronary 
Intervention Through the Addition of a "Compassionate Use" Variable to the 
National Cardiovascular Data Registry CathPCI Dataset A Study From the 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
37 | P a g e  
Massachusetts Angioplasty Registry. Journal of the American College of 
Cardiology. 2011; 57(8):904-911. PMID: 21329835; PMCID: PMC3061352 
45) Elicker BM, Cypel YS, Weinreb JC. IV contrast administration for CT: a survey of 
practices for the screening and prevention of contrast nephropathy. Ajr. 2006; 
186(6):1651-1658. PMID: 16714655  
46) 48. Montgomery DC. Design and analysis of experiments. Eighth edition. ed. 
Hoboken, NJ: John Wiley & Sons, Inc.; 2013. 
47) Harvey G, Loftus-Hills A, Rycroft-Malone J, Titchen A, Kitson A, McCormack B, 
Seers K. Getting evidence into practice: the role and function of facilitation. J Adv 
Nurs. 2002; 37(6):577-588. PMID: 11879422 
48) Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based 
practice: a conceptual framework. Qual Health Care. 1998; 7(3):149-158. PMID: 
10185141; PMCID: PMC2483604 
49) 51. Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. 
Evaluating the successful implementation of evidence into practice using the 
PARiHS framework: theoretical and practical challenges. Implementation science: 
IS. 2008; 3:1. PMID: 18179688; PMCID: PMC2235887 
50) McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K. Getting 
evidence into 
51) practice: the meaning of 'context'. J Adv Nurs. 2002; 38(1):94-104. PMID: 11895535 
52) Rycroft-Malone J, Harvey G, Kitson A, McCormack B, Seers K, Titchen A. Getting 
evidence into practice: ingredients for change. Nurs Stand. 2002; 16(37):38-43. 
PMID: 12068568 
53) Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, 
Estabrooks C. Ingredients for change: revisiting a conceptual framework. Qual Saf 
Health Care. 2002; 11(2):174-180. PMID: 12448812; PMCID: PMC1743587 
54) Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A Guide for applying a 
revised version of the PARIHS framework for implementation. Implementation 
science: IS. 2011; 6:99. PMID: 21878092; PMCID: PMC3184083 
55) Cannon CP, Battler A, Brindis RG, Cox JL, Ellis SG, Every NR, Flaherty JT, 
Harrington RA, Krumholz HM, Simoons ML, Van De Werf FJ, Weintraub WS, 
Mitchell KR, Morrisson SL, Anderson HV, Cannom DS, Chitwood WR, Cigarroa JE, 
Collins-Nakai RL, Gibbons RJ, Grover FL, Heidenreich PA, Khandheria BK, 
Knoebel SB, Krumholz HL, Malenka DJ, Mark DB, McKay CR, Passamani ER, 
Radford MJ, Riner RN, Schwartz JB, Shaw RE, Shemin RJ, Van Fossen DB, 
Verrier ED, Watkins MW, Phoubandith DR, Furnelli T. American College of 
Cardiology key data elements and definitions for measuring the clinical 
management and outcomes of patients with acute coronary syndromes. Journal of 
the American College of Cardiology. 2001; 38(7):2114-2130. PMID: 11738323 
56) Weisbord SD, Gallagher M, Kaufman J, Cass A, Parikh CR, Chertow GM, Shunk 
KA, McCullough PA, Fine MJ, Mor MK, Lew RA, Huang GD, Conner TA, Brophy 
MT, Lee J, Soliva S, Palevsky PM. Prevention of Contrast-Induced AKI: A Review 
of Published Trials and the Design of the Prevention of Serious Adverse Events 
following Angiography (PRESERVE) Trial. Clin J Am Soc Nephrol. 2013. PMID: 
23660180; PMCID: PMC3805082 

PROTOCOL  
(Version #: 6, version date: 9-21-20) 
 
38 | P a g e  
57) Brar SS, Aharonian V, Mansukhani P, Moore N, Shen AY, Jorgensen M, Dua A, 
Short L, Kane K. Haemodynamic-guided fluid administration for the prevention of 
contrast-induced acute kidney injury: the POSEIDON randomised controlled trial. 
Lancet. 2014; 383(9931):1814-1823. PMID: 24856027 
58) Brown JR, Rezaee ME, Nichols EL, Marshall EJ, Siew ED, Matheny ME. Incidence 
and In-Hospital Mortality of Acute Kidney Injury (AKI) and Dialysis-Requiring AKI 
(AKI-D) After Cardiac Catheterization in the National Inpatient Sample. J Am Heart 
Assoc. 2016; 5(3). PMID: 27068629; PMCID: PMC4943252 
59)  Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina 
MJ, Kattan MW. Assessing the performance of prediction models: a framework for 
traditional and novel measures. Epidemiology. 2010; 21(1):128-138. PMID: 
20010215; PMCID: PMC3575184 
60) Cox DR. Two further applications of a model for binary regression. Biometrika. 
1958; 45:562–565. 62. Leung FH, Savithiri R. Spotlight on focus groups. Can Fam 
Physician. 2009; 55(2):218-219. PMID: 19221085; PMCID: PMC2642503 
61) Stewart D SP. Focus group research: exploration and discovery. Thousand Oaks, 
CA: Sage Publications, Inc; 1998. 
62) Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual 
Health Res. 2005; 15(9):1277-1288. PMID: 16204405 
63) Charmaz K. Constructing Grounded Theory: A Practical Guide Through Qualitative 
Analysis Thousand Oaks: Sage; 2006. 
64) Strauss A, Corbin J. Basics of Qualitative Research. London: Sage; 1990. 
65) Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction 
to qualitative methods in health and health services research. BMJ. 1995 Jul 1; 
311:42-45. PMID: 7613329; PMCID: PMC2550091 
66) Emerson R, Fretz R, Shaw L. Writing Ethnographic Fieldnotes. Chicago: University 
of Chicago Press; 1995. 
67) Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. 
2nd Edition ed. Thousand Oaks: Sage Publications; 1994.  
68) Corbin J, Strauss A. Basics of Qualitative Research 3e. Los Angeles: Sage 
Publications; 2008.Sustainability Model and Guide. In: Improvement IfIa, ed: NHS 
England;  
69) Provost LP, Murray SK. The health care data guide: learning from data for 
improvement. 1st ed. San Francisco, CA: Jossey-Bass; 2011.