Integration of Machine Learning and Clinical Decision Support to Prevent Postoperative Delirium in Patients With Cognitive Impairment
Phase: NA
Conditions: Post-operative Delirium, Decision Support Systems, Clinical, Cognitive Impairment
Sponsor: Icahn School of Medicine at Mount Sinai

Integration of Machine Learning and Clinical Decision Support to Prevent 
Postoperative Delirium in Patients With Cognitive Impairment 
NCT05777187
PI: Ira Hofer, MD
Document Date: 9-14-2023

Protocol Template, Version 3.0 
i 
INTEGRATION OF MACHINE LEARNING AND CLINICAL DECISION SUPPORT 
TO PREVENT POSTOPERATIVE DELIRIUM IN PATIENTS WITH COGNITIVE 
IMPAIRMENT 
Principal Investigator: 
Ira Hofer, MD 
Associate Professor, Department of Anesthesiology and Medicine 
Icahn School of Medicine at Mount Sinai 
Supported by: 
The National Institute on Aging 
FY22_Demo1_Hofer 
 (Any modification to the protocol should be annotated on the coversheet or in an appendix.  The 
annotation should note the exact words that are changed, the location in the protocol, the date 
the modification was approved by the Executive Committee, and the date it became effective.)  
Version 3 (Please change version number with each amendment) 
09/14/2023 

Protocol Template, Version 3.0 
ii 
Summary of Changes Version 3  
 
The use of strictly Physician Assistants to complete the 4AT post-operative delirium screen was 
changed to include any medical provider that may be on the care team. This was done to increase 
the post-operative screens being done so the primary endpoint could be reached in a timely 
manner. Previously, using only the PA providers did not allow for comprehensive post-operative 
delirium screening for all study participants due to the difference in services that the patients 
were on. It was noted that many patients were on medical services rather than surgical services 
where the PA providers were available.  
 
 
Page/Section 
Description of Change 
Rationale  
Page xii/Design 
and Outcomes  
Changed “physician assistant (PA)” to 
“medical provider” under secondary outcomes 
in the following sentence: “The impact of 4AT 
screening prompts on medical provider post-
operative care practices”  
Replaced physician assistant with 
medical provider to broaden use of 
clinic care team to complete 4AT 
assessments. 
Page 3/1. Study 
Objectives/1.2 
Secondary 
Objectives  
Changed “physician assistant (PA)” to 
“medical provider” in bullet point e) “The 
impact of 4AT screening prompts on medical 
provider post-operative care practices” 
Same as above  
Page 6/3. Study 
Design  
Changed “PAs” to “medical provider” and 
corrected spelling of second use of the word 
“notification” in sentence “Postoperatively, for 
patients in both groups the medical providers 
caring for the patients will receive a 
notification to screen the patient for POD, and 
if the screen is positive a notification to screen 
the patient on each subsequent day until the 
delirium resolves.” 
Same as above and correction of 
spelling.  
Page 8/4. 
Selection and 
Enrollment of 
Participants/4.1 
Inclusion Criteria  
Changed sentence “For providers, all 
anesthesia providers (anesthesiologist, resident, 
and nurse anesthetist and PAs on the surgical 
services in all the hospitals will be included in 
the study.” to “For providers, all anesthesia 
providers (anesthesiologist, resident, and nurse 
anesthetist and medical providers completing 
the 4AT screens, including PAs on the surgical 
services, in all the hospitals will be included in 
the study” 
This sentence was changed to add all 
medical providers completing post-
operative delirium screens in addition 
to surgical physician assistants to the 
inclusion criteria for assessing clinical 
care.  
Page 11/5. Study 
Interventions/5.4 
Adherence 
Assessment   
Changed sentence “For the notification of the 
PAs we will examine postoperative length of 
stay as a time series before and after the 
intervention and for enrolled patients as 
opposed to patients who are unenrolled 
because they have had a prior surgical 
procedure during the study period” to “For the 
post-operative delirium screen (4AT) 
This sentence was changed to remove 
the mention of PA (physician 
assistant) specifically and focus 
directly on the 4AT assessment done 
by any medical provider on the care 
team.  

Protocol Template, Version 3.0 
iii 
notifications we will examine postoperative 
length of stay as a time series before and after 
the intervention and for enrolled patients as 
opposed to patients who are unenrolled 
because they have had a prior surgical 
procedure during the study period.”  
Page 13/6 Study 
Procedures/6.2 
Description of 
Evaluations/6.2.3 
Follow-up Visits  
Changed sentence “Patients will have routine 
postoperative care with the addition of 4AT 
assessments by the physician assistant on each 
postoperative day as part of routine care.” to 
“Patients will have routine postoperative care 
with the addition of at least one 4AT 
assessment by a medical provider on the care 
team”.  Added sentence “If post-operative 
delirium is present in the screen, providers will 
be encouraged to re-screen the following day, 
but only one assessment is needed for the 
present study’s primary end point.” 
Replaced PA with medical provider to 
broaden use of clinic care team to 
complete 4AT assessments. 
Additional sentence added to clarify 
amount of 4AT screens, and what is 
needed for primary outcome purposes 
as oppose to clinical.  
Page 17/9.4 
Outcomes/9.4.2 
Secondary 
Outcomes  
Replaced “physician assistant staff” with 
“medical providers”.  
Replaced PA with medical provider to 
broaden use of clinic care team to 
complete 4AT assessments. 
Page 18/9.5 Data 
Analyses 
Under bullet point d), changed “physician 
assistant” to “medical provider” in first 
sentence in paragraph: “Analysis of delirium 
screening prompt on medical provider post-
operative care practices”. Changed “the PAs” 
to “a medical provider” and “surgical service” 
to “care team” in second sentence: “At present 
not all patients with CI are screened for POD 
by their providers. Part of our intervention will 
be ensuring that all these patients are screened 
for POD by a medical provider on their care 
team.” 
Replaced PA with medical provider to 
broaden use of clinic care team to 
complete 4AT assessments. Replaced 
surgical service with care team to 
ensure accuracy of provider. Not only 
surgical services are part of patient’s 
care team.  
 
 
Summary of Changes Version 2: The use of the Confusion Assessment Method (CAM) was 
transitioned to the use of the 4 A's Test. These tests are similar both are reliable in measuring 
post-operative delirium. Anywhere in the protocol where CAM was previously named was 
changed to 4AT.  
 
Rationale  
Use of the CAM was changed to use of the 4AT for logistic reasons in the study.  
Change in use of CAM to 4AT. It was noted that the impact that the alert for need for screening may impact 
PA behavior.  
Same as above. 
Stratification of patients changed from risk grouping to by surgical service in conjunction with change of 
staff assessing delirium changing from nursing staff to PA staff.  

Protocol Template, Version 3.0 
iv 
Same as above.  
Misspelling.  
Including additional detail to asses adherence.  
Change in use of CAM to 4AT. 
Change in use of CAM to 4AT as well as specifiying that PA staff will complete post-operative delirium 
assessments.  
Change in use of CAM to 4AT. 
Change in use of CAM to 4AT and added description and justification for use of 4AT.  
Addition of measurement on PA care management as related to the study.  
The CAM no longer being used for study, and the 4AT is able to assess patients that are unable to respond 
due to drowsiness or sickness.  
It was noted that the impact that the alert for need for screening may impact PA behavior. 
 
  
 
 

Protocol Template, Version 3.0 
v 
TABLE OF CONTENTS 
Page 
PREFACE ...................................................................................................................................... ii 
FULL PROTOCOL TITLE .......................................................... Error! Bookmark not defined.i 
SUMMARY OF CHANGES ........................................................... Error! Bookmark not defined. 
TABLE OF CONTENTS ........................................................................................................... vii 
PRÉCIS .......................................................................................................................................... x 
Study Title................................................................................................................................... xix 
Objectives ................................................................................................................................ xix 
Design and Outcomes .............................................................................................................. xix 
Interventions and Duration ...................................................................................................... xix 
Sample Size and Population..................................................................................................... xix 
STUDY TEAM ROSTER............................................................................................................. 1 
Principal Investigator:  ............................................................................................................. 1 
Co-Investigators:  ..................................................................................................................... 1 
PARTICIPATING STUDY SITES ............................................................................................. 2 
1 
Study objectives ..................................................................................................................... 3 
1.1 
Primary Objective ........................................................................................................... 3 
1.2 
Secondary Objectives...................................................................................................... 3 
2 
BACKGROUND AND RATIONALE ................................................................................ 3 
2.1 
Background on Condition, Disease, or Other Primary Study Focus .............................. 3 
2.2 
Study Rationale ............................................................................................................... 4 
3 
STUDY DESIGN ................................................................................................................... 5 
4 
SELECTION AND ENROLLMENT OF PARTICIPANTS ............................................ 7 
4.1 
Inclusion Criteria ............................................................................................................ 7 
4.2 
Exclusion Criteria ........................................................................................................... 8 
4.3 
Study Enrollment Procedures ......................................................................................... 8 

Protocol Template, Version 3.0 
vi 
5 
STUDY INTERVENTIONS .............................................................................................. 10 
5.1 
Interventions, Administration, and Duration ................................................................ 10 
5.2 
Handling of Study Interventions ................................................................................... 10 
5.3 
Concomitant Interventions ............................................................................................ 11 
5.4 
Adherence Assessment ................................................................................................. 11 
6 
STUDY PROCEDURES .................................................................................................... 11 
6.1 
Schedule of Evaluations .............................................................................................. 132 
6.2 
Description of Evaluations ............................................................................................ 13 
6.2.1 Screening Evaluation ................................................................................................ 13 
6.2.2 Enrollment, Baseline, and/or Randomization ........................................................... 13 
6.2.3 Follow-up Visits........................................................................................................ 13 
7 
SAFETY ASSESSMENTS ................................................................................................. 13 
7.1 
Specification of Safety Parameters ............................................................................... 13 
7.2 
Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters .... 13 
7.3 
Adverse Events and Serious Adverse Events ............................................................... 13 
7.3.1 Reporting Procedures ................................................................................................ 14 
7.3.2 Follow-up for Adverse Events .................................................................................. 14 
7.4 
Safety Monitoring ......................................................................................................... 14 
8 
INTERVENTION DISCONTINUATION........................................................................ 14 
9 
STATISTICAL CONSIDERATIONS .............................................................................. 14 
9.1 
General Design Issues ................................................................................................... 14 
9.2 
Sample Size and Randomization .................................................................................. 15 
9.2.1 Treatment Assignment Procedures ........................................................................... 15 
9.3 
Interim analyses and Stopping Rules ............................................................................ 16 
9.4 
Outcomes ...................................................................................................................... 17 
9.4.1 Primary outcome ....................................................................................................... 17 
9.4.2 Secondary outcomes ................................................................................................. 17 
9.5 
Data Analyses ............................................................................................................... 17 
10 DATA COLLECTION AND QUALITY ASSURANCE ................................................ 19 
10.1 
Automated Collection of Data from the EHR ............................................................... 19 
 
 

Protocol Template, Version 3.0 
vii 
11 PARTICIPANT RIGHTS AND CONFIDENTIALITY ................................................. 19 
11.1 
Institutional Review Board (IRB) Review .................................................................... 20 
11.2 
Informed Consent Forms .............................................................................................. 20 
11.3 
Participant Confidentiality ............................................................................................ 20 
11.4 
Study Discontinuation ................................................................................................... 20 
12 ETHICAL CONSIDERATIONS ....................................................................................... 20 
13 COMMITTEES................................................................................................................... 20 
14 PUBLICATION OF RESEARCH FINDINGS ................................................................ 20 
15 REFERENCES .................................................................................................................... 22 
I. Procedures Schedule 
II. Informed Consent Form Template 
  
III. Other (add as many appendices as necessary) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Protocol Template, Version 3.0 
viii 
PRÉCIS 
Over one million surgical procedures are performed annually on patients with cognitive 
impairment (CI) in the United States. Up to 50% of these patients will develop 
postoperative delirium (POD) – a fluctuating disturbance in attention, altered level of 
consciousness and disorganized thinking lasting up to one-week post-procedure. 
POD is associated with in-hospital complications, longer hospitalization, post-discharge 
worsening of functional and psychological health, progressive cognitive decline, 
dementia, and death. 
 
It is estimated that 30-40% of POD cases are preventable and optimal perioperative 
management is associated with POD prevention. For example, glycemic control, 
medication management, blood pressure titration and anesthetic depth have all been 
associated with POD. Thus, anesthesiologists have an opportunity to improve safety in an 
area that is “directly attributable to anesthesia” with a serious impact on patient 
outcomes. While these studies demonstrate the potential for perioperative interventions to 
prevent POD, providers often struggle to identify at risk patients and ensure that the 
increasing number of best practice guidelines are consistently applied to each patient. 
 
The modern electronic health record (EHR) provides an unprecedented opportunity to 
address this shortcoming. However, a key challenge in applying best practice guidelines 
to patients is often patient identification. While clinicians struggle to read and absorb all 
the data in the EHR, computers excel at consistently and accurately processing large 
amounts of data. Clinical decision support (CDS) pathways have been demonstrated to be 
highly effective at identifying at risk patient populations and increasing anesthesiologist 
adherence to best practices such as glucose control, blood pressure, anesthetic 
management, lung protective ventilation, thereby improving postoperative outcomes. 
 
We have extensive experience in the pathogenesis and prevention of POD, EHR data, 
integrating risk stratification systems into the EHR to identify patients with CI, and 
providing perioperative decision support. We propose leveraging our experience in 
informatics and POD research to perform a prospective randomized controlled trial 
(RCT) that will test the effectiveness of a CDS system to promote adherence to best with 
the goal of decreasing POD in patients with baseline CI. 
 
In particular the protocol will involve 1) identifying patients with baseline cognitive 
impairment (CI) scheduled for surgery 2) randomizing these cases to a) CDS prompts to 
the anesthesiologist regarding 12 perioperative best practices or b) no prompts (controls) 
and measuring the efficacy of the protocol at reducing POD as measured by the “4 A’s 
Test” (4AT).  

Protocol Template, Version 3.0 
ix 
Study Title  
Integration of Machine Learning and Clinical Decision Support to Prevent 
Postoperative Delirium in Patients with Cognitive Impairment 
Objectives  
The primary objective of this pilot study is to increase adherence to best practices by 
anesthesia providers in order to reduce postoperative delirium (POD) in patients with 
underlying cognitive impairment. In order to accomplish this goal, we will identify 
patients with baseline cognitive impairment undergoing surgery using structured data 
from the electronic health record (EHR) and an existing natural language processing 
(NLP) model. Patients will then be randomized to usual care or clinical decision support 
(CDS) alerts directed at the anesthesia provider for 12 perioperative best practices. 
Design and Outcomes   
Design: Prospective Stage IV pragmatic randomized controlled trial 
Setting: Mount Sinai Health System – four hospitals 
Population: Patients with baseline cognitive impairment undergoing surgery.  
Intervention: 12 clinical best practices in 5 domains (see subsequent slides) 
Outcomes: 
Primary outcome:  
Postoperative delirium as measured by the 4 A’s Test (4AT) 
Secondary outcomes: 
The ability of CDS prompts to increase best practice adherence 
The impact of individual and overall practice adherence on POD prevention 
The efficacy of this tool in various sociodemographic subgroups to ensure that it is 
equitable. 
The impact of 4AT screening prompts on medical provider post-operative care practices  
 
Interventions and Duration  
In order to accomplish our study goal, we will identify patients with baseline cognitive 
impairment undergoing surgery using structured data from the electronic health record 
(EHR) and an existing natural language processing (NLP) model. Patients will then be 
randomized to usual care or clinical decision support (CDS) alerts directed at the 
anesthesia provider for 12 perioperative best practices. 
 
The study will run for 12 months.  
Sample Size and Population  
Study population (estimated): 12,000 prospectively and 12,000 patients as historical 
controls. 
We will identify patients with baseline cognitive impairment undergoing surgery using 
structured data from the electronic health record (EHR) and an existing natural language 

Protocol Template, Version 3.0 
x 
processing (NLP) model. Patients will be randomized to usual care or clinical decision 
support (CDS) alerts directed at the anesthesia provider for 12 perioperative best practice.

Protocol Template, Version 3.0 
1 
STUDY TEAM ROSTER  
Principal Investigator: 
          Dr. Ira Hofer 
   One Gustave L. Levy Place, 
 
New York, NY, 10029 
 
212-241-7473 
 
Email address: ira.hofer@moungsinai.org 
 
Main responsibilities: Dr. Hofer will be responsible for study 
design, coordinating meetings with the other investigators, and 
overseeing study execution. /Key roles: Principal Investigator 
Co-Principal Investigator: 
     Dr. Susana Vacas, MD PhD 
5 Fruit Street, Gray Bigelow 444-K 
Boston, MA 02114 
310-562-9407 
Main responsibilities: Dr. Vacas will be responsible for CDS 
pathway design and statistical analysis. Her role will not 
require IRB oversight /Key roles: Co-Investigator 
 
Co-Investigators: 
 
 
   Dr. Girish Nadkarni 
 
One Gustave L. Levy Place, Box 1243 
 
New York, NY, 10029 
 
Telephone: 212-241-1385 
 
Fax: 212-987-0389 
 
Email address: girish.nadkarni@mountsinai.org 
Main responsibilities: Dr. Nadkarni will be responsible for 
deployment of the NLP algorithm into clinical care. /Key roles: 
Co-Investigator 
 
 
 
 
Dr. Matthew Levin 
 
 
 
One Gustave L. Levy Place, 
 
New York, NY, 10029 
 
Telephone: 212-241-8382 
 
Email address: matthew.levin@mssm.edu 
 
Main responsibilities: Dr. Levin will be responsible for 
deployment of the NLP algorithm into clinical care and CDS 
implimentation/Key roles: Co-Investigator 
Dr. Alex Federman 
 
17 East 102nd Street, 7th Floor 
 
New York, NY 10029  
 

Protocol Template, Version 3.0 
2 
 
Telephone: 212-659-8551 
 
Email address: alex.federman@mountsinai.org 
 
Main responsibilities: Dr. Federman will be responsible for 
designing the tools to identify patients with CI. /Key roles: Co-
Investigator 
 
PARTICIPATING STUDY SITES  
 
Catherine Sarkisian, MD 
 
200 Medical Plaza Driveway Suite 365 
Los Angeles, CA 90095 
(310) 206-8272 
Main responsibilities: Dr. Sarkisian will be responsible for 
assisting with the clinical trial design. There is no need for 
IRB oversight in her role. /Key roles: Co-Investigator 
 
 

Protocol Template, Version 3.0 
3 
1 
STUDY OBJECTIVES 
1.1 Primary Objective 
The primary objective of this pilot study is to increase adherence to best practices by 
anesthesia providers in order to reduce postoperative delirium (POD) in patients with 
underlying cognitive impairment. In order to accomplish this goal we will identify 
patients with baseline cognitive impairment undergoing surgery using structured data 
from the electronic health record (EHR) and an existing natural language processing 
(NLP) model. Patients will then be randomized to usual care or clinical decision support 
(CDS) alerts directed at the anesthesia provider for 12 perioperative best practices. 
 
1.2 Secondary Objectives 
The secondary objective is to measure: 
– a) The ability of CDS prompts to increase best practice adherence 
– b) The impact of individual and overall practice adherence on POD prevention 
– c) The efficacy of this tool in various sociodemographic subgroups to ensure that 
it is equitable. 
– e) The impact of 4AT screening prompts on medical provider post-operative care 
practices 
 
2 
BACKGROUND AND RATIONALE  
2.1 Background on Condition, Disease, or Other Primary Study Focus 
Over one million surgical procedures are performed annually on patients with cognitive 
impairment (CI) in the United States1,2. Up to 50% of these patients will develop 
postoperative delirium (POD)3,4 – a fluctuating disturbance in attention, altered level of 
consciousness and disorganized thinking lasting up to one week post-procedure.5,6 POD 
is associated with in-hospital complications, longer hospitalization, post-discharge 
worsening of functional and psychological health, progressive cognitive decline, 
dementia, and death.1,7-11 
It is estimated that 30-40% of POD cases are preventable12,13 and optimal perioperative 
management is associated with POD prevention.14-19 For example, glycemic control20, 
medication management21, blood pressure titration16,17,22 and anesthetic depth23 have all 
been associated with POD. Thus, anesthesiologists have an opportunity to improve safety 
in an area that is “directly attributable to anesthesia” with a serious impact on patient 
outcomes. While these studies demonstrate the potential for perioperative interventions to 
prevent POD, providers often struggle to identify at risk patients and ensure that the 
increasing number of best practice guidelines are consistently applied to each patient.  
The modern electronic health record (EHR) provides an unprecedented opportunity to 
address this shortcoming. However, a key challenge in applying best practice guidelines 
to patients is often patient identification. While clinicians struggle to read and absorb all 
the data in the EHR, computers excel at consistently and accurately processing large 

Protocol Template, Version 3.0 
4 
amounts of data. Clinical decision support (CDS) pathways have been demonstrated to be 
highly effective at identifying at risk patient populations and increasing anesthesiologist 
adherence to best practices such as glucose control24-27, blood pressure28-30, anesthetic 
management28,31, lung protective ventilation32, thereby improving postoperative 
outcomes.  
We have extensive experience in the pathogenesis and prevention of POD, EHR data, 
integrating risk stratification systems into the EHR to identify patients with CI, and 
providing perioperative decision support. We propose leveraging our experience in 
informatics and POD research to perform a prospective randomized controlled trial 
(RCT) that will test the effectiveness of a CDS system to promote adherence to best with 
the goal of decreasing POD in patients with baseline CI. 
 
2.2 Study Rationale 
1. Preventing POD is one of the most important priorities for patients with cognitive 
impairment (CI). Adults over 65 years of age are 16% of the United States population and 
40% of surgical volume. Nearly 30% of this population has preoperative CI. Patients with 
CI have an overall incidence of postoperative delirium (POD) of 30-50% and an 
incidence as high as 70% depending on existing comorbidities or type of surgery. POD 
increases all-cause mortality by 10-20% for every 48 hours of delirium and is associated 
with increased duration of postoperative mechanical ventilation, ICU and hospital length 
of stay. POD is linked to long-term cognitive decline, including an 8-fold amplified risk 
for dementia, and intensified functional decline. It is estimated that worsening long-term 
cognitive dysfunction occurs in more than 10% of older non-cardiac surgical patients. 
The annual costs associated with POD are estimated to be over $150 billion. National 
associations, including the American Geriatrics Society (AGS) and the American Society 
of Anesthesiologists (ASA), have issued calls to action to prevent POD. 
2. Intraoperative best practices are efficacious in prevention of POD. 30-40% of POD 
cases are preventable and intraoperative management is critical to POD prevention. 
Inappropriate medications (defined by The Beers criteria) have been linked to POD and 
glycemic control (avoiding both hypoglycemia and hyperglycemia) has been shown to 
prevent POD. Several studies have also found an association between intraoperative 
hemodynamic management with the incidence of POD. Given the disruptive effect of 
anesthetics on the brain, some have hypothesized that anesthetics might contribute to 
POD directly – it has been shown that titrating anesthetic dose to age-adjusted anesthetic 
concentrations or to the electroencephalogram (EEG) reduces the incidence of POD and 
improves cognitive function at one year.  
3. Multipronged interventions are more effective at preventing postoperative 
complications. Several studies have demonstrated the ability of enhanced recovery after 
surgery (ERAS) pathways to improve perioperative outcomes. The studies span a variety 
of surgical procedures and patient populations and have consistently shown that multi-
intervention pathways are more successful at improving outcomes than singular 
interventions. To date there have been no ERAS pathways to prevent POD studied 
prospectively. 

Protocol Template, Version 3.0 
5 
4. Perioperative providers struggle to identify patients at risk of POD. POD develops 
through an interaction of predisposing patient vulnerabilities and precipitating factors that 
occur throughout the perioperative period. While CI and similar cognitive features are 
risk factors for POD, other risk factors such as metabolic derangements, polypharmacy, 
and poorly controlled pain are not directly related to cognition. Similarly, precipitating 
factors such as surgical duration, invasiveness of surgery, and some medications can 
directly and indirectly trigger neurological change. Several perioperative risk 
stratification instruments for POD have been published, however, they are based on small 
studies, limited predictive features, and narrow inclusion criteria, thereby limiting 
generalizability. Further, work by our group has shown that anesthesiologists often 
struggle to assimilate all of the information scattered throughout the modern electronic 
health record (EHR), leading them to underestimate patient risk. As a result, providers at 
the point of care – the operating room – often fail to identify patients at risk of POD. The 
struggle to apply the diverse array of existing risk instruments is exacerbated by the fact 
that anesthesiologists are often unaware that patients had POD, as it occurs well after 
their contact with the patient, making their “clinical intuition” less relevant. 
5. Clinical Decision Support (CDS) has been shown to increase provider awareness of 
risk and adherence to best practices. CDS refers to alerts imbedded in an EHR at the 
point of care to modify provider behavior. While CDS has shown mixed results in other 
settings it has been extremely effective at modifying perioperative provider behavior. 
Institutions have successfully used perioperative CDS to in areas such as glycemic 
control, postoperative nausea and vomiting66, anesthetic depth blood pressure control, 
and antibiotic prophylaxis. However, too many CDS alerts can cause burnout amongst 
providers and decrease effectiveness making generalized CDS alerts less effective than 
those targeted to high-risk patients. Thus, integrating risk-stratification targeting high-risk 
patients with CDS prompts to adhere to best practices carries the potential to be more 
effective for the patient and the provider’s well-being and practice. 
6. Overall Clinical Significance: POD is a major cause of morbidity in patients with CI, 
and perioperative management is implicated in the incidence of POD. Implementation of 
the various perioperative best practice guidelines for the prevention of POD remains 
inconsistent due to challenges of identifying patients at risk for POD and the rapid nature 
of perioperative care. This proposal will prevent POD by identifying patients with CI by 
performing a stage IV effectiveness randomized controlled trial using CDS to promote 
provider adherence to perioperative best practices while measuring the effectiveness of 
the CDS on reducing POD. 
 
3 
STUDY DESIGN 
Location: The trial will be conducted at the Mount Sinai Health System (MSHS) in 
Manhattan, containing one of the most diverse populations in the world, with 130 operating 
rooms across four hospitals, performing more than 100,000 surgical procedures each year. 
 
Identification of patients with CI: Patients will be identified as having CI if the established 
NLP model yields a positive prediction or if the patient has structured data in the EHR 
indicative of cognitive impairment (i.e.  Mini-Mental score exam <24, ICD codes for CI, 

Protocol Template, Version 3.0 
6 
etc.). The use of multiple criteria is crucial because our previous work has shown that using 
multiple techniques to identify clinical phenotypes is superior to any single technique 
independently. These models and techniques already exist at MSHS and the NLP model is 
currently under peer review.  
 
Randomization and risk stratification:  In the perioperative period an anesthesia attending 
supervises multiple junior providers in pairings that change daily. Thus, randomization at 
the level of the provider is impossible and patients will be randomized at the level of the 
surgical procedure. In order to avoid issues with statistical analysis, only the first surgical 
procedure for each patient will be included in the study. For each procedure, patients will be 
stratified based on their surgical service and the hospital. For each stratum, patients will be 
alternately assigned to the intervention or control groups (i.e., 1st intervention, 2nd control, 
etc.). The control group will receive usual care (no prompts) and the intervention group will 
receive an alert to the anesthesiologist that the patient is at increased risk for POD with CDS 
prompts for clinical best practices. Postoperatively, for patients in both groups the medical 
providers caring for the patients will receive a notification to screen the patient for POD, and 
if the screen is positive a notification to screen the patient on each subsequent day until the 
delirium resolves.  
 
Intervention Development: The intervention will be developed in line with the IMPACT 
Collaboratory steps for creating effective value propositions. Consistent with previous CDS 
deployments done by our team, the proposed interventions will be shared with departmental 
leadership for input and feedback on notification design and content. Specific feedback 
about frequency and thresholds of alerts will be solicited to gain buy-in and attempt to 
mitigate alert fatigue. Non-disruptive alerts that avoid hard stops will be used. Prior to 
intervention go live, education will be provided to the anesthesiologists via email 
communications as well as presentations at forums such as grand rounds, faculty meetings, 
residency meetings, etc. The education will specify that these alerts are part of a prospective 
randomized controlled trial. Selection of intraoperative best practices: The 12 intraoperative 
best practices across five domains that will be encouraged with CDS is contained in Table 1. 

Protocol Template, Version 3.0 
7 
 
Interventions were selected from the American College of Surgeons, the AGS, and ASA list 
of best practices, if they were relevant to the perioperative period, suggested as a specific 
objective action, and are documented in the EHR during usual care. All interventions are 
performed by the anesthesiologist and measured in the EHR. 
 
CDS Notifications of the POD risk and best practice recommendations: All CDS 
notifications will be designed to be non-disruptive and will be displayed on a banner and 
sidebars and will not contain hard stops. Thus, the notifications will be informative of the 
patient risk and recommended best practices but will not interrupt clinical workflows. The 
preoperative notification display in the banner on the top of the EHR will contain 
notifications of the presence of CI, risk for POD, and preoperative recommendations. 
Similar alerts are already in use at MSHS. In the control arm, there will be no notification on 
the EHR screen. Intraoperative recommendations will be non-disruptive notifications on the 
side of the intraoperative record; postoperative notifications will be incorporated into the 
CDS alerts already in place for medication orders in the postoperative anesthesia care unit. 
All of the interventions will be designed without any hard stops and in a way that avoids 
disruptions in the clinical workflow (i.e., no new pop-up alerts will be used) as our previous 
work has shown non-disruptive interventions have been highly effective at changing 
provider behavior. Similar notifications for other purposes are already in place at the MSHS 
and can be created within several weeks. Our team has extensive experience creating such 
notifications in the EHR. 
 
4 
SELECTION AND ENROLLMENT OF PARTICIPANTS  
4.1 Inclusion Criteria  
Inclusion Criteria: We will include patients with CI undergoing surgery, approximately 
12,000 patients. For each patient only the first surgical procedure during the study 
period will be included. Patients will be identified as having CI if the established NLP 

Protocol Template, Version 3.0 
8 
model yields a positive prediction (see preliminary data) or if the patient has structured 
data in the EHR indicative of cognitive impairment (i.e. Mini-Mental score exam <24, 
ICD codes for CI, etc.). The use of multiple criteria is crucial because our previous 
work has shown that using multiple techniques to identify clinical phenotypes is 
superior to any single technique independently. These models and techniques already 
exist at MSHS and there is no barrier to using the EHR data to identify patients with CI. 
 
Historical controls will be identified from the existing EHR data warehouse (included) 
if they would have met criteria for inclusion in the one year prior to study launch. 
 
For providers, all anesthesia providers (anesthesiologist, resident and nurse anesthetist) 
and medical providers completing the 4AT screens, including PAs on the surgical 
services, in all of the hospitals will be included in the study  
 
4.2 Exclusion Criteria  
Patients will be excluded if this not their first surgery since study start, if they do not have 
cognitive impairment based on EHR data, or if the surgery is for organ donation.  
 
There will be no exclusion criteria for providers or historical controls.  
4.3 Study Enrollment Procedures  
All participants who meet the inclusion/exclusion criteria will be enrolled to the study and 
we will be obtaining a waiver of informed consent from the IRB based on the following 
criteria: 
Waiver Criteria 
Justification 
Prospective Patients 
Prospective Provider 
who is receiving the 
CDS prompts 
Historical Controls  
the research involves 
no 
more 
than 
minimal risk to the 
subjects 
 recommendations 
strictly provide best 
practice 
guidance 
and do not mandate 
adherence to the 
recommendations 
 All 
recommendations 
are in current best 
practice guidelines 
 CDS alerts are 
non-disruptive to 
clinical care (i.e. 
no hard stops) 
and similar to 
multiple 
alerts 
already in use 
 Data 
will 
be 
deidentified 
and 
the only risk is in 
data loss 
the research could 
not practicably be 
carried out without 
the requested waiver 
or alteration 
 The 
trial 
is 
estimated to enroll 
about 
12,000 
patients over one 
year 
with 
some 
procedures 
taking 
 Anesthesia 
providers 
are 
assigned to cases 
often at the last 
minute 
and 
change 
 Contact 
information 
for 
historical patients 
may 
have 
changed or would 

Protocol Template, Version 3.0 
9 
place emergently at 
off hours 
throughout 
the 
day. It would be 
impossible 
to 
enroll 
and 
remove patients 
dynamically with 
these 
frequent 
changes 
no 
longer 
be 
available. 
if 
the 
research 
involves 
using 
identifiable 
private 
information 
or 
identifiable 
biospecimens, 
the 
research could not 
practicably 
be 
carried out without 
using 
such 
information 
or 
biospecimens in an 
identifiable format 
 The implementation 
of the trial in the 
EHR will not use 
any 
identifiable 
information and all 
data extracted will 
have PHI removed. 
In order to remove 
the PHI an honest 
broker not on the 
study 
team 
will 
perform the data 
extract and remove 
the 
PHI. 
Patient 
identifiers will be 
hashed, 
dates 
truncated to the year 
and all dates turned 
into relative dates.  
 No identifiers on 
individual 
providers will be 
collected 
and 
there will be no 
analysis at the 
provider level  
 No 
PHI 
on 
historical subjects 
will be collected 
the 
waiver 
or 
alteration will not 
adversely affect the 
rights and welfare of 
the subjects; 
 The CDS prompts 
will 
only 
be 
recommending 
codified 
best 
practices.  
 The final decision 
for 
all 
treatment 
rests in the hands of 
the 
anesthesia 
provider 
 The 
CDS 
prompts 
are 
similar to those 
already in use and 
providers are free 
to ignore them if 
they wish 
 There will be no 
identifiers of the 
patients collected 
as part of the 
historical review 
whenever 
appropriate, 
the 
subjects or legally 
authorized 
representatives will 
be 
provided 
with 
additional pertinent 
information 
after 
participation 
 Given that the trial 
is only providing 
decision 
support 
recommendations 
to providers (which 
is a standard part of 
clinical care) there 
will be no direct 
notification 
to 
Study results will 
be published in 
peer 
reviewed 
literature 
and 
those 
publications will 
be made available 
to the providers 
 In 
line 
with 
standard practices 
for retrospective 
studies there will 
be no notification 
of patients after 
enrollment as this 
would 
require 
patient 

Protocol Template, Version 3.0 
10 
patients 
after 
participation 
identification and 
introduce 
additional risk.  
 
Randomization: Patients will be randomized within each hospital site at the level of the 
surgical case. To adjust for different POD risk amongst cases, we will use an 
established risk stratification model to classify each case into low, medium, and high 
baseline risk and randomize patients equally across the groups. 
 
5 
STUDY INTERVENTIONS  
5.1 Interventions, Administration, and Duration  
Intervention: The intervention is a CDS alert to promote 12 evidence based best practices 
in five domains: high-risk medication avoidance, glycemic control, hemodynamic 
management, normothermia, and anesthetic depth management, spanning the 
preoperative, intraoperative, and immediate postoperative periods. The CDS alerts will be 
designed to avoid disruptions in provider workflow. 
Interventions were selected from the American College of Surgeons, the AGS, and ASA 
list of best practices, if they were relevant to the perioperative period, suggested as a 
specific objective action, and are documented in the EHR during usual care. 
The intervention will be developed in line with the IMPACT Collaboratory steps for 
creating effective value propositions. Consistent with previous CDS deployments done 
by our team, the proposed interventions will be shared with departmental leadership for 
input and feedback on notification design and content. Specific feedback about frequency 
and thresholds of alerts will be solicited to gain buy-in and attempt to mitigate alert 
fatigue. Non-disruptive alerts that avoid hard stops will be used. Prior to intervention go 
live, education will be provided to the anesthesiologists via email communications as 
well as presentations at forums such as grand rounds, faculty meetings, residency 
meetings, etc. All interventions are performed by the anesthesiologist and measured in 
the EHR. 
5.2 Handling of Study Interventions  
The intervention will be developed in line with the IMPACT Collaboratory steps for 
creating effective value propositions. Consistent with previous CDS deployments done 
by our team, the proposed interventions will be shared with departmental leadership for 
input and feedback on notification design and content. Specific feedback about frequency 
and thresholds of alerts will be solicited to gain buy-in and attempt to mitigate alert 
fatigue. Non-disruptive alerts that avoid hard stops will be used. Prior to intervention go 
live, education will be provided to the anesthesiologists via email communications as 
well as presentations at forums such as grand rounds, faculty meetings, residency 
meetings, etc. All interventions are performed by the anesthesiologist and measured in 
the EHR. These anesthesiologists will not be part of the study team and will be 
performing the interventions as part of routine clinical care.  

Protocol Template, Version 3.0 
11 
 
5.3 Concomitant Interventions 
Any and all additional interventions to prevent POD or treat any other elements of 
anesthesia care will be allowed and at the complete discretion of the attending 
anesthesiologist. Further, in the event that the attending anesthesiologist feels that any of 
the interventions in the study protocol are unwarranted they retain sole discretion to 
ignore the CDS recommendations and prescribe care as they see fit.  
5.4 Adherence Assessment  
We will conduct an analysis to assess best practice adherence in the intervention vs 
control groups. For each of the 12 practices adherence will be measured as a binary 
variable; overall protocol adherence will be defined as the proportion of the 12 practices 
performed by the anesthesia team. The average adherence rate by study arm will be 
compared between the two study arms using the mixed model (or hierarchical generalized 
linear model for binary outcomes), treating hospital sites and providers as random effects. 
To examine learning effects in the control group and the time trend alone, we will create 
three segments one segment for the intervention group, one segment for the control 
group, and the third the historical control period. We will compare the three segments on 
the outcomes of interest using a segmented time series regression analysis. 
For the post-operative delirium screen (4AT) notifications we will examine postoperative 
length of stay as a time series before and after the intervention and for enrolled patients as 
opposed to patients who are unenrolled because they have had a prior surgical procedure 
during the study period.  
 
6 
STUDY PROCEDURES 
See next page 
 

Protocol Template, Version 3.0 
12 
6.1 
Schedule of Evaluations 
 
 
Assessment 
Day of Surgery 
Postoperative Days 
Informed Consent Form   
 
 
Demographics 
X 
 
Structured data evidence of CI 
X 
 
Medical History  
X 
 
Current Medications 
X 
 
Previous mini mental exam scores 
X 
 
NLP Algorithm on clinical notes 
X 
 
Performance of the 12 best practice interventions 
X 
 
Vital Signs 
X 
 
Inclusion/Exclusion Criteria  
X 
 
Enrollment/Randomization 
X 
 
Postoperative 4AT Scores 
 
X 
Adverse Events  
 
X 

Protocol Template, Version 3.0 
13 
6.2 Description of Evaluations  
6.2.1 Screening Evaluation 
A waiver of informed consent is being requested for study patients, providers, and the 
historical controls. Patients scheduled for surgery will be automatically identified in 
the EHR as having CI based on one of  
- 
An existing natural language processing algorithm (NLP) currently under peer 
review 
- 
Structured data from the EHR (i.e. mini-mental score<25, prescription of 
Donepezil, etc. 
All providers who care for patients enrolled will be automatically enrolled. Prior to 
study go-live providers will be educated as to the protocol and informed that any CDS 
prompts can be followed or ignored at their discretion. No data on provider level CDS 
adherence will be collected by the study team. 
6.2.2 Enrollment, Baseline, and/or Randomization 
Enrollment 
Patients will be automatically enrolled based upon the presence of the above criteria 
in the EMR. 
Randomization 
Patients will automatically be randomized prior to surgery based on their underlying 
POD risk and hospital by an algorithm in the EHR. The randomization status of each 
patient will then be stored in the EHR in a way that can be extracted on the back end 
but is invisible to the provider.  
6.2.3 Follow-up Visits 
Patients will have routine postoperative care with the addition of at least one 4AT 
assessment by a medical provider on the care team. If post-operative delirium is 
present in the screen, providers will be encouraged to re-screen the following day, 
but only one assessment is needed for the present study’s primary end point.  
7 
SAFETY ASSESSMENTS  
Data on postoperative outcomes will be automatically extracted from the EHR.  
7.1 Specification of Safety Parameters 
All interventions are consistent with perioperative best practice and introduce no 
additional risk to the patients. Thus, no additional testing is necessary.  
7.2 Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters 
As per above, the study interventions introduce no additional risk to the patient and thus 
additional monitoring is not needed.  
7.3 Adverse Events and Serious Adverse Events  
There are no adverse events (AEs) or serious adverse events (SAEs) directly associated 
with the study protocol. Any and all AEs and SAEs (hypo/hyperglycemia, acute kidney 

Protocol Template, Version 3.0 
14 
injury, respiratory dysfunction, and death) are consistent with the risk of undergoing 
surgery and are not related to the study protocol.  
 
7.3.1 Reporting Procedures 
Severity of Event 
There are no adverse events that will be attributable to the study intervention of decision 
support prompts. All of the AEs and SAEs can be automatically identified from the EHR 
feed and a list of relevant patients generated.  
Relationship To Study Intervention 
As noted above, there are no AEs or SAEs expected to result from study intervention 
 
7.3.2 Follow-up for Adverse Events 
N/A 
7.4 Safety Monitoring 
PI will present data safety monitoring reports to the DSMB as follows: 
1. Prior to start of participant enrollment and data collection, submit DSMP for approval 
(1/31/22) 
2. Six months after participant enrollment initiation 
3. At end of study 
4. Or at the discretion of the DSMB/NIA 
Content of DSM report: (in addition to org., title, PI, IRB study #)  
1. Study status (Participants screened, enrolled, completed, discontinued, risk vs. 
benefit, site performance 
2. Safety information (AEs, SAEs, actions taken or planned to address issues) 
3. Study quality (data quality, timeliness of data collection, fidelity of intervention based 
on measures of completeness of intervention components) 
8 
INTERVENTION DISCONTINUATION  
N/A 
9 
STATISTICAL CONSIDERATIONS  
9.1 General Design Issues  
 
Primary hypothesis: CDS alerts that provide anesthesiologists caring for patients with CI 
real-time prompts regarding best practices, will increase adherence to these practices and 
reduce POD. 
 
Analysis: We will evaluate the difference in POD incidence, using the 4 A’s Test (4AT) 
in patients where providers received the CDS prompts vs controls. 

Protocol Template, Version 3.0 
15 
 
Secondary hypothesis: The CDS prompts will improve the individual practice adherence 
and the influence of each practice adherence on POD prevention will be heterogeneous 
within and between study arms and groups. 
 
Secondary analysis: We will perform a secondary analysis of the data to evaluate a) The 
ability of CDS prompts to increase best practice adherence, b) The impact of individual 
and overall practice adherence on POD prevention, and c) The efficacy of this tool in 
various sociodemographic subgroups to ensure that it is equitable. 
 
Analysis: We will evaluate practice adherence within each study arm and identify 
individual practices that have significant improvement resulting from the implementation 
of the CDS prompt. We will also study the influence of each practice on POD prevention 
and test whether individual and overall practice effects on reducing POD 
differ between the study arms. Lastly, we will examine pathway adherence and 
effectiveness to reduce POD in key subgroups of gender, social determinants of health 
(SDOH), and race/ethnicity. 
 
9.2 Sample Size and Randomization 
9.2.1 Treatment Assignment Procedures 
Treatment Assignment: In the perioperative period an anesthesia attending supervises 
multiple junior providers in pairings that change daily. Thus, randomization at the 
level of the provider is impossible and patients will be randomized at the level of the 
surgical procedure. In order to avoid issues with statistical analysis, only the first 
surgical procedure for each patient will be included in the study. 
For each procedure, where the patient has been identified as having CI, the 
nomogram will be used to estimate the patient’s risk of POD and assign the patient to 
one of three groups: high risk (>330 points, 30% risk of POD), medium risk (285-330 
points, 15-30% risk of POD), and low risk (<284 points, <15% risk of POD). Patients 
will be stratified based on their risk grouping and the hospital. For each stratum, 
patients will be alternately assigned to the intervention or control groups (i.e., 1st 
intervention, 2nd control, etc.). The control group will receive usual care (no prompts) 
and the intervention group will receive an alert to the anesthesiologist that the patient 
is at increased risk for POD with CDS prompts for clinical best practices. 
 
Patients will be automatically randomized by the EHR and the status of the 
randomization will only be visible in the back-end database. Providers caring for 
patients in the control group will not know the patient is included in the study. Given 
the short time frame of the CDS prompts and overall provider discretion there will be 
no need to unmask any participants.  
 

Protocol Template, Version 3.0 
16 
Primary Endpoint Power: Power was calculated for the primary endpoint. Assuming 
100,000 surgical procedures, with 40% over age 65 and 30% of those having CI, we 
will have roughly 12,000 patients to enroll - 5000 subjects in each study arm, Poisson 
regression achieves 80% power at a 0.05 significance level to detect a response rate 
ratio of at least 0.942 between the intervention group and the control group, assuming 
that the POD incident rate for the control is 0.30 and the mean hospital length of stay 
is 3 days. The detectable rate becomes 0.930 if we adjusted for an over-dispersion 
parameter of 1.5 (PASS 12. NCSS). 
 
Secondary Outcomes Power: 
a) Analysis of provider adherence: The table below shows the minimal detectable 
difference in the improvement of adherence due to use of CDS prompts assuming 
adherence for each individual item without intervention is no lower than 50%. 
The calculation was performed using the 2-sample z-test, using a conservative 
significance level (0.004) to account for multiple comparisons and the targeted 
power is at the 90% level. 
 
 
 
 
b) Effect of individual and overall practice adherence on POD: The AUC of the 
ROC curve for the base model that considers factors in nomogram is 0.8 (the 
null). The proposed study is powered at 90% to detect an increase of 0.016 when 
the study arm is added to the base model, using a two-sided z-test at a significance 
level of 0.05 (PASS 12. NCSS). 
c) Analysis of Health Equity: It is currently unknown the if there are differences in 
POD rates or best practice adherence for each of these groups, thus impossible to 
accurately calculate power. For each of these analyses post-hoc power will be 
calculated and reported in the results. There are 12 practice items, we will use 
0.004 alpha level for statistical significance in all individual practice adherence 
analyses. The usual 0.05 alpha level will be applied for all other analyses 
including the overall protocol adherence analysis. All tests will be 2-sided 
 
9.3 Interim analyses and Stopping Rules 
No interim analysis is planned.  

Protocol Template, Version 3.0 
17 
9.4 Outcomes  
9.4.1 Primary outcome 
Primary Outcome: The Primary outcome will be postoperative delirium as measured 
by the 4 A’s Test (4AT). The 4AT is a commonly used tool that identifies the 
presence/absence of POD. It consists of 4 sections; alertness, AMT4: Abbreviated 
Mental Test -4, attention, and acute change of fluctuating course. The 4AT is scored 
from 0-12, a score of 4 or more suggests delirium. We will estimate the incidence rate 
per person-day during postoperative days 0-7 for each treatment group. Patients free 
of POD within 7 days will be censored on day 7. Otherwise, follow-up time will be 
defined as time to POD or at the time of discharge, we will assume that patient do not 
have POD after discharge. 
9.4.2 Secondary outcomes   
The ability of CDS prompts to increase best practice adherence will be measured by 
documentation of the best practices in the EHR as part of the usual patient care. For 
example, the measure “Maintain Glucose <200” will be measured by documentation of 
a glucose value >200 in the EHR.  
 
The impact of individual and overall practice adherence on POD prevention will be 
measured using the combination of the POD measurements and practice adherence 
measurement.  
 
The efficacy of the tool in various sociodemographic subgroups will be measured using 
the socioeconomic and demographic information of the patient in the EHR. 
 
The impact of delirium screening prompts on the medical providers will be measured by 
a time series analysis looking at all enrolled patients and patients that would have been 
enrolled in the one year prior to go-live.  
9.5 Data Analyses 
Primary Endpoint: We will perform marginal structural model for Poisson regression 
using the generalized estimating equation method. Marginal structural models estimate 
parameters through the use of inverse-probability weighting and allow for adjustment of 
time-dependent confounders (e.g., daily medical treatment) and informative drop outs, 
while accounting for clustering effects within time series data and providers. 
Secondary Endpoints: 
a) Protocol adherence: For each of the 12 practices adherence will be measured as a 
binary variable; overall protocol adherence will be defined as the proportion of the 12 
practices performed by the anesthesia team. The average adherence rate by study arm 
will be compared between the two study arms using the mixed model (or hierarchical 
generalized linear model for binary outcomes), treating hospital sites and providers as 
random effects. To examine learning effects in the control group and the time trend 
alone, we will create three segments one segment for the intervention group, one 
segment for the control group, and the third the historical control period. We will 

Protocol Template, Version 3.0 
18 
compare the three segments on the outcomes of interest using a segmented time series 
regression analysis. The daily average of the protocol adherence rate will be 
calculated, excluding days with fewer than 10 cases. Daily averages will also be 
calculated for patient and case-level covariates (e.g., hospital site, and factors 
considered in the nomogram). We will assess the degree of autocorrelation in the data 
(using the Durbin-Watson statistics) with an auto-regressive linear model of lag 2 
using the Yule-Walker method, fitting a separate line for each of the four periods, 
with each period having its own slope (indicating the trend over time for that period) 
and intercept. To test for trends in adherence over time, we will test if the slopes for 
all workflow segments are equal to zero; if rejected, we will test whether the slope for 
the intervention group differs from that of the control group and if the slope fo the 
control group differs from the slope from a year before. 
b)  Effect of Individual and overall practice adherence on POD: We will examine the 
effect of each of the 12 best practices on POD using a hierarchical generalized linear 
mixed model for Poisson regression with log link for POD outcome that examines the 
effect of practice adherence (one practice item at a time and the overall protocol 
adherence), and its interaction with the study arm. The models will adjust for factors 
in the nomogram and treat hospital sites and providers as random effects. To 
understand if individual practice adherence and/or overall protocol adherence will 
improve the predictability of POD, we will perform a series of nested analyses that 
contrast the areas under the ROC curve between the base model that includes all 
factors in the nomogram and the testing model. For examples, the testing models 
could be 1) base model + study arm, and then 2) base model + study arm overall 
adherence rate. 
c) Analysis of health equity on CDS efficacy in preventing POD: We will conduct a 
series of analyses to examine if the efficacy of the CDS pathway and/or the individual 
and overall practice adherence are homogeneous across different patient subgroups, 
such as biological sex, social determinants of health (SDOH), and race. We will 
report the POD incidence rates and the practice adherence rates across the subgroups 
and check if randomization was achieved within each of the subgroups. For each 
demographic factor, we will repeat the SA1 and SA2 analyses, and add an interaction 
term between the factor of interest and the intervention group (SA1) or practice 
adherence (SA2) to test if the CDS pathway efficacy or practice adherence varies 
depending on the demographic factor, adjusting for covariates that show significant 
disparities among subgroups. 
d) Analysis of delirium screening prompt on medical provider post-operative care 
practices. At present not all patients with CI are screened for POD by their providers. 
Part of our intervention will be ensuring that all these patients are screened for POD 
by a medical provider on their care team. Because the patients are not currently being 
screened for POD we cannot do a direct time series analysis looking at POD rates as 
there is no pre-intervention control group. Thus we will do a segmented multivariate 
time series analysis looking at other clinical outcomes, in particular length of stay 
(LOS), which is associated with POD.  

Protocol Template, Version 3.0 
19 
10 
DATA COLLECTION AND QUALITY ASSURANCE 
10.1 Automated Collection of Data from the EHR 
All data for the study will be automatically extracted from the EHR database. The EPIC 
Clarity Database is a SQL relational database that is refreshed daily. Our group has 
already implemented a peer-reviewed perioperative data warehouse that extracts the 
necessary data from the Clarity database on a regular basis and cleans and organizes it to 
facilitate use.  
 
All data necessary for the study, including presence of CI, underlying risk factors for 
POD from the nomogram, practice adherence by the anesthesiologist, and POD are 
readily contained and extracted from the data warehouse.  
 
Any data extracts delivered to the study team for analysis will be deidentified and have 
dates binned at the level of the month to ensure patient anonymity. Data extracts required 
for reporting of AE/SAEs will have only the minimum necessary identifying information.  
11 
PARTICIPANT RIGHTS AND CONFIDENTIALITY  
 
Participant confidentiality and privacy is strictly held in trust by the participating 
investigators, their staff, the safety and oversight monitor(s), and the sponsor(s) and 
funding agency. This confidentiality is extended to the data being collected as part of this 
study. Data that could be used to identify a specific study participant will be held in strict 
confidence within the research team. No personally-identifiable information from the 
study will be released to any third party except as needed for reporting of AE/SAEs.  
 
All research activities will be conducted in as private a setting as possible.  
The study monitor, other authorized representatives of the sponsor or funding agency, 
representatives of the Institutional Review Board (IRB), regulatory agencies or 
representatives from companies or organizations supplying the product, may inspect all 
documents and records required to be maintained by the investigator, including but not 
limited to, medical records (office, clinic, or hospital) and pharmacy records for the 
participants in this study. The clinical study site will permit access to such records.  
The study participant’s contact information will be securely stored at each clinical site for 
internal use during the study but will not be available to the study team. At the end of the 
study, all records will continue to be kept in a secure location for as long a period as 
dictated by the reviewing IRB, Institutional policies, or sponsor/funding agency 
requirements.  
 
Study participant research data, which for purposes of statistical analysis and scientific 
reporting, will be transmitted to Mount Sinai for linkage, where data are analyzed and 
stored on a HIPAA compliant environment. This will not include the participant’s contact 
or identifying information. Rather, individual participants and their research data will be 
identified by a unique study identification number. The study data entry and study 
management systems will be secured and password protected. 

Protocol Template, Version 3.0 
20 
11.1 Institutional Review Board (IRB) Review 
This protocol and the informed consent document and any subsequent modifications will 
be reviewed and approved by the IRB or ethics committee responsible for oversight of 
the study.   
11.2 Informed Consent Forms 
N/A - we request a waiver of informed consent for all study participants.  See section 4.3 
for waiver justifications. 
We also request a HIPAA waiver of authorization for the use of HIPAA-protected EHR 
data for this study.    
11.3 Participant Confidentiality  
Any data, specimens, forms, reports, video recordings, and other records that leave the 
site will be identified only by a participant identification number (Participant ID, PID) to 
maintain confidentiality.  All records will be kept digitally on encrypted drives.  All 
computer entry and networking programs will be done using PIDs only. Information will 
not be released without written permission of the participant, except as necessary for 
monitoring by IRB, the sponsor or persons working on behalf of the sponsor (i.e. 
IMPACT research study staff, the DSMB and/or Safety Officer), the FDA, the NIA, and 
the OHRP. 
11.4 Study Discontinuation  
The study may be discontinued at any time by the IRB or DSMB as part of their duties to 
ensure that research participants are protected.  
12 
ETHICAL CONSIDERATIONS 
The protocol of the study consists only of decision support prompts to the provider. The 
anesthesiologist has complete discretion to follow or ignore any or all recommendations 
based on their clinical judgement. Thus, failure to follow a recommendation will not be 
considered a protocol deviation.  
13 
COMMITTEES 
The primary committee will consist of the PIs and Co-Is for this project (listed above) 
who will meet monthly to review study progress and address any issues. 
 
In addition, an implementation committee will be formed consisting of clinical 
anesthesiologists (who will ultimately be subjected to the alerts) from the participating 
hospitals who will provide feedback on any decision support alerts and address any 
concerns as they relate to provider workflow.  
14 
PUBLICATION OF RESEARCH FINDINGS 
Publication of the results of this trial will be governed by the policies and procedures 

Protocol Template, Version 3.0 
21 
developed by the Steering Committee as well as by the IMPACT Collaboratory.  Any 
presentation, abstract, or manuscript will be made available for review by the IMPACT 
Collaboratory prior to submission. 
 
 

Protocol Template, Version 3.0 
22 
15 
REFERENCES 
1. 
Culley, D.J., et al. Poor Performance on a Preoperative Cognitive Screening Test Predicts 
Postoperative Complications in Older Orthopedic Surgical Patients. Anesthesiology 127, 
765-774 (2017). 
2. 
Culley, D.J., et al. Preoperative Cognitive Stratification of Older Elective Surgical 
Patients: A Cross-Sectional Study. Anesth Analg 123, 186-192 (2016). 
3. 
Dyer, C.B., Ashton, C.M. & Teasdale, T.A. Postoperative delirium. A review of 80 
primary data-collection studies. Arch Intern Med 155, 461-465 (1995). 
4. 
Ansaloni, L., et al. Risk factors and incidence of postoperative delirium in elderly 
patients after elective and emergency surgery. Br J Surg 97, 273-280 (2010). 
5. 
Association, A.P. American Psychiatric Association: Diagnostic and Statistical Manual 
of Mental Disorders, Fifth Edition: DSM-5, (American Psychiatric Association, 
Arlington, VA, United States of America, 2013). 
6. 
Evered, L., et al. Recommendations for the nomenclature of cognitive change associated 
with anaesthesia and surgery-2018. British journal of anaesthesia 121, 1005-1012 
(2018). 
7. 
Vacas, S., Cole, D.J. & Cannesson, M. Cognitive Decline Associated With Anesthesia 
and Surgery in Older Patients. Jama (2021). 
8. 
Monk, T.G., Saini, V., Weldon, B.C. & Sigl, J.C. Anesthetic management and one-year 
mortality after noncardiac surgery. Anesthesia and analgesia 100, 4-10 (2005). 
9. 
Rudolph, J.L., et al. Delirium is associated with early postoperative cognitive 
dysfunction. Anaesthesia 63, 941-947 (2008). 
10. 
Silbert, B., et al. Preexisting cognitive impairment is associated with postoperative 
cognitive dysfunction after hip joint replacement surgery. Anesthesiology 122, 1224-1234 
(2015). 
11. 
Davis, D.H., et al. Delirium is a strong risk factor for dementia in the oldest-old: a 
population-based cohort study. Brain : a journal of neurology 135, 2809-2816 (2012). 
12. 
Brown, C.H.t., et al. Cognitive Decline after Delirium in Patients Undergoing Cardiac 
Surgery. Anesthesiology 129, 406-416 (2018). 
13. 
Fong, T.G., Tulebaev, S.R. & Inouye, S.K. Delirium in elderly adults: diagnosis, 
prevention and treatment. Nat Rev Neurol 5, 210-220 (2009). 
14. 
Moller, J.T., et al. Long-term postoperative cognitive dysfunction in the elderly 
ISPOCD1 study. ISPOCD investigators. International Study of Post-Operative Cognitive 
Dysfunction. Lancet 351, 857-861 (1998). 
15. 
Clegg, A. & Young, J.B. Which medications to avoid in people at risk of delirium: a 
systematic review. Age Ageing 40, 23-29 (2011). 
16. 
Wang, N.Y., Hirao, A. & Sieber, F. Association between intraoperative blood pressure 
and postoperative delirium in elderly hip fracture patients. PloS one 10, e0123892 (2015). 
17. 
Siepe, M., et al. Increased systemic perfusion pressure during cardiopulmonary bypass is 
associated with less early postoperative cognitive dysfunction and delirium. European 
journal of cardio-thoracic surgery : official journal of the European Association for 
Cardio-thoracic Surgery 40, 200-207 (2011). 
18. 
Sessler, D.I., et al. Hospital stay and mortality are increased in patients having a `Triple 
Low' of low blood pressure, low bispectral index, and low minimum alveolar 
concentration of volatile anesthesia. Anesthesiology 116(2012). 

Protocol Template, Version 3.0 
23 
19. 
MacKenzie, K.K., Britt-Spells, A.M., Sands, L.P. & Leung, J.M. Processed 
Electroencephalogram Monitoring and Postoperative Delirium: A Systematic Review and 
Meta-analysis. Anesthesiology 129, 417-427 (2018). 
20. 
Saager, L., et al. Intraoperative tight glucose control using hyperinsulinemic 
normoglycemia increases delirium after cardiac surgery. Anesthesiology 122, 1214-1223 
(2015). 
21. 
Franco, K., Litaker, D., Locala, J. & Bronson, D. The cost of delirium in the surgical 
patient. Psychosomatics 42, 68-73 (2001). 
22. 
Hirsch, J., DePalma, G., Tsai, T.T., Sands, L.P. & Leung, J.M. Impact of intraoperative 
hypotension and blood pressure fluctuations on early postoperative delirium after non-
cardiac surgery. Br J Anaesth 115, 418-426 (2015). 
23. 
Evered, L.A., et al. Anaesthetic depth and delirium after major surgery: a randomised 
clinical trial. British journal of anaesthesia 127, 704-712 (2021). 
24. 
Ehrenfeld, J.M., Wanderer, J.P., Terekhov, M., Rothman, B.S. & Sandberg, W.S. A 
Perioperative Systems Design to Improve Intraoperative Glucose Monitoring Is 
Associated with a Reduction in Surgical Site Infections in a Diabetic Patient Population. 
Anesthesiology 126, 431-440 (2017). 
25. 
Nair, B.G., et al. Intraoperative blood glucose management: impact of a real-time 
decision support system on adherence to institutional protocol. Journal of clinical 
monitoring and computing 30, 301-312 (2016). 
26. 
Nair, B.G., Horibe, M., Neradilek, M.B., Newman, S.F. & Peterson, G.N. The Effect of 
Intraoperative Blood Glucose Management on Postoperative Blood Glucose Levels in 
Noncardiac Surgery Patients. Anesth Analg 122, 893-902 (2016). 
27. 
Sathishkumar, S., et al. Behavioral Modification of Intraoperative Hyperglycemia 
Management with a Novel Real-time Audiovisual Monitor. Anesthesiology 123, 29-37 
(2015). 
28. 
McCormick, P.J., Levin, M.A., Lin, H.M., Sessler, D.I. & Reich, D.L. Effectiveness of an 
Electronic Alert for Hypotension and Low Bispectral Index on 90-day Postoperative 
Mortality: A Prospective, Randomized Trial. Anesthesiology 125, 1113-1120 (2016). 
29. 
Nair, B.G., et al. Anesthesia information management system-based near real-time 
decision support to manage intraoperative hypotension and hypertension. Anesth Analg 
118, 206-214 (2014). 
30. 
Nair, B.G., Horibe, M., Newman, S.F., Wu, W.Y. & Schwid, H.A. Near real-time 
notification of gaps in cuff blood pressure recordings for improved patient monitoring. J 
Clin Monit Comput 27, 265-271 (2013). 
31. 
Nair, B.G., et al. Reducing Wastage of Inhalation Anesthetics Using Real-time Decision 
Support to Notify of Excessive Fresh Gas Flow. Anesthesiology 118, 874-884 (2013). 
32. 
Josephs, S.A., Lemmink, G.A., Strong, J.A., Barry, C.L. & Hurford, W.E. Improving 
Adherence to Intraoperative Lung-Protective Ventilation Strategies at a University 
Medical Center. Anesth Analg 126, 150-160 (2018).