Determinants of Implementation Success Coordinating Ventilator, Early Ambulation and Rehabilitation Efforts in the Intensive Care Unit
Conditions: Implementation Science, Delirium, Intensive Care Unit Acquired Weakness, Pain, Sedation Complication, Mechanical Ventilation Complication
Sponsor: Ohio State University

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
 
Research Protocol: 
Determinants of Implementation Success Coordinating Ventilator, Early Ambulation 
and Rehabilitation Efforts in the ICU  
(DISCOVER-ICU) 
 
 
SPECIFIC AIMS 
Patients with heart, lung, and blood disorders severe enough to require an intensive care unit (ICU) stay 
frequently experience profound physical, mental, and cognitive health impairments that may persist for months 
to years after hospital discharge.1,2 These long-term impairments are commonly acquired in the ICU and are 
often initiated or exacerbated by sedation, mechanical ventilation, and symptom management decisions.3-6 
Indeed, ICU-acquired pain, anxiety, delirium, and weakness are associated with numerous adverse health 
outcomes including prolonged mechanical ventilation, post-traumatic stress disorder, depression, functional 
decline, new institutionalization, and severe neurocognitive impairment.7-20 Considering millions of adults face 
the challenges of ICU survivorship annually,21-25 it is essential findings from high-quality research are reliably 
adopted and sustained in everyday clinical practice.  
 
As outlined in “Clinical Practice Guidelines for the Management of Pain, Agitation, and Delirium (PAD) in Adult 
Patients in the ICU,”7 a robust and growing body of evidence demonstrates clinical outcomes are improved when 
integrated, interprofessional approaches to mechanical ventilation and symptom management are applied during 
the course of critical illness. One such approach is the ABCDEF bundle, an evidence-based, multicomponent 
strategy that incorporates many PAD guideline recommendations.6,26,27 While previously shown safe and 
effective when applied in routine ICU practice,28-30 our preliminary data demonstrate that this bundle has yet to 
be widely adopted. Pain and delirium remain underdiagnosed,31,32 and ICU clinicians often fail to maintain 
patients at a light level of sedation, use a protocolized mechanical ventilation discontinuation approach, get 
patients out of bed, or accept flexible bedside family presence.33-43 There is, therefore, a clear need for studies 
that use validated implementation frameworks and outcomes to assess strategies for improving ABCDEF bundle 
adoption and the removal of long-held and potentially harmful ICU practices. This need is most critical as our 
nation, and the world, face an exponential increase in the use of mechanical ventilation and ICU services to treat 
victims of the COVID-19 pandemic. 
 
In the proposed study, we will continue our three-year partnership with the Society of Critical Care Medicine’s 
ICU Liberation Collaborative, a national quality improvement (QI) network comprised of 68 ICUs. Guided by the 
Consolidated Framework for Implementation Research,44 the overall objective of this T4 research is to develop 
multilevel implementation strategies to enhance sustainable adoption of the ABCDEF bundle in routine ICU 
practice. We are particularly interested in discovering how various patient-, provider-, and organization-level 
characteristics and implementation strategies effect ABCDEF bundle adoption. Using a multiphase, sequential, 
mixed-methods design our “Determinants of Implementation Success Coordinating Ventilator, Early Ambulation 
and Rehabilitation Efforts in the ICU (DISCOVER-ICU)” study has four specific aims (1-4).  
 
Aim 1: Estimate the effects of patient-level characteristics on ABCDEF bundle adoption.  
 
Aim 2: Examine unit-level variation in ABCDEF bundle adoption and associated provider- and organization-level 
characteristics.  
 
Aim 3: Determine which implementation strategies result in the greatest adoption of the ABCDEF bundle.  
 
Aim 4: Identify and describe the micro-decisions (e.g., resource allocation, patient participation, preferences, 
and agreement) involved in implementing spontaneous awakening trials (SATs), spontaneous breathing trials 
(SBTs), and early mobility/exercise in everyday care. The following research questions will be addressed in this 
aim: 
1. What are the micro-decisions that clinicians make when implementing SATs, SBTs, and early mobility with 
ICU patients receiving mechanical ventilation? 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
2. How and to what extent do clinicians inform mechanically ventilated patients and consider patient 
preference about the procedures involved with SATs, SBTs, and exercise/early mobility?  
3. What are the primary conflicts and tradeoffs in decisions about implementing SATs, SBTs, and early 
mobility/exercise with MV patients?  
 
The results of this study will directly lead to the development of implementation strategies that are adaptable, 
responsive to community needs, and account for the cultural and organizational factors necessary to increase 
ABCDEF bundle adoption. The proposed work will also provide important information previously missing from 
the literature regarding micro-ethical decision making in the ICU and may illuminate previously unidentified 
ethical dilemmas clinicians face during a national health emergency. 
 
RESEARCH STRATEGY 
 
Scientific Premise 
Survivorship is an emerging challenge for critical care medicine. Over five million Americans are admitted 
to intensive care units (ICUs) annually, primarily for heart, lung, and blood disorders.21-23,46 This number will rise 
as the number of adults with complex comorbidities grows, our population ages, and the incidence of acute 
respiratory and heart failure increases.47,48 Advances in critical care medicine have improved overall survival 
rates,49 but, for some, survival comes with heavy personal and financial costs. Many ICU survivors experience 
profound physical, cognitive, and/or mental health impairments that often persist for months or years after 
hospitalization.1,2 The constellation of these impairments is now referred to as post-intensive care syndrome 
(PICS).1,2 While the exact etiology of PICS remains unclear, its incidence and severity is likely related to both 
pre-existing risk factors and commonly acquired ICU symptoms initiated or exacerbated by antiquated sedation, 
mechanical ventilation (MV), and mobility practices.3-6  
 
ICU-acquired conditions significantly impact the quantity and quality of life (QOL) after critical illness. In 
2013, the Society of Critical Care Medicine (SCCM) updated their “Clinical Practice Guidelines for the 
Management of Pain, Agitation, and Delirium in Adult Patients in the ICU” (PAD Guidelines).7 Version 3 of the 
PAD guidelines is due to be released later this year and to include the most recent evidence related to ICU sleep 
and immobility. Both versions highlight the importance, high incidence, and substantial physical, psychological, 
and cognitive toll various symptoms play during and after critical illness. Pain, occurring in  
> 70% of ICU patients,50-52 is one of the most prevalent, distressing, and under-treated physical symptoms 
experienced by the critically ill. Pain occurs both at rest and with routine clinical activities, such as wound care, 
tracheal suctioning, and turning.53 Air hunger, being dependent on health professionals, and not being able to 
effectively communicate needs cause many ICU patients to feel anxiety, fear, and loneliness. These feelings 
seem to depend on the ability of ICU providers and family members to be physically “present” with the patient.54 
Delirium, occurring in up to 80% of patients requiring MV,55,56 is recognized as a significant prognostic indicator. 
A recent meta-analysis found delirious patients were 6 times more likely to experience nosocomial complications, 
2.5 times more likely to be discharged to skilled placement, had longer ICU and hospital length of stay (LOS), 
and spent an average of 7 days longer on MV.15 Lastly, the ICU-acquired muscle weakness often caused by ICU 
medications and prolonged bedrest can be present in 25-50% of critically ill patients.57-59 The impact of ICU-
acquired pain, anxiety, delirium, and weakness extends well beyond hospitalization. ICU pain and anxiety are 
associated with insufficient sleep,60 post-traumatic stress disorder,16,17 chronic pain,7 depression,18 and lower 
health-related QOL17 post hospitalization. Patients with ICU delirium and weakness experience substantial 
functional decline,14,61-63 higher risk of re-hospitalizations,64 elevated mortality risk,10,11 and severe long-term 
neurocognitive impairment.13,19,65,66 Collectively, these findings suggest ICU-acquired pain, anxiety, delirium, and 
weakness are precursors to poor short- and long-term patient outcomes. 
 
A number of proven-effective ICU sedation, mechanical ventilation, and mobility interventions exist. The 
PAD Guidelines recommend a number of strategies to improve the care and outcomes of critically ill adults: (1) 
the routine monitoring of pain, agitation/sedation, and delirium using valid and reliable tools; (2) maintaining 
critically ill patients at a “light” levels of sedation; (3) using daily spontaneous awakening trials (SATs) or a light 
“target level” of sedation; and (4) performing early mobilization when feasible. Sedation strategies recommend 
nonbenzodiazepine sedatives (i.e., propofol or dexmedetomidine) versus benzodiazepines (i.e., midazolam or 
lorazepam). Additional patient benefit is accrued when SATs are coordinated with daily MV discontinuation 
protocols that include spontaneous breathing trials (SBTs).67 In well-designed studies, these symptom-focused 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
interventions were associated with reduced ICU and hospital LOS, time spent on MV, tracheostomy placement 
rates, delirium and coma, functional decline, and risk of death 67-78  
 
Widespread adoption of the ICU PAD Guidelines recommendations remains poor. Routine monitoring of 
pain, delirium, and sedation levels during an ICU stay remains inconsistent.31,32,79-82 Benzodiazepines and other 
potentially harmful sedative medications continue to be routinely administered,81 elevating patients’ risk for 
delirium and complicating ICU providers ability to mobilize patients.72,83 Patients remain on sedation and MV 
longer than needed, and resistance remains high on the part of clinicians to maintain patients at a light level of 
sedation, use a protocolized MV discontinuation approach, and get patients out of bed during their ICU stay.33-
38,40,41,81,84-88 Despite the broad consensus that liberalization of visiting hours in the ICU improves the care and 
experience of patients and families, 42 few critical care units are open for family visits 24 hours per day.89-91 
Clearly, there is an important and significant gap between the discovery of proven-effective ICU interventions 
and approaches that can equip ICU providers with the knowledge, skills, and tools necessary to adopt and 
sustain these interventions in everyday practice.  
 
The ABCDEF bundle facilitates the organizational and cultural changes needed to implement the PAD 
guidelines. To facilitate adoption of PAD guidelines into everyday ICU care, our team members helped develop 
and first-test an evidence-based, multicomponent, and interprofessional team-management framework, the 
ABCDEF bundle.3-6,26,27,35,92,93 Based on dozens of research studies published in high-impact journals,7 the 
ABCDEF bundle elements are intentionally interdependent and specifically designed to improve collaboration 
among ICU team members, standardize care processes, and break the cycle of over-sedation, prolonged MV, 
and immobility.3 These changes in care come about through daily use of: (1) the assessment tools recommended 
in the PAD guidelines, (2) both SATs and SBTs, (3) select sedative medications, (4) standardized 
exercise/mobility activities, and (5) family members as active participants in ICU care.  
 
Earlier versions of the ABCDEF bundle were associated with improvements in clinical outcomes. A 
number of studies and quality improvement (QI) projects demonstrated the benefits of incorporating earlier 
versions of the ABCDEF bundle into everyday clinical care. In 2014, Balas (PI) led a single-center, before/after 
study28 that demonstrated bundle implementation was associated with a 3-day improvement in ventilator-free 
days (p = 0.04). After adjusting for age, sex, severity of illness, comorbidity, and MV status, patients managed 
with the bundle experienced a near halving of the odds of delirium (odds ratio [OR], 0.55; 95% confidence interval 
[CI], 0.33-0.93; p = 0.03) and increased odds of mobilizing out of bed at least once during an ICU stay (OR, 2.11; 
95% CI, 1.29- 3.45; p = 0.003).94-98 Both ICU (pre 16.4% vs. post 9.3%, p = 0.07) and hospital (pre 19.9% vs. 
post 11.3%, p = 0.04) mortality rates were lower in the group treated with the bundle. Another single-system, 
multi-site QI project conducted in 2017 by Barnes-Daly29 studied over 6,000 patient days and showed a “dose-
response” relationship to bundle implementation. After adjusting for age, APACHE II, and MV status, 
multivariable analysis showed that, for every 10% increase in bundle compliance, patients had 15% higher odds 
of hospital survival (OR 1.15; 95% CI, 1.09-1.22; p < 0.001). Patients also experienced more days alive and free 
of delirium and coma with increased bundle compliance (incident rate ratio, 1.15; 95% CI, 1.09-1.22; p < 0.001).  
 
There is a need for research exploring the complex bioethical issues involved in delivering evidence-
based ICU interventions. Critical care clinicians and implementation science researchers face unique ethical 
challenges when translating scientific and technologic advances into everyday ICU practice. For example, while 
ethical justification for traditional clinical trials relies heavily on individual consent, implementation research and 
ICU practice often involve aspects of distributive justice, economics, and balancing beneficence/non-maleficence 
principles.99 In addition to these “macro” ethical concepts, critical care delivery often involves a series of “micro” 
decisions that are not always made on best evidence but rather occur in a continuous flux of relationships and 
dialogues.100 Data are needed on the complex decision-making processes and/or ethical dilemmas 
interprofessional teams encounter when delivering effective but time and resource-intensive clinical 
interventions. These ethical considerations are intrinsic to both the responsible conduct of biomedical research 
and the translation of scientific and technologic advances into ICU practice. Importantly, this knowledge will also 
help inform future NIH and other federal policy decisions. 
 
Micro-decisions and micro-ethics play an important, yet often unrecognized, role in everyday ICU 
practice. Micro-decisions are decisions made at the individual patient level aimed at capturing patient 
participation and preferences in care.100 A new observational study conducted in long-term acute care hospitals 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
shows that well-functioning interdisciplinary teams balance tradeoffs between patient preference and clinical 
practice guidelines in liberating patients from prolonged MV.101 That study, however, was conducted with patients 
with a lower severity of illness who were in the chronic (vs. acute) phase of MV liberation. Importantly, this work 
did not describe how, or how often, patient preferences were ascertained when implementing evidence-based 
interventions. How interprofessional teams view and approach micro-decisions involving patient participation 
and patient preference during ABCDEF bundle implementation may help to explain the variability in bundle 
adoption rates. This area of inquiry involves micro-ethics, which is primarily concerned with the process of 
communication between providers and patients as they negotiate health care decisions.102 In terms of healthcare 
providers, the degree of concern, manner in which questions are asked, content and clarity of the information 
provided, suggestions/responses made to patient's doubts, vocabulary, voice tone, facial expressions, providing 
comfortable, and safe environment during physical examination are few micro-ethical issues which helps in 
gaining trust of the patient.102 While believed to play an important role in clinical outcomes, the field of micro-
decisions and micro-ethics remains largely unexplored in implementation science.  
 
The complexity of decision making and challenges of patient-provider communication in the ICU may 
partially explain low ABCDEF bundle performance rates. To date, consistent and regular adherence to the 
ABCDEF bundle has been shown to be difficult and suboptimal.38,103-105  Process improvements have been made 
to standardize bundle processes and establish accountability through adaptations to daily rounds, nursing and 
physician documentation, and medical record charting to reflect adherence to its protocols, yet ICU teams still 
struggle to achieve ICU bundle adherence goals. A recent article by Stollings et al106 emphasizes the importance 
of the cooperative efforts by the interprofessional team on implementing the ICU liberation bundle. Unfortunately, 
these authors, along with many others on the ICU liberation topic, overlook accommodating and attending to 
patient preferences and communication as a component of the bundle or implementation model. Several studies 
show that the three bundle components shown to have poorest adherence are: SATs, SBTs, and early 
exercise/mobility.40,103,107-109 At the end of the ICU Liberation Collaborative, an average ≤ 40% of patients 
participated in early mobility and less than half of patients requiring MV received a daily SBT.103 
 
Patient participation and preference are important considerations in the implementation of evidence-
based practices (EBP) and recommendations110, however little attention has been given to patient 
participation and preference in EBP implementation in the ICU. This is because, in early stages of critical 
illness, patients are often unable to participate in care or care decisions due to illness severity, fluctuating states 
of consciousness, decisional incapacity and communication impairment. Prior observational research, 
conducted by Drs. Happ and Tate, found that a substantive proportion (40%) of patients on prolonged MV 
participated in decisions about daily care (e.g., weaning trials, wound care, physical activity) as well as in 
decisions about life sustaining treatments.111 Recent case reports of decisional participation during MV in the 
ICU provide further validation that patient preferences and input can be ascertained with the use of appropriate 
assistive communication tools and techniques.112-114 As sedation use with MV patients in ICU decreases, 
opportunities for patient participation in care and treatment increase. We have little systematically collected data 
about how interprofessional teams interact with MV patients about planned treatments or procedures in ICU, 
particularly when patients resist or disagree with the team. These interactions and micro-decisions about care 
and treatment may be sources of ethical conflict. Prior research by Happ, Tate and others show that 
misinterpretation of MV patients’ communication and preferences can result in care directly counter to ICU 
liberation principles. 112,115-117 
 
Preliminary Data 
Our team has a unique opportunity to leverage and build on existing data and ongoing relationships with 
a network of motivated ICUs and the Society of Critical Care Medicine. Our team recently partnered with 
the SCCM to create the “ICU Liberation Collaborative," a QI project to foster the bedside application of the PAD 
Guidelines through wide-scale dissemination and implementation of the ABCDEF bundle.6,27 The Collaborative 
was led by an internationally recognized, multidisciplinary group with expertise in bundle-related clinical trials, 
implementation and dissemination research, and large-scale QI efforts. These experts operationally defined the 
new ABCDEF bundle. The SCCM provided overall project management, and each regional collaborative was 
led by a team including a registered nurse (RN), medical doctor (MD), respiratory therapist (RT), 
physical/occupational therapist (PT/OT), and pharmacist to mirror the interprofessionalism implicit in the bundle. 
The Collaborative included 68 ICUs from academic, community, and Veterans Affairs (VA) hospitals located 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
throughout the United States. Guided by the Consolidated Framework for Implementation Research (CFIR), 44 
the Collaborative formally ran from August 2015 to April 2017.  
Collaborative participants were required to attend four in-person meetings, monthly co-learning calls, and 
database training sessions. They were also invited to participate in a listserv and select in-person site visits. All 
teams submitted bundle-related performance and outcome data and completed pre- and post-implementation 
questionnaires focused on assessment of teamwork, work environment, overall ICU care, and organizational 
structure (see APPROACH).  
Based on our prior work,28,29,118 evidence-based professional behavioral change interventions and 
implementation strategies were shared and used by Collaborative members. For example, during the in-person 
meetings, the importance of providing audit and feedback, conducting cyclical small tests of change, changing 
record systems, and providing ongoing multimodal educational offerings to ICU staff were discussed. Sites were 
further encouraged to engage staff in local consensus discussions, develop a formal implementation and quality-
monitoring plan, and identify and involve bundle champions, early adopters, local opinion leaders, and former 
ICU patients/families in the implementation process. The in-person meetings also helped to reinforce and provide 
beneficial strategies for facilitating effective ICU communication, collaboration, and cooperation. The 
heterogeneity in implementation strategies used by collaborative participants provides a rich and unique 
opportunity to learn what interventions are associated with greatest bundle adoption.  
 
Our preliminary data demonstrate that ABCDEF bundle performance is associated with improved short-
term outcomes. To date, we have explored the association between collaborative member’s ABCDEF bundle 
performance and patient-, symptom-, and system-related outcomes (under-review). We defined bundle 
performance as either complete or proportional. This analysis included 15,226 critically ill adults with a variety of 
diagnoses who spent at least one full day in a participating ICU. Adjusting for a minimum of 18 a priori determined 
potential confounders, complete bundle performance was significantly associated with higher likelihood of ICU 
and hospital discharge and lower likelihood of hospital death within 7 days. Complete bundle performance was 
also significantly associated with lower likelihood of next day MV, coma, delirium, and physical restraint use and 
lower odds of ICU readmission and discharge to a facility other than home. Consistent with prior work,29 our data 
also demonstrated a dose-response relationship between higher proportional bundle performance and 
improvements in each of the above clinical outcomes (all p < 0.002).  
 
Our preliminary data also show that collaborative participation resulted in significant but highly variable 
ABCDEF bundle adoption. Our next goal was to determine if collaborative participation led to increased bundle 
adoption. We were interested in exploring if some sites were “better” “ABCDEF bundle adopters than others. 
The presence of a substantial amount of variability in our data would present us the opportunity to learn both 
from high- and low-bundle-performing sites. We calculated monthly rates of complete bundle performance, 
proportional bundle performance, and element-specific performance for the 17,258 patients across 49,109 full 
ICU days. We used segmented regression analysis for interrupted time series data to model the linear trend of 
bundle performance rates as a function of time for each segment (1-6 months pre-Collaborative involvement and 
7-20 months during the Collaborative).  
Complete bundle performance levels were low in 
the baseline (pre-Collaborative) period (i.e., < 4%). 
Complete bundle performance increased 2.0% (SE = 
0.9%, P 
= 0.057) 
immediately after 
joining the 
Collaborative, and each month was associated with 0.6% 
greater increase (SE = 0.2%, P = 0.038) in complete bundle 
performance rates, compared to a relatively flat trend in the 
pre-Collaborative period. Proportional bundle performance 
rates also significantly increased by 3.8% (SE = 1.6%, P = 
0.026) immediately after joining the Collaborative, then 
increased monthly at the same speed as the trend in the 
pre-Collaborative period (2%, SE = 0.4%, P = 0.57).  
Bundle performance was highly variable among 
ICUs and across individual bundle elements; interquartile 
ranges of > 20% existed for all performance measures. 
(Figure 1) Elements A, C, D, and F were performed on 
average ≥ 70% of the time at the end of the Collaborative: 
Figure 1. Variation among ICUs in Unadjusted 
ABCDEF Bundle Performance Rates

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
adoption of elements B1, B2, and E (SATs, SBTs, and early mobility) proved far more challenging. For example, 
at the end of the Collaborative, an average ≤ 40% of patients participated in early mobility and less than half of 
patients requiring MV received a daily SBT. Although overall performance was relatively poor, there were indeed 
some very good bundle performers.  
Understanding remains limited of the factors contributing to incomplete penetrance and variable 
implementation of the ABCDEF bundle. We and others have found barriers at the patient, provider, and 
organization level believed to affect ABCDEF bundle adoption.35,36,40, 119-128 These include: (1) intervention-
related issues (e.g., timing of SAT and SBT trials); (2) communication challenges; (3) knowledge deficits; (4) 
workload concerns; and (5) documentation burden. Most studies conducted on this topic were performed in 
single academic hospitals or health care systems and therefore have reduced generalizability. Moreover, prior 
studies are limited by the fact that they often rely solely on what ICU providers “say they do” versus measuring 
“what they actually do” in terms of bundle performance.35,36,40,91,124-128 This is problematic in that “perceived” and 
“actual” performance data often differ. Data that we have captured, but not yet explored, from the Collaborative 
combined with additional targeted data collection will address these important limitations.  
In summary, our preliminary data show that most critically ill adults are not receiving a potentially life-
saving and QOL enhancing intervention on a daily basis. Because national ABCDEF bundle performance rates 
are so low, our data also suggest that implementation efforts should, for now at least, continue to focus on ways 
of increasing bundle adoption rather than on ways of improving sustainability. Finally, the variability that is 
present in our data affords us the unique opportunity to learn from both high- and low-performing sites what 
constitutes an optimal and sustainable bundle implementation strategy.  
 
APPROACH 
The overall objective of the DISCOVER-ICU study is to develop multilevel implementation strategies to enhance 
sustainable adoption of the ABCDEF bundle in routine ICU practice. We are particularly interested in discovering 
how various patient-, provider-, and organization-level characteristics and implementation strategies effect 
ABCDEF bundle adoption. Results of this study will inform the development of implementation strategies to be 
tested in a cluster randomized hybrid II implementation-effectiveness trial to evaluate simultaneously the effect 
that the ABCDEF bundle has on long-term patient outcomes (not yet studied) and the best methods of adopting 
the bundle into practice. Congruent with the goals of the NHLBI, both the DISCOVER-ICU and future trial will 
generate an improved understanding of the processes involved in translating research into practice and use that 
understanding to enable improvements in public health and stimulate further scientific discovery. 
Our primary process-focused outcome for Specific Aims 1, 2, and 3 is adoption. Building off the definition 
offered by Proctor, 129 we operationalize adoption as the decision of an organization to commit to and perform 
the ABCDEF bundle. The level of analysis for adoption in Aim 1 is the patient and for Aim 2 the provider and 
organization. This process outcome is particularly salient for early to mid-implementation-stage projects such as 
the ICU Liberation Collaborative. Adoption in Aims 1 and 2 will be measured through existing medical record, 
administrative, and survey data. Our process outcomes for Aim 3 include acceptability, feasibility, and costs. 
Acceptability is the perception among ICU stakeholders that a given implementation strategy is agreeable and 
satisfactory for meeting the objective of improved ABCDEF bundle performance. Feasibility is the extent to 
which a given implementation strategy can be successfully used or carried out within a particular ICU or 
organization. Finally, we define cost as the perceived cost impact of a given implementation strategy. 
Acceptability, feasibility, and cost will be measured through an online survey, a concept mapping exercise, and 
expert panel of key ICU Liberation Collaborative and critical care leaders.  
Our primary process-focused outcome for Aim 4 is micro-decision making. Through focus group 
interviews with key ICU Liberation Collaborative participants, we specifically aim to describe micro-decision 
making about ABCDEF bundle implementation and to determine how and to what extent patient preferences are 
ascertained and considered when implementing evidenced-based ICU pain, sedation, MV liberation, and early 
mobility practices.  
 
Conceptual Framework. Our approach is guided by the Consolidated Framework for Implementation Research 
(CFIR),44 which provides a comprehensive taxonomy of operationally defined constructs from multiple disciplines 
(e.g., psychology, sociology) that are likely to influence implementation of complex programs; 39 constructs are 
organized across 5 major domains that interact to influence implementation effectiveness.  
 
Aim 1: Estimate the effects of patient-level characteristics on ABCDEF bundle adoption.  

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
Design and Rationale. In Aim 1, we will use existing SCCM ICU Liberation Collaborative data to estimate the 
effects that various non-modifiable factors exhibited by ICU adults at admission (i.e., patient-level characteristics) 
have on ABCDEF bundle adoption. Previous surveys, qualitative investigations, and systematic reviews reported 
that certain patient-level characteristics (e.g., instability, patient weight/size) may impede bundle delivery, but 
these studies were limited in use of self-reported (vs. actual) rates of implementation. 35,36,125,126,128 Findings from 
this aim will for the first time objectively distinguish which patient-level characteristics affect delivery of the overall 
bundle and its individual elements. Findings will have immediate relevance to patient care by aiding in the design 
of specific implementation strategies focused on overcoming patient-level barriers.  
 
Setting and Sample. Included in the Collaborative were adults who were (1) initially admitted to a participating 
ICU and (2) designated as needing ICU level of care. Patients previously admitted to a non-participating ICU 
during their current hospitalization were not included. “ICU level of care” was defined as patients who (1) needed 
intensive medical and/or nursing care that not could not be delivered in a step-down and/or medical/surgical unit, 
(2) were managed by an ICU team, or (3) did not have transfer orders written or downgraded to a lower level of 
care. Excluded were patients who (1) died or were discharged within 24 hours of ICU admission or (2) were 
undergoing life support withdrawal and/or documented comfort care only within 24 hours of ICU admission.  
Aim 1 data consist of 15,226 patients who spent 24 hours in a participating ICU (~49,000 ICU patient-days). 
Our sample includes patients on and off MV, ranging from 18-90+ years old, admitted with a variety of diagnoses. 
Of the 15,226 patients, 63% (n = 9619) were hospitalized in academic medical centers and admitted to mixed 
medical/surgical (56%), medical (18%), surgical/trauma (12%), neurologic (5%), or cardiac/surgical (10%) ICUs.  
 
Procedures and Data Sources. The Vanderbilt University Medical Center Institutional Review Board (IRB) served 
as the Coordinating Center IRB and approved the SCCM ICU Liberation Collaborative QI project. All 
Collaborative participants acquired site-specific IRB evaluation and approval. No identifiable data were collected. 
All data were collected by standardized medical record review and anonymous staff surveys and were managed 
using a secure, web-based application designed for validated data entry, transmission, and storage (Research 
Electronic Data Capture [REDCap], Vanderbilt University, 2016 - grant support [(UL1 TR000445 from 
NCATS/NIH]).  
Local staff who attended a training webinar and as-needed support from SCCM managers entered de-
identified data into the collaborative database at their individual hospitals. Data were collected retrospectively 
during a 6-month baseline period (January 2015-June 2015) and prospectively during a 14-month 
implementation period (January 2016-March 2017). During the baseline (pre-Collaborative) period, staff from 
each site entered data on the first 5 consecutively admitted ICU patients each month (30 baseline patients per 
site). Throughout the implementation period, site staff collected data on the first 15 consecutively admitted 
patients per month. For each patient, daily data were collected for a maximum of seven ICU days (average of 
3.3 ICU-days per patient) or until patients transferred out of the ICU, were designated non-ICU status, or died. 
 
Measures. Patient-level variables include: age, sex, race, ethnicity, language, body mass index (BMI), admitting 
diagnosis, severity of illness, and preadmission residence and mobility status. The severity of illness scores 
collected in the Collaborative included the Acute Physiology and Chronic Health Evaluation (APACHE) II, 130 
APACHE III,131 and APACHE IV132 referent to the patient's status at the time of ICU admission.  
 
Data analysis. We will conduct similar analyses for each of the 
performance measures (performed vs. not performed), including 
complete, proportional, and each element performance. We will 
first use descriptive statistics to examine data distribution, check 
for outliers, and summarize patient characteristics. Bivariate tests 
(e.g., t test or Chi-square statistics) will be used to compare 
patient characteristics in the pre-Collaborative and Collaborative 
periods.  
Using multilevel segmented logistic regression modeling for 
interrupted time series data, we can model the likelihood of 
bundle performance change over time, both before and during 
the periods of Collaborative implementation. Figure 2 shows a 
hypothetical example. The basic form of the level-1 model (Figure 
x, solid black line) can be written as Logit(𝑌𝑖𝑗) = 𝛽0𝑗+
Figure 2. Segmented logistic regression for ITS 
data-A hypothetical example 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
𝛽1𝑗Month + 𝛽2𝑗Imp + 𝛽3𝑗Month ∗Imp + 𝑒𝑖𝑗, where (1) Logit(Yij) denotes the logit (or log-odds) of receiving 
bundle performance for the 𝑖𝑡ℎICU-day of the 𝑗𝑡ℎ patient; (2) Month is the month indicator of the study period (a 
total of 20 months coded as -5 to 0 for pre-implementation period and 1 to 14 for implementation period); and 
(3) Imp denotes an indicator variable coded as 0 for pre-implementation period and 1 for the implementation 
period. Therefore, Logit(𝑌𝑖𝑗) is modeled as a linear function of 𝛽0𝑗+ 𝛽1𝑗Month for the pre-implementation period 
and (𝛽0𝑗+ 𝛽2𝑗) + ( 𝛽1𝑗+ 𝛽3𝑗)Month for the implementation period. 𝛽2𝑗 and 𝛽3𝑗 are the estimates of level and 
slope changes due to the intervention implementation. A significant level change (𝛽2𝑗) indicates an immediate 
change in the level of bundle performance at the initiation of intervention implementation. A significant slope 
change (𝛽3𝑗) indicates different trends over time during the pre- vs. post-implementation periods. Under the 
multilevel modeling framework, level-2 models will fit 𝛽0𝑗, 𝛽1𝑗, 𝛽2𝑗, and 𝛽3𝑗 each as a linear function of fixed-
effects of patient characteristics and patient-specific random effects. The patient-specific random terms allow for 
adjustment of within-patient clustering of ICU-days. Take patient gender as an example. From the combined 
multilevel segmented logistic regression model, we will (1) estimate the overall trends during the pre- and 
implementation periods (solid black lines); (2) derive fixed-effect estimates by patient characteristics (Figure 4, 
long dashed line and short dashed line for each gender); (3) test the significance of hypothesized subgroup 
differences (e.g., older age, obese, and non-English speaking) in the these estimates; and (4) include patient-
specific random terms to allow for variability by patients (patient-specific lines not shown in Figure 4). The model 
can be simplified if supported by the preliminary results. For example, if the initial analyses for an outcome (e.g., 
element E performance) suggest that (1) there are no significant level change and slope change for both male 
and female patients and (2) the overall increasing trend over time does not differ by patient gender, we can 
reduce the model to a parsimonious form using multilevel logistic regression to model the probability of having 
an outcome as a function of fixed effect of patient gender, fixed effect of time, and patient-specific random effects. 
We can also further extend the model to (1) include multiple patient characteristics simultaneously in the model 
to generate multivariable adjusted estimates, (2) perform curve fitting for linear or non-linear trend, and (3) adjust 
for additional clustering (e.g., within-organization clustering). The above descried multilevel segmented logistic 
regression approach will be applied to analyzing complete and element bundle performance measures. For 
proportional bundle performance, we will use multilevel segmented Poisson regression instead. 
 
Power analysis. We used the simulated-based power calculation method by Zhang et al. 133 for interrupted time 
series analysis to conduct power analysis for Aim 1. With the expected ~49,000 ICU-days of ~15,000 patients in 
68 ICUs, we generated 1,000 simulations with various patient subgroup distributions and proportion of ICU days 
with bundle performance during the pre-implementation and implementation periods. Our preliminary analyses 
of the pre-implementation period data suggest that the probability of bundle performance ranged from 0.05 (for 
complete bundle performance) to 0.60 (for performance of element A) at the end of the pre-implementation 
period and a monthly increase in the probability ranged from zero (for element E) to 0.05 (for element D). To be 
conservative in power calculation, we used a probability of 0.05 for the level and a 0.05 monthly increase in the 
probability for the slope in the pre-implementation period. Our power analysis suggested that our data will have 
80% power to detect an odds ratio (OR) of 1.25 (corresponding to an increase of 0.02 in the probability of 
receiving bundle performance) and 80% power to detect a slope change, with an OR of 1.52 corresponding to 
pre- vs. post-difference of 0.03 in the monthly increase of the probability of receiving bundle care. Based on our 
preliminary data, we reasonably assumed an autocorrelation of 0.5 under the first order auto-regressive 
covariance structure, a within-patient correlation of 0.5, and a within-hospital correlation of 0.6 in our simulation-
based power calculation. Under the same assumptions, we also conducted power analyses on the effects of 
patient characteristics on the level and slope changes. For patient characteristics with 20:80 distribution, our 
sample size will have 80% power to detect an OR of 1.73 for the effect of patient characteristics on level change 
and 80% power to detect an OR of 2.65 for the effect of patient characteristic on slope change. The above power 
calculation is the most conservative scenario. The study power will be greater than 80% for other cases (e.g., 
pre-probability of 0.5, patient characteristics with 50:50 distribution). Regardless of study power, we will report 
point estimates, 95% confidence intervals, and use clinical significance to guide results interpretation. All above 
calculations were based on multilevel segmented logistic regression with a 0.05 two-sided significance level and 
adjusted for data dependencies described above. 
Missing data. Data for Aim1 have a relatively low rate of missing (< 5%). We will primarily report data from 
complete cases without data imputation. As a sensitivity analysis, we will examine the pattern of missingness 
and repeat the analyses with multiple imputation.  
 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
Aim 2: Examine unit-level variation in ABCDEF bundle adoption and associated provider- and 
organization-level characteristics.  
Design and Rationale. In Aim 2, we will use existing SCCM ICU Liberation Collaborative data to examine unit-
level variation in ABCDEF bundle adoption and associated provider- and organization-level characteristics. 
Myriad provider- and organization-level characteristics are believed to affect delivery of the bundle. 35,40,124-126,128 
However, there is little evidence outlining which, and to what degree, these characteristics affect bundle 
performance. Given significant financial pressure, hospitals are seeking a “silver bullet” to reduce the iatrogenic 
risks associated with prolonged MV, delirium, and ICU-acquired weakness. Completion of this aim would 
advance knowledge by identifying provider- and organization-level characteristics amenable to implementation 
interventions.  
 
Setting and Sample. We will use provider- and organization-level data from two discrete surveys administered 
before and after the Collaborative. All full- and part-time nurses (RNs and Advanced Practice), MDs (residents, 
fellows, and attending), respiratory therapists, pharmacists, PT/OTs, nursing assistants, case managers, and 
other health care providers practicing in an ICU participating in the Collaborative were eligible to participate in 
the first survey, “Interprofessional ICU Team Survey.” The second survey, “Organizational Survey,” was 
completed jointly by the nursing and medical director of a participating ICU. In total, 7,307 ICU providers (5472 
pre; 3247 post) completed the Interprofessional ICU Team Survey. Organizational surveys were completed by 
the nursing and medical directors of all 68 ICUs. 
 
Procedures and Data Sources. Teamwork, communication, and a healthy work environment are fundamental to 
ICU practice and specifically to effective ABCDEF bundle implementation. All collaborative in-person meetings 
and co-learning calls focused on teaching about and improving these concepts to ensure that all sites had a 
fundamental understanding of who comprises the team, how to evaluate team health, and what tools the teams 
could use to facilitate daily work and communication. Similarly, each site had varying experience with the 
individual components of the bundle, necessitating a formal assessment of providers’ knowledge and 
perceptions of the bundle. To help each site develop a better understanding of their interprofessional team, 
identify potential barriers and facilitators to ABCDEF bundle implementation, and foster educational and 
implementation efforts, an anonymous online survey (Interprofessional ICU Team Survey) was created and 
distributed to all members of the ICU team at the beginning and end of the Collaborative. Because ICU- and 
hospital-level variations in structure and process play important roles in patient- and family-centered outcomes, 
134 we developed an Organizational Survey modeled on the work of Checkley et al. 135 to identify characteristics 
of the ICUs. This survey also provided data for comparison of performance between like hospitals as well as for 
comparison of large/academic centers and community hospitals.  
 
Measures. The Interprofessional ICU Team Survey was comprised of three discrete parts: the Assessment of 
Interprofessional Team Collaboration Scale (AITCS), 136 American Association of Critical-Care Nurses (AACNs) 
Healthy Work Environment Assessment tool (HWEA), 137 and the ICU Care and Perceptions Survey. 
1) AITCS. The AITCS is 37-item survey designed to evaluate teamwork and collaboration. The AITCS includes 
three sections (cooperation, coordination, and partnership/shared decision-making) assessed on a 5-point 
Likert scale (5 = always, 1 = never). Internal consistency estimates for reliability of each subscale range from 
0.80 to 0.97, with an overall reliability of 0.98.118 The AITCS survey is being used as proxy measure for the 
CFIR’s “inner setting” domains of networks and communication, culture, implementation climate, and 
readiness for implementation.  
2) HWEA. The HWEA is an 18-item survey designed to evaluate the six AACN standards for establishing and 
sustaining a healthy work environment: skilled communication, true collaboration, effective decision making, 
appropriate staffing, meaningful recognition, and authentic leadership. Each standard is assessed on a scale 
of three unique questions. The 18 items are assessed on a 5-point Likert scale (5 = strongly agree, 1 = 
strongly disagree). The questions and scales were reviewed for face validity and tested for reliability and 
showed internal consistency with identical factor structures and Cronbach's alpha scores of ≥0.80. The 
HWEA is being used as proxy measure for the CFIR’s constructs “inner setting” and “characteristics of 
individuals" domains of networks/communication, culture, implementation climate, readiness for 
implementation, individual identification with organization, and other personal attributes.  
3) ICU Care and Perceptions Survey. This survey, developed by Collaborative faculty, contains 83 items to 
evaluate respondents’ beliefs, self-efficacy, perceptions, resources, reported practices, and awareness of 
polices and protocols related to the ABCDEF bundle and its individual components. Of the 83 questions, 11 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
are in a yes/no format, and the rest are based on a 5-point Likert scale (5 = strongly agree, 1 = strongly 
disagree). The ICU Care and Perceptions Survey is being used as proxy measure for the CFIR’s construct 
characteristics of the individuals' domains of knowledge and beliefs about the intervention, self-efficacy, 
individual’s stages of change, and other personal attributes. This survey relates to the CFIR’s “intervention 
characteristics” domains of evidence strength and quality, relative advantage, cost, and complexity.  
 
The Organizational Survey collected information on (1) hospital and ICU characteristics, (2) utilization rates, 
(3) ICU staffing patterns, (4) ICU organization, and (5) ICU rounding and ABCDEF bundle practices. The first 
section contains questions on characteristics of the hospital (i.e., type, locale, teaching status) and ICU (i.e., 
type, training programs). The next section contains questions focused on utilization data, such as total number 
of hospital, ICU, and step-down beds, and number of annual hospital and ICU admissions. The third section 
collects ICU MD, advanced practice provider, RN, and additional ICU team member staffing, education, and 
certification data. The fourth section has questions on the organization of the ICU, such as whether it is an 
open, semi-open, or closed unit, and if the ICU has a medical and/or nursing director. The final section has 
questions regarding ICU rounding and ABCDEF practices. This section collects specific data regarding the 
use of daily rounds, where rounds are performed, who regularly attends rounds, use of PAD assessment tools, 
which protocols and teamwork tools the ICU uses, and types of QI experience. This survey was completed by 
both the nursing and medical director of a participating Collaborative team. The Organizational Survey is being 
used as proxy measure for the CFIR’s construct “inner setting” domains of structural characteristics, networks 
and communication, culture, implementation climate, readiness for implementation. This survey also relates to 
the CFIR’s “process” domains of planning and engaging. 
 
Sample Size/Power Analysis. The reliability of ICU-specific performance rates depends on sample size in each 
hospital (ICU days after adjusting for within-patient clustering). Using the Spearman-Brown prophecy 
formula,138,139 a minimal sample size of 10 is needed to generate reliable ICU-specific estimates (reliability ≥ 0.8), 
assuming an intraclass correlation of 0.3 for between-ICU variation based on our preliminary analysis. Given the 
large number of observations in our data (~49,000 ICU patient-days nested in ~15,000 patients in 68 ICUs), 
there will be sufficient sample size to generate an ICU-specific performance estimate with ≥ 0.8 reliability, even 
conservatively assuming a large within-patient correlation of 0.8. We conducted power analysis for the effects of 
organization-level characteristics using the same simulation-based approach described for Aim 1 power analysis. 
Under the most conservative scenario (see power analysis of Aim 1), our sample size will have sufficient power 
(≥ 80%) to detect an OR of 1.80 for the effect of an implementation strategy (used vs. not used by the site) on 
level change and an OR of 2.96 for the effect of an implementation strategy in slope change, assuming 20:80 
distribution of the implementation strategy (used vs. not used by the site). 
 
Data Analysis. The overall analytic strategy for Aim 2 will be the same as described for Aim 1. That is, using 
multilevel segmented logistic regression modeling for complete and element bundle performance and multilevel 
segmented Poisson regression for proportional performance. First, we will examine the variation across ICUs in 
bundle performance by adding ICU-specific random effects to the model. All significant patient-level 
characteristics from Aim 1 will remain in the model as covariates so that ICU-level estimates are risk-adjusted 
for patient case mix. From the model, we can (1) quantify the degree of ICU variation using the intra-class 
correlation (ICC); (2) derive the ICU-specific adjusted rates of bundle performance at the last month of 
implementation and examine their distribution (mean, variance, median, interquartile range, and range); and (3) 
rank the ICU-specific adjusted rates from low to high and identify the top 5 and bottom 5 ICUs with best and 
worst adjusted performance rates. Next, we will estimate the effects of organizational characteristic on bundle 
adoption by adding them as fixed effects to the model. Organizational survey at the post-implementation period 
will be estimated as fixed effects to model the level change (𝛽2𝑗) and slope change (𝛽3𝑗). While data from the 
pre-implementation period will be estimated as fixed effects to model pre-implementation level (𝛽0𝑗), pre-
implementation slope (𝛽1𝑗), post-implementation level change (𝛽2𝑗), and slope change (𝛽3𝑗) (see Data Analysis 
section for Aim 1). Again, multilevel clustering (within-patient clustering of ICU days and within-ICU clustering 
of patients) will be adjusted in the model using random effects terms. As the provider-level data can be linked to 
specific ICUs, but not linkable to specific patient and patient ICU days, we will use multilevel modeling (level-1: 
time-point [pre- and post-survey], level-2: provider; level-3: ICU) to model the fixed effect of time, provider-level 
assessments (AITCS, HWEA, 136,137and ICU Care and Perceptions surveys), and their interactions on ICU-level 
adjusted performance rates. From the model, we can derive contrast estimates on the effect of change in 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
provider-level assessments on ICU-level bundle adoption. We will also extend the model to adjust for ICU-within-
organization clustering.  
 
Missing data. All 68 ICUs before and 42 ICUs after the Collaborative completed the organizational survey. The 
advantages of the multilevel model for analyzing longitudinal data lie in its flexibility in handling missing data and 
more efficient use of available data. Multilevel modeling allows for varying numbers of waves of data per study 
unit; the same number of waves of data is not required for all hospitals. Thus, missing data will be more easily 
managed, and all available data will be used for model estimates. Nonetheless, we will carefully examine the 
extent and pattern of missing data and impute missing values when appropriate. If missing not at random exists, 
we will use pattern-mixture modeling. We will also conduct sensitivity analyses to examine the robustness of 
study findings with and without multiple imputation or using pattern-mixture modeling.  
 
Aim 3: Determine which implementation strategies result in the greatest adoption of the ABCDEF bundle.  
Design and Rationale. Identifying feasible and effective ABCDEF bundle implementation strategies that are 
contextually appropriate remains a challenge for ICU researchers and practicing clinicians. In Aim 3, we will 
extend data collection and use both quantitative and qualitative approaches (i.e., using an online survey, concept 
mapping exercise, and an expert panel) to gain a deeper understanding of which implementation strategies are 
most likely to overcome barriers to ABCDEF bundle adoption.  
 
Setting and Sample. Healthcare providers from the 68 sites that participated in the ICU Liberation Collaborative 
will be included in the work outlined in Specific Aim 3. These healthcare providers will have experience working 
in one of the ICU Liberation Collaborative units and come from various professional groups (e.g., nursing, 
medicine, pharmacy, physical/occupational/respiratory therapy). We will also include a subgroup of 10-12 
healthcare providers who currently serve of the SCCM’s ICU Liberation and/or Quality and Safety committee 
and two members of the SCCM’s Council who are responsible for overseeing the society’s quality & safety 
initiatives in an expert panel.  
 
Procedures and Data Sources.  
1. Implementation Methods Survey. We will develop an online survey based on the implementation strategies 
described in the Expert Recommendations for Implementing Change (ERIC) Project.131 While identifying, 
developing, and testing implementation strategies are important goals of implementation science, these efforts 
had previously been complicated by the use of inconsistent language and inadequate descriptions of 
implementation strategies in the published literature. Based on an expert panel of implementation scientists and 
mental health clinical managers, the ERIC study addressed this need by publishing a final compilation of 73 
implementation strategy terms and definitions.131  
Recruitment. A letter of invitation to participate in the survey will be sent by a SCCM study team member 
(Harmon) via email to healthcare providers who were primarily responsible for ABCDEF bundle implementation 
efforts at the 68 sites who participated in the ICU Liberation Collaborative. The invitation letter will describe the 
purpose of the study, provide instructions on how the survey may be completed, and inform recipients that 
participation is voluntary, that no identifiable information will be collected, and who to contact with study-related 
questions. The healthcare providers will be informed that each ICU Liberation Collaborative site will be allowed 
to submit only one survey and that this survey should be completed with feedback from ICU team members 
representing various disciplines (e.g., nursing, medicine, pharmacy, respiratory/physical/ occupational therapy).  
The healthcare providers will be instructed that there is no set approach in garnering their team’s survey 
input. They will be encouraged to do what is best for them and their ICU team. The letter will inform potential 
participants that the study team acknowledges the fact that there may be some disagreements among 
professionals on various aspects of the survey and instruct them to complete the survey as well as they can and 
as closely aligned with team’s consensus as possible. The following options for completing the site survey will 
be suggested: 
1) Conduct a meeting specifically focused on completing the online survey. During this team meeting, the 
online survey could be accessed, questions discussed, and responses entered simultaneously. We 
recommend this approach as it is likely to be the most comprehensive and least time consuming.   
2) Print a hardcopy of the survey to discuss and complete during a regularly scheduled or small group team 
meeting. Once the survey is complete, one team member would then enter the site data into the online 
survey. 

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
3) Print hardcopies of the survey and ask each team member to complete it individually. Once responses 
are received, one team member would then resolve any conflicting answers and enter the site data into 
the online survey. 
4) Distribute a word or PDF version of the survey via email communication to team members. Completed 
surveys could then be given to one team member who would then would resolve any conflicting answers 
and enter the site data into the online survey. 
Each team member who participates in completing the survey, maximum of four team members per 
site, will be eligible to receive a $25 Amazon gift card. In addition, the person responsible for entering the final 
site data into the online survey will be eligible to receive an additional $25 Amazon gift card. Reimbursement will 
occur once all survey data is entered. A maximum of 272 providers will participate in the online survey.  
Methods. The first section of the survey will provide participants the name and brief description of each 
of the 73 implementation strategies recommended in the ERIC project. Participants will be asked to think about 
the strategies they used during the course of the ICU Liberation Collaborative to increase their units’ adoption of 
the ABCDEF bundle. Participants will then be asked to review each ERIC implementation strategy and to select 
“yes or no” as to whether they used the strategy during the time the spent in the ICU Liberation Collaborative. If 
“no” is selected, participants will be directed to the next question. If “yes” is selected, the participants will then 
be asked to rate, on a 5-point Likert scale, how (1) helpful, (2) acceptable, (3) feasible, and (4) costly it was to 
use the strategy. In the next section of the survey, participants will be provided the opportunity to report any 
additional implementation strategies they used to foster ABCDEF bundle adoption during their time in the ICU 
Liberation Collaborative that were not included in Section 1. If participants think of any additional strategies, they 
will be asked to provide the name and a short definition of the strategy. They will then be asked to rate, on a 5-
point Likert scale, how (1) helpful, (2) acceptable, (3) feasible, and (4) costly it was to use the strategy. If 
participants feel the list of terms in Section 1 was adequately comprehensive, they will be instructed to leave this 
section blank. The final section of this survey will collect basic information regarding site location and type of 
professionals who assisted in survey completion. Because we are requesting the engagement of multiple 
stakeholders, we anticipate that the time necessary to complete the survey may vary by site. We estimate it will 
take approximately 20 minutes for an individual to complete the survey and approximately 40-50 minutes for 
team gathering and joint completion of the survey. 
 
2. Concept Mapping Exercise. Results of the online survey will be used to inform the development of a group 
concept mapping exercise aimed at further understanding what implementation strategies result in the greatest 
adoption of the ABCDEF bundle. Concept mapping is considered a substantially stronger methodological 
approach for characterizing how complex concepts (like implementation strategies) are organized than less 
structured group consensus methods.110-111 Concept mapping in this project will use the Concept Systems Global 
MAX© web platform for participation and data analysis.  
Recruitment. A letter of invitation to participate in the concept mapping exercise will be sent by a SCCM 
study team member (Harmon) via email to the healthcare providers who were primarily responsible for ABCDEF 
bundle implementation efforts at the 68 sites who participated in the ICU Liberation Collaborative. The invitation 
letter will describe the purpose of the study, provide instructions on how the concept mapping exercise will be 
completed, and inform recipients that participation is voluntary, that no identifiable information will be collected, 
and who to contact with study-related questions. The healthcare providers will be asked to invite team members 
from various professions (e.g., nursing, medicine, pharmacy, respiratory/ physical/occupational therapy). A 
maximum of 204 providers will participate in the concept mapping exercise.  
Each participant who participates in the full concept mapping exercise will receive a $75 Amazon gift 
card. 
 
Methods. Group Concept Mapping (GCM) is a unique approach for planning and evaluation that uses 
the knowledge and opinions of stakeholders and specific statistical tools to produce visual results that lead to 
understanding, agreement, and action. Valuing the voices of stakeholders, experts, or other participants in the 
process, GCM consolidates perspectives into a concise, readable series of graphic representations and reports. 
The multi-step method involves input from stakeholders, in this case ICU providers,  whose knowledge and 
opinions are relevant to the problem being studied (i.e., ABCDEF bundle implementation) and the resulting maps 
feed back to the group a model, or “map” of their thinking that can be used for subsequent action.  

IRB Protocol Number: 2019H0092 
IRB Approval date:  
     Version:  03/30/2022 
The GCM process asks participants to brainstorm 
statements relevant to a topic of interest, sort the 
statements into piles based on each individual’s 
perception of similarity, and rate each statement on one 
or more scales. People can participate either 
electronically (as will be the case in the DISCOVER 
ICU study) or in face-to-face sessions. GCM computes 
maps and other figures and supports the investigative 
team to develop agendas, strategic plans, and products. 
The GCM process supports the post-map work of 
determining priorities, actions, and measures to inform 
the planning. GCM’s graphical representations make it 
easy to identify common themes, capture and represent 
priorities, and develop an implementation plan. For 
example, the hypothetical Concept Map displayed in 
Figure 3 illustrates the conceptual array based on 
participants’ ideas and how every individual perceived 
their relationships. Clusters contain many ideas and are 
grouped into meaningful concepts by the sorting activity.  
A Pattern Match (example provided in Figure 4) 
uses participant ratings to illustrate the degree of 
agreement between Rating 1 and Rating 2, as in this 
example, or between Subgroup 1 and Subgroup 2 on a 
Rating. Pattern matches help the investigative team 
identify whether agreement exists, disconnects that are 
evident, and whether these can be addressed.    
Finally, a Go Zone shows planners the specific 
items that are within a certain cluster, and describes their 
relative rating values, by arraying ratings on two scales. 
Figure 5 is an example for one cluster. Each point 
represents a specific idea, such as item 34: Support 
better family functioning and parental mental health and 
wellness, which was rated high on importance but lower 
on current presence, suggesting its priority in planning. 
Awareness of the issues and agreement on their 
priorities, emerge from the results, allowing a group to 
take action, development implantation strategies, and 
measure their progress and success. 
We anticipate that the time necessary for participants to complete all components of the concept mapping 
exercise will be approximately 30 minutes. All responses 
are entered into a secure web-based program and no 
identifying information will be collected other than 
professional category (e.g., nurse, physician) and site 
location. 
 
3. Expert Panel. The final step for Aim 3 is to solicit input 
from a panel of experts in the critical care field in order to 
gain a consensus on practice and policy recommendations 
for adoption of the ABCDEF bundle.  
Recruitment. A letter of invitation to participate in the 
expert panel will be sent by a SCCM study team member 
(Harmon) via email to current members of the Society of 
Critical Care Medicine’s ICU Liberation and Quality & 
Safety committees. We will invite a subgroup of 10-12 
interprofessional members to participate on the expert 
panel. Two members of the SCCM’s Council responsible  
Recognize Workplace
Influence on Wellbeing
Advocate for 
Community Change
Tackle Barriers 
through Policy
Invest in Systems 
for Equity
Sustain 
Collaboration
Focus on Children 
and Families
Use Research for 
Community
Improvement
Inspire Change 
through Knowledge
Figure 4. Example Pattern Match  
Figure 5. Example Go Zone  
Figure 3. Example Concept Map  

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
14 
 
for the organizations quality & safety line will also be asked to participate. The invitation letter will describe the 
purpose of the expert panel, provide instructions on how the meeting will be conducted, and inform recipients 
that participation is voluntary, that no identifiable information will be collected, and who to contact with study-
related questions. Participants will not be reimbursed for participation.   
 
Methods. Study team members with expertise in conducting focus groups and interviews with ICU 
providers (Balas, Mion) will lead the expert panel of ICU providers in a live polling and consensus building 
process using a web-based interactive discussion platform (i.e., Zoom). Before the webinar, panelists will be 
emailed the results of the concept mapping exercise and a summary of topics that will be discussed during the 
meeting. One researcher will lead the discussion and the other will record notes, monitor the discussion, and 
help facilitate keeping to the agenda. We expect the expert panel meeting to last 60-90 minutes. 
 
Data Analysis. Survey. Quantitative data of the survey will be analyzed using the same approach as in Aim 2 
for organization-level characteristics. In short, implementation strategies will be estimated as fixed effects to 
model the level change and slope change using multilevel segmented logistic regression modeling with  
patient-, hospital-, and organization-specific random effects to adjust for multilevel clustering effects. Concept 
Mapping. The study’s planned minimum enrollment of 40 is above the recommended sample size for concept 
mapping (≥ 15).146 In this stage, multidimensional scaling and hierarchical cluster analysis will be used to 
characterize how implementation terms were clustered by the healthcare provider panelists, providing the 
opportunity to quantitatively characterize the categories of terms developed by the providers in terms of how 
they were rated on key dimensions. Final data analyses will include visual summaries of data, including weighted 
and unweighted cluster maps, ladder graphs, and go-zone graphs, all specific tools from the web platform used 
for this analysis.141-143 As described above cluster maps provide a visual representation of the relatedness of 
concepts, and weighted cluster maps are used to depict how concepts within a cluster were rated on key 
dimensions (e.g., importance). Ladder graphs provide a visual representation of the relationship between 
dimensions of a concept (e.g., importance and feasibility, importance and changeability). Go-zone graphs are 
useful for illustrating the concepts that are most actionable (e.g., high importance and high feasibility) and which 
concepts are less actionable (low importance and low feasibility). Bridge values (i.e., quantitative 
characterizations of how closely individual concepts within a cluster are related) will also be reported. Expert 
Panel. Following the interactive web-based discussion, the investigator team will review the notes, summarize 
the main points of discussion and include the list of recommendations that emerged from the Expert Panel 
discussion. These findings and recommendations will be circulated to the participants for feedback and 
validation.  
 
Aim 4: Identify and describe the micro-decisions (e.g., resource allocation, patient participation, 
preferences, agreement) involved in implementing SATs, SBTs, and early mobility/exercise in everyday 
care.  
 
Design and Rationale. We propose a qualitative dominant, mixed-methods companion study to the Discover-
ICU Study. This work will be performed in Year 2 of the Discover ICU study. We will conduct focus group 
interviews with interprofessional teams from ten (5) high- and (5) low- bundle-performing ICUs. This is a modified, 
positive-negative deviance approach to exploit differences between high performing and low-performing units 
that will be used to identify the ethical challenges (e.g., conflicts, tradeoffs) experienced by clinicians in daily 
bedside decision making about bundle implementation. Focus groups will be composed of interprofessional 
members (e.g., physicians, nurses, physical therapists, respiratory therapists, pharmacists, occupational 
therapists, speech language pathologists) of the ICU liberation team from each unit.  
 
Setting/Sample. We will conduct the focus group interviews remotely via Zoom videoconferencing “rooms” (Zoom 
Video Communications, Inc), a procedure used successfully for interviews with clinicians in our organization and 
by others.147 A total of 50 – 80 interprofessional team members will participate (5-8 participants each for 10 
selected ICUs). This sample represents nearly 15% of the ICU Liberation Collaborative and will provide a 
substantive body of narrative data sufficient to address the research questions.  
The OSU biostatistician, Dr. Tan, will conduct purposive selection of five ICUs that are high- performing 
and five ICUs that are low- performing on SBTs based on analysis from the Discover ICU study dataset. Selection 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
15 
 
will also consider maximum variability on performance of SATs, early mobility/exercise, hospital geographic 
location, hospital size, and annual number of ICU admissions. We are conducting maximum variation sampling 
because high performing environments may have different approaches, considerations, and challenges in micro-
decisions about implementing SATs, SBTs and early mobility than low-performing units. This method will also 
produce the most representative sample to ensure credibility and transferability of findings. If selected ICUs 
refuse or are unable to participate, we will replace from the sampling pool. This method will also produce the 
most representative sample to ensure credibility and transferability of findings. 
ICU clinicians participating in focus group interviews will be full or part-time ICU physicians, nurses, 
respiratory therapists, occupational and physical therapists, pharmacists and speech language pathologists who 
are members of the ICU liberation team in selected ICUs. Inclusion criteria: clinicians must be in ICU practice 
for at least 1 year and at least 18 years of age Exclusion criteria: unit managers and administrators will be 
excluded to avoid any feelings of intimidation or fear of retribution among group members. Women of child-
bearing age or pregnant women will not be excluded from this study.  
 
Study Procedures 
 
Recruitment. Ms. Harmon, our SCCM research partner, will contact prospective ICU interprofessional 
teams to invite their participation and ascertain interest and willingness to participate in the focus groups. Once 
interest in focus group participation is obtained from the ICU teams by SCCM staff, the Ohio State University 
(OSU) research team will contact the ICU team lead by email (or phone, if emails do not elicit a response) and 
schedule the Zoom meeting.147 Electronic informed consent will be distributed by the OSU team and completed 
by all participants before beginning the focus group interview.  
 
Data Collection. Unit Organizational and Demographic Characteristics: We will collect information from 
the Organizational Survey 135 data contained in the parent study database to describe the ten participating units. 
This dataset includes information on (1) hospital and ICU characteristics and utilization, (2) ICU staffing patterns, 
and (3) ICU rounding and ICU liberation bundle practices. We will describe the 10 units in the focus group sample 
on characteristics of the hospital (i.e., type, locale, teaching status, total number of beds, annual admissions) 
and ICU (i.e., type, training programs; number of ICU and step-down beds, open, semi-open, or closed unit, and 
number of ICU admissions). We will also describe ICU staffing patterns (ICU MD, advanced practice provider, 
RN, and additional ICU team member staffing, education, and certification data) and ICU rounding and ABCDEF 
bundle practices. We will collect the following characteristics from focus group participants: profession, critical 
care accreditation, highest education, years in critical care, age category, sex, race/ethnicity. 
 
Focus Group Interviews: The focus groups will be conducted by Drs. Mary Beth Happ and Judith Tate, 
experienced qualitative researchers via an online Zoom meeting platform. The advantage of this platform is that 
(1) it is an affordable mechanism to access clinical teams throughout the nation and that (2) individual team 
members can participate from a variety of locations (i.e., home, hospital or office) to facilitate scheduling and 
attendance.147 Both researchers have experience conducting focus groups and interviews with critical care 
clinicians. 113,115,148-152 Dr. Tate recently conducted interviews with interprofessional long term acute care teams 
about their experiences with and perspectives on ICU liberation bundle components. 153  
Drs. Happ and Tate have experience as critical care clinicians and credibility as critical care researchers, 
but have not participated in the ICU Liberation Collaborative or in SCCM sponsored trainings. Therefore, ICU 
Liberation teams may be more forthcoming during focus group interviews conducted by this pair. One researcher 
will lead the discussion and the other will record notes regarding nonverbal communication, interactions, tone, 
and responses. Teams will first be introduced to the focus group leaders and made to feel comfortable with brief 
social dialogue. We will then orient participants to the ground rules of focus group participation (e.g., 
confidentiality, respectful dialogue and turn-taking, all opinions are valued, and nonjudgmental stance).  
Participants will be asked to select a recent case of a mechanically ventilated patient eligible for SATs, 
SBTs, and early mobility and to present the de-identified story of a single day in caring for the patient. We will 
elicit the participants’ stories and viewpoints about considerations, conflicts, and decisions that they made about 
SATs, SBTs, and early mobility/exercise. We will ask them to describe interactions with the patient including how 
the patient was informed of the aforementioned interventions, what input the patient had in the process, and how 
the patient responded. We will obtain clinicians’ perspectives on ascertaining and considering patient 
preferences about SATs, SBTs, and early mobility/exercise bundle procedures as well as their descriptions of 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
16 
 
conflicts and tradeoffs in decisions to implement the procedures. See draft focus group interview guide 
(Appendix A). Interview probes may change over the course of the study to address and integrate topics and 
concerns raised in prior focus groups. We will also ask the participants to compare and contrast this case 
experience to other cases of mechanically ventilated patients in their unit. Focus group interviews will be audio 
recorded via Zoom, saved electronically as an audio file on a secure, single sign-on server, and transcribed 
verbatim. These qualitative methods will allow us to comprehensively characterize patient-provider processes in 
micro-decisions regarding implementing the SAT, SBT, and early mobility/exercise components of the ABCDEF 
bundle, including how the interprofessional team responds when patients communicate dissent or preferences 
different from the team. Focus groups are expected to last 60-90 minutes. 
 
Data Management and Analysis. The qualitative study design and analysis follow the consolidated criteria for 
reporting qualitative research COREQ guidelines.154 Focus group interview recordings will be transcribed 
professionally along with researcher notes describing participant behaviors and tone. Transcripts will be 
reviewed with the recordings for accuracy. We will use Dedoose (dedoose.com) secure, on-line qualitative data 
management program to manage the data. Initial data coding of the focus group transcripts and notes will be 
conducted by Drs. Happ and Tate using basic qualitative description and constant comparative techniques.155-
157 Coding and analysis will be initiated after the first focus group and continue during subsequent focus group 
data collection. Portions of text will be labeled or coded with terms that are low inference (“data close”) and 
descriptive of participant words.158 Codes will be grouped into thematic categories, and subcategories. All 
transcripts will be dual coded by Happ and Tate where codes and categories will be defined and coding 
discrepancies will be resolved during weekly analytic meetings. We will identify relationships between categories. 
Analytic procedures will include writing individual case (unit) summaries and thematic summaries, constructing 
stem-leaf plots to assess strength and distribution of themes across cases (units) and unit categories (high vs 
low-performing), identifying properties of themes and subthemes and creating visual hierarchical or relational 
diagrams. We will compare and contrast high and low bundle performing units through dimensional analysis and 
by constructing matrices.159 Organizational Survey data will be categorically displayed and selected data from 
the Organizational Survey, such as unit characteristics, rounding practices, and ICU liberation bundle practices 
crossed with primary themes in a matrix analysis.  Drs. Happ and Tate will share the analysis of the first four 
focus groups with the larger DISCOVER group investigators to elicit their input on additional analytic directions, 
alternate interpretations of focus group data and recommendations for other information to be elicited in the 
remaining focus groups. Key focus group participants will be selected to provide feedback on initial findings; their 
feedback will be incorporated into the final analytic product. As a final step, we will subject the focus group 
thematic categories and subcategories to traditional qualitative concept mapping45 to identify relationships and 
potentially actionable intervention points in micro-decision processes to improve implementation of SATs, SBTs, 
and early mobility/exercise with MV patients. 
 
Potential Problems and Alternative Approaches and Benchmarks for Success 
Potential limitations of Aims 1 and 2 include the possibility of missing data and the large number of data elements 
to be analyzed. Our team has intimate knowledge of the database and the key measures we are using. We have 
collapsed the data into relatively broad categories based on the constructs/domains of the CFIR and will continue 
to refine these measures using the experience of the entire team. Should we find key elements missing, our 
continued relationship with the Collaborative sites will allow us to explore additional methods of addressing these 
limitations. While we have a rigorous plan to adjust for patient case mix in deriving ICU-level risk-adjusted bundle 
adoption rates, there will admittedly be unmeasured confounders (e.g., patient or caregiver preference) that may 
bias the ICU-level estimate. However, it is reasonable to expect that such unmeasured confounders will be 
distributed evenly across ICUs. Finally, there may be concern regarding our ability to recruit the necessary 
participants to complete the work proposed in Aims 3 and 4. As demonstrated in the sample letters of support, 
Collaborative members remain tremendously excited and committed to furthering the important work of bundle 
adoption.  
Potential Pitfalls foreseen for Aim 4 are due to selection of high and low performing units for focus group 
participation will be based on data collected in 2015-2017, therefore unit characteristics may have changed and 
unit adherence may have improved or regressed since the data were collected. We also recognize that ICU 
environments and experiences in the care of mechanically ventilated patients are rapidly changing in response 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
17 
 
to the COVID19 pandemic. This reality makes the addition of this micro-ethics study aim even more important 
as it will provide relevant context for the DISCOVER ICU study data integration and interpretation. 
Interprofessional team members may be reluctant to fully disclose ethical concerns in group interviews with other 
team members. Our experience has been that when techniques to engender social comfort are used and the 
discussion focuses on patient stories, interprofessional workgroups “open up.” Participation in focus group 
interviews may be low. We will employ the following strategies to boost participation in focus groups: 1) we will 
offer an incentive to each unit of $50 dollar Amazon gift card to each focus group participant (payment will occur 
once the focus group is completed) and 2) we will offer each group a 1-year subscription to online communication 
training and toolkit in techniques to improve patient-provider communication. The SCCM connection and 
membership in the ICU Liberation Collaborative provide added motivation and incentive to participate. We expect 
that we will be successful in recruiting 10 units to participate in focus groups.  
 
Expected Outcomes and Future Directions 
The bioethics of implementation research is in its infancy. While patient engagement in implementation research 
has been discussed, little evidence exists about the process individuals undertake to elicit patient preferences, 
negotiate conflicts or include patient responses to bedside care with mechanically ventilated patients in the ICU. 
The need to optimize patient recovery following treatment with MV while minimizing provider burden has never 
been more important as the COVID-19 pandemic dramatically increases the number of adults of all ages who 
will experience an ICU admission and MV treatment. Future research could be directed at enhancing patient 
engagement with everyday micro-decisions to improve implementation of SATs, SBTs and early 
mobility/exercise. An additional research direction will be to determine if improved communication and patient 
engagement improves bundle performance and/or decreases moral distress of caregivers. 
 
 
Study Timeline for Aim 1-4 
Year 
1 
2 
3 
Quarter 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
Submit IRB, recruit, hire, and train study personnel 
X 
X 
 
 
 
 
 
 
 
 
 
 
Study team in-person meetings 
 
X 
 
 
 
X 
 
 
 
X 
 
 
Data extraction and cleaning Aim 1 and 2 
 
X 
X 
X 
X 
 
 
 
 
 
 
 
Data analysis Aim 1 and 2 
 
 
 
 
X 
X 
X 
X 
 
 
 
 
Survey and concept mapping development Aim 3 
 
 
X 
X 
 
 
 
 
 
 
 
 
Aim 3 Administer survey, concept mapping exercise, and 
expert panel 
 
 
 
 
X 
X 
X 
X 
X 
X 
 
 
Write and submit papers, disseminate study findings 
 
 
 
 
 
 
X 
X 
X 
X 
X 
X 
Aim 4 Qualitative Focus group, identify and recruit sites 
 
 
 
 
 
 
X 
 
 
 
 
 
Schedule and conduct focus group interviews 
 
 
 
 
 
 
 
X 
X 
X 
X 
 
Data analysis and member checking 
 
 
 
 
 
 
 
X 
X 
X 
X 
 
Dissemination, abstract and manuscript submissions 
 
 
 
 
 
 
 
 
 
 
X 
X 
 
 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
18 
 
References 
1. 
Elliott D, Davidson JE, Harvey MA, et al. Exploring the scope of post-intensive care syndrome therapy 
and care: engagement of non-critical care providers and survivors in a second stakeholders meeting. 
Critical Care Medicine. 2014;42(12):2518-2526. 
2. 
Needham DM, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from 
intensive care unit: report from a stakeholders' conference. Critical Care Medicine. 2012;40(2):502-509. 
3. 
Vasilevskis EE, Ely EW, Speroff T, Pun BT, Boehm L, Dittus RS. Reducing Iatrogenic Risks: ICU-
Acquired Delirium and Weakness--Crossing the Quality Chasm. Chest. 2010;138(5):1224-1233. 
4. 
Morandi A, Brummel NE, Ely EW. Sedation, delirium and mechanical ventilation: the 'ABCDE' 
approach. Current Opinion In Critical Care. 2011;17(1):43-49. 
5. 
Pandharipande P, Banerjee A, McGrane S, Ely EW. Liberation and animation for ventilated ICU 
patients: the ABCDE bundle for the back-end of critical care. Critical Care (London, England). 
2010;14(3):157-157. 
6. 
Ely EW. The ABCDEF Bundle: Science and Philosophy of How ICU Liberation Serves Patients and 
Families. Crit Care Med. 2017;45(2):321-330. 
7. 
Barr J, Fraser GL, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, 
and delirium in adult patients in the intensive care unit. Critical Care Medicine. 2013;41(1):263-306. 
8. 
Milbrandt EB, Deppen S, Harrison PL, et al. Costs associated with delirium in mechanically ventilated 
patients. Critical Care Medicine. 2004;32(4):955-962. 
9. 
Shehabi Y, Riker RR, Bokesch PM, Wisemandle W, Shintani A, Ely EW. Delirium duration and mortality 
in lightly sedated, mechanically ventilated intensive care patients. Critical Care Medicine. 
2010;38(12):2311-2318. 
10. 
Pisani MA, Kong SY, Kasl SV, Murphy TE, Araujo KL, Van Ness PH. Days of delirium are associated 
with 1-year mortality in an older intensive care unit population. American Journal of Respiratory & 
Critical Care Medicine. 2009;180(11):1092-1097. 
11. 
Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated 
patients in the intensive care unit. JAMA: The Journal Of The American Medical Association. 
2004;291(14):1753-1762. 
12. 
Ely EW, Gautam S, Margolin R, et al. The impact of delirium in the intensive care unit on hospital length 
of stay. Intensive care medicine. 2001;27(12):1892-1900. 
13. 
Girard TD, Jackson JC, Pandharipande PP, et al. Delirium as a predictor of long-term cognitive 
impairment in survivors of critical illness. Critical Care Medicine. 2010;38(7):1513-1520. 
14. 
Balas MC, Happ MB, Yang W, Chelluri L, Richmond T. Outcomes Associated With Delirium in Older 
Patients in Surgical ICUs. Chest. 2009;135(1):18-25. 
15. 
Zhang Z, Pan L, Ni H. Impact of delirium on clinical outcome in critically ill patients: a meta-analysis. 
General hospital psychiatry. 2013;35(2):105-111. 
16. 
Granja C, Gomes E, Amaro A, et al. Understanding posttraumatic stress disorder-related symptoms 
after critical care: the early illness amnesia hypothesis. Critical Care Medicine. 2008;36(10):2801-2809. 
17. 
Schelling G, Stoll C, Haller M, et al. Health-related quality of life and posttraumatic stress disorder in 
survivors of the acute respiratory distress syndrome. Critical Care Medicine. 1998;26(4):651-659. 
18. 
Castillo MI, Cooke ML, Macfarlane B, Aitken nM. Trait Anxiety But Not State Anxiety During Critical 
Illness Was Associated With Anxiety and Depression Over 6 Months After ICU. Critical Care Medicine. 
2016;44(1):100-110. 
19. 
Pandharipande PP, Girard TD, Jackson JC, et al. Long-term cognitive impairment after critical illness. 
The New England journal of medicine. 2013;369(14):1306-1316. 
20. 
Witlox J, Eurelings LSM, De Jonghe JFM, Kalisvaart KJ, Eikelenboom P, Van Gool WA. Delirium in 
Elderly Patients and the Risk of Postdischarge Mortality, Institutionalization, and Dementia: A Meta-
analysis (English). JAMA, the journal of the American Medical Association. 2010;304(4):443-451. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
19 
 
21. 
Wunsch H, Angus DC, Harrison DA, et al. Comparison of medical admissions to intensive care units in 
the United States and United Kingdom. American Journal of Respiratory & Critical Care Medicine. 
2011;183(12):1666-1673. 
22. 
Wunsch H, Wagner J, Herlim M, Chong DH, Kramer AA, Halpern SD. ICU occupancy and mechanical 
ventilator use in the United States. Critical Care Medicine. 2013;41(12):2712-2719. 
23. 
Barrett ML, Smith, M.W., Elixhauserr, A., Hongiman, L.S., Pines, J.M. Utilization of Intensive Care 
Services, 2011. HCUP Statistical Brief #185. In: Quality AfHRa. Rockville, MD.2014. 
24. 
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability 
among survivors of severe sepsis. JAMA. 2010;304(16):1787-1794. 
25. 
Needham DM, Feldman DR, Kho ME. The functional costs of ICU survivorship. Collaborating to 
improve post-ICU disability. Am J Respir Crit Care Med. Vol 183. United States2011. 
26. 
Marra A. The ABCDEF Bundle in Critical Care. Critical care clinics.33(2):225-243. 
27. 
Barnes-Daly MB, Pun, B.T., Harmon, L.A., Byrum, D.G., Kumar, V.K., Devlin, J.W., Stollings, J.L., 
Puntillo, K.A., Engel, H.J., Posa, P.J., Barr, J., Schweickert, W.D., Esbrook, C.L., Hargett, K.D., Carson, 
S.S., Aldrich, M., Ely, E.W., Balas, M.C. Improving Health Care for Critically Ill Patients Using an 
Evidence-Based Collaborative Approach to ABCDEF Bundle Dissemination and Implementation. 
Worldviews on Evidence-Based Nursing. in-press. 
28. 
Balas MC, Vasilevskis EE, Olsen KM, et al. Effectiveness and safety of the awakening and breathing 
coordination, delirium monitoring/management, and early exercise/mobility bundle*. Critical Care 
Medicine. 2014;42(5):1024-1036. 
29. 
Barnes-Daly MA, Phillips G, Ely EW. Improving Hospital Survival and Reducing Brain Dysfunction at 
Seven California Community Hospitals: Implementing PAD Guidelines Via the ABCDEF Bundle in 
6,064 Patients. Critical Care Medicine. 2017;45(2):171-178. 
30. 
Kram SL, DiBartolo MC, Hinderer K, Jones RA. Implementation of the ABCDE Bundle to Improve 
Patient Outcomes in the Intensive Care Unit in a Rural Community Hospital. Dimensions of Critical 
Care Nursing. 2015;34(5):250-258 259p. 
31. 
Spronk PE, Riekerk B, Hofhuis J, Rommes JH. Occurrence of delirium is severely underestimated in 
the ICU during daily care. Intensive care medicine. 2009;35(7):1276-1280. 
32. 
van Eijk MM, van Marum RJ, Klijn IA, de Wit N, Kesecioglu J, Slooter AJ. Comparison of delirium 
assessment tools in a mixed intensive care unit. Crit Care Med. 2009;37(6):1881-1885. 
33. 
Baggs JG, Schmitt MH, Mushlin AI, et al. Association between nurse-physician collaboration and 
patient outcomes in three intensive care units. Critical Care Medicine. 1999;27(9):1991-1998. 
34. 
Bakhru RN, Wiebe, D.J., McWilliams, D., Spuhler, V.J. Schweickert, W.D. An International Survey of 
Early Mobilization Practices. American Thoracic Society Abstract2014. 
35. 
Balas MC, Burke WJ, Gannon D, et al. Implementing the Awakening and Breathing Coordination, 
Delirium Monitoring/Management, and Early Exercise/Mobility Bundle into Everyday Care: 
Opportunities, Challenges, and Lessons Learned for Implementing the ICU Pain, Agitation, and 
Delirium Guidelines. Critical Care Medicine. 2013:S116-S127. 
36. 
Carrothers KM, Barr J, Spurlock B, Ridgely MS, Damberg CL, Ely EW. Contextual Issues Influencing 
Implementation and Outcomes Associated With an Integrated Approach to Managing Pain, Agitation, 
and Delirium in Adult ICUs2013. 
37. 
Connolly B, O'Neill B, Salisbury L, Blackwood B. Physical rehabilitation interventions for adult patients 
during critical illness: an overview of systematic reviews. Thorax. 2016;71(10):881-890. 
38. 
Dubb R, Nydahl P, Hermes C, et al. Barriers and Strategies for Early Mobilization of Patients in 
Intensive Care Units. Annals of the American Thoracic Society. 2016;13(5):724-730. 
39. 
Laurent H, Aubreton S, Richard R, et al. Systematic review of early exercise in intensive care: A 
qualitative approach. Anaesthesia Critical Care & Pain Medicine. 2016;35(2):133-149. 
40. 
Miller MA, Govindan S, Watson SR, Hyzy RC, Iwashyna TJ. ABCDE, but in that order? A cross-
sectional survey of Michigan intensive care unit sedation, delirium, and early mobility practices. Annals 
of the American Thoracic Society. 2015;12(7):1066-1071. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
20 
 
41. 
Nydahl P, Parker Ruhl A, Bartoszek G, et al. Early Mobilization of Mechanically Ventilated Patients: A 1 
-Day Point-Prevalence Study in Germany. Critical Care Medicine. 2014;42(5):1178-1186. 
42. 
Davidson JE, Aslakson RA, Long AC, et al. Guidelines for Family-Centered Care in the Neonatal, 
Pediatric, and Adult ICU. Crit Care Med. 2017;45(1):103-128. 
43. 
Ouellette DR, Patel S, Girard TD, et al. Liberation From Mechanical Ventilation in Critically Ill Adults: 
An Official American College of Chest Physicians/American Thoracic Society Clinical Practice 
Guideline: Inspiratory Pressure Augmentation During Spontaneous Breathing Trials, Protocols 
Minimizing Sedation, and Noninvasive Ventilation Immediately After Extubation. CHEST. 
2017;151(1):166-180. 
44. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of 
health services research findings into practice: a consolidated framework for advancing implementation 
science. Implementation Science: IS. 2009;4:50-50. 
45. 
Wunsch H, Linde-Zwirble W, Angus DC, Hartman ME, Milbrandt EB, Kahn JM. The epidemiology of 
mechanical ventilation use in the United States. Critical Care Medicine. 2010;38(10):1947-1953. 
46. 
Carson SS, Cox CE, Holmes GM, Howard A, Carey TS. The changing epidemiology of mechanical 
ventilation: a population-based study. Journal of Intensive Care Medicine (Sage Publications Inc.). 
2006;21(3):173-182. 
47. 
Mehta AB, Syeda SN, Wiener RS, Walkey AJ. Epidemiological trends in invasive mechanical ventilation 
in the United States: A population-based study. J Crit Care. 2015;30(6):1217-1221. 
48. 
Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population Burden of Long-Term Survivorship After 
Severe Sepsis in Older Americans. Journal of the American Geriatrics Society. 2012;60(6):1070-1077. 
49. 
Iwashyna TJ. Survivorship will be the defining challenge of critical care in the 21st century. Ann Intern 
Med. Vol 153. United States2010. 
50. 
Puntillo KA. Pain experiences of intensive care unit patients. Heart & Lung: The Journal Of Critical 
Care. 1990;19(5 Pt 1):526-533. 
51. 
Gélinas C. Management of pain in cardiac surgery ICU patients: have we improved over time? 
Intensive & Critical Care Nursing. 2007;23(5):298-303. 
52. 
Puntillo KA, White C, Morris AB, et al. Patients' perceptions and responses to procedural pain: results 
from Thunder Project II. American Journal of Critical Care. 2001;10(4):238-251 214p. 
53. 
Baumgarten M, Poulsen I. Patients' experiences of being mechanically ventilated in an ICU: a 
qualitative metasynthesis. Scandinavian Journal of Caring Sciences. 2015;29(2):205-214. 
54. 
Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and 
reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA: The 
Journal Of The American Medical Association. 2001;286(21):2703-2710. 
55. 
Ely EW, Margolin R, Francis J, et al. Evaluation of delirium in critically ill patients: validation of the 
Confusion Assessment Method for the Intensive Care Unit (CAM-ICU). Critical Care Medicine. 
2001;29(7):1370-1379. 
56. 
Denehy L, Lanphere J, Needham DM. Ten reasons why ICU patients should be mobilized early. 
Intensive Care Med. 2017;43(1):86-90. 
57. 
Fan E, Dowdy DW, Colantuoni E, et al. Physical complications in acute lung injury survivors: a two-year 
longitudinal prospective study. Crit Care Med. 2014;42(4):849-859. 
58. 
Needham DM, Wozniak AW, Hough CL, et al. Risk factors for physical impairment after acute lung 
injury in a national, multicenter study. Am J Respir Crit Care Med. 2014;189(10):1214-1224. 
59. 
Jones J, Hoggart B, Withey J, Donaghue K, Ellis BW. What the patients say: A study of reactions to an 
intensive care unit. Intensive Care Medicine. 1979;5(2):89-92. 
60. 
Brummel NE, Jackson JC, Pandharipande PP, et al. Delirium in the ICU and Subsequent Long-Term 
Disability Among Survivors of Mechanical Ventilation. Critical Care Medicine. 2013. 
61. 
Herridge MS, Cheung AM, Tansey CM, et al. One-year outcomes in survivors of the acute respiratory 
distress syndrome. New England Journal of Medicine. 2003;348(8):683-693. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
21 
 
62. 
Herridge MS, Tansey CM, Matté A, et al. Functional disability 5 years after acute respiratory distress 
syndrome. The New England Journal of Medicine. 2011;364(14):1293-1304. 
63. 
L M, MA P. One-year outcomes following delirium in older ICU patients. J Am Ger Society. 2004;S2. 
64. 
Jackson JC, Gordon SM, Hart RP, Hopkins RO, Ely EW. The association between delirium and 
cognitive decline: a review of the empirical literature. Neuropsychology review. 2004;14(2):87-98. 
65. 
Wolters AE, Peelen LM, Veldhuijzen DS, et al. Long-Term Self-Reported Cognitive Problems After 
Delirium in the Intensive Care Unit and the Effect of Systemic Inflammation. J Am Geriatr Soc. 
2017;65(4):786-791. 
66. 
Girard TD, Kress JP, Fuchs BD, et al. Efficacy and safety of a paired sedation and ventilator weaning 
protocol for mechanically ventilated patients in intensive care (Awakening and Breathing Controlled 
trial): a randomised controlled trial. Lancet. 2008;371(9607):126-134. 
67. 
Kress JP, Pohlman AS, O'Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill 
patients undergoing mechanical ventilation. New England Journal of Medicine. 2000;342(20):1471-
1477. 
68. 
Shehabi Y, Bellomo R, Reade MC, et al. Early intensive care sedation predicts long-term mortality in 
ventilated critically ill patients. American Journal of Respiratory & Critical Care Medicine. 
2012;186(8):724-731. 
69. 
Tanaka LM, Azevedo LC, Park M, et al. Early sedation and clinical outcomes of mechanically ventilated 
patients: a prospective multicenter cohort study. Crit Care. 2014;18(4):R156. 
70. 
Balzer F, Weiss B, Kumpf O, et al. Early deep sedation is associated with decreased in-hospital and 
two-year follow-up survival. Crit Care. 2015;19:197. 
71. 
Pandharipande P, Shintani A, Peterson J, et al. Lorazepam is an independent risk factor for 
transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104(1):21-26. 
72. 
Kollef MH, Levy NT, Ahrens TS, Schaiff R, Prentice D, Sherman G. The use of continuous i.v. sedation 
is associated with prolongation of mechanical ventilation. Chest. 1998;114(2):541-548. 
73. 
Pandharipande PP, Sanders RD, Girard TD, et al. Effect of dexmedetomidine versus lorazepam on 
outcome in patients with sepsis: an a priori-designed analysis of the MENDS randomized controlled 
trial. Crit Care. 2010;14(2):R38. 
74. 
Pandharipande PP, Pun BT, Herr DL, et al. Effect of sedation with dexmedetomidine vs lorazepam on 
acute brain dysfunction in mechanically ventilated patients: the MENDS randomized controlled trial. 
JAMA: The Journal Of The American Medical Association. 2007;298(22):2644-2653. 
75. 
Riker RR, Shehabi Y, Bokesch PM, et al. Dexmedetomidine vs midazolam for sedation of critically ill 
patients: a randomized trial. Jama. 2009;301(5):489-499. 
76. 
Ely EW, Baker AM, Dunagan DP, et al. Effect on the duration of mechanical ventilation of identifying 
patients capable of breathing spontaneously. The New England journal of medicine. 
1996;335(25):1864-1869. 
77. 
Schweickert WD, Pohlman MC, Pohlman AS, et al. Early physical and occupational therapy in 
mechanically ventilated, critically ill patients: a randomised controlled trial. Lancet. 
2009;373(9678):1874-1882. 
78. 
Payen JF, Chanques G, Mantz J, et al. Current practices in sedation and analgesia for mechanically 
ventilated critically ill patients: a prospective multicenter patient-based study. Anesthesiology. 
2007;106(4):687-695; quiz 891-682. 
79. 
Elliott D, Aitken LM, Bucknall TK, et al. Patient comfort in the intensive care unit: a multicentre, 
binational point prevalence study of analgesia, sedation and delirium management. Crit Care Resusc. 
2013;15(3):213-219. 
80. 
Chawla R, Myatra SN, Ramakrishnan N, Todi S, Kansal S, Dash SK. Current practices of mobilization, 
analgesia, relaxants and sedation in Indian ICUs: A survey conducted by the Indian Society of Critical 
Care Medicine. Indian J Crit Care Med. 2014;18(9):575-584. 
81. 
Elliott R, McKinley S, Aitken L. Adoption of a sedation scoring system and sedation guideline in an 
intensive care unit. Journal of advanced nursing. 2006;54(2):208-216. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
22 
 
82. 
Pisani MA, Murphy TE, Araujo KL, Slattum P, Van Ness PH, Inouye SK. Benzodiazepine and opioid 
use and the duration of intensive care unit delirium in an older population. Critical care medicine. 
2009;37(1):177-183. 
83. 
Bakhru RN, McWilliams DJ, Wiebe DJ, Spuhler VJ, Schweickert WD. Intensive Care Unit Structure 
Variation and Implications for Early Mobilization Practices. An International Survey. Annals of the 
American Thoracic Society. 2016;13(9):1527-1537. 
84. 
Luetz A, Balzer F, Radtke FM, et al. Delirium, sedation and analgesia in the intensive care unit: a 
multinational, two-part survey among intensivists. PLoS One. 2014;9(11):e110935. 
85. 
Berney SC, Harrold M, Webb SA, et al. Intensive care unit mobility practices in Australia and New 
Zealand: a point prevalence study. Crit Care Resusc. 2013;15(4):260-265. 
86. 
Jolley SE, Moss M, Needham DM, et al. Point Prevalence Study of Mobilization Practices for Acute 
Respiratory Failure Patients in the United States. Crit Care Med. 2017;45(2):205-215. 
87. 
Kahn JM. Intensivist physician staffing and the process of care in academic medical centres. Qual Saf 
Health Care. 2007;16(5):329. 
88. 
Ramos FJ, Fumis RR, de Azevedo LC, Schettino G. Intensive care unit visitation policies in Brazil: a 
multicenter survey. Rev Bras Ter Intensiva. 2014;26(4):339-346. 
89. 
Garrouste-Orgeas M, Vinatier I, Tabah A, Misset B, Timsit JF. Reappraisal of visiting policies and 
procedures of patient's family information in 188 French ICUs: a report of the Outcomerea Research 
Group. Ann Intensive Care. 2016;6(1):82. 
90. 
Morandi A, Piva S, Ely EW, et al. Worldwide Survey of the "Assessing Pain, Both Spontaneous 
Awakening and Breathing Trials, Choice of Drugs, Delirium Monitoring/Management, Early 
Exercise/Mobility, and Family Empowerment" (ABCDEF) Bundle. Crit Care Med. 2017;45(11):e1111-
e1122. 
91. 
Balas MC, Vasilevskis EE, Burke WJ, et al. Critical care nurses' role in implementing the "ABCDE 
bundle" into practice. Critical Care Nurse. 2012;32(2):35. 
92. 
Balas MC, Devlin JW, Verceles AC, Morris P, Wesley Ely E, Ely EW. Adapting the ABCDEF Bundle to 
Meet the Needs of Patients Requiring Prolonged Mechanical Ventilation in the Long-Term Acute Care 
Hospital Setting: Historical Perspectives and Practical Implications. Seminars in Respiratory & Critical 
Care Medicine. 2016;37(1):119-135. 
93. 
Rhodes A, Phillips G, Beale R, et al. The Surviving Sepsis Campaign bundles and outcome: results 
from the International Multicentre Prevalence Study on Sepsis (the IMPreSS study). Intensive Care 
Medicine. 2015;41(9):1620-1628. 
94. 
Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related 
bloodstream infections in the ICU. New England Journal of Medicine. 2006;355(26):2725. 
95. 
Marcantonio ER, Flacker JM, Wright RJ, Resnick NM. Reducing delirium after hip fracture: a 
randomized trial. Journal of the American Geriatrics Society. 2001;49(5):516-522. 
96. 
Martinez FT, Tobar C, Beddings CI, Vallejo G, Fuentes P. Preventing delirium in an acute hospital 
using a non-pharmacological intervention. Age & Ageing. 2012;41(5):629-634. 
97. 
Inouye SK, Bogardus ST, Charpentier PA, et al. A multicomponent intervention to prevent delirium in 
hospitalized older patients. N Engl J Med. 1999;340(9):669-676. 
98. 
Klompas M, Anderson D, Trick W, et al. The Preventability of Ventilator-associated Events The CDC 
Prevention Epicenters Wake Up and Breathe Collaborative. American Journal of Respiratory and 
Critical Care Medicine. 2015;191(3):292-301. 
99. 
Payen J, Bru O, Bosson J, et al. Assessing pain in critically ill sedated patients by using a behavioral 
pain scale. Critical Care Medicine. 2001;29(12):2258-2263. 
100. 
Gélinas C, Fillion L, Puntillo KA, Viens C, Fortier M. Validation of the critical-care pain observation tool 
in adult patients. American Journal of Critical Care. 2006;15(4):420-427. 
101. 
Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: validity and reliability 
in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338-1344. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
23 
 
102. 
Riker RR, Picard JT, Fraser GL. Prospective evaluation of the Sedation-Agitation Scale for adult 
critically ill patients. Critical Care Medicine. 1999;27(7):1325-1329. 
103. 
Bergeron N, Dubois MJ, Dumont M, Dial S, Skrobik Y. Intensive Care Delirium Screening Checklist: 
evaluation of a new screening tool. Intensive care medicine. 2001;27(5):859-864. 
104. 
Boehm LM. ORGANIZATIONAL DOMAINS AND VARIATION IN ATTITUDES OF INTENSIVE CARE 
PROVIDERS TOWARD THE ABCDE BUNDLE. American Journal of Critical Care. 2017;26(3):e18-e26. 
105. 
Bassett R, Adams KM, Danesh V, et al. Rethinking critical care: decreasing sedation, increasing 
delirium monitoring, and increasing patient mobility. Joint Commission Journal On Quality And Patient 
Safety. 2015;41(2):62-74. 
106. 
Boehm LM, Vasilevskis EE, Mion LC. Interprofessional Perspectives on ABCDE Bundle 
Implementation. Dimensions of Critical Care Nursing. 2016;35(6):339-347. 
107. 
Costa DK, Valley TS, Miller MA, et al. ICU team composition and its association with ABCDE 
implementation in a quality collaborative. Journal Of Critical Care. 2017;44:1-6. 
108. 
Costa DK, White M, Ginier E, et al. Identifying barriers to delivering the ABCDE bundle to minimize 
adverse outcomes for mechanically ventilated patients: A systematic review. Chest. 2017. 
109. 
Morandi A, Piva S, Ely EW, et al. Worldwide Survey of the "Assessing Pain, Both Spontaneous 
Awakening and Breathing Trials, Choice of Drugs, Delirium Monitoring/Management, Early 
Exercise/Mobility, and Family Empowerment" (ABCDEF) Bundle. Critical Care Medicine. 
2017;45(11):e1111-e1122. 
110. 
Halpern NA, Pastores SM. Critical care medicine in the United States 2000-2005: an analysis of bed 
numbers, occupancy rates, payer mix, and costs. Critical Care Medicine. 2010;38(1):65-71. 
111. 
Deutschman CS, Ahrens T, Cairns CB, Sessler CN, Parsons PE. Multisociety task force for critical care 
research: key issues and recommendations. American Journal Of Respiratory And Critical Care 
Medicine. 2012;185(1):96-102. 
112. 
Tomson CRV, Van Der Veer SN. Learning from practice variation to improve the quality of care. Clinical 
Medicine. 2013;13(1):19-23. 
113. 
Matlock DD, Groeneveld PW, Sidney S, et al. Geographic variation in cardiovascular procedure use 
among Medicare fee-for-service vs Medicare Advantage beneficiaries. JAMA: Journal of the American 
Medical Association. 2013;310(2):155-162. 
114. 
Tsai TC, Joynt KE, Orav EJ, Gawande AA, Jha AK. Variation in surgical-readmission rates and quality 
of hospital care. New England Journal of Medicine. 2013;369(12):1134-1142. 
115. 
Jaramillo E, Tan A, Yang L, Kuo Y-F, Goodwin JS. Variation among primary care physicians in 
prostate-specific antigen screening of older men. JAMA: Journal of the American Medical Association. 
2013;310(15):1622-1624. 
116. 
Backes Kozhlmannil K, Law MR, Virnig BA. Cesarean Delivery Rates Vary Tenfold Among US 
Hospitals; Reducing Variation May Address Quality And Cost Issues. Health Affairs. 2013;32(3):527-
535. 
117. 
Nyberg L, Gustafson Y, Janson A, Sandman PO, Eriksson S. Incidence of falls in three different types 
of geriatric care. A Swedish prospective study. Scandinavian Journal Of Social Medicine. 1997;25(1):8-
13. 
118. 
Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of Interprofessional Team Collaboration 
Scale (AITCS): development and testing of the instrument. The Journal Of Continuing Education In The 
Health Professions. 2012;32(1):58-67. 
119. 
Nurses AAoCC. AACN Standards for Establishing and Sustaining Healthy Work Environments: A 
Journey to Excellence. 2005:http://www.aacn.org/wd/hwe/docs/execsum.pdf. 
120. 
Gill KV, Voils SA, Chenault GA, Brophy GM. Perceived versus actual sedation practices in adult 
intensive care unit patients receiving mechanical ventilation. Annals of Pharmacotherapy. 
2012;46(10):1331-1339. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
24 
 
121. 
Gude WT, Roos-Blom M-J, van der Veer SN, et al. Health professionals' perceptions about their clinical 
performance and the influence of audit and feedback on their intentions to improve practice: a theory-
based study in Dutch intensive care units. Implementation Science: IS. 2018;13(1):33-33. 
122. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual 
distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-
76. 
123. 
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification 
system. Critical care medicine. 1985;13(10):818-829. 
124. 
Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of 
hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619-1636. 
125. 
Zimmerman, J.E., Kramer, A.A., McNair, D.S., Malila, F.M.(2006). Acute Physiology and Chronic Health 
Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients Critical Care 
Medicine. 34 (5), 1297-1310.  
126. 
Zhang F, Wagner AK, Roos-Degnan D (2011) Simulation-based power calculation for designing 
interrupted time series analyses of health policy interventions. Journal of Clinical Epidemiology. 64: 
1252-1261 
127. 
Sakr Y, Moreira CL, Rhodes A, et al. The impact of hospital and ICU organizational factors on outcome 
in critically ill patients: results from the Extended Prevalence of Infection in Intensive Care study. Critical 
Care Medicine. 2015;43(3):519-526. 
128. 
Checkley W, Martin GS, Brown SM, et al. Structure, process, and annual ICU mortality across 69 
centers: United States critical illness and injury trials group critical illness outcomes study. Critical Care 
Medicine. 2014;42(2):344-356. 
129. 
Bravo G, Potwin L. Estimating the reliability of continuous measures with Cronbach’s alpha or the 
intraclass correlation coefficient: toward the integration of two traditions. J Clin Epidemol 1991; 44(4-5): 
381-90 
130. 
Carmines EG, Zeller RA. Reliability and validity assessment. Newbury Park, Calif: Sage Publications; 
1979. 
131. 
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results 
from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 
2015;10(1):1-14. 
132. 
Burke JK, O’Campo P, Peak GL, Gielen AC, McDonnel KA, Trochim WMK: An introduction to concept 
mapping as a participatory public health research method. Qual Health Res. 2005, 15: 1392-1410. 
10.1177/1049732305278876. 
133. 
Concept Systems. The concept system. n.d. Retrieved from 
http://www.conceptsystems.com/content/view/the-concept-system.html  
134. 
Trochim WMK, Kane M. Concept mapping: An introduction to structured conceptualization in health 
care. Int J Qual Health Care. 2005, 17: 187-191. 10.1093/intqhc/mzi038.View ArticlePubMedGoogle 
Scholar 
135. 
Brownson RC, Kelly CM, Eyler AA, Carnoske C, Grost L, Handy SL, Maddock JE, Pluto D, Ritacco BA, 
Sallis JF, Schmid TL. Environmental and policy approaches for promoting physical activity in the United 
States: A research agenda. J Phys Act Health. 2008, 5: 488-503.PubMedGoogle Scholar 
136. 
Green AE, Aarons GA. A comparison of policy and direct practice stakeholder perceptions of factors 
affecting evidence-based practice implementation using concept mapping. Implement Sci. 2011, 6: 1-
12. 10.1186/1748-5908-6-1. 
137. 
Trochim WMK. The reliability of concept mapping. 1993, Dallas, TX 
 
 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
25 
 
 
APPENDIX A.  Focus Group Interview Guide 
 
We are going to begin by asking you to think about a patient you recently cared for on you unit who required 
mechanical ventilation.  We are specifically interested in learning about the decisions you and your team made 
regarding SATs, SBTs, and early mobility. 
Please focus on whether or not you were able to actually conduct an awakening trial, a spontaneous breathing 
trial and mobility.  What were some the factors you all considered in implementing spontaneous awakening trials, 
spontaneous breathing trials, and early mobility/ exercise with this patient?  
 
Take a few minutes as a group to write down perhaps in bullet points or full sentences, if you like, the de-
identified (no names) story of a single day in caring for the patient. It doesn’t have to be a perfect case – just 
one that is typical or that you’d really like to talk about.  
 
[allow 10 minutes for group discussion] 
 
Select a member to begin telling the story. Others are encouraged to add. 
 
Tell me about the considerations you had and decisions that you made regarding SAT for this patient?  
• 
What conflicts did you encounter?  
• 
What tradeoffs did you make or think about? 
 
Tell me the considerations you had and decisions that you made regarding SBT for this patient? 
• 
What conflicts did you encounter?  
• 
What tradeoffs did make or think about? 
 
Tell me the considerations you had and decisions that you made regarding  early mobility  for this patient? 
• 
What conflicts did you encounter?  
• 
What tradeoffs did make or think about?  
 
Can you describe interactions with the patient about the SAT, SBT, and early mobility  procedures? 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
26 
 
• 
How did you inform the patient?  
• 
What methods of communication are used with MV patients during bundle implementation? 
• 
Is this the same or different for other interactions in the care of patients in your unit? 
 
What input did the patient have in the process?  
• 
How would you describe the patient’s preferences about A, B, and E bundle procedures? 
• 
How did the patient respond?  
• 
In what ways did you use the patient’s preferences and responses in decisions about beginning, 
continuing or stopping the A, B, or E bundle procedures? 
o How did you weigh the patient’s responses in A, B, and E? 
 
How does this patient compare with other typical patients on your unit? (ascertain if this is an extreme case) 
[probe for a typical or contrasting case] 
 
 
 
1. 
Elliott D, Davidson JE, Harvey MA, et al. Exploring the scope of post–intensive care syndrome therapy and care: 
engagement of non–critical care providers and survivors in a second stakeholders meeting. Critical care medicine 
2014;42:2518-26. 
2. 
Needham DM, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from intensive care 
unit: report from a stakeholders' conference. Critical care medicine 2012;40:502-9. 
3. 
Vasilevskis EE, Ely EW, Speroff T, Pun BT, Boehm L, Dittus RS. Reducing iatrogenic risks: ICU-acquired delirium 
and weakness—crossing the quality chasm. Chest 2010;138:1224-33. 
4. 
Morandi A, Brummel NE, Ely EW. Sedation, delirium and mechanical ventilation: the ‘ABCDE’approach. Current 
opinion in critical care 2011;17:43-9. 
5. 
Pandharipande P, Banerjee A, McGrane S, Ely EW. Liberation and animation for ventilated ICU patients: the 
ABCDE bundle for the back-end of critical care. Critical Care 2010;14:157. 
6. 
Ely EW. The ABCDEF bundle: science and philosophy of how ICU liberation serves patients and families. Critical 
care medicine 2017;45:321. 
7. 
Barr J, Fraser GL, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, and delirium 
in adult patients in the intensive care unit: executive summary. American Journal of Health-System Pharmacy 
2013;70:53-8. 
8. 
Milbrandt EB, Deppen S, Harrison PL, et al. Costs associated with delirium in mechanically ventilated patients. 
Critical care medicine 2004;32:955-62. 
9. 
Shehabi Y, Riker RR, Bokesch PM, Wisemandle W, Shintani A, Ely EW. Delirium duration and mortality in lightly 
sedated, mechanically ventilated intensive care patients. Critical care medicine 2010;38:2311-8. 
10. 
Pisani MA, Kong SYJ, Kasl SV, Murphy TE, Araujo KL, Van Ness PH. Days of delirium are associated with 1-year 
mortality in an older intensive care unit population. American journal of respiratory and critical care medicine 
2009;180:1092-7. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
27 
 
11. 
Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the 
intensive care unit. Jama 2004;291:1753-62. 
12. 
Ely E, Gautam S, Margolin R, et al. The impact of delirium in the intensive care unit on hospital length of stay. 
Intensive care medicine 2001;27:1892-900. 
13. 
Girard TD, Jackson JC, Pandharipande PP, et al. Delirium as a predictor of long-term cognitive impairment in 
survivors of critical illness. Critical care medicine 2010;38:1513. 
14. 
Balas MC, Happ MB, Yang W, Chelluri L, Richmond T. Outcomes associated with delirium in older patients in 
surgical ICUs. Chest 2009;135:18-25. 
15. 
Zhang Z, Pan L, Ni H. Impact of delirium on clinical outcome in critically ill patients: a meta-analysis. General 
hospital psychiatry 2013;35:105-11. 
16. 
Granja C, Gomes E, Amaro A, et al. Understanding posttraumatic stress disorder-related symptoms after critical 
care: the early illness amnesia hypothesis. Critical care medicine 2008;36:2801-9. 
17. 
Schelling G, Stoll C, Haller M, et al. Health-related quality of life and posttraumatic stress disorder in survivors of 
the acute respiratory distress syndrome. Critical care medicine 1998;26:651-9. 
18. 
Castillo MI, Cooke ML, Macfarlane B, Aitken LM. Trait anxiety but not state anxiety during critical illness was 
associated with anxiety and depression over 6 months after ICU. Critical care medicine 2016;44:100-10. 
19. 
Pandharipande PP, Girard TD, Jackson JC, et al. Long-term cognitive impairment after critical illness. New 
England Journal of Medicine 2013;369:1306-16. 
20. 
Witlox J, Eurelings LS, de Jonghe JF, Kalisvaart KJ, Eikelenboom P, Van Gool WA. Delirium in elderly patients and 
the risk of postdischarge mortality, institutionalization, and dementia: a meta-analysis. Jama 2010;304:443-51. 
21. 
Wunsch H, Angus DC, Harrison DA, Linde-Zwirble WT, Rowan KM. Comparison of medical admissions to 
intensive care units in the United States and United Kingdom. American journal of respiratory and critical care medicine 
2011;183:1666-73. 
22. 
Wunsch H, Wagner J, Herlim M, Chong D, Kramer A, Halpern SD. ICU occupancy and mechanical ventilator use in 
the United States. Critical care medicine 2013;41. 
23. 
Barrett ML, Smith MW, Elixhauser A, Honigman LS, Pines JM. Utilization of Intensive Care Services, 2011. HCUP 
Statistical Brief# 185. Agency for Healthcare Research and Quality, Rockville, MD 2014. 
24. 
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among 
survivors of severe sepsis. Jama 2010;304:1787-94. 
25. 
Needham DM, Feldman DR, Kho ME. The functional costs of ICU survivorship: collaborating to improve post-ICU 
disability. American Thoracic Society; 2011. 
26. 
Marra A, Ely EW, Pandharipande PP, Patel MB. The ABCDEF bundle in critical care. Critical care clinics 
2017;33:225-43. 
27. 
Barnes‐Daly MA, Pun BT, Harmon LA, et al. Improving health care for critically ill patients using an evidence‐
based collaborative approach to ABCDEF bundle dissemination and implementation. Worldviews on Evidence‐Based 
Nursing 2018;15:206-16. 
28. 
Balas MC, Vasilevskis EE, Olsen KM, et al. Effectiveness and safety of the awakening and breathing coordination, 
delirium monitoring/management, and early exercise/mobility (ABCDE) bundle. Critical care medicine 2014;42:1024. 
29. 
Barnes-Daly MA, Phillips G, Ely E. Improving hospital survival and reducing brain dysfunction at seven California 
community hospitals: implementing PAD guidelines via the ABCDEF bundle in 6,064 patients. Critical care medicine 
2017;45:171-8. 
30. 
Kram SL, DiBartolo MC, Hinderer K, Jones RA. Implementation of the ABCDE bundle to improve patient 
outcomes in the intensive care unit in a rural community hospital. Dimensions of Critical Care Nursing 2015;34:250-8. 
31. 
Spronk PE, Riekerk B, Hofhuis J, Rommes JH. Occurrence of delirium is severely underestimated in the ICU during 
daily care. Intensive care medicine 2009;35:1276-80. 
32. 
van Eijk MM, van Marum RJ, Klijn IA, de Wit N, Kesecioglu J, Slooter AJ. Comparison of delirium assessment tools 
in a mixed intensive care unit. Critical care medicine 2009;37:1881-5. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
28 
 
33. 
Baggs JG, Schmitt MH, Mushlin AI, et al. Association between nurse-physician collaboration and patient 
outcomes in three intensive care units. Critical care medicine 1999;27:1991-8. 
34. 
Bakhru RN, McWilliams D, Spuhler V, Schweickert WD. An International Survey Of Early Mobilization Practices.  
C15 CENTRAL NERVOUS SYSTEM AND MOTOR IMPAIRMENT IN CRITICAL ILLNESS: American Thoracic Society; 
2014:A3933-A. 
35. 
Balas MC, Burke WJ, Gannon D, et al. Implementing the awakening and breathing coordination, delirium 
monitoring/management, and early exercise/mobility bundle into everyday care: opportunities, challenges, and lessons 
learned for implementing the ICU Pain, Agitation, and Delirium Guidelines. Critical care medicine 2013;41:S116-S27. 
36. 
Carrothers KM, Barr J, Spurlock B, Ridgely MS, Damberg CL, Ely EW. Contextual issues influencing 
implementation and outcomes associated with an integrated approach to managing pain, agitation, and delirium in 
adult ICUs. Critical care medicine 2013;41:S128-S35. 
37. 
Connolly B, O'Neill B, Salisbury L, Blackwood B. Physical rehabilitation interventions for adult patients during 
critical illness: an overview of systematic reviews. Thorax 2016;71:881-90. 
38. 
Dubb R, Nydahl P, Hermes C, et al. Barriers and strategies for early mobilization of patients in intensive care 
units. Annals of the American Thoracic Society 2016;13:724-30. 
39. 
Laurent H, Aubreton S, Richard R, et al. Systematic review of early exercise in intensive care: A qualitative 
approach. Anaesthesia Critical Care & Pain Medicine 2016;35:133-49. 
40. 
Miller MA, Govindan S, Watson SR, Hyzy RC, Iwashyna TJ. ABCDE, but in that order? A cross-sectional survey of 
Michigan intensive care unit sedation, delirium, and early mobility practices. Annals of the American Thoracic Society 
2015;12:1066-71. 
41. 
Nydahl P, Ruhl AP, Bartoszek G, et al. Early mobilization of mechanically ventilated patients: a 1-day point-
prevalence study in Germany. Critical care medicine 2014;42:1178-86. 
42. 
Davidson JE, Aslakson RA, Long AC, et al. Guidelines for Family-Centered Care in the Neonatal, Pediatric, and 
Adult ICU. Crit Care Med 2017;45:103-28. 
43. 
Ouellette DR, Patel S, Girard TD, et al. Liberation From Mechanical Ventilation in Critically Ill Adults: An Official 
American College of Chest Physicians/American Thoracic Society Clinical Practice Guideline: Inspiratory Pressure 
Augmentation During Spontaneous Breathing Trials, Protocols Minimizing Sedation, and Noninvasive Ventilation 
Immediately After Extubation. Chest 2017;151:166-80. 
44. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health 
services research findings into practice: a consolidated framework for advancing implementation science. 
Implementation science 2009;4:50. 
45. 
Ferguson N, Laydon D, Nedjati Gilani G, et al. Report 9: Impact of non-pharmaceutical interventions (NPIs) to 
reduce COVID19 mortality and healthcare demand. 2020. 
46. 
Wunsch H, Linde-Zwirble WT, Angus DC, Hartman ME, Milbrandt EB, Kahn JM. The epidemiology of mechanical 
ventilation use in the United States. Critical care medicine 2010;38:1947-53. 
47. 
Carson SS, Cox CE, Holmes GM, Howard A, Carey TS. The changing epidemiology of mechanical ventilation: a 
population-based study. Journal of intensive care medicine 2006;21:173-82. 
48. 
Mehta AB, Syeda SN, Wiener RS, Walkey AJ. Epidemiological trends in invasive mechanical ventilation in the 
United States: a population-based study. Journal of critical care 2015;30:1217-21. 
49. 
Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population burden of long‐term survivorship after severe sepsis in 
older Americans. Journal of the American Geriatrics Society 2012;60:1070-7. 
50. 
Iwashyna TJ. Survivorship will be the defining challenge of critical care in the 21st century. Annals of internal 
medicine 2010;153:204-5. 
51. 
Puntillo K. Pain experiences of intensive care unit patients. Heart & lung: the journal of critical care 1990;19:526-
33. 
52. 
Gélinas C. Management of pain in cardiac surgery ICU patients: have we improved over time? Intensive and 
Critical Care Nursing 2007;23:298-303. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
29 
 
53. 
Puntillo KA, White C, Morris AB, Perdue ST. Patients' perceptions and responses to procedural pain: results from 
Thunder Project II. American Journal of Critical Care 2001;10:238. 
54. 
Baumgarten M, Poulsen I. Patients' experiences of being mechanically ventilated in an ICU: a qualitative 
metasynthesis. Scandinavian journal of caring sciences 2015;29:205-14. 
55. 
Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and reliability of the 
confusion assessment method for the intensive care unit (CAM-ICU). Jama 2001;286:2703-10. 
56. 
Ely EW, Margolin R, Francis J, et al. Evaluation of delirium in critically ill patients: validation of the Confusion 
Assessment Method for the Intensive Care Unit (CAM-ICU). Critical care medicine 2001;29:1370-9. 
57. 
Denehy L, Lanphere J, Needham DM. Ten reasons why ICU patients should be mobilized early. Intensive care 
medicine 2017;43:86-90. 
58. 
Fan E, Dowdy DW, Colantuoni E, et al. Physical complications in acute lung injury survivors: a 2-year longitudinal 
prospective study. Critical care medicine 2014;42:849. 
59. 
Needham DM, Wozniak AW, Hough CL, et al. Risk factors for physical impairment after acute lung injury in a 
national, multicenter study. American journal of respiratory and critical care medicine 2014;189:1214-24. 
60. 
Jones J, Hoggart B, Withey J, Donaghue K, Ellis B. What the patients say: A study of reactions to an intensive care 
unit. Intensive Care Medicine 1979;5:89-92. 
61. 
Brummel NE, Vasilevskis EE, Han JH, Boehm L, Pun BT, Ely EW. Implementing delirium screening in the intensive 
care unit: secrets to success. Critical care medicine 2013;41:2196. 
62. 
Herridge MS, Cheung AM, Tansey CM, et al. One-year outcomes in survivors of the acute respiratory distress 
syndrome. New England Journal of Medicine 2003;348:683-93. 
63. 
Herridge MS, Tansey CM, Matté A, et al. Functional disability 5 years after acute respiratory distress syndrome. 
New England Journal of Medicine 2011;364:1293-304. 
64. 
McNicoll L, Pisani M, Inouye S. One-year outcomes following delirium in older ICU patients. Journal of the 
American Geriatrics Society 2004;52. 
65. 
Jackson JC, Gordon SM, Hart RP, Hopkins RO, Ely EW. The association between delirium and cognitive decline: a 
review of the empirical literature. Neuropsychology review 2004;14:87-98. 
66. 
Wolters AE, Peelen LM, Veldhuijzen DS, et al. Long‐term self‐reported cognitive problems after delirium in the 
intensive care unit and the effect of systemic inflammation. Journal of the American Geriatrics Society 2017;65:786-91. 
67. 
Girard TD, Kress JP, Fuchs BD, et al. Efficacy and safety of a paired sedation and ventilator weaning protocol for 
mechanically ventilated patients in intensive care (Awakening and Breathing Controlled trial): a randomised controlled 
trial. The Lancet 2008;371:126-34. 
68. 
Kress JP, Pohlman AS, O'Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill patients 
undergoing mechanical ventilation. New England Journal of Medicine 2000;342:1471-7. 
69. 
Shehabi Y, Bellomo R, Reade MC, et al. Early intensive care sedation predicts long-term mortality in ventilated 
critically ill patients. American journal of respiratory and critical care medicine 2012;186:724-31. 
70. 
Tanaka LMS, Azevedo LCP, Park M, et al. Early sedation and clinical outcomes of mechanically ventilated 
patients: a prospective multicenter cohort study. Critical care 2014;18:R156. 
71. 
Balzer F, Weiß B, Kumpf O, et al. Early deep sedation is associated with decreased in-hospital and two-year 
follow-up survival. Critical care 2015;19:197. 
72. 
Pandharipande P, Shintani A, Peterson J, et al. Lorazepam is an independent risk factor for transitioning to 
delirium in intensive care unit patients. Anesthesiology: The Journal of the American Society of Anesthesiologists 
2006;104:21-6. 
73. 
Kollef MH, Levy NT, Ahrens TS, Schaiff R, Prentice D, Sherman G. The use of continuous iv sedation is associated 
with prolongation of mechanical ventilation. Chest 1998;114:541-8. 
74. 
Pandharipande PP, Sanders RD, Girard TD, et al. Effect of dexmedetomidine versus lorazepam on outcome in 
patients with sepsis: an a priori-designed analysis of the MENDS randomized controlled trial. Critical Care 2010;14:R38. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
30 
 
75. 
Pandharipande PP, Pun BT, Herr DL, et al. Effect of sedation with dexmedetomidine vs lorazepam on acute brain 
dysfunction in mechanically ventilated patients: the MENDS randomized controlled trial. Jama 2007;298:2644-53. 
76. 
Riker RR, Shehabi Y, Bokesch PM, et al. Dexmedetomidine vs midazolam for sedation of critically ill patients: a 
randomized trial. Jama 2009;301:489-99. 
77. 
Ely EW, Baker AM, Dunagan DP, et al. Effect on the duration of mechanical ventilation of identifying patients 
capable of breathing spontaneously. New England Journal of Medicine 1996;335:1864-9. 
78. 
Schweickert WD, Pohlman MC, Pohlman AS, et al. Early physical and occupational therapy in mechanically 
ventilated, critically ill patients: a randomised controlled trial. The Lancet 2009;373:1874-82. 
79. 
Payen J-F, Chanques G, Mantz J, et al. Current practices in sedation and analgesia for mechanically ventilated 
critically ill PatientsA prospective multicenter patient-based study. Anesthesiology: The Journal of the American Society 
of Anesthesiologists 2007;106:687-95. 
80. 
Green JV. Patient comfort in the intensive care unit: a multicentre, binational point prevalence study of 
analgesia, sedation and delirium management. Chest 2010;138:1224-33. 
81. 
Chawla R, Myatra SN, Ramakrishnan N, Todi S, Kansal S, Dash SK. Current practices of mobilization, analgesia, 
relaxants and sedation in Indian ICUs: A survey conducted by the Indian Society of Critical Care Medicine. Indian journal 
of critical care medicine: peer-reviewed, official publication of Indian Society of Critical Care Medicine 2014;18:575. 
82. 
Elliott R, McKinley S, Aitken L. Adoption of a sedation scoring system and sedation guideline in an intensive care 
unit. Journal of advanced nursing 2006;54:208-16. 
83. 
Pisani MA, Murphy TE, Araujo KL, Slattum P, Van Ness PH, Inouye SK. Benzodiazepine and opioid use and the 
duration of ICU delirium in an older population. Critical care medicine 2009;37:177. 
84. 
Bakhru RN, McWilliams DJ, Wiebe DJ, Spuhler VJ, Schweickert WD. Intensive care unit structure variation and 
implications for early mobilization practices. An international survey. Annals of the American Thoracic Society 
2016;13:1527-37. 
85. 
Luetz A, Balzer F, Radtke FM, et al. Delirium, sedation and analgesia in the intensive care unit: a multinational, 
two-part survey among intensivists. PloS one 2014;9. 
86. 
Berney S, Harrold M, Webb S, et al. Intensive care unit mobility practices in Australia and New Zealand: a point 
prevalence study. Critical Care and Resuscitation 2013;15:260-5. 
87. 
Jolley SE, Moss M, Needham DM, et al. Point prevalence study of mobilization practices for acute respiratory 
failure patients in the United States. Critical care medicine 2017;45:205. 
88. 
Kahn JM, Brake H, Steinberg KP. Intensivist physician staffing and the process of care in academic medical 
centres. BMJ Quality & Safety 2007;16:329-33. 
89. 
da Silva Ramos FJ, Fumis RRL, de Azevedo LCP, Schettino G. Intensive care unit visitation policies in Brazil: a 
multicenter survey. Revista Brasileira de terapia intensiva 2014;26:339. 
90. 
Garrouste-Orgeas M, Vinatier I, Tabah A, Misset B, Timsit J-F. Reappraisal of visiting policies and procedures of 
patient’s family information in 188 French ICUs: a report of the Outcomerea Research Group. Annals of intensive care 
2016;6:82. 
91. 
Morandi A, Piva S, Ely EW, et al. Worldwide ABCDEF (Assessing pain, both spontaneous awakening and breathing 
trials, choice of drugs, delirium monitoring/management, early exercise/mobility, and family empowerment) survey. 
Critical care medicine 2017;45:e1111. 
92. 
Balas MC, Vasilevskis EE, Burke WJ, et al. Critical care nurses’ role in implementing the “ABCDE bundle” into 
practice. Critical care nurse 2012;32:35-47. 
93. 
Balas MC, Devlin JW, Verceles AC, Morris P, Ely EW. Adapting the ABCDEF bundle to meet the needs of patients 
requiring prolonged mechanical ventilation in the long-term acute care hospital setting: historical perspectives and 
practical implications.  Seminars in respiratory and critical care medicine; 2016: Thieme Medical Publishers. p. 119-35. 
94. 
Rhodes A, Phillips G, Beale R, et al. The surviving sepsis campaign bundles and outcome: results from the 
international multicentre prevalence study on sepsis (the IMPreSS study). Intensive care medicine 2015;41:1620-8. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
31 
 
95. 
Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream 
infections in the ICU. New England Journal of Medicine 2006;355:2725-32. 
96. 
Marcantonio ER, Flacker JM, Wright RJ, Resnick NM. Reducing delirium after hip fracture: a randomized trial. 
Journal of the American Geriatrics Society 2001;49:516-22. 
97. 
Martinez FT, Tobar C, Beddings CI, Vallejo G, Fuentes P. Preventing delirium in an acute hospital using a non-
pharmacological intervention. Age and ageing 2012;41:629-34. 
98. 
Inouye SK, Bogardus Jr ST, Charpentier PA, et al. A multicomponent intervention to prevent delirium in 
hospitalized older patients. New England journal of medicine 1999;340:669-76. 
99. 
Hutton JL, Eccles MP, Grimshaw JM. Ethical issues in implementation research: a discussion of the problems in 
achieving informed consent. Implementation Science 2008;3:52. 
100. 
Guler S, Hurton S. C winn M, Molinari M. Levels in Decision Making and Techniques for Clinicians. International 
Journal of Digestive Diseases 2015;1:2. 
101. 
Rak KJ, Ashcraft LE, Kuza CC, et al. Effective Care Practices in Patients Receiving Prolonged Mechanical 
Ventilation. An Ethnographic Study. American Journal of Respiratory and Critical Care Medicine 2020;201:823-31. 
102. 
Mandal J, Dinoop K, Parija SC. Microethics in medical education and practice. Tropical parasitology 2015;5:86. 
103. 
Pun BT, Balas MC, Barnes-Daly MA, et al. Caring for critically ill patients with the ABCDEF bundle: results of the 
ICU Liberation Collaborative in over 15,000 adults. Critical care medicine 2019;47:3-14. 
104. 
Hsieh SJ, Otusanya O, Gershengorn HB, et al. Staged implementation of awakening and breathing, coordination, 
delirium monitoring and management, and early mobilization bundle improves patient outcomes and reduces hospital 
costs. Critical care medicine 2019;47:885-93. 
105. 
Trogrlić Z, van der Jagt M, Bakker J, et al. A systematic review of implementation strategies for assessment, 
prevention, and management of ICU delirium and their effect on clinical outcomes. Critical Care 2015;19:157. 
106. 
Stollings JL, Devlin JW, Lin JC, Pun BT, Byrum D, Barr J. Best Practices for Conducting Interprofessional Team 
Rounds to Facilitate Performance of the ICU Liberation (ABCDEF) Bundle. Read Online: Critical Care Medicine| Society of 
Critical Care Medicine 2020;48:562-70. 
107. 
Boehm LM, Dietrich MS, Vasilevskis EE, et al. Perceptions of workload burden and adherence to ABCDE bundle 
among intensive care providers. American Journal of Critical Care 2017;26:e38-e47. 
108. 
Kim C, Kim S, Yang J, Choi M. Nurses' perceived barriers and educational needs for early mobilisation of critical ill 
patients. Australian Critical Care 2019;32:451-7. 
109. 
Costa DK, White MR, Ginier E, et al. Identifying barriers to delivering the awakening and breathing coordination, 
delirium, and early exercise/mobility bundle to minimize adverse outcomes for mechanically ventilated patients: a 
systematic review. Chest 2017;152:304-11. 
110. 
Melnyk BM, Gallagher-Ford L, Fineout-Overholt E. Implementing the evidence-based practice (EBP) 
competencies in healthcare: a practical guide for improving quality, safety, and outcomes: Sigma Theta Tau; 2016. 
111. 
Happ MB, Swigart VA, Tate JA, Hoffman LA, Arnold RM. Patient involvement in health‐related decisions during 
prolonged critical illness. Research in nursing & health 2007;30:361-72. 
112. 
Altschuler T, Happ MB. Partnering with speech language pathologist to facilitate patient decision making during 
serious illness. Geriatric nursing (New York, NY) 2019;40:333. 
113. 
Meltzer EC, Gallagher JJ, Suppes A, Fins JJ. Lip-reading and the ventilated patient. Critical care medicine 
2012;40:1529-31. 
114. 
Szulecki D. Speaking up to save a life. AJN The American Journal of Nursing 2016;116:68-9. 
115. 
Happ MB. Interpretation of nonvocal behavior and the meaning of voicelessness in critical care. Social science & 
medicine 2000;50:1247-55. 
116. 
Happ MB, Garrett K, Thomas DD, et al. Nurse-patient communication interactions in the intensive care unit. 
American Journal of Critical Care 2011;20:e28-e40. 
117. 
Tate JA, Seaman JB, Happ MB. Overcoming barriers to pain assessment: communicating pain information with 
intubated older adults. Geriatric nursing (New York, NY) 2012;33. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
32 
 
118. 
Klompas M, Anderson D, Trick W, et al. The preventability of ventilator-associated events. The CDC prevention 
epicenters wake up and breathe collaborative. American journal of respiratory and critical care medicine 2015;191:292-
301. 
119. 
Payen J-F, Bru O, Bosson J-L, et al. Assessing pain in critically ill sedated patients by using a behavioral pain scale. 
Critical care medicine 2001;29:2258-63. 
120. 
Gélinas C, Fillion L, Puntillo KA, Viens C, Fortier M. Validation of the critical-care pain observation tool in adult 
patients. American Journal of Critical Care 2006;15:420-7. 
121. 
Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation–Sedation Scale: validity and reliability in adult 
intensive care unit patients. American journal of respiratory and critical care medicine 2002;166:1338-44. 
122. 
Riker RR, Picard JT, Fraser GL. Prospective evaluation of the Sedation-Agitation Scale for adult critically ill 
patients. Critical care medicine 1999;27:1325-9. 
123. 
Bergeron N, Dubois M-J, Dumont M, Dial S, Skrobik Y. Intensive Care Delirium Screening Checklist: evaluation of 
a new screening tool. Intensive care medicine 2001;27:859-64. 
124. 
Boehm LM, Vasilevskis EE, Dietrich MS, et al. Organizational domains and variation in attitudes of intensive care 
providers toward the ABCDE bundle. American Journal of Critical Care 2017;26:e18-e28. 
125. 
Bassett R, Adams KM, Danesh V, et al. Rethinking critical care: decreasing sedation, increasing delirium 
monitoring, and increasing patient mobility. The Joint Commission Journal on Quality and Patient Safety 2015;41:62-74. 
126. 
Boehm LM, Vasilevskis EE, Mion LC. Interprofessional perspectives on ABCDE bundle implementation: a focus 
group study. Dimensions of critical care nursing: DCCN 2016;35:339. 
127. 
Costa DK, Valley TS, Miller MA, et al. ICU team composition and its association with ABCDE implementation in a 
quality collaborative. Journal of critical care 2018;44:1-6. 
128. 
Costa DK, White M, Ginier E, et al. Identifying barriers to delivering the ABCDE bundle to minimize adverse 
outcomes for mechanically ventilated patients: a systematic review. Chest 2017;152:304. 
129. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, 
measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services 
Research 2011;38:65-76. 
130. 
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II-A Severity of disease classification system: Reply. 
Critical Care Medicine 1986;14:755. 
131. 
Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system: risk prediction of hospital mortality 
for critically III hospitalized adults. Chest 1991;100:1619-36. 
132. 
Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: 
hospital mortality assessment for today’s critically ill patients. Critical care medicine 2006;34:1297-310. 
133. 
Zhang F, Wagner AK, Ross-Degnan D. Simulation-based power calculation for designing interrupted time series 
analyses of health policy interventions. Journal of clinical epidemiology 2011;64:1252-61. 
134. 
Sakr Y, Moreira CL, Rhodes A, et al. The impact of hospital and ICU organizational factors on outcome in critically 
ill patients: results from the Extended Prevalence of Infection in Intensive Care study. Critical care medicine 
2015;43:519-26. 
135. 
Checkley W, Martin GS, Brown SM, et al. Structure, process and annual intensive care unit mortality across 69 
centers: United States Critical Illness and Injury Trials Group Critical Illness Outcomes Study (USCIITG-CIOS). Critical care 
medicine 2014;42:344. 
136. 
Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of Interprofessional Team Collaboration Scale (AITCS): 
development and testing of the instrument. Journal of continuing education in the health professions 2012;32:58-67. 
137. 
Nurses AAoC-C. AACN standards for establishing and sustaining healthy work environments: a journey to 
excellence. American Journal of Critical Care 2005;14:187-97. 
138. 
Bravo G, Potvin L. Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass 
correlation coefficient: toward the integration of two traditions. Journal of clinical epidemiology 1991;44:381-90. 
139. 
Carmines EG, Zeller RA. Reliability and validity assessment: Sage publications; 1979. 

IRB Protocol Number: 
2019H0092 
IRB Approval date:  
tbd 
     Version:  
09/25/2020 
 
33 
 
140. 
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the 
Expert Recommendations for Implementing Change (ERIC) project. Implementation Science 2015;10:21. 
141. 
Burke JG, O’Campo P, Peak GL, Gielen AC, McDonnell KA, Trochim WM. An introduction to concept mapping as a 
participatory public health research method. Qualitative health research 2005;15:1392-410. 
142. 
Rush SC. Concept system software. Taylor & Francis; 2014. 
143. 
Trochim W, Kane M. Concept mapping: an introduction to structured conceptualization in health care. 
International Journal for Quality in Health Care 2005;17:187-91. 
144. 
Brownson RC, Kelly CM, Eyler AA, et al. Environmental and policy approaches for promoting physical activity in 
the United States: a research agenda. Journal of physical activity and health 2008;5:488-503. 
145. 
Green AE, Aarons GA. A comparison of policy and direct practice stakeholder perceptions of factors affecting 
evidence-based practice implementation using concept mapping. Implementation Science 2011;6:104. 
146. 
Trochim W. The Reliability of Concept Mapping En actas de la Annual Conference of the American Evaluation 
Association. Dallas, Texas 1993. 
147. 
Archibald MM, Ambagtsheer RC, Casey MG, Lawless M. Using Zoom Videoconferencing for Qualitative Data 
Collection: Perceptions and Experiences of Researchers and Participants. International Journal of Qualitative Methods 
2019;18:1609406919874596. 
148. 
Tate JA, Happ MB. Neurocognitive Problems in Critically Ill Older Adults: The Importance of History. Geriatric 
nursing (New York, NY) 2011;32:285. 
149. 
Tate JA, Devito Dabbs A, Hoffman LA, Milbrandt E, Happ MB. Anxiety and agitation in mechanically ventilated 
patients. Qualitative health research 2012;22:157-73. 
150. 
Barnato AE, Tate JA, Rodriguez KL, Zickmund SL, Arnold RM. Norms of decision making in the ICU: a case study of 
two academic medical centers at the extremes of end-of-life treatment intensity. Intensive care medicine 2012;38:1886-
96. 
151. 
Happ MB, Tate JA, Swigart VA, DiVirgilio-Thomas D, Hoffman LA. Wash and wean: bathing patients undergoing 
weaning trials during prolonged mechanical ventilation. Heart & Lung 2010;39:S47-S56. 
152. 
Radtke JV, Tate JA, Happ MB. Nurses’ perceptions of communication training in the ICU. Intensive and Critical 
Care Nursing 2012;28:16-25. 
153. 
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist 
for interviews and focus groups. International journal for quality in health care 2007;19:349-57. 
154. 
Sandelowski M, Docherty S, Emden C. Focus on qualitative methods. Sample size in qualitative research 
Research in Nursing & Health 1995;18:179-83. 
155. 
DeCuir-Gunby JT, Marshall PL, McCulloch AW. Developing and using a codebook for the analysis of interview 
data: An example from a professional development research project. Field methods 2011;23:136-55. 
156. 
Sandelowski M. Whatever happened to qualitative description? Research in nursing & health 2000;23:334-40. 
157. 
Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and 
measures to achieve trustworthiness. Nurse education today 2004;24:105-12. 
158. 
Miles M, Huberman A, Saldana J. Qualitative Data Analysis: A Sourcebook, 3rd edn London. Sage; 2014. 
159. 
Wilson J, Mandich A, Magalhães L. Concept mapping: A dynamic, individualized and qualitative method for 
eliciting meaning. Qualitative Health Research 2016;26:1151-61.