Using the Electronic Health Record to Identify and Promote Goals-of-Care Communication for Older Patients With Serious Illness
Phase: NA
Conditions: Dementia, Chronic Disease, Neoplasm Metastasis, Lung Neoplasms, Pulmonary Disease, Chronic Obstructive, Heart Failure, Congestive, Liver Cirrhosis, Kidney Failure, Chronic, Lung Diseases, Interstitial, Peripheral Vascular Disease, Diabetes With End Organ Injury
Sponsor: University of Washington

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
1 
Study Protocol: Project to Improve Communication About Serious Illness--Hospital Study (PICSI-H) 
a. Project to Improve Communication about Serious Illness-Hospital study: Pragmatic Trial (Trial 1) 
(PICSI-H) : NCT04281784 
b. Project to Improve Communication about Serious Illness-Hospital study: Comparative 
Effectiveness Trial (Trial 2) (PICSI-H) : NCT04283994 
 
Date:  May 15, 2025 
 
Principal Investigator:  Erin Kross, MD 
 
Cambia Palliative Care Center of Excellence 
University of Washington, Seattle WA 
 
Table of Contents 
1) 
Brief Overview ......................................................................................................................................... 2 
2) 
Specific Aims: .......................................................................................................................................... 2 
3) 
Research Plan .......................................................................................................................................... 3 
4) 
Research Design and Methods ................................................................................................................ 4 
5) 
Outcomes ................................................................................................................................................ 7 
6) 
Analyses ................................................................................................................................................. 10 
7) 
Sample size ............................................................................................................................................ 10 
8) 
Data management and quality control to achieve scientific rigor ........................................................ 11 
9) 
Protocol modifications .......................................................................................................................... 11 
10) 
Potential limitations and alternative approaches ............................................................................. 12 
11) 
Anticipated findings .......................................................................................................................... 13 
  
 
 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
2 
1) Brief Overview 
 
For hospitalized patients with chronic illness, a key component of high-quality care includes goals-of-care 
discussions (GOCD) conducted early during a hospital stay to identify how patients’ goals of care should inform 
current care plans. These goals-of-care discussions are associated with improved patient and family outcomes 
and reduced intensity of care at the end of life. Despite their importance, GOCD during a hospitalization often 
do not occur. The failure to conduct these conversations and to ensure that care received is aligned with 
patients’ wishes for care is particularly a concern for older adults with chronic illness, and particularly those 
with Alzheimer’s disease and related dementias (ADRD).  
 
This study, funded by NIA, evaluates the Jumpstart intervention, the key to which is the “Jumpstart Guide.” 
There are two versions of the Jumpstart Guide being tested, one that is “EHR-based, clinician-facing” and 
includes information identifying the dates of prior advance care planning documents (e.g., code status, 
healthcare directives, durable power of attorney for healthcare) and Physician Orders for Life Sustaining 
Treatments (POLST forms) in the electronic health record (EHR) to the clinician. It also includes “just-in-time” 
suggestions for having a GOCD. The other guide is “survey-based, bi-directional.” It is survey-based because it 
provides goals of care information from patients’ self-reported surveys (or proxy reports from family members 
if patients do not have decisional capacity) and is then shared with their clinicians. It is bi-directional because 
there is one version of the Jumpstart Guide that goes to the clinician and another version that goes to the 
patient or, if the patient doesn’t have decisional capacity, a family member surrogate. These survey-based, 
bidirectional one-page Jumpstart Guides include patient-specific preferences with “just-in-time” tips to 
improve this communication, tailored either for clinicians or patients and family members. The clinicians’ 
Jumpstart Guide also includes information identifying the dates of prior advance care planning documents. 
 
The study comprises two linked, complementary randomized trials. Trial 1, a large pragmatic trial, compares 
usual care with the EHR-based clinician-facing Jumpstart for hospitalized older adults with serious illness 
(target n=2000; ADRD subset, target n=400). Trial 2, a comparative effectiveness trial, is a three-arm trial that 
compares the EHR-based clinician-facing Jumpstart, the survey-based bi-directional Jumpstart, and usual care 
(n=600). Subjects are enrolled from the UW Medicine hospitals (Harborview Medical Center, UW Medical 
Center - Montlake, UW Medical Center – Northwest). For Trial 1, eligible patients are ≥ 55 years, admitted for a 
minimum of 12 and a maximum of 96 hours prior to study enrollment to inpatient services, without a 
documented GOCD during admission, and meet criteria for serious illness; hospitalized patients >80 years are 
also eligible. For Trial 2, eligible patients meet the same criteria and, in addition, must be sufficiently fluent in 
English to complete questionnaires and not be under respiratory isolation precautions in order to allow in-
person recruitment. If patients are eligible but do not have decisional capacity, a family member or friend 
involved with their care may participate as their surrogate; family members or friends include individuals who 
are  legal guardians, durable power of attorney for healthcare, spouses, adult children, parents, siblings, 
domestic partners, other relatives, and friends. In addition, we will conduct qualitative interviews with key 
stakeholders including patients/families (n=60, subgroup of patients/families enrolled for Trial 2) and clinicians 
(n=50 across Trial 1 and Trial 2). 
 
2) Specific Aims: 
 
Specific Aim 1 (Trial 1): Evaluate the effectiveness of a novel intervention, the EHR-based clinician-facing 
Jumpstart, compared with usual care, for improving the quality of care; the primary outcome is EHR 
documentation of a goals-of-care discussion during the 30 days after randomization. Secondary outcomes 
focus on intensity of care:  ICU admissions and readmissions, ICU and hospital free days, costs of care during 
the hospitalization, and 7- and 30-day hospital readmission. 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
3 
Specific Aim 2 (Trial 2): Evaluate the effectiveness of the survey-based bi-directional Jumpstart compared to 
the EHR-based clinician-facing Jumpstart and usual care for improving quality of care; the primary outcome is 
EHR documentation of a goals-of-care discussion within 30 days after randomization. Secondary outcomes 
include intensity of care outcomes from Aim 1 and patient- and family-reported outcomes assessed by surveys 
at 3-5 days and 4-6 weeks after randomization, including occurrence and quality of goals-of-care discussions in 
the hospital, goal-concordant care, psychological symptoms, and quality of life.  
 
Specific Aim 3 (Trials 1, 2):  Conduct a mixed-methods evaluation of the implementation of both interventions, 
guided by the RE-AIM and CFIR frameworks for implementation science, incorporating quantitative evaluation 
of intervention reach and adoption, as well as qualitative analyses of interviews with participants, to explore 
barriers and facilitators to future implementation and dissemination.   
 
3) Research Plan 
 
a) Background and Significance 
 
People near the end of life often receive care they would not choose.1,2 The National Academy of 
Medicine has documented these discrepancies in care and identified advance care planning and goals-of-care 
discussions as primary mechanisms for addressing them.1 This type of communication is a focus for 
improvement for two key reasons: 1) when goals-of-care discussions occur, they are associated with improved 
quality of care and patient- and family-centered outcomes, including increased quality of life, reduced 
symptoms of psychological distress, and fewer intensive treatments at the end of life;3-6and 2) clinicians 
frequently do not have goals-of-care discussions with their patients until very late in the illness.3,7-10 
 
The value of advance care planning discussions with healthy individuals is a topic of debate; however 
improving goals-of-care discussions for those with serious illness facing difficult treatment decisions is widely 
agreed upon as an urgent need that can improve patient outcomes.11-16 There is an emerging consensus on this 
important distinction between advance care planning (ACP) for healthy individuals and goals-of-care 
discussions for those with chronic life-limiting illness and on the critical importance of timely goals-of-care 
discussions.11,17,18 Furthermore, even if advance care planning does occur in the outpatient setting, effective 
goals-of-care discussions (a component of advance care planning for more proximal decision-making) are still 
needed for hospitalized patients whose prior preferences may have changed or may not have been specific to 
the current circumstances.19-21 In short, for hospitalized patients with chronic illness, a key component of high-
quality care includes goals-of-care discussions conducted early during a hospital stay that build on prior 
advance care planning and identify how patients’ goals inform current care.8,11,22 These early hospital 
discussions are supported by the National Quality Forum.23 Despite their key importance to a large number of 
patients, early hospital goals-of-care discussions often do not occur.8,24 A recent research agenda for serious 
illness communication, supported by the National Institute on Aging and published in JAMA Internal Medicine, 
highlights the importance of promoting high-quality goals-of-care discussions, as well as the potential 
opportunity to use the EHR to both identify those patients who would benefit from goals-of-care discussions 
and to guide clinicians in high-quality discussions.25 We propose two complementary trials to examine the 
effectiveness of such interventions, and we use an innovative hybrid effectiveness-implementation approach 
that evaluates the interventions and their implementation.26 
 
b) Innovation 
 
Use of the EHR to identify seriously ill, hospitalized patients without a goals-of-care discussion: Recent research 
agendas highlight the lack of research utilizing the EHR to implement interventions that improve serious illness 
communication.25,27 We will use a validated EHR-based quality metrics program to identify hospitalized 
patients aged 55 years and older with chronic serious illness who do not have EHR documentation of a goals-

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
4 
of-care discussion. We have developed an innovative natural language processing/machine learning (NLP/ML) 
protocol to identify inpatient and outpatient documentation of goals-of-care discussions in any type of EHR 
note, including admission notes, progress notes, and discharge summaries. In this way, we not only target a 
population likely to benefit from the intervention but also do so with methods that are generalizable and 
scalable.28 
 
Examine the comparative effectiveness of an EHR-based clinician-facing Jumpstart, a survey-based bi-
directional Jumpstart and usual care in an innovative study design: The intervention is based on our recently 
completed trial in the outpatient setting of the survey-based bi-directional intervention (Jumpstart), which is 
an individualized communication-priming intervention, targeting both patients and clinicians and providing 
each with information obtained from patient surveys in order to guide a goals-of-care discussion.29 This 
intervention is innovative because it is one of the few that involves clinicians as well as patients and family 
members (bi-directional). However, a question raised by reviewers of our prior trial2 was whether the 
individualized survey-based component was necessary, especially given the resources needed to implement it. 
Therefore, we have developed this innovative study design that combines a large pragmatic trial of an EHR-
based clinician-facing Jumpstart compared to usual care (Trial 1, using a waiver of informed consent to 
facilitate enrollment and the pragmatic approach) and a smaller comparative effectiveness trial of three arms:  
an EHR-based clinician-facing Jumpstart, a survey-based bi-directional Jumpstart, and usual care (Trial 2). In 
contrast to Trial 1, this comparative effectiveness trial includes individual patient or surrogate consent for 
participation as well as completion of surveys to generate Jumpstart Guides. 
 
Develop an innovative effectiveness-implementation approach that advances implementation and 
dissemination: Few evidence-based communication interventions are widely adopted. Barriers to 
implementation include factors at the level of individual patients, clinicians, operating clinical units, and 
healthcare systems.30,31 The RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) 
framework systematically addresses these factors and, in turn, translates interventions like Jumpstart into the 
“real world”.32-35 Similarly, the Consolidated Framework for Implementation Research (CFIR) offers another 
framework that examines implementation of evidence-based interventions, including domains representing 
intervention characteristics, outer setting, inner setting, characteristics of individuals and process.36 This study 
will include a novel hybrid effectiveness-implementation approach that can accelerate implementation and 
dissemination of the interventions after the study by allowing us to evaluate–during the study–strategies and 
outcomes that will facilitate uptake of the interventions in the future.26,36-39 This innovative design offers the 
opportunity to advance implementation science in palliative care and increases the utility of this study. 
 
4) Research Design and Methods 
 
a) Overview 
 
We will conduct two linked, complementary randomized trials of an EHR-based clinician-facing or a survey-
based bi-directional intervention to promote and guide goals-of-care discussions for older, seriously ill, 
hospitalized patients using an innovative method for identifying eligible participants through the EHR. Trial 1 
assesses the effectiveness of the EHR-based clinician-facing Jumpstart compared with usual care in a large 
pragmatic trial not requiring patient or family consent (Aim 1, Trial 1). Trial 2 examines the efficacy of the 
survey-based, bi-directional Jumpstart as compared with the EHR-based, clinician-facing Jumpstart and usual 
care (Aim 2, Trial 2). In addition, we will examine the implementation of the interventions using the RE-AIM 
and CFIR frameworks32-36 (Aim 3). Figure 1 provides an overview of the design. 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
5 
 
 
b) Setting 
 
We will conduct this study with a diverse population drawn from 3 hospitals at UW Medicine, including 
a university hospital (UW Medical Center - Montlake), a county, safety-net hospital (Harborview Medical 
Center, HMC), and a community hospital (UW Medical Center - Northwest). UWMC - Montlake provides 
specialist care for the Pacific Northwest region and has 529 acute care beds and 75 ICU beds. HMC is operated 
by the University of Washington and has 413 acute care beds and 94 ICU beds. HMC is the only Level 1 Trauma 
Center serving five states, and its mission population includes inner city poor, recent immigrants, and persons 
with HIV/AIDS. UWMC - Northwest is a community hospital with 218 acute care beds and 15 ICU beds, serving 
north Seattle with a large geriatric and nursing home resident population. Our prior studies have included each 
of these sites.40-42 These 3 sites offer the advantage of caring for diverse patients while also using a unified EHR 
incorporating Epic systems into a platform we have been able to access and use.43-48 
 
c) Patient population 
 
Trials 1 and 2:  Eligible patients aged 55 years and over will be identified by ICD-10 codes for one or more of 
the nine chronic conditions used by the Dartmouth Atlas49: dementia, malignant cancer/leukemia, chronic 
pulmonary disease, coronary artery disease, heart failure, chronic liver disease, chronic renal disease, diabetes 
with end-organ damage, and peripheral vascular disease. These nine conditions account for 90% of deaths 
among Medicare beneficiaries in the US.50,51 To increase inclusivity of important and under-studied 
populations, we will also include all hospitalized patients over age 80. Among patients meeting these criteria, 
we will include only those with no identified documentation of goals-of-care discussions during the current 
hospitalization and prior to enrollment.  
 
Trial 2:  Eligible patients or their surrogates will have sufficient English language proficiency to complete 
surveys. Additionally, patient’s providers will be alerted via SecureChat in Epic that study staff are planning to 
recruit the patient or their family member and given the opportunity to notify study staff if they believe the 
patient or patient’s family should not be approached for any reason.  
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
6 
d) Description of participants 
 
We will collect the following information about study participants.  
 
Patients: For Trials 1 and 2, using the EHR, we will identify the following demographics:  age, race, ethnicity, 
sex, marital status, comorbidities, limited spoken English proficiency, and acute severity of illness including the 
Deyo-Charlson comorbidity index. For Trial 2, using survey-reported data, we will report education, health 
status (SF-1), and income.  
 
Families:  For Trial 2, using survey-reported data, we will report the following demographics for participating 
family members/surrogates:  age, race, ethnicity, sex, education, marital status, health status (SF-1), 
relationship to patient, and living situation related to patient.  
 
Clinicians:  For Trials 1 and 2, using information collected during the interview, we will report the following 
demographics for clinicians participating in qualitative interviews:  age, race, ethnicity, sex, clinical specialty. 
 
e) Sampling patients with ADRD 
 
Given the dramatically increasing prevalence of ADRD in the US and the rising intensity of care among these 
patients,52,53 it is particularly important that we understand the effect of interventions to improve quality of 
palliative care in this group. We designed both trials to be powered to examine heterogeneity of treatment 
effects (HTE) in this important group. For Trial 1, we plan to continue recruitment as needed to meet our 
sample requirements for this subgroup. For Trial 2, we will prioritize recruitment of patients with ADRD to 
maximize this subgroup sample size. We hypothesize that the interventions will be equally effective in this 
population, but that the proportion of patients with decisional capacity will be lower (enrolling more family 
members in Trial 2). However, we believe it is important to explicitly examine this hypothesis given the unique 
features of intensity of care at the end of life for this important and increasing population. 
 
f) Randomization 
 
Patients are randomized in a 1:1 ratio in Trial 1 and 1:1:1 ratio in Trial 2 using variable size blocks and stratified 
for hospital and ADRD vs. no ADRD. Participating family members or legal surrogate decision makers are 
assigned to the same arm as the corresponding patient.   
 
g) Intervention 
 
EHR-based clinician-facing Jumpstart (Trials 1 and 2):  First, we use automated methods to examine inpatient 
and outpatient EHR notes prior to the current admission, identifying current code status as well as all prior 
POLST forms and advance directives; this information is included on Jumpstart Guides to inform discussions. 
Second, we deliver the Jumpstart Guide to the primary hospital team (all attending and resident physicians and 
advanced practice providers) via secure email and either a page alerting the providers to the presence of the 
Jumpstart Guide in their email (Trial 1) or via in-person delivery (Trial 2). See Appendices 1 and 2 for Trial 1 
EHR-based clinician-facing Jumpstart Guides (PDF and HTML formats), and Appendix 3 for Trial 2 EHR-based 
clinician-facing Jumpstart Guide.  
 
Survey-based bi-directional Jumpstart (Trial 2 only):  First, information about the patient is abstracted from the 
EHR in the same way as for the EHR-based clinician-facing Jumpstart. Second, patients or their family 
member/surrogate complete baseline survey items assessing three domains: a) preferences for discussions 
about goals of care; b) barriers and facilitators for having such discussions; and c) current goals of care. Third, 
using the EHR and baseline survey, we use the automated algorithm from our prior trial29 adapted to the 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
7 
hospital setting using human-centered design methods54 to create a survey-based Jumpstart Guide to prompt 
and guide goals-of-care discussions between the patient and hospital team or, if the patient isn’t able, the 
family member and the hospital team. Finally, in the fourth step, we deliver the Jumpstart Guides to the 
primary team via secure email as well as in-person delivery to members of the team. We also provide a survey-
based Jumpstart Guide to the patient or family, adapted with phrasing and terminology specifically for the 
patient and family. All Jumpstart Guides are delivered on the day of randomization with the goal of prompting 
a goals-of-care discussion early during hospitalization, as supported by the National Quality Forum.23 The 
clinicians’ survey-based bi-directional Jumpstart includes both EHR-data and patient-tailored suggestions for 
conducting goals-of-care discussions based on survey responses. The patients’ survey-based bidirectional 
Jumpstart includes patient-tailored suggestions for starting a conversation with their doctor. The suggestions 
are guided by the educational experience of VitalTalk, a nationally acclaimed program for teaching serious 
illness communication, and adapted to the inpatient setting.55,56 See Appendix 4 for clinicians’ survey-based bi-
directional Jumpstart Guide, and Appendix 5 for patients’ survey-based bi-directional Jumpstart Guide.  
 
5) Outcomes 
 
a) Outcomes from the EHR and Death Certificates (Trials 1 and 2): 
 
The primary outcome for both trials is EHR documentation of goals-of-care discussions within 30 days after 
randomization. Our rationale for this as the primary outcome is that this is the primary target for all 
interventions and important to diverse stakeholders including patients and their families.11,43,57-59 We will use 
NLP/ML methods to identify goals-of-care discussions.60 We will manually review the EHR for goals-of-care 
discussions using our standard EHR abstraction methods40-42 for a randomly selected subset of patients in each 
trial to evaluate potential misclassification with NLP/ML methods. 
  
Secondary outcomes for both trials, obtained from the EHR, include utilization metrics associated with 
intensity of care (e.g., any ICU admissions, hospital readmissions, and ICU- and hospital-free days). ICU 
admissions will be assessed 30- and 90-days post-randomization, as well as 7- and 30-days post-discharge. 
Hospital readmissions will be assessed for 7- and 30-days post-discharge. ICU- and hospital-free days are 
defined as the number of days alive and outside of the ICU (or hospital) within the specified time period after 
randomization (i.e., 30- or 90-days).61,62 We will also examine the following outcomes: 1) emergency 
department visits within 30- and 90-days post-randomization (Trial 1 only); 2) palliative care consults 
completed in the 30- and 90-days post-randomization; 3) costs of care during hospital admission and within 
30- and 90-days following randomization (obtained from institutional billing systems); 4) all-cause mortality at 
30-days (both Trials) and 120-days (Trial 2 only) post-randomization (using EHR and Washington State death 
certificate data); 5) time to first goals-of-care discussions during the 30-days post-randomization (Trial 2 only); 
and 5) days spent in the hospital post-randomization during the index hospitalization (Trial 1 only). See Table 1 
for a summary of study outcomes. 
 
b) Outcomes derived from patient- and family-reports (Trial 2 only): 
 
Additional outcomes for Trial 2 will be obtained from patient-reported or family-reported surveys (Table 1). 
Depending on the research question, surveys will be completed during at least one of the following time 
points: 1) baseline; 2) 3-5 days after randomization; and 3) 4-6 weeks after randomization. Surveys may be 
completed in person, online, by mail, or by phone, based on respondents’ preferences.  
 
Occurrence and quality of discussions: We use previously validated items to assess the occurrence and quality 
of goals-of-care communication during the hospitalization after randomization.5,29,63-68 Communication 
occurrence is assessed with a single item.29,64 Quality of goals-of-care communication is assessed with the end-

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
8 
of-life communication subscale (QOC_eol) of the Quality of Communication (QOC) survey, developed from 
qualitative interviews and focus groups with a diverse group of patients, families, and clinicians.63,65,67  
 
Goal-concordant care: Concordance between the care patients want and the care they are receiving will be 
measured with two questions from SUPPORT.69 The first question defines patients’ priorities for extending life 
or ensuring comfort. The second question assesses patients’ perceptions of their current treatment using the 
same two options.69 Concordance is defined as a match between preference for care and the type of care 
currently received, as reported by patients or families. Although most patients want both quality and life-
extending care, requiring respondents to pick one is a useful way to identify patients’ top priority.70-72 If 
patients are unable to respond, goals of care are elicited from family as they would be in clinical practice.73  
Additionally, goal-concordant care will be assessed with an investigator developed question, “Is your care (or 
your family member’s care) in line with your (their) goals?”. Data from this question complements the 
SUPPORT items but also addresses some of the shortcomings that have been associated with responses to 
SUPPORT. 
 
Symptoms of anxiety and depression: Patient and family symptoms of anxiety and depression are assessed 
with the Hospital Anxiety and Depression Scale (HADS).74,75 Patients and families will complete these surveys 
for themselves only; we do not ask for surrogate report of patients’ psychological symptoms. The goal is not to 
diagnose the clinical syndromes of anxiety or depression, but rather to identify the burden of symptoms.  
 
Health related quality of life: The EQ-5D-5L is a 5-item questionnaire derived from the widely-used and well-
validated EQ-5D. In this revision, questions similarly represent 5 dimensions of health-related quality of life: 
mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. In contrast to the EQ-5D, each 
dimension has 5 levels rather than 3, ranging from no problems (1) to sever problems (5). Its psychometric 
properties including responsiveness have been supported across a broad range of populations, conditions and 
settings76, including dementia77 and for proxy use78, making it particularly appropriate for the current study.  
 
Shared decision-making: The CollaboRATE instrument is a 4-item survey that assesses patients’ perception of 
shared decision making—that is, feeling and reporting being informed and involved in decision-making steps as 
part of their interactions with their physician.79 It has been shown to demonstrate intra-rater validity, 
discriminative and concurrent validity, as well as sensitivity to change.80 In this study, we will be able to 
evaluate it both as an outcome and as a mediator, exploring whether the Jumpstart interventions alter 
patients’ and surrogates’ perceptions of shared decision-making or whether it mediates the occurrence of the 
primary outcome (EHR documentation of goals of care).  
 
c) Implementation outcomes (Trials 1 and 2) 
 
Assessment of the implementation of the interventions in Aim 3 is guided by the RE-AIM Framework for 
implementation research32-35 and the Consolidated Framework for Implementation Research (CFIR).36 RE-AIM 
is a multidimensional framework for evaluating the public health impact of efforts to translate research into 
practice.33 The five dimensions of RE-AIM are reach of the intervention within the target population, 
effectiveness of the intervention, adoption by target staff members or settings, implementation consistency 
and quality, and maintenance of intervention delivery and effects.32-35 CFIR is a pragmatic meta-theoretical 
framework that synthesizes constructs related to implementation of evidence-based interventions. The five 
overarching domains are intervention characteristics, outer setting, inner setting, characteristics of individuals, 
and process, and include a total of 37 constructs that can be used to understand what works, and why, in a 
certain setting.36 We collect quantitative and qualitative data on reach, effectiveness, adoption, 
implementation, and maintenance (RE-AIM) of the intervention. Quantitative data are collected as routine 
tracking as part of the implementation of both trials, with data on participation, intervention use, fidelity to 
the intervention, and changes over time. Qualitative data are collected through short, semi-structured 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
9 
interviews guided by the CFIR domains. The interviews are conducted in-person or by phone with patients and 
family members from Trial 2, and clinicians from either Trial 1 or 2 after study involvement.  Participants are 
selected using purposive sampling to provide diverse perspectives; we will consider level of participation in the 
study, race, ethnicity, age, gender, and, for clinicians, specialty. A trained qualitative interviewer will interview 
participants using an interview guide, and interviews will be audio-recorded and transcribed.58,81-97.  
 
Table 1: Outcome measures and data collection 
OUTCOME MEASURES 
CONCEPT 
TIME FRAME 
 
Outcomes derived from the EHR 
Primary Outcome 
 
 
EHR documentation of goals-of-care 
discussion  
Goals-of-care 
discussion 
30-days post-randomization 
Secondary Outcomes  
 
 
ICU admissions (post-randomization) 
Intensity of care 
30- and 90-days post-randomization 
ICU admissions (post-discharge) 
Intensity of care 
7- and 30-days post-discharge 
Hospital readmissions (post-
discharge) 
Intensity of care 
7- and 30-days post-discharge 
ICU- and hospital-free days 
Intensity of care 
30- and 90-days post-randomization 
Other pre-specified/exploratory outcomes 
Emergency department visits 
(Trial 1 only) 
Intensity of care 
30- and 90-days post-randomization 
Palliative care consults completed  
Intensity of care 
30- and 90-days post-randomization 
Costs of care 
Intensity of care 
During hospital stay, 30- and 90-days post-
randomization 
All-cause mortality  
All-cause 
mortality 
From EHR and death certificates:  
Trial 1:  30-days post-randomization  
Trial 2:  30- and 120-days post-randomization 
Time to first goals of care discussion 
(Trial 2 only) 
Goals-of care 
discussions 
30-days post-randomization 
Days spent in hospital (index 
hospitalization) (Trial 1 only) 
Intensity of care 
Length of index hospital stay post-
randomization 
Outcomes derived from survey data 
Secondary Outcomes 
Patient/family reported discussion of 
goals29,64 
Goals-of-care 
discussion 
occurrence 
3-5 days and 4-6 weeks post-randomization   
Quality of Communication 
(QOC)63,65,67 
Quality of 
communication 
3-5 days post-randomization 
SUPPORT question69 
Goal-concordant 
care 
3-5 days and 4-6 weeks post-randomization 
Other pre-specified/exploratory outcomes 
HADS – anxiety and depression74,75 
Symptoms of 
anxiety and 
depression 
4-6 weeks post-randomization 
EQ-5D-5L76-78 
Health-related 
QOL 
4-6 weeks post-randomization 
CollaboRATE79,80 
Shared decision-
making 
3-5 days post-randomization 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
10 
Goal Concordance 
Goal-concordant 
care 
3-5 days and 4-6 weeks post-randomization 
 
6) Analyses 
 
We will follow the intention-to-treat principle for all analyses.  
 
a) Primary Outcome (presence of goals of care discussion within 30 days after randomization) 
 
The effect of intervention on the primary outcome will be quantified by the difference in proportions and 
evaluated with a linear regression model with robust standard errors. The predictor of interest is 
randomization arm (EHR-based clinician-facing Jumpstart or usual care for Trial 1; or EHR-based clinician-facing 
Jumpstart, survey-based bi-directional Jumpstart, or usual care for Trial 2). The model will adjust for hospital 
site and ADRD status, since randomization is stratified on these factors. This model assumes the effect of 
intervention is the same for patients with and without ADRD. We will also include an interaction between 
randomization arm and ADRD, which allows the effect of intervention to vary by ADRD status and allows 
evaluation of the effect among those with and without ADRD. We will evaluate the timing of goals-of-care 
discussions with a Cox proportional hazards model.  
 
b) Additional Outcomes 
 
For the analysis of the other outcomes, we will use a strategy similar to that for the primary outcome. For 
continuous outcomes (e.g., ICU-free days, HADS score), the effect of intervention will be quantified by a 
difference in means. For survey outcomes which are collected at more than one time point after 
randomization, we will use a mixed model to account for the correlation between repeated measures. Our 
initial model will allow the average response to be different at each time point, but assume the intervention 
has the same effect at each time. We will also allow the effect of intervention to be different across time by 
including an interaction between time and intervention. The advantage of using the data at the multiple time 
points and a mixed model approach is that we can gain precision; it also allows for missing responses, 
assuming responses are missing at random. Missing data are more of an issue for the survey outcomes than 
the primary outcome; we will quantify the amount and type of missing data, evaluate associations of 
missingness with participant characteristics, and apply appropriate methods to account for missing data.98 
 
c) Evaluate implementation and identify barriers and facilitators to future implementation. 
 
We will perform thematic content analysis of transcribed interviews to explore feedback on the intervention, 
ways to improve intervention implementation, and aspects of care not adequately addressed by the 
intervention.99-101 Interview guides and analyses will be guided by the RE-AIM and CFIR frameworks as 
described above.32-36 Qualitative data will be imported to analytic software (Dedoose), where investigators will 
perform the following analytic steps using an iterative approach to thematic analysis102: 1) initially code 
material, devising a coding framework and using that framework to reduce the text into smaller segments; 2) 
identify themes from the coded text; 3) construct thematic networks that include basic themes, organizing 
themes and global themes; 4) describe and summarize thematic networks; and 5) interpret patterns that have 
emerged in and across thematic networks.  
 
7) Sample size  
 
a) Sample size considerations for the primary outcome   
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
11 
The focus for sample size considerations is the primary outcome: proportion of patients with documented 
goals-of-care discussions within 30 days after randomization. 
 
Trial 1: With a total sample size of 2000 (1000 per group), two-sided significance level (α) of 0.05, and a 
variance estimate based on the proportion in the control group only, we have 80% power to detect a 
difference in proportions between those randomized to the EHR-based clinician-facing Jumpstart and usual 
care of at least 0.06. We assumed a proportion in the control group of 0.54 based on preliminary data. If the 
total number of patients with ADRD in Trial 1 is 400 (200 per group), we would have 80% power with α=0.05 to 
detect a difference in proportions of 0.14 among those with ADRD. 
 
Trial 2: With a total sample size of 600 (200 per the EHR-based clinician-facing Jumpstart, 200 per survey-based 
bi-directional Jumpstart, and 200 per usual care), we have 80% power to detect a difference in proportions of 
16% for each of the 3 pairwise comparisons assuming an overall α=0.05 and a Bonferroni adjustment for the 3 
comparisons (α=0.017 for each comparison) and variance based on a proportion of 0.54.    
 
b) Sample size for qualitative analyses 
 
For Aim 3 qualitative analyses, it is important to achieve theoretical saturation (no new themes 
emerging).101,103 Based on our prior studies, we anticipate achieving saturation by 80 interviews for 
understanding patients/families and clinician perspectives.58,84,92-97 We will monitor for saturation and will 
recruit additional participants if needed. 
  
8) Data management and quality control to achieve scientific rigor 
 
This project requires the creation, maintenance, and analysis of a database that includes a variety of measures 
from multiple sources.  This study, like all studies, depends on the quality of the data and therefore systematic 
data collection, quality control, and data-management procedures will be implemented: 1) protocols for data 
collection; 2) rigorous training, certification, and periodic re-training of study staff, with ongoing monitoring of 
adherence to protocols; 3) regular review of questionnaire response rates, respondent burden,104 and missing 
items to identify and correct problems; 4) verification of all data through custom-designed data entry systems; 
and 5) weekly team meetings to provide feedback to study staff to ensure problems are resolved quickly. To 
ensure reliability and validity of data, we will use our current methods for training and quality control.105-109 
Staff conducting EHR review will undergo the following training: instruction on the protocol, guided practice 
abstraction, and independent abstraction with reconciliation by a trainer.  
 
9) Protocol modifications 
 
1. Changes to inclusion/exclusion criteria for Trials 1 and 2 (except where noted) 
a. Lowering the inclusion criteria age from ≥65 to ≥55 years 
b. Removing “English speaking” as an inclusion criterion (Trial 1 only) 
c. Adding hospital admission for a minimum of 12 hours as an inclusion criterion 
d. Removing markers of frailty as an eligibility criterion 
e. Adding pregnancy, suicide attempt, and same-day discharge as exclusion criteria 
f. Adding COVID-19 as a specific inclusion diagnosis (Trial 1 only) 
2. Study design 
a. For Trial 2, in addition to comparing the survey-based bi-directional Jumpstart to the EHR-
based clinician-facing Jumpstart, we will also include a third arm in which patients receive 
usual care.  

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
12 
b. We have increased the target sample size from n=400 to n=600 to accommodate the addition 
of a third “usual care” arm to the study. 
c. We have modified the questionnaires that patients receive at baseline such that only patients 
in the survey-based bi-directional Jumpstart arm are presented with items that are used to 
create the bi-directional, survey-based Jumpstart. These items are an integral part of the 
survey-based bi-directional intervention and therefore not appropriate for the other arms to 
complete.  
3. Procedures 
a. Removing data collection for acute severity of illness (e.g., SOFA score) for Trial 2 and replacing 
with a Deyo-Charlson comorbidity index 
b. Revisions to instruments included in surveys for Trial 2 
c. Increasing subject numbers for clinician interviews from 20 to 50 across both trials 
d. Addition of demographic items (required by study sponsor, NIH) to the clinician interview 
4. Materials 
a. Updated instructional video for clinicians in Trial 1 
b. Revision of the Jumpstart Guide into an HTML format as a delivery option. This version 
includes optional feedback buttons at the bottom (options: will definitely use; will use if time 
allows; maybe, will consider; not appropriate; will not see this patient; already done; opt out; 
other/free text) 
c. Updated language and formatting for all versions of the Jumpstart Guide (patient survey-based 
bidirectional, clinician survey-based bidirectional, EHR-based clinician facing) using a human-
centered design approach54  
 
10) Potential limitations and alternative approaches 
 
a) Including ADRD and other diseases in same study 
 
Patients with ADRD receive different intensity of care at the end of life compared to other chronic 
diseases,52,110 and there may be differences in the effectiveness of the interventions. Our hypothesis is that 
these interventions will work for all diseases, but because of the unique issues of increasing ICU use among 
those with ADRD,52,110 we will target adequate sample size for patients with ADRD to be able to test this 
hypothesis for the primary outcome of each trial. We could have proposed separate trials for ADRD and other 
illnesses, but this would decrease the generalizability of the interventions. 
 
b) Generalizability 
 
This study occurs in a single healthcare system but includes three diverse hospitals thus enhancing 
generalizability. 
 
c) Misclassification of goals-of-care discussions 
 
Goals-of-care discussions may be misclassified for two reasons: 1) the sensitivity and specificity of the NLP/ML 
algorithm is not perfect; and 2) documentation of goals-of-care discussions in the EHR will never perfectly 
reflect actual discussions. This misclassification could affect outcome assessment and patient identification. 
For outcome assessment, we will assess the accuracy of the NLP/ML algorithm against manual EHR review in a 
randomly selected sample of patients to evaluate the extent of misclassification.  For our final algorithm, we 
will use human abstractors to verify the first documentation of a goals-of-care discussion for each patient to 
maximize positive predictive value and specificity, as has been done by others.111,112  However, since our goal is 
to prompt and guide more discussions than would happen without the interventions, the limitations of the 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
13 
NLP/ML algorithm for patient identification do not invalidate the randomized trials. We could have included all 
hospitalized patients regardless of prior documentation, but we believe that untargeted prompts might limit 
the impact of the intervention. In addition, we will use the interviews in Aim 3 to understand clinicians’ 
perspectives on the effect of misclassification. 
 
d) Contamination 
 
It is possible that this intervention might change behavior for clinicians caring for patients randomized to the 
comparator arms. Our prior studies suggest that most clinicians require a patient-specific prompt to have 
timely goals-of-care discussions, which may mitigate this concern.29,64 However, we will assess for an increase 
in goals-of-care discussions in the comparator groups over time, which might signify contamination or 
temporal trends, but could be used to assess the potential degree of contamination if present. This issue 
would bias the results toward the null hypothesis and only be a major issue for a negative study. 
 
e) Scalability of surveys in Trial 2 
 
Study staff will distribute surveys, which is challenging for implementation in clinical practice. Aim 3 will 
provide insights into how best to address scalability for implementation.  
 
f) Objective assessments of the quality of goals of care discussions 
 
Our NLP/ML approach identifies goals-of-care discussions without assessing their quality. Since our prior trials 
demonstrated increased patient-assessed quality with the Jumpstart intervention, this is less of a concern.29,64 
Future NLP/ML advances may permit quality assessments. Trial 2 assesses quality of communication from 
patient and family perspectives. 
 
g) Costs assessments focus on UW Medicine 
 
Cost assessments for Aim 1 are limited to costs available in the UW Medicine EHR, and we will not be able to 
assess costs from other healthcare systems after hospital discharge. Most of the benefits we anticipate for this 
intervention will occur during the hospitalization, although there may be ongoing reductions in costs after 
hospitalization related to changes in the goals of care as a result of the intervention. We will evaluate for such 
effects in Trial 2, and the limitation of not having access to these costs from outside UW Medicine in Trial 1 is 
diminished somewhat because this is a randomized trial. 
 
11) Anticipated findings 
 
These interventions use the EHR to identify patients who should have documentation of a goals-of-care 
discussion but do not, and then prompt and guide such discussions with either: a) an EHR-based clinician-
facing prompt and guide for clinicians only, along with information about prior advance care planning 
completed prior to the hospitalization; or b) a survey based bi-directional intervention that provides patient-
specific support to clinicians, patients and family members. We anticipate that both interventions will be 
effective compared to the usual care arm, and that this study will provide important options for healthcare 
systems. Economic analyses will allow us to evaluate the effect on costs of care and may enhance the 
intervention’s dissemination. If either or both of these interventions are not effective, the results and the 
interviews in Aim 3 will provide important information to shape, direct and deliver future interventions. 
 
 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
14 
References 
 
1. 
Institute of Medicine. Improving Quality and Honoring Individual Preferences Near the End of Life. 
Washington, DC: National Academy Press; 2015. 
2. 
Lorenz K, Lynn J. Morton SC.  End-of-Life Care and Outcomes. Summary, Evidence Report/Technology 
Assessment: Number 110. AHRQ Publication Number 05-E004-1. Agency for Healthcare Research and Quality, 
2004. (Accessed September 17, 2018, at 
https://archive.ahrq.gov/downloads/pub/evidence/pdf/eolcare/eolcare.pdf.) 
3. 
Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient mental health, 
medical care near death, and caregiver bereavement adjustment. JAMA 2008;300:1665-73. 
4. 
Detering KM, Hancock AD, Reade MC, Silvester W. The impact of advance care planning on end of life 
care in elderly patients: randomised controlled trial. BMJ 2010;340:c1345. 
5. 
Hanson LC, Zimmerman S, Song MK, et al. Effect of the Goals of Care Intervention for Advanced 
Dementia: A Randomized Clinical Trial. JAMA Intern Med 2017;177:24-31. 
6. 
Torke AM, Callahan CM, Sachs GA, et al. Communication Quality Predicts Psychological Well-Being and 
Satisfaction in Family Surrogates of Hospitalized Older Adults: An Observational Study. J Gen Intern Med 
2018;33:298-304. 
7. 
Fakhri S, Engelberg RA, Downey L, et al. Factors Affecting Patients' Preferences for and Actual 
Discussions About End-of-Life Care. J Pain Symptom Manage 2016;52:386-94. 
8. 
Heyland DK, Dodek P, You JJ, et al. Validation of quality indicators for end-of-life communication: 
results of a multicentre survey. CMAJ 2017;189:E980-E9. 
9. 
Silveira MJ, Kim SY, Langa KM. Advance directives and outcomes of surrogate decision making before 
death. N Engl J Med 2010;362:1211-8. 
10. 
Teno JM, Gruneir A, Schwartz Z, Nanda A, Wetle T. Association between advance directives and quality 
of end-of-life care: a national study. J Am Geriatr Soc 2007;55:189-94. 
11. 
Sudore RL, Lum HD, You JJ, et al. Defining Advance Care Planning for Adults: A Consensus Definition 
From a Multidisciplinary Delphi Panel. J Pain Symptom Manage 2017;53:821-32 e1. 
12. 
Childers JW, Back AL, Tulsky JA, Arnold RM. REMAP: A Framework for Goals of Care Conversations. J 
Oncol Pract 2017;13:e844-e50. 
13. 
Morrison RS, Meier DE, Arnold RM. What's Wrong With Advance Care Planning? JAMA 2021;326:1575-
6. 
14. 
Halpern SD. Toward Evidence-Based End-of-Life Care. N Engl J Med 2015;373:2001-3. 
15. 
Halpern SD. Goal-Concordant Care - Searching for the Holy Grail. N Engl J Med 2019;381:1603-6. 
16. 
Halpern SD, Loewenstein G, Volpp KG, et al. Default options in advance directives influence how 
patients set goals for end-of-life care. Health Aff (Millwood) 2013;32:408-17. 
17. 
Mohan D, Sacks OA, O'Malley J, et al. A New Standard for Advance Care Planning (ACP) Conversations 
in the Hospital: Results from a Delphi Panel. J Gen Intern Med 2021;36:69-76. 
18. 
Sudore RL, Heyland DK, Lum HD, et al. Outcomes That Define Successful Advance Care Planning: A 
Delphi Panel Consensus. J Pain Symptom Manage 2018;55:245-55 e8. 
19. 
Sudore RL, Fried TR. Redefining the "planning" in advance care planning: preparing for end-of-life 
decision making. Ann Intern Med 2010;153:256-61. 
20. 
Fried TR, O'Leary J, Van Ness P, Fraenkel L. Inconsistency over time in the preferences of older persons 
with advanced illness for life-sustaining treatment. J Am Geriatr Soc 2007;55:1007-14. 
21. 
Ditto PH, Jacobson JA, Smucker WD, Danks JH, Fagerlin A. Context changes choices: a prospective 
study of the effects of hospitalization on life-sustaining treatment preferences. Med Decis Making 
2006;26:313-22. 
22. 
Sinuff T, Dodek P, You JJ, et al. Improving End-of-Life Communication and Decision Making: The 
Development of a Conceptual Framework and Quality Indicators. J Pain Symptom Manage 2015;49:1070-80. 
23. 
National Quality Forum, Measure 1626: Patients Admitted to ICU who Have Care Preferences 
Documented within 48 hours (RAND Corporation). 2017. (Accessed January 2, 2018, at 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
15 
https://www.qualityforum.org/News_And_Resources/Endorsement_Summaries/Endorsement_Summaries.as
px.) 
24. 
Heyland DK, Barwich D, Pichora D, et al. Failure to engage hospitalized elderly patients and their 
families in advance care planning. JAMA Intern Med 2013;173:778-87. 
25. 
Tulsky JA, Beach MC, Butow PN, et al. A Research Agenda for Communication Between Health Care 
Professionals and Patients Living With Serious Illness. JAMA Intern Med 2017;177:1361-6. 
26. 
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: 
combining elements of clinical effectiveness and implementation research to enhance public health impact. 
Med Care 2012;50:217-26. 
27. 
Aslakson RA, Reinke LF, Cox C, Kross EK, Benzo RP, Curtis JR. Developing a Research Agenda for 
Integrating Palliative Care into Critical Care and Pulmonary Practice To Improve Patient and Family Outcomes. J 
Palliat Med 2017;20:329-43. 
28. 
Mandl KD, Khorasani R, Kohane IS. Meaningful use of electronic health records. Health Aff (Millwood) 
2012;31:1365; author reply 6. 
29. 
Curtis JR, Downey L, Back AL, et al. A patient and clinician communication-priming intervention 
increases patient-reported goals-of-care discussions between patients with serious illness and clinicians: a 
randomized trial. JAMA Intern Med 2018;178:930-40. 
30. 
Zerhouni EA. Clinical research at a crossroads: the NIH roadmap. J Investig Med 2006;54:171-3. 
31. 
Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in 
external validation and translation methodology. Eval Health Prof 2006;29:126-53. 
32. 
Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J 
Public Health 2013;103:e38-46. 
33. 
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion 
interventions: the RE-AIM framework. Am J Public Health 1999;89:1322-7. 
34. 
Harden SM, Gaglio B, Shoup JA, et al. Fidelity to and comparative results across behavioral 
interventions evaluated through the RE-AIM framework: a systematic review. Syst Rev 2015;4:155. 
35. 
Shoup JA, Gaglio B, Varda D, Glasgow RE. Network analysis of RE-AIM framework: chronology of the 
field and the connectivity of its contributors. Transl Behav Med 2015;5:216-32. 
36. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of 
health services research findings into practice: a consolidated framework for advancing implementation 
science. Implement Sci 2009;4:50. 
37. 
Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do 
it. BMJ 2013;347:f6753. 
38. 
Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation 
science for the non-specialist. BMC Psychol 2015;3:32. 
39. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual 
distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38:65-76. 
40. 
Curtis JR, Nielsen EL, Treece PD, et al. Effect of a quality-improvement intervention on end-of-life care 
in the intensive care unit: a randomized trial. Am J Respir Crit Care Med 2011;183:348-55. 
41. 
Curtis JR, Treece PD, Nielsen EL, et al. Integrating palliative and critical care: evaluation of a quality-
improvement intervention. Am J Respir Crit Care Med 2008;178:269-75. 
42. 
Curtis JR, Treece PD, Nielsen EL, et al. Randomized Trial of Communication Facilitators to Reduce 
Family Distress and Intensity of End-of-Life Care. Am J Respir Crit Care Med 2016;193:154-62. 
43. 
Curtis JR, Sathitratanacheewin S, Starks H, et al. Using Electronic Health Records for Quality 
Measurement and Accountability in Care of the Seriously Ill: Opportunities and Challenges. J Palliat Med 
2018;21:S52-S60. 
44. 
Hicks K, Downey L, Engelberg RA, et al. Predictors of Death in the Hospital for Patients with Chronic 
Serious Illness. J Palliat Med 2017. 
45. 
Lavin K, Davydow DS, Downey L, et al. Effect of Psychiatric Illness on Acute Care Utilization at End of 
Life From Serious Medical Illness. J Pain Symptom Manage 2017;54:176-85 e1. 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
16 
46. 
Sathitratanacheewin S, Engelberg RA, Downey L, et al. Temporal Trends Between 2010 and 2015 in 
Intensity of Care at End-of-Life for Patients With Chronic Illness: Influence of Age Under vs. Over 65 Years. J 
Pain Symptom Manage 2018;55:75-81. 
47. 
Brown CE, Engelberg RA, Sharma R, et al. Race/Ethnicity, Socioeconomic Status, and Healthcare 
Intensity at the End of Life. J Palliat Med 2018;21:1308-16. 
48. 
Steiner JM, Kirkpatrick JN, Heckbert SR, et al. Identification of adults with congenital heart disease of 
moderate or great complexity from administrative data. Congenit Heart Dis 2018;13:65-71. 
49. 
The Dartmouth Atlas of Healthcare. 2016. (Accessed June 1, 2018, at www.dartmouthatlas.org.) 
50. 
Wennberg JE, Bronner K, Skinner JS, Fisher ES, Goodman DC. Inpatient care intensity and patients' 
ratings of their hospital experiences. Health Aff (Millwood) 2009;28:103-12. 
51. 
Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK. Use of hospitals, physician visits, 
and hospice care during last six months of life among cohorts loyal to highly respected hospitals in the United 
States. BMJ 2004;328:607. 
52. 
Teno JM, Gozalo P, Khandelwal N, et al. Association of Increasing Use of Mechanical Ventilation Among 
Nursing Home Residents With Advanced Dementia and Intensive Care Unit Beds. JAMA Intern Med 
2016;176:1809-16. 
53. 
2017 Alzheimer’s Disease Facts and Figures. Alzheimer's Association, 2017. (Accessed March 14, 2018, 
at https://www.alz.org/facts/.) 
54. 
Abedini NC, Merel SE, Hicks KG, et al. Applying Human-Centered Design to Refinement of the 
Jumpstart Guide, a Clinician- and Patient-Facing Goals-of-Care Discussion Priming Tool. J Pain Symptom 
Manage 2021;62:1283-8. 
55. 
Back AL, Arnold RM, Baile WF, et al. Efficacy of communication skills training for giving bad news and 
discussing transitions to palliative care. Arch Intern Med 2007;167:453-60. 
56. 
Back AL, Arnold RM, Tulsky JA, Baile WF, Fryer-Edwards KA. Teaching communication skills to medical 
oncology fellows. J Clin Oncol 2003;21:2433-6. 
57. 
Sudore RL, Schillinger D, Knight SJ, Fried TR. Uncertainty about advance care planning treatment 
preferences among diverse older adults. J Health Commun 2010;15 Suppl 2:159-71. 
58. 
Curtis JR, Wenrich MD, Carline JD, Shannon SE, Ambrozy DM, Ramsey PG. Understanding physicians' 
skills at providing end-of-life care: Perspectives of patients, families, and health care workers. J Gen Intern Med 
2001;16:41-9. 
59. 
Steinhauser KE, Christakis NA, Clipp EC, McNeilly M, McIntyre LM, Tulsky JA. Factors considered 
important at the end of life by patients, family, physicians, and other care providers. JAMA 2000;284:2476-82. 
60. 
Lee RY, Brumback LC, Lober WB, et al. Identifying Goals of Care Conversations in the Electronic Health 
Record Using Natural Language Processing and Machine Learning. J Pain Symptom Manage 2021;61:136-42 e2. 
61. 
Khandelwal N, Brumback LC, Halpern SD, Coe NB, Brumback B, Curtis JR. Evaluating the Economic 
Impact of Palliative and End-of-Life Care Interventions on Intensive Care Unit Utilization and Costs from the 
Hospital and Healthcare System Perspective. J Palliat Med 2017;20:1314-20. 
62. 
Auriemma CL, Taylor SP, Harhay MO, Courtright KR, Halpern SD. Hospital-Free Days: A Pragmatic and 
Patient-centered Outcome for Trials among Critically and Seriously Ill Patients. Am J Respir Crit Care Med 
2021;204:902-9. 
63. 
Engelberg R, Downey L, Curtis JR. Psychometric characteristics of a quality of communication 
questionnaire assessing communication about end-of-life care. J Palliat Med 2006;9:1086-98. 
64. 
Au DH, Udris EM, Engelberg RA, et al. A randomized trial to improve communication about end-of-life 
care among patients with COPD. Chest 2012;141:726-35. 
65. 
Curtis JR, Engelberg RA, Nielsen EL, Au DH, Patrick DL. Patient-physician communication about end-of-
life care for patients with severe COPD. Eur Respir J 2004;24:200-5. 
66. 
Curtis JR, Patrick DL, Caldwell E, Greenlee H, Collier AC. The quality of patient-clinician communication 
about end-of-life care: A study of patients with AIDS and their primary care clinicians. AIDS 1999;13:1123-31. 
67. 
Janssen DJ, Curtis JR, Au DH, et al. Patient-clinician communication about end-of-life care for Dutch 
and US patients with COPD. Eur Respir J 2011;38:268-76. 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
17 
68. 
White DB, Angus DC, Shields AM, et al. A Randomized Trial of a Family-Support Intervention in 
Intensive Care Units. N Engl J Med 2018;378:2365-75. 
69. 
Teno JM, Fisher ES, Hamel MB, Coppola K, Dawson NV. Medical care inconsistent with patients' 
treatment goals: association with 1-year Medicare resource use and survival. J Am Geriatr Soc 2002;50:496-
500. 
70. 
Coast J, Huynh E, Kinghorn P, Flynn T. Complex Valuation: Applying Ideas from the Complex 
Intervention Framework to Valuation of a New Measure for End-of-Life Care. Pharmacoeconomics 
2016;34:499-508. 
71. 
Finkelstein EA, Bilger M, Flynn TN, Malhotra C. Preferences for end-of-life care among community-
dwelling older adults and patients with advanced cancer: A discrete choice experiment. Health Policy 
2015;119:1482-9. 
72. 
Flynn TN, Bilger M, Malhotra C, Finkelstein EA. Are Efficient Designs Used in Discrete Choice 
Experiments Too Difficult for Some Respondents? A Case Study Eliciting Preferences for End-of-Life Care. 
Pharmacoeconomics 2016;34:273-84. 
73. 
Appelbaum PS. Clinical practice. Assessment of patients' competence to consent to treatment. N Engl J 
Med 2007;357:1834-40. 
74. 
Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361-
70. 
75. 
Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. 
An updated literature review. J Psychosom Res 2002;52:69-77. 
76. 
Feng YS, Kohlmann T, Janssen MF, Buchholz I. Psychometric properties of the EQ-5D-5L: a systematic 
review of the literature. Qual Life Res 2021;30:647-73. 
77. 
Keetharuth AD, Hussain H, Rowen D, Wailoo A. Assessing the psychometric performance of EQ-5D-5L 
in dementia: a systematic review. Health Qual Life Outcomes 2022;20:139. 
78. 
Michalowsky B, Hoffmann W, Mohr W, Radke A, Xie F. Comparing the psychometric properties of EQ-
5D-3L and EQ-5D-5L proxy ratings by informal caregivers and a health professional for people with dementia. 
Health Qual Life Outcomes 2022;20:140. 
79. 
Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaboRATE: a fast and 
frugal patient-reported measure of shared decision making in clinical encounters. Patient Educ Couns 
2013;93:102-7. 
80. 
Barr PJ, Thompson R, Walsh T, Grande SW, Ozanne EM, Elwyn G. The psychometric properties of 
CollaboRATE: a fast and frugal patient-reported measure of the shared decision-making process. J Med 
Internet Res 2014;16:e2. 
81. 
Engelberg RA, Wenrich MD, Curtis JR. Responding to families' questions about the meaning of physical 
movements in critically ill patients. J Crit Care 2008;23:565-71. 
82. 
Engelberg RA, Downey L, Wenrich MD, et al. Measuring the quality of end-of-life care. J Pain Symptom 
Manage 2010;39:951-71. 
83. 
Howell AA, Nielsen EL, Turner AM, Curtis JR, Engelberg RA. Clinician perceptions of the role of an 
interprofessional communication facilitator to improve family outcomes in the ICU Am J Crit Care 2014;23:380-
6. 
84. 
Curtis JR, Engelberg R, Young JP, et al. An approach to understanding the interaction of hope and 
desire for explicit prognostic information among individuals with severe chronic obstructive pulmonary disease 
or advanced cancer. J Palliat Med 2008;11:610-20. 
85. 
Curtis JR, Engelberg RA, Wenrich MD, Au DH. Communication about palliative care for patients with 
chronic obstructive pulmonary disease. J Palliat Care 2005;21:157-64. 
86. 
Curtis JR, Engelberg RA, Wenrich MD, et al. Studying communication about end-of-life care during the 
ICU family conference: Development of a framework. J Crit Care 2002;17:147-60. 
87. 
Curtis JR, Engelberg RA, Wenrich MD, Shannon SE, Treece PD, Rubenfeld GD. Missed opportunities 
during family conferences about end-of-life care in the intensive care unit. Am J Respir Crit Care Med 
2005;171:844-9. 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
18 
88. 
Reinke LF, Engelberg RA, Shannon SE, et al. Transitions regarding palliative and end-of-life care in 
severe chronic obstructive pulmonary disease or advanced cancer: themes identified by patients, families, and 
clinicians. J Palliat Med 2008;11:601-9. 
89. 
Reinke LF, Shannon SE, Engelberg RA, Young JP, Curtis JR. Supporting hope and prognostic information: 
nurses' perspectives on their role when patients have life-limiting prognoses. J Pain Symptom Manage 
2010;39:982-92. 
90. 
West HF, Engelberg RA, Wenrich MD, Curtis JR. Expressions of nonabandonment during the intensive 
care unit family conference. J Palliat Med 2005;8:797-807. 
91. 
White DB, Engelberg RA, Wenrich MD, Lo B, Curtis JR. Prognostication during physician-family 
discussions about limiting life support in intensive care units. Crit Care Med 2007;35:442-8. 
92. 
Back AL, Young JP, McCown E, et al. Abandonment at the end of life from patient, caregiver, nurse, and 
physician perspectives: loss of continuity and lack of closure. Arch Intern Med 2009;169:474-9. 
93. 
Carline JD, Curtis JR, Wenrich MD, Shannon SE, Ambrozy DM, Ramsey PG. Physicians' interactions with 
health care teams and systems in the care of dying patients: Perspectives of dying patients, family members, 
and health care providers. J Pain Symptom Manage 2003;25:19-28. 
94. 
Curtis JR, Patrick DL, Caldwell E, Collier AC. Why don't patients with AIDS and their clinicians talk about 
end-of-life care? Barriers to communication for patients with AIDS and their primary care clinicians. Arch Intern 
Med 2000;160:1690-6. 
95. 
Curtis JR, Wenrich MD, Carline JD, Shannon SE, Ambrozy DM, Ramsey PG. Patients' perspectives on 
physicians' skills at end-of-life care: Differences between patients with COPD, cancer, and AIDS. Chest 
2002;122:356-62. 
96. 
Wenrich MD, Curtis JR, Ambrozy DM, et al. Provision of emotional support and personalized care by 
physicians to patients nearing the end of life. J Pain Symptom Manage 2003;25:236-46. 
97. 
Wenrich MD, Curtis JR, Shannon SE, Carline JD, Ambrozy DM, Ramsey PG. Communicating with dying 
patients within the spectrum of medical care from terminal diagnosis to death. Arch Intern Med 2001;161:868-
74. 
98. 
Little R, Rubin D. Statistical analysis with missing data. Third ed. New York: Wiley; 2019. 
99. 
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 
2005;15:1277-88. 
100. 
Corbin J, Strauss A. Methods of qualitative research. Thousand Oaks, CA: Sage Publications; 1993. 
101. 
Strauss AL, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing 
Grounded Theory. Thousand Oaks: Sage Publications; 1998. 
102. 
Attride-Stirling J. Thematic networks: An analytic tool for qualitative research. . Qualitative Research 
2001;1:385-405. 
103. 
Giacomini M, Cook DJ, for the Evidence-Based Medicine Working Group. Qualitative research in health 
care: What are the results and how do they help me care for my patients? JAMA 2000;284:478-82. 
104. 
Kross EK, Nielsen EL, Curtis JR, Engelberg RA. Survey burden for family members surveyed about end-
of-life care in the intensive care unit. J Pain Symptom Manage 2012;44:671-80. 
105. 
Glavan BJ, Engelberg RA, Downey L, Curtis JR. Using the medical record to evaluate the quality of end-
of-life care in the intensive care unit. Crit Care Med 2008;36:1138-46. 
106. 
Gries CJ, Engelberg RA, Kross EK, et al. Predictors of symptoms of posttraumatic stress and depression 
in family members after patient death in the ICU. Chest 2010;137:280-7. 
107. 
Gries CJ, Curtis JR, Wall RJ, Engelberg RA. Family member satisfaction with end-of-life decision making 
in the ICU. Chest 2008;133:704-12. 
108. 
Gerstel E, Engelberg RA, Koepsell T, Curtis JR. Duration of withdrawal of life support in the intensive 
care unit and association with family satisfaction. Am J Respir Crit Care Med 2008;178:798-804. 
109. 
Kross EK, Engelberg RA, Gries CJ, Nielsen EL, Zatzick D, Curtis JR. ICU care associated with symptoms of 
depression and posttraumatic stress disorder among family members of patients who die in the ICU. Chest 
2011;139:795-801. 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
19 
110. 
Curtis JR, Engelberg RA, Teno JM. Understanding variability in end-of-life care in the ICU for the elderly. 
Intensive Care Medicine 2017;43:94-6. 
111. 
Udelsman B, Chien I, Ouchi K, Brizzi K, Tulsky JA, Lindvall C. Needle in a Haystack: Natural Language 
Processing to Identify Serious Illness. J Palliat Med 2019;22:179-82. 
112. 
Lindvall C, Deng CY, Moseley E, et al. Natural Language Processing to Identify Advance Care Planning 
Documentation in a Multisite Pragmatic Clinical Trial. J Pain Symptom Manage 2022;63:e29-e36. 
 
 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
20 
APPENDIX 1: Trial 1 EHR-based clinician-facing Jumpstart Guide (PDF format) 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
21 
APPENDIX 2: Trial 1 EHR-based clinician-facing Jumpstart Guide (HTML format) 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
22 
APPENDIX 3: Trial 2 EHR-based clinician-facing Jumpstart Guide 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
23 
APPENDIX 4: Trial 2 Clinicians’ survey-based bi-directional Jumpstart Guide 
 

Study Protocol: PICSI-H  
PI: Erin Kross 
Page 
24 
APPENDIX 5: Trial 2 Patients’ survey-based bi-directional Jumpstart Guide