Patient-centered and Efficacious Advance Care Planning in Cancer: the PEACe Comparative Effectiveness Trial
Phase: NA
Conditions: Cancer, Advance Care Planning
Sponsor: University of Pittsburgh

 
Patient-centered and efficacious advance care planning in cancer: the 
PEACe comparative effectiveness trial 
Grant number: 1R01CA235730-01 
Protocol Number: STUDY19080337 
Principal Investigator: Yael Schenker, MD, MAS, FAAHPM 
ClinicalTrials.gov: NCT03824158 
 
 
April 17th, 2024

 
April 17th, 2024 
 
   
 
2 
 
STATEMENT OF COMPLIANCE 
The study will be conducted in accordance with the International Conference on 
Harmonisation guidelines for Good Clinical Practice (ICH E6), the Code of Federal 
Regulations on the Protection of Human Subjects (45 CFR Part 46), and the NIDCR Clinical 
Terms of Award. All personnel involved in the conduct of this study have completed human 
subjects’ protection training. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
3 
 
SIGNATURE PAGE 
The signature below constitutes the approval of this protocol and the attachments, and 
provides the necessary assurances that this trial will be conducted according to all 
stipulations of the protocol, including all statements regarding confidentiality, and according 
to local legal and regulatory requirements and applicable US federal regulations and ICH 
guidelines. 
Principal Investigator: 
Signed: 
 
Date:  
 
Name: Yael Schenker, MD, MAS 
 
Title: Principal Investigator  
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
4 
 
TABLE OF CONTENTS 
PAGE 
STATEMENT OF COMPLIANCE ................................................................................................ 2 
SIGNATURE PAGE .................................................................................................................... 3 
TABLE OF CONTENTS .............................................................................................................. 4 
LIST OF ABBREVIATIONS ......................................................................................................... 6 
PROTOCOL SUMMARY ............................................................................................................. 7 
SCHEMATIC OF STUDY DESIGN ............................................................................................. 9 
1. 
BACKGROUND AND RATIONALE .................................................................................. 10 
2. 
AIMS AND OBJECTIVES ................................................................................................. 11 
3. 
STUDY DESIGN .............................................................................................................. 12 
3.1 
Study Setting ........................................................................................................ 12 
3.2 
Eligibility Criteria ................................................................................................... 12 
3.2.1 
Patient eligibility criteria ............................................................................. 13 
3.2.2 
Caregiver eligibility criteria ......................................................................... 14 
3.2.3 
Clinician eligibility criteria........................................................................... 14 
3.3 
Recruitment Methods ............................................................................................ 14 
3.4 
Consent Procedures ............................................................................................. 16 
3.5 
Intervention Arms .................................................................................................. 17 
3.6 
Randomization Procedures ................................................................................... 18 
3.7 
Blinding ................................................................................................................. 18 
3.8 
Intervention Fidelity ............................................................................................... 19 
3.9 
Data Collection Methods ....................................................................................... 20 
3.10 
Follow-up and Retention ....................................................................................... 20 
3.11 
Withdrawal ............................................................................................................ 21 
4. 
MEASURES AND ASSESSMENTS ................................................................................. 22 
4.1 
Overview ............................................................................................................... 22 
4.2 
Primary Outcome .................................................................................................. 22 
4.3 
Secondary Patient and Caregiver Outcomes ........................................................ 22 
4.4 
Healthcare Utilization Outcomes ........................................................................... 24 
4.5 
In-depth Interviews ............................................................................................... 24 
4.6 
Additional Measures ............................................................................................. 25 
5. 
STATISTICAL ANALYSIS PLAN ...................................................................................... 27 
5.1 
Baseline Data ....................................................................................................... 27 
5.2 
Primary and Secondary Outcomes ....................................................................... 27 
5.3 
Handling of Missing Values ................................................................................... 28 
5.4 
Interim & Final Analyses ....................................................................................... 28 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
5 
 
5.5 
Sample Size and Power Calculations ................................................................... 28 
5.6 
Qualitative Analysis Plan ...................................................................................... 29 
6. 
MONITORING .................................................................................................................. 30 
6.1 
Human Subjects Protections ................................................................................. 30 
6.2 
Minimizing Risks to Participants ............................................................................ 30 
6.3 
Data and Safety Monitoring Plan .......................................................................... 32 
6.3.1 
Adverse Event Monitoring ......................................................................... 32 
7. 
REFERENCES ................................................................................................................. 34 
8. 
APPENDIX ....................................................................................................................... 38 
Telephone Safety Protocol ............................................................................................... 38 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
6 
 
LIST OF ABBREVIATIONS 
ACP 
Advance care planning 
AD 
Advance directive 
CEQUEL 
Caregiver evaluation of the quality of end-of-life care 
CITI 
Centralized IRB Training Initiative 
CONSORT 
Consolidated Standards of Reporting Trials 
ECOG PS 
Eastern Cooperative Oncology Group – Performance Status 
EOL 
End of life 
eSYSDM 
Electronic system for data management 
GLMM 
Generalized linear mixed model 
HADS 
Hospital Anxiety and Depression Scale 
HRPO 
Human Research Protection Office 
ICU 
Intensive care unit 
IRB 
Institutional Review Board 
NQF 
National Quality Forum 
PI 
Principal investigator 
PTSD 
Post-traumatic stress disorder 
QOPI 
Quality Oncology Practice Initiative 
RN 
Registered nurse 
SAE 
Serious adverse event 
SW 
Social worker 
UAP 
Unanticipated problem 
UPMC 
University of Pittsburgh Medical Center 
 
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
7 
 
PROTOCOL SUMMARY 
Title: 
Patient-centered and efficacious advance care planning in cancer: the 
PEACe-compare trial 
 
Précis: 
 
PEACe-compare is a single-blind, patient-level randomized trial to 
compare the effectiveness of facilitated ACP vs web-based ACP among 
patients with advanced cancer and their family caregivers. 
 
Objectives: 
 
 
Aim 1: To compare the effectiveness of facilitated ACP versus web-
based ACP on patient and family caregiver outcomes. 
 
Aim 2: To assess implementation costs and the effects of facilitated 
ACP and web-based ACP on healthcare utilization at end of life. 
 
Aim 3: To identify contexts and mechanisms that influence the 
effectiveness of facilitated ACP versus web-based advance care 
planning. 
 
Population: 
 
400 adult patients with advanced cancer and their caregivers 
 
Sites: 
 
The study will be conducted at UPMC Hillman Cancer Center clinics in 
Western Pennsylvania. 
 
Description of  
Interventions: 
 
Facilitated ACP will be conducted using the Respecting Choices® 
facilitated ACP conversation model. Patients randomized to this arm 
will participate in an in-person or remote (telephonic or video) facilitated 
ACP discussion led by a trained facilitator using a structured interview 
tool designed for adults with chronic, incurable illness. 
 
Web-based ACP will use the PREPARE for Your Care tool, an online 
ACP tool that includes 5 ACP steps using interactive videos. Patients 
randomized to this arm will view PREPARE on a home computer or on 
a tablet in the oncology office. 
 
Study Duration: 
 
5 years 
 
 
Subject 
Participation 
Duration: 
 
 
 
 
 
up to 5 years 
 
Estimated Time to 
Complete 
Enrollment: 
 
 
 
3 years 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
8 
 
FUNDING 
This trial is funded through a National Cancer Institute R01 grant: 1R01CA235730-01. 
 
CLINICALTRIALS.GOV INFORMATION  
This trial is registered at ClinicalTrials.gov: NCT03824158, registered on January 31, 2019. 
 
ETHICS AND ADVISORY COMMITTEES 
The study is approved by the University of Pittsburgh IRB: STUDY19080337.  
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
9 
 
SCHEMATIC OF STUDY DESIGN 
  

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
10 
 
1. BACKGROUND AND RATIONALE  
Failure to deliver care near the end of life (EOL) that reflects the needs, values and preferences of 
patients with advanced cancer remains a key shortcoming of our cancer care delivery system.1, 2 
National organizations including the National Academy of Medicine and American Society of Clinical 
Oncology have called for increased attention to advance care planning (ACP) to improve patient-
centered care near EOL.1-4 A critical barrier to progress in the field is a lack of evidence about the 
most effective and efficient ACP strategy to improve treatment decisions near EOL and ensure 
patients’ wishes are honored. Two widely-used patient-facing ACP interventions entail sizeable 
differences in costs and complexity to deploy: (1) discussions with trained facilitators5 and (2) web-
based advance care planning using interactive videos.6, 7 These approaches have never been 
compared directly; it is therefore unclear whether one form of ACP is more potent—and if so, for 
whom and under what circumstances.8, 9 Understanding the relative effectiveness of facilitated 
versus web-based ACP is important because facilitated ACP requires far more resources. This 
proposal addresses this evidence gap, using mixed methods to compare the effectiveness of 
facilitated ACP vs web-based ACP among 400 patients with advanced cancer and their caregivers.  
 
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
11 
 
2. AIMS AND OBJECTIVES 
The purpose of this study is to compare the effectiveness and implementation costs of two different 
patient-facing ACP interventions among patients with advanced cancer and their caregivers: (1) 
facilitated ACP using the Respecting Choices® model versus (2) web-based ACP using the 
PREPARE online ACP tool. The study includes longitudinal outcomes that define successful 
advance care planning based on a recent international consensus panel10 and incorporates a 
qualitative evaluation to identify factors that influence ACP effectiveness.  
 
Aim 1 will compare the effectiveness of facilitated ACP versus web-based ACP on patient and family 
caregiver outcomes. Using a validated, patient-reported measure, we will compare the interventions’ 
impact on ACP engagement at 12 weeks (1° outcome), ACP discussions with caregivers and 
clinicians, advance directive (AD) completion, and caregiver symptoms of depression and anxiety. 
For patients who die during the study period, we will use validated, caregiver-reported measures to 
compare perceived quality of EOL care, patients’ EOL goal attainment, and bereaved caregiver 
symptoms of depression, anxiety, and post-traumatic stress. 
 
Aim 2 will assess implementation costs and the effects of facilitated ACP and web-based ACP on 
healthcare utilization at end of life. To inform future dissemination efforts and aid in understanding 
optimal financing models, we will calculate implementation costs of each intervention and determine 
effects on healthcare utilization at EOL, including hospitalizations, chemotherapy use, and hospice 
use. 
 
Aim 3 will identify contexts and mechanisms that influence the effectiveness of facilitated ACP 
versus web-based ACP. We will conduct a qualitative evaluation, using in-depth interviews with 
patients, caregivers, and clinicians, and perform thematic analysis to understand for whom, how, and 
in what circumstances facilitated ACP versus web-based ACP are most effective. 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
12 
 
3. STUDY DESIGN 
The study is a single-blind, patient-level randomized trial, using mixed methods to compare the 
effectiveness of facilitated ACP versus web-based ACP among 400 patients with advanced cancer 
and their caregivers. 
 
3.1 Study Setting  
The trial will be conducted with patients seen at the UPMC Hillman Cancer Center, one of the largest 
integrated community networks of oncologists in the United States with over 60 locations throughout 
Western Pennsylvania, Ohio, and New York. Within this network, we will focus recruitment at up to 
six sites representing a mix of academic and community practice models and a range of clinic sizes.  
 
3.2 Eligibility Criteria 
We designed patient eligibility criteria to be broadly representative of the population with cancer for 
whom ACP is most important. There are no inclusion or exclusion criteria based on gender, race, or 
ethnicity. There is a lack of good evidence regarding the optimal timing of ACP in advanced cancer11, 
12, but in most prior studies patients and clinicians preferred to delay the introduction of ACP to later 
in the illness trajectory rather than at the time of diagnosis.13-15 The validated ‘would not be surprised’ 
question is a simple and effective method for identifying patients with cancer at high risk of dying in 
one year (hazard ratio 7.8).16, 17 We do not include patients with hematologic malignancies because 
these diseases have very different trajectories and treatment options near the end of life,18, 19 and the 
majority of patients seen at participating practices have solid tumors. We include patients who have 
previously completed an AD because preferences change and patients may benefit from additional 
engagement in ACP with changing health contexts.12 We exclude patients < 18 because children 
with incurable cancer have unique ACP needs best served by clinicians experienced working with 
pediatric populations, and our study is conducted at adult oncology practices. Our teach-back 
method for informed consent is designed to ensure complete comprehension among vulnerable 
populations.20 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
13 
 
3.2.1 Patient eligibility criteria 
Inclusion Criteria 
18 years of age or older 
Solid tumor 
The oncologist ‘would not be surprised’ if the 
patient died within the next year 
Eastern Cooperative Oncology Group 
performance status (ECOG PS) of 0, 1, or 2 
Planning to receive ongoing care at a 
participating oncology clinic 
Willing to participate in either a web-based or 
facilitated program 
Exclusion Criteria 
Does not speak English 
Inability to consent, using a validated teach-
back method. 
Hematologic malignancy 
No phone for additional study contacts and 
follow-up interviews 
Unable to participate in advance care 
planning, as assessed by clinician 
Unable to complete the baseline interview 
 
 
 
 
 
 
 
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
14 
 
3.2.2 Caregiver eligibility criteria 
Inclusion Criteria 
18 years of age or older 
Family member or friend of an eligible patient 
Primary person involved in patient’s care and 
best able to participate in the study, as 
assessed by patient 
Exclusion Criteria 
Does not speak English 
No phone for additional study contacts and 
follow-up interviews 
Unable to complete the baseline interview 
3.2.3 Clinician eligibility criteria 
Inclusion Criteria 
Clinician of consented patients 
 
3.3  Recruitment Methods 
To facilitate recruitment, we use a process for identifying and screening study participants that has 
been IRB-approved and successful in our prior work.21-25 Research staff at each clinic will review 
administrative lists and clinic schedules weekly with oncologists or their designated representatives 
to identify potentially eligible patients with solid tumors for whom the oncologist ‘would not be 
surprised’ if the patient died within one year. These criteria are not shared with patient and caregiver 
participants. Patients who meet initial criteria will be approached in-person by a clinical member of 
their oncology team who is already involved in their clinical care and offered a study recruitment 
brochure, or securely referred to the study team remotely and then contacted via the telephone for 
study recruitment.  
For in-person recruitment, the referring clinician will obtain verbal permission from the patient before 
introducing a member of the research team. The research team will document that verbal permission 
has been obtained before approaching a patient. We will not introduce the study at the first 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
15 
 
appointment in which a new diagnosis of metastatic disease is discussed, because this is a time of 
heightened anxiety when patients are unlikely to be receptive to ACP research.15, 26 If verbal 
permission is obtained from the patient, this introduction from a member of the clinical team will be 
followed by a detailed, in-person explanation of the study from a trained research staff member. 
Patients may decline to hear more about the study and will be informed that their decision will in no 
way affect their medical care.  
 
Patients referred to the study team remotely will be mailed a letter by the study team, on behalf of 
their oncologist. This letter will include a brief introduction to the study, inform the patient that they 
may be eligible to participate, and include instructions to call or email the study coordinator if they do 
not wish to hear further about the study. If patients do not contact the study coordinator within one 
week after the letter is sent, a trained research assistant or the study coordinator will call to review 
key study highlights and assess interest in participating. 
 
A second invitation to participate will be offered to patients who decline initial participation (for 
example, because they are feeling too overwhelmed), but give permission to be re-contacted (either 
in person or by phone) at a later date. Study staff will review the tracking system weekly to maintain 
recruitment fidelity. 
 
Patients who have provided consent to participate will be asked to identify a potential caregiver 
participant, designated by the patient as the primary person involved in their care and best able to 
participate in the study. Patients will be encouraged to contact the potential caregiver in advance of 
the research team to avoid any impression of cold calling. All caregiver participants will provide 
informed consent (either written or verbal option). Patients unable to identify a caregiver will not be 
excluded because these patients also face treatment decisions near EOL and may benefit from 
ACP. 
 
Patients participating in an in-depth interview (Aim 3) will be asked to identify the clinician most 
involved in decisions about their care. Research staff will approach clinicians to introduce the study 
and offer a study information sheet. The consent discussion will be done at a time that is convenient 
for the clinician. All clinicians participating in an in-depth interview (Aim 3) will provide informed 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
16 
 
consent. 
 
3.4  Consent Procedures 
All consent forms and procedures have been approved by the University of Pittsburgh Institutional 
Review Board. 
 
Patient participants will provide written informed consent using the validated teach-back method. 
This is a modified informed consent process developed for vulnerable and low-literate populations.20 
The process involves: using a consent form written below a 6th-grade reading level and in a 14-point 
font; reading highlights from the consent form to potential subjects; allowing time for questions and 
discussion; allowing participants to read the form themselves, if desired; and verifying 
comprehension through repeated and targeted education until comprehension is achieved. If 
subjects require > 3 passes through the comprehension assessment, they will be deemed ineligible. 
Only after comprehension will we ask patient participants to sign the consent form. Patients recruited 
in-person by the study team will be asked to sign a paper copy of the consent form. Patients 
recruited remotely will be given three options to complete written consent: 1) paper consent form 
provided via the mail, 2) electronic consent form provided via email, and 3) electronic consent form 
sent through DocuSign, signed via electronic signature. All recruited study patients will be provided a 
copy of the completed consent form for their records.  
 
Caregiver participants will also provide informed consent (either written or verbal option) and may 
participate in the teach-back with the patient if they are present for this discussion.  
 
The study staff will consent clinicians verbally at a time that is convenient for them. Study staff will 
conduct a consent discussion with the clinician describing study procedures, anticipated risks and 
benefits, emphasizing the voluntary nature of participation, and eliciting any questions or concerns. 
After all questions have been addressed to the clinician’s satisfaction, study staff will request verbal 
consent. Consent will be documented in the study's secure electronic database. 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
17 
 
3.5  Intervention Arms 
Facilitated advance care planning arm 
Participants randomized to the facilitated ACP arm will participate in Respecting Choices®, a 
facilitated ACP conversation designed for adults with chronic incurable illness. Conversations are led 
by a nurse or social worker with supportive oncology experience who has been trained as a 
Respecting Choices® facilitator. Each patient will participate in up to three face-to-face, video, or 
telephonic visits, lasting 45-60 minutes each. The facilitator initiates contact with a call to each 
patient to establish rapport, determine the preferred visit mode, schedule the visit, and facilitate 
overall intervention participation. Visits will include the patient’s medical decision maker (health care 
agent), when available, and any other members of the family and/or caregivers the patient wishes to 
include. Visits will be held in a private location (oncology clinic, patient’s home, or research office) 
and audio-recorded for fidelity monitoring. During the ACP conversation visit, facilitators will use the 
“First Steps® ACP Conversation Guide: Adults with Chronic Illness” structured interview tool as a 
discussion roadmap and provide guidance in choosing a medical decision maker, exploring serious 
illness understanding and experiences, identifying goals and values, and making future treatment 
decisions. Facilitators will provide a copy of the AD used throughout UPMC, assist with completion 
and uploading to the patient’s electronic medical record, when appropriate, and make 
recommendations for communicating goals and sharing written preferences.  
 
 
Web-based advance care planning arm 
Participants randomized to the web-based ACP arm will use the PREPARE web-based ACP tool 
(available at https://prepareforyourcare.org) PREPARE is literacy- and culturally-appropriate and 
HIPAA-compliant. Instructions for accessing and using a secure, PEACe-compare study version of 
the PREPARE website, as well as a copy of the AD used throughout UPMC, will be shared with 
participants upon randomization. Patients and their caregivers review the 5 steps of PREPARE 
(approximately 10 minutes per step) and create an action plan for each step. After completing the 
steps, patients have the option of printing out a Summary and Action Plan. The website can be 
reviewed on a home computer or on a tablet in the patient’s oncology practice. Website analytics will 
be provided to the research staff to gauge patient participation. Patients who have not logged on to 
the website will be called by study staff to assess and address any technical difficulties. During the 
check-in call, study staff will offer three options to participants in order to help them overcome any 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
18 
 
practical or technological barriers preventing the patient from accessing the website: 1) to review the 
website on a study-supplied tablet during the patient’s next oncology clinical visit, 2) to help walk 
them through signing up for the website initially, and 3) setting a goal date to review the website. If 
patients do not choose any of these options, they will be reminded that accessing the website is part 
of their study participation, and will be called again in a few weeks if they have not logged on to the 
website. 
3.6  Randomization Procedures 
Upon completion of baseline patient and caregiver interviews, patient-caregiver pairs will be 
randomized 1:1 at the patient level to receive either the facilitated ACP or the web-based ACP 
intervention. We chose patient-level randomization, consistent with most prior ACP trials, because 
both interventions are at the patient level and do not involve clinician- or system-level changes. The 
risk of contamination is extremely low because both interventions will be conducted in private and 
are unlikely to influence systematic changes in physician practices. To further ensure against any 
potential contamination, we will ask enrolled participants and caregivers not to discuss either 
intervention with other patients or caregivers once they are told which group they have been 
randomized to. 
 
The randomization sequence will be generated by the lead statistician who will not be involved in 
study recruitment or treatment allocation. The statistician will use R (R Foundation for Statistical 
Computing, Vienna, Austria) to generate a randomization sequence that will be fed into the web-
based database portal, so that each patient-caregiver pair will be randomly assigned to a group upon 
completion of their baseline interview. 
 
3.7  Blinding 
All outcome assessments will be conducted by blinded research staff who are not involved in ACP 
interventions or follow-up contacts to assess intervention completion. At the start of each outcome 
assessment interview, participants are reminded not to discuss study materials with research staff. It 
is not possible to blind patient and caregiver participants. However, they are told during the consent 
process that there is a “50/50 chance” of getting one of two different tools to help patients have a 
voice in their care, and patient/caregiver participants provide informed consent before randomization. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
19 
 
We obtain clinicians’ permission to recruit their patients but do not share information on 
randomization or involve clinicians in intervention administration. However, it is possible that 
participants will discuss ACP participation with their clinicians. 
3.8  Intervention Fidelity 
Training of Respecting Choices® ACP facilitators is overseen by experienced Respecting Choices® 
staff members. We train nurses (RNs) or social workers (SWs) because these non-physician 
members of the clinical team are typically trained as ACP facilitators and are widely available in 
outpatient oncology settings, maximizing feasibility of future dissemination. We select facilitators who 
do not provide other clinical duties at participating sites to guard against potential contamination. 
Clinical leaders at participating sites are asked to meet with and approve ACP facilitators to ensure 
oncology-relevant experience and mutual trust. Selected facilitators complete on-line prerequisite 
modules and attend a competency-based, in-person Respecting Choices® ACP Facilitator 
Certification program. Participants demonstrate competency in ACP facilitation through role play 
demonstration in small group observation and feedback. Participants develop a plan for 
improvement of identified skill deficits as needed.  Upon completion of training, facilitators will 
conduct several mock ACP sessions using case scenarios provided by Respecting Choices® while 
being observed by trainers to certify acquisition of key skills.  To ensure consistent delivery of 
facilitated ACP at participating sites, identify any problems in delivery that may require immediate 
remediation or modifications in future settings, and reduce drift in adherence to ACP protocols over 
time, all ACP sessions conducted during the study are audio-recorded. Each month, one to two 
audio-recordings per facilitator are reviewed and scored for fidelity to key constructs per Respecting 
Choices® evaluation standards. Feedback on reviewed sessions is discussed during regular 
intervention team meetings and with a consultant from Respecting Choices®. 
 
We monitor fidelity in the PREPARE intervention arm by using analytics from a secure, study-
specific website that allows us to track the number of log-ins and number of ACP steps completed by 
each participant. Analytics data will be examined by an un-blinded research staff member. 
 
Participants in both arms will receive check-in calls between baseline and the 12 week follow-up. 
Participants in the Respecting Choices® arm will be called to schedule the Respecting Choices® 
intervention visits with an ACP Facilitator. Participants in the PREPARE arm will be called (up to 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
20 
 
three times) to assess for and troubleshoot any participation issues if they have not logged on to the 
PREPARE website. 
 
All contacts with study participants follow standardized study scripts.  All staff members are 
rigorously trained to follow study operating procedures and are not permitted to conduct study tasks 
independently until they have been observed and demonstrated mastery. 
 
3.9  Data Collection Methods 
We will use a secure electronic System for Data Management (eSYSDM) for all data entry and 
management. To minimize participant burden and disruption to clinical activities, baseline surveys 
may be completed either in-person, by telephone, or by mail. Medical record reviews will be 
conducted by blinded clinical research staff. Twelve-week and bereavement assessments will be 
conducted via telephone by blinded research assistants trained in study procedures. In the event a 
participant is unable to complete the interview by telephone or unreachable after seven attempts, a 
paper copy will be mailed. All data from paper documents will be entered into the eSYSDM by 
research staff and verified using standardized data entry procedures. Original paper source 
documents will be filed and locked in a filing cabinet accessible only by study staff members. 
 
3.10 Follow-up and Retention 
Follow-up assessments will be conducted with patients and caregivers at 12 weeks, and with 
caregivers at bereavement (12 weeks or later after patient’s death). To enhance retention, we 
distribute contact cards with the contact information for study staff, mail reminder cards, mail 
answer guides prior to telephone assessments, and send condolence cards to referred family 
caregivers when a patient dies. Our data collection strategy is designed to minimize participant 
burden by using parsimonious measures previously pilot-tested with this population, administering 
patient and caregiver interviews by phone, and providing compensation. Enrolled patients and 
caregivers will be paid $25 each upon completion of the 12-week follow-up assessment. Enrolled 
caregivers will be paid $25 each upon completion of the bereavement assessment, if applicable. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
21 
 
Enrolled patients and caregivers will be paid an additional $25 each if asked to complete the 
qualitative in-depth interview. 
 
3.11 Withdrawal 
Participants may withdraw from the study at any time. All previously collected information will 
continued to be used for study purposes until study completion. For participants who wish to 
withdraw, we will ask them why. For participants who do not have a response to this open-ended 
question, we will offer a list of reasons for withdrawal from previous studies (e.g., not enough time, 
no longer interested, no benefit, etc). 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
22 
 
4. MEASURES AND ASSESSMENTS 
4.1  Overview 
Successful ACP influences multiple important outcomes.  We selected priority outcomes related to 
patients, caregivers, and healthcare utilization based on an organizing framework developed by an 
international panel of ACP experts.10 We focused on outcomes for which validated and parsimonious 
measures have been developed for use with seriously-ill participants. We chose a 12-week time-
point for patient and caregiver assessments to balance the need to allow sufficient time for ACP to 
occur without incurring excessive loss to follow up in a seriously-ill population. For patients who die 
during the study period, we will conduct brief additional interviews with bereaved caregivers 12 
weeks or later after a patient’s death, allowing assessment of EOL care and caregiver bereavement 
adjustment while minimizing participant burden. Additionally, we will review medical records for 
documented care goals and EOL healthcare utilization. 
 
4.2  Primary Outcome 
We chose patient-reported engagement in ACP as our primary outcome because this validated 
measure reflects a patient- and caregiver-centered understanding of ACP as a complex process 
involving multiple behaviors, rather than simply completion of an advance directive. Importantly, the 
15-item version of the ACP Engagement Survey measures a broad range of ACP behaviors, is able 
to detect change in response to ACP interventions, and can be administered with minimal participant 
burden.27 
 
4.3  Secondary Patient and Caregiver Outcomes 
We will additionally measure ACP discussions with caregivers, ACP discussions with clinicians, and 
AD completion (using both a single question and by reviewing medical records for any new ACP 
documentation).  For patients who die during the study period, we chose caregiver-reported 
measures of goal-concordant EOL care because these validated measures capture overall receipt of 
goal-concordant care in the last month of life and incorporate the perspectives of bereaved family 
members. In contrast, measuring receipt of goal-concordant EOL care based on chart review does 
not account for frequently-encountered situations in which documented ADs do not reflect the 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
23 
 
current, real-time clinical scenario. Similarly, we chose a validated, caregiver-reported measure of 
quality of EOL care because cancer caregivers provide an important and reliable perspective on this 
outcome.28, 29 We include measures of caregiver depression, anxiety, and post-traumatic stress 
symptoms because many patients cite easing the burden on families as a motivator of ACP and it is 
possible that family caregivers may experience less distress after participating in a facilitated ACP 
intervention. 
 
Patient and Family Caregiver Outcomes (Aim 1) 
Specific 
Measure 
Description 
Timing 
ACP Engagement 
(primary 
outcome) 
15-item ACP engagement survey assessing ACP processes related to choosing a medical 
decision maker, discussing and documenting preference for care at EOL, flexibility for 
surrogate decision making, and asking questions of medical providers. Single summary 
score (range 0-5 with higher scores indicating higher engagement) has high internal 
consistency (Cronbach’s alpha=0.92) and responsiveness to change.27 
Baseline, 
12 weeks 
ACP discussions 
with caregivers 
“Have you talked with your family or friends about the kind of medical care you would 
want if you were very sick or near the end of life?”27, 30, 31 
Baseline, 
12 weeks 
ACP discussions 
with physicians 
“Have you talked with your doctor about the kind of medical care you would want if 
you were very sick or near the end of life?”27, 30, 31 
Baseline, 
12 weeks 
AD completion 
Single question will assess AD completion. We will additionally assess documented care 
goals by reviewing medical records for any new ACP documentation, including AD forms 
or documented discussion. 
Baseline, 12 
weeks 
Receipt of goal- 
concordant EOL 
care 
Two validated caregiver-reported questions have been used to measure receipt of goal-
concordant EOL care for patients with cancer: (1) “In your opinion, to what extent were 
[the patient’s] wishes followed in the medical care received in the last month of life?”; 
and (2) Caregivers asked about patient’s preferred and actual places of death, with 
questions separated in the survey to minimize conscious comparison.32 
Bereavement 
Quality of EOL 
care 
The Caregiver Evaluation of Quality of End-of-Life Care (CEQUEL) scale is a 13- item 
instrument (range of scores 13-26) with demonstrated reliability and convergent 
validity among cancer caregivers.28  Strengths include brevity and clinical relevance, with 
inclusion of 4 distinct but related factors (prolongation of death, perceived suffering, 
shared decision-making, and preparation for death) associated with caregiver 
bereavement outcomes.28 
Bereavement 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
24 
 
Depression and 
anxiety 
symptoms 
Hospital Anxiety and Depression Scale (HADS) is a widely-used 14-item instrument 
measuring symptoms of depression and anxiety.33 It has been extensively validated for 
screening emotional distress among advanced cancer patients34 and family members.35  
Score of ≥ 8 on either domain indicate significant symptoms of depression or anxiety 
with good sensitivity and specificity.36 
Baseline, 12 
weeks, 
Bereavement 
Post-traumatic 
stress symptoms 
Impact of Events Scale is a 22-item self-report instrument for PTSD symptoms with 3 
subscales measuring intrusion (8 items), avoidance (8 items) and hyperarousal (6 
items).37 Subscales have high internal consistency (Cronbach’s alpha ranging from 0.79-
0.92).38 A score of 30 or higher is considered a clinically significant burden. 
Bereavement 
 
 
4.4  Healthcare Utilization Outcomes 
Using methods refined in our previous work,21 we will assess healthcare utilization at the end of life 
via caregiver bereavement interviews and medical record review, allowing us to accurately assess 
use for participants hospitalized in multiple hospital systems.39 We will measure intensity of EOL 
care using the American Society of Clinical Oncology’s Quality Oncology Practice Initiative (QOPI)40 
and National Quality Forum (NQF)41-endorsed measures: chemotherapy administered within the last 
2 weeks of life; hospitalizations (>1) or ICU admission in the last month of life; >1 emergency room 
visit in the last month of life; hospice enrollment; hospice enrollment within 3 days of death; and 
number of days in hospice.  
 
We will additionally assess ACP implementation costs by tracking staff time spent on each 
intervention arm. Staff time costs for each intervention will be estimated by multiplying staff training 
and patient care time related to the intervention in hours by the average hourly wage for US nursing 
and social work staff of comparable levels.42 
 
4.5  In-depth Interviews 
We will conduct in-depth interviews with a subset of patients, caregivers, and clinicians to identify 
contexts and mechanisms that influence ACP effectiveness.  These interviews are based on a realist 
evaluation paradigm, drawing on qualitative methods to investigate factors and conditions under 
which certain outcomes will or will not be realized.43 This contextual evaluation expands on the 
comparative effectiveness data generated in Aims 1 and 2 to answer the guiding questions (1) “what 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
25 
 
makes each ACP intervention work,” and (2) “for whom and under what circumstances is each ACP 
intervention most effective.”44 
 
Patients and caregivers will be invited to participate in in-depth interviews after completing trial 
outcome assessments (12-week and bereavement, respectively) to avoid impacting trial integrity. 
We will recruit equal numbers of participants from each ACP intervention arm. Within each 
intervention arm, we will include participants for whom the intervention was succesful (defined as an 
increase in ACP engagement at 12 weeks) and not successful (defined as no increase in ACP 
engagement at 12 weeks). This sampling strategy will ensure elicitation of  a spectrum of 
experiences regarding what makes each intervention work (or not work) for changing ACP 
behaviors. We will aim for linked patient-caregiver pairs because caregivers may lend additional 
perspective on patient EOL experiences. Because ACP also involves clinicians and healthcare 
systems, we will ask patients to identify a clinician most involved in decisions about their care and 
interview these stakeholders after patient and caregiver interviews have been completed. As 
recommended for qualitative research, the sample size is not fixed. Rather, we will conduct 
interviews until thematic saturation is reached, meaning no new themes emerge from the data. 
Based on our prior work, we anticipate that this will involve at least 60 patients and 60 caregivers 
and approximately 60 clinicians.45-48 
 
Interviews will be conducted using semi-structured interview guides informed by (1) a theoretical 
framework for ACP based on behavior change theory and (2) the literature on factors that may affect 
the ability of ACP interventions to improve ACP outcomes. Interview guides will be developed by a 
multidisciplinary group (including patient and caregiver representatives) and piloted with 2-3 
representative stakeholders prior to use.  All interviews will be conducted by an experience 
qualitative researcher with extensive in-depth interviewing experience, audio-recorded, and 
transcribed verbatim.  Interview questions will be modified as new themes emerge from the data. 
 
4.6  Additional Measures 
We will collect basic demographic data from participants at time of enrollment, including age, sex, 
race/ethnicity, religion, social support (8-item modified Medical Outcomes Study Social Support 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
26 
 
Survey49), health literacy (single-item question validated among diverse patient populations50), length 
of patient relationship with oncologist, cancer type, and performance status. We will additionally 
assess illness understanding,29, 51 care preferences,52 past ACP and decision-making experiences 
(prior surrogate, prior experiences making decisions for self or others),53 distress (single-item 
distress thermometer),54 and decisional certainty (decisional conflict scale).55  
 
As recommended for trials of complex interventions, we will collect process data to understand ACP 
implementation in both arms.56, 57 We will track the timing, length, and content of ACP using 
standardized checklists and website usage data.  
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
27 
 
5. STATISTICAL ANALYSIS PLAN 
5.1  Baseline Data 
We will evaluate the statistical properties of baseline and follow-up outcome measures, including 
potential outliers, normality, and missing data, using summary statistics and graphical tools. 
Participant demographics and clinical characteristics will be presented as frequency (%) for 
categorical variables and mean (standard deviation) for continuous variables (or median and range 
for continuous variables with heavily skewed distributions). Results will be reported following the 
CONSORT guideline. 
 
5.2  Primary and Secondary Outcomes 
All analyses for treatment group comparisons will use an intention-to-treat approach.  
 
The primary outcome is ACP engagement. We will test the effect of treatment assignment on the 
primary outcome using linear mixed models. The model will include treatment group, and baseline 
ACP engagement as fixed effects. To adjust for possible clustering of patients seen by the same 
facilitator or provider, we will include random effects for facilitators (for patients in the facilitated ACP 
group only) and oncologists.  
 
The same analytic approach will be used for secondary outcomes: perceived quality of EOL care 
and caregiver symptoms of anxiety, depression and PTSD. Binary outcomes such as ACP 
discussions, AD completion, documented care goals, and receipt of goal-concordant EOL care will 
be compared using mixed effect logistic regression with the same set of fixed effects and random 
effects as in the models for continuous outcomes. We will also compare the rates of completion of 
intervention between the two intervention groups, using the same model as for other binary 
outcomes.  
 
Analyses of health care utilization data will use GLMM (generalized linear mixed model), with logit 
link (binary distribution) for dichotomous (yes/no) health care outcomes such as chemotherapy within 
last 2 weeks of life. Again, we will use intervention group and baseline ACP as fixed effects and 
oncologist and clinic as random factors. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
28 
 
 
5.3  Handling of Missing Values 
We will prevent and monitor missing data by using an eSYSDM, which will automatically produce an 
error message notifying the user that the field must be completed when required fields are left blank. 
The eSYSDM also performs important tracking duties by monitoring screening information, eligibility 
status, follow-up interviews due, and study group assignment to ensure that the required data 
collection instruments are administered within the time constraints dictated by the study protocol. We 
will compare baseline characteristics between patients with complete follow-up to those without by 
randomization group, in order to assess potential biases that may exist in the complete case 
analysis. We will also record and report all reasons for study drop-out using a withdrawal/termination 
form to assess the missing data mechanism (missing completely at random, missing at random, or 
non-ignorable missingness, meaning the data missingness is related to the actual value). We will 
conduct sensitivity analyses for primary and secondary outcomes using several validated methods: 
(1) complete case analyses, which assumes missing completely at random; (2) multiple imputation 
using M=10 imputations, which assumes missing at random; and (3) assigning poor scores and 
good scores for missing values differentially by treatment group, which aligns with non-ignorable 
missingness. 
 
5.4  Interim & Final Analyses 
We will not have any planned interim looks for stopping for efficacy.  The final analyses will be 
conducted once study follow-up is complete, after all data is cleaned and the study database is 
locked.   
 
5.5  Sample Size and Power Calculations 
We selected our sample size to provide ample power to assess differences in the primary outcome—
ACP engagement—as well as secondary patient, caregiver, and healthcare utilization outcomes. All 
calculations were conducted using α=0.05, based on two-sample t-tests for continuous outcomes 
and chi-squared tests for binary outcomes. To adjust for possible clustering effect within each 
oncologist, we used the design effect (DE=1+[(1+CV²)m-1)]ρ), where CV=the coefficient of variation 
for cluster size58 and m=the number of patients per oncologist. Conservatively estimating a 25% loss 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
29 
 
to follow-up for patient-reported outcomes, our sample size will provide >98% power to detect a 
moderate effect size (Cohen’s d=0.5) in our primary outcome—ACP engagement.27 While any 
increase in ACP engagement may be associated with clinically meaningful improvements in EOL 
outcomes,27 a minimal clinically important difference has not been established. A moderate effect 
size (0.35 on a 5-point score, based on a SD of 0.7),27 ensures that we will be able to detect a 
difference comparable to the changes seen with ACP interventions in prior trials.6, 27 We will have 
>78% power to detect a 15% difference in rates of ACP discussions with family and physicians 
between groups (assuming discussion rates of 25% in the web-based arm) and >99% power to 
detect a 20% difference in rates of AD completion (assuming an AD completion rate of 50% in the 
web-based ACP arm). Power is similarly adequate for all caregiver-reported outcomes, assuming a 
75% caregiver participation rate and 15% loss to follow-up, which results in an effective sample size 
of 254.  For caregiver-reported quality of EOL care we will have 97.6% power to detect a moderate 
effect size28, and for caregiver anxiety and depression we will have 83.2% power to detect a clinically 
meaningful difference (Cohen’s d=0.368).59 Anticipating that 90% of patients will die during the study 
period (resulting in an effective sample size of 360), we will have >73-78%  power to detect an 8% 
absolute decrease in ICU admissions and chemotherapy use at the EOL and >80% power to detect 
a 15% decrease in no hospice or hospice ≤ 3 days before death, based on national data.60 
 
5.6  Qualitative Analysis Plan 
We will perform iterative, thematic analysis to illuminate key contexts and mechanisms influencing 
outcomes for each ACP strategy. Preliminary coding will be done by the interdisciplinary 
investigative team, with robustness assessed through a kappa statistic. Once the coding scheme is 
standardized, the remainder of coding will be conducted by research staff using qualitative analysis 
software, with supervision from investigators and cross-checking to ensure confirmability. Regular 
interdisciplinary meetings will be held to identify themes emerging from the data and revise the initial 
conceptual model. The final product of this Aim will be a detailed conceptual model providing 
plausible explanations about how, for whom, and in what circumstances each ACP intervention 
influences outcomes. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
30 
 
6. MONITORING 
6.1  Human Subjects Protections 
All study staff will be required to complete standard CITI modules related to research ethics and 
compliance training in human subjects research. Certificates of completion are kept on file, and 
checked monthly for upcoming expiration. Study staff will be required to renew expiring certificates 
before continuing involvement in the study. Weekly project management meetings are held with 
study staff to ensure that research protocols are followed. Our informed consent process ensures 
that participants have a clear understanding of the study and can freely choose not to participate 
without affecting their care or relationships with UPMC or the University of Pittsburgh in any way. 
 
6.2  Minimizing Risks to Participants 
The potential risks to patient and caregiver participants in both arms of the trial consist mainly of 
concerns about invasion of privacy and perceived burden or emotional upset from completing 
questionnaires about personal physical and mental health and/or the health of a loved one.  In 
addition, there are potential risks to participants in both arms of emotional upset caused by the 
experience of thinking and talking about serious illness and advance care planning. However, based 
on prior experience with both interventions, these potential risks are quite small.  For example, in a 
prior trial of the PREPARE web-based ACP intervention, there were no adverse events and there 
was no increase in depression or anxiety when compared to written ADs alone.6 In a prior trial of 
facilitated ACP using the Respecting Choices® model, facilitated ACP was associated with less 
stress, anxiety and depression among bereaved family members when compared to a control 
group.5 Participants in the facilitated ACP arm may feel self-conscious about having facilitated ACP 
sessions audio-recorded and may additionally experience inconvenience or burden if additional 
appointments are required for facilitated ACP sessions. 
 
We take several steps to ensure confidentiality and avoid invasion of privacy. All study personnel will 
be properly trained in the protection of human subjects. To maintain confidentiality of all study 
materials, we will record names and other personally identifying information only on the tracking 
forms; all other material will use unique identification numbers. Identifying information and de-
identified study records will be kept in separate locked file cabinets and separate files on password 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
31 
 
protected computers. All study databases will be maintained on password protected servers and 
routinely backed up to an encoded password protected file. All facilitated ACP visits will be 
conducted in a comfortable and private or semi-private setting. Telephone interviews will be 
conducted from a private or semi-private research office. Study staff will not re-contact patients who 
initially decline to learn more about the study or be re-contacted. 
 
To reduce the risk of emotional distress, we will adhere to the following standardized procedures.  
First, to address potential emotional upset from answering questions about personal physical and 
mental health and/or the health of a loved one, study staff will remind participants of the voluntary 
nature of study participation and notify them at the start of administration of each questionnaire that 
they are free to skip or decline any questions they wish or to discontinue participation at any time, if 
desired. If a participant's emotional upset escalates while completing study procedures over the 
phone, the study coordinator/research assistant will follow a safety protocol that we have 
successfully deployed in our prior work (see Telephone Safety Protocol in Appendix). 
 
If a participant experiences emotional upset while in the clinical setting undergoing study procedures, 
the ACP facilitator will be with participants.  These ACP facilitators are experienced nurses who will 
receive additional training in supportive care in their role as Respecting Choices® ACP facilitators.  
In addition, they will have a palliative care physician investigator available at all times by telephone 
for any questions or concerns. They will therefore be capable of handling these issues appropriately. 
 
Participants will be given the option of in-person or remote (telephonic or video chat) intervention 
delivery. Remote intervention delivery will be completed utilizing secure, HIPAA-compliant audio and 
visual platforms. In order to decrease the burdens of travel and scheduling, we will make every 
attempt to schedule in-person facilitated ACP sessions at a time and place that is convenient for 
patients. Directions and parking information will be provided to participants prior to the date of their 
scheduled session to facilitate an easier travel plan. Participants will receive parking vouchers, if 
they choose to complete study procedures at the research office. Additionally, participants 
randomized to the Respecting Choices® arm of the study will have the option of having the ACP 
facilitator come to their homes to conduct the study visit, when appropriate and advisable. 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
32 
 
 
If the research staff or investigators learn that the patient or caregiver participant or someone with 
whom they are involved is in serious danger or potential harm, they will inform the appropriate 
agencies, as required by Pennsylvania law. The requirement to report such information is referenced 
in the study patient and caregiver participant informed consents. This research is covered by a 
Certificate of Confidentiality from the National Institutes of Health, which is explained in detail in the 
patient and caregiver participant informed consents.   
6.3  Data and Safety Monitoring Plan 
Data and safety monitoring will be the shared responsibility of the research team, led by the PI, Dr. 
Schenker. A Data and Safety Monitoring Board is not required for this minimal risk trial. The study PI 
and co-investigators will meet monthly to review study progress and any new developments or 
changes to the risk-benefit ratio.  In addition, data quality will be reviewed quarterly by the full study 
team. 
 
The data and safety monitoring plan will address the following areas: (1) the progress of the 
research study, including assessment of data quality and timeliness, and participant recruitment, 
accrual, and retention; (2) review of adverse event data to determine whether or not there is any 
change to the anticipated risk-benefit ratio of study participation, and whether or not the study should 
continue as originally designed, or be changed or terminated; (3) assessment of external factors or 
relevant information; (4) review of study procedures designed to protect the privacy of the research 
participants and the confidentiality of their research data. A yearly summary will be submitted to the 
University of Pittsburgh Human Research Protection Office (HRPO) during annual IRB renewal.   
 
6.3.1 Adverse Event Monitoring 
All adverse events will be tracked by the study coordinator and urgent problems (major complaints 
about the intervention or safety protocol triggers) will be brought to the PI in real time. The 
statistician will run a mortality report for review at quarterly data quality meetings.  
 
Adverse events, serious adverse events (SAEs), and unanticipated problems (UAPs) will be 
monitored using standard definitions. Deaths (other than those resulting from suicide) and 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
33 
 
hospitalizations will not be tracked as adverse events in this population. All SAEs and UAPs will be 
reported to the University of Pittsburgh HRPO as per their policy and procedures.   
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
34 
 
7. REFERENCES 
 
1. 
Institute of Medicine, Dying in America: Improving Quality and Honoring Individual Prefernces 
Near the End of Life. Washington, DC: The National Academies Press; 2014 Available at 
https://www.nap.edu/read/18748/chapter/1 [access date June 4. 2019]. 
2. 
Institute of Medicine, Delivering High Quality Cancer Care: Charting a New Course for a 
System in Crisis. 2013. Available at  https://www.nap.edu/read/18359/chapter/1 [access date 
June 4, 2019]. 
3. 
Bestvina, C.M. and B.N. Polite, Implementation of Advance Care Planning in Oncology: A 
Review of the Literature. J Oncol Pract, 2017. 13(10): p. 657-662. 
4. 
Kline, R.M., et al., Centers for medicare and medicaid services: using an episode-based 
payment model to improve oncology care. J Oncol Pract, 2015. 11(2): p. 114-6. 
5. 
Detering, K.M., et al., The impact of advance care planning on end of life care in elderly 
patients: randomised controlled trial. Bmj, 2010. 340: p. c1345. 
6. 
Sudore, R.L., et al., Effect of the PREPARE Website vs an Easy-to-Read Advance Directive 
on Advance Care Planning Documentation and Engagement Among Veterans: A 
Randomized Clinical Trial. JAMA Intern Med, 2017. 177(8): p. 1102-1109. 
7. 
Sudore, R.L., et al., A novel website to prepare diverse older adults for decision making and 
advance care planning: a pilot study. J Pain Symptom Manage, 2014. 47(4): p. 674-86. 
8. 
Houben, C.H., et al., Efficacy of advance care planning: a systematic review and meta-
analysis. J Am Med Dir Assoc, 2014. 15(7): p. 477-89. 
9. 
Brinkman-Stoppelenburg, A., J.A. Rietjens, and A. van der Heide, The effects of advance 
care planning on end-of-life care: a systematic review. Palliat Med, 2014. 28(8): p. 1000-25. 
10. 
Sudore, R.L., et al., Outcomes that Define Successful Advance Care Planning: A Delphi 
Panel Consensus. J Pain Symptom Manage, 2017. 
11. 
Johnson, S., et al., Advance care planning for cancer patients: a systematic review of 
perceptions and experiences of patients, families, and healthcare providers. Psychooncology, 
2016. 25(4): p. 362-86. 
12. 
Walling, A., et al., Evidence-based recommendations for information and care planning in 
cancer care. J Clin Oncol, 2008. 26(23): p. 3896-902. 
13. 
Fried, T.R. and J.R. O'Leary, Using the experiences of bereaved caregivers to inform patient- 
and caregiver-centered advance care planning. J Gen Intern Med, 2008. 23(10): p. 1602-7. 
14. 
Lamont, E.B. and M. Siegler, Paradoxes in cancer patients' advance care planning. J Palliat 
Med, 2000. 3(1): p. 27-35. 
15. 
Barnes, K., et al., Acceptability of an advance care planning interview schedule: a focus 
group study. Palliat Med, 2007. 21(1): p. 23-8. 
16. 
Moss, A.H., et al., Prognostic significance of the "surprise" question in cancer patients. J 
Palliat Med, 2010. 13(7): p. 837-40. 
17. 
Billings, J.A. and R. Bernacki, Strategic targeting of advance care planning interventions: the 
Goldilocks phenomenon. JAMA Intern Med, 2014. 174(4): p. 620-4. 
18. 
Hui, D., et al., Quality of end-of-life care in patients with hematologic malignancies: a 
retrospective cohort study. Cancer, 2014. 120(10): p. 1572-8. 
19. 
LeBlanc, T.W., A.P. Abernethy, and D.J. Casarett, What Is Different About Patients With 
Hematologic Malignancies? A Retrospective Cohort Study of Cancer Patients Referred to a 
Hospice Research Network. J Pain Symptom Manage, 2014. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
35 
 
20. 
Sudore, R.L., et al., Use of a modified informed consent process among vulnerable patients: 
a descriptive study. J Gen Intern Med, 2006. 21(8): p. 867-73. 
21. 
Becker, C.L., et al., A cluster randomized trial of a primary palliative care intervention 
(CONNECT) for patients with advanced cancer: Protocol and key design considerations. 
Contemp Clin Trials, 2017. 54: p. 98-104. 
22. 
Schenker, Y., et al., Care management by oncology nurses to address palliative care needs: 
a pilot trial to assess feasibility, acceptability, and perceived effectiveness of the CONNECT 
intervention. J Palliat Med, 2015. 18(3): p. 232-40. 
23. 
Tulsky, J.A., et al., Enhancing communication between oncologists and patients with a 
computer-based training program: a randomized trial. Ann Intern Med, 2011. 155(9): p. 593-
601. 
24. 
Maciasz, R.M., et al., Does it matter what you call it? A randomized trial of language used to 
describe palliative care services. Support Care Cancer, 2013. 21(12): p. 3411-9. 
25. 
Schenker, Y., et al., Do patients with advanced cancer and unmet palliative care needs have 
an interest in receiving palliative care services? J Palliat Med, 2014. 17(6): p. 667-72. 
26. 
Schenker, Y., et al., A Pilot Trial of Early Specialty Palliative Care for Patients with Advanced 
Pancreatic Cancer: Challenges Encountered and Lessons Learned. J Palliat Med, 2017. 
27. 
Sudore, R.L., et al., Measuring Advance Care Planning: Optimizing the Advance Care 
Planning Engagement Survey. J Pain Symptom Manage, 2017. 53(4): p. 669-681 e8. 
28. 
Higgins, P.C. and H.G. Prigerson, Caregiver evaluation of the quality of end-of-life care 
(CEQUEL) scale: the caregiver's perception of patient care near death. PLoS One, 2013. 
8(6): p. e66066. 
29. 
Zhang, B., M.E. Nilsson, and H.G. Prigerson, Factors important to patients' quality of life at 
the end of life. Arch Intern Med, 2012. 172(15): p. 1133-42. 
30. 
Wright, A.A., et al., Associations between end-of-life discussions, patient mental health, 
medical care near death, and caregiver bereavement adjustment. Jama, 2008. 300(14): p. 
1665-73. 
31. 
Mack, J.W., et al., End-of-life discussions, goal attainment, and distress at the end of life: 
predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol, 2010. 
28(7): p. 1203-8. 
32. 
Wright, A.A., et al., Family Perspectives on Aggressive Cancer Care Near the End of Life. 
Jama, 2016. 315(3): p. 284-92. 
33. 
Zigmond, A.S. and R.P. Snaith, The hospital anxiety and depression scale. Acta Psychiatr 
Scand, 1983. 67(6): p. 361-70. 
34. 
Vodermaier, A. and R.D. Millman, Accuracy of the Hospital Anxiety and Depression Scale as 
a screening tool in cancer patients: a systematic review and meta-analysis. Support Care 
Cancer, 2011. 19(12): p. 1899-908. 
35. 
Pochard, F., et al., Symptoms of anxiety and depression in family members of intensive care 
unit patients before discharge or death. A prospective multicenter study. J Crit Care, 2005. 
20(1): p. 90-6. 
36. 
Bjelland, I., et al., The validity of the Hospital Anxiety and Depression Scale. An updated 
literature review. J Psychosom Res, 2002. 52(2): p. 69-77. 
37. 
Sundin, E.C. and M.J. Horowitz, Horowitz's Impact of Event Scale evaluation of 20 years of 
use. Psychosom Med, 2003. 65(5): p. 870-6. 
38. 
Sundin, E.C. and M.J. Horowitz, Impact of Event Scale: psychometric properties. Br J 
Psychiatry, 2002. 180: p. 205-9. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
36 
 
39. 
Pinto, D., et al., Good agreement between questionnaire and administrative databases for 
health care use and costs in patients with osteoarthritis. BMC Med Res Methodol, 2011. 11: 
p. 45. 
40. 
Campion, F.X., et al., Advancing performance measurement in oncology: quality oncology 
practice initiative participation and quality outcomes. J Oncol Pract, 2011. 7(3 Suppl): p. 31s-
5s. 
41. 
National Quality Forum., National Voluntary Quality Consensus Standards for Quality of 
Cancer Care. 2009: Washington, DC: National Quality Forum; 2009. Available at 
http://www.qualityforum.org/publications/2009/05/National_voluntary_consensus_standards_f
or_Quality_of_Cancer_Care.aspx [access date June 4, 2019] 
42. 
Bureau of Labor Statistics. Occupational Employment and Wages: 29-1141 Registered 
Nurses.  May 2016. Available from: https://www.bls.gov/oes/current/oes291141.htm. [ access 
date June 4, 2019]. 
43. 
Kazi, M., Realist evaluation for practice. british Journal of Social Work, 2003. 33: p. 803-818. 
44. 
Greenhalgh, T., et al., How do you modernize a health service? A realist evaluation of whole-
scale transformation in london. Milbank Q, 2009. 87(2): p. 391-416. 
45. 
Schenker, Y., et al., I don't want to be the one saying 'we should just let him die': 
intrapersonal tensions experienced by surrogate decision makers in the ICU. J Gen Intern 
Med, 2012. 27(12): p. 1657-65. 
46. 
Schenker, Y., et al., Oncologist factors that influence referrals to subspecialty palliative care 
clinics. J Oncol Pract, 2014. 10(2): p. e37-44. 
47. 
Schenker, Y., et al., "It hurts to know... and it helps": exploring how surrogates in the ICU 
cope with prognostic information. J Palliat Med, 2013. 16(3): p. 243-9. 
48. 
Guest G, B.A., Johnson L, How Many Interviews Are Enough? An Experiment with Data 
Saturation and Variability. Field Methods, 2006. 18(1): p. 59-82. 
49. 
Moser, A., et al., The eight-item modified Medical Outcomes Study Social Support Survey: 
psychometric evaluation showed excellent performance. J Clin Epidemiol, 2012. 65(10): p. 
1107-16. 
50. 
Sarkar, U., et al., Validation of self-reported health literacy questions among diverse English 
and Spanish-speaking populations. J Gen Intern Med, 2011. 26(3): p. 265-71. 
51. 
Temel, J.S., et al., Longitudinal perceptions of prognosis and goals of therapy in patients with 
metastatic non-small-cell lung cancer: results of a randomized study of early palliative care. J 
Clin Oncol, 2011. 29(17): p. 2319-26. 
52. 
Weeks, J.C., et al., Relationship between cancer patients' predictions of prognosis and their 
treatment preferences. Jama, 1998. 279(21): p. 1709-14. 
53. 
Sudore, R.L., et al., An advance directive redesigned to meet the literacy level of most adults: 
a randomized trial. Patient Educ Couns, 2007. 69(1-3): p. 165-95. 
54. 
Jacobsen, P.B., et al., Screening for psychologic distress in ambulatory cancer patients. 
Cancer, 2005. 103(7): p. 1494-502. 
55. 
O'Connor, A.M., Validation of a decisional conflict scale. Med Decis Making, 1995. 15(1): p. 
25-30. 
56. 
Oakley, A., et al., Process evaluation in randomised controlled trials of complex interventions. 
Bmj, 2006. 332(7538): p. 413-6. 
57. 
Craig, P., et al., Developing and evaluating complex interventions: the new Medical Research 
Council guidance. BMJ, 2008. 337: p. a1655. 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
37 
 
58. 
Zimmermann, C., et al., Early palliative care for patients with advanced cancer: a cluster-
randomised controlled trial. Lancet, 2014. 383(9930): p. 1721-30. 
59. 
Puhan, M.A., et al., The minimal important difference of the hospital anxiety and depression 
scale in patients with chronic obstructive pulmonary disease. Health Qual Life Outcomes, 
2008. 6: p. 46. 
60. 
Earle, C.C., et al., Aggressiveness of cancer care near the end of life: is it a quality-of-care 
issue? J Clin Oncol, 2008. 26(23): p. 3860-6. 
 
 
 

 
April 17th, 2024 
 
 
 ________________________________________________________________________________________________________  
 
38 
 
8. APPENDIX 
Telephone Safety Protocol