Refining an Advance Care Planning Group Visit Intervention - A Novel Intervention to Engage Older Adults in Advance Care Planning.
Phase: NA
Conditions: Advance Care Planning, Primary Health Care
Sponsor: University of Colorado, Denver

Official title: Engaging in Advance Care Planning Talks Group Visit 
Intervention 
NCT number: NCT03141242 
Document date: August 24, 2020 
Document type: Study Protocol with Statistical Analysis Plan  
 
 
 

Study Protocol  
I. Hypotheses and Specific Aims:  The main goal of the ENACT (ENgaging in Advance Care 
planning Talks) Group Visit intervention is to integrate a patient-centered ACP process into 
primary care, ultimately helping patients to receive medical care that is aligned with their 
values. This study will determine: a) how the ENACT Group Visit intervention can be refined 
and adapted with input from patients, clinicians, staff, and healthcare system leaders to 
achieve high fidelity for essential components of the intervention while allowing adaptability to 
individual clinic settings and patients from diverse cultural backgrounds; b) meaningful ACP 
outcomes of the intervention to these key stakeholders; and c) feasibility, acceptability and 
preliminary efficacy of the ENACT Group Visit intervention compared to a comparison arm. 
We will pursue the following aims: 
Aim 1: Develop and refine the ENACT Group Visit intervention prototype to have standardized 
essential components of the intervention, to be adaptable across clinic settings, and to 
prioritize ACP outcomes important to key primary care stakeholders (i.e., patients, clinicians, 
staff, and the healthcare system). We will conduct structured focus groups with stakeholders 
to answer key questions for intervention refinement: 
Question 1a: What do patients identify as essential components of the intervention? What 
intervention changes do they suggest to address their ACP needs in the group visit context, 
including strategies to overcome common barriers to participation? 
Question 1b:  What do clinicians, staff, and system leaders identify as essential components of 
the ENACT Group Visit intervention and needed adaptations to promote implementation (e.g., 
clinician involvement, documentation, follow up)? 
Question 1c: What do stakeholders identify as meaningful ACP outcomes for the ENACT Group 
Visit intervention (e.g., patient self-efficacy, readiness for ACP, clinician knowledge of patient’s 
values, ACP documentation)? 
Aim 2: Conduct a pilot randomized control trial (RCT) to test the feasibility, acceptability, and 
preliminary efficacy of the ENACT (ENgaging in Advance Care planning Talks) Group Visit 
intervention compared to a mailed advance directive (AD) arm. 
Hypothesis 2a: It is feasible to recruit, randomize, and retain patients, with high intervention 
fidelity. 
Hypothesis 2b: Patients and clinicians will rate ENACT Group Visit intervention as clinically 
appropriate, acceptable, and low burden. 
Hypothesis 2c: Patients in the ENACT Group Visit intervention will have increased engagement 
in ACP outcomes, informed by aim 1 (i.e., primary outcome: documentation of surrogate 
decision maker; secondary outcomes: advance directive completion, readiness to complete 
ACP actions, ACP discussions), compared to a mailed AD arm. 
II. Background and Significance:  

Healthcare systems need interventions that promote integrated advance care planning (ACP) 
discussions for older adults in primary care. The 2014 Institute of Medicine “Dying in 
America” report emphasized that ACP is essential to ensuring that patients receive care 
reflecting their values, goals, and preferences.1 ACP discussions are associated with 
improved patient outcomes including satisfaction, quality of life, receipt of medical care 
aligned with their wishes, yet most people, particularly poorer, minority, and less-educated 
individuals, do not have these conversations.1-5 The report recommended that healthcare 
systems and payers should encourage clinicians to initiate ACP conversations, integrate 
discussions into ongoing care, and facilitate communication across the healthcare system. 
Specific to the care of older adults, primary care settings face barriers that limit effective 
ACP.6 Clinicians lack the time and training to have ACP conversations and lack clinic-based 
processes to facilitate ACP.(3) A novel ACP group visit in primary care may promote ACP for 
older adults by overcoming patient and clinician barriers to ACP. 
ACP interventions in primary care settings to promote advance directive completion have a 
moderate overall effect size and the most effective approaches combined educational 
materials for patients coupled with a patient-healthcare provider interaction that was typically 
in a one-on-one setting.7 However, implementation of ACP interventions into clinical care 
faces significant barriers such that many patients lack opportunities to discuss ACP with their 
healthcare provider.8 
Group visits effectively engage patients in health care promotion and disease management, 
suggesting that an ACP group visit may be feasible and effective. Older adults enrolled in 
chronic disease management group visits showed improved health status, satisfaction with 
care, and decreased healthcare utilization.9 While group visits have inherent strengths that 
can be leveraged to improve ACP engagement, only 3 group visit interventions focusing on 
supporting ACP discussions have been published, all between 1993 and 2003.10-12 These 
studies have several limitations. First, the group visits focused on completion of advance 
directives instead of a broader, contemporary understanding of ACP as a patient-centered 
process that includes multiple steps such as choosing and preparing a surrogate decision 
maker and values-based discussions. Secondly, these studies were conducted in a single 
clinical setting and have not been effectively implemented into real-world clinical practice.  
To address the need for a disseminable patient-centered ACP model of care for older adults in 
primary care settings, we developed an Advance Care Planning Group Visit (ACP-GV) 
intervention prototype at the University of Colorado Hospital to promote ACP engagement 
(e.g., discussing ACP) and documentation (e.g., surrogate decision makers, advance 
directives) in August 2013. This intervention is an innovative approach to promoting ACP in 
primary care settings by combining Collaborative Learning Theory13, the strengths of 
facilitated discussions within the group medical visit setting (i.e. group dynamics to promote 
behavior change), and ACP patient resources (e.g., PREPARE website14 and The 
Conversation Project Starter Kit15) to facilitate effective communication and patient 
engagement related to ACP. The initial intervention prototype involved groups of older adults 
(age 65+) who participated in two sessions, one month apart, facilitated by a physician and 
social worker. Findings from the clinical demonstration quality improvement intervention 

(COMIRB #13-2291) suggested that older adults were willing to engage in ACP discussions 
and document their wishes (described below).16 Ideally, this intervention will be able to be 
adapted to diverse patient populations and clinical settings. The current need is to refine and 
adapt the ACP-GV intervention for diverse patients and clinician stakeholders in preparation 
for implementation, and to test whether the ACP-GV intervention increases ACP outcomes 
compared to control conditions. Moving forward, the refined intervention will be known 
as the ENACT (ENgaging in Advance Care planning Talks) Group Visit intervention. 
The ENACT Group Visit intervention is an innovative model of care that needs to be rigorously 
tested in a randomized controlled trial to determine whether it improves ACP documentation 
and informed discussions for older adults compared to usual care. Currently, 50% of 
UCHealth Denver Metro primary care patients over the age of 60 say they have an advance 
directive, though it is unknown what proportion of these patients have had detailed 
conversations with loved ones or healthcare providers (UCHealth Outpatient Advance 
Directives Report, Feb 2017). Additionally, the rate of advance directives available within the 
electronic medical record so that they are available during a medical episode is also less 
than 50%. To this end, the current research study will refine, adapt and test the ENACT 
Group Visit intervention, focusing on real-world clinical settings. Because the first ACP group 
visit prototype focused on engaging patients in primary care settings in a patient-centered 
ACP process, the proposed research project will extend this work by developingan effective 
healthcare model of care that can be implemented into practice. The need for new models 
that promote ACP is highly relevant given the approval by the Centers for Medicare and 
Medicaid Services of reimbursement for ACP counseling on January 1, 2016.17  
III. Preliminary Studies:   
 
Intervention Structure (COMIRB #13-2291) - At University of Colorado Hospital (UCH), we 
conducted the initial ACP-GV prototype intervention, which was named the “Conversation 
Group Medical Visit.” The intervention aims to engage patients in an interactive discussion of 
key ACP concepts and support patient-initiated ACP actions (i.e. choosing decision-
maker(s), deciding on preferences during serious illness, discussing preferences with 
decision-makers and healthcare providers, and documenting advance directives). The group 
visits involve two 2-hour sessions, one month apart, facilitated by a geriatrician and a social 
worker or nurse. Table 1 provides an overview of the ENACT Group Visit intervention 
structure and facilitator considerations. The discussions include sharing experiences related 
to ACP, considering values related to serious illness, choosing a surrogate decision-
maker(s), flexibility in decision making, and having conversations with decision-makers and 
healthcare providers.18 The facilitators support an interactive discussion that promotes 
opportunities for patients to learn from others’ experiences.  
Table 1. ENACT Group Visit Intervention 
Overview of intervention 
Session format (2 hours) 
Time 
Sessions: Two sessions, 1 month apart 
Arrival, check-in, medical update   
Introductions and rapport building  
30 min 
20 min 

Participants: 8-10 adults; option to 
bring a potential surrogate to the 
sessions 
Facilitators: Provider and social worker 
or RN 
Location: Clinic conference room 
ACP discussion                              
Discussion of individual ACP 
goals 
Opt.: Individual visits with clinician 
60 min 
10 min 
10 min 
Preliminary data - We implemented the prototype ACP-GV intervention for older adults (age 
65+) in 3 UCHealth primary care clinics in 16 ACP-GV intervention cohorts (n=118) between 
Oct 2013-June 2015. This prior work and experience supports our ability to complete the 
current study. Compared to prior to participating, more patients reported having detailed 
ACP conversations with their loved ones about their wishes (19% to 42%, p<0.001). Medical 
record review showed increased rates of advance directives (20% to 57%, p<0.001) and 
documentation of a medical durable power of attorney [MDPOA] or medical proxy 
documentation (39% to 79%, p<0.001) (Table 2, McNemar’s Test). While the intervention 
focuses on patient-initiated ACP goals and actions via education and support from the 
group, patients often chose to complete an advance directive. 
Lessons learned and current gaps – From the initial quality improvement project, we identified 
specific intervention components of the ACP-GV intervention. These components are: 
patient and group characteristics, facilitator characteristics and communication skills, ACP 
resources, primary care provider integration, and clinic resources. Building on these 
preliminary insights, Aim 1 of this study will solicit key stakeholder input to identify essential 
and adaptable features of the ENACT Group Visit intervention and create an ENACT Group 
Visit intervention manual that can be used to adapt the intervention to individual clinic 
settings and populations. Aim 2 will conduct a pilot RCT of the ENACT Group Visit 
intervention 
IV. Research Methods 
A.  Outcome Measure(s):   
Aim 1: We have identified multiple potential outcomes for each key research question. In line 
with qualitative research methods, we anticipate that new outcomes of interest will be iteratively 
identified during the analytic process for each of the study questions.   
Table 2. ACP-GV outcomes 
Baseline, n (%) 3 month, n (%) p-value 
ACP documentation, n = 118 
Advance directive 
23 (20) 
67 (57) 
<0.001 
MDPOA or medical proxy 
46 (39) 
94 (79) 
<0.001 
Table 3. Intervention 
Components 
Potential Essential vs. Adaptable Features 

1.1. 
What are essential/adaptable features of the ENACT Group Visit intervention? Table 3. 
 
 
 
 
 
 
 
 
 
1.2. 
What are meaningful outcomes for the ENACT Group Visit intervention? Table 4. 
Table 4. Potential Patient-centered Outcomes 
ACP discussions, decisions, or documentation 
ACP knowledge 
Self-efficacy or patient readiness for ACP   
Patient, clinician satisfaction 
Knowledge of patient’s wishes 
 
Aim 2: Outcome measures for the pilot RCT of the ENACT Group Visit intervention include 
patient and implementation outcomes. These outcomes are guided by a Sudore et al conceptual 
framework of ACP engagement outcomes14, the RE-AIM evaluation framework19 and the 
Implementation Outcomes Framework20, as described in Table 5. In this pilot study, we will 
focus on measures of Reach, Efficacy (Effectiveness) and Implementation. 
 
 
Patient and Group 
characteristics 
Age; Gender; Interested in groups; Group size; Participation 
with spouse/partner; Patient cognitive status 
Facilitator skills 
Communication skills; Teaching skills; Medical and ACP 
knowledge 
ACP resources 
Conversation Starter Kit; PREPARE website; Colorado 
MDPOA 
PCP integration 
Desire for individualized follow up; Desire to share ACP with 
PCP 
Clinic resources 
Meeting space; Workflows; Staffing needs 

 
 
 
 
Primary Outcome: Change in documentation of surrogate decision maker 
Table 5. Patient-centered and implementation outcomes for Aim 2. 
Aim 2 outcomes 
Measure/Outcome 
Data Sources 
Time Point 
2.1. Feasibility (Implementation outcome) 
Recruitment 
(related to Reach)  
% individuals who 
participate of eligible 
patients by clinic-based 
screening 
Demographics of 
participants and 
non-participants 
After 
screening 
complete 
Randomization 
No difference in 
demographics across 
study arms (age, gender, 
racial composition, 
advance directive in EMR 
at baseline) 
Demographics of 
participants 
After 
randomization 
complete 
Retention  
% individuals who 
complete 6 month follow 
up; % who complete 
ENACT group visit 
intervention arm (both 
visits) 
Participant 
database 
0 to 6 months. 
Intervention fidelity 
(ENACT group visit 
intervention arm 
only) 
-Adherence to intervention 
manual 
-Quality of program 
delivery 
Facilitator 
checklist; team 
notes 
Post-
intervention 
2.2 Acceptability (Implementation outcome) 
Clinical 
appropriateness  
Perceived fit or relevance 
Patient survey 
after session A, , 
and patient/PCP 
interviews 
Post-
intervention 
(ENACT group 
visit 
intervention 
arm) 
Acceptability 
-Satisfaction with content; 
-Satisfaction with delivery  
Level of burden 
Usefulness 
 
2.3 Advance Care Planning Outcomes (patient-reported effectiveness) 
Primary Outcome Measure:  

1. Change in documentation of the surrogate decision maker 
Measure: An MDPOA form is in electronic medical chart  
Time Frame: 0, 3, 6, 12 months 
Secondary Outcome Measure: 
2. Change in advance directive in medical record  
Measure: Any advance directive is present in the medical chart 
Time Frame: 0, 3, 6, 12 months 
3. Change in readiness to engage in ACP  
Measure: Four patient reported questions regarding readiness to engage in specific parts of the 
advance care planning process (i.e. signing official papers to name a medical decision maker; 
talking to the decision maker; talking to the doctor; signing official papers putting their wishes in 
writing)  
Time Frame: 0, 6 months 
4. Change in participant choice of a surrogate decision maker 
Measure: Patient report - "Have you decided who you want your medical decision maker to be?" 
Time Frame: 0, 6 months 
5. Change in participant discussions of values and care preferences with surrogate decision 
maker  
Measure: Patient report - "Have you talked with your decision maker about what kind of medical 
care you would want if you were very sick or near the end of life?" 
Time Frame: 0, 6 months 
Other Pre-specified Outcome Measures:  
6. Percent of Recruitment (Reach) 
Measure: Percent of individuals who participate of eligible patients, by clinic-based screening 
Time Frame: From date of pre-screening until the date of participants' decision to enroll in study 
or not, up to 3 months 
7. Percent of Retention  
Measure: Percent of individuals who complete the intervention and the 6 month follow up  
Time Frame: Enrollment thru 6 month follow up 
 
B. Description of Population to be Enrolled:   
 
Population and Setting: UCHealth patients and associated patient-invited observers, primary 
care providers (PCPs), clinical staff, and healthcare system leaders. 
Aim 1 will enroll up to 42 individuals. Specifically, we will conduct 6 focus groups (5-6 
participants each), including each of the following groups: patients (including those who 
participated in ACP GV prototype and those who did not), PCPs, and staff. Additionally, 4-6 
healthcare system leaders will be interviewed. 
Aim 2 will recruit up to 120 older adults (≥ age 50) from University of Colorado Hospital (UCH) 
primary care clinics and UCHealth Northern Colorado (Snow Mesa Internal Medicine). We will 
invite patients, observers, and patient’s PCP. We will enroll 120 subjects to ensure that at least 
100 subjects complete the study. 
 
Inclusion Criteria:  

Aim 1: Patients will be ages 65 or older. Clinicians will be PCPs, including physicians, advanced 
practice nurses, and/or physician assistants. Multidisciplinary staff members will include nurses, 
medical assistants, social workers, and/or schedulers who are members of primary care clinics 
teams. Healthcare system leaders related to UCHealth primary care practices will include 
medical directors, nurse managers, ambulatory service chiefs, and other administrative leaders. 
Aim 2: Patients will be ages 50 years and older, and receive primary care at UCHealth. 
Observers will be adults (>age 21) who are invited by patients who are randomized to the 
ENACT Group Visit intervention arm. PCPs can participate if at least one of their patients is 
enrolled in either study arm. 
 
Exclusion Criteria:  
Exclusion criteria for Aim 1 and Aim 2 patients include known prior diagnoses of dementia or 
deafness, as determined by ICD-10 codes, clinician identification or self-report, that limits ability 
to participate in group discussions. We will exclude participants who have cognitive impairment 
as assessed by inability to consent to the study and deficits on the validated Short Portable 
Mental Status Questionnaire (SPMSQ). We are assessing for memory impairment and 
dementia in order to exclude subjects who may have decision-making challenges. We will 
exclude individuals who do not have a phone, ability to travel to clinic, or someone who can 
assist with reading ACP materials if they are visually impaired. We will also exclude individuals 
planning to move in the next six months. Individuals who have a spouse or partner currently 
enrolled in the study will also be excluded in order to maintain independence of participants.  
 
C. Study Design and Research Methods   
 
This study will refine the ENACT Group Visit intervention using key stakeholder input to adapt 
the intervention to individual clinical settings and prioritize stakeholder outcomes for ACP (Aim 
1), and conduct a pilot RCT of the ENACT Group Visit intervention (Aim 2) (Figure 3).  
 
Aim 1: To develop and refine the ENACT Group Visit intervention prototype to have 
standardized essential components of the intervention, to be adaptable across clinic settings, 
and to prioritize ACP outcomes important to key primary care stakeholders (i.e., patients, 
clinicians, staff, and the healthcare system).  
Recruitment:  
Patients - Using existing clinical relationships (including through the ACP-GV quality 
improvement effort), we will recruit for focus groups (patients, clinicians, staff) and interviews 
(healthcare system leaders). We will also ask permission to recruit patients from the 

UCHealth Patient Advisory Council. We will specifically recruit both men and women, ages 
65 and older, and diverse self-reported racial/ethnic backgrounds to maximize potential for 
generalizability. Patients will be contacted by study personnel by letter (or up to 2 emails, if 
that is their preferred contact method) and/or up to 3 phone call attempts.  
Clinicians, staff, healthcare system leaders- To gain broad perspectives on integration of the 
ENACT Group Visit intervention into clinical work flow, we will include clinicians from each 
discipline (MD, NP, PA) who had patients participate in the prototype, as well as University 
providers who did not have patients involved in the ACP-GV prototype. We will also invite 
clinical staff who were involved in the ACP-GV prototype and those who were not. 
Multidisciplinary staff members will include nurses, medical assistants, social workers, 
and/or schedulers. Healthcare system leaders will include medical directors, nurse 
managers, ambulatory service chiefs, and other administrative leaders. The study PI will 
contact clinicians, staff, and healthcare system leaders by email or in-person to recruit for 
the study. 
All participants will be consented in person and provide written consent if they agree to 
participate. The consent process will take place immediately prior to the focus groups or 
interviews. See Table 6. 
Research Methods: We will use a structured focus group process, Nominal Group Technique22, 
to obtain qualitative data from patients, clinicians, and staff to answer key questions for 
intervention refinement. Like an in-person Delphi panel, Nominal Group Technique provides a 
structure to achieve prioritized recommendations from participants. By using this method, we 
will receive individual stakeholder and group-level input to reach consensus on how to refine 
the ENACT Group Visit intervention so that it has high fidelity to essential components, 
adaptability to clinic settings, and focuses on clinically meaningful ACP outcomes.  
We will conduct focus groups with patients, clinicians, and staff (total of 6 groups). Each 
Nominal Group Technique Session will include 5-6 participants per group. To achieve input 
from health care system leaders while being respectful of their time and schedule constraints, 
we will conduct semi-structured 30-minute interviews in person or by phone based on the same 
topics with each leader (n=4-6). All groups and interviews will be conducted at a convenient 
location for participants. 
Nominal Group Technique Sessions: Participants will be given a description of the purpose of 
the Nominal Group Technique session and how they will actively provide input as individuals 
and as a group. We will provide a description of the ENACT Group Visit intervention and its 
goals, the prototype intervention manual, and the key discussion questions. For each topic, 
individuals will first independently generate their own ideas. Then, the group will share ideas 
and engage in a discussion that refines the suggestion. Lastly, the group will rank ideas to 
provide prioritized input. The focus group/interview guides, including key discussion questions, 
will be updated iteratively based on emerging data and analysis.  
For Topic 1, participants will review the intervention domains, including the potential essential 
components (Table 3), and identify areas that are unclear, poorly described, or missing. For 
Topic 2, participants will suggest adaptations or changes that are important to implementing the 
intervention into specific clinic settings or for individuals from diverse cultural backgrounds. We 

will solicit input related to the PCP’s role in the intervention, clinical documentation, or 
subsequent follow up needs. For Topic 3, participants will describe meaningful outcomes from 
the ENACT Group Visit intervention. Potential outcomes include self-efficacy, readiness to 
engage in ACP, and values-based discussions, decisions, and ACP documentation. Clinicians 
and healthcare system leaders will be asked for input on “minimum clinically important 
differences” for each outcome. At the end of a session, we will summarize the ranked ideas for 
participants. We will incorporate input into subsequent sessions for iterative input. After all 
groups are completed, we will conduct a final combined session with patients, clinicians, staff or 
leaders (up to 10 total participants), as a form of member checking to assure that we have 
reached consensus on intervention refinements.  
Aim 1 will lead to an ENACT Group Visit intervention that is refined for testing and 
implementation. Essential components will provide the basis for an intervention manual and 
fidelity checklist. Adaptable components will be included in the manual, as they outline how the 
intervention can be modified to a clinic setting or specific patient population.  
Aim 2: Conduct a pilot RCT to test the feasibility, acceptability, and preliminary efficacy of 
ENACT (ENgaging in Advance Care planning Talks) Group Visit intervention compared to a 
mailed advance directive (AD) arm. 
Study design: We will conduct a pilot RCT of the ENACT Group Visit intervention vs. an AD-
only control arm among older adults (≥ age 50) in a primary care clinic (Schematic). We will 
assess feasibility, patient and clinician acceptability, and preliminary efficacy on 3, 6, and 12-
month ACP outcomes using validated instruments (i.e., primary outcome: MDPOA completion, 
secondary outcomes: advance directive completion, self-efficacy, readiness, ACP discussions 
[informed by Aim 1]).  
Schematic for Aim 2: 
 
 
 
 
 
 
 

 
 
 
 
ENACT (ENgaging in Advance Care planning Talks) Group Visit Intervention Study 
Flowchart. 

*Short Portable Mental Status Questionnaire 
**ACP resources include Conversation Starter Kit, PREPARE website pamphlet 
*** Observers will be adults who are invited by patients who are randomized to the ENACT 
Group Visit intervention arm. 
**** Primary care providers includes Physicians, advance practice nurses, and physician 
assistants.  
Patient Pre-Screening – We will recruit patients from UCHealth primary care clinics, which are 
settings with which study team members have a treatment relationship. We will use chart 
review to screen potential patients for eligibility, with support from UCHealth Clinic personnel 

who have access to population-health tools. We will ask for a list of patients who are 49 
years of age or older, list a UCHealth clinician as their primary care provider. Although we 
will initially be recruiting patients 50 years of age and older, we will broaden the age range 
for the data extraction to age 49 because the recruitment phase may take more than 1 year 
and potential subjects may turn 50 during that time. This is an efficient use of clinical 
operations resources and avoids multiple data extractions. 
Upon obtaining a list of potential patients from chart review, primary care clinicians will be 
sent a letter/e-mail informing them about the study. We will ask them to review a list of their 
patients and to provide permission/endorsement for patient(s) who would be appropriate for 
the study based on their judgment that the patient is appropriate for a group visit (i.e. does 
not have severe cognitive impairment or hearing loss to make a group visit very difficult). 
Clinicians will be asked to provide permission for the study team to contact their patients by 
letter to describe the research study and offer patients the opportunity to decline to be 
contacted by study staff. Additionally, clinicians will be informed that if they do not respond 
one week after the 3rd attempt to contact them to review their patient list (including by email, 
phone, and/or in person), we will assume assent to contact their patients and a letter 
describing the study will be sent to patients on behalf of the study team. We will obtain 
permission from the Practice Directors before their clinicians are contacted. 
In addition, we will also recruit patients directly from clinic by posting flyers in approved 
areas in the clinics, provider referral, and utilizing University of Colorado's free web-based 
advertisements, and local online community boards. Participants can contact study staff in 
person, by phone, or by email.  
We expect that >90% of patients from the administrative data pull will be eligible based on 
study criteria. Prior to study consent, we will determine eligibility using the Pre-Screening 
template. Based on our preliminary experiences, around 25% will agree to participate and 
provide informed consent. To reach a goal of 120 enrolled subjects, we anticipate 
approaching 480 patients. We will over-enroll by 20% to reach the target study sample size 
of 100, allowing for individuals who chose not to complete the entire ENACT Group Visit 
intervention (both visits). We will monitor recruitment and retention to understand reasons 
for declining. 
In total, this study will approach up to 660 subjects. For Aim 1, we anticipate screening up to 
100 subjects (inclusive of patients, clinicians, staff, healthcare system leaders). For Aim 2, we 
will screen up to 480 patients, invite up to 60 observers and invite an estimated 20 primary care 
providers of enrolled patients. We estimate enrolling 200 subjects for study completion including 
40 participants in Aim 1 and 120 patients, an estimated 20 observers (not required for study 
outcomes), and up to 20 primary care providers (not required for primary outcome assessment) 
in Aim 2. 
 
 
 
Recruitment, Screening and Consent Visit: 

Patients - Patients will be contacted by study personnel by letter (or up to 2 emails, if available 
in the EMR) and/or up to 3 phone call attempts. Patients will answer brief questions by 
telephone using the participant eligibility script to determine study eligibility (assessing for 
lack of phone, ability to travel to clinic, ability to view written ACP materials, plans to move if 
any, and current enrollment in the study of a spouse or partner). A waiver of HIPAA 
authorization has been requested for this telephone pre-screen.  
Patients will then participate in a verbal consent process by phone. We will read the consent 
form to potential subjects, allowing time for questions and discussion, and then assess 
comprehension of the study using 5 true/false questions. (Supporting Materials: Consent 
Verification) If comprehension questions are not answered correctly, education and 
reassessment of comprehension are repeated. If subjects take more than two passes 
through the comprehension assessment, formal assessment for cognitive impairment will be 
completed using the Short Portal Mental Status Questionnaire. If patients are cognitively 
impaired they will be excluded. If they are not cognitively impaired, we will redo the teach-
back one more time, after which the patient will be deemed ineligible for the study if they are 
unable to answer all comprehension questions correctly. Once complete comprehension is 
achieved, we will ask patients to provide verbal consent which will be documented. We are 
requesting a waiver of documentation of consent. A copy of the consent form detailing the 
date and time of consent will be provided to patients. 
Observers - Once a patient has agreed to enroll in the study AND is randomized to the ENACT 
Group Visit intervention, we will ask patients if they want to include an "observer" to 
participate with them in the ENACT Group Visit intervention. We will obtain name and 
telephone contact information for the potential observer. Over the phone, we will describe 
the study, the dates of the patient’s assigned ENACT Group Visit intervention sessions, and 
consent the observer for the study over the phone if they are interested in participating. We 
have requested a waiver of documentation of consent for this activity. Observers who agree 
to participate will attend the group visits and be invited to complete brief surveys and/or 
interviews. Observers themselves will not be registered as patients, so their insurance will 
not be billed and co-pays will not apply to observers. Although the personal health 
information is minimal (name, telephone number), we are requesting a waiver of HIPAA 
authorization. 
Clinicians – For clinicians who have a patient enrolled in the study, study personnel will contact 
them by email about study participation after the 6 month patient-level outcomes have been 
collected to invite them to provide their perspective on the ACP intervention. If the clinician 
agrees, they will be consented by email and a waiver of documentation is requested. 
Clinicians will be asked to complete brief surveys (i.e. clinical relevance, acceptability, 
burden) and/or interviews to evaluate the potential impact of the study. This recruitment of 
PCPs is after the ENACT Group Visit intervention cohorts have been completed to limit PCP 
contamination, where PCP behavior may be directly influenced by reflecting on this ACP 
intervention study. Since the ENACT Group Visit intervention is medical care, we 
acknowledge that PCPs will be aware that their patients are involved in the study. 
 

 
Randomization: Patients will be randomized in blocks of 5 patients to maintain equal study arm 
size and gender balance. Based on pilot data, we will target scheduling 10 patients per ENACT 
Group Visit intervention cohort to achieve at 8-10 patients, allowing for cancellations. Patients 
randomized to the control arm will be offered the intervention after study completion. Offering 
the intervention to both arms limits dissatisfaction due to randomization to the control arm. In 
order to maintain independence, individuals with a spouse or partner consented to the study will 
be excluded. Prior to randomization, study personnel will conduct a chart review to assess ACP 
documentation and conduct a phone survey to assess ACP outcomes. 
ENACT Group Visit Intervention Arm: We will implement the ENACT Group Visit intervention. It 
will consist of up to 10 patients meeting in two 2-hour group medical visit sessions, one month 
apart. Six cohorts will be conducted so that at least 50 patients receive the ENACT Group Visit 
intervention arm, allowing for patient cancellations. The intervention will be conducted based on 
an intervention manual that will be refined and adapted during Aim 1. Participants will receive 
the Colorado Medical Durable Power of Attorney (MDPOA) form  and the Conversation Starter 
Kit by mail prior to the first session. An ENACT Group Visit intervention cohort will be started 
each month, with the goal of starting all cohorts within 6 months of starting Aim 2. 
Intervention resources, which will be updated based on Aim 1 input, are included as an 
appendix to this protocol. 
Table 6. Summary of methods of obtaining consent 
 
Participant type 
Method of 
consent 
Consent type 
Timing 
Aim 1 Patient, clinicians, 
staff, healthcare 
system leaders 
In person 
Written 
Immediately prior to 
focus 
group/interview 
Aim 2 Patient: Pre-screen 
By phone 
Waiver of HIPAA 
authorization 
After administrative 
selection, clinician 
permission, and 
outreach letter to 
patient. 
Patient: Eligibility, 
Consent, and 
Baseline 
Assessment 
Telephone [Waiver of 
documentation of 
consent, with waiver of 
HIPAA authorization] 
Observers 
By phone 
Telephone [Waiver of 
documentation of 
consent, with waiver of 
HIPAA authorization] 
After a patient is 
randomized to the 
ENACT Group Visit 
intervention arm 
Clinicians 
By email 
Email [Waiver of 
documentation of 
consent] 
After 6 month 
patient-outcomes 
are complete 

Control Arm: Patients randomized to the control arm will receive by mail a Colorado MDPOA, 
the Conversation Starter Kit, and a cover letter to encourage them to discuss ACP with their 
primary care provider. They will also receive the local standard of care, which includes 
opportunities to discuss ACP with their PCP during routine clinical one-on-one office visits. After 
collection of 6 month outcomes, control arm patients will be contacted and offered opportunity to 
be scheduled for a non-research ENACT Group Visit intervention cohort. While the ACP Group 
Visit is not systematically part of local standard of care, it has been previously offered at 3 
UCHealth primary care clinics as part of a quality improvement initiative. 
Follow up assessments: 
Patient-centered and implementation measures will be collected as study outcomes as 
described previously in Table 5. 
For Aim 2, after the 6 month patient-level outcomes have been collected, we will approach all 
PCPs who had a patient enrolled in the study (both ENACT Group Visit intervention or control 
arm) to invite them to provide PCP-level acceptability outcomes (i.e. clinical relevance, 
acceptability, burden) through brief interviews. We will recruit PCPs after the ENACT Group 
Visit intervention cohorts have been completed to limit PCP contamination, where PCP behavior 
may be directly influenced by reflecting on this ACP intervention study. 
D.   Description, Risks and Justification of Procedures and Data Collection Tools: 
Aim 1 study procedures are focus group or interviews with up to 40 patients, clinicians, staff, or 
healthcare system leaders and pose no more than minimal risk to subjects. The potential risks 
include breach of confidentiality and privacy. Information provided by subjects about the ENACT 
Group Visit intervention will be beneficial to its refinement and implementation, and has very low 
potential for psychological distress related to discussing ACP and/or end-of-life issues. 
Aim 2 study procedures include a) minimum necessary patient demographic information related 
to screening, recruitment and retention rates; b) participant self-reported demographics; c) 
advance care planning (ACP) outcomes (documentation of advance directives, documentation 
of medical durable power of attorney or medical proxy decision maker) at baseline, 3, 6, and 12-
months; d) ENACT Group Visit intervention evaluations after session 2; e) brief participant 
telephone interviews at 6 month follow up; f) group visit audio and video-recordings and study 
team de-briefing notes related to each ENACT Group Visit intervention; g) PCP interviews and 
clinic-level implementation measures. Data collection tools and request for audiovisual release 
have been included in this COMIRB application. The potential minimal risks include breach of 
confidentiality and privacy, which is not greater than the potential usual risk encountered 
through participating in routine medical care, and potential for psychological distress related to 
discussing advance care planning and/or end-of-life issues.    
Plans to minimize risk related to psychological discomfort - The ENACT Group Visit intervention 
has been developed, and will be iteratively refined in Aim 1, to promote effective group 
dynamics and a supportive environment, while minimizing participants' discomfort. Research 
staff will be trained to address psychological distress and will follow standard procedures for 
referral for mental health evaluation. The mailed letter that the control group and the ENACT 
Group Visit intervention participants receive will include a healthcare provider contact whom 
they can contact for support or concerns related to ACP and/or end-of-life issues. 

Plans to minimize risks related to loss of confidentiality -  At the beginning of each ENACT 
Group Visit intervention, in line with local standards of care for group medical visits practices 
at UCHealth, all participants will be asked to review and sign a standard clinical group visit 
consent form that outlines privacy guidelines including HIPAA and describes how standard 
medical visit insurance co-pays apply. The consent form from the ACP-GV prototype is 
provided as an example (supporting materials); in the quality improvement initiative, 100% 
of the participants signed this form. Although all participants will have already provided 
informed consent before arriving to the ENACT Group Visit intervention, this process will 
include a reminder that participants are participating in a research study, participation is 
voluntary, and to maintain privacy and confidentiality. All written and audio/visual recordings 
and consent materials will be in locked cabinets and on password-protected, encrypted 
computers. 
 
E.   Potential Scientific Problems:   
Aim 1: Potential problem - Patients and other key stakeholders may not be able to provide 
specific, actionable input to refine the ENACT Group Visit intervention.  
 
Compared to a less structured focus group, Nominal Group Technique is an efficient method 
to achieve stakeholder consensus for intervention refinement. Overall feasibility is supported by 
the study team’s experience using Nominal Group Technique and patient advisory boards are 
familiar with working with researchers, supporting the achievement of quality patient input. While 
stakeholders may not identify a “gold standard” ACP outcome for the intervention, the study 
team can suggest ACP outcomes from the literature. We desire stakeholder input on 
intervention outcomes to understand end-users’ priorities. If stakeholder input diverges, or to 
improve generalizability for primary care settings that are not well represented by stakeholders, 
the study PI can re-convene an expert Advisory Panel from The Colorado Health Foundation 
prototype study to reach consensus. 
Aim 2: Potential problem – Patient recruitment to a complex intervention related to advance care 
planning may be slow. 
 
Our prior experience suggests that scheduled patients may cancel at the last minute. Thus, 
we have built in use of administrative searches to identify potentially eligible patients and will 
ask PCPs to endorse the study in a recruitment letter sent to patients on their behalf. We have 
allotted sufficient recruiting resources and multiple methods, and will over-schedule each 
ENACT Group Visit intervention cohort so that a goal of 10 patients per cohort is more likely.  
F.   Data Analysis Plan:   
Aim 1: By design, initial data analysis occurs during the Nominal Group Technique sessions as 
participants generate ideas, refine ideas as a group, and rank ideas at the end of the session. A 
research assistant will create detailed field notes to outline this analysis and collect participants’ 
written notes. All sessions (~2 hours each) will be audiotaped and transcribed. After each 
session, the study team will conduct a team-based, deductive content analysis13,14 of transcripts 
and field notes to identify essential components for standardization, adaptable components that 

can be changed to meet particular clinic setting needs, and prioritize ACP outcomes. Detailed 
logs of analysis and interpretation, transcripts, and coding will be maintained to ensure 
transparency. The focus group and interview guides will be updated iteratively based on 
emerging data and analysis to ensure data is rich and trustworthy, thereby reflecting key 
stakeholders’ input. 
Aim 2: To assess feasibility, we will track recruitment and participation rates, including reasons 
for not participating. We will compare patient demographics by study arm to assess 
randomization using t-test. We will assess intervention fidelity using a Fidelity Checklist that we 
develop in Aim 1, confirming with field notes and transcript analysis. Analyses will be intent-to-
treat by assigned study arm. We will describe acceptability measures using means and 
standard deviations for 5-point Likert outcomes, and compare outcomes by study arm using t-
test. The intervention will be a) feasible if ≥85% of intervention arm participants attend both 
sessions (retention rate) and b) acceptable if acceptability measure means are at least 4.0 out 
of 5. The proportion of MDPOA completion at 6 months will be compared by study arm to 
estimate effect sizes. Within arms, McNemar’s test will be used to assess for pre- and post-
study change in ACP outcomes at 6 months. To address potential missing, a secondary 
analysis using linear mixed models will be conducted using all available data collected at 
baseline and 6 months.  
Power Calculation: While the proposed study is not powered to detect statistically significant 
differences for the ACP outcomes, we will compare 6 month ACP outcomes in the ENACT 
Group Visit intervention compared to the AD-only arm to provide preliminary estimates of effect 
sizes. The estimated effect sizes and stakeholder input on minimally clinically important 
differences in ACP outcomes will inform future trials. With a sample size of 50 patients per 
group and assuming a control arm advance directive completion rate of 20% based on pilot 
data, the detectable rate, with power=80% (alpha=0.05), in the ENACT Group Visit intervention 
arm will be 46% (26% increase). With power=90% (alpha=0.05), we can detect a 50% rate in 
the intervention arm. 
 
G.  Summarize Knowledge to be Gained:   
This research study will clearly define the key intervention components, facilitator skills, 
and patient perspectives on feasibility, acceptability, and meaningful ACP outcomes for this 
novel ENACT Group Visit intervention. This study will provide rational and justification for and 
data to inform design of a future pragmatic trial of the ENACT Group Visit intervention in 
multiple clinics. Findings will enable the ENACT Group Visit intervention to be adapted to 
individual clinic and patient populations. This knowledge is critical to future efforts to refine the 
intervention, train facilitators, implement and test the intervention with high fidelity. 
 
  
 
 

H. References: 
 
1. 
Institute of Medicine. Dying in America: Improving Quality and Honoring Individual 
Preferences Near the End of Life. Washington, DC: The National Academies Press, 
2015. doi:10.17226/18748. 
2. 
McMahan RD, Knight SJ, Fried TR, Sudore RL. Advance care planning beyond advance 
directives: perspectives from patients and surrogates. J Pain Symptom Manage. 
2013;46(3):355-365. 
3. 
Tierney WM, Dexter PR, Gramelspacher GP, Perkins AJ, Zhou XH, Wolinsky FD. The 
effect of discussions about advance directives on patients' satisfaction with primary care. 
J Gen Intern Med. 2001;16(1):32-40. 
4. 
Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient 
mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 
2008;300(14):1665-1673. 
5. 
Silveira MJ, Kim SY, Langa KM. Advance directives and outcomes of surrogate decision 
making before death. N Engl J Med. 2010;362(13):1211-1218. 
6. 
Sachs GA, Stocking CB, Miles SH. Empowerment of the older patient? A randomized, 
controlled trial to increase discussion and use of advance directives. J Am Geriatr Soc. 
1992;40(3):269-273. 
7. 
Ramsaroop SD, Reid MC, Adelman RD. Completing an advance directive in the primary 
care setting: what do we need for success? J Am Geriatr Soc. 2007;55(2):277-283. 
8. 
Lund S, Richardson A, May C. Barriers to advance care planning at the end of life: an 
explanatory systematic review of implementation studies. PLoS One. 
2015;10(2):e0116629. 
9. 
Scott JC, Conner DA, Venohr I, et al. Effectiveness of a group outpatient visit model for 
chronically ill older health maintenance organization members: a 2-year randomized trial 
of the cooperative health care clinic. J Am Geriatr Soc. 2004;52(9):1463-1470. 
10. 
High DM. Advance directives and the elderly: a study of intervention strategies to 
increase use. Gerontologist. 1993;33(3):342-349. 
11. 
Landry FJ, Kroenke K, Lucas C, Reeder J. Increasing the use of advance directives in 
medical outpatients. J Gen Intern Med. 1997;12(7):412-415. 
12. 
Dipko LR, Xavier K, Kohlwes RJ. Advance directive group education in a VA outpatient 
clinic. Soc Work Health Care. 2003;38(2):93-106. 
13. 
Bruffee KA. Collaborative Learning: Higher Education, Interdependence, and the 
Authority of Knowledge. Baltimore: Johns Hopkins University Press; 1993. 
14. 
Sudore RL, Knight SJ, McMahan RD, et al. A Novel Website to Prepare Diverse Older 
Adults for Decision Making and Advance Care Planning: A Pilot Study. J Pain Symptom 
Manage. 2013. 
15. 
The Conversation Project.  http://theconversationproject.org/starter-kit/intro/. Accessed 
October 13, 2012. 
16. 
Lum HD, Jones J, Matlock DD, et al. Advance Care Planning Meets Group Medical 
Visits: The Feasibility of Promoting Conversations. Ann Fam Med. 2016;14(2):125-132. 
17. 
Centers for Medicare and Medicaid Services. "CMS Finalizes 2016 Medicare Payment 
Rules for Physicians, Hospitals & Other Providers." 30 Oct. 2015. 
https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2015-Press-
releases-items/2015-10-30.html. Accessed October 31, 2015. 
18. 
Lum HD, Sudore RL, Bekelman DB. Advance Care Planning in the Elderly. Med Clin 
North Am. 2015;99(2):391-403. 

19. 
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health 
promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322-
1327. 
20. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: 
conceptual distinctions, measurement challenges, and research agenda. Adm Policy 
Ment Health. 2011;38(2):65-76. 
21. 
Sudore RL, Stewart AL, Knight SJ, et al. Development and validation of a questionnaire 
to detect behavior change in multiple advance care planning behaviors. PLoS One. 
2013;8(9):e72465. 
22. 
Dunham R. Nominal Group Technique: A User's Guide. 2006.; 
http://www.sswm.info/sites/default/files/reference_attachments/DUNHAM%201998%20N
ominal%20Group%20Technique%20-%20A%20Users'%20Guide.pdf. Accessed 
September 23, 2015.