Hopewell Hospitalist: A Video Game Intervention to Increase Advance Care Planning Conversations by Hospitalists With Older Adults
Phase: NA
Conditions: Advance Care Planning
Sponsor: Dartmouth-Hitchcock Medical Center

v. 11/22/2021 
 
 
1 
 
 
 
 
 
 
 
 
EFFECTIVENESS: Hopewell 
Hospitalist: A Video Game Intervention 
to Increase Advance Care Planning by 
Hospitalists 
 
STUDY00031186 
- B NCT04881968 
11/22/2021 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

v. 11/22/2021 
 
 
2 
Dartmouth College • Dartmouth-Hitchcock Medical Center 
COMMITTEE FOR THE PROTECTION OF HUMAN SUBJECTS 
CPHS.Tasks@Dartmouth.edu • 603-646-6482 
 
SOCIAL, BEHAVIORAL, and NON-CLINICAL RESEARCH PLAN 
CPHS template v. 04172017 
 
Please complete: CPHS#     31980             PI: BARNATO 
 
 
Important Note:  The CPHS Department (Chair & Scientific) Review Form is required with this 
application.  Find the form in the RAPPORT Library or on the CPHS Website.  
 
● Respond to each item, even if to indicate N/A or not applicable 
● Attach and/or upload this form as your ‘Investigator Protocol’ in Rapport 
● If you are completing this form on a Mac, indicate your answer to any checkboxes by bolding or 
highlighting, or by deleting any incorrect options. 
 
 
 
1. Introduction and Background 
 
Advance Care Planning (ACP) is an integral part of the National Academy of Medicine’s objective of ensuring 
that patients receive person-centered, family-oriented, and evidence-based care, particularly at the end-of-life. 
However, less than half of all people in the U.S. have documented advance care plans, such as an Advance 
Directive. Existing guidelines therefore advocate that physicians use the opportunity of hospitalization to initiate 
ACP conversations.  
 
ACP involves discussion of goals, values, and preferences for health care with the patient, family members, 
or surrogates. Discussions may include hospice care, end-of-life care options, power of attorney for health care 
decisions, living wills, and physician orders for life-sustaining treatment (POLST).  
 
Numerous barriers impede the initiation of ACP in the hospital. High-quality conversations require 
physicians to have both the willingness and the time necessary to engage in these emotionally complex 
interactions. As a result, physicians typically defer ACP for all except the most critically ill. In contrast, experts 
advocate that these conversations occur prior to discharge for every patient over the age of 65. How best to shift 
physicians’ decisional thresholds – convincing them to engage in ACP conversations with all eligible patients – 
remains unclear.  
 
In prior work, the PIs developed a customized video game to recalibrate physician heuristics (intuitive 
judgments) in trauma triage, drawing on research from other domains suggesting the potential effectiveness of 
using narrative engagement to change behavior. Narrative engagement interventions use messages designed to 
resonate with recipients in personally relevant and meaningful ways. The PIs found that physicians using the 
game made fewer diagnostic errors on a validated trauma triage simulation than those randomized to use a text-
based educational program, a difference that persisted to six-month follow up. Given this success, this study 
hypothesizes that narrative engagement might also change physicians’ decisional thresholds and adapted the 
video game to address the problem of ACP in the hospital.  
 
This study is part of a larger project, “Physician Cognition, Inpatient Advance Care Planning, and Outcomes 
for Seriously Ill Older Adults,” and uses a pragmatic, minimal risk behavioral intervention to test the effect of the 
intervention, compared to usual care quality improvement alone, on the likelihood of ACP discussions by 
hospitalists with hospitalized patients aged 65 and older. Hospitalists employed by Sound Physicians – a large 
national physician practice that employs more than 2000 acute care providers in hospital medicine at more than 
200 hospitals in 38 states – are our study population. The primary outcome is billed ACP discussions.  
 

v. 11/22/2021 
 
 
3 
 
2. Objectives and Hypotheses 
 
 
This study will test the efficacy and effectiveness of using a customized theory-based video game to increase 
rates of ACP for hospitalized older adults. The rationale for testing the efficacy and effectiveness simultaneously is 
that the video game App developed in October 2019 will no longer function after the Apple iOS update, 
anticipated in October 2020. 
 
The primary outcome is physician billing for ACP (data available directly from Sound Physician-submitted 
claims with a 1 quarter lag). The primary hypothesis (H1) is that hospital ACP billing practices after distribution of 
the video game will increase (more than would be expected by secular trends alone). A priori post-hoc hypotheses 
include: (H2) The larger the proportion of hospital physicians who play the game, the greater the increase in ACP 
rates, and (H3) Differences in hospital billing practices before-and-after distribution of the video game will be 
positively associated with the proportion of physicians who have completed their organization’s ACP e-
curriculum. 
 
The secondary outcomes are episode-based patient healthcare use and mortality (data available from CMS 
with a 2-year lag). 
 
 
3. Study Design 
 
Note: All aspects of this study design will need to be re-appraised in light of the evolving Covid-19 pandemic 
and the associated secular disruptions in hospital medicine staffing, medical decision-making processes, and 
mortality among older adults. 
 
This study has two conditions: 1) Efficacy, and 2) Effectiveness. (Note: the term “efficacy” reflects the study 
arm with the greatest likelihood of intervention adherence and intervention fidelity – the direct provision of an 
iPad pre-loaded with the App; the term “effectiveness” reflects study arm that is more akin to what would happen 
in real practice – the provision of a URL to download the App). Hospitals that meet the selection criteria, and 
thereby the hospitalist participants, will be randomly assigned to one of the two conditions.  The conditions differ 
in terms of how the game intervention is delivered and when the intervention is delivered. The first condition 
receives the intervention in steps over the course of 3 months.  The second condition receives the intervention 
after that 3-month period concludes (Figure 1). The trial runs for a total of 6 months.   
 
All data will be collected through the game itself with Google Analytics and Qualtrics.  This data will be linked 
with Sound data and with Medicare claims that is regularly and continually collected by those external parties. 
 
All potential participants will be contacted by email to request their participation.  Efficacy condition 
participants will be directed to a consent form on Qualtrics.  After consenting, each participant will receive 
The usual care quality improvement includes: 1) mandatory and voluntary CME-accredited e-learning, 
including modules on compliance with CMS billing codes 99497 (ACP first 30 mins) and 99498 (ACP additional 30 
mins), templated ACP documentation, and training in serious illness communication; 2) compensation incentives 
rewarding quality and productivity performance, including $20/ACP conversation; and 3) handheld and desktop 
IT tools (“SoundConnect”) that enable coding and billing and facilitate coordination and compliance with clinical 
pathways. The effect of these efforts will be treated as the baseline to which the effect of the game will be 
compared.  
 
          The planned study will test both the effect of the video game intervention on ACP billing practices as well 
as the impact of the distribution strategy in a hybrid efficacy-effectiveness clinical trial. We will use linked Sound-
submitted claims data to quantify the influence of the intervention on ACP billing (available with a 3-month lag) 
and linked CMS-assembled claims data to quantify the influence of the intervention and any shifts in ACP 
behavior on secondary episode-based patient healthcare use and outcomes (available with a 2-year lag). 

v. 11/22/2021 
 
 
4 
information through Qualtrics about how to access the game.  The information will reflect the condition into 
which the participant has been randomized. 
 
In the Efficacy condition, 30 hospitals (conditional on the response rate) will be randomized into an order for 
a stepped wedge cluster randomized trial.  The stepped wedge trial randomizes hospitalist participants at the 
group level (i.e., hospital) rather than the individual level (i.e. hospitalist).  We will use a stepped-wedge trial 
design for two reasons: 1) individual-level randomization risks misclassifying patients, contaminating unexposed 
physicians, and failing to address group-level attitudes to and practices of ACP; 2) a parallel cluster randomized 
design risks imbalance among groups because of the high intra-class correlation coefficient that exists for billing 
practices at the hospital-level. Each group ‘crosses over’ from control to intervention at a randomized timepoint 
and multiple ‘time steps’ of data collection occur. All the Sound-affiliated hospitalists at each hospital will be 
invited to use the video game intervention. If a participant accepts, the participant will receive an iPad with the 
video game pre-loaded.  The iPad will be mailed to either a participant’s home address or the hospital site where 
it will be distributed by the clinical performance nurse or nurse liaison. We will contact and consent a small 
sample of 20-30 physicians who completed the efficacy portion of the trial for a semi-structured interview via 
Zoom to discuss their experiences playing the game. These semi-structured interviews will be recorded and 
transcribed, and participants will be compensated with a $50 Amazon gift card for participating in the interview. 
Finally, we will ask Sound to abstract the charts of a random sample of 200-300 patients treated by trial 
physicians.  
 
          In the Effectiveness condition, all eligible hospitalists who staff 170 hospitals (not enrolled in the Efficacy 
condition) will receive an email with instructions on how to access the game on the iOS Application Store and 
encouragement to do so. The recruitment email will be sent by Sound Physicians and include both a link to play 
the game and a short video highlighting a selection of positive experiences from participants in the Efficacy 
condition. The email and informational video are intended to encourage participation.  The positive gameplay 
experiences will be collected via short 15-20 minute recorded zoom interviews. Ten physicians who played the 
game in the Efficacy condition and responded positively in their post-game surveys will be invited to participate 
by email The physicians who agree to be interviewed for the informational video will be allowed to review the 
content selected for use in the video, as well as approve or deny the use or distribution of that content.  
 
 
 
 

v. 11/22/2021 
 
 
5 
 
Figure 1. Overall trial design. The left branch displays the Efficacy condition. The Right Branch displays the Effectiveness condition. 
 
 
 
4. Analysis 
 
We plan to conduct intention-to-treat analyses with two-tailed significance testing at an alpha of 0.05, 
with no adjustment for multiple comparisons.  
 
 
We will begin by calculating ACP billing practices for each Sound hospital, defined as the proportion of 
eligible patients who had an ACP conversation billed during their hospitalization, during the quarter before 
and after distribution of the video game intervention. To test the efficacy of the video game, we will compare 
billing practices before and after distribution at hospitals enrolled in the efficacy trial using a Student’s t-test. 
To test the effect of the delivery method, we will fit a mixed effects patient-level logistic regression model 
with presence of ACP billing during the hospitalization as the dependent variable. The hospital’s assignment 
and the epoch of the hospitalization (i.e. before or after distribution of the intervention) will serve as the 
primary independent variables. Covariates will include patient characteristics (age [by 5 year increments], sex, 
admission diagnosis, selected comorbidities, organ failure), physician characteristics (age, sex, race, ethnicity, 
educational background [board certification, years since completion of residency, location of medical school, 
location of residency/fellowship], practice environment [number of patients evaluated/week of service, 
number of weeks on service/year], exposure to ACP-interventions [e-learning curriculum developed by Sound 
Physicians], amount of use of the game intervention), and hospital characteristics (geographic region, size, 
presence of palliative care specialists, proportion of physicians who used the video game, median length of 
time physicians used the video game, proportion of physicians who had previously completed Sound’s e-
curriculum training in ACP billing practices). Results from this analysis will also allow us to assess mediators of 
the effect of the video game. In exploratory analyses, we will test the correlation between billed and 
documented ACP conversations using Spearman’s correlation coefficient. We will test the effect of the video 
game on physician behavior using the self-reported MiPS-ACP quality score using Student’s t-tests.  We will 
assess the validity ACP billing as a measure of the effect of the intervention on physician performance through 
a planned chart review. Working with Sound Physicians our team will review the charts of 200-300 patients 
treated by physicians who in our efficacy trial. Our team will review each patient chart to determine if ACP 
conversations are documented in the chart. We will then compare this to billing data to quantify the 
agreement between these two measures of ACP conversations. 
 
 
Characteristics of the physicians who use the video game will be summarized means (standard 
deviations) for continuous variables and proportions (%) for categorical variables, and with comparisons of 
the distributions of characteristics between the ‘steps’ in the trial using Students t-test and chi-squares as 
appropriate. The data will be summarized and compared hospital characteristics for each arm/group of the 
trial similarly. Of physicians who use the video game, we will categorize quantitative and qualitative feedback 
as positive or negative and will assess opinions about the tolerability of the interventions. We will also 
summarize responses to the Narrative Engagement Scale and the Positive and Negative Affect Scale for 
participants in the efficacy condition.  
 
 
The participation rate will be calculated as the proportion of physicians at hospitals in each arm/group 
of the trial who log into the video game. We will summarize the length of time that physicians spend playing 
the game. We will compare participation at hospitals in different arms/groups of the trial using chi-square 
tests, and the duration of exposure using chi-square tests and Students t-tests. Information about adherence 
will provide insight into the tolerability of the game and will also allow for exploratory analyses into the 
mechanism of the intervention’s success or failure. The statistical model for the primary analysis in the 
Efficacy condition emulates a cluster-level stepped-wedge design in that each hospital begins in the control 
setting and at some-point transitions to the treated (post video game roll-out) setting. For patient i seen by 
physician j at hospital h in time-period t let 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 be a binary random variable indicator whether has an billed 

v. 11/22/2021 
 
 
6 
ACP conversation; 𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡𝑗𝑗𝑗𝑗 be a time-varying hospital indicator of whether the game has been rolled-out 
(𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡𝑗𝑗𝑗𝑗= 1 ) or has yet to be rolled-out (𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡𝑗𝑗𝑗𝑗= 0 ); 𝑥𝑥𝑖𝑖𝑖𝑖𝑖𝑖 be a vector of patient-level controls; 𝜃𝜃𝑗𝑗ℎ a random 
effect for physician within hospital and hospital and 𝜆𝜆ℎ a random-effect for hospital. Including time-period as 
a categorical predictor the model is given by: 
 
𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙(Pr (𝑌𝑌𝑖𝑖𝑖𝑖ℎ𝑡𝑡= 1|𝜃𝜃𝑗𝑗ℎ, 𝜆𝜆ℎ)) = 𝛽𝛽0 + 𝛽𝛽1𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡ℎ𝑡𝑡+ 𝛽𝛽2𝑥𝑥𝑖𝑖𝑖𝑖ℎ𝑡𝑡+ 𝛽𝛽3𝑡𝑡+ 𝜃𝜃𝑗𝑗ℎ+ 𝜆𝜆ℎ 
 
The key coefficient is 𝛽𝛽1, which captures the structural shift in the outcome that occurs when a patient 
attends a hospital after the roll-out of the game versus prior to this time. Because 𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡ℎ𝑡𝑡 does not vary within 
a time-period, the information in the observations made within a hospital and time-period depends on the 
size of the effects of hospital and time-period with the statistical significance of inferences about 𝛽𝛽1 likely to 
be reduced by the clustering of observations in physicians. The specification of this model in Stata is: 
 
melogit billed_acp patient_covariates epoch || hosp: || md: 
 
 
The primary outcome secondary analysis differs from the above model by conditioning on the extent 
to which the Sound physicians participated in the game. Let 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡 denote a measure of the extent to 
which the physicians that patient i encountered had played the game by time-period t and 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒ℎ𝑡𝑡 denote a 
measure of the extent to which all of the physicians at hospital h have played the game by time-period t. 
Several specifications for the statistical model will be considered. The simplest uses 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡 as a predictor 
and sets  𝐺𝐺𝑎𝑎𝑎𝑎𝑒𝑒𝑗𝑗ℎ𝑡𝑡= 1 if any physicians of patient i have played the game by time-period t and 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡= 0 
otherwise. We label this as scenario (i). Under scenario (i) if 𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡𝑗𝑗𝑗𝑗= 0 then 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡= 0 for all j in time-
period t. However, if 𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡𝑗𝑗𝑗𝑗= 0 then it can be the case that 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡= 0 (this occurs when the game has 
been rolled-out at the hospital but physician j hasn’t played it yet). Scenario (ii) sets 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡 to be the 
proportion of the physicians a patient actually saw who have played the game. Scenarios (iii) and (iv) are as 
for (i) and (ii) but replace 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡 with 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒ℎ𝑡𝑡. For all such scenarios, the model is given by: 
 
𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙(Pr (𝑌𝑌𝑖𝑖𝑖𝑖ℎ𝑡𝑡= 1|𝜃𝜃𝑗𝑗ℎ, 𝜆𝜆ℎ)) = 𝛽𝛽0 + 𝛽𝛽1𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡ℎ𝑡𝑡+ 𝛽𝛽2𝑥𝑥𝑖𝑖𝑖𝑖ℎ𝑡𝑡+ 𝛽𝛽3𝑡𝑡+ 𝛽𝛽4𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒∗ℎ𝑡𝑡+ 𝜃𝜃𝑗𝑗ℎ+ 𝜆𝜆ℎ 
 
where the * implies that the presence or absence of the j subscript distinguishes the scenarios when 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒𝑗𝑗ℎ𝑡𝑡 
and 𝐺𝐺𝐺𝐺𝐺𝐺𝑒𝑒ℎ𝑡𝑡 are used respectively. It is important ot note that the model retains 𝑃𝑃𝑃𝑃𝑃𝑃𝑡𝑡ℎ𝑡𝑡 as a predictor in 
order to allow for the possibility that the roll-out of the game affects outcomes even if physician(s) do not 
play the game. 
 
Stata: melogit billed_acp patient_covariates epoch physician_exposure proportion_physicians_exposed || 
hosp: || md: 
 
 
5. Study Progress Monitoring 
 
This study will be overseen by a Data Safety and Monitoring Board.  The DSMB has been empaneled.  
It will be approved and the first meeting will be held before the start of the trial. In the process of 
establishing the DSMB charter, DSMB members will provide guidance on safety surveillance and reporting 
requirements.  
 
 
As determined by NIH, the DSMB will “1) periodically review and evaluate the accumulated study 
data for participant safety, study conduct and progress, and, when appropriate, efficacy, and 2) make 
recommendations to NIDCR concerning the continuation, modification, or termination of the trial. The 
DSMB considers study-specific data as well as relevant background knowledge about the disease, test agent, 
or patient population under study.” 
 
 
As described by NIH, “The DSMB is responsible for defining its deliberative processes, including: 
event triggers that would call for an unscheduled review, stopping procedures that are consistent with the 

v. 11/22/2021 
 
 
7 
protocol, unmasking (unblinding), and voting procedures. The DSMB is also responsible for maintaining the 
confidentiality of its internal discussions and activities as well as the contents of reports provided to it. 
 
 
The DSMB should review each protocol for any major concern prior to implementation. During the 
trial, the DSMB should review cumulative study data to evaluate safety, study conduct, and scientific validity 
and integrity of the trial. As part of this responsibility, DSMB members must be satisfied that the timeliness, 
completeness, and accuracy of the data submitted to them for review are sufficient for evaluation of the 
safety and welfare of study participants. The DSMB should also assess the performance of overall study 
operations and any other relevant issues, as necessary.”  
 
 
All eligible participants will be contacted at the start of the trial and receive the interventions as 
described under Section 3.  The trial will run for a total of 6 months. 
 
 
 
 
 
6. Risks & Benefits 
 
Note: Risks may be physical, psychological, social, legal, economic, to reputation, or others.  
 
 
a. Describe any potential risks, their likelihood and seriousness: 
There are two categories of participants in this pragmatic trial: 1) Consented research participants 
(hospitalists); and 2) Unconsented research participants (patients of participating hospitalists). 
 
The consented research participants are the hospitalists employed by Sound Physicians. A key consideration 
in this study that the participants’ employer is a partner in conducting the trial. The primary outcome is the 
hospitalists’ behavior; specifically, the frequency with which hospitalists bill for ACP conversations with patients 
65 and older.  
 
The unconsented research participants are the patients of participating hospitalists. Since the trial seeks to 
influence hospitalist ACP behavior during patient encounters, hospitalists’ patients will be indirectly affected.  A 
key consideration is that this study randomizes participation at the level of the hospital, and consent at the 
physician level, with the intent to improve the quality and patient-centeredness of patient care.  However, 
neither the hospitals nor the physicians will obtain consent from their patients  
 
Consented research participants (hospitalists): 
Potential Risks: There are no health risks to consented research participants (hospitalists). The three major 
potential risks are lack of voluntariness (coercion), impact on employability, and loss of confidentiality.  
 
Lack of voluntariness (i.e., undue influence). A central tenet of clinical research is that participation is 
voluntary. This study will make it very explicit that research participation is not a condition of employment by 
Sound. However, there may yet be a perception of lack of voluntariness. This study cannot completely separate 
research participation from employer purview. Study implementation requires the disclosure of study 
participation to members of the Sound study team who are responsible for programming the information 
technology (IT) platforms that will be used to deliver the intervention(s).  
 
Confidentiality. Participant data will be collected and stored with physician identifiers (e.g., names and 
UPIN numbers linked to unique study IDs). Strict confidentiality and privacy procedures have been put in place 
to protect against any potential public disclosure of individual provider performance data. All physician-level 
data will be de-identified for analysis and reporting. A compromise of security of survey, interview, intervention 
adherence, and ACP performance data outside of the study would result in a breach of confidentiality and 
violation of privacy.  
 

v. 11/22/2021 
 
 
8 
Sound and TDI each have detailed policies and procedures in place for the physical and electronic 
security of the SoundConnect and Medicare claims data.  
 
Employability. Study participation is not a condition of employment. Sound collects and reports multiple 
measures of hospitalist performance. These are conditions of employment. Hospitalists already receive data on 
their own ACP performance monthly, which is reported to their direct hospital supervisors in comparison with 
all of the hospital’s other hospitalists’ performance. There is no central Sound policy regarding how hospital 
supervisors use any performance data, including ACP performance. This study will not change these reporting 
procedures. Therefore, responses should not influence employability. Indeed, if the intervention(s) are 
effective, participants randomized to active treatment will improve their performance. A theoretical concern is 
if increased attention to ACP crowded out hospitalists’ focus on other quality targets. While this was not 
originally a secondary outcome proposed in the original protocol, the administrative and statistical feasibility of 
adding this safety measure will be assessed (see below). 
 
Unconsented research participants (patients of hospitalists): 
The three main risks to unconsented research participants (patients of participating hospitalists) are 
violation of rights due to non-consent and lack of voluntariness, breach of confidentiality, and shortened 
survival time associated with preference-concordant treatment associated with modifying code status to “do 
not resuscitate” or completing ACP documents such as physician orders for life-sustaining treatment (POLST) as 
well as potential medical undertreatment associated these orders.  
 
ACP meets the minimal risk standard as defined at 45 C.F.R. § 46.102(i), “Minimal risk means that the 
probability and magnitude of harm or discomfort anticipated in the research are not greater in and of 
themselves than those ordinarily encountered in daily life or during the performance of routine physical or 
psychological examinations or tests.” ACP involves discussion about goals, values, and treatment preferences 
with the patient, family members, or surrogates. Discussions may include hospice care, end-of-life care options, 
power of attorney for health care decisions, living wills, and physician orders for life-sustaining treatment 
(POLST). Engagement in comprehensive ACP conversations are not strictly required and practices vary widely. 
Specifically, while the Joint Commission requires that patients be asked whether they have a pre-existing 
advance directive at the time of acute care admission and that code status be documented. Comprehensive ACP 
conversations are not currently a Joint Commission requirement. Medicare encourages comprehensive ACP 
conversations by paying for these services, but their policy states that the services are “voluntary,” meaning 
that if a provider broaches an ACP conversation with a patient, they must provide an opportunity to decline the 
services and documented in the record. Therefore, having or not having a comprehensive ACP conversation 
during an acute care admission are both currently medically recognized standards of care.  
 
Lack of voluntariness (i.e., non-consent). As outlined above, patients of consented and randomized 
hospitalist will not be provided with a choice regarding whether they are treated by an intervention or control 
hospitalist. This situation is common to cluster randomized studies designed to improve adherence with clinical 
practice guidelines.  
 
Confidentiality. The source of patient-level data for clinical trial outcomes is administrative billing data. 
This includes data collected by Sound Physicians for the purposes of billing the Centers for Medicare and 
Medicaid Services (CMS) and other payers for physician services during the index hospitalization (primary 
outcome: ACP billing) and additional claims data from CMS for data on predictors/risk-adjusters and secondary 
outcomes.  
 
The main procedures to protect against risks (lack of voluntariness, confidentiality, employability) to 
participation by the consented research participants (hospitalist physicians) involve separating research- and 
operations-related elements of the study. Specifically, Dartmouth team members, not Sound Physician team 
members, will complete key research-related activities such as recruitment, informed consent, primary data 
collection, data analysis and storage.  
 

v. 11/22/2021 
 
 
9 
To protect against lack of voluntariness, confidentiality, and threats to employability, no Sound 
Physician employee will participate in the recruitment and informed consent process beyond introducing the 
study and Dartmouth investigators to potential research participants. Any reports regarding undue influence 
will be treated as a SAE. No member of the study team will disclose study participation to Sound leadership 
(including co-investigator Birkmeyer) or to the hospitalists’ direct supervisors.  
 
Data elements will be entered directly by the research coordinator; these files are backed-up daily and 
archived weekly. All study subjects will be assigned unique study identifiers that will appear on all data 
collection instruments, documents, and files used in the statistical analysis and manuscript preparation. Limited 
team members will have access to identifiers needed for tracking and informed consent. We will follow TDI data 
security procedures to protect the confidentiality of physician research participant data along with Medicare 
beneficiary billing claims data (see below). 
 
The principal means of protecting against violation of patient rights and harming welfare of 
unconsented research participants (patients of participating hospitalists) for pragmatic trials like ours is 
thoughtful review by multiple experts in human subjects research ethics and regulation, including NIH staff, an 
independent DSMB, and the Dartmouth Institutional Review Board. The physician-directed behavioral 
interventions proposed in the current study augment a suite of non-randomized quality improvement 
interventions already deployed by Sound Physicians to increase ACP by hospitalists which are not subject to 
such review or to rigorous safety or efficacy evaluation.   
 
The PIs will audit a purposive sample of ACP conversations during the trial to assess quality. They will 
follow TDI data security procedures to protect the confidentiality of Medicare beneficiary billing claims data 
(see below).  
 
Data Security Procedures:  
 
 
TDI has a long and flawless history in maintaining and using extremely large confidential databases. The 
mission of the Data Analytic Core (DAC) at The Dartmouth Institute for Health Policy and Clinical Practice (TDI) is 
to provide support to research that contributes to the effectiveness, quality, equity, and efficiency of health 
care within the United States, which is complemented by its promise to respect personal privacy, safeguard the 
confidentiality of data and to provide a secure environment for the databases under its management. The DAC 
functions as the Data Coordinating Center for TDI. As detailed in the DAC Policies and Standards Manual (PSP), 
the DAC meets this commitment by ensuring data anonymity; maintaining principles and policies for the 
protection 55of health care data including strict policies which limit access to data and heightened security 
measures (organizational, technological, and physical) according to both FISMA (Federal Information Security 
Management Act) and HIPAA (Health Information Portability and Accountability Act) requirements; reviewing 
and approving research proposals requesting use of the data sets controlled by the DAC; assuring that 
appropriate and current data use agreements, institutional reviews and other approvals are in place for 
research projects; an active IT Security Committee, at the working and governance levels;  annual staff training 
to keep health information protection matters a constant priority; requiring that ALL TDI and the Data and 
Analytic Core (DAC) users sign a TDI DAC Policies, Standards, and Procedures Information Security 
Acknowledgement and Agreement; regularly reviewing the DAC policies to ensure they are in line with current 
health information legislation and protection practices.  
 
 
Additionally, the DAC must adhere to both FISMA and HIPAA requirements as well as CMS requirements 
to ensure the security of protected health information (PHI). PHI is the terminology used in HIPAA legislation 
and by the Centers for Medicare and Medicaid Services (CMS). Under HIPAA, PHI consists of any “individually 
identifiable data such as SSN, birth date, admission date, discharge date or zip code”. The claims data used for 
research within the DAC at TDI contain files with PHI. The safeguards vary depending on the amount, 
distribution, format of the information, and the method of storage.  
 
Description of Security Measures 

v. 11/22/2021 
 
 
10 
 
The methods of protection in place for both the claims data and the physician level data as outlined 
above include:   
1. Physical measures; e.g., locked facility with tracked key access, locked filing cabinets, and restricted 
access to offices; 
2. Organizational measures; e.g., strict employee confidentiality agreements (with restrictive actions as a 
sanction) and limiting access on a “need-to-use” basis;  
3. Technological measures; e.g., the use of firewalls, “moating” of administrative data making it 
inaccessible externally, passwords, and encryption of data; use of VPN terminals and networks; and,  
4. Anonymity of data by stripping off conventional identifiers for data provided to users (when 
conventional identifiers are not necessary for research).  
 
 
As a condition of employment, all staff (full-and part-time) must sign the DAC Policies, Standards, and 
Procedures Information Security Acknowledgement and Agreement prior to working with the data. Staff 
includes TDI faculty, adjunct faculty, fellows, research analysts, programmers, students, and other approved 
researcher staff seeking access to or utilizing the DAC.  
 
 
All DAC servers, data storage, switches, and other peripherals are stored in isolated rack cages with 
DAC-dedicated key locks limiting access to only the DAC Network Systems Administrator. All DAC servers, are 
maintained by Dartmouth computing. All data traffic for the project is contained within a private, dedicated 
DAC VLAN network. All network traffic to the servers must pass through the Dartmouth hardware firewall. The 
only access to the servers is via SSH and requires 2-factor authentication (NetId/password and DUO). All 
networks located on switches are segmented and behind Dartmouth firewalls where the DAC data resides.  
 
 
Derivative data variable definitions and labeling will be confidential and described as per data type and 
use. Labeling will not include any PHI. 
 
 
 
b. Describe why all the risks to subjects are reasonable in relation to both anticipated benefits 
and the knowledge expected to be gained from the study: 
  
 
There are no direct benefits to the study participants. Knowledge gained from the study will contribute 
to the academic literature on physician quality improvement and the efficacy of video games as behavioral 
interventions. This knowledge will inform how advances in technology may transform the means by which 
behavioral and social science interventions are delivered. These technologies ensure treatment fidelity and can 
extend treatment duration, thereby improving behavioral maintenance.  
 
 
Further, this study will inform how these technologies may be used in the context of ACP, as well as 
broadly because ACP is an extension of the game’s original use-case of trauma triage. This study will provide 
information about how hospital-based physicians initiate ACP conversations for patients admitted acutely and if 
the decision to initiate a conversation can be improved with a customized theory-based video game. 
 
 
 
7. Unexpected Events or Incidental Findings 
 
Note: It may be important to consider the potential for certain unanticipated events to occur, for example: 
● finding an anomaly in a MRI 
● discovering child abuse 
● causing distress in interviews of a sensitive nature 
 
Describe potential events and provide a plan of action: 
 
 
Unexpected events may include: unexpectedly high rates of advance care planning billing that raise 
concerns regarding Medicare billing fraud, or systematic biases or disparities in advance care planning 

v. 11/22/2021 
 
 
11 
conversations. No action will be taken regarding unexpectedly high advance care planning billing rates. Systemic 
biases will be reported back to the hospital without any identifying information. There is the potential that 
participants may begin to feel uncomfortable during the semi-structured interviews. While unexpected, if 
such adverse event occurs, we will discontinue the interview and connect participants with any relevant 
resources.  
 
8. Deception 
 
Does any part of this study involve deception or withholding of information from participants? 
 
☐  Yes 
 
☒  No 
 
If Yes, provide an explanation which addresses the following: 
● A description of the deception being used 
● Why the deception is necessary 
● A plan for debriefing, or providing subjects with the pertinent information after participation 
 
 
9. Equitable Participant Selection 
 
a. Estimated number of participants at Dartmouth CPHS reviewed sites:  
 
Sound Physicians is a national physician practice that employs acute care providers in hospital medicine, 
emergency medicine and critical care. We selected this physician practice as a partner in our efforts to test the 
effect of the video game for three reasons: 1) it staffs over 200 hospitals with a wide variety of geographic and 
organizational characteristics, increasing the generalizability of our observations; 2) it has already implemented 
best-practice quality improvement efforts to improve ACP practices at its hospitals, including providing employees 
with web-based didactic education on the importance of ACP in the hospital and a small financial incentive ($20) 
for each documented ACP conversation that meets Medicare’s criteria for reimbursement; 3) billing practices 
remain lower than desired.   
 
 
We will partition the 200 Sound hospitals into two groups: Efficacy condition hospitals (n=30), 
approximately 120 physicians (20-30 of whom will be invited for semi-structured interviews), and Effectiveness 
condition hospitals (n=170), approximately 1700 physicians with an average of 10 physicians at each site in this 
group.).  We will begin by stratified sampling of hospitals for the efficacy trial, such that there is a range of ACP 
billing practices and numbers of hospitalists. Eligible hospitals must employ clinical performance nurses (nurse 
liaison), who will both distribute the iPads to participants and collect secondary outcome measures. The remainder 
of the hospitals will be in the effectiveness arm of the trial.  
 
 
We will allocate hospitals to the efficacy and effectiveness arms of the trial. Hospitals in the efficacy arm 
will be randomized to the order in which they receive the video game. We will generate randomization schemas 
using Stata 13.0 (Statacorp, College Station, TX, USA) statistical software, using random block sizes of 4. Although 
we cannot blind study personnel and participants, we will mask the arm and group assignment during the analysis 
phase. 
 
 
b. Provide a justification of the proposed sample size 
 
For the Efficacy condition, we plan to recruit 120 physicians at 30 hospitals (conditional on willingness to 
participate) and will expose them sequentially to the video game intervention. If fewer physicians per hospital 
agree to participate, we will plan to increase the number of hospitals allocated to the efficacy trial. 
 
Assuming a baseline ACP rate of 14% (rising by 1.25%/quarter), a hospital intra-class correlation 
coefficient of 0.008-0.115, and 160 patients/physician/quarter, with 120 physicians, we can detect a 3% 

v. 11/22/2021 
 
 
12 
difference between ACP practices before and after the distribution of the intervention with an alpha of 0.05 and 
80% power. 
 
For the Effectiveness condition, we will recruit 170 hospitals. This represents the remainder of the 
Sound hospitals. As part of our agreement with the funding agency, we are to deliver the intervention to all 
Sound hospitalists. The effectiveness study, therefore, is not specifically powered. 
 
c. Define the target population: 
 
The target population is physicians whose job responsibilities include ACP.  In this study, all inpatient 
hospitalist physicians who have worked for Sound Physicians at a study hospital for at least one quarter are 
eligible to participate. There are approximately 200 hospitals across the US providing a total of approximately 
2000 eligible hospitalists.  The study aims to balance the sample in the following characteristics:  
● A hospital’s ACP billing rate. 
● Number of hospitalists at a hospital. 
● If the hospital is an academic medical center. 
● Availability of Clinical Performance Nurse to pull charts. 
 
Additionally, for the Efficacy condition, the inclusion criteria are: a hospitalist must have worked for Sound for at 
least 6 months, report use of ACP billing codes, provide contact information that matches a contact in our list of 
eligible physicians (i.e. full name match or partial name match and employer-based email match), and receive a 
working iPad. If participants do not meet any of these criteria, they will be informed that they are ineligible, but 
that they may play the video game intervention voluntarily and without compensation for their time. They will 
receive instructions on how to download and play the game, as well as complete the post-game questionnaire. 
Communications with eligible and ineligible participants may be modified if a participant contacts the study team 
directly with specific questions or concerns. The response will be modified to address the participant’s needs. 
 
 
 
 
d. Vulnerable populations 
 
Note: Certain populations are considered vulnerable to coercion and undue influence and are provided 
with additional protections when participating in a research study.   
 
Identify any of the below populations which you plan to recruit for this study.  In addition, 
complete the form(s) linked with each population as necessary and upload on the 
‘Supporting Documents’ page in Rapport.  
☐ Pregnant Women, Fetuses and Neonates 
☐ Children 
☐ People with impaired decision-making capacity 
 
The following populations may also be considered vulnerable to coercion or other undue 
influence: 
● Prisoners 
● People who are economically disadvantaged  
● The elderly  
● People who are illiterate or do not speak English 
● Students and employees 
 
Describe any other potentially vulnerable population(s) and the additional protections 
provided to them: 
 
Employees. See section 6. 

v. 11/22/2021 
 
 
13 
 
 
10. Recruitment 
 
Describe method(s) of recruitment.  Associated advertisements and other materials to be used for 
recruitment should be uploaded to the ‘Consent Forms and Recruitment Materials’ page in 
Rapport. 
 
All participants will be offered the opportunity to “opt in” to participate.  
 
Those who engage with the study procedures are provided with consent information at each step of 
engagement; including an agreement to proceed with pre- and post-game survey questionnaire completion 
on the face page of the Qualtrics surveys (efficacy); and information about the data captured by game use at 
the time of downloading the game App (effectiveness condition). 
 
Recruitment and “Opt-In” Consent Process: Game Use. All Sound hospitalist participants in the efficacy 
condition will provide “opt-in” consent for active participation (e.g., use of the game and provision of 
personal information). Sound Physician leadership will introduce the research study to regional and hospital 
team leaders at the annual leadership conference and to all eligible hospitals via e-mail, with a web-based 
link to a Qualtrics survey to opt-in to study participation. We seek a waiver of documentation from the 
Dartmouth IRB to use electronic consent documentation. 
 
Claims analysis. Sound hospitalist participants and their patients will not provide consent for use of their 
billing data. We have relevant data use agreements with Sound Physicians and CMS. We seek a waiver of 
informed consent from the Dartmouth IRB. 
 
11. Informed Consent, Assent, and Authorization 
 
All forms discussed in this section should be uploaded to the ‘Consent Forms and Recruitment 
Materials’ page in Rapport 
 
a. Please describe the consent and/or assent process, addressing the following: 
● Who will obtain consent/assent from participants 
● Where the consent/assent process will take place 
● The timeframe for providing information potential participants about a study, having the consent 
form signed, and beginning study activities 
● Any precautions taken to minimize the possibility of coercion or undue influence 
● The forms which will be used as well as any aids used to simplify scientific or technical 
information 
● How comprehension will be ensured 
 
 
Those who engage with the study procedures are provided with consent information at each step of 
engagement; including an agreement to proceed with pre- and post-game survey completion on the face 
page of the Qualtrics surveys (electronic research consent, efficacy consent); and information about the 
data captured by game use at the time of downloading the game App (electronic “terms of use” agreement, 
effectiveness condition).       
 
b. Waiver(s) or alteration(s) may be requested for research that involves no more than minimal 
risk.   
Indicate requested waiver(s) or alteration(s) below.  In addition, complete the corresponding section 
of the Waivers and Alterations Request Form and upload it to the ‘Consent Forms and 
Recruitment Materials’ page in Rapport.      
☐  For the informed consent process 

v. 11/22/2021 
 
 
14 
☒ For the documentation of informed consent 
☐  For the HIPAA Authorization to use and/or disclose PHI 
☐ For a waiver of the requirement for medical record documentation 
 
 
12. Compensation or Gifts 
 
Please describe any payments, gifts or reimbursements participants will receive for taking part in the study: 
 
 
All participants in the Efficacy condition will be compensated for their time.  These participants will keep the 
iPads provided during the study. Participants from the efficacy condition who complete a semi-structured interview 
will be compensated with a $50 Amazon gift card for participating in the interview.  
All participants in the Effectiveness condition who are sent the game URL only will receive up to 2 AMA PRA 
Category 1 Credit(s) TM if they complete the continuing medical education (CME) evaluation form. 
 
 
13. Privacy of Participants  
 
Note: Methods used to obtain information about participants may have an effect on privacy.  For example: 
● Consent discussions or interviews held in public which concern sensitive subjects or behaviors 
● Observations of behavior, especially illicit behavior, in quasi-public settings 
 
Describe any activities or interactions which could lead to a breach of privacy and provide a plan to 
protect participant privacy: 
 
See section 6. 
 
 
 
14. Confidentiality of Data 
 
Note: Any person engaged in research collecting information that could cause financial, social or legal harm 
to participants may apply for a Certificate of Confidentiality.  Certificates of Confidentiality are 
issued by the National Institutes of Health (NIH) to protect identifiable research information from 
forced disclosure. They are intended to allow the investigator and others who have access to research 
records to refuse to disclose identifying information on research participants in any civil, criminal, 
administrative, legislative, or other proceeding, whether at the federal, state, or local level.  
 
a. If disclosed, could any of the data collected be considered sensitive, with the potential to 
damage financial standing, employability, insurability, or reputation? 
 
☐  No 
 
☒  Yes 
 
If Yes, describe the data or information, the rationale for their collection, and whether a 
Certificate of Confidentiality will be obtained: 
 
See Section 6. 
 
 
b. Describe the safeguards employed to secure, share, and maintain data during the study, 
addressing any of the following which may apply:  
● Administrative, ie. coding of participant data  
● Physical, ie. use of locked file cabinets 
● Technical, ie. encrypted data systems 
 
See Section 6. 

v. 11/22/2021 
 
 
15 
 
c. Describe the plan for storage or destruction of data upon study completion: 
 
See Section 6.