Primary Palliative Care for Emergency Medicine (PRIM-ER)
Conditions: Emergencies
Sponsor: NYU Langone Health

Official Title: 
Primary Palliative Care Education, Training, and Technical Support for 
Emergency Medicine (PRIM-ER)
NCT Number:
NCT03424109
Study Number:
18-00607 
Document Type:
Study Protocol and Statistical Analysis Plan
Date of the 
Document: 
 
October 30, 2022 

1 
Protocol Name
Primary Palliative Care Education, Training, and Technical 
Support for Emergency Medicine (PRIM-ER) 
Principal Investigator 
Keith Goldfeld, DrPH, MS, MPA
NYU Grossman School of Medicine
Department of Population Health
180 Madison Avenue, Room 5-51
New York, NY 10016 
Keith.Goldfeld@nyulangone.org
(646) 501-3650
Primary Contact Name/Info
Allison Cuthel
212-263-8631  
Allison.Cuthel@nyulangone.org
NYULMC Study Number
18-00607
Initial Version Date: 07/25/2018
Version 9: 10/26/2022 
1. Objectives 
 
We propose the implementation and testing of a novel, highly efficient pragmatic intervention that seeks to shift 
the clinical practice paradigm of emergency medicine. We propose a pragmatic, cluster-randomized stepped 
wedge design to test the effectiveness of primary palliative care education, training, and technical support for 
emergency medicine (PRIM-ER) in 35 Emergency Departments (EDs). PRIM-ER includes four core 
components: 1) evidence-based multidisciplinary primary palliative care education, 2) simulation-based 
workshops on communication in serious illness, 3) clinical decision support, and 4) provider audit and 
feedback. These core components will be implemented in each participating health system as part of a quality 
improvement initiative to improve the care older adults with serious illness receive in the ED setting. In the UG3 
phase of the project, we will: 1) tailor PRIM-ER to the emergency provider workforce and a more diverse ED 
context using an agile implementation framework approach; and 2) pilot test PRIM-ER at two sites for 
feasibility, fidelity, and usability. In the UH3 phase, we will: 3) implement PRIM-ER in a cluster-randomized, 
stepped wedge design in the remaining 33 EDs; and 4) measure the effect of PRIM-ER on aspects of: a) ED 
disposition to an acute setting; b) healthcare utilization in the 6 months following the ED visit; and c) survival 
following the index ED visit. The project will thus be implemented in a total of 35 EDs (2 pilot sites and 33 
remaining EDs) as previously approved and outlined in the below table. We hypothesize that it will be feasible 
to test PRIM-ER at two EDs with a high level of fidelity and usability and implement at all sites. We also 
hypothesize that older adult visitors with serious, life-limiting illness cared for by providers with primary 
palliative care skills will be less likely to be admitted to an inpatient setting, more likely to be discharged home 
or to a palliative care service, and will have higher home health and hospice use, fewer inpatient days and ICU 
admissions at 6 months, and longer survival than those seen prior to implementation. Additionally, we 
hypothesize that sites with higher baseline ED disposition to an acute care setting and less primary palliative 
care knowledge and skills will demonstrate greater change after implementation. 
 
Embedded under the PRIM-ER study, will be expanding our analysis to consider factors related to home and 
community services available for older adults with serious life-limiting illness, by considering contextual 
(regional, state, and healthcare-system) factors that influence access to home and community health services 
for older adults with serious life-limiting illness among the 18 PRIM-ER healthcare systems across the United 

2 
States. To understand the contextual factors we will conduct 1) an environmental scan/grey grey literature 
review to identify contextual factors impacting access to home and community health services, and 2.) Semi-
Structured qualitative interviews with emergency medicine practitioners to capture provider perspectives and 
expand our contextual factors to include available resources in the healthcare service area. Throughout the 
protocol, we will continue to refer to these items as the PRIM-ER Contextual Assessment.  
 
 
2. Background 
 
The high intensity of end-of-life care in the United States (US) is now considered an epic public health problem. 
Persons receiving many life-sustaining therapies do not appear to show a benefit of better health or longer 
life.1 Emergency Departments (EDs) care for society’s most vulnerable older adults who present with 
exacerbations of chronic disease at the end of life, yet the clinical paradigm continues to focus on treatment of 
acute illness and injury. Palliative care interventions in the ED capture high-risk patients at a time of crisis and 
can dramatically improve patient-centered outcomes.2,3
Half of Americans 65 years and older are seen in the ED in the last month of life, and three-quarters visit the 
ED in the six months before their death.4 Emergency care has not fully adapted to the needs or goals of 
seriously ill patients who prefer to have care delivered at home.5,6 Palliative care teams are now present in over 
two-thirds of hospitals, as well as 98 percent of National Cancer Institute-designated cancer centers.7 
Consultation by palliative care teams, however, is typically available Monday through Friday during business 
hours, and palliative care teams are not routinely available to come to the ED when a patient is in crisis.  
 
An ED visit is often described as a sentinel event signifying a breakdown in care coordination for older 
adults.8,9 Since EDs sit at the crossroads of ambulatory and inpatient care, they can and often play a pivotal 
role in balancing the potential harms and benefits of hospitalization for seriously ill, vulnerable older adults.10-13
Hospitalization for older adults carries significant risks such as iatrogenic complications, functional and 
cognitive decline, and loss of independence14-19 but emergency providers may be unaware of safe alternatives. 
Emergency medicine developed as a specialty to treat the acutely ill and injured, yet EDs increasingly care for 
older adults with multiple comorbid conditions who present for acute exacerbations of chronic illness. Visits to 
the ED by older adults are increasing both in frequency and as a proportion of all ED visits. In 2011, adults 
aged 65 years and older comprised 15% of total ED visits, had the highest severity of illness, and represented 
44% of all admissions from the ED.20 The number and rate of admissions to the Intensive Care Unit (ICU) by 
emergency providers have also increased, especially among older adults.21 The proportion of the US 
population 65 years and older will continue to grow, and EDs will see an increase in both the number of older 
adults and the complexity of care they are required to provide.22 The ED presents a key decision point at which 
providers set the subsequent care trajectory, including whether an older adult is hospitalized and to which 
setting. Emergency physicians can thus play an integral role in transforming care for older adults through 
evidence-based models of care delivery that emphasize tradeoffs between potential benefits and potential 
harms.23 However, until recently, little attention has been paid to the delivery of goal-concordant care in the ED 
for older adults with serious illness.  The default treatment plan is to deliver treatment intensive care that favor 
life-sustaining therapies, many of which may be contrary to what older adults desire. 
 
3. Settings of the Human Research 
 

3 
The PRIM-ER quality improvement (QI) initiative will be implemented in the Emergency Departments at the 
following sites: 
 
Site
Location 
NYU School of Medicine 
    Perelman Center for Emergency Care 
    Bellevue Hospital Center  
    NYU Langone Hospital – Brooklyn 
    NYU Long Island 
New York, NY 
New York, NY 
Brooklyn, NY 
Mineola, NY 
Allegheny Health Network
    Allegheny General Hospital 
Pittsburgh, PA 
Baystate Health  
    Baystate Medical Center 
Baystate Franklin 
Springfield, MA 
Greenfield, MA
Beaumont Health System
    Beaumont Royal Oak 
    Beaumont Troy
Royal Oak, MI 
Troy, MI 
Brigham and Women’s/Dana Farber Cancer Institute
Brigham and Women’s Hospital
    Brigham and Women’s Faulkner 
Boston, MA
Boston, MA 
Christiana Care Health System 
    Christiana Hospital 
    
Newark, DE 
Henry Ford Health System 
    Henry Ford Hospital 
Henry Ford Fairlane 
    Henry Ford West Bloomfield  
Detroit, MI 
Fairlane, MI 
West Bloomfield, MI
Icahn School of Medicine at Mount Sinai
    Mount Sinai Hospital 
Mount Sinai Beth Israel 
    Mount Sinai West
New York, NY 
New York, NY 
New York, NY 
Mayo Clinic Health System
    Mayo Clinic, St. Mary’s 
    Mayo Clinic Austin-Albert Lea  
    Mayo Clinic Health Mankato
Rochester, MN 
Austin/Albert Lea, MN 
Mankato, MN 
Ochsner Health System
    Ochsner Medical Center 
New Orleans, LA 
The Ohio State University 
    Wexner Medical Center 
Columbus, OH 
Rutgers New Jersey Medical School 
University Hospital Newark
Newark, NJ
University of California, San Francisco
    UCSF Medical Center 
    Zuckerberg San Francisco General 
San Francisco, CA 
San Francisco, CA 
University of Florida Health 
    UF Health Shands Hospital 
    UF Springhill 
    UF Kanapaha 
Gainesville, FL 
Gainesville, FL 
Gainesville, FL
University of Pennsylvania Health System
    Hospital of the University of Pennsylvania 
    Pennsylvania Hospital
    Penn Presbyterian Medical Center
 
Philadelphia, PA 
Philadelphia, PA 
Philadelphia, PA
University of Texas
   MD Anderson 
Houston, TX
University of Utah Health 
    University of Utah Hospital 
Salt Lake City, UT 
Yale New Haven Health System
    Yale New Haven Hospital 
New Haven, CT

4 
The research component of this initiative, consisting of analyzing the Medicare Claims Database, will solely 
occur at the Ronald O. Perelman Department of Emergency Medicine at NYU Langone Health. The 
qualitative interviews under the PRIM-ER Contextual Assessment will also be conducted at the Ronald O. 
Perelman Department of Emergency Medicine at NYU Langone Health.  
 
4. Subject Identification, Recruitment, and Consent 
 
A) Methods and Procedures 
 
This is a cluster-randomized QI initiative that will implement and test the impact of PRIM-ER in various 
healthcare settings. We will first develop and pilot test the QI initiative at two sites in the UG3 phase, and then 
use a cluster-randomized, stepped wedge design to implement the education, training, and technical support in 
our network of EDs in the UH3 phase. Randomization will occur at the ED level and be done in advance by the 
biostatistician to determine the order in which the training will occur. Due to the COVID-19 pandemic, training 
will occur either in-person or virtually. The virtual trainings will occur using NYULH’s Zoom account, meeting ID 
and set password. A member of the PRIM-ER team will be on each virtual training in order to monitor learner’s 
attendance and assist with any technical glitches. The PRIM-ER team will provide the training link, meeting ID, 
and password to each Site Principal Investigator (PI) and the Site PI will distribute the link and meeting ID 
accordingly to his/her staff at their own home institution. Reminder e-mails about the training will be sent from 
each Site PI to his/her staff. The overall approach involves ongoing asynchronous learning and technical 
support to bolster skills, conduct interdisciplinary case reviews, and reinforce clinical pathways and protocols 
via provider audit and feedback. Electronic triggers for palliative care will be embedded in the electronic health 
record (EHR) to identify patients who may benefit from hospice or palliative care services. These electronic 
triggers already existing in the Perelman Center for Emergency Services EHR as part of standard, clinical 
workflow, but will be further tailored for each participating health system. Palliative care champions at each site 
will facilitate attendance at didactic and workshop sessions, disseminate information about local resources for 
outpatient palliative care, home care and hospice, and work with the local informatics team to reinforce 
protocols and implement trigger criteria to identify older adults who may benefit from further needs assessment 
and follow-up. We will engage with the palliative care champions to better understand the QI implementation 
process including but not limited to obtaining champion’s feedback on the initiative, suggestions for 
improvements, and adaptations- all which are common approaches in implementing and reflecting on QI 
initiatives. Engagement will be in the form of discussions post QI about what went well, and what can be 
improved upon in moving forward as well as understanding from the champion’s perspective what they could 
have done differently so the project team can gather lessons learned. Physicians and nurses will receive audit 
and feedback reports to monitor their performance over time, and a learning monitoring system will track 
participation in educational activities. 
 
Prior to initiating this QI project, members from the palliative care team, emergency nursing, social work/case 
management, informatics, and ED operations from each of the 18 health systems will participate in workgroups 
to discuss how to best incorporate primary palliative care into the clinical workflow at each site. Pilot testing of 
PRIM-ER will also occur at two sites to optimize feasibility, fidelity, and usability. Emergency physicians at 
each pilot site will be invited to participate in usability testing of the clinical decision support (CDS) system. To 
assess usability, MORAE software will be utilized to perform screen captures and audio record participants 
verbalizing their actions, thoughts, and feelings as they progress through a simulated CDS system. 
 
In the UH3 phase of the study, we will engage eligible providers at the 33 additional sites based on the random 
sequential order in which the ED implementation occurs. Throughout the duration of the project, we will actively 

5 
engage each health system by providing all ED staff with audit and feedback reports to monitor their 
performance over time. These reports will be provided on a weekly basis during the study period, as well as 
incorporated into ED-specific continuous quality improvement processes. By providing this continuous and 
consistent feedback to ED personnel, we hope to encourage continued participation and active engagement 
with the initiative throughout its duration. To understand the implementation barriers and facilitators for the 
CDS component, we will be asking each Site PI to complete a short REDCap quantitative survey describing 
the changes they made to their CDS component of the intervention and the rationale (instrument attached). 
Any PHI related to the person completing the quantitative survey will be completely optional (Title, Role, Age, 
Years in practice). We will require the Site name in order to understand the changes made at each site. We will 
be sending each survey to the Site PI, but if they are not equipped to complete the survey in entirely they can 
forward it to another team member (e.g. Information Technology analyst) within their institution. We only are 
interested in understanding the changes made at the site, nothing related to the person completing the survey 
on the site’s behalf. We will also be conducting 30-60 minute qualitative interviews with Site PIs who are 
interested in an effort to gain more in-depth insights. Both the REDCap quantitative and qualitative interviews 
are completely optional. Verbal consent will be requested for qualitative interviews.  
 
For the research component of the study, we will use Medicare claims of the beneficiaries in our patient cohort 
to measure outcomes, including ED disposition to an acute care setting, healthcare in the 6 months following 
the ED visit and survival following the index ED visit as a result of the intervention. The patient cohort will be 
extracted via the Centers for Medicare and Medicaid Services (CMS) Research Data Assistance Center 
(ResDAC) using a two-step process to maximize diversity, and minimize intentional or unintentional exclusions 
based on risk, age, health literacy, demographics, or expected adherence. First, we will provide a 
comprehensive list of facility codes for the 35 participating EDs. Inpatient and ambulatory claims will be used to 
identify community-dwelling ED visitors 66 years and over who made a visit to any of the EDs from 2 years 
prior to study initiation until the last day implementation day of PRIM-ER. ED claims will be identified via 
Revenue Center Code values of 0450-0459 (Emergency room) or 0981 (Professional fees-Emergency room) 
according to ResDAC. We will then examine all inpatient, ambulatory, and carrier claims for the 12 months 
prior to each older adult’s index ED visit to calculate each beneficiary’s Gagne Index, a score developed to 
predict one-year mortality in community-dwelling older adults.24 The Gagne Index has been adapted from the 
Romano-Charlson Index and the Elixhauser system.25,26 It calculates a score based on the presence or 
absence of ICD-9s from inpatient and ambulatory claims in the prior year. Beneficiaries with a one-year 
mortality of at least 30% (score > 6) based on claims from the previous 12 months will be included in the 
analysis. 
 
We will estimate the baseline rate of acute care admission, healthcare utilization, and survival following the 
index ED visit using Medicare claims data for visitors to each ED. We will use the Master Beneficiary Summary 
File, Inpatient, Outpatient, Home Health, and Hospice files to monitor acute care admission, healthcare 
utilization, and survival monthly for up to 6 months after the index ED visit to evaluate whether there is a 
change before and after implementation. Measurement of what will be considered the baseline rate will 
continue until the month prior to implementation at each site, and post-implementation rates will be considered 
one month after implementation and continue on a monthly basis until 6 months after the last site has 
undergone implementation. To reduce prevalence-incidence bias27, we will include a roll-in period of 12 months 
before we begin to include baseline rates of our outcomes in the analysis. The index ED visit will be defined as 
the first ED visit to one of our 35 facilities during which the beneficiary has 12 months of prior inpatient, 
outpatient, or carrier claims consistent with a Gagne Index > 6, or >30% mortality. If a beneficiary’s index ED 
visit occurs during the roll-in period, they will be excluded from the baseline rate calculations if they return to 
one of our participating EDs and would otherwise meet our inclusion criteria. 

6 
To account for primary palliative care knowledge and skills on patient outcomes in analysis, we will use survey 
data that assessed knowledge and attitudes of palliative and end-of-life care collected before PRIM-ER 
implementation from the emergency physicians, physician assistants, nurse practitioners, and nurses at all 35 
participating EDs. 
 
As part of the PRIM-ER Contextual Assessment, we will examine state, regional, and healthcare system 
characteristics from the 18 healthcare systems included in the parent PRIM-ER study. The PRIM-ER 
Contextual Assessment will include 3 components conducting: 1) A grey literature review, 2) Semi-Structured 
qualitative interviews and 3) An exploratory analysis to examine if access to home and community health 
services predicts healthcare utilization in the 6 months after an ED visit at the end of life.  
 
Grey Literature Review 
We will conduct a grey literature review to identify contextual factors influencing access to home and 
community health services for the selected states, regions, and healthcare systems included in the PRIM-ER 
study. We choose to conduct a grey literature review versus a systematic review due to the nature of data we 
desire to obtain. Information on state and federal licensing, for example, are unlikely to be included in a formal 
research publication. We will also conduct an environmental scan to complete a census report on the type, 
amount, and location of home and community health services relative to the location of the healthcare systems 
of interest. The home and community health services of interest for this study include assisted living, skilled 
nursing care, adult day care, home health services, naturopathic medicine, chiropractic medicine, acupuncture, 
and massage therapy. Our search strategy will reflect the recommendations from Paez 2017 to balance search 
specificity and sensitivity without being overly inclusive to a fault.31 We will use a combination of resources for 
our search, including grey literature databases (OpenGrey, WONDER, SCOPUS, Grey Matters, Grey 
Literature Report, National Technical Information Service, PsycEXTRA, Web of Science, Zetoc), unpublished 
clinical trials (ClinicalTrials.gov, Cochrane Central Registry of Controlled Trials, World Health Organization 
International Clinical Trials Platform) conference papers (Conference Papers Index), dissertations, theses, and 
academic papers (ProQuest Dissertation & Theses Global, WorldCatDissertations) web searches (Google, 
Google Scholar, Mednar), and healthcare system information collected through PRIM-ER. Data of interest 
includes applicable laws/regulations, number and type of facilities, distance/location of facilities in relation to 
healthcare systems, and availability of services both within and external to the healthcare system. 
 
Semi- Structured Qualitative Interviews 
We will recruit an emergency medicine physician and nurse champion (one each) from all 18 healthcare 
systems within the PRIM-ER parent study. This will provide us a diverse population of emergency providers 
from different geographic locations and healthcare systems. Prior to starting the interviews we will explain the 
purpose of the interview to the participant, and obtain verbal consent. Interviews will be recorded and 
transcribed. Confidentiality will be protected by immediately downloading the recording to a secure computer 
and deleting the recordings once they are transcribed. No identifiable information will be recorded. The 
interviews pose minimal risk to the participants. 
 
Interviews will be completed and recorded using ZOOM teleconferencing and REDCap survey software. The 
main subsections of the interview guide will include 1.) Provider demographics and healthcare system 
characteristics 2.) Provider familiarity and attitudes regarding home and community health services for older 
adults with serious life-limiting illness, 3.) Barriers and facilitators to home and community health services 
within and out of their healthcare network, and 4.) Availability of integrative medicine services to support older 
adults with serious life-limiting illness 

7 
For interviews conducted aiming to understand CDS implementation in greater detail, we will be inviting 1 Site 
PI from each of the 18 healthcare systems within the PRIM-ER parent study. We will follow the same protocol, 
as outlined above for previous qualitative interviews. Prior to starting the interviews we will explain the purpose 
of the interview to the participant, and obtain verbal consent. Interviews will be recorded and transcribed. 
Confidentiality will be protected by immediately downloading the recording to a secure computer and deleting 
the recordings once they are transcribed. No identifiable information will be recorded. The interviews pose 
minimal risk to the participants.  
Interviews will be completed and recorded using ZOOM teleconferencing and REDCap survey software. The 
main subsections of the interview guide will include: Reach, Effectiveness and Maintenance. This will help us 
understand the specific barriers or facilitators to the effectiveness implementation and practical use of the 
PRIM-ER CDS instrument.  
Exploratory Analysis
We will conduct an exploratory analysis to examine if access to home and community health services predicts 
healthcare utilization in the 6 months after an ED visit at the end of life. We will examine correlations between 
home and community health access factors and healthcare utilization from the 18 healthcare systems included 
in the PRIM-ER study using the Senior Care Services Scale (SCSS) developed by Arbaje et al.32 Data from all 
18 healthcare systems collected from the Grey Literature Review and Qualitative Interviews will be eligible for 
this exploratory analysis. 
 
Contextual factors from the Grey Literature Review and the Qualitative Interviews will be used to analyze if 
access to home and community health services predicts healthcare utilization. Healthcare utilization data will 
be provided from the PRIM-ER study and will be collected from Medicare claims data to include ED revisits 
(count), inpatient days (count), home health use (Yes/No), and hospice use (Yes/No) in the 6 months from the 
index PRIM-ER ED visit. These measures of healthcare utilization are based on the Dartmouth Atlas Decedent 
Cohort Care Intensity Measures to monitor the quality of end-of-life care in Medicare patients.  The SCSS will 
be adapted to PRIM-ER using contextual factors from the Grey Literature Review and Qualitative Interviews 2.  
Each healthcare system will be assigned a score based on their access to home and community health 
services. 
 
 
 
B) Inclusion and Exclusion Criteria 
 
Eligible patients will include ED patients 66 years or older with serious, life-limiting illness who visited any of 
our EDs during the implementation of PRIM-ER. Patients must demonstrate one-year mortality of at least 30 
percent (score > 6) according to the Gagne Index, a validated instrument used to measure all cause one-year 
mortality in community-dwelling older adults, calculated based on their prior 12 months before the index ED 
visit of Medicare claims. ED patients transferred from a nursing home on the index ED visit will be excluded 
since prediction of mortality and disposition of such patients differs from community-dwelling adults. Patients 
currently receiving hospice at the time of the index ED visit will also be excluded since they have already 
received services.  
 
Qualitative Interviews Contextual Assessment 

8 
Participants for the provider interviews will include one emergency medicine physician and one nurse 
champion from each of the 18 healthcare systems already enrolled within the PRIM-ER. As part of PRIM-ER 
each site has already identified a provider and nurse champion that the study team will outreach. Inclusion 
criteria includes: Must be a practicing licensed physician or licensed registered nurse within a department of 
emergency medicine within a healthcare system included in the PRIM-ER study, must be 18 years or older, 
must be willing to provide verbal consent.  
Qualitative Interviews CDS 
Participants for the CDS interviews will include one Site PI from each of the 18 healthcare systems 
already enrolled within the PRIM-ER study. This is completely voluntary and Site PIs do not have to 
participate if they do not want to and/or do not have the time. As part of PRIM-ER project, each site has 
already identified a Site PI that the study team will outreach. Inclusion criteria includes: Must be a practicing 
licensed physician within a department of emergency medicine within a healthcare system included in the 
PRIM-ER study, must be 18 years or older, must be willing to provide verbal consent.  
C) Number of Subjects 
 
We expect to analyze the Medicare claims of over 57,000 patients with serious illness who have made their 
index ED visit to any of the 35 EDs. 
 
Qualitative Interviews Contextual Assessment  
We will recruit 18 licensed emergency medicine physicians and 18 emergency medicine registered nurses for 
a total of 36 participants for the provider interviews. 
 
Qualitative Interviews CDS 
We will recruit a maximum of 18 licensed emergency medicine physicians/Site PIs for the CDS interviews.   
D) Recruitment and Informed Consent 
 
Medicare claims of patients 66 years and older with serious, life-limiting illness who made a visit to any of our 
EDs during the study period will be used to measure outcomes in our patient cohort. We will seek a waiver of 
Health Insurance Portability and Accountability Act (HIPAA) authorization for ED patients as this study 
presents no more than minimal risk and cannot be practicably conducted without the waiver given the study’s 
geographic breadth and sheer number of participants (>57,000 eligible patients). Obtaining informed consent 
for participation and use of Medicare claims from all patients in this study is not feasible and will interfere with 
the conduct of this study. 
 
Qualitative Interviews 
We will recruit our participants for the provider interviews through a combination of word-of-mouth and email 
invitation through the healthcare systems points of contact established via the PRIM-ER study. After a 
participant expresses interest in participation, they will be contacted by Dr. Hill for Contextual Assessment or 
Senior Research Project Manager (A. Cuthel) or Co-Investigator Dr. Lawrence for CDS interview for an 
expanded explanation of the study, study procedures, their role as a participant, and potential risks and 
benefits. It will be clearly disclosed that the interview will occur via ZOOM teleconferencing and be recorded. 
Once the participant is read all study materials and information and had a chance to have all their questions 
answered, the recording device will be started, and the provider will be asked to provide verbal consent for 
participation. All qualitative interviews for this project, will follow the same protocol as outlined here.  

9 
E) Data Analysis 
 
a. Dependent variables 
 
ED disposition will be measured on the index ED visit, and will be a dichotomous variable for an acute care 
admission (Yes/No).  Acute care admission will be defined as admission to a non-palliative service, and non-
acute care admission will include admission to a palliative care service or unit, discharge to home, observation 
(without a change to inpatient status), or transfer to inpatient or outpatient hospice.  
 
Healthcare utilization will be measured as ED revisits (count), inpatient days (count), home health use 
(Yes/No), and hospice use (Yes/No) in the 6 months from the index ED visit. These will be identified through 
revenue codes in each site’s administrative data. We developed these measures of healthcare utilization 
based on the Dartmouth Atlas Decedent Cohort Care Intensity Measures to monitor the quality of end-of-life 
care in Medicare patients with serious chronic illness. 28-30
 
Survival will be measured in days from the index ED visit to death or 6 months, whichever is sooner. 
 
b. Independent variables 
 
Table 1 outlines the independent variables. Independent variables were previously assessed at the time of site 
implementation. Healthcare system- and provider-level variables were collected by the project manager and 
via a provider survey at the level of each participating ED. Patient-level variables will be assessed using the 
CMS Research Data Assistance Center Master Beneficiary Summary File, Base (A/B/D) Segment. The Project 
Manager will also document if trainings occurred via in-person or virtually to understand more deeply the 
implementation of the QI initiative at each site. 
 
Table 1. Independent Variables
Variable
Coding
Source
Implementation Period
Weeks from Time 0 
Project Manager
Healthcare system/ED-level variables
Health System 
Allegheny, Bay State, Beaumont, Brigham and Women’s, 
Christiana Care, Henry Ford, Mayo Clinic, MD Anderson, NYU 
Langone, Ohio State University, Ochsner, Rutgers, Sinai, UC 
San Francisco, University of Florida, University of Pennsylvania, 
University of Utah, Yale New Haven 
Project Manager 
ED 
1—35 
Project Manager
ED Volume 
30,000—49,999 visits, 50,000—69,999 visits, 70,000—89,999 
visits, > 90,000 visits  
Project Manager 
Ownership 
Nonprofit, Government, For Profit
Project Manager
Emergency medicine residency 
training site 
Yes/No
Project Manager
Free-standing ED
Yes/No
Project Manager
Dedicated ED social 
worker/care manager
Yes/No
Project Manager
US Region
Northeast, Midwest, Southeast, Southwest, West 
Project Manager
Metropolitan Status+
Yes/No
Project Manager
Outpatient palliative care
Yes/No
Project Manager
EHR
Epic, Cerner 
Project Manager
Trauma center
Yes/No
Project Manager
Patient variables

10 
For the PRIM-ER Contextual Assessment that data analysis will vary per specific component.  
 
Grey Literature Review: Data will be collected and analyzed on the regional, state, and healthcare system 
level. Data will be coded and collated per topic of interest. A combination of descriptive statistics, narrative 
description, and thematic coding will be used to summarize and present the data. 
 
Qualitative Interviews: For the qualitative data, the interview transcriptions will undergo verbatim transcription 
and thematic coding. We will develop deductive codes based on the discussion guide, and inductive codes 
based on salient topics identified during review and coding of the transcripts. We will compare codes across all 
of the transcripts during the final analysis phase and condense codes into meaningful categories. We will use 
ATLAS.ti software to facilitate coding as well as cross-investigator and cross-interview analyses. We will also 
develop data matrices to highlight prominent themes related to facilitators and barriers to home and community 
health services.  For quantitative analysis, we will use descriptive statistics to summarize the data. 
 
Exploratory Analysis: This exploratory analysis will mirror the healthcare utilization analysis and statistical 
model described in the PRIM-ER. Each healthcare system index score will be used as an independent variable 
in our predictive model to determine the relationship between home and community health access factors and 
healthcare utilization in the 6 months after an ED visit at the end of life. 
 
The healthcare system dependent (Ea.) and independent variables (Eb.) described above for PRIM-ER grant 
will also be used for the analysis of the PRIM-ER Contextual Assessment. 
c. Methods 
 
The analytic plan accounts for the nested structure of the data, assesses normality assumptions of dependent 
variables, and addresses issues related to missing data, study participation bias, and baseline covariate 
balance. We address each of these in turn. All analyses will be conducted in R 3.3.2 (R Foundation for 
Statistical Computing, Vienna). 
 
Prior to conducting the outcome analyses, we will compare patients in each ED cluster with respect to patient, 
provider, and facility characteristics. We will assess whether any adjustments will need to be made in the final 
statistical models based on whether the differences are clinically meaningful. To account for nesting in the data 
structure (patients nested in hospitals), we will use mixed effect multi-level models to estimate effect sizes. We 
anticipate two sources of variation. 
 
The primary outcome is the proportion of eligible patients whose disposition is to an acute care setting 
(inpatient, non-palliative service). The secondary outcomes include healthcare service utilization in the 6 
months following the ED visit and survival times following the ED visit. The health utilization outcomes include 
receipt of ED revisits (count), home health services (yes/no), inpatient days (count), admission to an ICU 
Age 
Years 
Master Beneficiary Summary 
File, Base Segment
Gender
Female, Male, Other
Master Beneficiary Summary 
File, Base Segment
Race/Ethnicity
Asian, Black, Hispanic, White, North American Native, Unknown, 
Other 
Master Beneficiary Summary 
File, Base Segment
Gagne index24 
Count of conditions 
Inpatient and outpatient RIF
+Population estimates by MSA are based on estimates of the civilian non-institutionalized population of the US as of July 1, 2013, 
from the 2013 National Health Interview Survey, National Center for Health Statistics, compiled according to the 2013 Office of 
Management and Budget definitions of core-based statistical areas. See http://www.census.gov/population/metro/ for more about 
metropolitan statistical area definitions.

11 
(yes/no), and admission to hospice (yes/no). The analysis of the effect of PRIM-ER on ED disposition in the 
context of a stepped-wedge design will be based on a Generalized Linear Mixed Model (GLMM). In particular, 
to assess the intervention effect, we will use a generalized linear binomial model with random site level effects.  
The analysis of site, provider, and patient-level characteristics that are associated with variation in impact of 
PRIM-ER will be based on extending the models used in the analysis plan for ED disposition, healthcare 
utilization in 6 months following the ED visit, and survival times to include independent variables related to the 
characteristics of interest. 
 
5. Risks to Subjects 
 
Any information collected from Emergency Department providers will be utilized solely for QI purposes and not 
analyzed for research.  
 
The study involves using Medicare claims of patients in our patient cohort that contain identifiable personal 
health information. The largest risk to ED patients is a breach of confidentiality. This will be managed by 
ensuring that only qualified study team members have access to patient data; all personal identifiers will be 
removed after final analysis, and all reporting and/or publication of data based on Medicare claims will be in 
aggregate form. Study team members will also be approved by ResDAC to access the Medicare claims 
through the Virtual Research Data Center (VRDC), a virtual research environment allowing researchers to 
have direct access to approved data files to conduct their analysis within the CMS secure infrastructure. All 
research personnel who have access to electronic records will undergo extensive training to safeguard against 
this potential risk to emergency provider, key informant, and patient participants, which will include HIPAA 
certification and CITI training in biomedical research and social and behavioral research. 
 
For the PRIM-ER Contextual Assessment and CDS quantitative survey or qualitative interviews provider 
information including demographics and interview responses will be recorded and analyzed. The largest risk to 
ED providers is a breach of confidentiality. There is no expected risk of physical, mental, professional, or 
financial harm to study participants. 
 
6. Potential Benefits to Subjects 
 
Future patients with serious illness who present to the ED may benefit from the findings of this study. 
 
7. Protections Against Risk 
 
Medicare claims obtained from CMS will be stored in the VRDC. The VRDC is a virtual research environment 
that allows researchers to have direct access to approved data files and be able to conduct their analysis 
within the CMS secure infrastructure. The VRDC contains its own VPN and virtual desktop. All reporting and/or 
publication of data will be in aggregate form. Additional protection of participant confidentiality mandated by 
HIPAA will be strictly adhered to. 
 
All reporting and/or publication of data will be in aggregate form. Additional protection of participant confidentiality 
mandated by HIPAA will be strictly adhered to.  
 
For the PRIM-ER Contextual Assessment and CDS quantitative survey and qualitative interviews the risks listed above 
will be managed and minimized by ensuring that only qualified study team members have access to patient 
data, unique identifiers are assigned to participant data, and all personal identifiers will be removed after final 

12 
analysis. The ZOOM teleconferencing interface will be an encrypted connection between the interviewer and 
interviewee. Participant information and data will be stored on REDCap software, which is an encrypted and 
secure data collection and storage system. This study will also adhere to, and exceed, federal, state, and 
institutional regulations regarding ethical conduct of research to protect subjects who choose to participate in 
this study. Interviews will be recorded and transcribed and confidentiality will be protected by immediately 
downloading the recording to a secure computer and deleting the recordings once they are transcribed. No 
identifiable information will be recorded. The interviews pose minimal risk to the participants.   
 
Regarding the potential virtual trainings, NYULH’s Zoom account will be used and the link, meeting ID and password 
will be distributed by each Site Principal Investigator to his/her staff members. 
 
8. Data Collection, Safety, and Monitoring 
 
A) Data Collection 
 
We will estimate the baseline rate of acute care admission, healthcare utilization, and survival following the 
index ED visit using Medicare claims data for visitors to each ED. To evaluate the effect of PRIM-ER, we will 
use the Master Beneficiary Summary File, Inpatient, Outpatient, Home Health, and Hospice files to monitor 
acute care admission, healthcare utilization, and survival monthly for up to 6 months after the index ED visit to 
evaluate whether there is a change before and after implementation. 
 
B) Provisions to Monitor Data and Ensure the Safety of Subjects 
 
All study data will be stored and accessed via secure systems. Data will not be accessed or analyzed by 
individual sites; these activities will be performed exclusively by authorized individuals at the lead study site 
(NYU School of Medicine). Only authorized personnel who have been appropriately trained will be granted 
permission by the PI to access study data. A Data Safety Monitoring Plan (DSMP) will be submitted for 
reporting procedures of adverse events and serious adverse events. 
 
C) Steering Committee 
 
The PRIM-ER Steering Committee (SC) is the primary governing body of PRIM-ER. In consultation with the 
NIH Program Officer, NIH Scientific Officer, and NIH Collaboratory leadership, it formulates and monitors 
policies and procedures guiding the research activities. All major scientific and operational decisions are made 
by majority vote with the concurrence of the NIH Program Officer, NIH Scientific Officer, and NIH Collaboratory 
leadership. The Steering Committee may appoint Subcommittees and Working Groups as needed to carry out 
specific tasks identified by the Steering Committee. The Steering Committee will function in accordance with 
the Terms and Conditions of the NIH Collaboratory Demonstration Project RFA and other applicable policies of 
NIH, NIA, and NCCIH. All participating PRIM-ER sites must agree to abide by the policies approved by the 
Steering Committee. 
 
The voting membership of the committee is to consist of the Principal Investigator, a site Principal Investigator 
from each of the other 17 health care systems, the NIH Program Officer, the NIH Scientific Officer, and 
leadership from the NIH Collaboratory as requested. Other (non-voting) memberships also include the 
Program Manager and other Subcommittee and Work Group Members. 
 

13 
This committee will establish bylaws, policies, and standard operating procedures to govern all aspects of 
PRIM-ER. This committee will review and approve the collaborative research agenda as well as formulate and 
monitor policies and procedures guiding the research activities, review and approve procedures for data 
acquisition, analysis and management, oversee communication within the PRIM-ER as well as with the greater 
scientific community and the public. 
 
The Steering Committee will be responsible for ensuring that there are well documented policies and operating 
procedures guiding all aspects of PRIM-ER activities (e.g., protocol development, review, initiation, conduct, 
and closure, data collection, publication, etc.) and bylaws delineating the requirements and expectations of 
collaborating institutions, membership criteria, review of research progress and performance, establish 
standards of performance, and procedures for removing institutions due to poor performance. 
 
The Steering Committee will establish subcommittees and workgroups to assist it in carrying out its functions. 
The Steering Committee may meet up to four times a year. 
 
D) Data and Safety Monitoring Plan 
 
The PI, in cooperation with her co-investigators, the DSMB, and the IRB at NYU School of Medicine, will monitor the 
safety of the implemented project. The project manager will inform the PI immediately of any adverse events (AEs) that 
meet the collection and reporting criteria of the Data Safety Monitoring Plan (DSMP). All serious adverse events (SAEs) 
and Unanticipated Problems (UPs) related to study participation will be reported to the IRB and the NIA according to the 
criteria outlined in the DSMP. Events that might be considered AEs related to this proposal include emotional distress 
resulting from discussions surrounding palliative care, and any breaches in subject confidentiality. Related AEs and 
related SAEs will also be reported annually in the IRB for continuation or termination of the research. Given the minimal 
risk entailed by this project for all participating populations, we do not anticipate the occurrence of many AEs or SAEs. 
The PI and co-investigators will be versed in these reporting procedures, as they are currently required for all research 
conducted at NYU School of Medicine. All investigators and staff involved in this project have completed an extensive 
course and passed a certifying exam on the protection of human subjects in research. Independent Monitors comprised of 
a researcher in palliative care, biostatistician, and palliative care physician and content expert will monitory the data safety 
of this study. The study team will generate Study Reports for the Independent Monitors and will provide information on 
the following study parameters: 
 
Demographic information pertaining to patient subjects obtained in Medicare Claims. 
 
Stopping and reporting rules for UPs and related AEs/SAEs. A summary report will be generated consisting of the 
number of related AEs and SAEs by site and in total and delineated by severity. 
 
Any protocol deviations that have occurred since the previous report. 
 
Quality management activities since the last review, including frequency. A summary of findings and corrective 
actions taken to address the findings will be included. 
 
Interim analyses as requested by the IMC to assess safety concerns or study futility. 
 
Study Report tables will be generated only from aggregate (not by group assignment) baseline and aggregate safety data 
for the study population. 
 
For the PRIM-ER Contextual Assessment and CDS quantitative survey, data collection, safety, and monitoring plan 
will adhere to, the data safety and monitoring plan outlined above for PRIM-ER grant, as applicable. For all the 
Semi-structures qualitative interviews confidentiality will be protected by immediately downloading the 
recording to a secure computer and deleting the recordings once they are transcribed. No identifiable 
information will be recorded. The interviews pose minimal risk to the participants. 

14 
9. Economic Impact to Subjects 
 
There is no expected economic impact to subjects participating in this study. 
 
10. Payments to Subjects 
 
Patient participants will not receive compensation in this study. 
 
11. Vulnerable Populations 
 
Given the magnitude of the Medicare Claims Database, it is possible that adults unable to consent will be 
included. Since we are requesting a waiver of authorization, this should not pose any additional risk to 
these subjects. 
Include
Exclude
Vulnerable Population Type
X
Adults unable to consent
 
X 
Individuals who are not yet adults (e.g., infants, children, teenagers
 
X
Wards of the State (e.g., foster children)
 
X
Pregnant women
 
X 
Prisoners 
REFERENCES 
 
1.
Barnato AE, Chang CC, Farrell MH, Lave JR, Roberts MS, Angus DC. Is survival better at hospitals 
with higher "end-of-life" treatment intensity? Med Care. 2010;48(2):125-132.
2. 
Wu FM, Newman JM, Lasher A, Brody AA. Effects of initiating palliative care consultation in the 
emergency department on inpatient length of stay. Journal of palliative medicine. 2013;16(11):1362-
1367. 
3. 
Grudzen CR, Richardson LD, Johnson PN, et al. Emergency Department-Initiated Palliative Care in 
Advanced Cancer: A Randomized Clinical Trial. JAMA Oncol. 2016. 
4. 
Smith AK, McCarthy E, Weber E, et al. Half of older Americans seen in emergency department in last 
month of life; most admitted to hospital, and many die there. Health Aff (Millwood). 2012;31(6):1277-
1285. 
5. 
Grudzen CR, Richardson LD, Major-Monfried H, Kandarian B, Ortiz JM, Morrison RS. Hospital 
administrators' views on barriers and opportunities to delivering palliative care in the emergency 
department. Annals of emergency medicine. 2013;61(6):654-660. 
6. 
Stone SC, Mohanty S, Grudzen CR, et al. Emergency medicine physicians' perspectives of providing 
palliative care in an emergency department. Journal of palliative medicine. 2011;14(12):1333-1338. 
7. 
Hughes MT, Smith TJ. The growth of palliative care in the United States. Annual review of public 
health. 2014;35:459-475. 
8. 
Clancy CM, Eisenberg JM. Emergency medicine in population-based systems of care. Ann Emerg Med. 
1997;39:800-803. 

15 
9.
Coleman EA, Eilertsen TB, Kramer AM, Magid DJ, Beck A, Conner D. Reducing emergency visits in 
older adults with chronic illness. A randomized, controlled trial of group visits. Eff Clin Pract. 
2001;4(2):49-57.
10. 
Morganti KG, Bauhoff S, Blanchard JC, et al. The Evolving Role of Emergency Departments in the 
United States. Santa Monica, CA: RAND Corporation;2013. 
11. 
McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United 
States. N Engl J Med. 2003;348(26):2635-2645. 
12. 
Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' 
care. Lancet. 2003;362(9391):1225-1230. 
13. 
Hwang U, Shah MN, Han JH, Carpenter CR, Siu AL, Adams J. Transforming Emergency Care for 
Older Adults. Health Aff. 2013;32(12):2116-2121. 
14. 
Corsonello A, Pranno L, Garasto S, Fabietti P, Bustacchini S, Lattanzio F. Potentially inappropriate 
medication in elderly hospitalized patients. Drugs Aging. 2009;26 Suppl1:31-39. 
15. 
Covinsky KE, Palmer RM, Fortinsky RH, et al. Loss of independence in activities of daily living in 
older adults hospitalized with medical illnesses: Increased vulerability with age. J Am Geriatr Soc. 
2003;51:451-458. 
16. 
Sager MA, Franke T, Inouye SK, et al. Functional outcomes of acute medical illness and hospitalization 
in older persons. Arch Intern Med. 1996;156(6):645-652. 
17. 
Hirsch CH, Sommers L, Olsen A, mullen L, Winograd CH. The natural history of functional morbidity 
in hospitalized older patients. J Am Geriatr Soc. 1990;38(12):1296-1303. 
18. 
Wakefield BJ, Holman JE. Functional trajectories associated with hospitalization in older adults. West J 
Nurs Res. 2007;29(2):161-177. 
19. 
Lakhan P, Jones M, Wilson A, Courtney MC, Hirdes J, Gray LC. A prospective cohort study of geriatric 
synrdomes among older medical patients admitted to acute care hospitals. J Am Geriatr Soc. 
2011;59(11):2001-2008. 
20. 
National Center for Health Statistics (U.S.). National Hospital Ambulatory Medical Care Survey. 
Emergency department summary. In. DHHS publication. Hyattsville, Md.: U.S. Dept. of Health and 
Human Services, Public Health Service, Centers for Disease Control and Prevention:v. 
21. 
Mullins PM, Goyal M, Pines JM. National growth in intensive care unit admissions from emergency 
departments in the United States from 2002 to 2009. Acad Emerg Med. 2013;20(5):479-486. 
22. 
Vincent GK, Velkoff VA, U.S. Census Bureau. The next four decades : the older population in the 
United States : 2010 to 2050. Washington, D.C.: U.S. Dept. of Commerce, Economics and Statistics 
Administration, U.S. Census Bureau; 2010. 
23. 
Hwang U, Shah MN, Han JH, Carpenter CR, Siu AL, Adams JG. Transforming emergency care for 
older adults. Health affairs. 2013;32(12):2116-2121. 
24. 
Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted 
mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749-759.
25. 
Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM 
administrative data: differing perspectives. J Clin Epidemiol. 1993;46(10):1075-1079; discussion 1081-
1090. 
26. 
Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. 
Med Care. 1998;36(1):8-27. 
27. 
Neyman J. Statistics; servant of all sciences. Science. 1955;122(3166):401-406. 
28. 
Wennberg JE, Fisher ES, Baker L, Sharp SM, Bronner KK. Evaluating the efficiency of california 
providers in caring for patients with chronic illnesses. Health Aff (Millwood). 2005;Suppl Web 
Exclusives:W5-526-543. 
29. 
Wennberg JE, Fisher ES, Stukel TA, Sharp SM. Use of Medicare claims data to monitor provider-
specific performance among patients with severe chronic illness. Health Aff (Millwood). 2004;Suppl 
Variation:VAR5-18.

16 
30.
Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK. Use of hospitals, physician 
visits, and hospice care during last six months of life among cohorts loyal to highly respected hospitals 
in the United States. BMJ. 2004;328(7440):607.
31.  
Paez A. Gray literature: An important resource in systematic reviews. J Evid Based Med. 
2017;10(3):233-240. doi:10.1111/jebm.12266 
32.  
Arbaje AI, Yu Q, Newhall KA, Leff B. Prevalence, Geographic Variation, and Trends in Hospital 
Services Relevant to the Care of Older Adults: Development of the Senior Care Services Scale and 
Examination of Measurement Properties. Med Care. 2015;53(9):768-775. 
doi:10.1097/MLR.0000000000000408