Improving Medical Care With Electronic Interventions Based on Automated Text and Phone Messages
Phase: NA
Conditions: General Medicine, Emergency Medicine, Mobile Health
Sponsor: Washington University School of Medicine

 
 
 
 
 
Improving Emergency Department Discharge 
Referrals with Automated Text and Phone 
Messages 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Principal Investigator: Dr. Brent Ruoff 
Emergency Medicine Department 
Study Protocol, version 4.1 
Sponsor: None 
IRB# 
June 26, 2020 
 
 
 
 
 
 
 
 
 
 

 
2
 
 
Content 
A Introduction …..................……………..........……………………………………………………………………………………….. 3 
A1 List of Abbreviations 
A2 Protocol Synopsis 
B Background & Rationale …………………………………………….………………………………………………………………….. 3-4 
B1 Background (clinical problem) 
B2 Pilot Data 
B3 Study Overview (hypothesis) 
C Study Objectives …………….………………………………………………………………………………….………………………….. 4-5 
C1 Specific Aims 
C2 Outcomes of Interest 
D Research Team …................................................................................................................................. 5 
D1 Key Personnel 
D2 Key Personnel Training Certifications 
D3 Organization and Participating Centers 
D4 Funding Source and Conflicts of Interest and Their Management 
E Data Collection and Analysis …............................................................................................................. 5-6 
E1 Data Quality Assurance 
E2 Data Analysis, Sample Size Estimation, and Study Power 
F General Assurances …........................................................................................................................... 6 
G Risks and Benefits …............................................................................................................................. 6 
H Publication Plan …................................................................................................................................ 6 
I References ….......................................................................................................................................... 6-7 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
3
 
A. Introduction 
A1. List of Abbreviations 
ECRC – Emergency Care Research Core 
ED – Emergency Department 
EMR – Electronic Medical Record System 
HIPAA – Health Insurance Portability and Accountability Act 
SMS – Short Message Service (texting) 
 
A2. Protocol Synopsis 
This was a prospective randomized open blinded end-point (PROBE) trial that took place at a large urban 
Emergency Department (ED) to test if a two-way phone messaging system improved patient adherence 
to referral follow-up visits at outpatient clinics. The primary outcome was patient attendance for follow-
up care and the secondary outcome was ED re-admissions within 4 months after initial discharge.  
 
B. Background & Rationale  
B1. Background: Adherence to Follow-up Visits After Emergency Department Discharge 
Patient adherence to ED follow-up instructions and appointments varies from 26-56% [1]. According to 
Kangovi et al. (2012), the most common reason a patient gave for readmission to the ED was “feeling 
unprepared for discharge” after the first visit [2]. The transition from ED to discharge requires the 
patient to be responsible for following discharge instructions, obtaining prescriptions, and finding time 
and resources to attend follow-up visits. Readmissions to the ED are not primarily a result of lack of 
access to services [3], pointing attention toward follow-up communication, which can improve transition 
to outpatient care [4-6]. ED clinical staff try to sit down with patients after ED visits to review 
instructions and sometimes schedule follow-up appointments [7]. An outpatient hemodialysis clinic that 
tried this model found it to be successful initially, but could not afford the extra staff to sustain it.[8] 
 
Time and money could be saved by implementing automated appointment reminders through voice or 
text messages. No-show patients were reduced by 10% when one primary care clinic began using a text 
and phone call reminder for appointments, resulting in a positive cost-benefit analysis [9]. A similar 
improvement in successful follow-up appointments was found in an ED setting with a diverse population 
[10]. Historically, successful follow-up appointment rates of patients discharged from the Barnes Jewish 
Hospital Emergency Department (BJH-ED) have ranged from 9% to 37.5%, the highest rate of follow-ups 
were for next-day appointments [11]. It’s difficult to know how much of a difference a simple 
appointment reminder system would make since a previous study of BJH-ED patients found the major 
identifiable causes for missing appointments were “other” (34%), no transportation (25%), fear of 
medical cost (18%), feeling better (13%), and unsure of where to go for the visit (9%) [12].  A 
sophisticated and robust reminder system may be capable of addressing some of these barriers..  
Recently complex modalities of electronic follow-up, such as video conferencing or remote physiological 
monitoring [13-15] have been introduced, further complicating follow-up care adherence,  especially in 
an ED setting.  
 
A two-way automated SMS system that provides a way for patients to respond has shown promise in 
improving communication and follow-up care with complex patients, with engagement rates between 
80-90% [16]. A pilot study found a two-way system helped dialysis patients improve attendance and had 
potential to reduce emergency hospitalization [17].   
 

 
4
 
We hypothesize that a bi-directional automated text and phone system which facilitates  scheduling 
follow-up referrals for discharged ED patients will improve follow-up adherence.  
 
B2. Hypotheses  
We hypothesize that compared to standard of care, an automated bidirectional phone messaging 
system will: (1) improve ED patient adherence to follow-up appointments and (2) reduce ED re-
admissions. 
 
C. Study Objectives 
C1. Specific Aims 
The primary objective of this study is to determine whether follow-up appointment adherence is 
improved within 120 days after an initial discharge from the ED by using an automated bi-directional 
phone messaging system. 
• 
Specific Aim 1: Did more patients successfully attend follow-up appointments? 
• 
Specific Aim 2: Was there a reduction in ED re-admissions? 
 
C2. Outcomes of Interest 
• 
Primary: Time to attending a follow-up visit with referred primary or specialty care within 120 
days post-discharge 
• 
Secondary:  
o Time to re-visit to the ED (BJH or other) within 120 days post-discharge. 
• 
Tertiary: 
o Patient engagement levels with the intervention: number of successful attempts to 
reach participant, number of successful attempts to schedule a follow-up appointment, 
length of phone call to schedule an appointment, and number of successful follow-up 
appointments with a specialist provider versus a primary care provider. 
 
D. Research Team 
 
D1. Key Personnel 
PI: Brent Ruoff, MD 
 
D2. Key Personnel Training Certification 
All current research team members complete Washington University HIPAA and CITI training courses. 
They are also registered with the Washington University Human Research Protection Office at 
myIRB.wusm.wustl.edu and added to the study’s research team prior to viewing PHI. The research 
assistant and medical student are given “View Only” access to the EMR to help with patient chart 
reviews. 
 
D3. Organization and Participating Centers 
Washington University in St. Louis School of Medicine 
Barnes Jewish Hospital Emergency Department 
Clinical Research and Innovation Student Program at Washington University School of Medicine 
 

 
5
 
D4. Funding Sources and Conflict of Interest 
There is no funding being used for this research project and there are no financial conflicts of interest.  
 
E. Data Collection and Analysis 
 
E1. Data Quality Assurance 
 
Brent Ruoff, M.D. is the principal investigator who is the Chief of Division of Emergency Medicine and 
Associate Professor of Emergency Medicine at Washington University School of Medicine. He is an 
expert on helping BJC reduce re-admissions and has been doing clinical research since 1990.  
 
E2. Data Analysis, Sample Size Estimation, and Study Power 
The sample size from the trial was determined to meet criteria to be sufficiently powered at 90% for a 
two-sided α = 0.05, for a 10% improvement of the primary outcome. We will conduct an Intention-to-
Treat analysis to test the impact of the intervention on participant follow-up attendance and ED re-
admissions. We will also conduct descriptive statistics regarding user engagement. 
 
We might experience missing outcome data if a participant set up a follow-up appointment with a 
provider who was not their referral provider and is not a part of the BJH EMR network or Missouri 
Health Connection. These appointments will not be accessible to our study team and a participant may 
be labeled as non-compliant erroneously; however, since most Missouri health clinics participate in the 
Missouri Health Connection, this issue will be rare and should apply to both the control and intervention 
arm equally.  
 
F. General Assurances 
The study team agrees to comply with all applicable Washington University policies and procedures, and 
applicable federal, state and local laws. The research will only be performed by qualified personnel. All 
persons assisting with the research are adequately informed about the protocol and their research-
related duties and functions. We will not implement any changes in the approved IRB application 
without prior IRB approval. 
 
G. Risks and Benefits 
Potential risks include possible breach of confidentiality involving PHI/study data. 
 
Efforts will be dedicated to minimize the risk of breach of privacy. All patient information and study data 
will be password protected in computer files stored on Box and only shared with research team 
members. Participants are assigned study ID numbers that will be used to minimize risk of exposure of 
patient information. Data collected from the Epharmix/CareSignal Intervention Platform are deidentified 
before being stored in Box. 
 
H. Publication Plan  
We will submit a manuscript describing our results within approximately 12 months of finalizing them. 
Any results published from this study will not contain any participant identifiers. 
 

 
6
 
References 
1. Arora S, Burner E, Terp S, Nok Lam C, Nercisian A, Bhatt V, Menchine M. Improving Attendance 
at Post–Emergency Department Follow-up Via Automated Text Message Appointment 
Reminders: A Randomized Controlled Trial. Academic Emergency Medicine. Jan 2015; 22(1):31-
37.  
2. Kangovi S, Grande D, Meehan P, Nitra N, Shannon R, Long JA. Perceptions of readmitted patients 
on the transition from hospital to home. J Hosp Med. Nov-Dec 2012; 7(9): 709-12.  
3. McCarthy ML, Hirshon JM, Ruggles RL, Docimo AB, Welinsky M, Bessman ES. Referral of 
medically uninsured emergency department patients to primary care. Acad Emerg Med. Jun 
2002; 9(6): 639-42.  
4. Cain CH, Neuwirth E, Bellows J, Zuber C, Green J. Patient experiences of transitioning from 
hospital to home: an ethnographic quality improvement project. J Hosp Med. May-Jun 2012; 
7(5): 382-7.  
5. D’Amore J, Marray J, Powers H, Johnson C. Does telephone follow-up predict patient satisfaction 
and readmission? Popul Health Manag. Oct 2011; 14(5): 249-55. PMID: 21504311. 
6. Soong C, Kurabi B, Wells D, Caines L, Morgan MW, Ramsden R, Bell CM. Do Post Discharge 
Phone Calls Improve Care Transitions? A Cluster-Randomized Trial. PLoS One. 2014; 9(1): 
e112230.  
7. Murnik M, Randal F, Guevara M, Skipper B, Kaufman A. Web-based primary care referral 
program associated with reduced emergency department utilization. Fam Med. Mar 2006; 
38(3): 185-9.  
8. Lamping, Donna L., and Kent A. Campbell. "Hemodialysis Compliance: Assessment, Prediction, 
and Intervention—Part II." Seminars in Dialysis. Vol. 3. No. 2. Blackwell Publishing Ltd, 1990.   
9. Cummings, K. Michael, et al. "Intervention strategies to improve compliance with medical 
regimens by ambulatory hemodialysis patients." Journal of Behavioral Medicine 4.1 (1981): 111-
127.  
10. Sanjay A, Burner E, Terp S, Nok Lam C, Nercisian A, Bhatt V and, Menchine M. Improving 
attendance at post-emergency follow-up via automated text message appointment reminders: a 
randomized controlled trial. Academic Emergency Medicine. Nov 2014; 22(1): 31-37. 
11. Scherer TM, Lewis LM. Follow-up to a Federally Qualified Health Center and Subsequent 
Emergency Department Utilization. Academic Emergency Medicine. 2010;17(1): 55-62.  
12. Dziuba D, Lewis LM. Barriers to follow-up from an urban ED. ACEP Research Forum Abstracts 
2009.  
13. Dixon RF. Enhancing primary care through online communication. Health Aff (Millwood). Jul 
2010; 29(7): 1364-9.  
14. Kilo CM. Transforming care: medical practice design and information technology. Health Aff 
(Millwood). Sept-Oct 2005; 24(5): 1296-301.  
15. Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health 
record with secure messaging: impact on primary care utilization. Am J Manag Care: Jul 2007; 
13(7): 418-24.  
16. Som A, Patel K, Sink E, Peters RM, Javaherian K, Groenendyk J, An T, Xu Z, Polites GM, Blanchard 
M, Ross W. A novel patient engagement platform using accessible text messages and calls 
(Epharmix/CareSignal): feasibility study. JMIR Formative Research. 2017; 1(1):e2. 
17. Som A, Groenendyk J, An T, Patel K, Peters RM, Polites GM, Ross W. Improving dialysis 
adherence for high risk patients using automated messaging: proof of concept. Sci Rep. Jun 
2017; 7: 4177. 
 
 

 
7
 
 
Protocol from IRB #201504079: Improving medical care in ED discharge referral in patients with 
electronic interventions based on automated text and phone messages  
 
Content 
A Overview of Study ……………………………………………………………………………………………………………....……… 9-10 
A1 Study Design Summary 
A2 Pilot Data 
A3 Epharmix/CareSignal Intervention Builder Summary 
B Pre-Enrollment Period, Screening, Informed Consent & Study Entry ……………..…………………….…….. 10-12 
B1 Participant Identification 
B2 Eligibility Criteria 
B3 Screening 
B4 Informed Consent 
B5 Study Entry 
B6 Randomization and Blinding 
B7 Follow-Up Communication 
B8 Clinical Workflow Precautions 
B9 Participant Withdrawal 
C Study Procedure ..……………………………………………………………………………………………………..…………..…… 12-13 
C1 Intervention Treatment 
C2 Participant Education on Research vs Clinical Care 
C3 Changes to Standard of Care 
D SMS and Phone Application ……………………………….…………………………………..…………………………………… 13-14 
D1 Epharmix/CareSignal  
D2 Regulatory Status 
D3 Method of Use of Epharmix/CareSignal 
D4 Alert Management 
E Follow-up Visit Schedule.................................................................................................................. 14 
F Data Collection and Analysis …………………….……………………….………..………………………………..…….….…. 14-15 
F1 Outcome Assessments 
F2 Confidentiality and Security  
F3 Data Quality Assurance 
F4 Data Analysis and Sample Size Estimation  
G Safety and Adverse Events............................................................................................................... 15-16 
G1 Definitions 
G2 Reporting of Adverse Events 
G3 Data Collection Procedures for Adverse Events 
G4 Follow-up of Serious Adverse Events 
H Risk and Benefits ……………………..……………………………..……………………………………….………….………..…… 16-17 
I Study Administration …...................................................................................................................... 17 
I1Key Personnel 
I2 Key Personnel Training Certification  
I3 Funding Source and Conflicts of Interest 
I4 Subject Stipends or Payments 

 
8
 
Appendix ……….…................................................................................................................................. 18-37 
1 Written Consent Form 
2 Demographic Survey 
3 Verbal Consent Form 
4 Pilot Study Timeline 
5 Pseudocode 
6 Adverse Events Log 
7 Security Clearance 
8 Text Message Example Screenshots 
 
A. Overview of Study 
 
A1. Study Design Summary 
 
Key elements of study design under 2015040479 
 
Type of study 
PROBE 
Total N 
~330 
Intervention Arm 
Automated text and phone messages + Standard 
of Care 
Control Arm 
No phone messages + Standard of Care 
Power 
90% 
Hypothesis 
Adherence to follow-up visits will improve in 
intervention arm 
 
Data will be collected regarding scheduling and attendance of follow-up appointments by assessing an 
automated bi-directional text or phone intervention and reviewing patient charts [see Section C1. 
Intervention Treatment].  
 
This intervention was an add-on to standard of care with the goal to research the efficacy of an 
automated two-way phone system in improving follow-up appointment adherence in ED discharge 
patients. 
 
A2. Epharmix/CareSignal Intervention Builder Summary 
An electronic intervention platform was developed by Epharmix/CareSignal, a WUSTL IDEA Labs 
(ideas.wustl.edu) company, which uses an automated two-way text or phone system for various uses to 
supplement clinical care, such as tracking patient adherence, collecting additional data, or delivering 
educational follow-up. The platform is called Epharmix/CareSignal Intervention Builder and is uploaded 
on Washington University (WU) servers as an application. On this platform our research team has a 
unique account on which we built a series of messages [see Appendix 5. Pseudocode] to test our 
hypothesis. Automated text or phone messaging was delivered from this account to intervention 
participants and participant responses returned directly to this unique account, allowing research team 
members to supervise responses and have control over the intervention.  
 
Epharmix/CareSignal, Inc. and Epharmix/CareSignal employees are only involved in this project as a 
service provider, giving WUSTL personnel access to create an account on their platform. One employee, 
the Chief Developer, assists in aggregating and deidentifying data from our account [see Section D3. 
Method of Use of Epharmix/CareSignal Intervention Builder]. 

 
9
 
 
B. Pre-Enrollment, Screening, Informed Consent & Study Entry 
 
B1. Participant Identification 
The trial enrolls any interested adult who is discharged from the Barnes Jewish ED and is recommended 
to see a referral provider. Discharge instructions have to be complete and the patient needs to be able 
to use a consistent phone, have a way to understand English communication, and be capable of 
consenting. 
 
B2. Eligibility Criteria 
Inclusion criteria: 
1. 18 years and older. 
2. Accessible SMS capable mobile phone or residential landline. 
3. Able and willing to provide consent and authorize access of participant medical record for study 
use.  
4. Able to read English or have English-speaking family member to assist with phone 
communications. 
5. Supplied a clinical referral with recommended outpatient follow-up timeframe or date within 4 
months of discharge. 
 
Exclusion criteria: 
1. Unable to be contacted by a phone call and/or SMS text message, or unwilling to provide a 
phone number. 
2. Unwilling to consent and follow the assigned regimen and complete the required follow-up. 
3. Non-English speaking. 
4. Neurologic, anatomic, or cognitive disorders and thus unable to consent and/or answer text 
messages/phone calls. 
5. Already had a follow-up appointment scheduled before being discharged. 
 
B3. Screening 
Enrollment for the trial occurs on a rolling basis for a six-month period. Treating physicians in the ED 
identify potential participants after completing a patient’s discharge and follow-up instructions. The 
treating physician notifies the Emergency Care Research Core (ECRC) research coordinator with the 
patient’s name and ED room number. While the patient is in their room, the research coordinator 
approaches the patient to introduce the study and discuss participation. 
 
B4. Informed Consent 
The research coordinator identifies him/herself as a research staff member that is separate from clinical 
care. Follow-up care is not affected by a patient’s choice to enroll. The research coordinator reviews the 
consent form [see Appendix 1. Informed Consent Form] with patients and obtains written consent along 
with a demographic survey [see Appendix 2. Demographic Survey]. 
 
Patients who are unsure about enrollment can take the consent form home with them and decide to 
join from home, but before they attend a follow-up appointment. A phone number to the study’s 
primary contact is listed at the top of the consent form. Some patients are also contacted by phone by 
the research coordinator about participating in advance of a scheduled follow-up appointment. When 

 
10
 
consent is given over the phone, the research coordinator follows the verbal consent form [see 
Appendix 3. Verbal Consent Document]. 
 
B5. Study Entry 
Once a patient provides written consent, they are enrolled as a study participant. After a research 
coordinator obtains consent, s/he enters a participant’s preferred phone number, contact method (text 
or phone message), and preferred contact times into the research account on the Epharmix/CareSignal 
Intervention Builder platform via the web portal epx.wustl.edu. They also include the referral clinic 
name, clinic phone number, and clinic address that the treating ED staff instructs the patient to call for a 
follow-up appointment. It is possible to include more than one referral clinic if the ED staff provides a 
second option.  
 
B6. Randomization Method and Blinding 
Once a participant is added to the Intervention Builder account, the integrated software independently 
randomizes participants to either the control or the intervention arm. The software s randomly allocates 
participants in a 1:1 ratio to intervention and control arms.  
 
Even though participants are not informed if they were enrolled in the intervention or control arm, they 
are capable of inferring this information (if they did or did not receive a text or phone call). 
 
Treating clinical staff are blinded to arm allocation.  They complete treatment before a participant 
enrolls in the study.  
 
The research team will be blinded to arm allocation whenever possible. EMR review to identify attended 
follow-up appointments and ED re-visits will be completed before intervention responses are reviewed 
and analysis commences.  
 
B7. Follow-Up Communication 
There was not any follow-up communication in this research study. 
 
B8. Clinical Workflow Precautions 
Following discharge from an emergency department (ED), it is standard of care that patients receive 
written instructions and counseling from ED staff (physicians, nurses and as-needed social workers). 
Discharged patients are not normally contacted by ED staff after leaving the ED. The intervention is 
considered an add-on to this standard of care by facilitating communication to the participant’s 
designated provider  for their follow-up appointment.  
 
In the event that a discharged patient begins showing new signs or symptoms that required clinical 
evaluation before leaving the ED, the patient is required to start the standard clinical care process over 
again in compliance with ED guidelines and the Emergency Medical Treatment and Labor Act. 
Depending on the outcome of the subsequent evaluation, the patient may or may not meet research 
criteria, which is determined by the patient’s final disposition. 
 
If a participant’s ED visit results in scheduling a follow-up appointment at the time of their discharge, 
such as for a specialist consultation or social worker visit, they are not included in the study. 
 

 
11
 
B9. Participant Withdrawal 
If any participant wants to withdraw from the #201504079 study they may call the number listed at the 
top of the written consent form and ask the project lead or principal investigator to send a copy of a 
withdrawal letter to fill out, or they could follow instructions in the consent form to go to the Human 
Research Protection Office website and print off their own copy of a withdrawal letter. They would mail 
the letter to the address given at the top of the consent form [see Appendix 1. Written Consent Form].  
 
The research coordinator reviews this process with the participant at the time of their written consent. 
Participant data collected up to the time of withdrawal will still be analyzed. If the participant was still 
receiving automated messages when the research team received a withdrawal letter, the project lead 
located the participant’s information in the Epharmix/CareSignal Intervention Builder and discontinues 
the intervention from the research account. 
 
C. Study Procedure 
C1. Intervention Treatment  
After providing written informed consent, participants in the intervention arm begin receiving text or 
voice messages to help them schedule a follow-up appointment with the referral provider per the 
pseudocode template attached [see Appendix 5. Pseudocode]. By default, participants receive 
automated messages starting one hour following discharge if it is normal business hours (Monday 
through Friday 0900 to 1700), or at 1000 the next business day if they are discharged outside of normal 
business hours. Automated messages are sent up to three days in a row or until the participant 
responded or the participant opted out, whichever occurs first. The participant is able to respond to all 
messages by either using alphanumeric (1-3) or binary answers (Yes or No). The primary goal of the 
intervention is to connect the participant directly to the referral provider’s office to schedule an 
appointment. Once the participant hangs up with the office, the intervention asks if they were 
successful with scheduling an appointment.  
 
If the participant fails to respond to the intervention, the intervention leaves a voicemail detailing 
instructions on how to reach the referral provider’s office via a toll-free phone number. This toll-free 
number is the same number that showed up on the participant’s cell phone or landline caller ID when 
the intervention reached out to them. If the participant calls the toll-free number, the intervention 
works as it normally would.  
 
Once an appointment is made, the system sends reminders at 14 days, 7 days, 3 days, and 1 day before 
the appointment date. The day after the appointment, the intervention contacts the participant to see if 
they attended it. If the participant did attend, the intervention stops. If the participant did not attend 
and needs to reschedule, the intervention resets and starts over again.  
 
Participants can voluntarily opt-out of receiving any further messages at any time if they text STOP or if 
they press * on the keypad during an automated phone message. 
 
Participants in the control arm do not receive any messages.  
 
C2. Participant Education on Research vs Clinical Care 
At the time of written consent the research coordinator will advise participants that the messages they 
may receive are not coming from their treating physicians or clinical staff, but from an automated 
system that is separate from their medical care because it is only part of a research study. Participants 

 
12
 
are advised that if they have any health concerns, the automated system would not be able to help 
them and they need to reach out to their doctor or the emergency room as they normally would. If the 
participant feels they are having a health concern related to the study, they are encouraged to call the 
phone number to the main study contact at the top of their consent form. If the research team is 
notified, they willfollow reporting procedures for an adverse event [see Section F2. Reporting of Adverse 
Events]. 
 
C3. Changes to Standard of Care 
No automated clinical judgements are included in this study. The purpose of utilizing the 
Epharmix/CareSignal Intervention Builder in this study was purely to facilitate follow-up visits to referral 
providers. 
 
D. SMS and Phone Application 
D1. Epharmix/CareSignal 
This study under umbrella IRB 201504079 uses the Epharmix/CareSignal Intervention Builder platform to 
develop an automated outreach that facilitates patient follow-up visits with referral providers and 
collects appointment follow-up rates. The platform utilizes two-way text messaging or automated phone 
calls to acquire data from participants, at intervals set by the principal investigator. The research team’s 
unique account on the Epharmix/CareSignal Intervention Builder collected follow-up appointment data. 
Only one Epharmix/CareSignal employee, the Chief Developer, had access to the Intervention Builder 
platform in order to provide routine maintenance. This employee had updated HIPAA and CITI training. 
The Epharmix/CareSignal Intervention Builder was not a phone application; participants did not need to 
download or install anything on their phones in order to receive the research texts or phone messages. 
Participant phone responses were automatically stored on the Epharmix/CareSignal Intervention Builder 
site, epx.wustl.edu, in the specific research account that is password protected. Epharmix/CareSignal, 
Inc does not have access to this password. In addition, the site is SSL-encrypted, ensuring password 
safety in transmission.  
 
D2. Regulatory Status 
The FDA has not made an assessment of risk with regard to this study, and as a communication tool we 
are not requesting to conduct this study under FDA NSR requirements. Epharmix/CareSignal 
Intervention Builder is not an implant and does not present a potential for serious risk, does not support 
or sustain human life, and is not of substantial importance in diagnosing, curing, mitigating, or treating 
disease, and does not have serious risk for health, safety, or welfare of a subject. 
 
D3. Method of Use of Epharmix/CareSignal Intervention Builder 
After enrollment, the research coordinator will initiate use of the research account’s intervention that 
was built on the Epharmix/CareSignal Intervention Builder. The research account will send an initial text 
message or phone call as described in Section C1. Intervention Treatment. Since patients discharged 
from the ED do not have any further contact with ED providers, this intervention is not designed to 
sustain communication between the clinical provider and the participant. This intervention is instead 
fully automated to connect participants to a referral provider or service. It does not request health 
information from participants nor deliver any health information to them. The pseudocode responds to 
a participant’s success in setting up a follow-up appointment and provides alternate choices if a follow-
up appointment could not be made [see Appendix 5. Pseudocode]. 
 

 
13
 
Once the intervention is completed with the last enrolled participant, the project lead will request  that 
the Epharmix/CareSignal Chief Developer aggregate and extract a raw data file (.csv) from the research 
account that the study manager encrypted so that the data is unviewable by the Chief Developer. The 
Chief Developer places this encrypted data file on an encrypted flash drive that is handed off to the 
project lead. The project lead uploads the encrypted file onto a secure database on box.wustl.edu. 
 
D4. Alert Management 
No alerts were used as a part of the intervention. 
 
E. Follow-up Visit Schedule 
No additional follow-up visits were required for this study. Participants followed up with their referral 
providers as they normally would. 
 
F. Data Collection and Analysis 
F1. Outcome Assessments 
The outcomes of interest are: 
• 
Primary: Participant follow-up attendance to a referral clinic or provider (compliant / non-
compliant). 
• 
Secondary:  
o Number of re-admissions to the ED (BJH or other) within 30 days and again within four 
months post-discharge. 
• 
Tertiary: 
o Patient engagement levels with the intervention: number of successful attempts to 
reach participant, number of successful attempts to schedule a follow-up appointment, 
length of phone call to schedule an appointment, and number of successful follow-up 
appointments with a specialist provider versus a primary care provider. 
 
To minimize bias, all clinical staff and research team members conducting patient chart reviews will be 
blinded to participant arm allocation [see Section B6. Randomization Method and Blinding]. We 
anticipate the study will have no impact on clinical workflow since the intervention will take place during 
a gap in clinical care when a participant is transferred between providers and we will only be completing 
patient chart assessments and data analysis to finalize the study. 
 
F2. Confidentiality and Security 
The Epharmix/CareSignal Intervention Builder has been evaluated by WU IT security before and 
approved for other studies [see Appendix 7. Security Clearance]. Participant responses to the 
intervention are stored on a HIPAA compliant WU server at epx.wustl.edu that has both encryption and 
penetration testing. Data was exported upon request after being de-identified in a .csv file and uploaded 
to Box [see D3. Methods of Use of Epharmix/CareSignal Intervention Builder]. 
 
Hard copies of consent forms and demographic surveys will be kept in a research binder in a locked 
cabinet in a locked office at the ECRC by the research coordinator until digitally archived. All patient 
identifiers will be destroyed at the earliest   time possible. 
 

 
14
 
F3. Data Quality Assurance 
An interim patient chart assessment and analysis of intervention responses will be performed four 
months after enrollment of the 85th participant. The principal investigator is responsible for monitoring 
participant responses. 
 
F4. Data Analysis, Sample Size Estimation, and Study Power 
As detailed in Section A.2 Pilot Data, we calculated that we need to recruit at least 333 participants to 
have a power of 90% for a two-sided α = 0.05, to detect a 10% improvement of the primary outcome. As 
a Phase II study, we are attempting to identify the effect size of the intervention. 
 
We obtained the estimated improvement in the primary outcome from our pilot study. The number of 
eligible patients in the ED over a 6-month period were approximately 21% of admitted patients. We 
needed a sample size of 266 patients, and anticipated that roughly 20% of participants would be lost to 
follow-up, which means we needed to recruit 333 participants. We calculated our sample size using the 
G*Power3 program. 
 
 
G. Safety and Adverse Events 
 
G1. Definitions 
Adverse Events (AE) would include a data breach in this study.  
 
Serious Adverse Events (SAE) are defined as any medical occurrence that results in death, is life 
threatening, requires inpatient hospitalization, results in persistent or significant disability, is a 
congenital anomaly or birth defect, or is an event requiring medical intervention to prevent any of these 
examples of an SAE. SAEs may be mild (transient, easily tolerated by participant), moderate (causes 
discomfort or interrupts the study or the participant’s usual activities), or severe (causes considerable 
interference with usual activities). 
 
G2. Reporting of Adverse Events 
In the study, all adverse events, whether expected or unexpected, will be reported to Washington 
University’s Human Research Protection Office (HRPO) [see Appendix 6. Adverse Events Log]. The HRPO 
office will be notified of any serious adverse experience within ten working days of the occurrence. If the 
event is fatal or life threatening, HRPO will be notified immediately, but not more than twenty-four 
hours after occurrence. Other adverse events will be reported to HRPO in annual reports. Adverse 
events will be reported from the time of participant enrollment to four months post-enrollment. 
 
In the case of a data breach, Information Security (IS) will be notified immediately and the IS security 
date and number will be recorded. Participants will be notified by a phone call from the research 
coordinator. 
 
G3. Data Collection Procedures for Adverse Events 
In cases where the principal investigator deems it necessary for a participant to stop receiving the 
intervention prior to end point (e.g. change in health rendering patient unable to participate), data will 
be included up to the date of the event. 
 

 
15
 
G4. Follow-up of Serious Adverse Events 
All SAEs will be followed until resolution, permanent outcome of the event or until stabilization. Some 
SAEs will require study discontinuation. For example, participants will discontinue the study if they are 
unable to tolerate study participation. These participants will continue to undergo follow-up evaluation 
by the investigator, direct consultation between the investigators and the treating physicians and 
establishment of ongoing care plans before being discharged from the study. For participants 
experiencing an SAE that does not require study discontinuation, continuing study participation will 
require discussions of the potential risks and benefits of continuing participation, involving the 
participant and their treatment team, the investigators and the participant’s treating physicians. 
 
H. Risks and Benefits 
Potential risks include possible breach of confidentiality involving PHI/study data and possible risk of 
annoyance by the subject due to repetition of messages. 
 
Efforts will be dedicated to minimize the risk of breach of privacy. All patient information and study data 
will be password protected in computer files and all paper forms will be kept in a secure locked office 
within a locked suite, controlled by the research staff. Additionally each patient will be assigned a study 
ID number to minimize risk of exposure of patient information. The ID number will be used to code all 
the data assigned to each patient as they are included sequentially into the study. This number will be 
used for the electronic spreadsheet and placed on the paper documents (if created) and a list of the 
assigned numbers will be maintained in a password-protected electronic spreadsheet.  
 
Communication via phone and text are unsecured and potentially non-confidential. Additionally the 
information stored on a patient’s phone is outside the control of the study investigator. The patient was 
informed of these risks and was required to consent to exchange potential Protected Health Information 
(PHI) via these communication lines to participate in the study. To prevent the accidental disclosure of 
PHI from pop-up phone notifications, the text message read: “Confidential message” and the content 
began 3 lines later, which was not seen on an automatic notification preview [see Appendix 8. Text 
Message Example Screenshots]. 
 
The major side-effect of this intervention is potential annoyance and disruption to patient lives. To 
reduce these risks, text messages and phone calls are reduced to the minimum possible to meet 
reminder or patient communication guidelines. In addition, efforts are made to ensure that the text 
message and phone calls are kind, courteous, respectful, and timely.  Patients are also given the 
opportunity to opt out at any time via a “STOP” message or the * dial key. 
 
 
I. Study Administration 
 
I1. Key Personnel 
PI: Brent Ruoff, MD 
 
I2. Key Personnel Training Certification 
All research team members complete Washington University HIPAA and CITI training courses. They are 
also registered with the Washington University Human Research Protection Office at 
myIRB.wusm.wustl.edu and added to the study’s research team prior to viewing PHI.  
 

 
16
 
I3. Funding Source and Conflicts of Interest  
This study was supported by the NIH/National Center for Advancing Translational Sciences (NCAT), grant 
ULI TR000448. Epharmix/CareSignal, Inc is also providing in kind services that allows us to create a free 
account for this study on the Epharmix/CareSignal Intervention Builder platform where we can design 
and build our own unique intervention. Automated phone or text messages that we send out from our 
account are free of charge to the study, as well as the phone responses that participants send back to 
our account. Phone companies may still charge participants for their responses based on participants’ 
individual data and phone plans. These charges are outside of our control. 
 
I4. Subject Stipends or Payments 
No payment or reimbursement is offered for this study. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
17
 
Appendix 
1. Informed Consent Document  
 
 

 
18
 
 

 
19
 
 
Brent Ruoff 

 
20
 
 

 
21
 
 

 
22
 
 

 
23
 
 

 
24
 
 
 
 
 

 
25
 
2. Demographic Survey 
 
 
 
 

 
26
 
3. Verbal Consent Document  
 
 
 

 
27
 
 
 
 
 

 
28
 
4. Pilot Study Timeline  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
29
 
5. Pseudocode 
 
 
 
 
 
 
 
 
 
 

 
30
 
 
6. Adverse Events Log 
 
 

 
31
 
7. Security Clearance 
 

 
32
 
 

 
33
 
 
 
 

 
34
 
8. Text Message Example Screenshots 
 
 
 
 

 
35