Improving Influenza Vaccination Delivery Across a Health System by the Electronic Health Records Patient Portal RCT 5 (LADHS)
Phase: NA
Conditions: Influenza, Respiratory Tract Infections
Sponsor: University of California, Los Angeles

Study Title: Improving Inﬂuenza Vaccination Delivery Across a Health System by the Electronic 
Health Records Patient Portal RCT 5 (LADHS) 
Initial IRB Approval: 12/20/2017 
RCT 5 (LADHS) IRB Approval: 8/29/2022 
Clinical Trials Registration: NCT05535777 
 
 
 

Significance: 
Previous studies have shown that low-cost, behavioral nudges through texting can increase 
influenza vaccination uptake compared to usual care.  However, there are limited studies that 
evaluate the effect of decreasing friction/barriers to scheduling especially within safety net 
populations.  This study will examine the effect of strategies to reduce friction/barriers to 
scheduling including bidirectional text messages, texts for patients to call for direct scheduling 
without wait time, and texts highlighting MediCal health plan transportation resources on 
influenza vaccination rates in adults within the 2022-2023 flu season.  
 
Objectives: 
This will be a randomized clinical trial (RCT) to test the impact of text message innovations on 
receipt of influenza vaccination during the 2022-23 season. Specifically, we will simultaneously 
test, using a comparative effectiveness trial design, the effect of the following 3 interventions, all 
compared to standard text messages: 
• 
Enhanced texting (with a callback by a trained call-center staff member to schedule a 
vaccine visit if the patient presses “1” in response to the text 
• 
Enhanced Bidirectional texting- with a texting response from a trained call-center staff 
member who will help the patient schedule a vaccine visit through a series of back-and-
forth texts 
• 
Provision of transportation information- with information within the text messages that 
highlight to patients that MediCal will pay for transportation to a vaccine visit   
 
Study Design: 
The study design is shown in this figure. Intervention arms are described below. 
 
 
 
We will send the total patient population of around 250,000 one of several types of text 
message reminders.  
 
 
Selection of Patients: 

Patient eligibility: Any patient who is deemed by LADHS to be an LADHS patient will become 
the patient denominator. Primary care patients who are empaneled to a primary care provider 
within DHS are assigned as follows: (1) assigned through managed care plans, (2) self-pay but 
linked with a provider, or (3) uninsured but assigned. We will include all patients over 6 months 
of age. 
 
Identification of families (i.e., households): We will group families using one of two possible 
methods. First, LADHS has an identifier that ties family members together. It is called the 
“Family Budget Unit.” We will explore this method. If it does not work, we will use another 
method that we have utilized successfully in the UCLA-based study from which this study is 
derived.  We will assume that people living in the same address belong to the same family. As a 
secondary measure, we use a telephone number as a linking number (i.e., if two people have 
the same telephone number, we assume they are from the same household).  
 
We will randomly allocate families to one of the study arms shown above and send family 
members the allocated text message. All family members will therefore receive the same text 
message. This is meant to reduce confusion among patients. For the analysis, we will randomly 
select one index patient per household for analysis. This follows Cochrane criteria for being 
included in meta-analyses. We will attempt to affiliate index patients with practices, and 
randomize patients (families) within these practices.   
 
Assignment to the intervention: All patients will be eligible for the intervention. There are no 
specific inclusion or exclusion criteria. Since we will send up to 3 text messages in October-
December 2022, we will search the EHR for whether or not the patient received an influenza 
vaccination this season. If yes, the patients will be dropped form the denominator of patients 
needing a text message reminder. We will search the EHR prior to each of the text message 
rounds. 
 
Review of medical records is needed in order to obtain the information about (a) influenza 
vaccination dates this upcoming season, and (b) address (in order to group patients into 
households as described above. We will review the EHR prior to each round of text messages 
to see if patients had received an influenza this season.  
 
Description of the Interventions: 
Text messages will begin to be sent in early October. We will randomly spread out the calls 
throughout the day and week in order to minimize unnecessary overload for the call center 
when patients text back a “1” response that they would like a call or text message to help them. 
We can do this by randomizing patients into batches so that there is no systematic time-of-day 
or day-of-week for any patients. We will send up to 3 reminders this upcoming season.  
 
Standard Texts (n=232,950): Patient randomized to this study arm will receive up to 3 text 
messages, reminding them about the importance of influenza vaccination. The messages 
themselves are shown below. The standard texts will include a clinic call back number and 
patient portal self-scheduling for patients to schedule their influenza vaccines. The direct 
scheduling texts includes a direct number to an agent that can help schedule and answer 
questions on the phone in real time. The texts with a direct number to schedule will link a 
specified phone number to call and schedule.  This number would be answered by a central 
agent quickly and a patient could schedule their flu shot at any clinic site.  This specific phone 
number would not go through the multiple option menus a patient would normally experience 
when calling their clinic. 
 

Standard Texts with Transportation (n=3,050): The standard texts with transportation includes 
phone number for patients to reserve transportation to a vaccination appointment based on 
MediCal health plan transportation resources.  
 
Enhanced Texts- Callback by a person (n=7,000): Patient randomized to this study arm will 
receive a phone call back by a call center agent if they press “1” in response to a question on 
the original text message. The call center agent’s job is to schedule patients for clinical visits. 
These call center agents will be trained by our faculty and staff and will have the usual HIPAA 
and other patient confidentiality training.  
 
Enhanced Bidirectional Texts- to help schedule a vaccine visit (n=7,000): Patients randomized 
to this study arm will receive a text message from a call center agent if they press “1” in 
response to a question on the original text message. The bidirectional texts will have an agent 
who can answer questions and schedule an appointment through text message back-and-forth 
conversations with the patient.  Bidirectional texts will be exchanged on a HIPAA compliant 
bidirectional text messaging platform.  The patient will use the regular SMS function on their 
cellphone and the agent will receive and respond to the bidirectional text on the HIPAA 
compliant platform.  
 
Text Message Wording: 
The wording of each of the text messages is shown in the figure below. We will have these 
translated into Spanish. LADHS patients have the option of receiving text messages in English 
or Spanish.  
 
Prepped Text Messages for DHS Flu Campaign 2022 - 2023 
Standard Text:  
#PCMH#:  #FirstName#, your flu shot is ready! Call your clinic at #ExternalRefNo# or visit the LA Health 
Portal at http://bit.ly/BookFluShot if you want this shot held for you. 
 
Standard Texts (+ Transportation)  
#PCMH#:  #FirstName#, your flu shot is ready! Call your clinic at #ExternalRefNo# or visit the LA Health 
Portal at http://bit.ly/BookFluShot if you want this shot held for you.  You can call #Health Plan Phone 
Number# to schedule a free ride your flu shot through #Health Plan Name# 
 
Examples (Transportation) 
• 
Standard Texts (+ Transportation) - LA Care 
#PCMH#:  #FirstName#, your flu shot is ready! Call your clinic at #ExternalRefNo# or visit the 
LA Health Portal at http://bit.ly/BookFluShot  if you want this shot held for you.  You can call 
888-839-9909 to schedule a free ride for your flu shot through your LA Care health plan. 
 
• 
Standard Texts (+ Transportation) - Health Net 
#PCMH#:  #FirstName#, your flu shot is ready! Call your clinic at #ExternalRefNo# or visit the 
LA Health Portal at http://bit.ly/BookFluShot  if you want this shot held for you  can call 855-253-
6863 to schedule a free ride for your flu shot through your Health Net health plan. 
 
Enhanced Prompt for Phone Scheduling: 
#PCMH#: #FirstName#, your flu is ready! Call our direct number xxx-xxx-xxxx to connect with an agent 
to hold this flu shot and schedule an appointment now. 
 
Bidirectional Texts:  
#PCMH#: #FirstName#, your flu shot is ready! Reply Y if you want this shot held for you and an agent 
will text you back to schedule an appointment. 

For more details, please see: 
https://docs.google.com/document/d/12ERynaBMjzpIpjcPJDoBCu-
8GA7cVBbYV8zko3mCBnM/edit 
And a brief here: 
https://docs.google.com/document/d/1Au5l7zjVqJSpQzi5NF7YvLR07mmyaMlz89dpDHWOob8/
edit 
 
Data Management: 
All study data, including CAIR2 data and DHS EMR and administrative data will be prepared by 
the DHS-CTSI Informatics Core in accordance with the DHS Research Oversight Board policy. 
Dr. Abhat will submit all data requests to the DHS helpdesk, which includes verification of 
procedural compliance with IRB and ROB approval screening for DHS-based research. Only 
HIPAA-limited or deidentified data sets will be transferred for analysis purposes; compliance 
with the DHS secure data transfer process to UCLA and/or USC will be managed by Dr. Abhat 
as DHS Principal Investigator. Any links between identified data and deidentified data will be 
stored only on DHS premises. Patient contact lists and personalized data elements will be 
exclusively maintained on premises by the Informatics Core, responsible for implementing the 
patient assignment and personalization templates.  
 
 
Statistical Analysis of Randomized Trial. We will perform separate evaluations of the impact of 
the text enhancements (callback and bidirectional texting) on the one hand, and transportation 
messaging on the other hand. For both analyses, we will use multivariable log-binomial 
regression models with vaccination status at the end of the campaign as the outcome variable. 
Regressions will include indicators for each intervention as fixed effects, with the standard text 
condition as the reference group, and potentially clinic random effects, depending on 
retrospective analysis results. Models will also adjust for patient age, gender, race/ethnicity, 
insurance, and whether the patient was vaccinated in at least one of the prior two flu seasons. 
Secondary analysis will include subgroup interactions with intervention indicators to assess 
heterogeneity of treatment effects.  In exploratory analyses, we will investigate interactions 
between interventions, questions related to modality of message delivery, as well as whether 
encounter type, click rates on messages, and distance to the health care facility from the home 
address affect influenza vaccination rates.  
 
 
In addition, we will evaluate the loss to follow-through rates (scheduling an appointment, but did 
not complete the appointment) by patient demographics, as distance to the health care facility 
from the home address).  
 
 
For the evaluation of text enhancements, we will perform a 3-fold Bonferroni correction 
accounting for pairwise comparison of intervention arms, using a two-sided significance level of 
0.017. For the evaluation of transportation messaging, we will use a two-sided significance level 
of 0.05. All analyses will be intention-to-treat, and will be performed using SAS v. 9.4 (SAS 
Institute Inc., Cary, NC).