Improving Influenza Vaccination Delivery Across a Health System by the Electronic Health Records Patient Portal RCT 4
Phase: NA
Conditions: Influenza, Respiratory Tract Infections
Sponsor: University of California, Los Angeles

Study Title: Improving Influenza Vaccination Delivery Across a Health System by the Electronic 
Health Records Patient Portal RCT 4 
Initial IRB Approval: 12/20/2017 
RCT 4 IRB Approval: 10/18/2021 
Clinical Trials Registration: NCT05052190 
-Terminated early- 
 
 

Protocol 
Brief Summary: 
This trial is taking place in Los Angeles, CA among patients from primary care practices within 
the UCLA Health System. 
Despite the Advisory Committee on Immunization Practices (ACIP) recommendation in 2010 
that all people above 6 months of age should receive an annual flu vaccine, vaccination rates 
remain low. The investigators plan to assess the effectiveness of text and portal reminder/recall 
messages with direct appointment scheduling (pre-appointment, and no pre-appointment) 
encouraging influenza vaccinations sent to patient portal users using a 2 x 2 nested factorial 
design. 
Methods and Procedures 
Sub-optimal vaccination rates are a significant problem in the U.S., despite their effectiveness in 
preventing morbidity and mortality from vaccine-preventable illness. For influenza specifically, 
annual epidemics of influenza cause substantial morbidity in the U.S. with up to 40,00-80,000 
deaths/year and many hospitalizations, emergency and outpatient visits, and significant costs. 
Reminder/recall (R/R), sent by phone, mail or other modality, can improve child and adult 
influenza vaccination rates. However, the majority of pediatric or adult primary care practices 
do not conduct R/R. Barriers are lack of finances, personnel, and algorithms to identify eligible 
patients. 
A technological breakthrough that might overcome these barriers involves patient portals-- 
secure, web-based communication systems, embedded within electronic health records (EHRs), 
for patients and providers to communicate with each other via email and the internet. Portals 
are used by about half of Americans and half of UCLA patients. Another is text messaging at the 
health system level. 
This randomized controlled trial will assess the effectiveness of reminders messages sent to 
portal users, encouraging influenza vaccination, on increasing influenza vaccination rates within 
a health system. Patients will be randomized into three groups: 1) one-third of patients will 
receive reminder letters via the portal, 2) one-third will receive reminders via text messages 
sent by the health system and 3) one-third will not receive reminders. 
Patients in the reminder arms will be further randomized (separately) for two additional 
components: 1) direct appointment scheduling and 2) pre-appointment reminders. 
 
Direct scheduling: The convenience of scheduling an appointment for influenza vaccination may 
increase the likelihood that patients receive an influenza vaccination. Patients receiving 
reminders will be randomized to receive one of two types of reminder letters: 1) one half will 

receive flu vaccine reminder letters with a link to the MyChart direct appointment scheduling 
page and 2) one half will receive flu vaccine reminder message without the direct appointment 
scheduling link. 
Pre-appointment flu reminders: Finally, a missed opportunities for vaccination occurs when 
patients are eligible for vaccination but are not offered a vaccine at their medical appointment. 
Pre-appointment reminders, sent in advance of an upcoming appointment, encouraging the 
patient talk to their doctor about influenza vaccination, may increase the likelihood of that 
patient receiving a vaccine. Patients will be randomized to one of two groups: 1) one half will 
receive modified pre-appointment reminders that mention influenza vaccination and 2) one 
half will receive the standard pre-appointment reminders for an upcoming appointment (no 
mention of influenza vaccination). 
Interventions that apply Behavioral Economics principles in vaccine promotion messaging can 
increase vaccine receipt. We will include ownership language in the pre-appointment messages. 
Additionally, the convenience of reminders sent via text and with a Direct Appointment 
Schedule link included enables patients to self-schedule a flu vaccine only visit, reducing friction 
for vaccine receipt. 
For the primary analysis, the primary outcome will be the patient's end of flu season 
vaccination status. Intervention effects will be assessed using mixed effects log-binomial 
models. Models will contain terms for messaging modality (text v. portal v. none), pre-
appointment reminders (sent v. not sent), and direct scheduling link (included v. not included). 
The R/R text and portal messages (with and without direct scheduling links) and pre-
appointment reminders and will all begin in October of 2021.Languages: messages will be in 
English.   
Health Literacy: We will use plain language <7th grade reading level per Flesch-Kincaid analysis.  
Eligibility: Our EHR will identify subjects who are eligible for either 1 or 2 influenza vaccinations 
based on ACIP algorithms (2 are recommended for children <9 years who have not had prior 
vaccination). 
Inclusion criteria 
Inclusion criteria will involve-- all UCLA primary care patients over 4 months of age who have 
made at least 1 visit to the practice within the prior 3 years and who have signed up for 
MyUCLAhealth (UCLA’s portal) and have used the portal at least once within 12 months, from 
these patients, index patients per household will be randomly selected. 
a. Active patient: We selected >1 visit in 3y based on our prior studies and the literature to 
focus the intervention on active patients; practices do not have a good mechanism to inactivate 
patients.  

b. Focusing on Portal Users: We can only send portal R/R messages to patients who have signed 
up for the portal. Since some patients sign up for the portal but never use it we will restrict our 
inclusion criteria to patients who used the portal within 12m (75% of UCLA patients). This 
standard has been applied before. We will assess intervention effects for the included 
population, and also for the entire practice, since the impact of the intervention reflects the 
extent of portal penetration.  
Exclusion criteria 
Patients will be excluded if they are not a patient at one of the 67 primary care practices within 
UCLA’s Health System, and/or if they do not meet the above specified requirements for being 
an active patient or portal user. 
Main Outcomes and Measures 
Primary Measures:  
1) Receipt of >1 influenza vaccine among index patients between 9/1/2022-4/1/2023 as 
documented in the EHR which is supplemented electronically by the health system with 
influenza vaccination data from external sources (e.g., pharmacies) and claims data.   
2) Receipt of >1 influenza vaccine among pediatric index patients between 9/1/2022-
4/1/2023 as documented in the EHR which is supplemented electronically by the health 
system with influenza vaccination data from external sources (e.g., pharmacies) and 
claims data.   
Secondary Measures: Receipt of annual influenza vaccine between 10/14/2021-4/1/2022 
among all included patients in identified households. 
 
 
Arms 
Assigned Interventions 
Active Comparator: Text Message R/R 
with Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by text. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. 
Behavioral: Text Message R/R with 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by text reminding them 
to get a flu vaccine with a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Active Comparator: Text Message R/R 
without Direct Appointment Schedule 
Link 
Behavioral: Text Message R/R without 
Direct Appointment Schedule Link 

Participants in this arm will receive up to 
3 R/R messages by text with no link to 
schedule an appointment. 
Participants in this arm will receive up to 
3 R/R messages by text reminding them 
to get a flu vaccine without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Active Comparator: Text Message R/R 
with Direct Appointment Schedule Link 
+ text Pre-Appointment Reminder 
Participants in this arm will receive up to 
3 R/R messages by text. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. Participants 
will receive a Pre-Appointment 
Reminder via text when they schedule a 
visit during the study period. 
Behavioral: Text Message R/R with 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by text reminding them 
to get a flu vaccine with a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Behavioral: Text Pre-Appointment 
Reminder 
Participants will receive a Pre-
Appointment Reminder via text when 
they schedule a visit during the study 
period advising them to get the flu 
vaccine at their upcoming appointment. 
Active Comparator: Text Message R/R 
without Direct Appointment Schedule 
Link + Text Pre-Appointment Reminder 
Participants in this arm will receive up to 
3 R/R messages by text. R/R message 
will be sent without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. Participants will receive a 
Pre-Appointment Reminder via text 
when they schedule a visit during the 
study period. 
Behavioral: Text Message R/R without 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by text reminding them 
to get a flu vaccine without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Behavioral: Text Pre-Appointment 
Reminder 
Participants will receive a Pre-
Appointment Reminder via text when 
they schedule a visit during the study 
period advising them to get the flu 
vaccine at their upcoming appointment. 

Active Comparator: Portal Message R/R 
with Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. 
Behavioral: Portal Message R/R with 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. 
Active Comparator: Portal Message R/R 
without Direct Appointment Schedule 
Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Behavioral: Portal Message R/R without 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. 
Active Comparator: Portal Message R/R 
with Direct Appointment Schedule Link 
+ Portal Pre-Appointment Reminder 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. Participants 
will receive a Pre-Appointment 
Reminder advising them to ask their 
doctor for the flu vaccine at their 
upcoming appointment via portal when 
they schedule a visit during the study 
period. 
Behavioral: Portal Message R/R with 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent with a Direct Appointment 
Scheduling link and instructions to make 
a flu vaccine appointment. 
Behavioral: Portal Pre-Appointment 
Reminder 
Participants will receive a Pre-
Appointment Reminder via the portal 
advising them to ask their doctor for the 
flu vaccine at their upcoming 
appointment via text when they schedule 
a visit during the study period. 
Active Comparator: Portal Message R/R 
without Direct Appointment Schedule 
Link + Portal Pre-Appointment 
Reminder 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
Behavioral: Portal Message R/R without 
Direct Appointment Schedule Link 
Participants in this arm will receive up to 
3 R/R messages by portal. R/R message 
will be sent without a Direct 
Appointment Scheduling link and 

 
 
 
will be sent without a Direct 
Appointment Scheduling link and 
instructions to make a flu vaccine 
appointment. Participants will receive a 
Pre-Appointment Reminder advising 
them to ask their doctor for the flu 
vaccine at their upcoming appointment 
via portal when they schedule a visit 
during the study period. 
instructions to make a flu vaccine 
appointment. 
Behavioral: Portal Pre-Appointment 
Reminder 
Participants will receive a Pre-
Appointment Reminder via the portal 
advising them to ask their doctor for the 
flu vaccine at their upcoming 
appointment via text when they schedule 
a visit during the study period. 
No Intervention: No R/R Message or 
Pre-Appointment Reminder 
Participants do not receive any flu 
vaccine R/R message or Pre-
Appointment Reminder advising them to 
ask their doctor for the flu vaccine at 
their upcoming appointment during the 
specified flu season.