PROmotion of FLU Vaccine Uptake in the Emergency Department - PROFLUVAXED
Phase: NA
Conditions: Influenza Vaccination
Sponsor: University of California, San Francisco

 
 
 
Project Title: PROmotion of FLU Vaccine uptake in the 
Emergency Department – PROFLUVAXED 
 
 R01 AI166967-01 
 
Sub-study: RTC for flu vaccine messaging platforms 
Date of Document: September 16, 2022 
Final Revisions made November 3, 2022 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
Study Procedures Manual for PROFLUVAXED  – A 3-arm Cluster 
Randomized Trial:  
 
Contents 
 
I. 
Overview .............................................................................................................................. 3 
II. 
IRB ........................................................................................................................................ 5 
III. 
Deposition of Protocol into ClinicalTrials.Gov ..................................................................... 5 
IV. 
Sites ..................................................................................................................................... 6 
V. 
Randomization ..................................................................................................................... 6 
VI. 
Site Orientation and Training .............................................................................................. 7 
VII. 
Study Hotline and Quality Assurance .................................................................................. 7 
VIII. 
Recruitment, Inclusions, Exclusions and Consent: .............................................................. 7 
IX. 
Study Procedures: Intervention Blocks ................................................................................ 8 
X. 
Study Procedures: Control Blocks ...................................................................................... 11 
XI. 
Research Staff Informing of ED Providers when Participants will accept Covid vaccine ... 16 
XII. 
Consent for 1- Month Follow Up Surveys for patients saying No to Vaccine in ED ......... 161 
XIII. 
Community Covid Vaccine Information for patients saying No to Vaccine in ED ........ Error! 
Bookmark not defined.2 
XIV. 
Primary Outcomes and Ascertainment: .......................................................................... 173 
XV.  
Data Recording and Entry………………………………………………………………………………………………………….14 
XVI. 
Data Analysis ................................................................................................................... 185 
XVII. Data management plan ................................................................................................... 236 

XVIII. Sample Size Considerations ............................................................................................. 217 
XIX. 
Consents and Rationale ................................................................................................... 248 
 
 
Overview 
This PROmotion of inFLUenza VAX(ccine) in the Emergency Department PROFLUVAXED 
MOP is a sub-study and extension of the PROCOVAXED trial. The PROCOVAXED trial is a 
multicenter study that seeks to decrease COVID-19 vaccine hesitancy and increase COVID-19 
vaccine uptake through the use of vaccine messaging platforms in the emergency department 
(ED). In this trial we found that implementation of our COVID-19 messaging platforms (videos, 
information sheets and scripted, direct messaging) was associated with greater COVID-19 
vaccine acceptance and uptake in unvaccinated ED patients. (primary manuscript currently 
under review). 
 
Other aims for our PROCOVAXED NIAID Grant R01 AI166967-01 included the similar 
development and testing of influenza vaccine messaging platforms. Toward this influenza 
vaccine aim, we conducted multiple in-depth qualitative interviews with ED patients whose 
primary health care access is the ED. During these interviews we queried about flu vaccine 
access and reasons for hesitancy. We have reviewed these interviews in detail and developed 5 
PROFLUVAXED messaging videos, 5 flyers and a scripted message to be used in this trial. In 
this PROFLUVAXED sub-study of PROCOVAXED, we seek to address the specific aims below. 
Of note, while the general procedures and analysis closely mirror the PROCOVAXED study, we 
have adapted this protocol in response to notable findings and limitations in the parent 
PROCOVAXED study, as well as specifics about influenza vaccination, which only occurs 
during limited periods of the year.  
 
In our PROCOVAXED trial we noted that approximately 10% of patients who did not receive 
COVID-19 vaccine messaging agreed to receive the COVID-19 vaccine when questioned in the 
ED and 8% received the COVID-19 vaccine at 30 days, leading the investigator team to believe 
that merely asking patients whether or not they will accept vaccines (without any messaging) in 
the ED and informing their ED providers when they say they would accept it is an intervention 
unto itself that can lead to greater vaccine uptake. We intend to test that hypothesis in Specific 
Aim III of this PROFLUVAXED trial by adding a third arm in which participants will not be asked 
whether they would accept the influenza vaccine in the ED (and correspondingly, research staff 
will not inform ED providers that the participant would accept the influenza vaccine). This third 
arm will allow us to determine whether the intervention of asking patients whether they would 
accept an influenza vaccine in the ED increases vaccine uptake. It will also allow us to dissect 
reasons for success (or failure) of the primary intervention (vaccine messaging) and refine 
recommendations about vaccine messaging programs accordingly. This third study arm may 
also allow the study team to discern which of the two primary intervention components (the 
vaccine messaging vs the asking of the vaccine acceptance question and informing providers) 
has the greatest effect (or lack of effect) on influenza vaccine uptake.  
 
Because of Omicron variant associated surges during the COVID-19 pandemic with 
corresponding research staff illness and ED overcrowding, we also found wide week-to-week 

fluctuations in enrollment in the PROCOVAXED study. To reduce this variability of enrollment, 
we have changed the unit of randomization from 1-week to 1-day.  
 
Change in Time Horizon for Primary Outcome In our original study MOP from September 16, 
2022, we had designated the primary time horizon for vaccine uptake as the index study day 
(vaccine uptake during that ED visit). After consideration of data from the parent study regarding 
COVID-19 vaccine messaging in the ED and reviews from the editorial board regarding the 
manuscript, we decided that the more relevant time horizon was vaccine uptake at 30 days. 
https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2799590 
We made this change to the study analysis plan within the first month of patient enrollment 
(November 2002), before any study analysis had taken place.  
 
We seek to accomplish the following specific aims: 
 
Specific Aim I: To determine whether implementation of influenza vaccine trusted 
messaging platforms is associated with increased influenza vaccine uptake in 
unvaccinated ED patients.  
 
At six EDs (Zuckerberg San Francisco General, UCSF Parnassus Medical Center [San 
Francisco, CA], Thomas Jefferson University Hospital [Philadelphia, PA], Ben Taub Hospital 
[Houston, TX], Harborview Medical Center [Seattle, WA], and Duke University Medical Center 
[Durham, NC]), we will conduct a cluster-randomized controlled trial of implementation of 
PROFLUVAXED trusted messaging platforms, with influenza vaccine uptake in the ED as the 
primary outcome. Hypothesis: Implementation of PROFLUVAXED trusted messaging platforms 
in EDs will be associated with increased influenza vaccine uptake in unvaccinated ED patients. 
Specific Aim II: To determine whether implementation of influenza vaccine trusted 
messaging platforms in EDs is associated with increased influenza vaccine acceptance 
in unvaccinated ED patients. For this specific aim influenza vaccine acceptance in the ED 
assessed via ED survey will be the primary outcome. Hypothesis: Implementation of 
PROFLUVAXED trusted messaging platforms in EDs will be associated with increased 
influenza vaccine acceptance in unvaccinated ED patients. 
Specific Aim III: To determine whether implementation of a protocol in which ED patients 
are asked whether they will accept an influenza vaccine in the ED (and notifying ED 
providers when they say they will accept it) is associated with increased influenza 
vaccine uptake in unvaccinated ED patients.  Hypothesis: Implementation of an ED protocol 
in which patients are asked whether they will accept an influenza vaccine (and notifying ED 
providers when they say they will accept it) will be associated with increased influenza vaccine 
uptake in unvaccinated ED patients. 
 
 
 

General Design: This is a three-arm cluster-randomized controlled trial (CRCT) to accomplish 
Specific Aims I, II and III. 
Study arms 
PROFLUVAXED Intervention M 
arm (Messaging and Vaccine 
Question) 
• 
Vaccine messaging given 
• 
Vaccine acceptance 
question asked 
Intervention Q arm (Vaccine 
Question, but No Messaging)  
• 
No vaccine messaging 
• 
Vaccine acceptance 
question asked 
Control arm (No Messaging, No 
Vaccine Acceptance Question)  
• 
No vaccine messaging 
• 
No vaccine acceptance 
question  
 
I. 
IRB  
We will submit our protocol to the UCSF Committee on Human Research as a modification. We 
will continue with multi-site reliance mechanism for the PROCOVAXED study as per NIH 
guidelines for randomized trials.  
II. 
Deposition of Protocol into ClinicalTrials.Gov 
As per federal regulations, we will deposit our full study protocol into the repository 
https://clinicaltrials.gov/ 
 

Setting and Sites 
We will conduct this over 5 and a half months (mid-September 2022 to February 28, 2023) at six 
high-volume EDs in five cities: (Zuckerberg San Francisco General, UCSF Parnassus Medical 
Center [San Francisco, CA], Thomas Jefferson University Hospital [Philadelphia, PA], Ben Taub 
Hospital [Houston, TX], Harborview Medical Center [Seattle, WA], and Duke University Medical 
Center [Durham, NC]). We have chosen this time-period to coincide with the influenza 
vaccination season.  
Unit of Randomization 
 
Sites will be assigned be assigned to a condition for a day. Randomization within each of the 
site uses pseudorandom number to permute blocks of time. The blocks consist of 15 days 
duration during which each condition appears for 5 days. Hence, in any 15 day period there will 
be a balance of interventions within each of the sites. The particular days of each week 
(Monday, Tuesday, Wednesday, Thursday and Friday) for each of the study arms will thus vary 
randomly. This randomization scheme will minimize secular trends (changes in perceptions 
about the flu vaccine that may occur through the course of the influenza vaccine season). We 
will generate a full study calendar based on this randomization scheme. To try to maintain 
masking of allocation, sites will be given a blacked out study calendar and will be instructed to 
open the calendar for a particular study day the morning of that study day (other than that study 
day, the rest of the calendar will remain blacked out). 
 
 
 

Site Orientation and Training  
The Core UCSF Site will develop orientation materials to familiarize the ED Sites with the study 
protocol. Each site will employ one or more Clinical Research Coordinators (CRCs), who will 
report to the Site PI and be responsible for day-to-day study implementation. We will develop 
and disseminate a manual of operating procedures (MOP) with standard personnel training 
methods, including education kits with scripts, summary cards, and PowerPoint presentations to 
assist coordinators in the orientation of site clinicians and other staff to our study protocol. We 
will convene ZOOM conference calls to review this summary and develop plans for optimization 
of PROFLUVAXED messaging platforms to improve usability and workflow. We will refine 
procedures with updates delivered to the site PIs during weekly ZOOM conferences.  
Study Hotline and Quality Assurance 
We will maintain a study hotline and encourage study personnel to contact the PI and Central 
Study Coordinator for all issues and queries. Hotline hours will be during primary study hours 
(weekday 8 a.m. to 5 p.m. PST).  
We will enact rigorous methods for clinical trial quality assurance and performance 
improvement, including: 1) systematic review of enrollment logs, 2) weekly audits of random 
samples of data for accuracy and missing elements, and 3) structured review of protocol 
deviations or violations. The Central Study Coordinator will prepare monthly summary report 
cards, tabulating individual site quality assurance metrics for review during scheduled Steering 
Committee calls. The overall study PI (Dr. Rodriguez) will discuss site-specific data with site PIs 
individually and summarize these data collectively during Steering Committee calls, with prompt 
dissemination of plans for process improvement. 
Recruitment, Inclusions, Exclusions and Consent 
Practical budget considerations and limits on research personnel in patient care areas during 
the COVID-19 pandemic, preclude 24/7 delivery of the PROFLUVAXED trusted messaging 
platforms and enrollment in this study. We will use a convenience sample technique to 
approach all eligible adult patients who present to our study EDs during 8-hour (one-day) 
blocks, typically beginning at approximately 9 a.m. and continuing to approximately 5 pm. Sites 
will be given leeway to vary their particular study time hours, as long as these study hours 
remain consistent from week to week.  
Inclusions will be: 
1) Adults 
2) Presenting to ED  
3) Not already vaccinated for influenza in the current year 
4) Able to provide informed consent 
5) Fluent in English or Spanish  
6) Anticipated ability to complete study intervention in ED i.e., able to watch a 3-minute 
videoclip 
Reviewing ED triage information, we will exclude patients with the following characteristics:  

1) Age < 18 years 
2) Major trauma such that it will preclude survey 
 3) Inability to participate in a survey because of intoxication, altered mental status, or critical 
illness 
 4) Incarceration  
5) Psychiatric hold  
6) We will also exclude patients who state that they have already received an influenza vaccine 
and patients who are in the ED for suspected acute Covid or influenza illness.   
 
NOTE: For all 3 study arms, study procedures should be performed in patient waiting 
times and not interfere or disrupt patient care in any way. 
 
 
 

Procedures and workflow during PROFLUVAXED Intervention M study arm 
 
The anticipated flow of the study during the Intervention M study blocks is summarized in Figure 
1. CRCs and research personnel will begin by setting up their home base of Consents and 
platforms (video clips, printed materials and scripts for messaging).   
 
 
Introduction to ED Staff:  Clinical Research Coordinators (CRCs) will set up their workstation 
in the ED and introduce themselves to ED staff (nurses, physicians and mid-levels), informing 
them that they will be doing the PROFLUVAXED study that day. They will avoid telling 
providers whether this is an intervention vs control arm.  
Initial Screening and Scripted Consent for Surveys: CRCs will review ED dashboards for 
inclusion and exclusion information. When an eligible patient is identified, the CRC will ask the 
nurse or doctor caring for the patient whether it is Ok for them to approach the patient about the 
study. For provider approved patients:  CRCs will approach eligible patients and deliver a 
scripted consent for two short surveys (Pre-intervention) Intake Survey and the (Post-
Intervention) Vaccine Acceptance Survey. See Scripted Consent for the Intervention period. 
They will also get written HIPAA authorization for review of their ED EHR. If the patient 
does not agree to this HIPAA review of their EHR, they will be excluded from the study. 
Participants will not be compensated for participation. 
CRCs will complete screening and enrollment log indicating whether or not they agreed to 
participate. If they agreed to participate, the CRC will assign a Study ID#. 
Intake Survey:  We will administer the INTAKE SURVEY to participants. CRCs will have the 
option of inputting surveys to REDCap on iPads in real time or using paper surveys (and later 
inputting into REDCap). These surveys are to be delivered orally (CRC asks questions), not via 
handing them out. The Intake Surveys are the same for all three arms of the study.  
Intervention M (messaging):  The intervention will consist of three messaging platforms that 
were developed specifically to reduce influenza vaccine hesitancy. All platforms have been 
reviewed by the UCSF Committee on Human Research. 
 
1) Video clips – short (approximately 3-minute) Public Service Announcement type videos 
to be shown to the participant on an iPad. 
2) Printed materials – one page information sheets handed to subjects by CRCs. 

3) Face to face messaging – short (< 1 minute), scripted message from the patient’s 
providers in the ED (nurse or provider) 
 
Each site will maintain a library of  
A. 5 versions of the videos - the version used in any participant will be tailored to that 
participant’s stated race/ethnicity. See ***below 
B. 5 versions of printed flyers – likewise, the version will be tailored to the participant’s stated 
race/ethnicity. See ***below 
C. 1 version of scripted message to be delivered in English or Spanish. 
 
Influenza Vaccine Flyer, Videos and Telling Provider to Deliver Message: 
 
Interventions will be delivered in real-time patient visits in site EDs, during waiting times such 
that they will not interfere with patient care. At the end of the survey, the CRC will deliver the 
influenza vaccine information flyer and ask the patient if they will watch a short video about 
influenza vaccines. If they agree to watch the video, the CRC will give show them video on the 
iPad. After finishing with the video, the CRC will tell the subject that they will be back in about an 
hour for the Vaccine Acceptance survey. The CRC will then leave the room and ask the 
patient’s primary provider (doctor, mid-level practitioner, or nurse) to deliver the influenza 
vaccine message (hand them the scripted message). This message is short and should not 
significantly impact provider workflow. Notably, vaccine messaging is recommended in the ED 
by the American College of Emergency Physicians and other health care organizations (Centers 
for Disease Control).  
 
***We will deliver messaging from our platform libraries in patients’ preferred language (English, 
Spanish). To the extent possible, we will follow recommendations to choose platforms from site 
libraries that match video clip and printed material messengers with subjects’ likely preferences 
for race, ethnicity, age and gender (e.g., Latinx messenger on video clip with Latinx participant).  
 
Vaccine Acceptance Survey (Post-Intervention) in the ED: We will administer the Vaccine 
Acceptance Survey: INTERVENTION GROUP at some time (generally 60 minutes but up to 6 
hours) after the Intake Survey.   
Primary and Other Outcome Ascertainment: Primary outcome ascertainment of influenza 
vaccination within 30 days will occur by 3 methods:  
1) Ascertainment of vaccination in the ED as above by direct questioning and review of EHR 
records. 
2) Blinded review of EHR at 30-days.  
3) For participants who have not received an influenza vaccine upon questioning in the ED, 
staff will ask whether they will agree to phone follow-up at 30 days. For those agreeing to 
follow-up, we will obtain written consent for phone follow-up (phone follow-up consent form). 
We will then ask participants for their best phone number(s) to reach them for a follow-up 
phone call. Sites will maintain a separate password protected database of subject IDs and 
follow up phone #s.  
This EHR review will be conducted in a blinded fashion – the research staff person 
reviewing the EHR will be unaware of participant’s study group assignment. This method 

will assure that all study participants, even those who refuse phone follow-up, will have at least 
2 ways of 30-day study outcome ascertainment (1 and 2 above).  
Secondary outcome ascertainment of influenza vaccine during ED visit will occur via 
review of each participant’s ED EHR by research staff on the next workday after their index ED 
visit to confirm receipt (or non-receipt) of an influenza vaccine in the ED.  
The secondary outcome of stated vaccine acceptance (Aim II) will be ascertained during the 
Second Survey (Vaccine Acceptance Survey) via their response to the question “Would you 
accept the influenza vaccine in the emergency department today if your doctor asked you to get 
it?”. An answer of “yes” would be a positive for this outcome. 
 
 
 
 
 

Procedures and workflow during Intervention Q: (Vaccine Question, but No Vaccine 
Messaging) 
 
 
The workflow during this arm is identical to the Intervention M Arm except there will be no 
messaging platforms delivered. The anticipated flow of the study during Intervention Q Blocks 
is summarized in Figure 2. 
 
 
 
Introduction to ED Staff:  Clinical Research Coordinators (CRCs) will set up their workstation 
in the ED and introduce themselves to ED staff (nurses, physicians and mid-levels), informing 
them that they will be doing the PROFLUVAXED study that day. They will avoid telling 
providers whether this is an intervention vs control arm.  
Initial Screening and Scripted Consent for Surveys: CRCs will review ED dashboards for 
inclusion and exclusion information. When an eligible patient is identified, the CRC will ask the 
nurse or doctor caring for the patient whether it is Ok for them to approach the patient about the 
study. For provider approved patients:  CRCs will approach eligible patients and deliver a 
scripted consent for two short surveys – the Intake Survey and the (Post-Intervention) Vaccine 
Acceptance Survey. See Scripted Consent for the No Messaging arm period. 
CRCs will complete screening and enrollment log indicating whether or not they agreed to 
participate. If they agreed to participate, the CRC will assign a Study ID#. 
Intake Survey:  We will administer the INTAKE SURVEY to participants. CRCs will have the 
option of inputting surveys to REDCap on iPads in real time or using paper surveys (and later 
inputting into REDCap). These surveys are to be delivered orally (CRC asks questions), not via 
handing them out. The Intake Surveys are the same for all three arms of the study.  
Vaccine Acceptance Survey: We will administer the Vaccine Acceptance Survey at some time 
(generally 60 minutes but up to 6 hours) after the Intake Survey. These surveys in the control 
group retain the same key primary and secondary outcome questions as in the intervention 
group Vaccine Acceptance surveys. See Vaccine Acceptance Survey: No Messaging arm.  
Primary and Other Outcomes Ascertainment: Ascertainment of primary and secondary 
outcomes will occur in the same manner as in the Intervention M arm.  
 

 
 
 
 
 
 
 

Procedures and workflow during Control Arm: No Messaging, No Vaccine Acceptance 
Question Arm 
 
The workflow during this arm is identical to the Intervention Q arm, except there will be no 
Vaccine Acceptance Question Survey. The anticipated flow of the study during Control Blocks 
is summarized in Figure 3. 
 
Introduction to ED Staff:  Clinical Research Coordinators (CRCs) will set up their workstation 
in the ED and introduce themselves to ED staff (nurses, physicians and mid-levels), informing 
them that they will be doing the PROFLUVAXED study that day. They will avoid telling 
providers whether this is an intervention vs control arm.  
Initial Screening and Scripted Consent for Surveys: CRCs will review ED dashboards for 
inclusion and exclusion information. When an eligible patient is identified, the CRC will ask the 
nurse or doctor caring for the patient whether it is Ok for them to approach the patient about the 
study. For provider approved patients:  CRCs will approach eligible patients and deliver a 
scripted consent for two short surveys – the Intake Survey and the (Post-Intervention) Vaccine 
Acceptance Survey. See Scripted Consent for the No Messaging, No Vaccine Acceptance 
Question arm period. 
CRCs will complete screening and enrollment log indicating whether or not they agreed to 
participate. If they agreed to participate, the CRC will assign a Study ID#. 
Intake Survey:  We will administer the INTAKE SURVEY to participants. CRCs will have the 
option of inputting surveys to REDCap on iPads in real time or using paper surveys (and later 
inputting into REDCap). These surveys are to be delivered orally (CRC asks questions), not via 
handing them out. The Intake Surveys are the same for all three arms of the study.  
Primary and Other Outcome Ascertainment: Ascertainment of the primary outcome and 
secondary outcomes of vaccine uptake will occur in the same manner as in the Intervention 
arms. Since this group will not receive the Vaccine Acceptance Survey, the secondary outcome 
of vaccine acceptance (Aim II) is not applicable or tested in this group. 
 
 
 

 

Research staff informing ED providers when participants will accept Influenza vaccine 
for Intervention M and Intervention Q arms 
Our study site EDs will usually have the capability of administering Influenza vaccines, and we 
expect that this will continue for the duration of the trial. The last question in the Vaccine 
Acceptance Survey in both the Intervention M and Intervention Q arms of the study is 
“Would you accept the flu vaccine in the emergency department today if your doctor asked 
you?” When a participant says they will accept the vaccine, the CRC or research staff will ask 
the participant if it is OK to notify that participant’s ED provider(s) – nurse and/or physician that 
they said they will accept the vaccine and confirm whether or not they receive it in the ED. 
Research staff will not tell patients that they qualify for the flu vaccine and will not advise them in 
any manner. When participants agree to notification of the ED provider, research staff will notify 
the ED provider that they stated they will accept the vaccine. They will not tell the provider that 
they meet criteria for the influenza vaccine and will not push that they get vaccinated. 

Primary Outcome and Ascertainment of Specific Aim I  
Our primary outcome for Specific Aim I is influenza vaccination uptake within 30-days of the 
index ED visit, comparing the Intervention M arm vs the Control (No Messaging and No 
Vaccine Acceptance Question) arm. Primary outcome ascertainment will occur by the 
mechanisms described above. Staff will check the EHR to confirm receipt (or non-receipt) of an 
influenza vaccine at 30 days and conduct follow-up phone calls for those who have not received 
the vaccine per record review. This review will be conducted in a blinded fashion – the research 
staff person reviewing the EHR will be unaware of participant’s study group assignment. 
Participants who have confirmed receipt by EHR review or follow-up phone call will be deemed 
“vaccinated at 30 days”. Conversely, participants who do not have confirmed receipt by review 
or phone call will be deemed to be “not vaccinated at 30 days” 
Our secondary outcomes include 30-day influenza vaccine uptake, comparing the 
Intervention Q vs Control arms and the Intervention M vs Intervention Q arms. These 
outcomes will be ascertained and designated in the same manner as the primary outcome. 
Other secondary outcomes include influenza vaccine uptake during their index ED visit, 
comparing all 3 permutations of the arms above (M vs Control, Q vs Control and M vs Q) . For 
ascertainment of these outcomes, we will ask participants and their providers whether they 
received the influenza vaccine in the ED above by direct questioning and conduct blinded 
review of EHR records the working day after their index ED visit. 
Outcome and Ascertainment of Specific Aim II  
Our outcome for Specific Aim II is Influenza vaccine acceptance (defined as a response of “yes” 
to the question “Would you accept the flu vaccine in the emergency department today if your 
doctor asked you?”), comparing the Intervention M arm vs Intervention Q arm. This outcome will 
be ascertained during the Vaccine Acceptance Survey in both arms.  
 
 
 
 

30 Day phone and EHR Follow-Up  
All participants in this study will have given consent for EHR review – this is a requirement of 
study enrollment. CRCs will check the EHR at 2 time periods – the workday after their index visit 
and, if not vaccinated during their ED visit, again 30 days after their visit. CRCs will review 
Master Data Flow daily (workdays) to determine which subjects have reached the 1-month 
follow-up time period. By convention, we will use the next month’s day that has the same 
number as the index study visit date, i.e., if the study index visit was on November 5, then the 1-
month follow-up should occur on December 5. If December 5 falls on a weekend, then the CRC 
will use the next study workday (typically the next Monday) as the follow-up date. Study 
subject’s medical record #s and telephone #s will be accessed from the Flu Vaccine Follow up 
sheet. The CRC who conducts EHR and phone follow-up will be blinded to the subject’s study 
group assignment (intervention vs control arms), i.e., a separate CRC who did not recruit at that 
site during that day will conduct this phone follow-up.  
1) The CRC will first review the EHR to determine whether there is any record of an influenza 
vaccine received in the preceding time period from the study index visit. If there is a record 
of vaccination, the CRC will record what date and where the participant received it (if 
available). See Follow up Data Collection form. 
2) If there is no record of vaccination in the EHR, the CRC will proceed with a phone call to the 
study subject. CRCs will conduct phone follow-up with study subjects who have given 
written consent for phone contact. See Follow up Phone Call Collection form. CRCs will 
enter follow up data on both the Master Data Flow and Follow up spreadsheets via REDCap 
links.  
a. If the patient does not answer the phone that morning, the CRC will place two more 
calls to the study subject over the next 2 workdays. They will vary the time of these 
calls to improve response.  
b. If the patient does not answer the phone by the third call, the CRC will leave a 
message with the phone # of the study team. No more calls will be initiated by the 
study team after this third call.  
 
Data Recording and Entry 
CRCs will record survey responses and other data via two mechanisms: 
1) Direct entry into the Flu Vaccine Study REDCap database in real time during surveys via 
secure links  
2) Recording onto paper forms first. Then entry of survey information and data after each 
participant enrollment. 
CRCs will keep a running log of all study flow and enrollment, recording the following data for all 
patients approached: study date, study arm, “Yes” and “No” agreeing to surveys, delivery or 
non-receipt of messaging platforms, agreeing to receipt of study vaccines, receipt of vaccines in 
the ED, “Yes” and “No” agreeing to follow-up calls and EHR review.  See Master Data Flow.  
 
 
 

Data Analysis 
Summary of Analysis Plan Change 
Our Specific Aim I and primary outcome (to determine whether implementation of influenza 
vaccine trusted messaging platforms is associated with increased influenza vaccine uptake in 
unvaccinated ED patients) and our other two specific aims remained the same throughout the 
original design, implementation and analysis stages of the study. The only change that occurred 
was the change of the primary outcome time horizon for influenza vaccine uptake from 1-day 
(during index emergency department visit) to 30-day uptake. This change occurred in the first 
month of study enrollment in response to our analysis and editorial reviews of our other 
preceding study of COVID-19 vaccine messaging platforms, which revealed that 30-day uptake 
was the more relevant primary outcome time horizon: 
https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2799590 
 
 
Rodriguez RM, Nichol G, Eucker SA, Chang AM, O'Laughlin KN, Pauley A, Rising KL, Eswaran 
V, Morse D, Li C, Patel A, Duber HC, Arreguin M, Shughart L, Glidden D; PROCOVAXED Study 
Network. Effect of COVID-19 Vaccine Messaging Platforms in Emergency Departments on 
Vaccine Acceptance and Uptake: A Cluster Randomized Clinical Trial. JAMA Intern Med. 2023 
Feb 1;183(2):115-123. doi: 10.1001/jamainternmed.2022.5909. PMID: 36574256; PMCID: 
PMC9856883 
Primary Analysis for Specific Aim I 
The primary study comparison is uptake (receipt) of an influenza vaccine within 30 days of the 
index ED visit, comparing participants seen on Intervention M dates with those seen on Control 
(No messaging, no vaccine acceptance question) dates to test our study hypothesis: 
Implementation of PROFLUVAXED trusted messaging platforms in EDs will be associated with 
increased Influenza vaccine uptake in unvaccinated ED patients.  
There are multiple secondary analysis comparisons for Specific Aims I and III: 
1) Influenza vaccination uptake within 30 days, comparing the Intervention Q arm vs the 
Control arm and comparing the Intervention M arm vs the Intervention Q arm. 
2) Receipt of an influenza vaccine during the index ED visit, comparing all 3 permutations of 
the study arms (M vs Control, Q vs Control, and M vs Q). 
 
Outcomes will be compared using mixed logistic regression with a fixed effect for randomization 
assignment, a normally distributed random effect to allow for clustering by enrolling center, and 
restricted cubic splines to allow for secular trends during the study period. The treatment effects 
will be tested by the coefficient for the fixed effect of study arm along with 95% confidence 
intervals. 
 
Analysis for Specific Aim II 
For Specific Aim II, we will compare outcomes in participants seen on Intervention M arm dates 
with those seen on Intervention Q arm dates to test our study hypothesis: Implementation of 

PROFLUVAXED trusted messaging platforms in EDs will be associated with increased 
influenza vaccine acceptance in unvaccinated ED patients. 
The outcome is Influenza vaccine acceptance (defined as a response of “yes” to the question 
“Would you accept the flu vaccine in the emergency department today if your doctor asked 
you?”).  This outcome will be compared using mixed logistic regression with a fixed effect for 
randomization assignment, a normally distributed random effect to allow for clustering by 
enrolling center, and restricted cubic splines to allow for secular trends during the study period. 
The treatment effects will be tested by the coefficient for the fixed effect of study arm along with 
95% confidence intervals. 
 
 
Subgroup Analyses 
Another focus of this research is on ED patients who lack primary care group, defined on the 
Intake survey question: “Do you have a regular clinic or doctor for medical care?” We will 
analyze outcomes according to the binary indicator of having primary care – yes vs no (and 
unsure). 
We will additionally stratify outcomes by study site (representing different regions of the country 
and different communities), age, gender, primary language, and race/ethnicity. 
Subgroups will be tested by adding a subgroup by intervention interaction to the mixed logistic 
regression. A subgroup will be considered significant if the pairwise intervention by subgroup 
omnibus test is significant at the 0.05 level. 
Rationale for time (1-day unit) cluster and consideration of Alternative Study Designs:  
In the study Overview, we described our rationale for switching from a one-week unit to a one-
day unit cluster. Our primary goal with this research is to determine whether real-world 
implementation of influenza vaccine messaging as an ED-site level intervention results in 
greater acceptance and uptake of influenza vaccines in vulnerable ED populations. Each site 
sees approximately 150-250 patients per day and applying or not applying the intervention 
(delivery of influenza vaccine messaging) for individual patients in this high workflow, rapid 
patient turnover ED environment is simply impractical. Given that influenza vaccine messaging 
may be seen and received by all patients non-selectively in the EDs, patient level randomization 
would result in high risk of cross-contamination between intervention and control arms. 
Therefore, removal of the messaging intervention from the site completely during specified time 
periods (1-day units) of Intervention Q, and removal of both interventions (messaging and the 
vaccine acceptance question) during Control days is the optimal approach. Although single 
switches of turning on the interventions at each site (i.e., stepped-wedge trial design) is easier 
to enact, changes in general population attitudes over time introduce bias that limit the validity of 
this trial method. We expect gradually increasing acceptance and uptake of the influenza 
vaccine over time, which would introduce substantial bias toward the intervention. Finally, cost 
considerations and feasibility limit the number of sites in this trial, negating the potential 
advantages of a cross-over trial with randomization by sites. These practical and methodological 

benefits of the 1-day unit cluster RCT far outweigh the smaller sample size and easier analysis 
with an individual patient unit RCT or a stepped-wedge design. 
Statistical approach: In terms of statistical approach, this is a superiority trial in which we seek 
to verify our central study hypothesis that provision of flu vaccine messaging will result in 
greater acceptance and uptake of the flu vaccine. Following the recommendations of Hussey 
and Hughes, our statistical analyses will focus on comparing the vaccine uptake rates during the 
time periods when influenza vaccine messaging is in place (Intervention Q) and when the 
system is not in place (Control - usual care) using mixed effects logistic models. The outcome of 
interest is the binary indicators of whether they have received a flu vaccine in the ED (uptake - 
yes/no). Models will include a normally distributed random center effect (on the logit scale) to 
accommodate potential within-center characteristics (e.g., case mix, demographics), as well as 
terms for time and intervention. Hypotheses testing will focus on the statistical significance of 
the intervention indicator. We will fit the mixed effects models using maximum likelihood and 
routines in Stata.  
We will test our primary hypotheses and analyze outcomes according to the study arm (index 
visit in Intervention Q day vs Control day) to which patients were allocated, regardless of 
whether they received Influenza vaccine messaging or not - intention to treat analysis.  
In addition to the effects on total vaccine acceptance, we will also examine the effect of 
Influenza vaccine messaging on acceptance in patient sub-groups, especially African American 
and Latinx persons. Influenza vaccine messaging may work for one patient sub-group and not 
others - these additional analyses will guide future directions and modifications of influenza 
vaccine messaging.   
Sample Size Considerations 
The sample size calculations for this research are governed by hypothesis testing of Specific 
Aim I -- Implementation of influenza messaging platforms will be associated with increased 
influenza vaccine uptake in unvaccinated ED patients. Considering the high benefit of 
increasing vaccine uptake and the negligible risk of the intervention (a trusted messaging 
program), even a small effect size of increased uptake would be a clinically important difference. 
By investigator consensus, we have determined that the intervention would be clinically useful if 
it increased influenza vaccine uptake by 7%.  
We base the sample size calculation on the comparison of the proportion of patients who 
receive the vaccine between the Intervention M and Control time periods using standard 
formulae for individual randomization. We have verified that these sample sizes are 
conservative by simulation of data using a mixed random effects model. Our baseline level of 
vaccine uptake in the Control arm is estimated to be approximately 5%. With this uptake level 
we find that at an alpha=0.05 level and a power of at least 0.80, we will need to enroll 744 
participants (248 in each arm) in the study to detect the difference of interest (a setting in 
which the vaccine acceptance rate will increase by 7% during Intervention periods vs Control 2 
periods).  
We must emphasize that this is a pragmatic trial intended to be completed during a 
single influenza vaccination season. Enrollment in this study will simply not make sense after 
March 2023. Carry-over of the study into a second influenza vaccination season (7 months 
later) would introduce significant confounders (differences in perceptions about influenza 
vaccines from year to year, influence of different stages of the COVID-19 pandemic). We 

therefore will terminate the study at the end of the influenza vaccination season (as 
determined by the PI in consultation with the San Francisco [and other] Departments of 
Public Health), regardless of enrollment numbers.   
 
 
 

Sample Size for Specific Aim II 
 
From our previous work in the PROCOVAXED study, we expect a baseline vaccine rate (Yes 
answer to “Would you accept the flu vaccine in the emergency department today if your doctor 
asked you?”) of approximately 10%. With the same other assumptions of Specific Aim I (an 
alpha=0.05 level and a power of at least 0.80), we will need to enroll 1119 participants (373 in 
each arm) in the study to detect the difference of interest (a setting in which the Influenza 
vaccine acceptance rate will increase by 7% during Intervention M periods vs Intervention Q 
periods.  
As per above, however, this is a pragmatic trial with a sample size that is governed by Specific 
Aim I and the intention to complete the study during a single influenza vaccination season. We 
will terminate the study at the end of the influenza vaccination season (as determined by the PI 
in consultation with the San Francisco [and other] Departments of Public Health), regardless of 
enrollment numbers.   
Sample Size for Specific Aim III 
 
Under the same assumptions of Specific Aim I (baseline uptake rate of 5%, an alpha=0.05 level 
and a power of at least 0.80), we will need to enroll 744 participants (238 in each arm) in the 
study to detect the difference of interest (a setting in which the Influenza vaccine uptake rate in 
the ED will increase by 7% during Intervention Q periods vs Control periods.  
As per above, however, this is a pragmatic trial with a sample size that is governed by Specific 
Aim I and the intention to complete the study during a single influenza vaccination season. We 
will terminate the study at the end of the influenza vaccination season (as determined by the PI 
in consultation with the San Francisco [and other] Departments of Public Health), regardless of 
enrollment numbers.   
Data management plan 
We will manage data using REDCap, hosted by the core site (UCSF) for secure data entry and 
management. Patient identifiers (medical record numbers and phone numbers) only link will be 
to unique study ID numbers, which will be housed in files that are kept separate from other 
study data. We will develop a detailed data dictionary to ensure consistent standards across 
sites. We will reduce missing or erroneous data using the REDcap data quality tool. 
 
 

Consents and Rationale 
 
We will obtain scripted verbal consent for Intake surveys in the same manner that we have 
conducted with the PROCOVAXED study, which is nearly identical in design and scope. We will 
obtain written HIPAA consent for EHR review for all patients and separate written consent for 
30-day follow-up phone calls.  
With regards to consent for delivery of the messaging intervention, we must emphasize that 
messaging for vaccine hesitancy is firmly a part of standard best-practice emergency 
department care (messaging of this type is currently be enacted in EDs across the US). Delivery 
of the vaccine messaging platforms is therefore an accepted common best practice not 
requiring consent. To add an extra layer of consent could lead to even greater vaccine 
hesitancy. We therefore plan the following processes with verbal assent for the Intervention: 
1) At the end of the Intake Survey, asking patients if they are willing to watch an influenza 
vaccine messaging video(s). If the patient says Yes, then we will play the video. If the 
patient says No, we will not play the video.  
2) Asking participants whether they are willing to read an informational flyer about Influenza 
vaccines. If the patient says Yes, then we will give them the glyer. If the patient says No, we 
will not give them the flyer. 
3) Handing the participant’s ED provider(s) the scripted message about influenza vaccines to 
deliver to the participant. Research staff will not mandate or check with providers whether 
they deliver the message.