VAX-MOM COVID-19: Increasing Maternal COVID-19 Vaccination
Phase: NA
Conditions: Immunization; Infection, Pregnancy Related, COVID-19
Sponsor: University of Rochester

STUDY00007717 
Page 1 of 30           
Version Date: 2.4.25 
 
Official Title: VAX-MOM COVID-19: Increasing Maternal COVID-19 Vaccination 
(Supplemental Award to “The VAX-MOM Study: Increasing Influenza and Tdap Vaccination of 
Pregnant Women”)  
 
NCT #: NCT05570630 
 
Protocol Document Date: Version 2/4/25 (the most recently approved protocol version for the 
study) 
 
PLEASE NOTE: This protocol has sentences/sections redacted to maintain the confidentiality 
of participating individuals and practice sites. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

STUDY00007717 
Page 2 of 30           
Version Date: 2.4.25 
 
VAX-MOM COVID-19: Protocol for Phase 2 
 
Lead Principal Investigator (URMC Site) – Cynthia Rand, MD, MPH 
Investigator (UCLA Site) – Peter Szilagyi, MD, MPH 
Investigator (RRH Site) – Elizabeth Westen, MD   
 
 
NOTE: The VAX-MOM COVID-19 study is divided into 2 main phases, baseline and 
intervention. The baseline phase [STUDY00007624: VAX-MOM: COVID-19 (Phase 1)] was 
previously submitted and approved independently through each participating health system’s 
regulatory board.  
 
The current protocol covers the intervention phase (Phase 2), which is being submitted for 
single IRB review, with the University of Rochester RSRB acting as the IRB of record for all 
study sites. 
 
 
1. PURPOSE OF STUDY  
 
COVID-19 infection during pregnancy is associated with increased risk of pre-eclampsia, 
preterm birth and stillbirth.1 Pregnant women with COVID-19 have a higher rate of ICU 
admission and intubation than those who are not pregnant.2 COVID-19 vaccine is 
recommended before pregnancy and during pregnancy to decrease the risk of severe illness 
and death. Pregnant women should also receive a booster dose of COVID-19 vaccine after 
the original series. A 2022 study showed the effectiveness of maternal vaccination against 
hospitalization for COVID-19 among infants was 52% overall, 80% during the delta period, 
and 38% during the omicron period.3 Importantly, studies show that the vaccine is safe and 
not associated with any negative maternal or infant outcomes.4 
 
Despite the benefits of COVID-19 vaccination, only 71% of pregnant women were 
vaccinated for COVID-19 as of June 2022 (most prior to pregnancy), with a much lower rate 
of 58% among non-Hispanic Black women.5 Additionally, only 56% of pregnant women 
overall have received a booster, but only 39% of Black pregnant women have received a 
COVID-19 booster vaccine. An effective intervention is needed to improve COVID 
vaccination rates for pregnant women overall, and particularly for Black women. 
 
Using participating OB/GYN offices affiliated with 3 health systems in NY and CA, the 
VAX-MOM COVID-19 intervention phase will implement a quality improvement 
initiative aimed at increasing maternal COVID-19 vaccination rates at participating 
sites. 
 
2. BACKGROUND AND RATIONALE  
 
Burden of COVID-19 
Pregnant women are at risk of significant morbidity and mortality from COVID-19 
infection, compared to the general population. Complications include increased need for ICU 
care, need for extracorporeal membrane oxygenation, prolonged ventilation, and death.2 
 

STUDY00007717 
Page 3 of 30           
Version Date: 2.4.25 
 
Effectiveness of maternal COVID-19 vaccination  
Maternal IgG raised by vaccination during pregnancy crosses the placenta to the infant, 
and remains detectable in more than half of infants at 6 months of age.  Early estimates 
suggest that vaccinating pregnant women after 20 weeks is 80% effective, and before 20 
weeks is 32% effective at preventing hospitalization of infants younger than 6 months with 
COVID-19.6   
 
Barriers to Vaccination 
There are many barriers to COVID-19 vaccination for pregnant women. Patient Barriers 
include vaccine hesitancy, lack of knowledge of the benefits and risks of COVID-19 vaccine 
or protection for the baby, concerns about vaccine safety generally and during pregnancy due 
to misinformation, and low perceived susceptibility to infection.7,8 Provider Barriers likely 
include missed opportunities for vaccination, lack of a vaccine recommendation or weak 
endorsements by health providers9-12 and various suboptimal practice operations.13-15 Since 
most women vaccinated during pregnancy receive their immunizations from their 
OB/GYN,16 it is critical for OBs to discuss, recommend, and offer immunizations. System 
Barriers include lack of audit and feedback, and lack of a standing orders as well as 
challenges with vaccine purchase and storage.11,17 
 
The VAX-MOM COVID-19 study will ultimately address multiple components by 
focusing on provider and staff communication, workflow optimization and vaccine rate 
feedback to overcome these barriers. 
 
3. ADMINISTRATIVE ORGANIZATION   
 
Project staff are affiliated with three participating health systems across two states. In 
New York, the University of Rochester Medical Center (URMC) and Rochester Regional 
Health (RRH), and in California, the University of California, Los Angeles (UCLA) (see 
Table 1).  
 
Table 1: Project Sites and Leaders/Team Members 
State 
NY 
CA 
State-Level Leader 
Cynthia Rand, MD, MPH  
(Lead PI) 
Peter Szilagyi, MD, MPH (Site PI) 
Team Members 
Site PI, Consultant, OB/GYN 
liaisons, Study Coordinator, HSRC 
Site PI, Consultant, OB/GYN liaison, 
Data Specialists 
Health System 
URMC 
RRH 
UCLA 
# of Participating 
OB/GYN Practices  
7 
13 
7 
 
 
 
 

STUDY00007717 
Page 4 of 30           
Version Date: 2.4.25 
 
PRINCIPAL INVESTIGATOR (PI) OVERSIGHT PLAN 
 
Cynthia Rand (PI) will oversee all project activities, which will be identical across the 
participating URMC, RRH and UCLA study locations. All sites will utilize the same site 
communication methods, training materials, study measures, feedback tools, online 
dissemination platforms, etc. Additionally, all sites will follow the same study design, 
procedures and timeline. 
 
Study staff will be grouped into distinct administrative teams, and each team will focus 
on defined project tasks. Team details and staff descriptions are presented below.  
 
Table 2: Administrative Team Organization & Responsibilities  
Core Leadership Team (both NY & CA combined) 
Joint operations of the study, strategic planning, procedural 
and study design decisions 
(Weekly joint meetings across states) 
C Rand,* P Szilagyi*, S Humiston, C Albertin, R Bender, A 
Schrader, S Vangala,  
State-level Teams (separate teams in NY & CA) 
Daily operations of the study 
(As needed within each state) 
NY: C Rand,* R Bender, A Schrader 
CA: P Szilagyi,* S Vangala, C Albertin, R Bender, A 
Schrader 
Practice Communication & Coordination Team 
Practice-level contacts, communications, surveys/interviews 
(As needed) 
 C Rand,* P Szilagyi,* C Olsen-Chen, C Dwyer, J Hsu, R 
Bender, A Schrader, S Humiston 
Data Analysis Team:  
Randomization, EHR data analyses, data management  
C Rand,* P Szilagyi,*  S Vangala 
*Team leader 
Principal Investigators and Consultants:  
Cynthia Rand, MD, MPH (Lead PI), an expert on adolescent immunization delivery, 
adolescent preventive services, QI and informatics, will lead the study. She is a practicing 
pediatrician in a large urban practice where she leads QI activities and is an experienced 
health services researcher. She is PI of the original VAX-MOM study aimed at increasing flu 
and Tdap vaccination for pregnant individuals. Peter Szilagyi, MD, MPH (Site PI), Professor 
of Pediatrics (UCLA), has led one of the most impactful US immunization delivery teams for 
25 years. He studies barriers to vaccination and leads research trials aimed at reducing missed 
vaccine opportunities, and serves as site PI for the original VAX-MOM study in CA. 
Elizabeth Westen, MD (Site PI), is an OB/GYN affiliated with the Rochester Regional Heath 
(RRH) system whose research interests include maternal vaccination. Christina Albertin, 
BSN, MPH, who has worked with both Drs. Szilagyi and Rand for over a decade as a senior 
health project coordinator, will act as a project consultant. Sharon Humiston, MD, MPH, a 
practicing pediatrician and expert on vaccine communication and qualitative research, will 
help lead the qualitative and educational components of the project. 
 
Additional NY Study Team Members: 
Courtney Olson-Chen, MD (URMC liaison) is an OB/GYN and Director of Research, 
URMC Division of Maternal-Fetal Medicine. Robin Bender, LMSW and Ann Schrader, MS 

STUDY00007717 
Page 5 of 30           
Version Date: 2.4.25 
 
will act as the study coordinators for all participating health systems and sites. 
 
Additional CA Study Team Members: 
 Sitaram Vangala, MS, Principal Statistician and Associate Director of DOMSTAT, will 
assist with data transfer and perform analyses for the study.  Jessica Hsu, MD (UCLA 
liaison) is a practicing OB/GYN at UCLA who directs the department’s QI efforts. 
 
Cynthia Rand, study PI, will ensure study and regulatory compliance, by: 
• Participating in and/or remaining updated regarding routine prescheduled 
meetings with project investigators and project staff for the duration of the study, 
including: 
o Weekly Core Leadership Team Meetings (State-level investigators, study 
coordinator, study consultants, data specialist, support staff): discussion of 
global project goals, timeline, regulatory requirements and adherence, 
protocol requirements and adherence, measure development, e-learning 
content, data analysis, workflow details, etc. 
o Weekly State-Level Team Meetings (State-level investigator, study 
coordinator, support staff, data specialist and/or consultant as needed): 
discussion of day-to-day operations needed to support global study goals 
(including protocol and regulatory items). 
o As-Needed Health System Liaison Meetings (State-level investigator, study 
coordinator, OB/GYN site liaisons): discussion of provider/nurse/staff 
workflow, development of study measures specific to 
providers/nurses/staff, optimal measure dissemination procedures, 
identification of possible vaccine champions. 
o As-Needed Data Development & Analysis Meetings (State-level 
investigators, study coordinator, EHR report builders, data specialists, 
consultants as needed): discussion of data collection methods, EHR data 
extraction methods, data analysis, data storage/management.  
• Keeping all study personnel abreast of prearranged study changes and subsequent 
regulatory determinations (e.g., changes to study measures, workflow procedures, 
study personnel, etc.) via a predetermined communication chain: 
o Core Leadership Team discusses necessary study changeStudy 
coordinator submits amendment via sIRB IRB notifies Study PI and 
study coordinator of regulatory determination Study PI and coordinator 
notify Core Leadership Team As appropriate, study coordinator will 
further notify local IRBs and site-level personnel and/or data specialists 
o Communication with study site personnel will most often take place 
during prescheduled meeting times, but in more urgent cases, will take 
place as soon as needed. 

STUDY00007717 
Page 6 of 30           
Version Date: 2.4.25 
 
o Communication will be both verbal (during or outside of weekly meetings 
depending upon level of urgency) and written as needed (via e-mail, e-
mail attachments, or the secure shared Box folder).  
• Keeping all study personnel abreast of unforeseen study events and subsequent 
regulatory determinations (e.g., protocol deviations, breaches in confidentiality, 
subject/site withdrawal, etc.) via a predetermined communication chain: 
o Study personnel reports study event to PI PI notifies study 
coordinator PI and study coordinator notify IRB, as well as other study 
personnel as appropriate any action requested by IRB is communicated 
to Core Leadership Team As appropriate, study coordinator will further 
notify site-level personnel and/or data specialists 
o As described above, communication will be both verbal and written. 
o Also see section #16 “Data & Safety Monitoring Plan” 
4. STUDY DESIGN & PROCEDURES 
 
Brief Overview of Phase 2:  
Using a clustered RCT (randomizing practices), we will allocate half of the practices within each 
health system to the VAX-MOM COVID-19 intervention and the other half to standard of care 
(control). We will measure the impact of the intervention on vaccination rates (primary 
outcomes) as well as rates of vaccination by subgroup (secondary outcomes) (see Table 4). 
Finally, if VAX-MOM COVID-19 is successful, we will provide the control practices with core 
intervention materials (e.g., learning module, rate feedback template) at the conclusion of the 
study.  
 
Design and Procedural Details of Phase 2: (also see Figure 1 for Timeline) 
 
Step 1) Deliver VAX-MOM COVID-19 training to randomized practices: 
 
Randomization Details:  
 
Practices will be the unit of randomization. We will assign practices using a covariate 
constrained randomization strategy. Practices will be stratified by health system, as well 
as arm assignment in the primary VAX-MOM trial. Within strata, we will perform 
constrained randomization to allocate practices to intervention/control arms, and ensure 
each arm has similar baseline COVID vaccination rates, percent of patients covered by 
Medicaid, number of OB providers, and number of patients. Specifically, we will use 
these variables to construct and evaluate a balance criterion, which is the sum of the 
squared difference between standardized group means on these variables. We will 
generate all possible combinations of eligible practices in 2 arms (using a SAS macro), 
and define an acceptable set of randomizations that result in balanced variables (generally 
the lowest 10% on the balance criterion). From this set of randomizations, we will select 
one set at random, and then randomly assign each practice to intervention vs control 
group. 

STUDY00007717 
Page 7 of 30           
Version Date: 2.4.25 
 
Intervention Details: 
 
Once randomization has been completed, and “intervention group” practices have been 
identified, multiple quality improvement (QI) techniques will be aimed at practice 
providers, nurses, and staff.  
 
• Identification of a “vaccine champion” at each intervention site: At each site, 
the affiliated medical director or health system liaison will select one “vaccine 
champion” for each practice. Each identified vaccine champion will be e-mailed 
an Information Sheet (see “InfoSheet_VaccineChampion…” document) by the 
study coordinator, outlining the basic study goals as well as their involvement in 
tasks throughout the training and intervention adoption phases of the study (see 
“Consent Process” section). The position of vaccine champion is voluntary, and if 
they do not wish to accept the role, they may decline and another individual will 
be identified as a replacement. The vaccine champion will play a lead role 
throughout the study intervention, including: 
o assisting with the dissemination of e-learning training modules and the 
scheduling of follow-up site-specific meetings 
o acting to ensure training content is being appropriately adopted into 
practice workflow procedures 
o fielding questions from site personnel regarding the logistics of study 
interventions  
o acting as an ongoing liaison between the site personnel and study staff 
o leading regular discussions regarding immunization rate feedback with site 
personnel (see step 2 for further detail) 
o planning for and completing monthly PDSA cycles (see step 2 for further 
detail) 
o monthly completion of the “Practice Time/Cost Survey” (see step 4 for 
further detail) 
o participating in monthly learning collaborative meetings with the study 
team 
 
• Delivery/discussion of an online training module: Each site will receive 
training focused on four main areas of content: a) the importance of COVID-19 
vaccination during pregnancy, b) vaccine communication, c) optimization of 
workflow and d) review and resources (see Table 3). This training will be 
delivered first via an online e-learning platform, and next via an in-person or 
virtual site meeting. 
 
 
 
 
 
 

STUDY00007717 
Page 8 of 30           
Version Date: 2.4.25 
 
Table 3: Training Module Content  
General Topic: 
Training Lessons: 
Importance of 
COVID-19 
vaccination during 
pregnancy 
1- How bad is COVID-19 disease during pregnancy? 
2- Why does ACOG recommend vaccination? 
3- What are ACOG’s COVID-19 vaccination recommendations? 
Streamlining vaccine 
communication 
4- A range of patient concern levels 
5- For everyone: An evidence-based recommendation 
6- For hesitant patients: Motivational interviewing 
7- For patients who refuse at this visit 
Optimizing workflow 
to increase 
vaccination 
8- Strategies for increasing immunization rates 
9- Putting the strategies into action 
Review and resources 
10- Test your knowledge 
11- Provider & Patient Resources 
 
o E-learning module dissemination: Module lessons will first be 
disseminated to practice site personnel via an e-mail link taking them to 
the Articulate/Rise 360 platform. The study coordinator will obtain e-mail 
addresses for the site personnel from either the health system liaison or the 
associated vaccine champion, and will then e-mail a training content link 
to all appropriate site providers, nurses, and staff. Attached to this initial e-
mail will be an Information Sheet describing the study goals and training 
phase details, as well as their involvement throughout the duration of the 
6-month QI intervention (see “IntroEmail_TrainingPhase…” and 
“InfoSheet_TrainingPhase…” documents; also see “Consent Process” 
section for more detail). After reviewing the Information Sheet and 
discussing any questions/concerns with study staff, site personnel will be 
asked to access and complete the training lessons prior to their scheduled 
site meeting.  
 
Embedded at the conclusion of the online training module, will be a link to 
a brief REDCap survey (see “TrainingModuleCompletionSurvey…” 
document) to obtain information regarding those who have fully 

STUDY00007717 
Page 9 of 30           
Version Date: 2.4.25 
 
completed the training. A list of practice personnel who have successfully 
completed the online training will be stored in a password-protected Excel 
file by the study coordinator. The names will not be linked with any other 
study data. The list will be accessible only by study staff for the purposes 
of confirming those who should receive MOC/CME/CNE credit for their 
participation and to calculate overall completion rates for each site.  
 
o Site meeting: Following the online training, all site personnel will meet 
with study staff either in-person or via a virtual Zoom meeting. The 
meetings (both virtual and in-person) will be audio and video recorded 
using a laptop or handheld recording device (in-person meetings) or via 
the Zoom recording option (virtual meetings) (see “Audio/Video 
Recording” and “Privacy and Confidentiality” sections for more detail). 
These archived video files will allow for the review of meeting content by 
site personnel who were unable to attend the meeting live, or by those who 
simply wish to review the meeting discussion for a second time. 
 
During the site meeting, study staff will briefly review online training 
content, allow time for the sharing of comments/questions from site 
personnel, and lead a discussion regarding the application of training 
content ideas specifically to that practice site. During this meeting, study 
staff will also present site personnel with their current immunization rates. 
Each designated vaccine champion will track meeting attendance or 
participation will be tracked via the Zoom “participant list.” The study 
coordinator will store this roster in a password-protected Excel file. The 
names will not be linked with any other study data. This list will be 
accessed for the purposes of confirming those who should receive MOC 
credit for their participation in the project, and for calculating overall 
attendance rates for each site. 
 
o MOC/CME/CNE credit: To encourage the completion of online trainings 
and attendance at site meetings, providers and nurses will be offered 
CME/CNE credit for completing the online learning module, and 
providers will be offered MOC/QI credit for attending office systems 
change meetings and reviewing practice rates.  
 
Step 2) Implement intervention ideas and monitor progress at intervention practices: 
 
To monitor the adoption of VAX-MOM COVID-19 training content (changes in 
communication techniques, workflow optimization, etc.) at each practice site, each 
vaccine champion will participate in the following activities: 
 
1. Lead a discussion regarding immunization rate feedback with practice providers, 
nurses, and staff on a monthly basis. The site-specific vaccination information 
will be obtained from either: 1) the EHR report builders for each health system, 
or 2) manual chart review (i.e., accessing eRecord to look up patient charts and 

STUDY00007717 
Page 10 of 30           
Version Date: 2.4.25 
 
entering information into a REDCap data entry template) completed by study 
staff or the site’s vaccine champion. Obtained vaccination information will be 
uploaded to the secure UCLA Health Box folder. The project’s data analyst will 
plug this vaccination information into a template that will create visual graphs 
and tables, allowing practice personnel to better understand trends within their 
site. The study coordinator will then disseminate the visual vaccination rate 
feedback to each site for review and discussion (see “RateFeedbackTemplate…” 
document).  
 
2. Develop PDSA cycle goals on a monthly basis utilizing knowledge about current 
workflow efficiency and data from vaccine rate reports. Vaccine champions will 
utilize the PDSA template (see “PDSAandTimeCostSurvey…” document) to 
guide progress with their respective teams. The template will help them decide 
upon specific intervention activities appropriate for their setting, track those 
involved with each activity, compare results from the activity to previous 
performance, and focus on changes that may need implementation during future 
cycles. PDSA cycle logs will be sent to vaccine champions for completion via a 
secure REDCap survey. 
 
For participating NY practice sites, the 6-month intervention (entailing PDSA 
cycles described above) will be extended by 3-5 months (depending upon the 
original start date of the specific location) until April of 2024, to optimize 
vaccine intervention efforts and data collection during the peak of the COVID 
season. 
 
During the 6+-month intervention phase, a study team member may observe 
intervention sites in-person in order to collect qualitative data regarding 
workflow and communication techniques as described in the sites’ submitted 
PDSA cycle logs. Specifically, the study team member will document the ways 
in which the targeted intervention techniques are implemented within the 
practice (see “InPersonObservationNotes…” document). Observation notes will 
then be compared with submitted PDSA cycle log information and analyzed for 
consistency.  
 
Of Note: Although study staff will observe OB/GYN practice personnel during 
patient encounters, practice patients will NOT be the target of the observation, 
and no identifying patient information will be collected. Rather, the focus of 
observation will be the select practice personnel only. Any patient-level 
information recorded during the observation will be done so in a generic manner 
such that the patient could not be identified on the basis of the notes (e.g., 
“…when patient indicated that they did not want the COVID-19 vaccine, the 
nurse stated, ‘The doctor may talk to you more about that when she comes in.’”).  
 
3. Complete the “Practice Time/Cost Survey” on a monthly basis (see step 4 for 
further detail). The vaccine champion will track site personnel attending each rate 
feedback and PDSA discussion as well as their role (e.g., Ob/Gyn, nurse, desk 

STUDY00007717 
Page 11 of 30           
Version Date: 2.4.25 
 
staff, etc.) within the practice.  They will then complete a monthly REDCap 
survey reporting aggregate information regarding total time spent on project-
specific activity (see“PDSAandTimeCostSurvey…” document).  
 
Additionally, to monitor study staff efforts during this phase (to parallel surveys 
completed by practice champions), study staff will participate in the completion 
of a weekly “Study Staff Time Survey” (see step 4 for further detail). 
 
Again, for participating NY practice sites, the 6-month intervention (including 
the Practice Time/Cost Survey and Study Staff Time Survey described above) 
will be extended by 3-5 months depending upon the specific location, to 
optimize vaccine intervention efforts and data collection during the peak of the 
COVID season. 
 
4. Participate in learning collaborative meetings on a monthly basis with the study 
team. The vaccine champions from all participating intervention sites, along with 
the study team, will meet together as a group to discuss current COVID-19 
information, intervention strategies/tips and overall project progress. 
 
 
Step 3) Compare intervention vs. control practices: 
 
Using the “RE-AIM” framework (see Table 4) we will compare intervention practices to 
control practices using multiple data sources. We will focus on one primary outcome and 
multiple secondary outcomes. 
 
Primary Outcome: Following the conclusion of phase 2 (6-month duration), we will 
assess the “Effectiveness” of the intervention by comparing intervention group COVID-
19 vaccination rates against control group COVID-19 vaccination rates. 
 
Secondary Outcomes: Following the conclusion of phase 2 (6-month duration), we will 
assess the remaining “RE-AIM” domains including “Reach,” “Adoption,” 
“Implementation,” and “Maintenance.” 
 
Table 4:  Summary of Outcome Measures and Tools 
RE-AIM 
Category 
Outcome Measure(s) 
Data Source 
Reach 
• Number of patients seen in the practice within study period. 
• Number of providers/nurses/staff completing the training 
modules. 
EHR data & 
training 
attendance 
records 

STUDY00007717 
Page 12 of 30           
Version Date: 2.4.25 
 
 
 
Details for EHR data extraction: 
 
• EHR Measure Description: We will work with health system EHR report builders 
and our experienced analysts to build EHR reports which evaluate vaccination 
coverage at participating sites (see Table 5).  
 
Baseline data was previously gathered during STUDY00007624, STUDY00005115 
and STUDY00004408. The remainder of this data (COVID intervention data), as well 
as any missing baseline data from the protocols listed above, will be gathered under 
the current protocol STUDY00007717. 
 
Effectiveness 
• Primary Outcome: Rate of COVID-19 vaccination (intervention 
vs control)  
• Secondary Outcomes:  
• 
COVID-19 vaccination rates by subgroups including (i)  
insurance groups, (ii) race/ethnicity, (iii) number of 
pregnancy, (iv) vaccine in prior year 
• 
Flu and Tdap vaccination rates by subgroups including (i)  
insurance groups, (ii) race/ethnicity, (iii) number of 
pregnancy, (iv) vaccine in prior year 
EHR data 
Adoption 
• Number and proportion of personnel involved in the VAX-MOM 
office changes (receiving MOC credit, attending in-office 
meetings) 
Training 
attendance 
records 
Implementation 
• Provider and office staff perceptions of feasibility, acceptability, 
barriers and facilitators to implementation, adherence to 
intervention, perceived time and cost, and impact on patient flow. 
• Perceived strength of vaccine recommendations. 
• Perceived adherence to staff checking whether vaccine is due. 
• Costs of implementing interventions 
 
Post-
Intervention 
Survey & 
Time/Cost 
Survey 
Maintenance 
• Sustainability 
Post-
Intervention 
Survey 
Table 5. Data obtained from EHR for women with live births during intervention period 
(NOTE: any data missing from the baseline period may also be gathered during Phase 2) 
Primary Information 
• 
COVID-19 vaccination status (both dates and type) 
• 
Flu vaccination status (dates) 
• 
Tdap vaccination status (dates) 

STUDY00007717 
Page 13 of 30           
Version Date: 2.4.25 
 
 
• EHR Measure Procedures: EHR data report builders for each health system will 
extract the requested information (data in Table 5, also see “EHRData…” document) 
using live births occurring from July 2022 until the conclusion of the study, and place 
into an excel file which will then be uploaded to a secure UCLA Health Sciences Box 
“initial report” folder for the study team.  
 
The data will be extracted for intervention and control sites both a) on a monthly 
basis, for the purposes of providing monthly rate feedback reports to intervention 
sites, and b) if needed, at the conclusion of the study to obtain any missing data from 
sites not obtained in the previous months. 
 
Initial automated report files will go through an auditing process in which a small 
percentage of patient charts will be reviewed manually (i.e., charts accessed in 
eRecord) by study staff and compared to automated report findings. Study staff 
completing this process will have received the proper Epic/EHR training and 
clearance. During the auditing process, files will be placed temporarily in secure 
“SMDNAS Research Storage” folders, to reduce the risk of altering original report 
files, or confusing audited files for originals.  
 
Once the auditing process is complete, the finalized EHR reports will be uploaded 
back into a secure UCLA Health Sciences Box “final report” folder for analysis 
and/or for creation of the monthly rate feedback form by the UCLA data specialist. 
De-identified analyzed data and/or the rate feedback reports will then be placed in a 
“analyzed data/rate feedback reports” folder on UCLA Health Box by the data 
specialist, accessible by the study team. For monthly rate feedback reports, the study 
coordinator will then disseminate the vaccination rate reports for each site to all 
corresponding vaccine champions via secure e-mail. 
 
In the event automated reports cannot be generated in a timely manner (i.e., as needed 
for monthly feedback or final analyses), study staff or vaccine champions will extract 
this same information by conducting manual chart reviews. A list of delivery MRNs 
will be passed from the health system liaison into UCLA Health Box, to identify 
appropriate patient charts that should be reviewed. Study staff will then manually 
access patient charts in eRecord for the appropriate health system and enter the 
information indicated in Table 5 into a REDCap “Manual Chart Review Tool” (i.e., a 
form into which study staff can easily enter the information needed).  Once manual 
data entry into REDCap has been completed, the information will be exported as an 
excel file and saved to the secure UCLA Health Sciences Box “final report” folder (as 
auditing will not be needed).  
Additional Information: Patient Demographics 
• 
DOB 
• 
MRN 
• 
Date of Delivery 
• 
GA at delivery 
• 
Race/ethnicity  
• 
Insurance 
• 
Language      
• 
Parity 
• 
High risk status 
• 
Other social 
determinants 
Additional Information: Practice Demographics 
• 
Site 
• 
Provider type (resident, midwife, MD) 

STUDY00007717 
Page 14 of 30           
Version Date: 2.4.25 
 
Details for Post-Intervention Survey: 
 
• Survey Measure Description: The Post-Intervention Survey (see 
“VAXMOMCOVID_PostInterventionSurvey…” document) is a survey consisting of 
Likert scale, multi-option and open-ended questions. The survey takes approximately 10 
minutes to complete. The survey assesses opinions regarding COVID-19 infection and 
vaccination, patient vaccine refusal, barriers to vaccination, vaccine workflow, RSV 
vaccine, practice culture, and basic demographic information. 
 
• Survey Measure Procedures: At the conclusion of phase 2, all providers (MD, 
residents, NPs, PAs and CNMs) and 1-2 select nurses from both the intervention and 
control sites will be asked to complete the survey. A notification e-mail (see 
“NotificationEmail…” document) may first be sent out by the health system liaison 
and/or Chair of Obstetrics and Gynecology affiliated with the appropriate health system. 
 
Surveys will be e-mailed to providers and select nurses by project staff using the secure 
web application REDCap (see “IntroEmail_PostInterventionSurvey…” document). E-
mail addresses will have previously been obtained from the health system liaison or 
vaccine champion and uploaded to the REDCap platform. After initial survey 
distribution, if the subject does not click on the survey link embedded within the e-mail, 
the REDCap platform will automatically disseminate reminder e-mails to the subject 
(identical content as initial invite e-mail). Once clicked, the survey link will first take 
potential respondents to the Survey Information Sheet for review (see “Consent Process” 
section for more details), and if they agree, will then allow them to continue to the full 
survey. Surveys will be sent in 3 “waves” in order to allow subjects from each health 
system to view their own system-specific Information Sheet (e.g., URMC subjects will 
first see the URMC Info Sheet, followed by the full survey). Subjects may answer survey 
questions at their own pace and may stop and restart at any time. At the conclusion of the 
survey, they will receive an automated confirmation message to assure them that the 
process is complete. 
 
Step 4) Evaluate cost of the VAX-MOM COVID-19 intervention: 
 
Practice Time & Cost Survey (see“PDSAandTimeCostSurvey…” document): A survey 
aimed at evaluating the amount of time and money devoted to VAX-MOM COVID-19 
intervention activities will be sent by the study coordinators via REDCap to the vaccine 
champion for each site on a monthly basis. The survey will be completed by the vaccine 
champion using their best time/cost estimates pertaining to the month prior. 
 
Study Staff Time Survey (see “StudyStaffTimeSurvey…” document): A survey aimed at 
evaluating the efforts of VAX-MOM COVID-19 study staff will be sent to study 
personnel on a weekly basis via REDCap. 
 
Final Time & Cost Analysis:  
Upon completion of phase 2 we will conduct a final time/cost analysis. Our two-fold cost 
measures are (a) total intervention cost to implement the VAX-MOM COVID-19 
program aggregated at the three health-system level under a cost analysis and (b) cost per 
additional vaccination under a subsequent cost-effectiveness analysis. 

STUDY00007717 
Page 15 of 30           
Version Date: 2.4.25 
 
Phase:
Study Item:
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
W1
W2
W3
W4
Randomization & Vaccine 
Champions Assigned and 
Consented
Online Module
Site Meetings
Rate Feedback (monthly)
PDSA Cycles
(monthly)
Practice Time & Cost Survey 
(monthly)
Learning Collaborative Meetings
(monthly)
Study Staff Time & Cost Survey 
(weekly)
Surveys: 
Provider & Nurse and Culture
EHR Data Collection
EHR Rate Analysis
Time & Cost Analysis
Step 4:
Data Analysis & 
Cost Comparison
Step 2:
Intervention
Step 3:
Post-Intervention 
Data Collection
VAX-MOM COVID Timeline
Month 8
Step 1:
Training
Month 2
Month 3
Month 4
Month 5
Month 9+
Month 6
Month 7
Month 1
Training Phase
6-month Intervention Phase
Figure 1: Timeline of Intervention Phase Procedures 
 
 
 
 
*extension of 
intervention phase 
for NY sites 

STUDY00007717 
Page 16 of 30           
Version Date: 2.4.25 
 
5. SUBJECT POPULATION 
 
For the purposes of this study, the term “subject” refers to both a) the personnel at participating 
OB/GYN practice sites and b) practice patient information (pregnant women with a live birth on 
record) gathered via electronic health record (EHR) extraction. All participating practices, 
approximate subject pool totals, and corresponding demographic information is listed in Table 6. 
The breakdown of optimal subject numbers per measure per health system is listed in Table 7. 
 
Table 6: Participating NY & CA Practice Sites and Subject Information (Total Subject Pool) 
 
NEW YORK URMC PRACTICES: 
Participating Practice 
Approx. Number of 
Providers/Nurses  
(provider & nurse subject pool) 
Approx. Annual 
Deliveries 
(patient subject pool) 
Strong Perinatal Associates (SPA) 
17 
220 
University OB/GYN (UOG) 
12 
290 
G. Wellness Obstetrics & Gynecology (GOG)  formerly WHP 
30 
600 
Community OB (COB) 
28 
320 
URMC Midwifery at Culver 
14 
210 
Highland Women's Health (HWH) (POB) 
15 
335 
URMC Midwifery at Lattimore 
14 
340 
URMC Insurance (% Private/Public/Other) 
45/55/5 
URMC Race/Ethnicity (% White/Black/Hispanic) 
 65/25/10 
NEW YORK RRH PRACTICES: 
Participating Practice 
Approx. Number of 
Providers/Nurses  
(provider & nurse subject pool) 
Approx. Annual 
Deliveries 
(patient subject pool) 
Bay Creek Midwifery 
5 
110 
Finger Lakes Geneva 
4 
240 
Park Ridge OB/GYN 
5 
240 
Parkway OB/GYN 
5 
240 
TWC at Alexander Park 
8 
180 
TWC at RGH (Portland) 
24 
460 
Unity OB/GYN at Ridgeway 
7 
300 
Unity OB/GYN at Brockport 
9 
330 
TWC at Clinton 
7 
115 
TWC at Newark 
10 
400 
Clifton Springs Hospital & Clinic  
10 
Shared delivery site 
with Newark 
UMG OB/GYN Clinton Crossings 
12 
210 
UMG OB/GYN St. Mary’s 
5 
120 
RRH Insurance (% Private/Public/Other) 
55/45/5 
RRH Race/Ethnicity (% White/Black/Hispanic) 
65/15/20 
 

STUDY00007717 
Page 17 of 30           
Version Date: 2.4.25 
 
CALIFORNIA UCLA PRACTICES: 
Participating Practice 
Approx. Number of 
Providers/Nurses  
(provider & nurse subject pool) 
Approx. Annual 
Deliveries 
(patient subject pool) 
Pasadena* 
4+ 
Not yet known 
Porter Ranch* 
2+ 
Not yet known 
Burbank* 
2+ 
Not yet known 
Santa Monica 
16 
1,100 
South Bay: Torrance 
7 
380 
Westwood  
14 
1,200 
Westlake/Thousand Oaks 
7 
200 
UCLA Insurance (% Private/Public/Other) 
Majority private 
UCLA Race/Ethnicity (% White/Black/Hispanic) 
Not yet known 
* = new site (not included in any previous VAX-MOM study) 
 
Table 7: Optimal Subject Numbers Per Measure Per Health System 
 
 
Total Subject 
Number for 
URMC 
Total Subject 
Number for 
RRH 
Total Subject 
Number for 
UCLA 
GRAND 
TOTAL 
EHR Data Reports (annual numbers) 
N= approx. 
2,300  
N= approx. 
2,950 
N= approx. 
>2,900 
N= approx. 
>8,150 
Post-Intervention Survey  
N= approx.    
100  
N= approx. 
100 
N= approx.     
45 
N= approx. 
235 
 
Inclusion of women and minorities 
Every effort will be made within this project to ensure that women, racial and ethnic 
minorities will be included in all aspects of the research. Practices selected include a broad 
spectrum of size, regional location as well as public and private practices. These variables 
will ensure racial/ethnic diversity among study subjects that will mirror the state of New 
York’s and California’s racial and ethnic distribution as shown in the enrollment table. An 
estimated 57% of practicing Obstetrician/Gynecologists (OB) and 85% of graduating OB 
residents are female.  Therefore, we anticipate a higher percentage of females among the 
provider study subjects. Patient study subjects will all be female.  
 
Inclusion of children 
Pregnant patients receiving care at eligible practices will be considered part of the study 
population and will include pregnant patients that are aged less than 18 years of age.   
 
Inclusion of Pregnant Women 
Although the research project involves OB/GYN practice sites, study staff will have 
minimal contact with pregnant patients, and study efforts do not place the patients’ 
pregnancy at risk. All pregnant patients, at both the intervention and control sites, will 
continue to receive their expected standard of care for the entirety of their pregnancy. 
 
 
 

STUDY00007717 
Page 18 of 30           
Version Date: 2.4.25 
 
Inclusion of Employees 
Employees of URMC, RRH and UCLA will be included as subjects. All Information 
Sheets disseminated to study subjects during this study phase will emphasize that the 
decision to participate will have no impact upon: performance evaluations, job advancement, 
or the loss/gain of benefits (e.g., salary increases, time off).  
 
6. INCLUSION/EXCLUSION CRITERIA & RECRUITMENT METHODS 
 
All Participating Sites (Intervention & Control) 
URMC, RRH and UCLA sites that participated in the previous VAX-MOM studies 
(STUDY00007624, STUDY00005115 and STUDY00004408) were invited to participate 
in this study. Prior to the start of the current VAX-MOM COVID-19 project, Drs. Rand 
and Szilagyi contacted leadership within each health system to confirm interest in 
continued participation. 
 
Originally (for previous VAX-MOM studies), health systems (URMC/RRH/UCLA) 
within each state were chosen because a) their size allowed for an ample sampling of 
patients and practitioners (i.e., large subject pool) and b) they allowed for a broad 
spectrum of geographic and socio-economic diversity (e.g., privately and publicly 
insured, ethnic multiplicity, rural and urban locations, etc.).  
    
Intervention Sites 
As described in the “Randomization” subsection of the “Study Design & Procedures” 
portion of the protocol, study sites from the larger pool will be selected to be part of the 
“intervention group” using specific constrained randomization procedures within each 
health system (see section 4 of protocol for more details). All remaining sites will be 
allocated to the “control group.” 
 
Practice-Level Personnel 
 
Criteria/Recruitment Method for Vaccine Champion: 
At each site, the affiliated medical director or health system liaison will select one 
“vaccine champion” for the practice. The vaccine champion will be selected due to their 
perceived ability to play a lead role throughout the study intervention, including their 
ability to complete all monthly tasks listed in “Step 1” of the “Study Design and 
Procedures” section. The position of vaccine champion is voluntary, and if they do not 
wish to accept the role, they may decline and another individual will be identified as a 
replacement. 
 
Criteria/Recruitment Method for Remaining Site Personnel: 
By default (and with the support of health system and practice site leaders) all remaining 
providers, nurses, and practice staff employed by the practice location, are able to 
participate in the quality improvement initiative. 
 
 
 

STUDY00007717 
Page 19 of 30           
Version Date: 2.4.25 
 
Patient-Level Study Subjects 
 
Criteria for EHR Data from Practice Patients (Pregnant Women): 
Baseline vaccine rates (reported to each site during their site meeting) were established 
during the prior protocols (STUDY00007624, STUDY00005115 and STUDY00004408), 
and contained EHR information from practice patients who were identified as having a 
live birth within a 6 month time period prior to the start of the intervention.  
 
To establish vaccine rates for the ongoing monthly feedback reports (July 2022 onward), 
EHR information will be collected every month from intervention sites during the 
entirety of the intervention phase (see timeline depicted in Figure 1). EHR information 
will reflect practice patients who are identified as having a live birth on record for that 
month. Again, all subjects will be female and may be <18 years of age.  
 
To establish the final vaccine data reports (for intervention vs. control comparison), 
detailed EHR information (see Table 5) will be collected from both intervention and 
control sites for the entire intervention period, and additionally, any missing data from 
the baseline period will be collected at this time. 
 
7. CONSENT PROCESS 
 
Health System, Practice Site, & Site Personnel Level  
 
Health System & Practice Site Consent: 
 
Health system support, and subsequently site-level support, was previously 
obtained during phase 1 of this study (STUDY00007624), as well as during our 
original VAX-MOM studies (STUDY00005115 and STUDY00004408), and is 
described in the “Inclusion/Exclusion Criteria & Recruitment Methods” section of 
the protocol.  
 
Site Personnel Consent: 
 
 
Delivery of Training Content & Involvement in 6-Month QI Intervention  
(All site personnel: completion of e-learning module, attendance of follow-up site 
meeting, and participation in monthly discussions of rate feedback and PDSA 
cycles)  
 
This study involves commonly accepted quality improvement efforts aimed to 
improve upon the standard of care in medical settings, such as the completion of 
e-learning modules and in-person trainings specifically related to practice goals, 
the review of practice metrics (vaccine rates) by site personnel, discussion of 
office workflow procedures/efficacy, and trainings in optimal communication 
techniques.  
 
We are therefore seeking a waiver of documentation of consent for the training 
portion of the study. More specifically, we are requesting this waiver because: a) 
the training portion of the study is no greater than minimal risk, b) the purpose of 

STUDY00007717 
Page 20 of 30           
Version Date: 2.4.25 
 
the study/intervention and a basic framework of the trainings will be clearly 
outlined for participants in the Information Sheet (see 
“InfoSheet_TrainingPhase…” document) e-mailed to them at the onset of the 
intervention phase, and c) ample time will be given to personnel to review the 
Information Sheet in full and ask questions of study staff regarding the quality 
improvement effort and/or specific tasks involved in the training phase. 
 
We will attach an Information Sheet (cited above) to the first e-learning invite e-
mail, describing the study goals and training phase details, as well as site 
personnel involvement throughout the duration of the 6-month QI intervention (e-
mailed to site personnel just prior to the training phase). Study staff will field any 
and all questions from potential study subjects prior to their module training and 
subsequent site meeting. 
 
Assessment of Intervention Adoption: Surveys & Rate Feedback  
(Selected Vaccine Champions: completion of monthly rate feedback discussions 
and monthly PDSA cycles and Practice Time & Cost Surveys; participation in 
monthly learning collaborative meetings) 
 
The assessment of intervention adoption by study sites involves commonly 
occurring procedures within a medical setting, including the ongoing review of 
practice metrics (vaccine rate feedback sessions), the development and assessment 
of practice goals (PDSA Cycles and learning collaborative meetings), and the 
evaluation of personnel efforts to achieve these goals (Time & Cost Survey). 
 
We are therefore seeking a waiver of documentation of consent for the 
intervention adoption portion of the study. More specifically, we are requesting 
this waiver because: a) the assessment portion of the study is no greater than 
minimal risk, b) the purpose of the study/intervention and a description of the 
assessment tasks will be clearly outlined for vaccine champions in the 
Information Sheet (see “InfoSheet_VaccineChampion…” document) e-mailed to 
them at the onset of the assessment phase, and c) ample time will be given to each 
vaccine champion to review the Information Sheet in full and ask questions of 
study staff regarding the quality improvement effort and/or specific tasks involved 
in the assessment phase. 
 
As soon as they are selected, each identified vaccine champion will be e-mailed 
(see “IntroEmail_VaccineChampion…” document) an Information Sheet by the 
study coordinator, outlining the basic study goals as well as their involvement in 
tasks throughout the training and intervention adoption portions of the study. 
After the Information Sheets have been disseminated, the study coordinator will 
conduct a follow-up phone/Zoom call with each vaccine champion, reviewing 
their role within the project and fielding any and all questions they may have. The 
position of vaccine champion is voluntary, and if they do not wish to accept the 
role, they may decline and another individual will be identified as a replacement. 
 
 

STUDY00007717 
Page 21 of 30           
Version Date: 2.4.25 
 
Assessment of Intervention Adoption: In-Person Observation  
(Select Personnel: observation of workflow and communication techniques) 
 
The in-person observation of intervention adoption involves commonly occurring 
procedures within a medical setting, including the ongoing documentation and 
review of daily workflow and communication procedures.  
 
We are therefore seeking a waiver of documentation of consent for the in-person 
observation portion of the study. More specifically, we are requesting this waiver 
because: a) the observation portion of the study is no greater than minimal risk, b) 
the purpose of the observation will be clearly outlined for practice personnel in 
the Information Sheet (see “InfoSheet_InPersonObs…” document) e-mailed to 
them prior to the scheduled observation, and c) ample time will be given to 
practice personnel to review the Information Sheet in full and ask questions of 
study staff regarding the observation. 
 
Prior to in-person observation, the study coordinator will contact each site’s 
vaccine champion to determine days that would be optimal for study staff to be 
onsite. Once the observation schedule has been determined, all personnel who are 
scheduled to work on the specified day(s) will be e-mailed an Information Sheet 
(see “IntroEmail_InPersonObs…” document) by the study coordinator, which 
they may review at their own pace prior to the observation date. They may e-mail 
the project coordinator with any questions, or bring questions to the attention of 
the study staff on the scheduled visit day prior to the start of any observation. The 
content of the Information Sheet will be reviewed again by study staff with 
practice personnel on the visit day prior to the start of any documented 
observation (any personnel not previously known to be working that day will be 
given a hard copy of the Information Sheet upon arrival by the study staff and 
given ample time to review). 
 
As stated previously, although study staff will observe OB/GYN practice 
personnel during patient encounters, practice patients will NOT be the target of 
the observation, and no patient information (in either individual or aggregate 
form) will be collected. Rather, the focus of observation will be the select practice 
personnel only. Information recorded during the observation session will be 
written in a generic manner such that specific personnel/patients will not be 
identifiable. Notes will describe the general workflow processes and 
communication methods used, rather than specific unique behaviors or exact 
dialogue. For example, study staff may document “…when patient declined to 
receive the COVID vaccine, the practice nurse gave them an informational 
handout and told them the doctor may speak with them further.” Although the 
patient is not the study subject, prior to each observation, practice personnel or the 
study staff will explain to the patient in simple language the purpose of the 
observation (i.e., to take notes regarding the nurse/staff work routine), and any 
patient that communicates discomfort or preference to not be observed, will not 
be observed.           

STUDY00007717 
Page 22 of 30           
Version Date: 2.4.25 
 
Post-Intervention Survey Dissemination 
 
We are seeking a waiver of documentation of consent for the Post-Intervention 
Survey portion of the study. We are doing so because: a) the survey is no greater 
than minimal risk (a brief online survey with unobtrusive questions), b) a detailed 
Information Sheet will be made available to each subject prior to the start of the 
survey (embedded within REDCap with subjects matched to the correct health-
system Information Sheet), c) ample time will be given to each subject to consider 
participation (subjects are notified of the survey via e-mail and can review the 
information sheet and/or ask questions of study staff for as long as necessary 
before deciding about survey completion), and d) no identifiers will be included 
on the survey form (only subject IDs), and the separate document linking subject 
name to subject ID will only be made available to a limited number of study staff, 
will be kept under double-locked conditions, and will be destroyed three years 
after study completion. 
 
Patient (EHR) Level  
 
EHR Data Extraction: We are seeking a waiver of consent and a waiver of HIPAA 
authorization for the EHR data extraction portion of the study. We are doing so because: 
a) we believe that this research cannot be practicably conducted without such a waiver, as 
we cannot feasibly obtain consent and HIPAA authorization from all potential subjects 
(thousands in total subject pool), b) site liaisons and report builders have routine access 
to patient records, c) a minimal number of patient-level data fields will be extracted from 
the EHR (just enough to complete analysis) d) a plan to protect EHR data during all data 
transfers will be implemented (see “Privacy & Confidentiality” section for details), e) any 
individual-level data will be destroyed three years after study completion, and f) PHI will 
not be reused or disclosed to any other person or entity except (i) as required by law, (ii) 
for authorized oversight of the research study, or (iii) for other research for which the use 
or disclosure of PHI would be permitted by the HIPAA Privacy Rule. 
 
8. AUDIO/VIDEO RECORDINGS  
 
All site meetings held during the training phase and all learning collaborative meetings held 
during the intervention adoption phase may be audio and/or video recorded using a laptop or 
handheld recording device (in-person meetings) or via the Zoom recording option (virtual 
meetings) (see “Privacy and Confidentiality” section for more detail). These archived 
audio/video files will allow for the review of meeting content by site personnel who were 
unable to attend the meetings live, or by those who simply wish to review the meeting 
discussion for a second time. 
 
The archived meeting audio/video files will be stored on a secure Box platform, accessible 
only to study staff and site personnel from the corresponding practice site (each site will have 
their own Box folder ensuring meetings from other sites are not viewable to outside 
personnel). Invites to the secure Box folder will be e-mailed from study coordinators directly 
to site personnel. As soon as the video files have been uploaded to the secure Box platform, 

STUDY00007717 
Page 23 of 30           
Version Date: 2.4.25 
 
they will be deleted from the laptop or handheld recording device. Once the study has been 
fully completed, all video files will be deleted from the shared Box folder. 
 
9.   RISKS TO SUBJECTS 
 
Practice Level Risk: This project involves QI trainings/interventions which aim to improve 
upon the existing communication techniques and workflow procedures at each intervention 
practice site. Planned study trainings/interventions are widely accepted among medical 
practices (e.g., practice providers/nurses/staff attending an informational meeting regarding 
the risk of COVID-19 in the pregnant population), and do not involve any novel high-risk 
changes to practice structure or procedures. Practice sites in the control group will continue 
to utilize “best-practice” guidelines. Thus, at the practice level, anticipated overall study risk 
is very low.  
 
Site Personnel & Patient Level Risk: The primary risk to site personnel and practice 
patients is the risk of a breach of confidentiality, though overall risk remains low.  
 
To reduce risk, personal information being collected from site personnel or practice patients 
(EHR data) has been minimized, and when possible is being collected in aggregate form or 
using study IDs.  
 
A number of policies, procedures, and technical safeguards (described in the “Privacy & 
Confidentiality” section of this protocol) will be in place to ensure that there is no breach of 
confidentiality as a result of this study.   
 
10.  POTENTIAL BENEFITS TO SUBJECTS 
 
Practice personnel will receive targeted training related to: COVID-19 disease as it pertains 
to the pregnant population, communication, and optimization of office workflow as it relates 
to the improvement of immunization rates within their practice. As such, practice personnel 
who participate in both the training and ongoing QI portions of the study will have the 
opportunity to apply for MOC, CME, or CNE credit. 
 
There are no anticipated benefits for practice patients beyond those inherent to the 
overarching study goals (i.e., an improvement in immunization rates). 
 
11.  COSTS FOR PARTICIPATION 
 
There will be no costs incurred by participants.  
 
12.  PAYMENT FOR PARTICIPATION  
 
For assisting with learning module dissemination/completion and site meeting 
scheduling/engagement, each vaccine champion will receive $50.  For completing the monthly 
“PDSA Cycle” and “Practice Time & Cost” surveys for the full 6 month duration, each vaccine 
champion will receive $150. Payments will be in the form of an eGift card and will be e-mailed 
to the vaccine champion at the conclusion of the site meeting ($50) and conclusion of the 6 
month intervention phase ($150). For NY sites participating in the extended intervention period, 

STUDY00007717 
Page 24 of 30           
Version Date: 2.4.25 
 
vaccine champions will be paid an additional $25 for each continued month. 
 
Additionally, if manual chart review is necessary to obtain vaccine rates for any practice sites, 
each vaccine champion will receive $20 for each monthly review. Payments will be in the form 
of an eGift card and will be e-mailed to the vaccine champion at the conclusion of the 6 month 
intervention phase.  
 
For completing the “Post-Intervention Survey” portion of the study, each subject will receive 
$50. Payment will be in the form of an eGift card and will be e-mailed to the subject within 6 
weeks of survey completion.  
 
13.  SUBJECT WITHDRAWALS 
 
During phase 1 of this study all health systems and affiliated practice sites agreed to participate 
in this quality improvement effort. While we do not anticipate a high level of subject/site 
withdrawals, any site wishing to end their involvement in the quality improvement initiative may 
do so at any time. Any practice personnel wishing to abstain from participation in quality 
improvement activities (trainings, ongoing meetings, adoption of new workflow procedures) are 
free to do so. Any vaccine champion who does not fully complete all monthly study activities 
during the 6 month intervention period, will still receive payment for all completed surveys prior 
to their withdrawal using a prorated amount. All subjects (practice personnel) are free to 
discontinue participation at any time, without consequence.  
 
14.  PRIVACY AND CONFIDENTIALITY OF SUBJECTS AND RESEARCH DATA 
 
The following data collection and storage procedures will be implemented to ensure that subject 
privacy and confidentiality are maintained throughout the entirety of the research process:  
 
Data obtained via e-mail and REDCap during the training phase: 
(Site personnel names collected during the e-learning modules and site meetings) 
 
The study coordinator will maintain a list of site personnel affiliated with all project 
locations, and those who have successfully completed their e-learning modules and who 
attended follow-up site meetings.  These names will be obtained via e-mails from the vaccine 
champions and/or REDCap surveys. The lists will be stored in a password protected Excel 
file on a password protected computer for the purposes of confirming requests for MOC, 
CME and CNE credit and to track the overall level of active site participation. No additional 
study information (e.g., future survey responses) will be contained within this file.  All files 
will be stored on a secure server hosted by the university. These individual-level training 
completion files will be deleted once they are no longer necessary (i.e., once all credit 
applications are in and the project has concluded). Only aggregate data, not individual names, 
will be stored long-term by study staff regarding the completion of online training content by 
site personnel (i.e., the total number of providers, nurses, and staff who did/didn’t complete 
the training). 
 
 

STUDY00007717 
Page 25 of 30           
Version Date: 2.4.25 
 
Data Obtained via REDCap during the intervention adoption & assessment phase: 
(PDSA cycles, Time & Cost Surveys, Post-Intervention Survey) 
 
For study measures completed via a REDCap survey, all subject names and corresponding e-
mail addresses will first be collected by the study coordinator from either the health system 
liaisons or the assigned vaccine champions for each site. The names, e-mail addresses, site 
locations, role and an assigned subject ID for each person, will be stored in a password 
protected Excel file on a password protected computer. No additional study information (e.g., 
survey responses) will be contained within this file. All files will be stored on a secure server 
hosted by the university.  
 
Once e-mail addresses have been obtained, the study coordinator will upload the e-mail 
addresses onto the secure REDCap platform. Surveys will automatically be disseminated to 
the appropriate subjects through REDCap programming completed by the study coordinator. 
Completed REDCap survey data will be exported in an Excel file and stored under double-
locked conditions (password protected file) on the secure “SMDNAS Research Storage” 
folders, before transfer to the UCLA Health Sciences Box platform for analysis by the 
project’s UCLA data specialists.  
 
Data Obtained via EHRs: 
 
EHR data reports will be generated only by the specified EHR report builder for each health 
system, or via manual chart review as described on page 12. A minimal number of patient-
level data fields (only those needed for analyses) will be extracted from the EHRs. For data 
verification purposes, identified study staff will check a small percentage of patient eRecord 
files to ensure the accuracy of generated EHR reports. The identified individuals will have 
access to EHR files and will complete all appropriate eRecord training prior to conducting 
data verification. All individual level PHI data will be stored on either the secure UCLA 
Health Sciences Box platform (original and final files) or the secure “SMDNAS Research 
Storage” folders (files currently being audited). Whenever possible, data will be stored in 
aggregate form.  
 
All Research Data:  
 
ORPA will be consulted and a Data Use Agreement (DUA) will be implemented as required 
between all participating health systems. 
 
Please also see the “Human Subject Research Data Security Questionnaire” for all utilized 
methods of data collection/transfer/management/analyses for all study activities. 
 
15. DATA / SAMPLE STORAGE FOR FUTURE USE 
 
Hard copy data: Due to the online (e.g., REDCap and excel files) nature of data collection 
during phase 2 of the study, no hard-copy documents will be created or stored. As we are seeking 
a waiver of documentation of consent or waiver of consent for all portions of the study, there will 
be no need to store consent forms with subject names.   
 

STUDY00007717 
Page 26 of 30           
Version Date: 2.4.25 
 
Electronic data: 
 
Site Meeting Attendance:  
Files containing site meeting attendance records will be stored in a password protected 
Excel file on a password protected computer. No additional study information (e.g., 
survey responses) will be contained within these files. All files will be stored on a secure 
server within the university system.  
 
E-learning modules, PDSA cycles, Time & Cost Surveys, Post-Intervention Survey: 
Survey data is stored immediately upon completion by the REDCap website. Study staff 
will then access the secure password-protected survey site to generate data reports when 
needed.  These data reports (exported excel files) will be stored in “SMDNAS Research 
Storage” folders or UCLA Health Box.  
 
EHR data output:  
All individual level PHI data will be stored on either the secure UCLA Health Sciences 
Box platform (original and final files) or in the secure “SMDNAS Research Storage” 
folders (files currently being audited). Whenever possible, data will be stored in 
aggregate form.  Analyzed data reports from the UCLA data specialist will be sent in a 
de-identified form via secure e-mail to the project PI for review.  All data will be stored 6 
years beyond study completion. 
 
16. DATA AND SAFETY MONITORING PLAN 
 
This study presents no more than minimal risk, however there is still a risk to privacy and 
confidentiality. Prior to the start of phase 1, the research team from both NY and CA met to 
create a data safety monitoring plan. PIs, team members, and representatives from each study 
clinic regularly discuss communication and action plans in the unlikely event that an adverse 
event occurs. Information on how to contact the study team via phone, mail or email is readily 
apparent to all participating providers and care team members. Though adverse events are not 
anticipated, should any occur they will be reported to the site PI (“team leader”) in each state and 
then to the IRB at the time of the event, and copies of all correspondence regarding the event 
with the IRB will be shared with the CDC, as needed. 
 
17. DATA ANALYSIS PLAN 
 
The primary outcome (receipt of 1+ COVID vaccine) is binary and our main explanatory 
variable will be an indicator for study arm. We will employ intent-to-treat analyses using 
generalized linear mixed models (GLMMs) with practice random effects, an approach 
recommended for group-randomized RCTs in which the goal is to estimate the causal effects of 
interventions on individuals, adjusted for clustering within groups. This method performs well in 
situations where the number of observations per cluster is large and for unequal cluster sizes. 
Models will assume a binomial distribution and a log link function in order to compare 
vaccination rates between study arms in terms of risk ratios. We will adjust for all practice-level 
variables included in the randomization balancing criterion, as well as patient-level 
race/ethnicity. Hypothesis tests will be two-sided with alpha = 0.05. Analyses will be performed 

STUDY00007717 
Page 27 of 30           
Version Date: 2.4.25 
 
using SAS v. 9.4 (SAS Institute Inc., Cary, NC). 
 
Power Analysis: We conservatively assume a control group vaccination rate of 50% (maximum 
variance). Adjusting for clustering of patients in practices, and assuming an intraclass correlation 
(ICC) of 1% (consistent with previous work), this sample size provides 80% power to detect an 
overall increase of 8.0 percentage point increase in vaccination rates between the QI intervention 
and control arms. This assumes a chi-squared test (a simplification of the planned mixed model 
analysis described above), an alpha of 0.05, and a sample size of 5,000 pregnant women 
organized into 16 practices per study arm. 
 
Final Time/Cost Analysis:  
Our two-fold cost measures are (a) total intervention cost to implement the VAX-MOM 
COVID program aggregated at the three health-system level under a cost analysis and (b) cost 
per additional COVID vaccination under a subsequent cost-effectiveness analysis. To derive a 
policy implication regarding the sustainability of programs, we will estimate costs and ICER 
estimates from the health system perspective.  
 
Costs: To make these cost measures comparable to similar past interventions of reminders 
and educational programs, we will estimate cost with one-year time horizon, excluding the cost 
to purchase, store and administer vaccines. We will estimate the total intervention cost, summing 
non-personnel costs (e.g., EHR hardware, software, and materials) and personnel costs. The 
personnel costs will distinguish research costs from intervention costs (e.g., practice-level 
meeting and collecting EHR data). The dollar values of these personnel costs will be calculated 
by multiplying “time efforts” (weekly reported by study personnel using a REDCap email 
survey) with the nationally representative “hourly-wage rates” by occupation codes of study 
personnel, derived from the Bureau of Labor Statistics. We are using this method in an NIH-
funded R01 study (the STOP-HPV study) which involves a practice-level intervention to raise 
HPV vaccination rates and in the original VAX-MOM study; surveys take <1 minute to 
complete.  
 
Cost-Effectiveness: We will develop a standard decision model for our cost-effectiveness 
analyses as conducted in our past studies. The effectiveness measures are the rates of COVID 
vaccination estimated under Aim 2b. We will estimate incremental cost effectiveness ratios 
(ICERs) for the intervention defined as: ICER (Study vs control) = (costStudy – costControl) / 
(COVID vaccine rateStudy – COVID vaccine rateControl) 
 
In the numerator of the equation, the costStudy is standardized to be equal to average 
intervention cost per patient in a study arm to account for the potential sample size difference 
between study and control arms. 
 
As explained above, this intervention cost will exclude the cost to purchase, store and 
administer vaccines, which is assumed to be identical among all practices. Applying the same 
assumption, the average cost for the control group (costControl) is zero. Using the developed 
standard decision model, we will conduct probabilistic analyses to generate point estimates and 
95% CIs for ICERs and one-way sensitivity analyses to determine conditions for being lower 
than the thresholds of cost-effectiveness, e.g., healthcare-system based quality improvement 

STUDY00007717 
Page 28 of 30           
Version Date: 2.4.25 
 
interventions to improve flu vaccination uptake targeting general populations (median ICER $51 
among 23 interventions) and healthcare workers (median ICER $125 among 6 interventions) 
reported by a systematic review. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

STUDY00007717 
Page 29 of 30           
Version Date: 2.4.25 
 
18. REFERENCES 
 
1. Wei SQ, Bilodeau-Bertrand M, Liu S, Auger N. The impact of COVID-19 on pregnancy 
outcomes: a systematic review and meta-analysis. CMAJ. 2021;193(16):E540-E548. 
2. Metz TD, Clifton RG, Hughes BL, et al. Association of SARS-CoV-2 Infection With Serious 
Maternal Morbidity and Mortality From Obstetric Complications. JAMA. 2022;327(8):748-
759. 
3. Halasa NB, Olson SM, Staat MA, et al. Maternal Vaccination and Risk of Hospitalization for 
Covid-19 among Infants. N Engl J Med. 2022. 
4. Lipkind HS, Vazquez-Benitez G, DeSilva M, et al. Receipt of COVID-19 Vaccine During 
Pregnancy and Preterm or Small-for-Gestational-Age at Birth - Eight Integrated Health Care 
Organizations, United States, December 15, 2020-July 22, 2021. MMWR Morb Mortal Wkly 
Rep. 2022;71(1):26-30. 
5. COVID Data Tracker. Centers for Disease Control and Prevention. 
https://covid.cdc.gov/covid-data-tracker. Published 2022. Accessed 7/8/2022. 
6. Halasa NB, Olson SM, Staat MA, et al. Effectiveness of Maternal Vaccination with mRNA 
COVID-19 Vaccine During Pregnancy Against COVID-19-Associated Hospitalization in 
Infants Aged <6 Months - 17 States, July 2021-January 2022. MMWR Morb Mortal Wkly 
Rep. 2022;71(7):264-270. 
7. Henninger M, Naleway A, Crane B, Donahue J, Irving S. Predictors of seasonal influenza 
vaccination during pregnancy. Obstetrics and gynecology. 2013;121(4):741-749. 
8. Eppes C, Wu A, You W, Cameron KA, Garcia P, Grobman W. Barriers to influenza 
vaccination among pregnant women. Vaccine. 2013;31(27):2874-2878. 
9. Kriss JL, Albert AP, Carter VM, et al. Disparities in Tdap Vaccination and Vaccine 
Information Needs Among Pregnant Women in the United States. Maternal and child health 
journal. 2018. 
10. Henninger ML, Irving SA, Thompson M, et al. Factors associated with seasonal influenza 
vaccination in pregnant women. Journal of women's health. 2015;24(5):394-402. 
11. New S, Winter K, Boyte R, et al. Barriers to Receipt of Prenatal Tetanus Toxoid, Reduced 
Diphtheria Toxoid, and Acellular Pertussis Vaccine Among Mothers of Infants Aged <4 
Months with Pertussis - California, 2016. MMWR Morbidity and mortality weekly report. 
2018;67(38):1068-1071. 
12. Kahn KE, Black CL, Ding H, et al. Influenza and Tdap Vaccination Coverage Among 
Pregnant Women - United States, April 2018. MMWR Morbidity and mortality weekly 
report. 2018;67(38):1055-1059. 
13. National Vaccine Advisory C. The National Vaccine Advisory Committee: reducing patient 
and provider barriers to maternal immunizations: approved by the National Vaccine 
Advisory Committee on June 11, 2014. Public health reports. 2015;130(1):10-42. 
14. Shavell VI, Moniz MH, Gonik B, Beigi RH. Influenza immunization in pregnancy: 
overcoming patient and health care provider barriers. American journal of obstetrics and 
gynecology. 2012;207(3 Suppl):S67-74. 
15. Healy CM, Rench MA, Montesinos DP, Ng N, Swaim LS. Knowledge and attitiudes of 
pregnant women and their providers towards recommendations for immunization during 
pregnancy. Vaccine. 2015;33(41):5445-5451. 
 

STUDY00007717 
Page 30 of 30           
Version Date: 2.4.25 
 
16. Centers for Disease C, Prevention. Influenza vaccination coverage among pregnant women: 
2011-12 influenza season, United States. MMWR Morbidity and mortality weekly report. 
2012;61:758-763. 
17. MacDougall DM, Halperin SA. Improving rates of maternal immunization: Challenges and 
opportunities. Human vaccines & immunotherapeutics. 2016;12(4):857-865.