COVID-19 Vaccination Take-Up in a County-Run Medicaid Managed Care Population
Phase: NA
Conditions: Covid19, Vaccination
Sponsor: National Bureau of Economic Research, Inc.

 
 
 
 
 
 
COVID-19 Vaccination Take-Up in a Medicaid Managed Care 
Population 
 
Pilot PI:  Mireille Jacobson 
 
Manual of Procedures (MOP) 
Created 04/29/2021, Revised 06/03/2021 

MOP version 1.1 dated 03Jun2021 
 
Page 2 of 69 
Table of Contents 
1.0 INTRODUCTION ............................................................................................................................. 3 
2.0 BRIEF OVERVIEW OF THE STUDY PROTOCOL ........................................................................ 3 
3.0 STUDY STAFF RESPONSIBILITIES ............................................................................................. 3 
4.0 STUDY FLOW DIAGRAM ............................................................................................................... 4 
Figure 1: Study Flow Diagram ........................................................................................................... 4 
5.0 RECRUITMENT AND RETENTION ................................................................................................ 4 
5.1 Screening and Eligibility Criteria .................................................................................................. 5 
5.2 Screening Log .............................................................................................................................. 5 
5.3 ELIGIBILITY CRITERIA .................................................................................................................. 5 
6.0 INFORMED CONSENT ................................................................................................................... 5 
6.1 HIPAA Authorization .................................................................................................................... 5 
7.0 STUDY INTERVENTION ................................................................................................................. 5 
8.0 RANDOMIZATION .......................................................................................................................... 5 
9.0 BLINDING AND UNBLINDING (MASKING AND UNMASKING) .................................................. 8 
10.0 SAFETY REPORTING .................................................................................................................. 8 
11.0 STUDY COMPLIANCE ................................................................................................................. 8 
12.0 DATA COLLECTION AND STUDY FORMS ................................................................................ 9 
12.1 Participant Data ......................................................................................................................... 9 
12.2 Study Forms .............................................................................................................................. 9 
12.3 General Instructions for Completing Forms .............................................................................. 9 
12.4 Data Flow ................................................................................................................................... 9 
12.5 Administrative Forms ................................................................................................................. 9 
12.6 Retention of Study Documentation............................................................................................ 9 
13.0 DATA MANAGEMENT .................................................................................................................. 9 
13.1 External Data ............................................................................................................................. 9 
13.2 Quality Control Procedures ....................................................................................................... 9 
14.0 CONCOMITANT MEDICATIONS ................................................................................................. 9 
15.0 DATA AND SAFETY MONITORING ACTIVITIES ..................................................................... 10 
15.1 Study Completion and Close-Out Procedures ........................................................................ 10 
15.1.1 Participant Notification ...................................................................................................... 10 
15.1.2 Confidentiality Procedures ............................................................................................... 10 
16.0 MOP MAINTENANCE ................................................................................................................. 10 
APPENDIX A: IRB-APPROVED STUDY PROTOCOL ...................................................................... 12 
APPENDIX B: INFORMED CONSENT DOCUMENTS AND HIPAA AUTHORIZATION .................. 50 
APPENDIX C: MOP MODIFICATIONS LOG ..................................................................................... 69 
 
 
 

MOP version 1.1 dated 03Jun2021 
 
Page 3 of 69 
1.0 Introduction  
We will measure COVID-19 related preventative health behaviors, including COVID-19 
vaccinations, in the Contra Costa County, CA Medicaid managed care population. Our 
goal is to test ways to increase COVID-19 vaccination uptake. Our hypothesis is that as 
a result of financial, cultural, or access issues, among other things, vaccinations and other 
preventative health behaviors are too low from a public health standpoint, meaning 
relative to virus exposure and the potential adverse consequences of infection in low-
income and racial/ethnic minority populations. Furthermore, we hypothesize that small 
financial incentives and other low-cost behavioral nudges can be used to reduce 
disparities, increasing vaccination rates and other behaviors known to mitigate the spread 
of the virus. 
2.0 Brief Overview of the Study Protocol 
Approximately 10,000 adult subjects will be recruited from the Contra Costa Health Plan 
(CCHP) via baseline survey. Subjects who complete the baseline survey will be 
randomized to the following arms: 
1. Control Arm (n=2,500) 
2. Informational Arm: no information/emotional message vs. safety and effectiveness 
information vs. information on consequences of going unvaccinated, race and/or 
gender concordant or discordant [7,500] 
 
Each of these arms will be interacted with a financial incentive of $10 (N=2,500) or $50 
(N=2,500) and, separately with a convenient link to the county public vaccine appointment 
scheduling system highlighted for participants (N=5,000).    
 
The above treatments are designed to test the role of the following on vaccine take-up:  
 
• Financial incentives [N=5,000] vs. no financial incentives [N=5,000] 
o 2,500 will be randomized to a $10 incentive and 2,500 to a $50 incentive 
• Convenient scheduling link highlighted [N=5,000] vs. not [N=5,000] 
• Messaging [N=7,500] vs not [2,500]  
o Message type: emotion [N=2,500] vs. safety and effectiveness [N=2,500] 
vs. consequences of not vaccinating [N=2,500] 
• Race concordant [N=2,500] vs. race discordant messenger [N=2,500] 
• Gender concordant [N=2,500] vs. gender discordant messenger [N=2,500] 
 
We will obtain survey data on preventative health behaviors, including mask-wearing, 
hand washing, and willingness to vaccinate. We will obtain EMR data on vaccine take-
up. 
3.0 Study Staff Responsibilities 
Pilot PI Jacobson will have ultimate responsibility for the study’s design and 
implementation. She will be supported by Other Significant Contributors Tom Chang and 
Manisha Shah. Jacobson will develop all study materials, including this MOP. She will be 

MOP version 1.1 dated 03Jun2021 
 
Page 4 of 69 
responsible for monitoring and reporting adverse events (AEs), serious adverse events 
(SAEs), and other issues per the terms of the relevant Data and Safety Monitoring Plan 
(DSMP). She will obtain informed consent and HIPAA authorization from all subjects; 
recruit and randomize subjects; receive EMR data from CCHS; following the procedures 
in this MOP and the IRB-approved protocol; and report any protocol deviations to the IRB 
and other relevant parties. She will be responsible for protecting subjects’ rights on a daily 
basis. She will oversee all data management procedures and quality control procedures.  
 
Collaborator Rajiv Pramanik of Conta Costa Health System (CCHS) will facilitate subject 
contacts and sharing of EMR data. CCHS will provide data on eligible CCHP members 
and will be responsible for screening members for eligibility. 
4.0 Study Flow Diagram 
Figure 1: Study Flow Diagram 
 
5.0 Recruitment and Retention 
A random sample of eligible CCHP members will be invited to participate in the study via 
email, mail, and/or text. We aim to recruit 10,000 subjects and recruitment will continue 
until we have reached 10,000 baseline survey responses. 
We will provide gift cards for baseline survey completion ($5 or $25, depending on survey 
take-up). In addition, respondents will be entered into a raffle for $250 gift cards. 

MOP version 1.1 dated 03Jun2021 
 
Page 5 of 69 
5.1 Screening and Eligibility Criteria  
CCHS will provide a list of eligible members and will be responsible for eligibility 
screening. Pilot PI Jacobson will not be responsible for eligibility screening. See 5.3 for 
Eligibility Criteria. 
5.2 Screening Log 
As screening will not be performed by Pilot PI Jacobson or her academic study team, this 
group will not maintain a screening log. Collaborator Pramanik will maintain 
documentation of eligibility determinations. 
5.3 Eligibility Criteria 
Eligible subjects are adult CCHP members with no previous COVID-19 vaccination 
history and no contraindications to COVID-19 vaccination, as determined in CCHP 
medical staff. 
6.0 Informed Consent  
Informed consent will be obtained during the baseline survey to recruit subjects. The IRB-
approved consent documents are attached as an appendix. 
6.1 HIPAA Authorization 
Personal health information (PHI) will be obtained in keeping with the Health Insurance 
Portability and Accountability Act (HIPAA). The IRB approved a partial HIPAA waiver for 
purposes of study participant recruitment. The study outcome, COVID-19 vaccination, is 
PHI. The IRB-approved HIPAA authorization form is attached as an appendix. 
7.0 Study Intervention  
Subjects in treatment arms will receive interventions: 
• Financial incentives [N=5,000] vs. no financial incentives [N=5,000] 
o 2,500 will be randomized to a $10 incentive and 2,500 to a $50 incentive 
• Convenient scheduling link highlighted [N=5,000] vs. not [N=5,000] 
• Messaging [N=7,500] vs not [2,500]  
o Message type: emotion [N=2,500] vs. safety and effectiveness [N=2,500] 
vs. consequences of not vaccinating [N=2,500] 
• Race concordant [N=2,500] vs. race discordant messenger [N=2,500] 
• Gender concordant [N=2,500] vs. gender discordant messenger [N=2,500] 
8.0 Randomization  
We will stratify our sample based on based on race/ethnicity and age-group. 
Randomization will be performed in the RedCAP system. In practice, the randomization 
will divide the sample into 60 possible conditions: 
 
1. Control [N=625] 
2. Control x $10 financial [N=312.5] 
3. Control x $50 financial [N=312.5] 

MOP version 1.1 dated 03Jun2021 
 
Page 6 of 69 
4. Control x link [N=625] 
5. Control x $10 financial x link [N=312.5] 
6. Control x $50 financial x link [N=312.5] 
7. No information/Emotional (language concordant) [N=625]   
8. No information/Emotional (language concordant) x $10 financial [N=312.5]  
9. No information/Emotional (language concordant) x $50 financial [N=312.5] 
10. No information/Emotional (language concordant) x link [N=625] 
11. No information/Emotional (language concordant) x $10 financial x link [N=312.5] 
12. No information/Emotional (language concordant) x $50 financial x link [N=312.5] 
13. Safety and effectiveness (race concordant, male) [N=156.25]  
14. Safety and effectiveness (race concordant, male) x $10 financial incentive 
[N=78.125]  
15. Safety and effectiveness (race concordant, male) x $50 financial incentive 
[N=78.125] 
16. Safety and effectiveness (race concordant, male) x link [N=156.25] 
17. Safety and effectiveness (race concordant, male) x $10 financial incentive 
[N=78.125]  
18. Safety and effectiveness (race concordant, male) x $50 financial incentive 
[N=78.125] 
19. Safety and effectiveness (race concordant, female) [N=156.25]  
20. Safety and effectiveness (race concordant, female) x $10 financial incentive 
[N=78.125] 
21. Safety and effectiveness (race concordant, female) x $50 financial incentive 
[N=78.125] 
22. Safety and effectiveness (race concordant, female) x link [N=156.25] 
23. Safety and effectiveness (race concordant, female) x $10 financial incentive x 
link [N=78.125] 
24. Safety and effectiveness (race concordant, female) x $50 financial incentive x 
link [N=78.125] 
25. Safety and effectiveness (race discordant, male) [N=156.25]  
26. Safety and effectiveness (race discordant, male) x $10 financial incentive 
[N=78.125]  
27. Safety and effectiveness (race discordant, male) x $50 financial incentive 
[N=78.125]  
28. Safety and effectiveness (race discordant, male) x link [N=156.25] 
29. Safety and effectiveness (race discordant, male) x financial incentive x link 
[N=78.125] 
30. Safety and effectiveness (race discordant, male) x financial incentive x link 
[N=78.125] 
31. Safety and effectiveness (race discordant, female) [N=156.25] 
32. Safety and effectiveness (race discordant, female) x $10 financial incentive 
[N=78.125]  

MOP version 1.1 dated 03Jun2021 
 
Page 7 of 69 
33. Safety and effectiveness (race discordant, female) x $50 financial incentive 
[N=78.125]  
34. Safety and effectiveness (race discordant, female) x link [N=156.25] 
35. Safety and effectiveness (race discordant, female) x $10 financial incentive x link 
[N=78.125] 
36. Safety and effectiveness (race discordant, female) x $50 financial incentive x link 
[N=78.125] 
37. Consequences of going unvaccinated (race concordant, male) [N=156.25]  
38. Consequences of going unvaccinated (race concordant, male) x $10 financial 
incentive [N=78.125]  
39. Consequences of going unvaccinated (race concordant, male) x $50 financial 
incentive [N=78.125]  
40. Consequences of going unvaccinated (race concordant, male) x link [N=156.25] 
41. Consequences of going unvaccinated (race concordant, male) x $10 financial 
incentive x link [N=78.125]  
42. Consequences of going unvaccinated (race concordant, male) x $50 financial 
incentive x link [N=78.125]  
43. Consequences of going unvaccinated (race concordant, female) [N=156.25]  
44. Consequences of going unvaccinated (race concordant, female) x $10 financial 
incentive [N=78.125]  
45. Consequences of going unvaccinated (race concordant, female) x $50 financial 
incentive [N=78.125]  
46. Consequences of going unvaccinated (race concordant, female) x link 
[N=156.25] 
47. Consequences of going unvaccinated (race concordant, female) x $10 financial 
incentive x link [N=78.125] 
48. Consequences of going unvaccinated (race concordant, female) x $50 financial 
incentive x link [N=78.125] 
49. Consequences of going unvaccinated (race discordant, male) [N=156.25]  
50. Consequences of going unvaccinated (race discordant, male) x $10 financial 
incentive [N=78.125]  
51. Consequences of going unvaccinated (race discordant, male) x $50 financial 
incentive [N=78.125]  
52. Consequences of going unvaccinated ( race discordant, male) x link [N=156.25] 
53. Consequences of going unvaccinated (race discordant, male) x $10 financial 
incentive x link [N=78.125] 
54. Consequences of going unvaccinated (race discordant, male) x $50 financial 
incentive x link [N=78.125] 
55. Consequences of going unvaccinated (race discordant, female) [N=156.25] 
56. Consequences of going unvaccinated (race discordant, female) x $10 financial 
incentive [N=78.125]  
57. Consequences of going unvaccinated (race discordant, female) x $50 financial 
incentive [N=78.125]  

MOP version 1.1 dated 03Jun2021 
 
Page 8 of 69 
58. Consequences of going unvaccinated (race discordant, male) x link [N=156.25] 
59. Consequences of going unvaccinated (race discordant, male) x $10 financial 
incentive x link [N=78.125] 
60. Consequences of going unvaccinated (race discordant, male) x $50 financial 
incentive x link [N=78.125] 
 Randomization assignments will be documented for future reference. 
9.0 Blinding and Unblinding (Masking and Unmasking) 
There is no scope for masking in this study. Subjects will know that treatment they have 
received, if any. We will not mask care providers or outcomes assessors. 
10.0 Safety Reporting 
Per our DSMP, we will be unable to observe AEs and SAEs in our survey and EMR data. 
 
Routine monitoring of EMR data for AEs and SAEs will not be conducted.  
 
However, if the research team becomes aware through other channels of a specific event, 
the report will immediately be relayed to the PIs.  
 
The PI will determine whether the event constitutes an AE or SAE, and classify any AEs 
or SAEs on their severity, expectedness, and relatedness, as above. The PI will consult 
with a qualified clinician as necessary to make such determinations. Affected subjects will 
be advised to visit a local, qualified medical practitioner. 
 
Unexpected SAEs will be reported to the Safety Officer (SO), IRB, and NIA within 48 
hours of the study’s knowledge, and summarized in periodic electronic reports to the SO.  
 
Any unanticipated problem, defined as an issue related to the research that suggests the 
research places participants or others at greater risk than was expected, will be reported 
to NIA and the SO within 48 hours of the study’s knowledge. Reporting will occur within 
24 hours if the problem involves death. The report will include a plan to correct the 
problem and prevent its reoccurrence. 
 
Any SAEs related to the study and any unanticipated problems will also be promptly 
reported to the sIRB. 
11.0 Study Compliance  
Protocol deviations for this study, while exceedingly unlikely, could include randomizing 
an ineligible participant; obtaining incomplete or inadequate informed consent; 
administering the wrong treatment to a subject; or failing to keep the IRB protocol up to 
date. 
 
Any deviations that impact participant safety will be reported to NIA and the SO within 24 
hours of occurrence, or as soon as they are discovered. All other deviations will be 

MOP version 1.1 dated 03Jun2021 
 
Page 9 of 69 
captured in routine reports to NIA and the SO. All deviations will be reported to the sIRB 
of Record in accordance with local policies and captured in the protocol deviations log in 
the appendix of this MOP.  
12.0 Data Collection and Study Forms 
12.1 Participant Data 
Individual-level survey data, EMR data, and randomization information will be maintained 
electronically by the study team. Survey data will be received directly by the study team. 
EMR data will be transmitted from CCHS to the study team. 
12.2 Study Forms 
There are no study-specific forms for this study.  
12.3 General Instructions for Completing Forms  
Not applicable. 
12.4 Data Flow  
Data, which will include PHI, will be stored on secure servers only available to the 
research team. Coded data will be used for analysis and will be stored separately from 
the identifiable data.  
12.5 Administrative Forms 
We will use the Adverse Event Form and Serious Adverse Event Form from the NIA 
Clinical Research Study Investigator’s Toolbox to record any AEs or SAEs.  
12.6 Retention of Study Documentation  
Participant data will be retained by the study team for at least the NIH minimum of three 
years. We have no plans for data destruction at this time. Informed consent will be 
included in the baseline survey data and maintained indefinitely.  
13.0 Data Management 
13.1 External Data  
The study team will receive data from CCHS on eligible CCHP members and on enrolled 
subjects’ vaccine outcomes. These data transfers will rely on Box.com, which encrypts 
the data during transfer and at rest. USC and CCHS will execute appropriate data use 
agreement(s) before transferring any data. 
13.2 Quality Control Procedures  
The study team will review data from CCHS, as well as survey data, as soon as 
possible after receipt. The team will perform standard data quality checks, to include 
flagging missing, out-of-range, or illogical data.  
14.0 Concomitant Medications 
Not applicable. 

MOP version 1.1 dated 03Jun2021 
 
Page 10 of 69 
15.0 Data and Safety Monitoring Activities 
15.1 Study Completion and Close-Out Procedures  
At close-out, Pilot PI Jacobson will ensure that all required data has been received and 
that all data and analysis files are organized and accessible for review. She will ensure 
that all required study information has been reported to the NIA and other funders. She 
will notify the IRB of Record of the study’s completion and retain a copy of this notification. 
Finally, she will prepare a report summarizing the study’s conduct. 
15.1.1 Participant Notification 
We do not plan to notify subjects or their health providers of the results of the study. 
15.1.2 Confidentiality Procedures 
Data will be stored securely as described in 12.4 and 13.1 above. Data will be accessed 
only by staff with a legitimate need. All staff will be properly trained in human subjects 
research practices and appropriately supervised.  
 
If computers are used to store and/or analyze clinical data, the investigator should 
address elements of computer security to ensure that the data remain confidential. These 
elements include but are not limited to: utilization of computer and system passwords, 
user security training, system testing and verification, and routine system backups to 
prevent any loss of electronic data. 
16.0 MOP Maintenance 
Each page of the MOP is numbered, dated, and includes a version number. Modifications 
are logged in the appendix. 

MOP version 1.1 dated 03Jun2021 
 
Page 11 of 69 
 

MOP version 1.1 dated 03Jun2021 
 
Page 12 of 69 
APPENDIX A: IRB-Approved Study Protocol and 06/01/2021 Amendment 

MOP version 1.1 dated 03Jun2021 
 
Page 13 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 14 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 15 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 16 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 17 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 18 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 19 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 20 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 21 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 22 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 23 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 24 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 25 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 26 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 27 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 28 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 29 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 30 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 31 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 32 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 33 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 34 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 35 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 36 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 37 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 38 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 39 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 40 of 69 
 
 
 
 

MOP version 1.1 dated 03Jun2021 
 
Page 41 of 69 
 
Amended Protocol – April 2021 (0.05) 
 
• Participants: Adult patients impaneled at Contra Costa Regional Medical Center 
(CCRMC) who are eligible for COVID-19 vaccination based on county and 
CCRMC determination. 
o Adults ages 18+, eligible for vaccination (as determined by the county) 
o Self-identified as African American/Black, White or Latino (irrespective of 
race). 
o Children are not included as they do not make their own health care 
decisions. We are interested in how adults make decisions about their own 
COVID-19 vaccination and other preventive health behaviors. 
o We are not explicitly studying a protected population. Prisoners are not 
impaneled at CCRMC. Pregnant women, to the extent they are not excluded 
in the county’s vaccination plans, may be enrolled in the study. However, 
we will not explicitly target pregnant women or ask about pregnancy. 
Vaccination offers will be determined by medical personnel from the county 
and CCRMC, separate from the study.  
 
• Recruitment Process (sampling strategy) 
o Eligible adults will be invited to participate in the study via email, mail, 
and/or text.  
o CCRMC will provide us with a list of impaneled adult patients who are 
eligible for vaccination based on county prioritization categories and 
individual health status and who self-identify as African American/Black, 
White or Latino (irrespective of race). The list will be shared by CCHS via 
USC Marshall’s box.com accounts and will be managed by Tom Chang. 
o We will randomly sample and invite adults from this list to participate in a 
baseline survey. Recruitment in this way will continue until we reach a goal 
of 10,000 baseline survey responses. 
 
• Enrollment Process 
o Consent will occur through the baseline survey available via USC 
RedCAP and through the included consent documents.  
o We will also ask for HIPAA authorization so that we can link the survey 
data to information on vaccinations. 
 
• Methods 
Our work will begin with a large online survey (N=10,000) of a random sample of the 
members of the county Medicaid managed care plan in Contra Costa County, CA. Contra 
Costa Health Plan (CCHP), has over 100,000 adult members. Over a quarter of adult 
members are Latino, a quarter non-Hispanic White and about 15% each are Black, Asian 
or of other/unknown race. Our sample will be drawn from CCHP members ages 18 and 
over who have not been previously vaccinated against COVID-19 and who have no 
contraindications to vaccination, as determined by county health plan medical staff. 
Because we are interested in testing race concordant messaging, we will also restrict to 

MOP version 1.1 dated 03Jun2021 
 
Page 42 of 69 
members who are African American/Black, White or Latino (irrespective of race). Those 
of other/unknown race are excluded because they cannot be matched based on race to 
a messenger as are Asians because the group is comprised of populations that are both 
linguistically and culturally quite distinct. In particular, about a third of the “Asian” group is 
Filipino, a third Vietnamese, 25% Mandarin-speaking Chinese, 20% Cantonese-speaking 
Chinese, 15% Punjabi-speaking Indian, 10% Laotian, 10% Korean, and so on. As such, 
we do not think these populations can credibly be treated as one group for the purposes 
of studying race concordance and yet each group is individually too small to consider 
alone. Survey respondents will receive a $5 gift card for their time and be entered into a 
raffle for a $250 gift card. 
 
In the survey, much of which we have previously piloted for a related project, we will 
capture rates of preventive health behaviors, including mask-wearing, hand washing, and, 
most importantly, willingness to vaccinate. Stratifying based on race/ethnicity, age-group 
and other characteristics predictive of vaccine take-up, we will randomize the 10,000 
respondents to one of four arms: 
 
1. Control arm [2,500] 
2. Messaging/Information Arm 1: no information/emotional message [N=2,500] 
3. Messaging/Information Arm 2: safety and effectiveness information [N=2,500]  
4. Messaging/Information Arm 3: information on consequences of going 
unvaccinated].  
The information arms have 
been 
designed 
with 
the 
experimental 
literature 
on 
vaccination intentions in mind 
(see Brewer et al. 2017 for a 
review). That literature finds, 
among 
other 
things, 
that 
messages aimed at clarifying the negative outcomes of not getting vaccinated are much 
more effective at changing intentions than messages that try to correct misperceptions 
about vaccine safety. Within the arm 3 and arm 4 message groups, we will further 
randomize participants to race/ethnicity concordant/discordant video messages and to 
gender concordant/discordant messages. This intervention builds on recent evidence that 
race-concordant health care providers increase health screening take-up (Alsan et al. 
2019).  
 
Each of these four arms will be interacted with a financial incentive of $10 (N=2,500) or 
$50 (N=2,500) and, separately with a convenient link to the county public vaccine 
appointment scheduling system highlighted for participants (N=5,000).  
 
The above treatments are designed to test the role of the following on vaccine take-up:  
 
• Financial incentives [N=5,000] vs. no financial incentives [N=5,000] 
o 2,500 will be randomized to a $10 incentive and 2,500 to a $50 incentive 
Sample Population:
Respondents from 
Aim 1 Survey 
(n=10,000)
Arm 1:
Control (n=2,500)
Arm 2: Emotional 
mesage/no 
information (n=2,500)
Arm 3:
Safety and 
Effectivness message 
(n=2,500)
Arm 4: 
Consequences of not 
vaccinating(n=2,500)
Figure. Randomization Design  

MOP version 1.1 dated 03Jun2021 
 
Page 43 of 69 
• Convenient scheduling link highlighted [N=5,000] vs. not [N=5,000] 
• Messaging [N=7,500] vs not [2,500]  
o Message type: emotion [N=2,500] vs. safety and effectiveness [N=2,500] 
vs. consequences of not vaccinating [N=2,500] 
• Race concordant [N=2,500] vs. race discordant messenger [N=2,500] 
• Gender concordant [N=2,500] vs. gender discordant messenger [N=2,500] 
These financial incentives of $10 or $50 are in line with the prior literature. For example, 
Bronchetti, Huffman, and Magenheim (2015) paid college students $30 for flu 
vaccinations in 2012. In addition, the $50 incentive is in-line with 2 hours of time at the 
minimum wage plus transportation costs. These incentives will be paid for with funding 
from J-PAL. While financial incentives for vaccination have faced some criticism (Largent 
and Miller 2021), prior work suggests small incentives can be quite effective (Bronchetti,  
Huffman, and Magenheim 2015). Given the urgency of the issue, we think it is crucial to 
test the effectiveness of financial incentives against other potential tools to improve 
vaccine take-up. We will also examine study impacts for subpopulations of interest 
identified in our baseline survey (e.g., groups at a high risk for adverse health outcomes 
or young adults who have high potential to be super-spreaders of COVID-19).  
 
We understand that the IRB is concerned that the incentives for vaccination are not equal 
across participants in the incentive arm. However, this approach is fairly standard in 
behavioral economic studies and has been approved by many IRBs at peer institutions. 
We highlight just a few such health studies here but there are dozens of other relevant 
examples: 
 
1. Alsan et al. (2019), which studied preventive health care among African 
American men, included a condition that randomly offered subjects incentives of 
$5 or $10 for selecting an influenza vaccination. These rewards were for 
intentions and were irrespective of actual vaccination. This study was approved 
by Stanford’s IRB. The protocol is available in Alsan et al. (2018) 
https://doi.org/10.1257/rct.2497-2.0.      
 
2. Bachireddy et al. (2019), which studied physical activity, compared 3 different 2-
week incentive programs with rewards for daily steps taken. Incentives varied 
from $0.10 per 10,000 steps to $2 per 10,000 steps, a 20-fold difference, over 
the 2-week intervention period. This study was approved by the IRB at University 
of Pennsylvania. 
 
3. Jones et al. (2019), which studied workplace wellness programs, offered varying 
levels of cash rewards ($100 vs. $200) to University of Illinois employees for 
completing a biometric screening and health risk assessment and an additional 
cash reward for completing each wellness activity ($25 vs. $75). The study was 
approved by the IRBs at University of Illinois Urbana Champaign, the University 
of Chicago and the National Bureau of Economic Research.  
 

MOP version 1.1 dated 03Jun2021 
 
Page 44 of 69 
We believe our work is in line with these and many other studies. It is also extremely 
policy relevant given concern about vaccine take-up. 
Our primary outcomes are vaccine take-up at one month and vaccine intentions at the 
time of survey. Secondary analysis will consider any vaccine take-up over longer time 
periods.  
 
Overview of procedures to be used for  
o Data collection tools 
• Baseline survey – Survey will be translated into Spanish and 
thus available in both Spanish and English for respondents 
to choose from in RedCAP. 
• Administrative data from CCRMC on COVID-19 tests, 
vaccinations, hospitalizations 
o Vaccination appointments in Contra Costa have been open to all age 
groups since March 30, 2021. 
o Research procedures will end once survey respondents have “received” 
an intervention.  The research team will not take part in any delivery of 
care. Thus, any actual take-up of vaccinations will be subject to normal 
CCHS standard of care.   
o Data Analysis  
 
• Randomization 
Randomization will be performed in the RedCAP system. In practice, the randomization 
will divide the sample into 60 possible conditions: 
 
1. Control [N=625] 
2. Control x $10 financial [N=312.5] 
3. Control x $50 financial [N=312.5] 
4. Control x link [N=625] 
5. Control x $10 financial x link [N=312.5] 
6. Control x $50 financial x link [N=312.5] 
7. No information/Emotional (language concordant) [N=625]   
8. No information/Emotional (language concordant) x $10 financial [N=312.5]  
9. No information/Emotional (language concordant) x $50 financial [N=312.5] 
10. No information/Emotional (language concordant) x link [N=625] 
11. No information/Emotional (language concordant) x $10 financial x link [N=312.5] 
12. No information/Emotional (language concordant) x $50 financial x link [N=312.5] 
13. Safety and effectiveness (race concordant, male) [N=156.25]  
14. Safety and effectiveness (race concordant, male) x $10 financial incentive 
[N=78.125]  
15. Safety and effectiveness (race concordant, male) x $50 financial incentive 
[N=78.125] 
16. Safety and effectiveness (race concordant, male) x link [N=156.25] 

MOP version 1.1 dated 03Jun2021 
 
Page 45 of 69 
17. Safety and effectiveness (race concordant, male) x $10 financial incentive 
[N=78.125]  
18. Safety and effectiveness (race concordant, male) x $50 financial incentive 
[N=78.125] 
19. Safety and effectiveness (race concordant, female) [N=156.25]  
20. Safety and effectiveness (race concordant, female) x $10 financial incentive 
[N=78.125] 
21. Safety and effectiveness (race concordant, female) x $50 financial incentive 
[N=78.125] 
22. Safety and effectiveness (race concordant, female) x link [N=156.25] 
23. Safety and effectiveness (race concordant, female) x $10 financial incentive x 
link [N=78.125] 
24. Safety and effectiveness (race concordant, female) x $50 financial incentive x 
link [N=78.125] 
25. Safety and effectiveness (race discordant, male) [N=156.25]  
26. Safety and effectiveness (race discordant, male) x $10 financial incentive 
[N=78.125]  
27. Safety and effectiveness (race discordant, male) x $50 financial incentive 
[N=78.125]  
28. Safety and effectiveness (race discordant, male) x link [N=156.25] 
29. Safety and effectiveness (race discordant, male) x financial incentive x link 
[N=78.125] 
30. Safety and effectiveness (race discordant, male) x financial incentive x link 
[N=78.125] 
31. Safety and effectiveness (race discordant, female) [N=156.25] 
32. Safety and effectiveness (race discordant, female) x $10 financial incentive 
[N=78.125]  
33. Safety and effectiveness (race discordant, female) x $50 financial incentive 
[N=78.125]  
34. Safety and effectiveness (race discordant, female) x link [N=156.25] 
35. Safety and effectiveness (race discordant, female) x $10 financial incentive x link 
[N=78.125] 
36. Safety and effectiveness (race discordant, female) x $50 financial incentive x link 
[N=78.125] 
37. Consequences of going unvaccinated (race concordant, male) [N=156.25]  
38. Consequences of going unvaccinated (race concordant, male) x $10 financial 
incentive [N=78.125]  
39. Consequences of going unvaccinated (race concordant, male) x $50 financial 
incentive [N=78.125]  
40. Conseqences of going unvaccinated (race concordant, male) x link [N=156.25] 
41. Conseqences of going unvaccinated (race concordant, male) x $10 financial 
incentive x link [N=78.125]  

MOP version 1.1 dated 03Jun2021 
 
Page 46 of 69 
42. Conseqences of going unvaccinated (race concordant, male) x $50 financial 
incentive x link [N=78.125]  
43. Conseqences of going unvaccinated (race concordant, female) [N=156.25]  
44. Conseqences of going unvaccinated (race concordant, female) x $10 financial 
incentive [N=78.125]  
45. Conseqences of going unvaccinated (race concordant, female) x $50 financial 
incentive [N=78.125]  
46. Conseqences of going unvaccinated (race concordant, female) x link [N=156.25] 
47. Conseqences of going unvaccinated (race concordant, female) x $10 financial 
incentive x link [N=78.125] 
48. Conseqences of going unvaccinated (race concordant, female) x $50 financial 
incentive x link [N=78.125] 
49. Conseqences of going unvaccinated (race discordant, male) [N=156.25]  
50. Conseqences of going unvaccinated (race discordant, male) x $10 financial 
incentive [N=78.125]  
51. Conseqences of going unvaccinated (race discordant, male) x $50 financial 
incentive [N=78.125]  
52. Conseqences of going unvaccinated (race discordant, male) x link [N=156.25] 
53. Conseqences of going unvaccinated (race discordant, male) x $10 financial 
incentive x link [N=78.125] 
54. Conseqences of going unvaccinated (race discordant, male) x $50 financial 
incentive x link [N=78.125] 
55. Conseqences of going unvaccinated (race discordant, female) [N=156.25] 
56. Consequences of going unvaccinated (race discordant, female) x $10 financial 
incentive [N=78.125]  
57. Consequences of going unvaccinated (race discordant, female) x $50 financial 
incentive [N=78.125]  
58. Consequences of going unvaccinated (race discordant, male) x link [N=156.25] 
59. Consequences of going unvaccinated (race discordant, male) x $10 financial 
incentive x link [N=78.125] 
60. Consequences of going unvaccinated (race discordant, male) x $50 financial 
incentive x link [N=78.125] 
 
• Protection of Data 
 
All identifiable data (names and either phone, email or physical address) will be 
available only for the purposes of 1) inviting study participants and 2) linking data. 
These data will be kept on secure password-protected USC Marshall servers and will 
only be accessible to Drs. Chang and Pramanik. Data sharing will occur through 
Marshall administered Box accounts. Box encrypts data at rest and in transit. Tom 
Chang will handle all matching of surveys and administrative data. 
 

MOP version 1.1 dated 03Jun2021 
 
Page 47 of 69 
Name and address information, not linked to any health information, will also be made 
available to the minimum number of research staff needed to invite participants to our 
surveys. 
 
Analytic datasets available to the research team will be limited datasets, without name, 
address, or any contact information. That data will be stored in a password protected 
shared study drive. 
 
References 
 
Alsan, Marcella, Owen Garrick and Grant Graziani. 2018. "The Oakland Men's Health 
Disparities Study." AEA RCT Registry. March 21. https://doi.org/10.1257/rct.2497-2.0  
 
Alsan M., Owen G. and Graziani G. 2019. "Does Diversity Matter for Health? 
Experimental Evidence from Oakland." American Economic Review 109(12): 4071-
4111. DOI: 10.1257/aer.20181446. 
 
Bachireddy C, Joung A, John LK, et al. 2019. Effect of Different Financial Incentive 
Structures on Promoting Physical Activity Among Adults: A Randomized Clinical 
Trial. JAMA Netw Open. 2019;2(8):e199863. doi:10.1001/jamanetworkopen.2019.9863  
 
Bronchetti ET, Huffman DB and Magenheim E. 2015. Attention, intentions, and follow-
through in preventive health behavior: Field experimental evidence on flu vaccination. 
Journal of Economic Behavior & Organization, 116, 270–291. DOI: 
10.1016/j.jebo.2015.04.003. 
 
Damon Jones, David Molitor, Julian Reif, 2019. What do Workplace Wellness Programs 
do? Evidence from the Illinois Workplace Wellness Study, The Quarterly Journal of 
Economics, 134(4): 1747-1791 qjz023, https://doi.org/10.1093/qje/qjz023 
 
Largent EA, FG Miller. 2021. Problems With Paying People to Be Vaccinated Against 
COVID-19 
JAMA Published online January 6, 2021. PMID: 33404585 
  

MOP version 1.1 dated 03Jun2021 
 
Page 48 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 49 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 50 of 69 
APPENDIX B: Informed Consent Documents and HIPAA Authorization 

MOP version 1.1 dated 03Jun2021 
 
Page 51 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 52 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 53 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 54 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 55 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 56 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 57 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 58 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 59 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 60 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 61 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 62 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 63 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 64 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 65 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 66 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 67 of 69 

MOP version 1.1 dated 03Jun2021 
 
Page 68 of 69 
 
 

MOP version 1.1 dated 03Jun2021 
 
Page 69 of 69 
APPENDIX C: MOP Modifications Log 
 
Date 
Modified 
Version # 
Section #s 
Page #s 
Brief Summary 
06/03/2021 
1.1 
Section 5.0; 
Appendix A; 
Appendix B 
Pages 4; 
48-53; 59-
63 
Increase in baseline survey 
incentive (from $5 to $25), to 
include IRB-approved protocol 
amendment and revised 
informed consent documents.