Lottery Incentive Nudges to Increase Influenza Vaccinations
Phase: NA
Conditions: Influenza, Vaccination, Health Promotion, Health Behavior
Sponsor: National Bureau of Economic Research, Inc.

Full Study Protocol  
GENERAL INFORMATION 
Lottery Incentive Nudges to Increase Influenza Vaccinations 
Christopher Chabris, Ph.D. 
cfchabris@geisinger.edu 
June 28, 2021 
IRB#: 2021-0484 
BACKGROUND AND OBJECTIVES
On average, 8% of the US population gets sick from flu each flu season (Tokars et al., 2018). 
Since 2010, the annual disease burden of influenza has included 9-45 million illnesses, 140,000-
810,000 hospitalizations, and 12,000-61,000 deaths (CDC 2020). The CDC recommends flu 
vaccination to everyone aged 6+ months, with rare exception; almost anyone can benefit from 
the vaccine, which can reduce illnesses, missed work, hospitalizations, and death (CDC 2019). In 
light of the poor health outcomes for patients and substantial costs realized by healthcare systems 
in treating flu and related complications, it is important to improve vaccination. 
Successful efforts to motivate patients to get vaccinated have included text message reminders 
(Milkman et al., 2021) and financial incentives (Lau et al., 2012), including a lottery for grocery 
gift certificates (Moran et al., 1996). This suggests that a lottery or raffle approach could be a 
worthwhile investment for healthcare systems, given the presumed emotional appeal of a very 
large reward and the anticipated savings in resource utilization costs of getting patients 
vaccinated. For example, Prospect Theory (Kahneman & Tversky, 1979) posits that people tend 
to overweight very low probabilities, which can enhance the relative salience of high payoffs in 
state lottery tickets with a type of “possibility effect” that is more pronounced as you approach 
0% likelihood with the same expected value (EV). However, high-stakes, low-probability lottery 
incentives and direct payments have not yet been attempted in the domain of immunizations. 
Within healthcare, research on high-stakes lotteries as an incentive appears limited to efforts to 
encourage survey responses—arguably not a behavior that personally benefits or affects 
participants in the same way as flu shots—and to examine this among clinicians, who are 
arguably accustomed to high compensation in a way that the average patient is not (Halpern et 
al., 2011).  
With the current study, we hypothesize that, among incentives of equal perceived EV or equal 
implementation costs, people are more likely to get vaccinated in response to lotteries with very 
high payoffs than to small certain cash payout or slightly higher-probability, more moderate 
payoffs. In particular, given the potential appeal of official state lottery tickets, we will include 
the promise of Pennsylvania scratch-off lottery tickets among our study arms. We further 

hypothesize that lotteries will outperform simple reminders (encouraging respondents to get the 
flu shot at their upcoming appointment) and the standard of care, representing the ambient 
healthcare system and public health campaigns to increase vaccination. 
Our specific aims are: 
1.
Evaluate whether providing any monetary incentive has a larger effect on vaccination rates than
simple reminders or no message.
2.
Evaluate whether lotteries are more effective than an equivalent-cost, certain $1 cash payout.
3.
Subject to available sample, evaluate which variations in prize-probability combinations and their
presentation are most effective, allowing us to model the curvature of the prize-probability
frontier.
PROCEDURES 
Research Design 
Patients at Geisinger who have a primary or specialty care appointment scheduled during the 
study period will be randomly assigned to a study arm. Outcomes during the 2021-2022 flu 
season will be compared between study arms. 
Study Population 
There are 2 separate populations to consider in the current study: the primary target population 
that will be randomized into an experimental condition and potentially contacted, and the 
household members of the primary population, who will not experience an intervention and will 
only be included for data analysis purposes. 
1) Primary target population
Inclusion Criteria: 
•
Aged 18 or older
•
Geisinger primary care provider (PCP) assigned
•
Upcoming appointment with PCP or select specialist who stocks and can administer the vaccine
during the study period
Exclusion Criteria: 
•
Has contraindications for flu vaccination
•
Note: we will respect opt-out communication preferences, but will include patients in the study if
they haven’t opted out of at least one communication modality employed
2) Secondary population (data analysis only): Household members
Inclusion Criteria: 

•
Household members of primary target population
•
Current Geisinger patient
Recruitment and Enrollment. All patients will be recruited and enrolled from Geisinger; 
MIT/NBER will not be involved in recruitment or enrollment. Patients meeting the primary 
target population criteria will be enrolled into one of 4-9 experimental conditions. Anticipated 
enrollment number for this primary target population is at least 39,224. Only patients from this 
primary target population will be contacted. However, health record data will be accessed to 
assess secondary outcomes for household members of this target population (expected N of 
80,000).  This brings the total planned enrollment to 119,224 participants. 
Detailed Study Procedures. 
Eligible subjects, as described above and in 2.2 Eligibility Criteria and 2.3 Age Limits, will be 
randomized into one of the intervention arms described below. At a minimum, assuming 
n=39,224, subjects will be randomized among four arms:  
#1: Pennsylvania Lottery Scratch-Off. PA Lottery $1 scratch-off ticket, with prizes 
including $5,000 (e.g., 1-in-500,000 chance). Odds of specific prizes will not be 
presented in the text, to mimic their absence on the actual ticket.  
#2: Certain Cash Payout. $1 cash incentive. 
#3: Reminder / Active Control Arm. Reminder message with no financial incentive. 
#4: No Treatment Control Arm. No additional contact beyond standard Geisinger flu shot 
communications. 
In August 2021, we will reevaluate the number of primary care and specialty appointments 
scheduled for the study period. If the projected number of appointments is sufficiently high and 
funding suffices, we may add up to five additional arms: 
#5: Additional Reminder / Active Control Arm. Reminder message with no financial 
incentive that uses “Reserved for you” messaging, which was the most effective 
messaging in our prior study (IRB Protocol #2021-0184). 
#6: High-Payoff Raffle Absent Upfront Odds. Entry to win $5,000. Odds will not be 
presented, to help equate perceptions between this raffle and Arm #1. 
#7: High-Payoff Raffle with Odds (Expected Value = $1). Entry to win $5,000 (1-in-
5,000 chance). 
#8: Low-Payoff Raffle with Odds (EV  = $1). Entry to win $50 (1-in-50 chance). 
#9: Moderate-Payoff Raffle with Odds (EV = $1). Entry to win $500 (1-in-500 chance). 
Based on our power calculation, we need at least 9,806 patients in each arm of the study. 
Therefore, for each additional 9,806 patients beyond our minimum projected sample size of 
39,224, we will add an additional arm. 
If our sample is sufficiently high to run any of Arms 6-9, we will obtain a Small Games of 
Chance (SGOC) license, and we have already initiated this process in anticipation of running 
these arms. 

Because this intervention is timed to the 2021–22 flu season, the intervention period will begin 
~9/1/21. In the active conditions, participants will receive SMS text messages, phone, and/or 
letter reminding them of their upcoming appointment (3 days prior to the appointment, and again 
1 day prior to the appointment) and telling them to ask for their flu shot. In the incentive 
conditions, to comply with state and federal regulations, participants will be told of a possible 
“cash bonus” and that they must reply to learn more about the incentive details. Participants will 
be informed that receipt of incentives is contingent upon documented vaccination in the 
electronic health record. We will also obtain data on cohabitants of the experimental subjects 
(anticipated n=80,000).   
The primary outcome will be whether the patient was vaccinated at the scheduled appointment, 
as recorded in the EHR. A secondary outcome will be flu vaccination within 7 days of receiving 
the intervention. Additional secondary outcomes will include pre-commitment via “Y” response 
to the initial text, vaccination during the subsequent flu season (to assess if behavior is 
maintained absent subsequent incentives); rates of flu diagnoses (both using the most rigorous 
biological tests, "high confidence flu", and using broader criteria that also include diagnosis 
codes and treatment information, "likely flu"), rates of flu complications and other relevant 
healthcare utilization outcomes such as emergency department visits, hospitalizations, and 
insurance claims (for an expected ~40% of participants who are members of Geisinger Health 
Plan). We will also measure rates of flu vaccination, diagnoses, and complications by non-
targeted household members who are likely to include older parents of patients and their 
spouses/partners. Finally, we will measure rates of COVID-19 vaccination in target participants 
and their household members. 
Data Sources 
In order to identify patients in the primary target population, Business Intelligence & Advanced 
Analytics (BIAA), working with Geisinger’s Phenomics and Clinical Data Core (PACDC), 
will provide a list of patients (including patient identifiers such as Medical Record Number) that 
meet our eligibility criteria. 
We will also obtain contact information (address, phone number, email address) to message 
participants in the primary target population from BIAA & PACDC. 
After the intervention is complete, we will obtain outcomes data for both target and household 
population groups from BIAA & PACDC. This will include data on patient flu vaccination, 
diagnosis of flu, diagnosis of flu-like symptoms, presence of flu-associated complications, 
hospital visits, emergency department visits, and COVID-19 vaccination status. This data set will 
additionally include patient characteristics that will be useful as analysis covariates, such as: age, 
patient primary care provider, and flu-related behavior and outcomes during previous flu 
seasons. 
STUDY DATA DETAILS 

Data Management Procedures and Confidentiality. All data will be electronic. Datasets with 
full identifying information will only be stored on Geisinger-managed, password-protected 
computers of the data brokers (Gail Rosenbaum and Amir Goren) for the purpose of linking 
datasets from different sources.  
A limited data set containing dates of service and ZIP codes will be shared with any non-
Geisinger collaborators (e.g., Additional Principal Investigator Doyle and his team) under a Data 
Use Agreement (pending approval) in compliance with HIPAA’s Privacy Rule, and using data 
security protocols reviewed and approved by the Geisinger Security Office, Privacy Office, and 
Information Technology department. 
After all data have been linked in a de-identified, coded file and analyzed, the datasets with 
identifiable information and any codes needed to link the identifiable information will be deleted. 
After the de-identified data have been fully analyzed, the de-identified dataset will be shared 
along with publications from this study. The deidentified data will not be destroyed or removed 
after any prespecified period of time has elapsed. We intend to permanently and securely archive 
the deidentified dataset at a research repository such as Open Science Framework (OSF) in order 
to be consistent with the best practices for open and reproducible science, as well as our 
obligation to the public as NIA-funded researchers. 
The only study team members who will have access to fully identifiable and protected health 
information will be Gail Rosenbaum and Amir Goren. Non-Geisinger collaborators will have 
access to a limited data set containing dates of service and ZIP codes. The remaining 
investigators will have access to the de-identified, coded data during data analysis. 
All data analysis will be conducted by Gail Rosenbaum, Amir Goren and our non-Geisinger 
Collaborators. We will analyze the data using standard behavioral research analysis methods, 
including computing bivariate correlations, using generalized linear models, using non-
parametric models for non-normally distributed data, and entering variables as independent 
predictors in regression models to attempt to predict desired outcomes.   
RESEARCH ACTIVITIES AT OTHER SITES 
Gesinger is the lead research team for this multi-site, single IRB study. Massachusetts Institute of 
Technology (MIT) and National Bureau of Economic Research (NBER) will cede to Geisinger’s 
IRB. Research activities conducted at MIT and NBER will be limited to data analysis, using 
limited data and de-identified data, as described above. MIT and NBER will not be involved in 
study recruitment/enrollment or intervention administration. 
BIBLIOGRAPHY 
Centers for Disease Control and Prevention. (2019). Who Needs a Flu Vaccine and When. 
https://www.cdc.gov/flu/prevent/vaccinations.htm (Oct 11). 
Centers for Disease Control and Prevention. (2020). Disease Burden of Influenza. 
https://www.cdc.gov/flu/ about/burden/index.html (Jan 10). 

_________. (2019). Who Needs a Flu Vaccine and When.  
https://www.cdc.gov/flu/prevent/vaccinations.htm (Oct 11). 
Halpern, S.D., Kohn, R., Dornbrand-Lo, A., Metkus, T., Asch, D.A., Volpp, K.G. (2011). 
Lottery-based versus fixed incentives to increase clinicians' response to surveys. Health Services 
Research 46(5),1663–1674. 
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. 
Econometrica 47, 263-291. 
Lau, D., Hu, J., Majumdar, S. R., Storie, D. A., Rees, S. E., & Johnson, J. A. (2012). 
Interventions to improve influenza and pneumococcal vaccination rates among community-
dwelling adults: a systematic review and meta-analysis. The Annals of Family Medicine, 10(6), 
538-546.
Milkman, K. L., Patel, M. S., Gandhi, L., Graci, H. N., Gromet, D., Ho, H., … Duckworth, A. L. 
(2021). A Mega-Study of Text-Based Nudges Encouraging Patients to Get Vaccinated at An 
Upcoming Doctor’s Appointment. Available at SSRN: https://ssrn.com/abstract=3780267 
Moran, W. P., Nelson, K., Wofford, J. L., Velez, R., & Case, L. D. (1996). Increasing influenza 
immunization among high-risk patients: education or financial incentive?. The American journal 
of medicine, 101(6), 612-620. 
Tokars, J.I., Olsen, S.J., & Reed, C. (2018). Seasonal incidence of symptomatic influenza in the 
United States. Clinical Infectious Diseases, 66, 1511–1518. PMID: 29206909