Community-based Behavioral Intervention to Increase COVID-19 and Influenza Vaccination for African American/ Black and Latino Persons: An Optimization Randomized Controlled Trial
Phase: PHASE2
Conditions: COVID 19, Influenza
Sponsor: New York University

2 
 
STUDY 
PROTOCOL 
Title: 
Community-based behavioral intervention to increase COVID-19 
and influenza vaccination for African American/Black and Latino 
persons: An optimization randomized controlled trial 
Short Title: 
COVID vax ORCT 
Field name: 
NCAP2 
Sponsor: 
National Institute on Minority Health and Health Disparities 
(NIMHD) 
Grant number 
R01MD019686 
IRB/UCAIHS Number: 
IRB-FY2025-9851 
Cayuse SP record 
24-0252  
Clinicaltrials.gov registration 
and date of first registration 
NCT06614361, September, 2024 
Silver grant number 
F8104 (25-43000-WSQPG-F8104) 
Protocol Date: 
October 3, 2024 
Amendment 1 Date: January 29, 2025 
Amendment 7 Date: 
Amendment 2 Date:  
Amendment 8 Date: 
Amendment 3 Date: 
Amendment 9 Date: 
Amendment 4 Date: 
Amendment 10 Date: 
Amendment 5 Date: 
Amendment 11 Date: 
Amendment 6 Date: 
Amendment 12 Date: 
 
Study Principal Investigator  
Marya Gwadz, PhD 
NYU Silver School of Social Work  
ERA Commons user name: mgwadz 
 
Co-Investigator 
Siyu Heng, PhD 
eRA Commons: SIYUHENG 
Co-Investigator 
Lalitha Parameswaran, MD, MPH 
eRA Commons: PARAML01 

3 
 
Co-Investigator 
Rauly Chero, LMHC 
eRA Commons: raulychero@nmic.org 
Co-Investigator 
Heather Gold, PhD 
eRA Commons: HTGOLD 
 
 
Project Director 
Samantha Serrano, MPH, MPA 
eRA Commons user name: SS13443.2022 
Cohort/data manager 
Dawa Sherpa, MPH 
eRA Commons user name: DS4578.2022 
Team members 
Paola Ramirez 
eRA Commons user name: PR2445.2022 
 
Consultants 
Angela Banfield, RN 
eRA Commons user name: 
ANGELABANFIELD.2023 
Scientific and Community Advisors 
 

4 
 
SPECIFIC AIMS 
Aim 1. Identify which of four candidate intervention components contribute meaningfully to 
improvement in the primary outcome, the receipt of the available COVID vaccine at the 6-month FU, 
with documentary evidence. Receipt of influenza vaccination is a 
secondary outcome.  
Aim 2. Identify mediators (e.g., perceived risk, altruism) and moderators 
(e.g., sociodemographic characteristics, medical distrust) of the effects 
of each component to better understand the components’ mechanisms 
of action and conditions under which they are most effective, to inform 
dissemination and future research and intervention programs. We will 
also explore perspectives in qualitative research (N=45). 
Aim 3: Build the most cost-effective intervention package(s) from the 
components found to be efficacious in Aim 1.  
CANDIDATE INTERVENTION COMPONENTS (SUMMARY) 
Core intervention: Health education session (30 minutes) 
Component A. Nurse-led shared decision-making (1 session, <60 
min.). 
Component B. Health & wellness interactive text message (TM) 
intervention (2 texts/week for 12 weeks, 1 text/week for 8 weeks, 20 
weeks total, 32 texts total) 
Component C: Prize at 6-month FU if vaccinated for COVID-19 with 
documentary evidence (gift bag with $25 gift card and low-cost items 
costing no more than $25), 3 reminder messages during intervention 
period 
Component D. Peer navigation (5 months duration, introductory 
meeting, bi-weekly personal phone calls, texts, or emails) 
SIGNIFICANCE   
The proposed four-year study responds to NOT-MD-23-008: Research 
to Address Vaccine Uptake and Implementation among Populations 
Experiencing Health Disparities. The multiphase optimization strategy 
(MOST) framework will be used to test a set of candidate intervention components. Then we will 
optimize a brief, efficient, and cost-effective behavioral intervention to increase COVID-19 and 
influenza vaccination for populations with high levels of vaccine hesitancy, namely, African 
American/Black and Latino (AABL) persons who are not up-to-date on vaccination. The new 
intervention developed in the study can be scaled up in community and outpatient health settings 
annually. While vaccine hesitancy is not new, COVID-19 is a novel and devastating disease and new 
solutions are needed. There is a scientific consensus that SARS-CoV-2 (the virus that causes COVID-
19) will continuously circulate in the human population, and new variants will evolve and spread, similar 
to influenza44,45. But, unlike influenza19, which has a seasonal predominance and is unlikely to be 
contracted repeatedly in a year, COVID-19 can be contracted several times each year, with potentially 
serious consequences46. Because SARS-CoV-2 mutates continuously and immunity from vaccination 
and infection diminishes over time47-49, the FDA and CDC have signaled that updated COVID-19 
vaccinations will be needed annually13-15,47-49. Further, annual influenza vaccination is more important 
than ever to reduce the impact of respiratory illnesses in the population and resulting burdens on the 
healthcare system18,50,51. But, COVID-19 and influenza vaccination rates are insufficient among AABL 
persons. This section focuses first on COVID-19, then influenza. 
Table 2. Abbreviations used in this 
proposal  
AABL 
African American/Black or Latino 
BE 
Behavioral economics 
BL 
Baseline (interview) 
CAB 
Community Advisory Board 
CAPI 
Computer-Assisted Personal 
Interview format 
CDC 
Centers for Disease Control and 
Prevention 
CFIR 
Consolidated Framework for 
Implementation Research 
ERCT 
Evaluation randomized controlled 
trial (a standard RCT) 
FU 
Follow-up (interview) 
IIT-ICM Intervention Innovations Team 
integrated conceptual model 
MI 
Motivational interviewing  
MOST 
Multiphase optimization strategy 
NMIC 
Northern Manhattan Improvement 
Corporation 
NYU 
New York University 
ORTC 
Optimization randomized 
controlled trial 
SARS-
CoV-2 
Severe acute respiratory 
syndrome coronavirus 2 
TM 
Text messages 

5 
 
COVID-19 case rates have varied and are currently climbing. The COVID-19 pandemic has shown a 
variable course. As of the summer of 2023, although surveillance is no longer comprehensive, 
numerous indicators show a marked rise in COVID-19 cases52, including the National Wastewater 
Surveillance System7,53. Rates of hospitalization and death from COVID-19 are also rising52,54. 
The effects of COVID-19 have been devastating. Over 6 million persons in the U.S. have been 
hospitalized, and more 1 million have died from COVID-19.52 The acute effects of COVID-19 include 
fatigue, shortness of breath, chest pain, abnormal heart rhythm, parosmia, and joint pain55. As many as 
1 in 5 persons with COVID-19 will develop Long COVID (signs, symptoms, and conditions that 
continue/develop after the initial COVID-19 infection).56,57,58. COVID-19 can lead to other serious 
adverse effects such as heart problems, blood clots, impaired fertility, and depression59. The risk of 
adverse effects of COVID-19, including serious illness, hospitalization, and death, increases with 
subsequent infections, particularly for those with co-morbidities60-62.  
Racial/ethnic disparities in COVID-19. There is a consensus that the COVID-19 pandemic exposed and 
intensified deep-rooted racial and economic inequities in health and health care63. Indeed, racial/ethnic 
inequities in COVID-19 incidence, morbidity, and mortality are marked64-66. For example, the NIH 
RECOVER study found racial/ethnic disparities in some symptoms of Long COVID and diabetes67. 
Importantly, although disparities have narrowed somewhat over the course of the pandemic, AABL 
persons are significantly more likely to be hospitalized with and to die from COVID-19 compared to 
White persons3,65,68.  
Mitigating the impact of COVID-19 illness. Staying up-to-date on COVID-19 vaccination is primary 
among the CDC’s recommendations for mitigating risk, along with testing and treatment, face 
coverings, and indoor air filtration69. Vaccination for COVID-19 is highly effective in reducing the 
incidence of illness, severe illness, hospitalization, disability, and death70,71. Vaccination partially 
mitigates the risk for Long COVID72,73. Equitable vaccine uptake is essential to reduce COVID-19-
related disparities in morbidity and mortality69. As of February 2023, 90% of adult Americans (and 
>80% of the AABL population) have received at least one COVID-19 vaccination dose and ~80% 
completed the primary series54,74. Yet, rates of those up-to-date on COVID-19 vaccination are 
unacceptably low. An updated COVID vaccine was approved in September, 202316,17 and data on 
uptake of the previous recommended vaccine (the bivalent booster) can be used to estimate the 
proportion that will receive this updated vaccine. Nationally only <30% received the bivalent booster, 
and disparities were notable: only 20% of Latino, 28% of African American/Black, and 31% of White 
persons who received the primary series also received the bivalent booster74,75. Thus, as of September 
2023 only a small proportion of AABL persons were up-to-date on COVID-19 vaccination, and at the 
same time they continue to experience disproportional adverse health effects from COVID-19. Without 
intervention, insufficient uptake of COVID-19 vaccines will prolong the social and economic 
repercussions of the pandemic on AABL communities76. As COVID-19 transitions to an ongoing health 
challenge, strategies to tackle its long-term threats are needed76. 
Current operational definition of being up-to-date on vaccines. As of 9/2023 updated COVID-19 
vaccines for 2023-24 has been approved, a monovalent COVID-19 vaccine with an XBB-lineage of the 
Omicron variant (Pfizer-BioNTech and Moderna vaccine for the XXB.1.5 variant)16,54. This updated 
vaccine is recommended regardless of whether the primary vaccine series or bivalent booster was 
received.69,75 Novavax is under review at the FDA77. CDC tracking on variants highlights the XXB.1.5 
variant is currently waning in the US, and the EG.5 variant is driving an uptick in COVID-1978. Because 
EG.5 is a subvariant or sub-strain of Omicron, the updated vaccine will provide protection against EG.5 
and similar Omicron sub-strains54.  
Racial/ethnic disparities in influenza and influenza vaccination. There are serious racial/ethnic 
disparities in influenza-associated hospitalization rates; they are highest in AABL populations 
compared to White populations, with AABL hospitalization rates ranging from 1.5 to 2.4 times the rates 
for White adults6,79. AABL people are also more likely to die from influenza than White people6. As 
noted above, annual vaccination for influenza is vital18. But, racial/ethnic disparities in influenza 

6 
 
vaccination rates are serious and persistent: Only 30-40% of adult AABL persons receive the influenza 
vaccine annually compared to >55% among White persons6,47. Thus, COVID-19 and influenza share 
important similarities, including the need for annual updated vaccinations, persistent racial/ethnic 
disparities in vaccine uptake, and resultant greater morbidity and mortality among AABL populations 
compared to White populations.  
MOST is an engineering-inspired framework for intervention development35. MOST allows for the 
testing the effects of individual intervention components in a fully powered experiment called an 
optimization randomized controlled trial (ORCT). Then, using findings from the ORCT, pre-specified 
criteria are applied to develop a multi-component intervention. These pre-specified criteria are called 
the optimization objective. For the proposed study, the optimization objective is to identify the most 
cost-effective combination(s) of components. The NIH has funded >250 ORCTs to date. We are highly 
experienced with the MOST framework, developing intervention components to reduce racial/ethnic 
disparities, and ORCTs36-38. MOST is described in detail below. 
Addressing racial/ethnic disparities. For over two decades, our research group, the Intervention 
Innovations Team (IIT), has studied the structural, social, cultural, and individual-level factors that 
impede health among AABL populations and created effective interventions to improve health 
outcomes36,81-85. Our studies are informed by a model we developed and have applied in numerous 
efforts called the IIT integrated conceptual model (IIT-ICM)85,86. The IIT-ICM combines critical race 
theory, harm reduction, and self-determination theory. As such, it centers the perspectives of AABL 
communities and addresses the role of systemic and structural factors, history, and culture in creating 
and shaping impediments to health behavior, supports any positive change, and guides toward health 
behavior without pressure or judgment, thereby supporting participants’ autonomy. The IIT-ICM taps 
into the importance of structural salience (health disparities have structural roots)126 and cultural 
salience (culture informs health decisions)127 in intervention content. The IIT-ICM has shown high 
acceptability and has contributed to intervention engagement and improved health outcomes in past 
studies with AABL populations36-38, 81-85,86. The IIT-ICM informs the intervention components to be 
tested in the proposed study, and the study is guided by a conceptual model that aligns with the IIT-
ICM (Fig. 1). 
Expertise in COVID-19 vaccination hesitancy. We are currently carrying out a study on COVID-19 
testing (U01MD017418) as part of the RadX-UP initiative, in collaboration with our community partner, 
the Northern Manhattan Improvement Corporation (NMIC). That study is a community-based ORCT to 
test intervention components and then optimize an intervention to increase COVID-19 testing for AABL 
frontline essential workers who are not up-to-date on COVID-19 vaccinations (U01MD017418). That 
study does not intervene to improve COVID vaccination rates, but we use quantitative and qualitative 
methods to study vaccination knowledge, attitudes, intentions, and uptake, and these data inform the 
present study, along with the larger empirical literature and a pilot study we carried out. In the COVID 
testing study (N=400/448 enrolled to date), half have at least one COVID vaccine dose but vaccine 
hesitancy is high. Concern about COVID-19 is low (63% not at all worried). Trust in COVID-19 vaccine 
safety is low (22%), and only 13% intend to get recommended booster shots. Impediments to COVID-
19 vaccination include cost (67%), lack of convenience (65%), safety concerns (59%), and questions 
about vaccine efficacy (55%). The main reason in favor of vaccination for COVID-19 would be to keep 
family safe (43%). Only 9% received an influenza vaccine in the past year. During the follow-up period, 
vaccination rates are persistently low (~2%), consistent with our focus on COVID testing in the study. 
We incorporate these findings into the description of vaccine barriers below.  
There is a growing literature on interventions to increase COVID‐19 vaccine uptake, and gaps that 
remain. Andreas et al.87 conducted a scoping review on COVID-19 vaccination research. They 
characterized promising interventions as focused on communication, education, multi-dimensions, and 
using incentives. A smaller number of studies focused on increased access. The studies’ outcome was 
generally attitudes (willingness to be vaccinated). Few studies examined COVID‐19 vaccine uptake87, 
focused exclusively on AABL populations, and almost none were carried out in Spanish87. Another 
review also found that behavioral outcomes are rarely used in COVID vaccine studies to date88. The 

7 
 
proposed study addresses these gaps, along with using the MOST framework. Interventions to 
increase influenza vaccination have been similar in scope and focus and included those based on 
communications research (improving knowledge, changing attitudes)89,90. The proposed study 
incorporates and extends the existing literature, as we describe throughout this proposal.   
Barriers to vaccination are largely comparable in AABL populations. AABL populations are among 
those with the lowest rates of COVID-19 and influenza vaccination but the highest rates of adverse 
consequences from COVID-19 and influenza. Past research has found that the major barriers to 
vaccination, described below, operate comparably for the two racial/ethnic groups28,31,32,91,92. It would 
be possible for the proposed study to focus on either African American/Black or Latino populations. 
But, for maximum reach and impact, the proposed study focuses on AABL populations together, as we 
and others have shown to be feasible and effective in past research37,81,85,93,94. To do so, the 
intervention components to be tested focus on the primary barriers that AABL populations experience 
to vaccination, and any staff-delivered components are flexible and individually tailored to respond to 
individual and cultural concerns. 
Factors that drive multi-level vaccine hesitancy in AABL populations. To be acceptable and effective, 
behavioral interventions must address the specific barriers to health behavior experienced by the 
population under study. This section describes the primary barriers that AABL populations experience 
to COVID-19 and influenza vaccination. Barriers to COVID-19 and influenza vaccination are similar for 
AABL33,34. But, in part because COVID-19 is new, AABL have greater barriers to COVID-19 vaccination 
than to influenza33,34. Thus, we focus mainly on COVID-19 in this section. There is a consensus that 
vaccine hesitancy is driven by complex barriers and impediments20 and must be considered in its 
systemic/structural and cultural contexts. We organize the impediments to vaccination for COVID-19 
and influenza using the theory of triadic influence 21. This is a social-cognitive theory, which aligns with 
the IIT-ICM, highlighting three streams of influence on health behavior: the individual, social, and 
structural. At the individual level, barriers include lack of knowledge of COVID, the belief that COVID-
19 is not a risk/not severe and vaccination is not necessary, along with medical and institutional 
distrust and counter-narratives about COVID-19 (sometimes called conspiracy theories)22,26-28,95-97. 
The lack of information regarding and level of distrust of the COVID-19 vaccination cannot be 
overstated, even among those who have received a dose of the vaccine98,99. Together, these attitudes 
and beliefs reduce behavioral intentions to be vaccinated for COVID-19100. Further, cognitive biases 
impede health behavior such as COVID-19 vaccination101. Cognitive biases are systematic thought 
processes resulting from the human mind's tendency to simplify information processing through a filter 
of personal experience and preferences102. Individuals typically show evidence of biases in judgment 
and reliance on heuristic ‘‘shortcuts’’ for health decisions.103,104 Relevant to vaccination are biases such 
as information salience (acting on the information that first comes to mind rather than on all the 
relevant information available)105-107, and present bias (the tendency to meet current desires or needs 
at the price of future beneficial outcomes). Interventions grounded in behavioral economics (BE) can 
circumvent cognitive biases to support health behavior108,109.  
At the social level of influence, peer norms discourage COVID-19 vaccination110-112.  Peer norms 
are the perceived informal rules that define acceptable and appropriate actions within a social group or 
community and that guide human behavior110. Altruism and a sense of collective responsibility are 
powerful social forces that can be harnessed to support vaccination113,114. At the structural level, 
access to COVID-19 vaccination can be challenging due to poor access to primary care, perceived 
discrimination, perceived cost, work schedules, and geographical inaccessibility (lack of transportation, 
inconvenient hours, lack of local vaccination sites)20. These are structural barriers. Barriers at these 
three levels of influence combine to reduce motivation and readiness to vaccinate for COVID-19 and 
influenza and thereby reduce vaccination rates in AABL populations (as noted above, barriers operate 
similarly for COVID-19 and influenza but barriers are more numerous and potent for COVID-19 than 
influenza33,34). Taken together, these barriers comprise multi-level vaccine hesitancy.  
The MOST framework has three phases: Preparation (developing promising components and a 
conceptual model), optimization (testing components, optimizing the intervention), and evaluation (if 

8 
 
appropriate, testing the new optimized intervention in a standard RCT, referred to here as an 
evaluation RCT [ERCT]). In collaboration with our community partner, NMIC, we have completed the 
proposed study’s preparation phase (developed components and a conceptual model) and the 
proposed study comprises an optimization phase. We propose to test four promising structurally and 
culturally salient candidate intervention components grounded in the literature, a pilot study we carried 
out in 2023, and our past research. Each component addresses a critical theoretical barrier or set of 
barriers to COVID-19 and influenza vaccination, and, to promote future scalability, some components 
are either brief or require only minimal staff time to implement. The primary outcome of the trial is being 
up-to-date on COVID-19 vaccination at the 6-month FU, that is, receiving the available vaccine dose. 
Influenza vaccine receipt is the secondary outcome. The candidate intervention components to be 
tested in the proposed study address barriers at the three levels of influence (Fig. 1). The components 
are: A) nurse-led shared decision-making, B) a text message intervention, C) prize for vaccination, and 
D) peer navigation to vaccination appointments. All participants will also receive the standard of care 
(health education, referrals). Components are described in the Approach. 
The main product from the proposed study will be an efficient and cost-effective optimized behavioral 
intervention or interventions to actively reach out to AABL communities annually and increase 
COVID-19 and influenza vaccination rates. The components are constructed as having key elements 
and core characteristics (that cannot be changed), and modules that can be updated as COVID-19 and 
influenza recommendations change, as we note in Approach. The study focuses on the large 
population of AABL persons who are not up-to-date on COVID-19 and influenza vaccinations but who 
have received at least one dose of the COVID-19 vaccination42,43. The majority of AABL persons in the 
U.S. (>80%) have at least one COVID-19 vaccine dose8. The modest population of AABL persons who 
have never been vaccinated for COVID-19 require different interventions than those not up-to-date but 
who have received at least one dose115. COVID-19 and influenza vaccine hesitancy are serious 
concerns among AABL populations42,43, and active outreach/community-based intervention approaches 
are urgently needed, as we propose here.  
 
 
PRIMARY 
OUTCOME
Up-to-date
on COVID-19
vaccination
SECONDARY
OUTCOME
Up-to-date on 
influenza 
vaccination
Beliefs about vaccination 
(perceived risk, safety, efficacy, 
trust, fears, and concerns),
Behavioral intentions
PROXIMAL MEDIATORS
CONCEPTUAL PATHWAY
A. Nurse-led
shared decision making
D. Peer navigation to 
vaccine 
appointments
INTERVENTION
Access to
vaccination
Fig. 1: Conceptual Model guiding the proposed study (rev)
Motivation 
and 
Readiness
B. Health and wellness
text messages
C. Prize at 6-month FU if 
vaccinated
Circumvent cognitive biases
Core (standard of care): 
Brief health
education, referrals
Knowledge of COVID-19 
disease and vaccine
Moderators
Age; Sex; Gender; SES; Race/Ethnicity; Vaccination history
Altruism & collective 
responsibility, Peer norms, 
Structural barriers
Optimize intervention
using cost-effectiveness
analysis 

9 
 
B. INNOVATION. NIH has called for research that “seek(s) to shift current research…paradigms or 
methodologies”116. 1) The MOST framework is a paradigm shift in intervention development methods. 
This project will be one of the first applications of the MOST framework to COVID-19 and influenza 
vaccine hesitancy research. 2) A second innovation entails the underlying approach taken in 
intervention components, which are both structurally and culturally tailored to the barriers that AABL 
persons experience to vaccination. E.g., The understandable causes of vaccine hesitancy are 
validated, and fear, distrust, and counter-narratives about COVID-19 and influenza vaccination are 
elicited/discussed, which foster project trustworthiness and participant engagement. Also, barriers to 
COVID-19 and influenza vaccination are addressed in new ways (e.g., informed by BE, peer 
navigation). 3) We will optimize the intervention for cost-effectiveness. This is innovative. 4) As 
described in the Approach, we will develop an implementation strategy manual to facilitate rapid scale-
up of the new optimized intervention in community-based and outpatient health care settings.  
C. APPROACH 
Overview. This is a four-year community-engaged study to be carried out by a collaborative research 
team at NMIC and NYU. For maximum scientific and real-world impact, it has both pragmatic (flexibility, 
closeness to real-world settings, real-world outcomes) and explanatory (e.g., examination of mediators) 
aspects. Study activities will take place in English and Spanish. The candidate intervention 
components will be tested in an efficient factorial experiment. Participants (N=560) are randomly 
assigned to an experimental condition. Follow-up (FU) assessments will be carried out at 3- and 6-
months post-baseline (BL), and a brief check of vaccination status will take place at 12-months post-
BL. The optimization objective is to create the most efficient combination(s) of components. A subset of 
participants will engage in qualitative research, and qualitative and quantitative results will be 
integrated, consistent with a concurrent parallel mixed methods design117. We will create an 
implementation manual to support uptake of the new optimized multi-component intervention(s) by 
community-based and outpatient health care settings. The proposed study has three phases: 1) Set-up 
and Refinement (6 mos.), 2) Implementation and Evaluation (34 mos.), and 3) Analysis, Decision-
making, and Dissemination (8 mos.). Study activities can take place in person or virtually (over Zoom). 
The proposed study has a strong scientific premise, rigor, and reproducibility. The proposed study is 
grounded in past research, strengthening its scientific premise. The study has rigor: we will use a 
factorial design, follow a detailed protocol (which will be published), randomly assign to an 
experimental condition, use fidelity and quality assurance checks, use PhenX Toolkit and NIH Public 
Health Emergency and Disaster Research Response (DR2) measures for assessment (as NOT-MD-
23-008 recommends)118,119, and empirically validate study endpoints. The study methods and 
components, and, therefore the optimized intervention, are designed to be reproducible; that is, this 
method should yield comparable results in a range of locations and settings.  
The Investigative Team. The study will be carried out by a multi-disciplinary team (psychology, 
medicine [infectious diseases], nursing, health services research/policy, social work, quantitative 
psychology, biostatistics, anthropology), diverse with respect to sex, gender, age, sexual orientation, 
language, race/ethnicity, and work setting. The team members have worked together on past projects. 
Dr. Gwadz (Principal Investigator) has led research to address racial/ethnic disparities in health for 
over two decades. Lalitha Parameswaran, MD, MPH (Co-Investigator) is a Clinical Assistant 
Professor at the NYU Grossman School of Medicine, a co-leader of the NYU Vaccine Treatment 
Evaluation Unit, and Co-I on Dr. Gwadz’s COVID testing study. She will review all materials for medical 
accuracy and provide guidance on medical aspects of vaccination and the health care system. Dr. 
Charles Cleland (Co-Investigator) will serve as the senior biostatistician. Dr. Cleland is experienced 
with designing and analyzing data from studies using MOST including factorial designs. Dr. Siyu Heng 
(Co-Investigator) is an Assistant Professor and statistician experienced with MOST and will carry out 
the primary analyses, with Dr. Cleland. Dr. Heather Gold (Co-Investigator) is an expert on cost-
effectiveness research, and will advise on the collection of cost data and cost-effectiveness analyses. 
Rauly Chero, LMHC (Co-Investigator) is the coordinator of wellness services at NMIC, a member of 
our COVID testing team, and an expert on COVID vaccination in the community context.  

10 
 
Consultants and Community Advisory Board (CAB). Angela Banfield, RN is a registered nurse, a 
COVID-19 Vaccine RN, a member of our COVID testing team, and expert in health education and 
interventions for AABL populations. She will advise on the nurse-led study component and study 
implementation generally. Dr. Robert Hawkins, a consultant to the proposed study, is an expert on the 
role of racism in health outcomes among AABL populations (including COVID-19). He will provide 
guidance on refining the intervention content and messaging, and the interpretation and integration of 
findings. Mx. Robin Freeman is an anthropologist with expertise in qualitative methods, COVID-19, 
and AABL populations. They will carry out the qualitative study component. A CAB will be active in all 
phases. CAB members are AABL persons and diverse with respect to age, sex, occupation, and 
vaccination experiences. They are highly knowledgeable about barriers to COVID-19/influenza 
vaccination in their communities, and potential solutions. The study is community-engaged in a number 
of respects (consistent with NOT-MD-23-008): It is co-led by Ms. Chero, a leader at our community 
partner, NMIC; a CAB is engaged in all phases; and participants will be recruited in their communities.  
The study’s primary outcome is receipt of the available COVID-19 vaccine, confirmed with 
documentary evidence (e.g., CDC COVID-19 Vaccination Record Card, MYVAXRECORDS, a doctor’s 
note or patient portal electronic health record note [e.g., from MyChart]). The available COVID-19 
vaccine will change during the study. CDC Advisory Committee on Immunization Practices guidelines 
will be followed. The secondary outcome is receipt of the available influenza vaccination50,120, 
confirmed with documentary evidence.  
The study eligibility criteria are:  
1. age 18-64 years 
a. Note: Those ages 65 and older have a different COVID-19 vaccination schedule than 
those ages 64 years and younger  
 
2. can engage in study activities in English or Spanish 
 
3. Black or African American or Latino/Hispanic race/ethnicity 
 
4. resides in New York City 
 
5. has received at least one dose of a COVID-19 vaccination in their lifetime, confirmed with 
documentary evidence (CDC vaccine card, My Vaccine Record, MyChart) 
 
6. not up-to-date on COVID-19 vaccination, defined as has not received the most recently 
available vaccine, confirmed with documentary evidence.  
a. As of October 3, 2024 the most recent vaccine is the “updated 2024-2025 COVID-19 
vaccine.” Documentary evidence includes: 
 
i. MY VACCINE RECORD (the preferred method) 
ii. MyChart record 
iii. Other electronic health record 
iv. Doctor’s note 
 
7. eligible to receive the COVID-19 vaccine (has not had anaphylaxis, myocarditis, pericarditis, or 
thrombosis with thrombocytopenia syndrome or other adverse effects deemed by a physician to 
be related to the COVID-19 vaccine121 by self-report 
 
8. if previously diagnosed with COVID-19, a minimum of 10 days has elapsed since last test 
positivity122,123 by self-report 
 

11 
 
9. is not up-to-date on influenza vaccination defined as not yet receiving the most recently 
available influenza vaccine. This will be documented where possible, but self-report is 
acceptable since not all influenza vaccines are reported to MY VACCINE RECORD. 
a. As of October 3, 2024 the most recent vaccine is the “updated 2024-2025 influenza 
vaccine.”  
 
10. has a phone that can be used for study participation and can receive text messages 
 
11. willing to follow NYU’s COVID protocols, if any are in place at that time (e.g., face coverings, 
not presenting to NYU when symptomatic with or testing positive for COVID) 
 
Those not found eligible will be told it is critical to discuss vaccination with their health care provider 
and given referrals. 
Seasonality. COVID-19 incidence waxes and wanes but it is not a seasonal disease124. In temperate 
climates such as the US, seasonal influenza epidemics occur mainly during winter. Updated 
vaccinations for influenza are available each fall, and there is less utility to receiving it after influenza 
season has ended. The proposed research study will take place year-round, and will address 
seasonality in influenza vaccine by taking into consideration that this vaccine may not be available or 
recommended for part of the year.  
Study design. We will carry out a factorial experiment to test four candidate intervention components. 
Each component has two levels: “off” (the component is not received) or “on” (the component is 
received). The design is comprised of 16 experimental conditions (24, Fig. 2), where each condition 
comprises a unique combination of intervention components. We wish to point out the design is not a 
16-arm RCT. Factorial experiments separate component effects, enabling estimation of the main effect 
contribution of each component. Factorial experiments can be economical compared to alternative 
designs, because they require substantially fewer participants to achieve the same goals35. For 
example, conducting four individual experiments using the RCT design, one for each component, 
would require N=2240 (560 participants per trial). Thus, the purpose and logical underpinnings of the 
factorial experiment are different from those of an RCT. The purpose of an RCT is a direct comparison 
of the efficacy of two or more versions of an intervention. By contrast, a factorial design never calls for 
a direct comparison of experimental conditions to see which one is best. Instead, the purpose is to 
identify which components show effectiveness. Efficiency comes from basing all estimated main effects 
on all 16 conditions in the factorial experiment. For example, the main effect of Component D will be 
estimated by comparing the mean outcome across Conditions 1-8 vs. across Conditions 9-16. All 
participants are included in the estimate of each main effect. Factorial experiments can have a small 
per-condition N and still achieve study aims if the total N is sufficient. All participants receive the core 
intervention. Condition 1 receives no other component. Conditions 2, 3, 5, and 9 receive one candidate 
component, Conditions 4, 6, 7, 10, 11, and 13 receive two candidate components, Conditions 8, 12, 
14, and 15 receive three components, and Condition 16 receives all components. 
 

12 
 
Rationale for the design. NOT-MD-23-008 calls for “appropriate intervention study designs.” The 
proposed study uses a factorial design and cost-effectiveness as its optimization objective. The 
intervention developed in the study will have a substantial evidence base, as we describe below, will 
be brief and efficient, and, if cost-effective, can be disseminated at the end of the proposed study. 
Indeed, urgent problems such as vaccine hesitancy call for innovative and timely solutions. We will also 
consider the potential benefits of a future ERCT.  
REDCap (Research Electronic Data Capture) will be used through all study phases. REDCap is a 
secure web-based application for building and managing online surveys and databases125. We have 
developed REDCap databases for past MOST studies and will adapt this database architecture for the 
proposed study. 
General description of the candidate intervention components. Candidate intervention components 
must show “promise” (i.e., acceptability, feasibility, and potential for effects) to be included in an 
ORCT35. In the proposed study, the core elements and key characteristics of the components are 
drawn from the existing literature (past effective interventions), past acceptable, feasible, and effective 
interventions created by our team, and a pilot study on COVID-19 and influenza vaccination we 
conducted. Our preliminary studies also yielded information on components or aspects of components 
that are challenging to implement for this public health problem or have insufficient acceptability (e.g., 
peer-to-peer education, frequent text messages about COVID, large financial incentives). The 
candidate components are designed to be as brief and efficient as possible, while still showing promise 
with respect to changing mediators and vaccination behavior. The candidate components are guided 
by manuals in English and Spanish. Manuals are comprised of a series of exercises and modules and 
are constructed to be interactive and engaging (core, Components A and D); Components B and C 
have written guidance but staff time to implement is minimal. Since recommendations for vaccination 
will continue to change, each component has key elements and core characteristics that cannot be 
modified, and modules that can be updated as COVID-19 and influenza recommendations evolve, in 
accordance with the CDC's Advisory Committee on Immunization Practices. The study interventionists 
do not make medical decisions. They serve as health educators. Medical decisions will be made by 
participants and their health care providers.  
Underlying approach in components. The IIT-ICM (described above) underlies the intervention 
components, which are designed to be structurally and culturally salient (as defined above)126,127. 
Aspects of structural and cultural salience include validating the understandable roots of vaccine 
hesitancy and institutional/medical distrust in all components, primary among them structural racism 
and structural inequality, and supporting participants’ own decisions about health behavior without 
pressure or judgment85,86. This is consistent with the motivational interviewing (MI) counseling 
approach85,86. In particular, past research has found that in the context of medical/institutional distrust, 
emphasizing autonomy and participants’ choices (and providing and respecting their choices) fosters 
meaningful engagement in intervention content85,86. A general MI approach is woven throughout 
components (e.g., roll with resistance, support autonomy), and each component has its own theoretical 
mediator(s) and behavior change techniques (e.g., navigation, BE). The study has a “stance” that 
COVID-19 and influenza vaccination are beneficial for individuals and communities but participants do 
not need to agree with the stance to engage in the study. The concept of vaccination as helpful to 
individuals and communities will be integrated into all components. It will be made clear that 
participants’ decisions about vaccination will be respected. This underlying approach and stance 
enhance components’ acceptability, feasibility, engagement, and effects85,86. Health information within 
the components will be drawn from the CDC and local health department and will be reviewed for 
medical accuracy by an expert (Dr. Parameswaran, a study Co-I). Candidate components address 
different theoretical mediators and are designed to be methodologically and/or theoretically distinct 
from each other. We will elicit (Component A) and/or attend to (core, Component D) and discuss fear, 
distrust, and counter-narratives about COVID-19, influenza, and vaccination, which are important 
aspects of project trustworthiness and building trust, as well as the participant’s behavior change 
process. In part this is because knowledge, attitudes, and emotions about COVID-19 cannot easily be 

13 
 
disentangled and must be addressed together85,86. AABL persons report there are few venues where 
they can discuss fears, distrust, and counter-narratives85,86. Although we address/validate medical and 
institutional distrust in components, we have not found in past research that levels of distrust change in 
response to interventions, even when the behavioral outcome is achieved81,140. Thus, we examine 
distrust as a moderator. The empirical evidence base for each component is provided below and draws 
from the existing literature, our past research, and a pilot study we conducted on COVID/influenza 
vaccination with AABL persons who are not up-to-date on vaccines. As noted above, because barriers 
to COVID-19 vaccination are greater than to influenza, components focus mainly on COVID-19 and 
secondarily on influenza33,34.  
Core session: Standard of care (health education, referrals, ~30 minutes). Knowledge about COVID-19 
vaccination is insufficient among AABL populations, it is challenging for people to access reliable 
health information (particularly as COVID-19 is less in the media and public health campaigns have 
largely waned), and health education is needed99,129. This component, which all participants receive, 
takes a health education approach and is interactive128. The first aim of the session is to introduce the 
participant to the study, its goals, and its ethos (e.g., supports autonomy, respects personal decisions) 
to promote engagement and retention. The second is to provide basic information regarding current 
COVID-19 and influenza vaccination guidelines and written referrals to no-cost vaccination sites. 
Modules include: Current CDC COVID-19 and influenza recommendations, the purpose of COVID-19 
and influenza vaccines at individual and population levels (e.g., prevent serious disease, 
hospitalization, and death), the importance of vaccination for community health, expected side effects, 
and written referrals to free vaccinations130. Because all participants receive the core session, its 
effects on the primary outcome are not assessed. It would be included in any optimized intervention. 
Theoretical target: knowledge.   
Component A. Nurse-led shared decision-making (1 session, <60 min.). AABL persons want and 
benefit from the opportunity to discuss COVID-19/influenza vaccination with and ask questions of 
trusted health professionals from AABL backgrounds, such as nurses24,31,32,131. We selected a shared 
decision making (SDM) approach for this component. SDM is a well-established and acceptable model 
with a promising empirical evidence base, particularly for affective, cognitive, and longer-term 
outcomes132,133. SDM is an approach to assist patient decision-making, where clinicians and patients 
share the best available evidence and where patients are supported to consider options and achieve 
informed preferences. Instead of assuming that health-related decisions should be guided by scientific 
consensus about effectiveness, SDM proposes that informed preferences, that is, what matters to 
patients, should play a major role in decision-making processes134,135. The principles of SDM are well 
documented and the common elements have been summarized in the literature136. In brief, SDM draws 
on and deepens the principles of patient-centered care134,135. SDM represents an important shift in the 
roles of both patients and clinicians. SDM recognizes the need to support autonomy by respecting both 
individual competence and interdependence on others. At its core, SDM is a process where decisions 
are made in a collaborative way, where trustworthy information is provided in accessible formats about 
a set of options, typically in situations where the concerns, personal circumstances, and contexts of 
patients play a major role in decisions134,135. Experts have called for SDM for COVID-19 vaccination 
decisions137-139. This component uses a practical and empirically-tested SDM model developed by 
Elwyn and colleagues called the three-step model comprised of team talk, option talk, and decision talk 
to guide a process of collaboration and deliberation134,135. Theoretical targets: perceived risk and 
necessity, behavioral intentions. 
Component B. Health & wellness interactive text message (TM) intervention (2 texts/week for 12 
weeks, 1 text/week for 8 weeks, 20 weeks total, 32 texts total, participants will be asked to 
acknowledge receipt). Cognitive biases interfere with health behavior101. This component is grounded 
in BE, a field that combines insights from psychology and economics105. Past studies that have taken 
or advocated for a BE approach for COVID-19108,141,142. The component’s main aims are to support 
participants staying actively engaged in the study and serve as a reminder to consider COVID-19 and 
influenza vaccinations. In BE terms, the TMs will “nudge” participants to stay engaged and toward 

14 
 
vaccination. TM interventions can be effective and cost-efficient for improving mental and physical 
health behavior, including related to COVID143-145. Our team is experienced with TM interventions 
grounded in BE83,84. Our past research indicates that when participants have ambivalent views on 
COVID, as expected in the proposed study, bi-weekly texts about COVID are not acceptable and can 
raise resistance, but a modest number of TMs on COVID are useful.  
Thus, TMs for the present study will provide content in four main areas:  
▪ 
General information about COVID-19 and influenza 
▪ 
Information about COVID-19 and influenza vaccination 
▪ 
Vaccination as protection for the community as well as individuals 
▪ 
General health information 
▪ 
Reminder about project activities and the need to let us know if contact information changes  
Messages are informative, instructional, motivational, and/or supportive. Overall, messages will take a 
gain (rather than loss) frame and will be clear and simple.  Links for more information will be provided 
where possible. Messages were developed with the CAB during the study’s preparation phase and 
informed by the health communications literature146. Health-related messages will be drawn from the 
CDC website and reviewed for medical accuracy.  
Specific structure of Component B: TMs are programmed into the Twilio program and sent 
automatically. The component is structured as follows: Participants first receive a brief orientation to 
the component (5 minutes) after the core session or baseline, and will put the study phone number into 
his/her/their phone. Then a test TM will be sent. TMs will be sent twice a week for 12 weeks, then 1 
text/week for 8 weeks (total TMs = 32). Participants will be asked to provide a numerical response to 
the TM to indicate that it was read and received. These responses will be used to generate points (1 
point/response), and participants will receive $1 for each point (maximum $32). This modest 
compensation will add interest to the component. “Gamification” principles will be used where 
participants will be sent a message at every 10 points such as  “Achievement unlocked!” This is 
referred to as a “badge.” Theoretical targets: circumvent cognitive biases.  
Component C. Prize at 6-month FU if vaccinated for COVID-19 with documentary evidence (a gift bag 
with $25 gift card and low-cost items costing no more than $25, 3 reminder messages sent at regular 
intervals during intervention period). The goal of this component is to provide the participant with a type 
of “deadline” for vaccination and nudge them toward vaccination. This component will be explained to 
participants after the core session, ideally, or baseline (<5 minutes). We will put the study phone 
number into his/her/their phone. Then a test text message will be sent. 
Timing will be: 1 month post-BL, 3 months post-BL, and 5 months post-BL. 
The three reminder messages were refined by the CAB.  
The reminder messages will read as follows: 
1.  “PRIZE REMINDER! Just a reminder that you are eligible to receive a prize at your 6-month follow-
up interview if you get vaccinated for COVID-19 before that time! Just bring some documentation of the 
vaccination. Let us know if you have questions at PHONE or NCAP2@nyu.edu”  
2. Don’t miss out! A prize is waiting for you. Just bring some documentation of the COVID vaccination. 
Let us know if you have questions at PHONE or NCAP2@nyu.edu 
3. Win a prize! Your 6-month follow-up can come with a reward – if get your COVID vaccine before 
then and bring proof to your visit. If you have any questions, reach out at PHONE or NCAP2@nyu.edu 
Documentation of COVID-19 vaccination can include information from the pharmacy that shows the 
date and type of vaccine, along with the other methods listed in this protocol. 
Participants who provide documentary evidence of vaccination at the 3-month follow-up can receive 
the prize then. 

15 
 
Theoretical targets: circumvent cognitive biases.  
Component D. Peer navigation (5 months duration, introductory meeting, twice-a-month personal 
phone calls, texts, or emails). Peer navigation is a flexible, individualized, and effective approach to 
guiding clients through the health care system and helping overcome barriers that prevent them from 
getting care156-158. In these models, services are provided by peers or near-peers who are from similar 
sociodemographic backgrounds as participants and who serve as role models. Peer navigation was 
selected as a means of tapping into the concepts of vaccination as an altruistic act, the importance of 
collective responsibility, and to challenge norms that AABL people do not commonly get updated 
COVID-19 and influenza vaccinations113,114,159. We are experienced with peer navigation 
interventions36,37. The component also includes strategies and supports to circumvent structural 
barriers around access to vaccination. It will have a different look and feel than Component B (e.g., will 
not provide information about COVID-19 but will focus on scheduling issues).  
The navigation period (5 months) starts at baseline. The introductory session will take place as soon as 
possible after baseline (0-2 weeks after baseline). It can be done virtually, preferably on Zoom because 
we are showing slides. If the introductory session is conducted late, the navigation period will be 
shortened.  
Weekly automated messages start after the introductory session. 
The component is structured as follows:  
▪ 
In-person session lasting < 60 min. (introductions; discussion of vaccination as a form of 
altruism and a contribution to the community with handouts; assessment of 
readiness/willingness to be vaccinated; determine if participant want an appointment at this time 
and make appointments or referrals to free vaccines as appropriate) 
 
▪ 
Bi-weekly (every two weeks) personal phone calls, texts, or emails (10 contacts initiated total 
maximum); e.g., “Checking in. I am here if you have any questions about scheduling an 
appointment.” 
 
▪ 
Automated message and in-person contact will be on alternate weeks where possible. 
Theoretical targets: altruism, collective responsibility, peer norms, structural barriers. 
SET UP AND REFINEMENT PHASE 
The main goals of the Set-up and Refinement Phase are to program the instrument to collect cost 
data, screening instrument, randomization table, and assessment batteries in REDCap. Assessment 
batteries will be tested for clarity and length. In collaboration with the CAB and scientific advisors, we 
will refine recruitment plans. We will carry out refinements of the candidate intervention components. 
We anticipate these to be minor (e.g., wording and length of exercises). To do so, we will bring 
together an intervention working group led by Dr. Gwadz and comprised of team members and the 
CAB as well as scientific advisors.  
REFINEMENT PHASE 
Recruitment – refinement phase 
Community Advisory Board (CAB) members will be recruited from our prior study of COVID testing 
interventions (NCAP) and the local community via social media and outreach. CAB members are not 
human subjects.  
Procedures and activities 
Activities can take place in-person at an NYU field site or virtually.  
CAB members will engage in the core intervention and 1-2 of the components (Components A, B, or 
D). This will include exploration on experiences with the component and how it can be improved in the 

16 
 
form of an in-depth qualitative cognitive interview161 and assessment of acceptability. Cognitive 
interviewing is a psychologically oriented method for empirically studying the ways in which individuals 
process and respond to intervention content161 (e.g., critique of each handout, exercise, or TM). We will 
ask participants if we can send TMs or call them for navigation after the meeting, as part of testing the 
component (Component B and D). We will collect minimal locator information (email address, phone 
number). This meeting will last two hours or less. Participants may engage in more than one pilot study 
session but will not receive the core session twice. 
Compensation – refinement phase 
Participants will receive $75 for each CAB meeting and funds for local public transportation if the 
meeting takes place in person.  
CONCLUSION OF REFINEMENT PHASE 
The feedback and acceptability ratings will be brought to the intervention working group for discussion 
and recommendations for refinement. If the intervention is not adequately refined at this point it will 
undergo another round of review and refinement. Materials will be professionally translated into 
Spanish using the back-translation method160. The activities are feasible: We will modify an existing 
REDCap MOST database, use measures from the PhenX Toolkit and NIH Public Health Emergency 
and Disaster Research Response (DR2) where possible (as recommended in NOT-MD-23-008)118,119, 
and have already created intervention manuals.  
Tracking program time and costs. Time and resources will be tracked following procedures developed 
by Dr. Gold (study Co-Investigator)39-41. The costs are those needed to implement the candidate 
intervention components. Research-related costs will not be included. We will capture the following: 
location costs, participant compensation, 
supplies, and personnel costs. The 
intervention components will have different 
durations, consist of different in-person or 
remote activities, and involve different 
personnel, categorized as either 
administrative staff or intervention facilitators. 
Administrative staff (e.g., research assistant, 
clinical supervisor) work on all program 
components. Intervention facilitators work only 
on the intervention, with their time estimated 
by tracking the number and duration of 
intervention sessions and contacts. We also 
will capture additional program time for 
training, supervision, and other related 
intervention time. For each unit of staff time, 
we will assign a dollar value based on the 
wage and fringe rate by occupation class of 
those who perform the activity using national 
average labor and fringe rates from the US 
Bureau of Labor Statistics202, as outlined in 
US guidelines. Sensitivity analysis will be conducted using actual staff wages and fringe.  
Implementation and Evaluation Phase 
We will use a hybrid recruitment plan with active outreach and passive strategies to efficiently the study 
population. The recruitment approach includes: 1) flyers describing the study in English and Spanish 
that can be used to directly recruit potential participants using ethnographic street recruitment methods 
(e.g., recruitment in parks and on the street)162; 2) ads placed in the medical research section of free 
newspapers (e.g., amMetro, Latino Impact), 3) ads disseminated on social media and Craig’s List, and  

17 
 
4) peer-to-peer recruitment 
(recruiters will be compensated 
$15 for each peer referral, 
maximum 10 
referrals/participant). In past 
studies, peer-to-peer 
recruitment was the most 
efficient and productive 
recruitment method38,83,163.   
Contact form. Potential 
participants will complete an 
online contact form linked to a 
QR code in recruitment 
materials or they can call the 
study directly. 
Screening for eligibility in two 
stages. First, verbal informed 
consent for screening is 
obtained following an IRB-
approved script followed by a 
brief (< 15 minute) structured 
screening interview using the 
Computer-Assisted Personal 
Interview (CAPI) format in 
REDCap to determine eligibility. 
Those found eligible will provide 
locator information. In a second 
step, documentary evidence of 
vaccination will be checked and 
logged. Screening can take 
place in recruitment venues, the 
study field site, or virtually (Fig. 
3). Those found eligible can 
enroll.  
Enrollment and study activities. 
Enrollment will take place at a 
study field site or virtually. 
Participants will provide 
electronic signed informed 
consent in REDCap (using the “eConsent” feature) for enrollment (or verbal consent if virtual), 
complete a more detailed locator form to facilitate longitudinal FU, and participate in a structured BL 
interview in REDCap. They will be randomly assigned to an experimental condition using REDCap. 
Participants have the opportunity to engage in the core session on the same day, or as soon as 
possible (ideally within 1-2 weeks). Then, other assigned components will be administered.  
Assessment battery in REDCap (Table 3). The BL will last ~30-45 minutes. Follow-up (FU) 
assessments will be carried out 3- and 6-months post-BL. FU interviews last ~25-35 minutes. BL 
interviews will assess the lifetime and past 3-month period, and the FUs will assess the period since 
the last interview118,119.  
At 12-months post-baseline we will check COVID-19 and influenza vaccination status by self-report 
and with documentary evidence for COVID-19. 
Table 3. Description of the measures in the assessment battery (Phenix toolkit and DR2 will be used where 
available) 
 
Moderating influences 
Socio-demographics, 
background factors 
Age, race/ethnicity, sex assigned at birth, occupation, sexual/gender minority status, 
immigration status, preferred language, ZIP code of residence, health status, co-
morbidities, racism and discrimination experiences, past COVID-19 diagnosis, 
vaccination, past influenza vaccination, children in the home  
Distrust of vaccination 
6-items assessed on a 5-point Likert-type scale 100,166 
Counter-narratives 
(conspiracy theories) 
6-items assessed on a 5-point Likert-type scale drawn from existing reliable scales 
168,169 
 
Individual/attitudinal influences 
Vaccine and COVID 
Knowledge  
10-item true-false questionnaire on aspects of vaccination, influenza, and COVID-19170   
Perceived risk  
Risk of COVID-19 and influenza, 2 items 
Necessity of vaccination Perceived necessity of vaccination for COVID-19 and influenza, 2 items 
Behavioral Intentions 
Vaccination scenarios assessed with 4 items  
Cognitive biases 
Measurement of perceived cognitive biases105, engagement in Component B 
Motivation/readiness 
Importance of COVID/influenza vaccination and confidence in ability to get vaccinated 
on a 1-100 scale.171  
 
Social influences 
Peer norms 
Assess peer norms re: COVID/influenza vaccination using 6 items172 
Altruism & collective 
responsibility 
6 items on COVID/influenza vaccination as an altruistic act/a collective responsibility 100  
 
Structural influences 
Structural barriers  
Perceived structural barriers: insufficient local sites, cost, perceived access  
 
PRIMARY OUTCOME 
COVID-19 vaccination 
Documentary evidence, date, name of vaccine(s), place of vaccination(s) 
 
SECONDARY OUTCOME 
Influenza vaccination 
Documentary evidence, date, name of vaccine, place of vaccination 
 
Intervention process domains  
Acceptability 
Client Satisfaction Questionnaire (YCSQ) –17 items 173 
Social harms 
Assessment of possible harms experienced in social, occupational domains.  
 
Feasibility/Intervention dose 
Intervention dose 
Attended core, Component A: Component B: number of TM responses; Component C: 
size of prize; Component D: number of navigation contacts completed and their 
characteristics/content 

18 
 
Qualitative interviews. Participants will be purposively sampled for maximum variability on the following 
criteria: age, sex, language (English or Spanish), race/ethnicity, and whether is up-to-date on COVID-
19 and/or influenza or not (N=45). We will explore participants’ experiences with and perspectives on 
the intervention components (including acceptability) and on barriers to and facilitators of vaccination. 
The qualitative interview will follow a semi-structured guide that has main questions linked to the 
conceptual model and probes. We will also attend to emergent topics. Interviews will be audio-recorded 
and transcribed. The qualitative sample size was determined following procedures outlined by Malterud 
called information power164. Greater sample heterogeneity as we expect here calls for larger samples; 
N=45 is a larger sample size. 
Compensation is provided using the Greenphire ClinCard system, a refillable debit card designed for 
research study reimbursement165.  The maximum compensation amount is $407. 
Participants can receive:  
▪ 
$60 for the BL 
▪ 
$25 for the core session 
▪ 
$25 for Component A 
▪ 
Up to $32 for Component B at conclusion of component  
▪ 
Prize for Component C (Gift bag with $25 and small items such as pens) 
▪ 
$25 for Component D 
▪ 
$50 for each FU interview and $20 for bringing documentary evidence of COVID-19 vaccination 
(regardless of whether vaccinated or not) or allowing us to help them access it or observe them 
accessing it; e.g., allowing us to help them check MY VACCINE RECORD or My Chart health 
records  
▪ 
$25 for 12-month check of vaccination with documentary evidence  
▪ 
$50 for the qualitative interview 
▪ 
funds for local public transportation 
Project staff will be highly trained, supervised, and diverse with respect to age, race/ethnicity, sex, 
sexual/gender minority status, socioeconomic status backgrounds, and language (bilingual in English 
and Spanish). Intervention components will be led by trained, supervised, health educators or 
clinicians. Component A will be led by a nurse/nursing student from an AABL background. Component 
D will be led by an AABL individual (peer/near-peer) with an understanding of the cultural/structural 
roots of vaccine hesitancy and background in health education.  
Retention/tracking. We are leaders in successful retention strategies163,174-177. The research team has 
over 20 years of experience with longitudinal studies, typically with follow-up rates of 85%-95% and 
intervention retention of 70-98%. Successful retention is a multifaceted effort requiring simultaneous 
strategies at the management, staff, project, and compensation levels. We estimate a >80% retention 
rate.  
In this study, COVID vaccination status is not protected health information (PHI) subject to HIPAA. 
Vaccination information can in some cases be classed as PHI and is covered by the HIPAA Rules. 
However, HIPAA only applies to HIPAA-covered entities – healthcare providers, health plans, and 
healthcare clearinghouses – and their business associates. NYU is not a HIPAA-covered entity. 
Nonetheless, it is not a HIPAA violation to ask for vaccination status. The participant can decide 
whether to provide the information to the study or not. The use of PHI involves no more than a minimal 
risk to the privacy of individuals in this study. See: https://www.hhs.gov/hipaa/for-professionals/special-
topics/research/index.html 
Fidelity, process ratings, and quality assurance. After each session or contact, the interventionist will 
complete process ratings in REDCap to assess fidelity to exercises in manuals178. Sessions (Core, 
Component A, D) will be audio-recorded and ~10% selected at random will be reviewed for quality 
assurance and supervision purposes, then destroyed. The intervention facilitators will attend monthly 
supervision.  

19 
 
Implementation strategy manual. We will draw on implementation science principles to support timely 
implementation of the optimized intervention(s)179. We will develop an implementation strategy manual, 
detailing requirements and recommendations for implementation in community-based and outpatient 
health settings. We will conduct meetings with staff stakeholders at NMIC and similar organizations. 
Guided by the Consolidated Framework for Implementation Research (CFIR)180, the meetings will 
explore potential barriers to and facilitators of implementation of the optimized intervention. The CFIR 
is one of the most widely utilized frameworks in implementation science180. It is a meta-theoretical 
framework that provides a standardized list of constructs found to be implementation determinants 
across five major domains180. We will probe for barriers and facilitators organized by CFIR domains: 
outer setting factors (i.e., funding, policies), inner setting (i.e., leadership, organizational readiness), the 
intervention (i.e., complexity), the people involved (i.e., self-efficacy) and the implementation process 
(i.e., planning). After each meeting, we will map group suggestions onto existing implementation 
strategies181. Results from Aims 1-3 will be integrated into the manual.  
Quantitative data analysis. Intent-to-treat analysis will be our primary analytic approach and exploratory 
analyses will examine complier average effects of intervention components182,183. Approaches to 
missing data will include full information maximum likelihood estimation184 and multiple imputation185. In 
sensitivity analysis, missing data will be treated as failure to achieve the desired outcome. If data are 
missing not at random (MNAR), we will employ sensitivity analysis, using selection186 or pattern 
mixture187,188 models. 
Aim 1: Identify which of four candidate intervention components contribute meaningfully to 
improvement in the primary outcome, receipt of the available COVID vaccine, with documentary 
evidence at the 6-month FU. The primary outcome for Aim 1 is receipt of the available COVID 
vaccine (documented) by the final follow-up. Logistic regression will estimate effects of components on 
the odds of receiving the available COVID vaccine. Intervention components will be effect-coded to 
estimate main effects and interactions of all four components (see Equation 1). The coefficient for an 
effect-coded main effect term (e.g., b1), multiplied by two and exponentiated, will estimate the effect of 
the component (e.g., Component A) on the odds of vaccination. Similarly, the coefficient for an effect-
coded interaction term, multiplied by two and exponentiated, will estimate interaction effects on the 
odds of vaccination. Similar logistic regression analyses will estimate effects on the secondary 
outcome. 
  
Aim 2: Identify mediators and moderators of the efficacy of each intervention component. To 
examine potential mediators, analysis for Aim 2 will use the potential outcomes framework189-191. This 
framework highlights assumptions needed to identify direct and indirect effects of interest: no 
unmeasured cofounders of the exposure (an intervention component) and outcome (vaccination) 
relation; no unmeasured confounders of the mediator and outcome relation; no unmeasured 
confounders of the exposure and mediator relation; and no measured or unmeasured confounders of 
the mediator and outcome relation affected by exposure. Since intervention components are randomly 
assigned, the key issue for the proposed study is addressing confounding of the relation between 
mediators and outcomes. Mediators measured at baseline will be included as confounders of the 
relation between follow-up mediators and vaccination. Sensitivity analysis will determine how the size 
of the correlation between error for the mediator model and error for the outcome model impacts 
inferences for direct and indirect effects. The total natural indirect effect (TNIE) and pure natural direct 
effect (PNDE) of each component will be estimated using the mediation R package192. The TNIE 
compares the outcome when subjects are exposed (e.g., receive a component), and the mediator 
varies as it would naturally under exposure, versus the outcome when subjects are exposed but the 
mediator varies as it would naturally in the absence of exposure (i.e., component not received). The 
PDNE compares participants at different levels of a component (e.g., On vs. Off) when a mediator is 
Equation 1 
log (
𝜋𝑖
1−𝜋𝑖) = 𝑏0 + 𝑏1𝑋𝐴+ 𝑏2𝑋𝐵+ 𝑏3𝑋𝐶+ 𝑏4𝑋𝐷+ 𝑏5𝑋𝐴∗𝐵+ 𝑏6𝑋𝐴∗𝐶+ 𝑏7𝑋𝐴∗𝐷+ 𝑏8𝑋𝐵∗𝐶+ 𝑏9𝑋𝐵∗𝐷+ 𝑏10𝑋𝐶∗𝐷+
𝑏11𝑋𝐴∗𝐵∗𝐶+ 𝑏12𝑋𝐴∗𝐵∗𝐷+ 𝑏13𝑋𝐴∗𝐶∗𝐷+ 𝑏14𝑋𝐵∗𝐶∗𝐷+ 𝑏15𝑋𝐴∗𝐵∗𝐶∗𝐷
   

20 
 
blocked. Potential moderator effects will be examined by adding interaction terms to the model 
described for Aim 1. We will include sociodemographics and other covariates (e.g., medical distrust) 
and explore the interactions of these variables with intervention components. When interaction effects 
are detected, we will estimate the simple main effects of the intervention component. Identified 
moderators will inform dissemination and future adaptive interventions193. 
Power Analysis. For the primary outcome, vaccination with the available COVID vaccine, we used 
PASS 2023194 to estimate the sample size needed for main effects of intervention components 
corresponding to odds ratios of 1.9 in logistic regression, given α=.05. Assuming a 40% chance of 
vaccination by the final follow-up overall, a sample size of n=448 (n=28 in each of 16 conditions) 
provides 87% power to detect an odds ratio of 1.9. To account for attrition of up to 20%, we propose a 
total sample size of 560 participants (n=35 in each of 16 conditions), ensuring complete data for at 
least n=28 per condition. When the main effect of an intervention component on a continuous mediator 
is estimated in a linear model, the sample size provides 80% power to detect a small standardized 
mean difference (d = .27). Moderator effects corresponding to an odds ratio of OR=1 in one subgroup 
and OR=4 in another can be detected with 83% power if subgroups sizes are roughly equal. To 
estimate the size of a mediated effect that can be detected given the proposed sample size, we use 
the approach described by Vittinghoff and colleagues195 as implemented in PASS 2023. Given a 
substantial correlation between an intervention component and a hypothesized mediator (r=.50), an 
odds ratio of 1.40 can be detected with >80% power for the effect of a mediator on the vaccination 
outcome. 
Qualitative data analysis and integration. Qualitive analyses will focus on experiences with the 
candidate intervention components, to inform intervention implementation and future research. Coding 
and analytic methods of qualitative data will employ a directed content analysis approach196. Analyses 
will begin with pre-determined codes based on the conceptual model (i.e., a “start list”), which will be 
expanded based on emergent findings. We will attend to trustworthiness and rigor. Integration of 
qualitative/quantitative results will use the joint display method197. A joint display is a state-of-the-art 
visual tool to integrate data sources198-200.  
Aim 3: Build the most cost-effective intervention package(s) from the components found to be 
efficacious in Aim 1. We will estimate the total cost of each experimental condition, rank them low to 
high, and remove any due to dominance (higher cost with lower effectiveness). Then, we will calculate 
the incremental cost-effectiveness ratio comparing each condition with the next lower cost condition, 
[(cost1 – cost2)/(outcome1 – outcome2)], to yield for each component the incremental cost per 
participant receiving vaccination and incremental cost per percentage point change in vaccination rate 
(from Aim 1); we also will explore the incremental cost per participant.  
The decision-making process will be led by Dr. Gwadz and include the study Co-Investigators (Cleland, 
Parameswaran, Heng, Gold, Chero) and consultants35. The decisions will be reviewed with the CAB 
before being finalized. STEP 1: Carry out Aim 1 analyses, which detail the effectiveness of each 
component (% vaccinated) as well as interaction effects. This allows us to predict % vaccinated for 
each experimental condition, adjusting for any covariates. STEP 2: Carry out Aim 2 analyses, 
qualitative analyses, and the joint display to determine how components work and under what 
conditions, along with acceptability and feasibility. These data will be integrated into the 
implementation manual. STEP 3: Use cost estimates and incremental cost-effectiveness ratios from 
Aim 3 and rank-order experimental conditions by incremental cost-effectiveness ratio. STEP 4: Carry 
out same steps for influenza and consider that in decision-making. FINAL STEP: There may be several 
good options for improving outcomes at different levels of cost. We will create an efficient frontier for 
these experimental conditions using methods developed by our colleagues at NYU called decision 
analysis for intervention value efficiency (DAIVE), a decision-making framework for use in MOST201. 
DAIVE is a user-friendly method for evaluating interventions based on a wide variety of decision-maker 
preferences, including those that involve multiple outcome variables. We have used DAIVE in a past 
study201. Using data from the previous steps as inputs, DAIVE will be used to create an efficient frontier 
of interventions for COVID alone, influenza alone, and COVID and influenza together (where we can 

21 
 
weight the relative importance of COVID and influenza vaccination equally or prioritize one based on 
prevalence and harm at the time of our cost-effectiveness analysis). The final decisions and 
recommendations and the implementation manual will be disseminated widely and the PI will provide 
technical assistance to interested parties. Ultimately, end users in community-based and health 
settings can decide which intervention package is most feasible for their setting.  
The future optimized intervention can adapt to changing vaccine recommendations. The candidate 
components include modules that can be revised as vaccines are updated and/or if CDC 
recommendations change. Thus, the intervention produced will keep current with the recommendations 
of the CDC's Advisory Committee on Immunization Practices. Dr. Gwadz’s team has a website and the 
most up-to-date intervention manuals, guidance, and implementation manual will be located there, 
along with updates pushed out to users.  
 
LITERATURE CITED 
 
1. 
Centers for Disease Control and Prevention. COVID-19: About COVID-19. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/index.html. 
2. 
Hill L, Artiga S. COVID-19 Cases and Deaths by Race/Ethnicity: Current Data and Changes 
Over Time. 2022. https://www.kff.org/racial-equity-and-health-policy/issue-brief/covid-19-cases-and-
deaths-by-race-ethnicity-current-data-and-changes-over-time/. 
3. 
Centers for Disease Control and Prevention. Health disparities: Provisional death counts for 
COVID-19. 2023. 
https://www.cdc.gov/nchs/nvss/vsrr/covid19/health_disparities.htm#RaceHispanicOrigin. 
4. 
Lundberg DJ, Wrigley-Field E, Cho A, et al. COVID-19 Mortality by Race and Ethnicity in US 
Metropolitan and Nonmetropolitan Areas, March 2020 to February 2022. JAMA Netw Open 2023; 6(5): 
e2311098. 10.1001/jamanetworkopen.2023.11098 PMC10155069 
5. 
Luck AN, Preston SH, Elo IT, Stokes AC. The unequal burden of the Covid-19 pandemic: 
Capturing racial/ethnic disparities in US cause-specific mortality. SSM - Population Health 2022; 17: 
101012. https://doi.org/10.1016/j.ssmph.2021.101012  
6. 
Black CL, O’Halloran A, Hung M-C, et al. Vital signs: influenza hospitalizations and vaccination 
coverage by race and ethnicity—United States, 2009–10 through 2021–22 influenza seasons. 
Morbidity and Mortality Weekly Report 2022; 71(43): 1366.  
7. 
Centers for Disease Control and Prevention. Wastewater Surveillance. 2023. 
https://www.cdc.gov/nwss/wastewater-surveillance.html. 
8. 
Centers for Disease Control and Prevention. COVID-19 Vaccination Coverage and Vaccine 
Confidence Among Adults: COVIDVaxView. 2023. https://www.cdc.gov/vaccines/imz-
managers/coverage/covidvaxview/interactive/adults.html. 
9. 
Grohskopf LA. Prevention and control of seasonal influenza with vaccines: recommendations of 
the Advisory Committee on Immunization Practices—United States, 2023–24 influenza season. 
MMWR Recommendations and Reports 2023; 72.  
10. 
Kini A, Morgan R, Kuo H, et al. Differences and disparities in seasonal influenza vaccine, 
acceptance, adverse reactions, and coverage by age, sex, gender, and race. Vaccine 2022; 40(11): 
1643-54.  
11. 
Webb Hooper M, Nápoles AM, Pérez-Stable EJ. No populations left behind: vaccine hesitancy 
and equitable diffusion of effective COVID-19 vaccines. Journal of General Internal Medicine 2021; 36: 
2130-3.  

22 
 
12. 
Yang Z-R, Jiang Y-W, Li F-X, et al. Efficacy of SARS-CoV-2 vaccines and the dose–response 
relationship with three major antibodies: a systematic review and meta-analysis of randomised 
controlled trials. The Lancet Microbe 2023.  
13. 
U. S. Food and Drug Administration. Future Vaccination Regimens Addressing COVID-19. 
2023. https://www.fda.gov/media/164699/download (accessed 9/12/23). 
14. 
Advisory Committee on Immunization Practices. ACIP Presentation Slides: Coronavirus 
Disease 2019 (COVID-19) Vaccines. 2023. https://www.cdc.gov/vaccines/acip/meetings/slides-2023-
09-12.html (accessed 9/12/2023). 
15. 
Cohen A. New CDC director urges COVID-19 booster as cases increase. 2023. 
https://rollcall.com/2023/08/31/new-cdc-director-urges-covid-19-booster-as-cases-increase/ (accessed 
9/12/23). 
16. 
U.S. Food and Drug Administration. FDA Takes Action on Updated mRNA COVID-19 Vaccines 
to Better Protect Against Currently Circulating Variants. Food & Drug Administration; 2023. 
17. 
Centers for Disease Control and Prevention. Interim Clinical Considerations for Use of COVID-
19 Vaccines in the United States. 2023. https://www.cdc.gov/vaccines/covid-19/clinical-
considerations/interim-considerations-us.html (accessed 9/15/2023). 
18. 
GSK. SimpliFlu: Know the facts about flu vaccination during the COVID-19 pandemic. 2023. 
https://www.gskflu.com/flu-and-covid-19/. 
19. 
Centers for Disease Control and Prevention. 2022-2023 Preliminary In-Season Burden 
Estimate. 2023. https://www.cdc.gov/flu/about/burden/preliminary-in-season-estimates.htm. 
20. 
Pena JM, Schwartz MR, Hernandez-Vallant A, Sanchez GR. Social and structural determinants 
of COVID-19 vaccine uptake among racial and ethnic groups. J Behav Med 2023; 46(1-2): 129-39. 
10.1007/s10865-023-00393-y PMC9846662 
21. 
Flay BR, Snyder F, Petraitis J. The theory of triadic influence. In: DiClimente RJ, Kegler MC, 
Crosby RA, eds. Emerging theories in health promotion practice and research. New York, NY: Jossey-
Bass; 2009: 451-510. 
22. 
Azarpanah H, Farhadloo M, Vahidov R, Pilote L. Vaccine hesitancy: evidence from an adverse 
events following immunization database, and the role of cognitive biases. BMC Public Health 2021; 
21(1): 1686. 10.1186/s12889-021-11745-1 PMC8444164 
23. 
Shearn C, Krockow EM. Reasons for COVID-19 vaccine hesitancy in ethnic minority groups: A 
systematic review and thematic synthesis of initial attitudes in qualitative research. SSM-Qualitative 
Research in Health 2022: 100210.  
24. 
Momplaisir F, Haynes N, Nkwihoreze H, Nelson M, Werner R, Jemmott J. Understanding 
drivers of COVID-19 vaccine hesitancy among Blacks [published online ahead of print February 9, 
2021]. Clin Infect Dis 2021; 10.  
25. 
Izeogu C, Gill E, Van Allen K, Williams N, Thorpe LE, Shelley D. Attitudes, perceptions, and 
preferences towards SARS CoV-2 testing and vaccination among African American and Hispanic public 
housing residents, New York City: 2020–2021. PloS one 2023; 18(1): e0280460.  
26. 
Morgan KM, Maglalang DD, Monnig MA, Ahluwalia JS, Avila JC, Sokolovsky AW. Medical 
mistrust, perceived discrimination, and race: A longitudinal analysis of predictors of COVID-19 vaccine 
hesitancy in US adults. Journal of racial and ethnic health disparities 2022: 1-10.  
27. 
McFadden SM, Demeke J, Dada D, et al. Confidence and hesitancy during the early roll-out of 
COVID-19 vaccines among black, Hispanic, and undocumented immigrant communities: A review. 
Journal of urban health : bulletin of the New York Academy of Medicine 2022; 99(1): 3-14.  

23 
 
28. 
Khubchandani J, Macias Y. COVID-19 vaccination hesitancy in Hispanics and African-
Americans: A review and recommendations for practice. Brain, behavior, & immunity-health 2021; 15: 
100277.  
29. 
Njoku A, Joseph M, Felix R. Changing the Narrative: Structural Barriers and Racial and Ethnic 
Inequities in COVID-19 Vaccination. Int J Environ Res Public Health 2021; 18(18). 
10.3390/ijerph18189904 PMC8470519 
30. 
Lin Q, Paykin S, Halpern D, Martinez-Cardoso A, Kolak M. Assessment of structural barriers 
and racial group disparities of COVID-19 mortality with spatial analysis. JAMA network open 2022; 
5(3): e220984-e.  
31. 
Kricorian K, Turner K. COVID-19 Vaccine Acceptance and Beliefs among Black and Hispanic 
Americans. PLoS One 2021; 16(8): e0256122. 10.1371/journal.pone.0256122 PMC8384224 
32. 
Kricorian K, Civen R, Equils O. COVID-19 vaccine hesitancy: misinformation and perceptions of 
vaccine safety. Human Vaccines & Immunotherapeutics 2022; 18(1): 1950504.  
33. 
Lennon RP, Block Jr R, Schneider EC, et al. Underserved population acceptance of 
combination influenza-COVID-19 booster vaccines. Vaccine 2022; 40(4): 562-7.  
34. 
Bazan IS, Akgün KM. Focus: Health Equity: COVID-19 Healthcare Inequity: Lessons Learned 
from Annual Influenza Vaccination Rates to Mitigate COVID-19 Vaccine Disparities. The Yale journal of 
biology and medicine 2021; 94(3): 509.  
35. 
Collins LM. Optimization of behavioral, biobehavioral, and biomedical interventions. Cham: 
Springer International Publishing 2018; 10(1007): 978-3.  
36. 
Gwadz MV, Collins LM, Cleland CM, et al. Using the multiphase optimization strategy (MOST) 
to optimize an HIV care continuum intervention for vulnerable populations: a study protocol. BMC 
Public Health 2017; 17(1): 383. 10.1186/s12889-017-4279-7 PMC5418718 
37. 
Gwadz M, Cleland CM, Lizardo M, et al. Using the multiphase optimization strategy (MOST) 
framework to optimize an intervention to increase COVID-19 testing for Black and Latino/Hispanic 
frontline essential workers: A study protocol. BMC Public Health 2022; 22(1): 1235. 10.1186/s12889-
022-13576-0 PMC9210062 
38. 
Cleland CM, Gwadz M, Collins LM, et al. Effects of Behavioral Intervention Components for 
African American/Black and Latino Persons Living with HIV with Non-suppressed Viral Load Levels: 
Results of an Optimization Trial. AIDS and behavior 2023: 1-18. 10.1007/s10461-023-04086-0 
PMC10211286 
39. 
Gold HT, McDermott C, Hoomans T, Wagner TH. Cost data in implementation science: 
categories and approaches to costing. Implement Sci 2022; 17(1): 11. 10.1186/s13012-021-01172-6 
PMC8796347 
40. 
Gold HT, Pirraglia E, Huang ES, et al. Cost and healthcare utilization analysis of culturally 
sensitive, shared medical appointment model for Latino children with type 1 diabetes. Pediatr Diabetes 
2021; 22(5): 816-22. 10.1111/pedi.13218 PMC8627428 
41. 
Makarov DV, Feuer Z, Ciprut S, et al. Randomized trial of community health worker-led decision 
coaching to promote shared decision-making for prostate cancer screening among Black male patients 
and their providers. Trials 2021; 22(1): 128. 10.1186/s13063-021-05064-4 PMC7876807 
42. 
Hallgren E, Moore R, Purvis RS, et al. Facilitators to vaccination among hesitant adopters. 
Human Vaccines & Immunotherapeutics 2021; 17(12): 5168-75.  
43. 
Willis DE, Selig JP, Andersen JA, et al. Hesitant but vaccinated: assessing COVID-19 vaccine 
hesitancy among the recently vaccinated. J Behav Med 2023; 46(1-2): 15-24. 10.1007/s10865-021-
00270-6 PMC8760868 

24 
 
44. 
Cubanski J, Kates J, Tolbert J, Guth M, Pollitz K, Freed M. What happens when COVID-19 
emergency declarations end? Implications for coverage, costs, and access. Kaiser Family Foundation 
2023.  
45. 
Centers for Disease Control and Prevention. End of Public Health Emergency. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/your-health/end-of-phe.html. 
46. 
Centers for Disease Control and Prevention. Reinfection. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/your-health/reinfection.html. 
47. 
Control CfD, Prevention. Estimates from the Behavioral Risk Factor Surveillance System 
(BRFSS), National Immunization Survey (NIS), and the National 2009 H1N1 Flu Survey (NHFS). 2012. 
48. 
Townsend JP, Hassler HB, Dornburg A. Infection by SARS‐CoV‐2 with alternate frequencies of 
mRNA vaccine boosting. Journal of medical virology 2023; 95(2): e28461.  
49. 
U.S. Department of Health and Human Services. COVID-19 Vaccines. 2023. 
https://www.hhs.gov/coronavirus/covid-19-vaccines/index.html (accessed 9/13/2023). 
50. 
Centers for Disease Control and Prevention. Flu Shot. 2023. 
https://www.cdc.gov/flu/prevent/flushot.htm. 
51. 
Centers for Disease Control and Prevention. Flu and COVID-19 Vaccine Coadministration. 
2023. https://www.cdc.gov/flu/prevent/coadministration.htm. 
52. 
Centers for Disease Control and Prevention. COVID data tracker. 2023. 
https://covid.cdc.gov/covid-data-tracker/#datatracker-home. 
53. 
Biobot. The Biobot Network of Wastewater Treatment Plants. 2023. https://biobot.io/data/. 
54. 
Centers for Disease Control and Prevention. Updated COVID-19 Vaccine Recommendations 
are Now Available. 2023. https://www.cdc.gov/respiratory-viruses/whats-new/covid-vaccine-
recommendations-9-12-2023.html. 
55. 
Centers for Disease Control and Prevention. Symptoms of COVID-19. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html. 
56. 
Centers for Disease Control and Prevention. Long COVID. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/long-term-effects/index.html. 
57. 
U.S. Department of Health and Human Services. Guidance on “Long COVID” as a Disability 
Under the ADA, Section 504, and Section 1557. 2023. https://www.hhs.gov/civil-rights/for-
providers/civil-rights-covid19/guidance-long-covid-disability/index.html. 
58. 
Nusbaum NJ. Long COVID, Disability, and the Workplace. South Med J 2023; 116(8): 718-20. 
10.14423/SMJ.0000000000001586 PMC10417233 
59. 
Bull-Otterson L, Baca S, Saydah S, et al. Post–COVID conditions among adult COVID-19 
survivors aged 18–64 and≥ 65 years—United States, March 2020–November 2021. Morbidity and 
Mortality Weekly Report 2022; 71(21): 713.  
60. 
Bowe B, Xie Y, Al-Aly Z. Acute and postacute sequelae associated with SARS-CoV-2 
reinfection. Nat Med 2022; 28(11): 2398-405. 10.1038/s41591-022-02051-3 PMC9671810 
61. 
Centers for Disease Control and Prevention. Underlying medical conditions. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-care/underlyingconditions.html. 
62. 
Centers for Disease Control and Prevention. Race/ethnicity, ≥6 months: Estimates from the 
Behavioral Risk Factor Surveillance System (BRFSS), National Immunization Survey (NIS), and the 
National 2009 H1N1 Flu Survey (NHFS). 2023. https://www.cdc.gov/flu/fluvaxview/trends/persons-over-
6months.htm. 

25 
 
63. 
Lewis TJ, Boykin M. We the people: A Black strategy to end the HIV epidemic in the United 
States of America. Journal of Healthcare, Science and the Humanities 2021; 11(1): 173.  
64. 
Bassett MT, Chen JT, Krieger N. Variation in racial/ethnic disparities in COVID-19 mortality by 
age in the United States: A cross-sectional study. PLoS Med 2020; 17(10): e1003402. 
10.1371/journal.pmed.1003402 PMC7575091 
65. 
Mackey K, Ayers CK, Kondo KK, et al. Racial and ethnic disparities in COVID-19–related 
infections, hospitalizations, and deaths: a systematic review. Annals of internal medicine 2021; 174(3): 
362-73.  
66. 
Tabatabai M, Juarez PD, Matthews-Juarez P, et al. An Analysis of COVID-19 Mortality During 
the Dominancy of Alpha, Delta, and Omicron in the USA. Journal of Primary Care & Community Health 
2023; 14: 21501319231170164. 10.1177/215013192311701  
67. 
National Institutes of Health. Symptoms of long COVID differ for people of different racial and 
ethnic groups. 2023. https://covid19.nih.gov/news-and-stories/symptoms-long-covid-differ-people-
different-racial-and-ethnic-groups# (accessed 4/6/2023). 
68. 
Whittington BJ, Buttazzoni G, Patel A, et al. Disparities in COVID-19 Hospitalization at the 
Intersection of Race and Ethnicity and Income. Journal of racial and ethnic health disparities 2023: 1-8.  
69. 
Centers for Disease Control and Prevention. How to protect yourself & others. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. 
70. 
Tenforde MW, Self WH, Gaglani M, et al. Effectiveness of mRNA Vaccination in Preventing 
COVID-19-Associated Invasive Mechanical Ventilation and Death - United States, March 2021-January 
2022. MMWR Morb Mortal Wkly Rep 2022; 71(12): 459-65. 10.15585/mmwr.mm7112e1 PMC8956334 
71. 
Massetti GM, Jackson BR, Brooks JT, et al. Summary of Guidance for Minimizing the Impact of 
COVID-19 on Individual Persons, Communities, and Health Care Systems - United States, August 
2022. MMWR Morb Mortal Wkly Rep 2022; 71(33): 1057-64. 10.15585/mmwr.mm7133e1 
PMC9400529 
72. 
Brannock MD, Chew RF, Preiss AJ, et al. Long COVID risk and pre-COVID vaccination in an 
EHR-based cohort study from the RECOVER program. Nature Communications 2023; 14(1): 2914. 
10.1038/s41467-023-38388-7  
73. 
Flemming A. Vaccines only partially protect against Long COVID. Nature Reviews Immunology 
2022; 22(7): 410-.  
74. 
Centers for Disease Control and Prevention. For immunization managers: Adults. 2023. 
75. 
Centers for Disease Control and Prevention. How to protect yourself & others. 2023. 
https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. 
76. 
Omer SB, Benjamin RM, Brewer NT, et al. Promoting COVID-19 vaccine acceptance: 
recommendations from the Lancet Commission on Vaccine Refusal, Acceptance, and Demand in the 
USA. Lancet 2021; 398(10317): 2186-92. 10.1016/S0140-6736(21)02507-1 PMC8592561 
77. 
Novavax. Novavax Intends to Deliver Protein-based XBB COVID Vaccine as Specified in U.S. 
HHS Letter to COVID Manufacturers. 2023. https://ir.novavax.com/press-releases/Novavax-Intends-to-
Deliver-Protein-based-XBB-COVID-Vaccine-as-Specified-in-U-S-HHS-Letter-to-COVID-
Manufacturers#:~:text=Novavax%20has%20developed%20an%20updated,%E2%80%9CUpon%20aut
horization%2C%20Novavax's%20XBB. (accessed 7/13/2023). 
78. 
Centers for Disease Control and Prevention. COVID data tracker: Variant proportions. 2023. 
https://covid.cdc.gov/covid-data-tracker/#variant-proportions. 

26 
 
79. 
Gavidia M. Racial Disparities Shown for Influenza Hospitalizations, Vaccination Coverage. 
2022. https://www.ajmc.com/view/racial-disparities-shown-for-influenza-hospitalizations-vaccination-
coverage#. 
80. 
Pfizer. Pfizer and BioNTech Initiate Phase 1 Study of Single Dose mRNA-Based Combination 
Vaccine Candidate for Influenza and COVID-19. 2022. 
https://www.pfizer.com/news/announcements/pfizer-and-biontech-initiate-phase-1-study-single-dose-
mrna-based-combination. 
81. 
Gwadz MV, Leonard NR, Cleland CM, et al. The effect of peer-driven intervention on rates of 
screening for AIDS clinical trials among African Americans and Hispanics. American journal of public 
health 2011; 101(6): 1096-102. 10.2105/Ajph.2010.196048 PMC3093288 
82. 
Stevens ER, Nucifora K, Zhou Q, et al. Cost-effectiveness of peer-versus venue-based 
approaches for detecting undiagnosed HIV among heterosexuals in high-risk new York City 
neighborhoods. Journal of acquired immune deficiency syndromes (1999) 2018; 77(2): 183. PMCID: 
PMC5762425 
83. 
Filippone P, Serrano S, Gwadz M, et al. A virtual pilot optimization trial for African 
American/Black and Latino persons with non-suppressed HIV viral load grounded in motivational 
interviewing and behavioral economics. Front Public Health 2023; 11: 1167104. 
10.3389/fpubh.2023.1167104 PMC10205984 
84. 
Gwadz M, Serrano S, Linnemayr S, et al. Behavioral intervention grounded in motivational 
interviewing and behavioral economics shows promise with Black and English-speaking Latino persons 
living with HIV with unsuppressed HIV viral load in New York City: A mixed methods pilot study. Front 
Public Health 2022; 10: 916224. 10.3389/fpubh.2022.916224 PMC9522600 
85. 
Gwadz M, Cluesman SR, Freeman R, et al. Advancing behavioral interventions for African 
American/Black and Latino persons living with HIV using a new conceptual model that integrates 
critical race theory, harm reduction, and self-determination theory: a qualitative exploratory study. 
Research Square [Preprint] 2022; 21(1): 97. 10.1186/s12939-022-01699-0 PMC9286957 
86. 
Cluesman S, Gwadz M, Freeman R, et al. Exploring behavioral intervention components for 
African American/Black and Latino persons living with HIV with non-suppressed HIV viral load in the 
United States: a qualitative study. International Journal for Equity in Health 2023; 22(1): 22. 
10.1186/s12939-023-01836-3 PMCID: PMC9886533 
87. 
Andreas M, Iannizzi C, Bohndorf E, et al. Interventions to increase COVID-19 vaccine uptake: a 
scoping review. Cochrane Database Syst Rev 2022; 8(8): CD015270. 10.1002/14651858.CD015270 
PMC9347311 
88. 
Batteux E, Mills F, Jones LF, Symons C, Weston D. The Effectiveness of Interventions for 
Increasing COVID-19 Vaccine Uptake: A Systematic Review. Vaccines (Basel) 2022; 10(3). 
10.3390/vaccines10030386 PMC8949230 
89. 
Nowak GJ, Sheedy K, Bursey K, Smith TM, Basket M. Promoting influenza vaccination: insights 
from a qualitative meta-analysis of 14 years of influenza-related communications research by US 
Centers for Disease Control and Prevention (CDC). Vaccine 2015; 33(24): 2741-56.  
90. 
Rutten LJF, Zhu X, Leppin AL, et al. Evidence-based strategies for clinical organizations to 
address COVID-19 vaccine hesitancy.  Mayo Clinic Proceedings; 2021: Elsevier; 2021. p. 699-707. 
91. 
Campbell J, Kaur A, Gamino D, Benoit E, Amos B, Windsor L. Individual and structural 
determinants of COVID-19 vaccine uptake in a marginalized community in the United States. Vaccine 
2023.  

27 
 
92. 
Nguyen LH, Joshi AD, Drew DA, et al. Self-reported COVID-19 vaccine hesitancy and uptake 
among participants from different racial and ethnic groups in the United States and United Kingdom. 
Nat Commun 2022; 13(1): 636. 10.1038/s41467-022-28200-3 PMC8807721 
93. 
Lieu TA, Elkin EP, Escobar PR, et al. Effect of electronic and mail outreach from primary care 
physicians for COVID-19 vaccination of black and Latino older adults: a randomized clinical trial. JAMA 
Network Open 2022; 5(6): e2217004-e.  
94. 
Anderson A, O’Connell SS, Thomas C, Chimmanamada R. Telehealth interventions to improve 
diabetes management among Black and Hispanic patients: a systematic review and meta-analysis. 
Journal of Racial and Ethnic Health Disparities 2022: 1-12.  
95. 
Kafadar AH, Tekeli GG, Jones KA, Stephan B, Dening T. Determinants for COVID-19 vaccine 
hesitancy in the general population: a systematic review of reviews. Z Gesundh Wiss 2022: 1-17. 
10.1007/s10389-022-01753-9 PMC9483252 
96. 
Madorsky TZ, Adebayo NA, Post SL, O’Brian CA, Simon MA. Vaccine distrust: A predictable 
response to structural racism and an inadequate public health infrastructure. American Public Health 
Association; 2021. p. S185-S8. 
97. 
Nawas GT, Zeidan RS, Edwards CA, El-Desoky RH. Barriers to COVID-19 vaccines and 
strategies to improve acceptability and uptake. Journal of pharmacy practice 2023; 36(4): 900-4.  
98. 
Zheng H, Jiang S, Wu Q. Factors influencing COVID-19 vaccination intention: The roles of 
vaccine knowledge, vaccine risk perception, and doctor-patient communication. Patient Education and 
Counseling 2022; 105(2): 277-83.  
99. 
Khairat S, Zou B, Adler-Milstein J. Factors and reasons associated with low COVID-19 vaccine 
uptake among highly hesitant communities in the US. American journal of infection control 2022; 50(3): 
262-7.  
100. 
Woko C, Siegel L, Hornik R. An investigation of low COVID-19 vaccination intentions among 
Black Americans: The role of behavioral beliefs and trust in COVID-19 information sources. Journal of 
Health Communication 2020; 25(10): 819-26.  
101. 
Mohamed K, Yazdanpanah N, Saghazadeh A, Rezaei N. Cognitive Biases Affecting the 
Maintenance of COVID-19 Pandemic. Acta Bio Medica: Atenei Parmensis 2021; 92(2).  
102. 
Halpern SD, Truog RD, Miller FG. Cognitive bias and public health policy during the COVID-19 
pandemic. Jama 2020; 324(4): 337-8.  
103. 
Kahneman D, Slovic SP, Slovic P, Tversky A. Judgment under uncertainty: Heuristics and 
biases. Cambridge, UK: Cambridge University Press; 1982. 
104. 
Albert D, Steinberg L. Judgment and decision making in adolescence. J Res Adolescence 2011; 
21(1): 211-24. 10.1111/j.1532-7795.2010.00724.x  
105. 
Linnemayr S, Stecher C. Behavioral Economics Matters for HIV Research: The Impact of 
Behavioral Biases on Adherence to Antiretrovirals (ARVs). AIDS and behavior 2015; 19(11): 2069-75. 
10.1007/s10461-015-1076-0 PMC4720130 
106. 
Kahneman D. Thinking, fast and slow. 1st ed. New York, NY: Farrar, Straus and Giroux; 2011. 
107. 
O'Donoghue T, Rabin M. Doing it now or later. The American economic review 1999; 89(1): 
103-24. 10.1257/aer.89.1.103  
108. 
Saleska JL, Choi KR. A behavioral economics perspective on the COVID-19 vaccine amid 
public mistrust. Translational behavioral medicine 2021; 11(3): 821-5.  
109. 
Wang TT, Wang S. Applying behavioral economics to address COVID-19 fear among oral and 
maxillofacial surgery patients. Journal of Oral and Maxillofacial Surgery 2021; 79(4): 741-2.  

28 
 
110. 
Latkin C, Dayton LA, Yi G, et al. COVID-19 vaccine intentions in the United States, a social-
ecological framework. Vaccine 2021; 39(16): 2288-94.  
111. 
Chu H, Liu S. Integrating health behavior theories to predict American’s intention to receive a 
COVID-19 vaccine. Patient education and counseling 2021; 104(8): 1878-86.  
112. 
Bogart LM, Dong L, Gandhi P, et al. COVID-19 vaccine intentions and mistrust in a national 
sample of Black Americans. Journal of the National Medical Association 2022; 113(6): 599-611.  
113. 
Kirchoff C, Penn A, Wang W, et al. COVID-19 Vaccine Acceptance Among Latino/a Immigrants: 
The Role of Collective Responsibility and Confidence. Journal of Immigrant and Minority Health 2023: 
1-8.  
114. 
McCready JL, Nichol B, Steen M, Unsworth J, Comparcini D, Tomietto M. Understanding the 
barriers and facilitators of vaccine hesitancy towards the COVID-19 vaccine in healthcare workers and 
healthcare students worldwide: An Umbrella Review. Plos one 2023; 18(4): e0280439.  
115. 
MacDonald NE, Butler R, Dubé E. Addressing barriers to vaccine acceptance: an overview. 
Human vaccines & immunotherapeutics 2018; 14(1): 218-24.  
116. 
National Institutes of Health. Definitions of Criteria and Considerations for Research Project 
Grant (RPG/R01/R03/R15/R21/R34) Critiques. http://grants.nih.gov/grants/peer/critiques/rpg.htm. 
117. 
Creswell J, Clark V. Designing and conducting mixed methods research. 3rd ed. Thousand 
Oaks, CA: SAGE Publications Inc.; 2017. 
118. 
Hamilton CM, Strader LC, Pratt JG, et al. The PhenX Toolkit: get the most from your measures. 
American journal of epidemiology 2011; 174(3): 253-60.  
119. 
National Institute of Environmental Health Sciences. Disaster Research Response (DR2) 
Program Common Data Elements (CDE). 2023 (accessed 4/11/2023). 
120. 
Centers for Disease Control and Prevention. 2023-2024 ACIP Summary. Table: Influenza 
vaccines — United States, 2023–24 influenza season, 2023. 
121. 
Centers for Disease Control and Prevention. Allergic Reactions. 2022. 
https://www.cdc.gov/coronavirus/2019-ncov/vaccines/safety/allergic-reaction.html (accessed 9/12/23). 
122. 
Centers for Disease Control and Prevention. Myths & Facts: Bust Common Myths and Learn 
the Facts. 2023. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/facts.html (accessed 9/12/23). 
123. 
Centers for Disease Control and Prevention. Frequently Asked Questions: Getting Your 
Updated COVID-19 Vaccine(s). 2023. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/faq.html 
(accessed 9/12/23). 
124. 
Wiemken TL, Khan F, Puzniak L, et al. Seasonal trends in COVID-19 cases, hospitalizations, 
and mortality in the United States and Europe. Sci Rep 2023; 13(1): 3886. 10.1038/s41598-023-31057-
1 PMC9994397 
125. 
Research Electronic Data Capture (REDCap). Vanderbilt; 2020. 
126. 
Metzl JM, Hansen H. Structural competency: Theorizing a new medical engagement with 
stigma and inequality. Social science & medicine 2014; 103: 126-33. 10.1016/j.socscimed.2013.06.032 
PMC4269606 
127. 
Health Research & Educational Trust. Becoming a culturally competent health care 
organization. Chicago, IL: American Hospital Association, 2013. 
128. 
Chin MH, Clarke AR, Nocon RS, et al. A roadmap and best practices for organizations to reduce 
racial and ethnic disparities in health care. Journal of general internal medicine 2012; 27: 992-1000.  
129. 
Kumari A, Ranjan P, Chopra S, et al. Knowledge, barriers and facilitators regarding COVID-19 
vaccine and vaccination programme among the general population: a cross-sectional survey from one 

29 
 
thousand two hundred and forty-nine participants. Diabetes & Metabolic Syndrome: Clinical Research 
& Reviews 2021; 15(3): 987-92.  
130. 
Centers for Disease Control and Prevention. HHS Launches Bridge Access Program to 
Safeguard Free COVID-19 Vaccination for Uninsured and Underinsured Adults. 2023. 
https://www.cdc.gov/media/releases/2023/p0914-uninsured-vaccination.html. 
131. 
Wiley Z, Khalil L, Lewis K, et al. A framework for inspiring COVID-19 vaccine confidence in 
African American and latino communities. Vaccines 2022; 10(8): 1319.  
132. 
Joosten EA, DeFuentes-Merillas L, De Weert G, Sensky T, Van Der Staak C, de Jong CA. 
Systematic review of the effects of shared decision-making on patient satisfaction, treatment 
adherence and health status. Psychotherapy and psychosomatics 2008; 77(4): 219-26.  
133. 
Shay LA, Lafata JE. Where is the evidence? A systematic review of shared decision making 
and patient outcomes. Med Decis Making 2015; 35(1): 114-31. 10.1177/0272989X14551638 
PMC4270851 
134. 
Elwyn G, Durand MA, Song J, et al. A three-talk model for shared decision making: multistage 
consultation process. bmj 2017; 359.  
135. 
Elwyn G, Frosch D, Thomson R, et al. Shared decision making: a model for clinical practice. 
Journal of general internal medicine 2012; 27: 1361-7.  
136. 
Makoul G, Clayman ML. An integrative model of shared decision making in medical encounters. 
Patient education and counseling 2006; 60(3): 301-12.  
137. 
Abrams EM, Shaker M, Oppenheimer J, Davis RS, Bukstein DA, Greenhawt M. The challenges 
and opportunities for shared decision making highlighted by COVID-19. The Journal of Allergy and 
Clinical Immunology: In Practice 2020; 8(8): 2474-80. e1.  
138. 
Durand M-A, Scalia P, Elwyn G. Can shared decision making address COVID-19 vaccine 
hesitancy? BMJ Evidence-Based Medicine 2022; 27(3): 159-61.  
139. 
Simpson N, Milnes S, Steinfort D. Don't forget shared decision‐making in the COVID‐19 crisis. 
Internal Medicine Journal 2020; 50(6): 761-3.  
140. 
Gwadz M, Cleland CM, Leonard NR, et al. Predictors of screening for AIDS clinical trials among 
African-Americans and Latino/Hispanics enrolled in an efficacious peer-driven intervention: uncovering 
socio-demographic, health, and substance use-related factors that promote or impede screening. AIDS 
and behavior 2013; 17(2): 801-12. 10.1007/s10461-012-0194-1 PMC4181555 
141. 
Soofi M, Najafi F, Karami-Matin B. Using insights from behavioral economics to mitigate the 
spread of COVID-19. Applied health economics and health policy 2020; 18: 345-50.  
142. 
Strickland JC, Reed DD, Hursh SR, et al. Behavioral economic methods to inform infectious 
disease response: Prevention, testing, and vaccination in the COVID-19 pandemic. PloS one 2022; 
17(1): e0258828.  
143. 
Haro-Ramos AY, Rodriguez HP, Aguilera A. Effectiveness and implementation of a text 
messaging intervention to reduce depression and anxiety symptoms among Latinx and Non-Latinx 
white users during the COVID-19 pandemic. Behaviour research and therapy 2023; 165: 104318. 
10.1016/j.jvacx.2022.100225  
144. 
Agyapong VIO, Hrabok M, Vuong W, et al. Changes in Stress, Anxiety, and Depression Levels 
of Subscribers to a Daily Supportive Text Message Program (Text4Hope) During the COVID-19 
Pandemic: Cross-Sectional Survey Study. JMIR Ment Health 2020; 7(12): e22423. 10.2196/22423 
PMC7752184 
145. 
Hall AK, Cole-Lewis H, Bernhardt JM. Mobile text messaging for health: a systematic review of 
reviews. Annual review of public health 2015; 36: 393-415.  

30 
 
146. 
Nan X, Iles IA, Yang B, Ma Z. Public health messaging during the COVID-19 pandemic and 
beyond: Lessons from communication science. Health communication 2022; 37(1): 1-19.  
147. 
Operario D, Kuo C, Sosa-Rubí SG, Gálarraga O. Conditional economic incentives for reducing 
HIV risk behaviors: integration of psychology and behavioral economics. Health Psychology 2013; 
32(9): 932. 10.1037/a0032760  
148. 
Haff N, Patel MS, Lim R, et al. The role of behavioral economic incentive design and 
demographic characteristics in financial incentive-based approaches to changing health behaviors: a 
meta-analysis. American Journal of Health Promotion 2015; 29(5): 314-23.  
149. 
Higgins ST, Klemperer EM, Coleman SR. Looking to the empirical literature on the potential for 
financial incentives to enhance adherence with COVID-19 vaccination. Preventive medicine 2021; 145: 
106421.  
150. 
Persad G, Emanuel EJ. Ethical Considerations of Offering Benefits to COVID-19 Vaccine 
Recipients. JAMA 2021; 326(3): 221-2. 10.1001/jama.2021.11045  
151. 
Campos-Mercade P, Meier A, Schneider F, Meier S, Pope D, Wengström E. Monetary 
incentives increase COVID-19 vaccinations, nudges do not. VoxEU org 2021; 19.  
152. 
Huang Y, Huang X, Yu R. The effectiveness of nonfinancial interventions and monetary 
incentives on COVID-19 vaccination: A meta-analysis. Health Psychol 2023; 42(6): 411-24. 
10.1037/hea0001288  
153. 
Mardi P, Djalalinia S, Kargar R, Jamee M, Esmaeili Abdar Z, Qorbani M. Impact of incentives on 
COVID-19 vaccination; A systematic review. Front Med (Lausanne) 2022; 9: 810323. 
10.3389/fmed.2022.810323 PMC9492889 
154. 
Chang TY, Jacobson M, Shah M, Kopetsky M, Pramanik R, Shah SB. Reminders, but not 
monetary incentives, increase COVID-19 booster uptake. Proceedings of the National Academy of 
Sciences 2023; 120(31): e2302725120.  
155. 
Schwalbe N, Hanbali L, Nunes MC, Lehtimaki S. Use of financial incentives to increase adult 
vaccination coverage: a narrative review of lessons learned from COVID-19 and other adult vaccination 
efforts. Vaccine: X 2022: 100225.  
156. 
Chan RJ, Milch VE, Crawford‐Williams F, et al. Patient navigation across the cancer care 
continuum: An overview of systematic reviews and emerging literature. CA: A Cancer Journal for 
Clinicians 2023.  
157. 
Cunningham WE, Weiss RE, Nakazono T, et al. Effectiveness of a peer navigation intervention 
to sustain viral suppression among HIV-positive men and transgender women released from jail: the 
LINK LA randomized clinical trial. JAMA internal medicine 2018; 178(4): 542-53.  
158. 
Ali-Faisal SF, Colella TJ, Medina-Jaudes N, Scott LB. The effectiveness of patient navigation to 
improve healthcare utilization outcomes: A meta-analysis of randomized controlled trials. Patient 
Education and Counseling 2017; 100(3): 436-48.  
159. 
Lin C, Parker T, Pejavara K, Smith D, Tu R, Tu P. “I Would Never Push a Vaccine on You”: A 
Qualitative Study of Social Norms and Pressure in Vaccine Behavior in the US. Vaccines 2022; 10(9): 
1402.  
160. 
Brislin RW. Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology 
1970; 1: 185-216.  
161. 
Beatty PC, Willis GB. Research synthesis: The practice of cognitive interviewing. Public Opinion 
Quarterly 2007; 71(2): 287-311. 10.1093/poq/nfm006  

31 
 
162. 
Jenness SM, Kobrak P, Wendel T, Neaigus A, Murrill CS, Hagan H. Patterns of exchange sex 
and HIV infection in high-risk heterosexual men and women. Journal of urban health : bulletin of the 
New York Academy of Medicine 2011; 88(2): 329-41. 10.1007/s11524-010-9534-5 3079035 
163. 
Gwadz M, Cleland CM, Perlman DC, et al. Public health benefit of peer-referral strategies for 
detecting undiagnosed HIV infection among high-risk heterosexuals in New York City. Journal of 
acquired immune deficiency syndrome: JAIDS 2017; 74(5): 499-507. 10.1097/QAI.0000000000001257  
164. 
Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: Guided by 
information power. Qualitative Health Research 2016; 26(13): 1753-60. 10.1177/1049732315617444  
165. 
Des Jarlais DC, Perlis TE, Settembrino JM. The use of electronic debit cards in longitudinal 
data collection with geographically mobile drug users. Drug Alcohol Depen 2005; 77(1): 1-5. 
10.1016/j.drugalcdep.2004.06.010  
166. 
Guidry JP, Laestadius LI, Vraga EK, et al. Willingness to get the COVID-19 vaccine with and 
without emergency use authorization. American Journal of Infection Control 2021; 49(2): 137-42.  
167. 
Graupensperger S, Abdallah DA, Lee CM. Social norms and vaccine uptake: College students’ 
COVID vaccination intentions, attitudes, and estimated peer norms and comparisons with influenza 
vaccine. Vaccine 2021; 39(15): 2060-7.  
168. 
Bertin P, Nera K, Delouvée S. Conspiracy beliefs, rejection of vaccination, and support for 
hydroxychloroquine: A conceptual replication-extension in the COVID-19 pandemic context. Frontiers in 
psychology 2020; 11: 2471.  
169. 
Islam MS, Kamal A-HM, Kabir A, et al. COVID-19 vaccine rumors and conspiracy theories: The 
need for cognitive inoculation against misinformation to improve vaccine adherence. PloS one 2021; 
16(5): e0251605.  
170. 
Centers for Disease Control and Prevention. Benefits of Getting a COVID-19 Vaccine. 2021. 
https://www.cdc.gov/coronavirus/2019-ncov/vaccines/vaccine-
benefits.html?s_cid=11235:covid%20vaccine%20benefits:sem.ga:p:RG:GM:gen:PTN:FY21. 
171. 
Rollnick S, Heather N, Gold R, Hall W. Development of a short 'readiness to change' 
questionnaire for use in brief, opportunistic interventions among excessive drinkers. British Journal of 
Addiction 1992; 87(5): 743-54.  
172. 
Latkin CA, Dayton L, Yi G, Konstantopoulos A, Boodram B. Trust in a COVID-19 vaccine in the 
US: A social-ecological perspective. Social science & medicine 2021; 270: 113684.  
173. 
Shapiro JP, Welker CJ, Jacobson BJ. The Youth Client Satisfaction Questionnaire: 
development, construct validation, and factor structure. Journal of clinical child psychology 1997; 26(1): 
87-98. 10.1207/s15374424jccp2601_9  
174. 
Gwadz M, Cleland CM, Belkin M, et al. ACT2 peer-driven intervention increases enrollment into 
HIV/AIDS medical studies among African Americans/Blacks and Hispanics: A cluster randomized 
controlled trial. AIDS and behavior 2014; 18(12): 2409-22. 10.1007/s10461-014-0829-5 4451823 
175. 
Gwadz M, Applegate E, Cleland C. HIV-infected individuals who delay, decline, or discontinue 
antiretroviral therapy: comparing clinic- and peer-recruited cohorts. Front Public Health 2014; 2. 
10.3389/fpubh.2014.00081  
176. 
Gwadz M, Rotheram-Borus MJ. Tracking high-risk adolescents longitudinally. AIDS Education & 
Prevention 1992; Suppl: 69-82.  
177. 
Leonard NR, Lester P, Rotheram-Borus MJ, Mattes K, Gwadz M, Ferns B. Successful 
recruitment and retention of participants in longitudinal behavioral research. Aids Education and 
Prevention 2003; 15(3): 269-81. 10.1521/aeap.15.4.269.23827  

32 
 
178. 
NIMH Multisite HIV Prevention Trial. Quality control and quality assurance in HIV prevention 
research: model from a multisite HIV prevention trial. AIDS 1997; 11: 49-53.  
179. 
Colditz GA, Emmons KM. The promise and challenges of dissemination and implementation 
research. Dissemination and Implementation Research in Health: Translating Science to Practice 2012: 
3-22.  
180. 
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the 
use of the consolidated framework for implementation research. Implementation science : IS 2015; 
11(1): 1-13.  
181. 
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: 
results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation 
science : IS 2015; 10(1): 1-14.  
182. 
Jo B. Statistical power in randomized intervention studies with noncompliance. Psychological 
methods 2002; 7(2): 178-93. 10.1037//1082-989x.7.2.178  
183. 
Little RJ, Yau LHY. Statistical techniques for analyzing data from prevention trials: Treatment of 
no-shows using Rubin's causal model. Psychological methods 1998; 3(2): 147-59. 10.1037/1082-
989x.3.2.147  
184. 
Allison PD. Missing data. Thousand Oaks, CA: Sage Publications, Inc.; 2002. 
185. 
Little RJA, Rubin DB. Statistical Analysis With Missing Data, 2nd Edition. Hoboken, NJ: Wiley; 
2002. 
186. 
Heckman J. The common structure of statistical models of truncation, sample selection and 
limited dependent variable and a simple estimator for such models. Annals of Economic and Social 
Measurement 1976; 5(4): 475-92.  
187. 
Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data 
in longitudinal studies. Psychological methods 1997; 2(1): 64-78. 10.1037/1082-989x.2.1.64  
188. 
Little RJA. Pattern-mixture models for multivariate incomplete data. Journal of the American 
Statistical Association 1993; 88(421): 125-34. 10.2307/2290705  
189. 
Muthén BO, Muthén LK, Asparouhov T. Regression and mediation analysis using Mplus. Los 
Angeles, CA: Muthén & Muthén; 2017. 
190. 
VanderWeele T. Explanation in causal inference: Methods for mediation and interaction. New 
York: Oxford University Press; 2015. 
191. 
Robins JM, Greenland S. Identifiability and exchangeability for direct and indirect effects. 
Epidemiology 1992: 143-55.  
192. 
Tingley D, Yamamoto T, Hirose K, Keele L, Imai K. Mediation: R package for causal mediation 
analysis. 2014.  
193. 
Collins LM, Murphy SA, Bierman KL. A conceptual framework for adaptive preventive 
interventions. Prevention science : the official journal of the Society for Prevention Research 2004; 
5(3): 185-96. 3544191 
194. 
Hintze J. PASS 12. Kaysville, Utah, USA: NCSS,LLC; 2013. 
195. 
Vittinghoff E, Sen Ś, McCulloch C. Sample size calculations for evaluating mediation. Statistics 
in medicine 2009; 28(4): 541-57. 10.1002/sim.3491  
196. 
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qualitative Health 
Research 2005; 15(9): 1277-88. 10.1177/1049732305276687  

33 
 
197. 
Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles 
and practices. Health Services Research 2013; 48(6 Pt 2): 2134-56. 10.1111/1475-6773.12117 
PMC4097839 
198. 
Gwadz M, Cleland CM, Freeman R, et al. Stopping, starting, and sustaining HIV antiretroviral 
therapy: A mixed-methods exploration among African American/Black and Latino long-term survivors of 
HIV in an urban context. BMC Public Health 2021; 21(1): 419. 10.1186/s12889-021-10464-x 
PMC7912958 
199. 
Gwadz M, Campos S, Freeman R, et al. Black and latino persons living with HIV evidence risk 
and resilience in the context of COVID-19: A mixed-methods study of the early phase of the pandemic. 
AIDS and behavior 2021; 25(5): 1340-60. 10.1007/s10461-021-03177-0 PMC7873114 
200. 
Leonard NR, Freeman R, Ritchie A, et al. "Coming from the place of walking with the youth -- 
that feeds everything": A mixed methods case study of a runaway and homeless youth organization. 
Child and Adolescent Social Work Journal 2017; 34(5): 443-59. 10.1007/s10560-016-0483-z 
PMC6774622 
201. 
Strayhorn JC, Cleland CM, Vanness DJ, Wilton L, Gwadz M, Collins LM. Using decision 
analysis for intervention value efficiency to select optimized interventions in the multiphase optimization 
strategy. Health Psychol 2023. 10.1037/hea0001318  
202. Neumann, PJ, Sanders, G., Russell, LB, Siegel, JE, Ganiats, TG. Cost-Effectiveness in Health 
and Medicine 2nd Edition. Oxford University Press, 2016. 
 
 
 
 
 
 

34