"Less Pain, Less Fuss, Right Now!" and "Make It Count!"--Multilevel Interventions for Patient, Parent, and Practice to Enhance Provider Recommendations for HPV Vaccination
Phase: NA
Conditions: Papillomavirus Vaccines
Sponsor: Mayo Clinic

NCI HPV R01 Protocol 
Co-PIs: RM Jacobson, MD, & JM Griffin, PhD 
Mayo Clinic – R01CA217889 
Page 1 of 5 
 
 
 
“Less Pain, Less Fuss, Right Now!” and 
“Make It Count!”—Multilevel Interventions 
for Patient, Parent, and Practice to Enhance 
Provider Recommendations for HPV 
Vaccination 
 
Protocol 
R01CA217889 
 
May 2, 2024 

NCI HPV R01 Protocol 
Co-PIs: RM Jacobson, MD, & JM Griffin, PhD 
Mayo Clinic – R01CA217889 
Page 2 of 5 
 
 
 
 
Co-Principal Investigator: Robert M Jacobson, MD, and Joan M Griffin, PhD 
 
Study Title: “Less Pain, Less Fuss, Right Now!” and “Make It Count!”—Multilevel Interventions for 
Patient, Parent, and Practice to Enhance Provider Recommendations for HPV Vaccination 
Protocol version number and date: version 0.8, May 2, 2024 
 
 
Hypothesis: Our overarching hypothesis is that multilevel interventions for patient, parent, and practice can enhance provider 
recommendations for HPV vaccination. 
 
Aims, purpose, or objectives: We are testing this hypothesis through three phases for which we have developed three 
protocols. In this third phase, with this protocol, we have three primary aims. 
 
Aim 1: Test the hypothesis that, as compared to no intervention, a practice-level intervention utilizing reminder-recalls 
featuring the availability of non-medication and medication anesthetics, the convenience of nurse-only visits, and the use of 
persuasive language for early, on-time vaccinations—“Less Pain, Less Fuss, Right Now!”—will improve the odds of a child 
receiving an HPV vaccine dose by at least 20%. 
 
Aim 2: Test the hypothesis that, as compared to no intervention, a provider-level intervention utilizing 
missed opportunities audit-and-feedback and equipping providers with a strong-recommendation toolkit—“Make It 
Count!”—will improve the odds of a child receiving an HPV vaccine dose by at least 20%. 
 
Aim 3: Test the hypothesis that simultaneous implementation of interventions targeting individual, interpersonal, and 
organizational factors will have a synergistic effect more than doubling the odds of a child receiving an HPV vaccine dose. 
 
Background:  Our relevant experience includes our team’s considerable work in vaccine delivery research with particular 
emphasis on HPV vaccination. This work has focused on parental attitudes toward HPV vaccination, population knowledge 
and attitudes about HPV vaccination, awareness and knowledge of HPV vaccination among uninsured, low-income 
populations; factors associated with HPV vaccination initiation and completion, and strategies to address vaccine hesitancy. 
We have also studied population health interventions that health care organizations can adopt including reminder-recall 
strategies and others. Our research team partners closely with primary care practice leadership to identify priorities for 
research to improve the clinical practice and community health. The clinical sites involved in this effort serve as a laboratory 
for health care delivery research, and our team has significant experience conducting research in these clinics. (See 
attachments 1, 2, and 3 for letters of support.) 
 
Current knowledge suffers significant gaps. Nearly 39,000 human papillomavirus (HPV)–associated cancers occur each year 
including 23,000 in females and 16,000 in males. Of these, 30,700 result from HPV infection, and 28,500 result from HPV 
strains preventable by the highly immunogenic and efficacious 9-valent HPV vaccine. While HPV vaccination is 
recommended routinely for all males and females 11 to 12 years of age, HPV vaccination rates in the United States (US) are 
failing to reach national goals; thus, leaving millions at risk for HPV-associated cancer. Rates of HPV vaccine series 
completion are significantly lower than rates observed for the other two adolescent vaccines introduced at approximately the 
same time for the same age group. The HPV vaccination rates in our community and region reflect the low rates observed in 
Minnesota and the rest of the US. Barriers to vaccination reported by parents of adolescents who have not started or 
completed the series include the beliefs that the vaccine is optional, not recommended, unnecessary, and unsafe. Because the 
vaccine series has involved three injections over six months in the past, the series required several visits. However, 
adolescents infrequently use health services; thus, they lack frequent opportunities for vaccination. Educational campaigns 
General Study Information 
Research Question and Aims 

NCI HPV R01 Protocol 
Co-PIs: RM Jacobson, MD, & JM Griffin, PhD 
Mayo Clinic – R01CA217889 
Page 3 of 5 
 
 
Study Design and Methods 
have failed to produce substantial improvements in HPV vaccine delivery rates. While other countries have addressed HPV 
vaccine delivery successfully through school-based programs, the US relies on primary-care-practice delivery of routine 
adolescent vaccination. Other vaccines in the US have achieved desired national goals in large part through state-based school 
vaccine mandates, but states have avoided mandating HPV vaccine for school attendance because of political concerns.  By 
contrast, several proven practice-level interventions offer small but incremental improvements in HPV vaccine delivery rates. 
Reminder-recalls directed to the patients and their parents are such practice-level interventions. They have produced varying 
effect sizes with improving HPV vaccination rates. Tailoring the message to address both patient and parent issues with HPV 
vaccination offers an opportunity to improve the effect size, but this has not been tested directly. Other innovative practice-
level strategies make sense in theory but have not been tested empirically—such as routinely offering medication anesthetics 
to make the HPV vaccination relatively painless and emphasizing nurse visits (not requiring a clinician encounter) with the 
nurse utilizing standing orders (or nursing protocols) to facilitate easy access. 
 
Our preliminary data supports our work. Using patient-level data geocode-matched to publically available data from the 
American Community Survey, we characterized HPV vaccination initiation and completion rates in our local population. We 
observed a greater likelihood of vaccination among females and older adolescents. We also found our composite, 
environmental level measure of socioeconomic status to be significantly associated with both initiation and completion, even 
after controlling for individual level variables known to be associated with vaccination. Across a 7-county area in southeast 
MN where we have sufficient population coverage to ascertain population estimates we found 4,066 (27.1%) of children and 
adolescents aged 9 to 14 had received one or more doses of HPV vaccine, and 1,524 (10.1%) of those of the same age had 
received three or more doses of HPV vaccine. Receipt of one or more doses of HPV vaccine ranged from 11.7% to 33.8% 
across counties. Receipt of the three doses needed to complete the series ranged from 2.5% to 13.7%. These rates are like the 
rest of Minnesota and the overall US population. None of the rates approximated the 80% Healthy People 2020 goal.  
 
 
Methods:   
 
The overall study grant of which this protocol is the third of three parts is designed as a stepped wedge cluster randomized 
trial. The cluster approach prevents cross-contamination between patients or between providers as we allocate two separate 
interventions (i.e. “Less Pain, Less Fuss, Right Now!” and “Make It Count!”) — to the six primary care practices. The 
stepped-wedge design  permits us to test the presence of each of the interventions in each primary care practice, making trial 
participation more attractive to each primary care practice, while also allowing each practice to serve as its own control, 
reducing the bias due to imbalanced risk factors across practices. The incorporation of a factorial design allows us to use a 
single trial to test two interventions and assess each individually and in combination. The design also provides opportunity to 
conserve overall sample size while maintaining power. 
 
Research Setting: We will conduct our study in six primary care practices (Community Pediatric and Adolescent Medicine, 
Family Medicine, Northeast Family Clinic, Northwest Family Clinic, Southeast Family Clinic, and Kasson Clinic) that have 
expressed a commitment to improving HPV vaccination rates. These practices employ salaried nurse-practitioners, 
pediatricians, and family physicians. Two of the practices train residents in family medicine and pediatrics. The study sites 
provide care to children including those in the target age group, 11 to 12 years of age. Five of the primary care practices are in 
Rochester, Minnesota and one of the practices is 15.7 miles west of Rochester in Kasson, Minnesota. Uniformity across the 
six clinical practices will enhance the success and efficiency of the study and contribute to the fidelity of the interventions as 
well as the interpretability of the results. None of the practices have had or currently have campaigns in place to reduce pain 
of HPV vaccination, although nurses do have access to an instant topical anesthetic skin refrigerant or vapo-coolant 
(Gebauer's Pain Ease Mist Spray®, Gebauer Co.) that does not require a provider order and have been taught comfort holds 
for infants receiving vaccinations. None have a reminder-recall process in place for HPV vaccination. None have provider-
performance assessments regarding HPV vaccination. All six practices ask patients receiving HPV vaccine to wait 15 minutes 
post vaccination to identify symptoms of syncope and utilize the same protocol for timing the 15 minutes. 
 
Study Design: To accomplish Aims 1-3, we will use a stepped-wedge cluster randomized trial with process evaluation. The 
cluster approach prevents cross-contamination between patients or providers as we allocate two separate interventions to the 
six primary care practices. The stepped-wedge design permits us to test the presence of each of the interventions in each 

NCI HPV R01 Protocol 
Co-PIs: RM Jacobson, MD, & JM Griffin, PhD 
Mayo Clinic – R01CA217889 
Page 4 of 5 
 
 
primary care practice, making trial participation more attractive to each primary care practice, while also allowing each 
practice to serve as its own control, reducing the bias due to imbalanced risk factors across practices. The incorporation of a 
factorial design allows us to use a single trial to test two interventions and assess each individually and in combination. The 
design also provides opportunity to conserve sample size while maintaining power. 
 
Allocation: We will allocate participating practices as illustrated in the table. There will be four 12-month steps in our design, 
for a total of 48 months. The first step will be a baseline period in which no intervention is implemented; data collected during 
this period will provide a within-practice control group for each practice. For the next step, two practices will be randomly 
selected to receive intervention 1 (Aim 1) and two practices randomly selected to receive intervention 2 (Aim 2). For the third 
step, the two practices with no intervention will be randomly allocated to 1 or 2; practices initially with intervention 1 will be 
randomly allocated to interventions 1+2 or intervention 1; and practices initially with intervention 2 will be randomly 
allocated to interventions 1+2 or 2. For the final step, all practices will receive both interventions. To ensure balance of patient 
numbers across interventions, we will block randomize at the first step, with the six practices grouped into three pairs 
according to volume. Physicians, NPs, residents, and patients belong to the individual, geographical separated practices and 
do not practice or obtain care at more than one practice. The lack of overlap minimizes the risk of contamination. 
 
Table: Factorial Design Utilized by the Proposed Stepped-Wedge, Cluster-Randomized Trial 
Practice 
Step 1 
Step 2 
Step 3 
Step 4 
A 
0 
0 
1 
1+2 
B 
0 
1 
1+2 
1+2 
C 
0 
2 
2 
1+2 
D 
0 
0 
2 
1+2 
E 
0 
1 
1 
1+2 
F 
0 
2 
1+2 
1+2 
0=Current care 
1=”Less Pain, Less Fuss, Right Now! reminder-recall 
2=”Make It Count!” missed opportunities audit-and-feedback and strong recommendation provider toolkit 
 
Interventions: As optimized in Phase 1, the “Less Pain, Less Fuss, Right Now!” intervention consists of a program of 
reminder-recall communication. For practices randomized to the “Less Pain, Less Fuss, Right Now!” intervention, reminder-
recall communication will go out at the beginning of each month. Each month the practice will send a secure electronic 
communication to the parent or legal guardian of the patient through the patient’s electronic health record portal. We will send 
mailed letters to parents or guardians of patients who either have opted-out of the portal messaging or who do not access the 
portal within one week of delivering the reminder-recall through the portal. To support the “Less Pain, Less Fuss, Right 
Now!” reminder-recall intervention, we will conduct a broad education of the practice staff—nurses, medical secretaries, 
receptionists, and clinical assistants—regarding the nature of the intervention, its goals, and its likely impact on the practice. 
The broad education will be conducted through supervisory communications only to practice staff in practices allocated to the 
intervention at the beginning of the step. As optimized in Phase 2, the “Make It Count!” intervention refers to the two 
components including the missed opportunities audit-and-feedback and the provision of a strong recommendation provider-
toolkit, along with a broad education of the practice staff—nurses, medical secretaries, receptionists, and clinical assistants—
regarding the nature of the intervention, its goals, and its likely impact on the practice. The broad education will be conducted 
through supervisory communications only to practice staff in practices allocated to the intervention at the beginning of the 
step. 
 
Data Collection: For each 12-month period, we will collect patient empaneled data for each participating practice in our 
region. Demographic and geographic data (including ethnicity, race, sex, age, insurance, street addresses and ZIP codes) will 
obtained electronically for all empaneled patients ages 11-12. Our administrative database will be searched electronically to 
identify the occurrence and dates of all HPV vaccination of children ages 11-12 in our entire system from January 1, 2006, to 
the end of the study using current procedural terminology (CPT) codes (90649, 90650 and 90651 ). Vaccinations prior to the 
study period will be used to identify patients who have previously completed 1, 2 or 3 HPV vaccine doses with appropriate 
spacing (defined according to ACIP recommendations).  
 

NCI HPV R01 Protocol 
Co-PIs: RM Jacobson, MD, & JM Griffin, PhD 
Mayo Clinic – R01CA217889 
Page 5 of 5 
 
 
Subject Information 
 
Target accrual: We will allocate participating practices in a stepped wedge. There will be four 12-month steps in 
our design, for a total of 48 months. The first step will be a baseline period in which no intervention is 
implemented; data collected during this period will provide a within-practice control group for each practice. For 
the next step, two practices will be randomly selected to receive intervention 1 (Aim 1) and two practices randomly 
selected to receive intervention 2 (Aim 2). For the third step, the two practices with no intervention will be 
randomly allocated to 1 or 2; practices initially with intervention 1 will be randomly allocated to 1+2 or 1; and 
practices initially with intervention 2 will be randomly allocated to 1+2 or 2. For the final step, all practices will 
receive both interventions. To ensure balance of patient numbers across interventions, we will block randomize at 
the first step, with the six practices grouped into three pairs according to volume. Physicians, NPs, residents, and 
patients belong to the individual, geographical separated practices and do not practice or obtain care at more than 
one practice. The lack of overlap minimizes the risk of contamination. 
  
Subject population (children, adults, groups):  The units of allocation as mentioned above are the six primary care 
practices with outcomes being measured at the individual patient-level. For each 12-month-long step, we will 
measure the HPV vaccination status of the eligible patients empaneled to the providers at that primary care practice. 
 
Inclusion Criteria: 1) Empaneled in one of the six participating primary care practices 2) 11 to 12 years of age at the first day 
of each of the 12-month-long steps 3) Due during that 12-month-long step for at least one dose of the HPV vaccine. (For 
immunocompetent children (the overwhelming majority), for children less than 15 years of age, the newly revised ACIP 
recommendations call for two doses at zero- and six-months with the minimum interval being five months between the first 
and second dose. Some eligible patients may have received two doses too close together for this because of the previous three-
dose recommendation was in effect until December 16, 2016, making the ACIP recommendations official. In these cases 
where the second dose is less than five months after the first, the individual should receive a third dose of HPV vaccine 12 
weeks after the second dose and at least 24 weeks after the first dose. Thus, patients who are eligible and due comprise of 
three groups: those having received no valid HPV vaccine; those having received one valid HPV dose AND it has now been 
five calendar-months or more; and those having received two valid doses, but Dose 2 was given less than five months after the 
first. Valid doses include doses given at nine years or older (the minimum age per ACIP) and meet the minimum intervals. 
 
Exclusion Criteria: 1) Not empaneled in one of the six participating practices. 2) Empaneled in one of the six 
participating practices but less than 11 years of age or more than 12 years of age on the first day of each 12-month 
long step. 3) Not due during that 12-month-long step for a dose of HPV vaccine. 
 
Waiver for Parental Permission and Child Assent: We are asking for a waiver of parental permission and child assent 
appealing to the exceptions permitted under 45 CFR 46 section §46.116. Given that the interventions are allocated at the 
practice level, the research could not practicably be carried out without the waiver, and the interventions are do not exceed 
minimal risk, the waiver or alteration will not adversely affect the rights and welfare of the subjects, and whenever 
appropriate, the participants will be provided with additional pertinent information after participation. Parental permission and 
child assent are impracticable with the overall sample size over the four years of study involve approximately 10,000 children 
11 to 12 years of age. It would be impracticable to arrange prospective sessions that would achieve timely informational 
discussions and provide for documentation of parental permission and adolescent assent for all 10,000 children. Even if we 
could overcome the logistics, the requirement of consent would compromise scientific validity as the very act of 
communicating the nature of the interventions (the reminder-recall) would a) serve as an intervention itself altering subjects’ 
behaviors and responses and b) serve to create a selection bias as it would cause parents less likely to vaccinate to opt out of 
the study leaving those more likely to vaccinate to opt in, biasing the findings. The use of deception is justified as any 
notification or invitation would serve as its own intervention and confound the impact of the communication of the reminder-
recall.