Evaluating the Feasibility of a Mobile Coaching Intervention to Improve HPV Vaccine Delivery
Phase: NA
Conditions: HPV Vaccine, Provider Communication Practices, Vaccine Recommendation Practices
Sponsor: UNC Lineberger Comprehensive Cancer Center

 ____________________________________________________________________________________________ 
i 
EVALUATING THE FEASIBILITY OF A MOBLIE COACHING 
INTERVENTION TO IMPROVE HPV VACCINE DELIVERY 
 
 
Support Provided by: The National Cancer Institute (R21CA241518); Lineberger 
Comprehensive Cancer Center; Cancer Control Education Program at the University of 
North Carolina Lineberger Comprehensive Cancer Center. 
Clinicaltrials.gov: 20-0810 
Principal Investigator: Melissa B. Gilkey 
May 23, 2022 

 ____________________________________________________________________________________________ 
ii 
TABLE OF CONTENTS 
 
Page 
Table of Contents ............................................................................................................. ii 
Protocol Summary ........................................................................................................... iv 
1 
Key Roles ................................................................................................................ 1 
2 
Introduction: Background Information and Scientific Rationale ............................... 3 
2.1 
Background Information ............................................................................... 3 
2.2 
Rationale ...................................................................................................... 3 
2.3 
Potential Risks and Benefits ......................................................................... 4 
2.3.1 Potential Risks ................................................................................... 4 
2.3.2 Known Potential Benefits ................................................................... 4 
3 
Objectives ............................................................................................................... 5 
3.1 
Study Objectives ........................................................................................... 5 
3.2 
Study Outcome Measures ............................................................................ 5 
3.2.1 Primary Outcome Measures .............................................................. 5 
3.2.2 Other Measures ................................................................................. 5 
4 
Study Design ........................................................................................................... 7 
5 
Study Enrollment and Withdrawal ........................................................................... 8 
5.1 
Subject Inclusion Criteria .............................................................................. 8 
5.2 
Subject Exclusion Criteria ............................................................................. 8 
5.3 
Strategies for Recruitment and Retention ..................................................... 8 
5.4 
Treatment Assignment Procedures .............................................................. 8 
5.4.1 Randomization Procedures................................................................ 8 
5.4.2 Masking Procedures .......................................................................... 8 
5.4.3 Reasons for Withdrawal ..................................................................... 8 
5.4.4 Handling of Withdrawals .................................................................... 9 
5.4.5 Termination of Study .......................................................................... 9 
6 
Study Intervention ................................................................................................. 10 
6.1 
Study Intervention Description .................................................................... 10 
6.2 
Dosage, Preparation and Administration of Study Intervention .................. 11 
6.3 
Modification of Study Intervention/Investigational Product for a Subject .... 11 
6.4 
Accountability Procedures for the Study Intervention ................................. 11 
6.5 
Assessment of Subject Compliance with Study Intervention ...................... 11 
6.6 
Concomitant Medications/Treatments ........................................................ 11 
7 
Study Schedule ..................................................................................................... 12 
7.1 
Screening ................................................................................................... 12 
7.2 
Enrollment/Baseline .................................................................................... 12 
7.3 
Follow-up .................................................................................................... 12 

 ____________________________________________________________________________________________ 
iii 
7.4 
Final Study Visit .......................................................................................... 12 
7.5 
Early Termination Visit ................................................................................ 12 
7.6 
Unscheduled Visit ....................................................................................... 12 
8 
Statistical Considerations ...................................................................................... 13 
8.1 
Study Hypotheses ...................................................................................... 13 
8.2 
Sample Size Considerations ...................................................................... 13 
8.3 
Planned Interim Analyses (if applicable) ..................................................... 13 
8.4 
Final Analysis Plan ..................................................................................... 13 
9 
Ethics/Protection of Human Subjects .................................................................... 15 
9.1 
Ethical Standard ......................................................................................... 15 
9.2 
Institutional Review Board .......................................................................... 15 
9.3 
Informed Consent Process ......................................................................... 15 
9.4 
Exclusion of Women, Minorities, and Children (Special Populations) ......... 15 
9.5 
Subject Confidentiality ................................................................................ 15 
9.6 
Study Discontinuation ................................................................................. 15 
9.7 
Future Use of Stored Specimens................................................................ 16 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 ____________________________________________________________________________________________ 
iv 
PROTOCOL SUMMARY 
Title: 
Evaluating the Feasibility of a Mobile Coaching 
Intervention to Improve HPV Vaccine Delivery 
Summary: 
 
The purpose of this study is to examine the feasibility of 
using Checkup Coach, a mobile coaching intervention, to 
improve the way that primary care providers recommend 
HPV vaccination to adolescent patients and their parents. 
To conduct this feasibility study, we will deliver the 
Checkup Coach intervention to Kaiser Permanente 
Washington primary care providers who routinely 
recommend HPV vaccine to adolescent patients. We will 
use a single-arm, pre-post design. Participating primary 
care providers will attend a 1-hour virtual communication 
workshop and then use a mobile phone app to receive 
additional coaching for 12 weeks. Providers will complete 
surveys at three time points: before the communication 
workshop, immediately after the workshop, and at 12-week 
follow-up. Surveys will assess changes in providers' self-
reported HPV vaccine recommendation practices and 
beliefs about HPV vaccine, as well as the acceptability of 
the intervention. We hypothesize that providers' HPV 
vaccine communication will improve between baseline and 
12-week follow-up. 
Objectives: 
 
To evaluate whether the Checkup Coach intervention 
improves providers’ HPV vaccine recommendation quality. 
Primary outcome measure: The change in providers’ 
composite score on a 5-item index of self-reported use of 
HPV vaccine communication practices from baseline to 12-
weeks post-intervention. 
Sample: 
Primary care providers who treat 11-12 year old patients in 
participating Kaiser Permanente Washington pediatric and 
family medicine practices. 
Phase: 
Not applicable 

 ____________________________________________________________________________________________ 
v 
Study Duration: 
June 21, 2021 through January 14, 2022 
Participation Duration: 
Providers will attend a one-hour communication training 
workshop and use the Checkup Coach mobile phone app 
at their discretion for 12 weeks. 
Description of 
Intervention: 
 
The Checkup Coach intervention is a 1-hour virtual 
workshop on HPV vaccine communication, followed by 12 
weeks of access to a mobile app that provides additional 
coaching.  
Estimated Time to 
Complete Enrollment: 
4 months 

 ____________________________________________________________________________________________ 
vi 
Schematic of Study Design: 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
KPWA primary care clinics 
invited to participate 
Exclude 
Not primary care clinics; 
Outside of Puget 
Sound/Spokane region; 
Decline to participate; 
Do not see patients, ages 
11-12 
Participating primary care 
clinics 
Primary care providers attend 
communication workshop 
Primary care providers do 
not consent to participate 
Primary care providers 
invited to participate 
Deliver communication workshop 
  
Mobile app usage 
  
Asses change in HPV vaccine 
recommendation quality score 

 ____________________________________________________________________________________________ 
1 
1 
KEY ROLES 
 
Principal Investigator: 
Melissa B. Gilkey, PhD1,2 
Intervention Design:  
Melissa B. Gilkey, PhD1,2 
Nora Henrikson, PhD, MPH3 
Annie-Laurie McRee, Dr P.H., MPH4 
John Dunn, MD, MPH5 
Jennifer Heisler-MacKinnon, MPH1,2 
Brigid K. Grabert, PhD, JD, MPH1,2 
Consuelo M. Norris, MA6 
Paula R. Blasi, MPH3 
Data Analysis and 
Reporting 
Melissa Gilkey, PhD1,2 
Brigid K. Grabert, PhD, JD, MPH1,2 
Nora Henrikson, PhD, MPH3 
Annie-Laurie McRee, Dr P.H., MPH4 
Jennifer Heisler-MacKinnon, MPH1,2 
Matthew B. Nguyen3 
Mary Catharine McKeithen2 
Institutions: 
 
1Lineberger Comprehensive Cancer Center 
University of North Carolina 
CB 7295 
Chapel Hill, NC 27599 
Phone: 919-966-3036 
 
2Gillings School of Global Public Health, University of 

 ____________________________________________________________________________________________ 
2 
North Carolina 
 
3Kaiser Permanente Washington Health Research Institute 
 
4Population Sciences and Epidemiology Branch, Center for 
Scientific Review, National Institutes of Health 
 
5Kaiser Permanente Washington 
 
6Office of Performance, Strategy and Budget, King County, 
Seattle, Washington 

 ____________________________________________________________________________________________ 
3 
2 
INTRODUCTION: BACKGROUND INFORMATION AND 
SCIENTIFIC RATIONALE 
2.1 Background Information 
 
HPV is a highly prevalent sexually transmitted infection associated with over 34,000 
cancers in the US each year. The annual cost of screening and treatment for HPV-
related diseases totals $8 billion. Widespread HPV vaccination could prevent the 
infections that lead to most of these diseases, making the vaccine a powerful tool for 
cancer prevention. 
 
National recommendations are for the completion of the 2-dose HPV vaccine series at 
ages 11-12. Timely completion is critical for protecting youth before HPV exposure and 
may enhance immune response. On-time vaccination, however, is not the norm in the 
US, with coverage at just 39% for 13-year-olds in 2017. This coverage is far below the 
Healthy People 2020 goal of 80% for ages 13-15, and suggests that interventions are 
urgently needed to improve uptake.  
 
The present study seeks to improve healthcare providers’ communication about HPV 
vaccine which is critical to increasing uptake. A provider’s recommendation is among 
the strongest predictors of HPV vaccination. Adolescents who receive a high-quality 
recommendation versus none have over 9 times higher odds of series initiation. 
Additionally, interventions that include provider communication training improve HPV 
vaccination coverage.  
 
Mobile apps are a promising way to extend provider training. Most providers use mobile 
devices professionally, and apps are an increasingly common vehicle for provider 
education. Whether app-based coaching could improve providers’ HPV vaccine 
communication is unknown. However, the approach has been successfully piloted in 
other contexts (e.g., critical care), suggesting promise.  
 
2.2 Rationale 
 
Our study seeks to evaluate the feasibility of Checkup Coach, a quality improvement 
(QI) coaching intervention, including a mobile app, that we have developed to deliver 
HPV vaccine communication training. At the provider level, Checkup Coach is designed 
to improve HPV vaccine communication via ongoing assessment and tailored feedback. 

 ____________________________________________________________________________________________ 
4 
At the clinic level, the app will provide quality improvement tools, including graphs of 
clinics’ and providers’ HPV vaccine coverage (i.e., % of patients vaccinated), for 
tracking shared goals for improvement. We will conduct this study to evaluate the 
feasibility of using the QI coaching intervention to improve HPV vaccine communication 
in primary care settings. If successful, this study will inform the future development of a 
randomized controlled trial to examine the impact of the Checkup Coach intervention on 
HPV vaccine uptake.  
2.3 Potential Risks and Benefits 
2.3.1 Potential Risks 
Potential short-term risks associated with participation include healthcare providers’ loss 
of time and slight risk of embarrassment or discomfort in answering questions about 
their HPV vaccine recommendation practices. However, because the communication 
workshop is brief (<1 hour) and app use is discretionary, we expect loss of time to be 
minimal. Additionally, providers will have the option to refuse to answer any survey 
questions and can decline to use the mobile app. All answers to survey questions and 
data entered into the app will be kept confidential. Thus, the risk of embarrassment is 
anticipated to be very slight. 
We do not anticipate immediate or long-range risks associated with participation.  
2.3.2 Known Potential Benefits 
By participating in this study there is a potential to increase provider knowledge and 
confidence for recommending HPV vaccination as a routine preventive service for 
adolescents. This study will also help inform a future larger randomized trial of the 
Checkup Coach intervention. The goal of that trail would be to develop a quality 
improvement tool that will help increase HPV vaccination coverage which could reduce 
the future incidence of HPV-related cancers.   

 ____________________________________________________________________________________________ 
5 
3 
OBJECTIVES 
3.1 Study Objectives 
The purpose of this study is to examine the feasibility of using Checkup Coach, a QI 
coaching intervention, to improve primary care providers’ use of evidence-based 
communication practices to support routine HPV vaccination. 
3.2 Study Outcome Measures 
Outcome measures will be changes in providers’ HPV vaccine recommendation 
practices and the feasibility of the Checkup Coach quality improvement intervention. 
Data will come from provider surveys at three time points and mobile app usage data. 
3.2.1 Primary Outcome Measures 
For each participating provider, we will assess the change in composite score on a 5-
item index of self-reported HPV vaccine recommendation quality from baseline to 12-
week post-intervention. Scores range from 5-25 (5 indicating the lowest quality score to 
25 indicating the highest quality score). 
3.2.2 Other Measures 
We will assess the feasibility of the Checkup Coach intervention by evaluating: 
• Use of the Checkup Coach mobile coaching app (measured as % of providers 
who use the Checkup Coach mobile app, assessed using app analytics; and 
measured as the mean number of times that app users access the app, 
assessed using app analytics). 
• Intervention acceptability (measured as mean ratings for intervention 
convenience, organization, helpfulness, and whether participants would 
recommend it to a colleague, assessed using 5-point survey items). 
 

 ____________________________________________________________________________________________ 
6 
4 
STUDY DESIGN 
This study will evaluate the Checkup Coach intervention using a single-arm, pre-post 
design with approximately 20 primary care providers at Kaiser Permanente 
Washington (KPWA). The intervention has two main components. In the first part of the 
intervention, participants attend a communication training workshop via a virtual 
conferencing platform. The workshop will be conducted by study staff and the KPWA 
Director of Preventive Care. The brief (<1 hour) workshop reviews evidence on HPV 
vaccination, models guideline-consistent recommendation practices, trains providers to 
address common parent concerns, and facilitates discussion of shared goals for 
improving. Workshop content is evidence-based and reflects communication practices 
supported by research evidence and recommended by the Centers for Disease Control 
and Prevention. At the end of the workshop, providers are asked to download and 
practice using the Checkup Coach mobile app.  
In the second part of the intervention, which starts after the communication training, 
providers are invited to use the Checkup Coach mobile app for 12 weeks. The app is a 
QI tool designed to provide tips and tools for improving HPV vaccine communication. 
App functions include: (a) provider self-assessment of HPV vaccine recommendation 
practices; (b) feedback on their HPV recommendation quality in the form of a 
“recommendation goal score”; (c) research-tested messages for addressing parent 
concerns that are tailored to in-app self-assessments; and (d) a dashboard with graphs 
of provider- and clinic-level HPV vaccination coverage (i.e. % of 11-12 year old 
adolescents vaccinated).  

 ____________________________________________________________________________________________ 
7 
5 
STUDY ENROLLMENT AND WITHDRAWAL 
5.1 Subject Inclusion Criteria 
In order to be eligible to participate in this study, a provider must: 
• Be a primary care provider in an invited KPWA pediatric or family medicine 
clinics in the Puget Sound/Spokane area 
• Regularly treat adolescent patients, ages 11-12 
5.2 Subject Exclusion Criteria 
Providers not eligible to participate: 
• Provide care in a clinic not invited to participate in the study 
• Do not regularly treat adolescent patients, ages 11-12 
5.3 Strategies for Recruitment and Retention 
 
The KPWA Medical Director for Preventive Care will identify KPWA primary care clinics 
in the Puget Sound/Spokane area that administer HPV vaccine to adolescent patients, 
ages 11-12. He will invite clinics to participate in the study by contacting each site’s 
clinical director. Once clinical directors agree to have their clinics participate, study staff 
will invite all primary care providers who regularly treat 11-12 year old patients in each 
clinic to participate.  
5.4 Treatment Assignment Procedures 
5.4.1 Randomization Procedures 
None. 
5.4.2 Masking Procedures 
None. 

 ____________________________________________________________________________________________ 
8 
5.4.3 Reasons for Withdrawal 
Participants who request to withdraw will be given the option to discontinue participation 
in the intervention but continue participation in follow-up data collection activities. 
Participants who refuse this offer and request to completely withdraw from the study will 
no longer be contacted. There will be no investigator-initiated withdrawal. 
5.4.4 Handling of Withdrawals 
See above. 
5.4.5 Termination of Study  
Given the low-risk nature of this study, we do not anticipate grounds for premature 
termination. 

 ____________________________________________________________________________________________ 
9 
6 
STUDY INTERVENTION 
6.1 Study Intervention Description 
 
The Checkup Coach intervention offers two main components to primary care 
providers who care for adolescent patients: a webinar communication workshop and 
use of the Checkup Coach mobile app. 
 
1. Communication Workshop. The Checkup Coach intervention begins with a 
brief (<1 hour), communication training workshop. Using a script and slides, the 
KPWA Director for Preventive Care and the study PI will meet with participating 
providers over a web-based meeting platform to: review evidence on HPV 
vaccination; model high-quality, guideline-consistent recommendations; train 
providers to address parent concerns; and facilitate discussion of shared goals 
for improving.  
 
At the end of the training, providers will be asked to download and begin using 
the Checkup Coach app. Downloading and using the app will be voluntary, and 
providers can consent to participate in the webinar training but decline all other 
study activities.  
 
2. Checkup Coach app. Providers will be invited to use the Checkup coach mobile 
app at their discretion for the 12 week study period. If they have push 
notifications enabled, providers will be sent push notifications reminding them to 
use the app up to six times over the 12 week period. Providers will complete the 
in-app self-assessments and receive tailored feedback every time they use the 
app. 
A detailed description of the Checkup Coach self-assessment and tailored 
feedback is as follows: 
 
• Self-assessment of recommendation quality. Providers complete a very 
brief in-app self-assessment to determine how often in the last week they 
used 5 practices effective for recommending HPV vaccine according to 
national guidelines. Guidelines include delivering recommendations on 
time (i.e., ages 11-12), consistently to all eligible patients, and offering 
same-day vaccination services when possible. The app adds 5-point 

 ____________________________________________________________________________________________ 
10 
response options to give an overall score and to identify areas for 
improvement. 
• Recommendation Goal Score. The app displays a bar chart of providers’ 
overall self-assessed recommendation quality score in reference to the 
goal of 5 (i.e., “almost always” using all 5 practices).  
• Dashboard. The app presents line graphs of aggregate HPV vaccination 
coverage (i.e., % of patients vaccinated) over the study period for both the 
individual provider and their clinic. 
• 
Tips for addressing HPV vaccine communication challenges. Providers 
complete an in-app 5-item self-assessment to identify parent concerns 
they “often” or “almost always” find difficult to ease. The app pulls from a 
library of 15 messages to suggest a research-tested response tailored to 
each concern (called “Tips for Talking to Parents” in the app). Those who 
indicate no concern receive randomly selected messages with 
encouragement to keep up their good work. 
6.2 Dosage, Preparation and Administration of Study 
Intervention 
Each primary care provider will attend one communication workshop which will last less 
than one hour. They will then be able to use the mobile app, at their discretion, for up to 
12 weeks. 
6.3 Modification of Study Intervention for a Subject 
Not applicable. 
6.4 Accountability Procedures for the Study Intervention 
Not applicable. 
6.5 Assessment of Subject Compliance with Study Intervention  
Not applicable. 
6.6 Concomitant Medications/Treatments 
Not applicable. 

 ____________________________________________________________________________________________ 
11 
7 
STUDY SCHEDULE 
7.1 Screening 
Screening of eligible clinics and providers will be conducted by the KPWA Director of 
Preventive Care, using data from KPWA’s internal records. 
7.2 Enrollment/Baseline 
Enrollment and pre-workshop data collection will occur simultaneously from July – 
September 2021. Communication workshops and post-workshop data collection will 
occur in the same period, from July – September 2021. 
7.3 Follow-up  
Mobile app use and data collection through the app will occur from July – December 
2021. Twelve-week follow-up data collection will occur from October 2021 – January 
2022. 
7.4 Final Study Visit 
Not applicable. 
7.5 Early Termination Visit 
Not applicable. 
7.6 Unscheduled Visit  
Not applicable. 

 ____________________________________________________________________________________________ 
12 
8 
STATISTICAL CONSIDERATIONS 
8.1 Study Hypotheses 
The Checkup Coach intervention will increase providers’ HPV vaccine recommendation 
quality score. 
8.2 Sample Size Considerations 
 
We will test our hypothesis that Checkup Coach will increase providers’ overall 
recommendation quality over the 12-week study period with approximately 20 providers. 
We acknowledge that our trial is underpowered to detect small differences or control for 
multiple testing. The purpose of a feasibility study is to estimate effect sizes for 
powering a large-scale RCT. By piloting our intervention in 5 or more different clinics, 
our study is designed to provide the data needed to lay this groundwork. 
8.3 Planned Interim Analyses (if applicable) 
Not applicable. 
8.4 Final Analysis Plan 
To assess change in providers’ HPV vaccine recommendation behavior, we will 
compare providers’ mean recommendation quality index scores between baseline and 
12-week follow-up using Wilcoxon signed-rank tests. To assess changes in providers’ 
cognitions and perceptions of their social environments, we will use Wilcoxon signed-
rank tests to compare mean scores for each survey item and the knowledge index item 
(scores ranging from 1-5) between the baseline and post-workshop surveys and the 
baseline and 12-week follow-up surveys.  

 ____________________________________________________________________________________________ 
13 
9 
ETHICS/PROTECTION OF HUMAN SUBJECTS 
9.1 Ethical Standard 
The investigator will ensure that this study is conducted in full conformity with the 
principles set forth in The Belmont Report: Ethical Principles and Guidelines for the 
Protection of Human Subjects of Research, as drafted by the US National Commission 
for the Protection of Human Subjects of Biomedical and Behavioral Research (April 18, 
1979) and codified in 45 CFR Part 46 and/or the ICH E6; 62 Federal Regulations 25691 
(1997). 
9.2 Institutional Review Board 
Study procedures have been approved by the Kaiser Permanente Washington Health 
Research Institute Human Subjects Review Office. 
9.3 Informed Consent Process 
Informed consent will be obtained from all participating providers, via Qualtrics online 
survey software, prior to participation in baseline data collection and the communication 
workshop. 
9.4 Exclusion of Women, Minorities, and Children (Special 
Populations) 
Not applicable. 
9.5 Subject Confidentiality 
Subject confidentiality is strictly held in trust by the participating investigators, their staff, 
and the sponsor(s) and their agents. The study team will take precautions to ensure that 
subject confidentiality is maintained. All data will be stripped of patient-level identifiers 
and stored on a password-protected server accessible only to UNC study staff. Data will 
be reported in aggregate and will not identify participating clinics or providers by name. 
9.6 Study Discontinuation 
Not applicable. 

 ____________________________________________________________________________________________ 
14 
9.7 Future Use of Stored Specimens 
Not applicable.