Improving HPV Vaccination Delivery in Pediatric Primary Care: The STOP-HPV Trial 1. Comparison of Communication Skills and Standard of Care for Uptake of the HPV Vaccine
Phase: NA
Conditions: Immunization, Vaccination
Sponsor: University of California, Los Angeles

NCT03599583 
Improving HPV Vaccination Delivery in Pediatric Primary Care: The STOP-HPV Trial 1. 
Comparison of Communication Skills and Standard of Care for Uptake of the HPV Vaccine. 
Statistical  Analysis Protocol 
UCLA IRB Approval Date 1/9/2018 (IRB#17-001772) 

Protocol and Analyses 
1 
Study Title: Impact of a Communication Intervention for Clinicians on Missed Opportunities for 
2 
HPV Vaccination: A Randomized Controlled Trial from the American Academy of Pediatrics 
3 
Pediatric Research in Office Settings (PROS) Network 
4 
 
5 
Clinical Trials Registration: NCT03599557 
6 
 
7 
 
8 
Brief Summary: 
9 
This trial, part of a multi-year comprehensive intervention, is located in pediatric practices and 
10 
health systems within the American Academy of Pediatrics Pediatric Research in Office Settings 
11 
(PROS) Network. Most adolescents who receive human papillomavirus (HPV) vaccine are 
12 
vaccinated in pediatric practices, yet missed opportunities (MOs) for HPV vaccination occur 
13 
often and lead to low HPV vaccination rates. This cluster randomized clinical trial (RCT) will test 
14 
the effectiveness (and cost-effectiveness) of training providers on HPV vaccine communication 
15 
to reduce MOs and increase HPV vaccination rates. 
16 
 
17 
Methods and Procedures 
18 
 
For this first part of the intervention, there are 2 study periods. We will randomize by practice 
19 
(not provider) to avoid contamination. We will assess missed opportunities (MOs) and HPV 
20 
vaccine rates using monthly and period-aggregate data, excluding the periods of ramping up of 
21 
each intervention.   
22 
Period 0: This is the 12-month baseline period.     
23 
Period 1: Arm-1 practices will receive communication skills training. Arm-2 control practices 
24 
will receive standard of care.  The duration of the communication intervention period will be 6 
25 
months.    
26 
  
27 
All analyses will be stratified by visit type (well or sick) and dose (initial vs subsequent). We 
28 
consider MOs by visit type and dose separately as we hypothesized that MOs for different visit 
29 
types and doses are driven by different factors. For the test of this communication intervention, 
30 
we hypothesized that since almost all HPV vaccinations (and nearly 100% of initial doses) are 
31 
provided during well-child care visits, an intervention focused upon communication may be 
32 
potentially effective especially during well-child care visits.  
33 
 
34 
Future interventions that focus upon changing practice procedures (e.g., providing feedback to 
35 
providers with suggestions for utilizing sick/chronic visits, or prompts to remind providers to to 
36 
consider HPV vaccinations) may have more of an impact at sick/chronic visits and for 
37 
subsequent doses.  
38 
 
39 
 
40 
Study Design 
41 
Study Type: Interventional (Clinical Trial) 
42 
 
43 
Actual Enrollment: 48 participants, where each participant is a pediatric primary care practice, to 
44 
include all eligible patients at the practices during the two periods.   
45 
 
46 
Allocation: Randomized 
47 
 
48 
Intervention Model: Parallel Assignment 
49 
 
50 

Intervention Model Description: This study will use a cluster RCT study design to test the impact 
51 
of the communication intervention to reduce missed opportunities (MOs) and raise HPV vaccine 
52 
rates.  
53 
 
54 
Arms and Interventions: 
55 
 
Arm 1 will receive the STOP-HPV communication intervention 
56 
 
Arm 2 will receive standard of care 
57 
 
58 
 
59 
 
60 
Inclusion and Exclusion criteria 
61 
Practice Inclusion Criteria: 
62 
 
The practice provides HPV vaccination services to adolescents. 
63 
 
The practice is part of Physician's Computer Company (PCC), Office Practicum (OP) or 
64 
(a) yet-to-be selected health system(s). 
65 
 
The practice has had the same EHR system in place for a year or more (with special 
66 
consideration on a case by case basis if they are close to but not do not reach a year). 
67 
 
The practice agrees to not participate in other HPV-related QI projects or research 
68 
interventions during the study period (with special consideration on a case by case 
69 
basis). 
70 
 
71 
Practice Exclusion Criteria: 
72 
 
The practice plans to change EHR systems in the next three years. 
73 
 
The practice participated in the last year, is currently engaged in, or plans to participate 
74 
in an office-based HPV-related quality improvement (QI) project or research intervention 
75 
during the study period (with special consideration on a case by case basis). 
76 
 
Estimated 20% or more of adolescents at the practice receive HPV vaccinations at 
77 
schools or health department clinics (given standard practice and published data, the 
78 
investigators expect that few to no practices will need to be excluded based on this 
79 
restriction). 
80 
 
81 
Patient inclusion criteria: 
82 
 
All patients of participating practices (intervention and comparison) aged 11-17 years 
83 
who have at least 1 visit to the practice within the past two years. 
84 
 
85 
Patient exclusion criteria: 
86 
 
None apart from age of patients (above). 
87 
 
 
88 

 
89 
Main Outcomes and Measures 
90 
Primary Outcome Measures: 
91 
1. Rate and change in the rate of missed vaccination opportunities among all clinicians 
92 
[ Time Frame: Monthly from months 0 (baseline) to month 6, and also aggregated over 
93 
the 6-month period from month 0, where month 0 is site specific and depends on 
94 
completion of site staff training and readiness to proceed. ] 
95 
Change in the rate of missed vaccination opportunities from baseline through the end 
96 
of the communication intervention period among all clinicians. 
97 
2. Rate and change in the rate of missed vaccination opportunities among consenting 
98 
clinicians [ Time Frame: Monthly from months 0 (baseline) to month 6, and also 
99 
aggregated over the 6-month period from month 0, where month 0 is site specific and 
100 
depends on completion of site staff training and readiness to proceed. ] 
101 
Change in the rate of missed vaccination opportunities from baseline through the end 
102 
of the communication intervention period among consenting clinicians. 
103 
Analysis 
104 
Primary Analysis 
105 
All analyses will be stratified by visit type (well or sick) and dose (initial vs subsequent). Visit-
106 
level analyses will implement marginal models (such as estimated with generalized estimating 
107 
equations (GEE) (Fitzmaurice 2011) to estimate relative and absolute changes in rates between 
108 
intervention and control practices, and over time. These methods facilitate estimation of 
109 
standardized (for patient-level characteristics)  (Korn and Graubard 1999) rates of MOs over 
110 
time. Assumptions include a large number of practice sites (Hayes and Moulton 2017), an 
111 
assumption that our study satisfies for comparisons of all 48 sites.  With fewer than 40 clusters, 
112 
however, some form of correction is warranted to correct confidence bounds.  (Huang 2016).  
113 
GEE methods will use marginal mean models with a logit link (binary family) as is appropriate 
114 
for binary outcomes for missed opportunities. The clustering level will be the practice site for all 
115 
analyses.   This approach will not consider clustering of clinicians within practice site.   This 
116 
approach will also not consider the repeated measures of visits within child.    Children are not 
117 
nested within clinician, as a child can be seen by more than one clinician.    
118 
 
119 
GEE methods will produce robust standard errors and respective confidence bounds directly for 
120 
odds ratios. Odds ratios, however, are not easy to interpret for even rates that are common.  For 
121 
that reason, we will transform log odds estimates and their ratios into risk ratios and risk 
122 
differences, and then use the delta method to estimate robust confidence bounds on the risk 
123 
difference scale.  Secondary analyses will include standardization by visit and patient-level 
124 
factors prior to transformation. 
125 
 
126 
Survey methods provide a computational alternative to GEE methods, which can fail with large 
127 
cluster sizes, and assume independence working correlation structures.   Survey methods will 
128 
be used to replace GEE algorithms if the GEE algorithms fail.   Such failure can occur when 
129 
clusters (practice sites) are large.   SAS, Stata, and R all have survey algorithms for logit link 
130 
model.      
131 

 
132 
Confidence bounds, alternatively, will be generated using percentile-based, bootstrap 
133 
resampling of the practices to account for correlation of visit-level measures within practices to 
134 
verify the validity of robust standard errors output by standard GEE procedures (Davison 1997) 
135 
at our sample size of practices. Confidence bounds based on resampling will protect against 
136 
large-sample assumptions made by standard implementations of GEE(Goodman 1994).   
137 
 
138 
Programs for resampling are well described in R (Cluster.Bootstrap (2018), Stata, and in SAS 
139 
(PROC SURVEY SELECT).   We will use whichever one is easiest.   
140 
 
141 
Secondary Analysis 
142 
Person-level analyses (vaccination rates) will employ the same marginal model methods as 
143 
visit-level analyses. 
144 
 
145 
 
146 
 
147 
 
148 
 
149 
 
150 
References 
151 
 
152 
Davison AC, Hinkley DV. Bootstrap methods and their application. Cambridge ; New York, NY, 
153 
USA: Cambridge University Press; 1997. 
154 
 
155 
Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis. 2nd ed. Hoboken, N.J.: 
156 
Wiley; 2011. 
157 
 
158 
Goodman, Steven N., and Jesse A. Berlin. "The use of predicted confidence intervals when 
159 
planning experiments and the misuse of power when interpreting results." Annals of internal 
160 
medicine 121.3 (1994): 200-206. 
161 
 
162 
Hayes RJ, Moulton LH. Cluster randomised trials, second edition. CRC Press, 2017 
163 
 
164 
Huang S, Fiero MH, Bell ML. Generalized estimating equations in cluster randomized trials with 
165 
a small number of clusters: Review of practice and simulation study. Clin Trials. 2016; 
166 
13(4):445-9. [PMID: 27094487] 
167 
 
168 
Korn EL, Graubard BI. Analysis of health surveys. New York: Wiley; 1999. 
169 
 
170 
 
171 
.  
172 
 
173