Comparative Effectiveness of Interactive, Health Literacy Promoting Text Messages on HPV Vaccine Completion In Minority Adolescents
Phase: NA
Conditions: Human Papillomavirus
Sponsor: Columbia University

 
 
 
 
 
 
 
Interactive, Health Literacy 
Promoting Text Messages and HPV 
Vaccine Completion In Minority 
Adolescents 
 
 
ClinicalTrials.gov ID: NCT02236273 
IRB approval date: January 5, 2021 
 
 

Study Purpose and Rationale 
Vaccination is one of the most effective public health interventions1, yet fundamental 
problems exist in translating highly effective vaccines into vaccination coverage levels 
sufficient to achieve population immunity or to realize their prevention potential. 
Phenomenal successes in eradication of older vaccine-preventable disease such as 
diphtheria, smallpox and polio have been unachievable for many other vaccinations 
such as influenza, human papillomavirus, and measles. Additionally, significant 
disparities in vaccination coverage exist for low-income and other disadvantaged 
populations.2,3 While other factors play a role, limited vaccine health literacy such as 
decreased knowledge about disease risk or misperceptions about vaccination is 
associated with pediatric and adult under-vaccination4-12. Emerging communication 
technologies, such as text messaging, linked to electronic health records (EHR) data 
offer low-cost, scalable opportunities to improve health literacy and to promote healthy 
behaviors, such as vaccination. Our work is at the forefront of this emerging field.13,14 
Nearly all U.S. adults (87%) have a cell phone, with higher rates (93%) among 30–49-
year-olds.15 Additionally, most cell phone owners use text messaging.15 Text messaging 
platforms such as ours can deliver thousands of messages simultaneously, allowing 
significant and rapid penetration of large target populations. Parents value health-
related text messages, including those for vaccination,16-19 and we have reported 
successful trials of text message vaccinationreminders.13,14,20 These trials took place in 
low-income populations for whom vaccination disparities exist, but for whom traditional 
mail and phone vaccination reminders have had limited effects.21-24 While our first text 
messages were non-enhanced reminders that solely notified parents that their child was 
due for vaccination, our next generation of messages included health literacy promoting 
information.14,20 These messages primarily used an untailored approach that did not 
maximize their effect. The trans-theoretical model of behavior change supports tailoring 
interventions to an individual’s stage of decision-making.25 While there is evidence for 
the effectiveness of this model in changing vaccination behavior,26-28 it has not been 
tested in a clinical trial. We propose to assess the next generation of text messages 
enhanced with interactive, vaccine health literacy-promoting information tailored 
individual families’ stages of vaccination decision-making. Human papillomavirus (HPV) 

is the most prevalent sexually transmitted infection in the U.S., and can lead to genital 
warts, and cervical, anal and penile cancer.29-36 Despite the availability of a highly 
effective vaccine,37-40 completion rates of the 3-dose vaccine series remain low, with 
only one third of female adolescents and 1% of male adolescents protected.3,41-45 
Minorities have higher HPV prevalence and greater cervical and penile cancer 
incidence,29,46 but lower vaccine completion rates.3,42,47 Using a preliminary non-
randomized design, we demonstrated the effectiveness of non-enhanced text message 
HPV vaccine reminders on increased receipt of the next dose due.20 While substantial 
vaccination differences were seen, overall completion rates remained low. Limited 
health literacy about the HPV-vaccine can affect series completion,48,49 and there is an 
indication for the potential use of the trans-theoretical model for changing HPV 
vaccination behavior.26,27 As HPV vaccine recommendations have not translated into 
high vaccination coverage levels, these new generation of tailored, text messages have 
potential for stronger disease prevention, but have not been assessed. In this study, 960 
parents of female and male 9-17 year-olds receiving their first HPV vaccine dose will be 
randomized to: (1) enhanced text messages with interactive, vaccine health literacy-
promoting information tailored to vaccine decision making-stage vs. (2) non-enhanced 
text messages that only notify when the next dose is due. The primary outcome is 
completion of the 3-dose HPV vaccine series within 12 months of initiation. 
Aim: To compare the effectiveness, in a randomized controlled trial, of the addition of 
interactive, tailored vaccine-health-literacy-promoting information to text-message 
vaccine reminders in improving HPV vaccine series completion for minority adolescents. 
We hypothesize that completion will be higher in those whose families receive 
enhanced text message reminders vs. non-enhanced text message reminders. Linking 
a new technology with a novel use of a well-tested behavioral theory builds on our 
previous work, and has the potential to provide important evidence for the next 
generation of text message reminders both for vaccination and potentially for other 
health problems. It also has high potential for dissemination.  
Approach Overview: propose to build upon our prior research to determine, using a 
RCT, whether the provision of individualized vaccine health literacy-promoting 

information combined with reminders improves HPV vaccine series completion. We will 
randomize 960 primarily minority adolescents aged 9-17 years who have received their 
first HPV vaccine in equal numbers to: (1) enhanced text messages with interactive, 
vaccine health literacy-promoting information tailored to vaccine decision making-stage 
plus vs. (2) non-enhanced text messages solely notifying when the next dose is due. 
The primary outcome will be completion of the entire 3-dose series within 12 months 
from the randomized adolescent’s first HPV vaccine dose. Although we recognize we 
could include a partial vaccination outcome represented by receiving 2 doses, we define 
success as series completion since vaccine efficacy data are based on 
thisdefinition.37,39,50 If subsequently it is determined that two doses are protective, we 
will be able to analyze the intervention’s impact on 2 doses. 
Preliminary Studies 
1) HPV vaccine coverage: At the study sites, 87.8% of adolescent females and 52.2% 
of adolescent males 13-21 years-old received1 HPV vaccine dose; this exceeds 
other established rates in 13-17 year old females: 53.0% nationally, 46.6% New York 
(NY) State, and 56.8% NY City and in 13-17 year old males 8.0% nationally, 6.4% 
New York (NY)State, and 10.1% NY City.51 However, at the study sites, among 13-
17 year old adolescents who received their first HPV vaccine dose between June 
2010 and May 2011, only 56.8% of females and 45.6% of males received all 3 
doses. This is lower than reported at the national (69.6%), and at the NY state 
(76.9%) and NY City (72.7%) levels during the same period.51 Higher initiation, but 
poorer completion rates is reflective of the national pattern for minority 
femaleadolescents.51 Among 11-17 years-olds, only 40.0% of females and 30.7% of 
males received 3 doses within 12 months. 
 
2) Use of text messaging, including for HPV and other adolescent vaccines reminders: 
In addition to the HPV text message reminder study, we also assessed the impact of 
text message reminders on receipt of meningococcal (MCV4) and/or tetanus, 
diphtheria, acellular pertussis (Tdap) vaccines.14 In a randomized intervention with 
urban adolescents without a MCV4 and/or Tdap vaccine and age-, gender- matched 

controls (n=361), adolescents whose parents were texted were more likely to be 
vaccinated at weeks 4 (15.4% vs. 4.2%; p<0.001). 
 
3) Experience conducting randomized controlled trials of text message vaccine 
reminders: We recently published the first large-scale RCT of text message 
vaccination reminders.13 We randomized 9,213 children and adolescents (6 months 
to 18 years) receiving care at 4 community-based clinics. This study is relevant to 
the present application for several reasons: (1) it was conducted in the same 4 
clinical sites we propose to use; and (2) we implemented this influenza vaccine text 
messaging/health literacy intervention successfully in English and Spanish. 
Specifically, we found that children whose parents were assigned to the text 
messaging intervention were more likely to receive an influenza vaccine both by a 
fall review date, 27.1% of the intervention vs. 22.8% of the usual care group 
(difference, 4.3% [95% CI, 2.3,6.3]; RRR, 1.19 [95% CI, 1.10,1.28]; P < .001), and 
by March 31 (intervention: 43.6% vs. 39.9% usual care (difference, 3.7% [95% CI, 
1.5,5.9]; [RRR], 1.09 [95% CI, 1.04,1.15]; P = .001). 
 
4) Experience conducting health literacy related interventions: In an NIH-funded study, 
we assessed an educational intervention to improve health literacy regarding upper 
respiratory infections in low-income, Latino families. Compared with controls, school 
age children of intervention parents were more likely to receive the 2009 
H1N1vaccine (56.5% vs. 26.1%; p<0.05) and their parents were more likely to 
receive a seasonal influenza vaccine (OR 2.7,95% CI 1.3,5.7) (unpublished data). 
 
5) Use of interactive vaccine health literacy-associated text messaging: We assessed 
the impact of text messages on receipt of influenza vaccine for 5,462 
children/adolescents still unvaccinated by early winter. In this difficult to vaccinate 
population, those whose parents received interactive vaccine health literacy-
promoting text messages were more likely to be vaccinated (38.5%) than both usual 
care (34.8%, p<0.05) and those receiving non-interactive text messages (35.3%, 
p<0.05) (unpublished data). In another study of 453 children in need of a second 

dose of influenza vaccine this (2012-13) season, (76.7%) of those whose parents 
received health literacy-promoting text messages including an interactive message 
were more likely to receive a second dose than those receiving non-enhanced 
reminders (67.8%) or solely a written reminder (58.7%;p<<0.01). 
 
6) Factors that affect HPV vaccine completion: We conducted a preliminary study 
addressing the need to identify key health literacy-related themes associated with 
HPV vaccine series completion. In the same population where this proposed RCT 
study would take place, we audio-recorded and transcribed 40 in-depth structured 
interviews of parent-adolescent dyads with adolescents who had received their first 
HPV vaccine 12 months previously; including adolescents who did and did not 
complete the HPV vaccine series within 12 months. Transcripts were analyzed 
utilizing thematic analysis.52 Most interviews (65.0%) were in Spanish. Adolescents 
were primarily Latino (88%), publicly insured (95%) and female (65.0%). One-third 
(36%) of parents had less than a high school education, and20% only graduated 
from high school. All parents (100%) reported initiating vaccination to protect their 
child, largely from cancer or less commonly from sexually transmitted infections 
(HPV). Nearly all (93%) viewed their health care provider as an important source of 
HPV vaccine information. Families were asked questions to elicit factors affecting 
their attitude toward completing the HPV vaccine series. The majority of responses 
(95%) fell into three themes: (1) once the decision was made to start, the series 
should be completed and all needed doses received; (2) protection of their child; (3) 
presumption of a good experience with subsequent doses if the first one went well. 
Half of parents mentioned side effect or vaccine safety concerns. These concerns 
were similar for the parents of female vs. male adolescents (46% vs. 50%) and 
Spanish vs. English-speaking parents (50% vs. 43%). In nearly all families, parents 
were seen as responsible for vaccination decisions and the logistics of ensuring the 
adolescent received the needed doses. Although most parents and adolescents 
knew the HPV vaccine was a multidose series, the majority 90% did not know the 
correct intervals between doses. Of note, all who knew the correct timing for both 
doses completed the series within 12 months. Over 90% of parents and adolescents 

thought that they received their HPV vaccine doses “on-time”, but when records 
were checked, only (45%) received second and third doses on-time. All but one 
parent was interested in receiving HPV vaccine text message reminders, and none 
had concerns, confirming our previous study findings.16 Overall, parents wanted 
messages to include both positive and negative information. In addition to 
information regarding the number and timing of doses, all participant comments 
regarding desired content for messages, except one, could be divided into 3 themes: 
(1) what the vaccine protects against and why it is important; (2) what can happen if 
someone is not fully vaccinated, and (3) vaccine side effects and safety. These 
themes were the same across all gender and language sub-groups. These results 
provide important information that will be useful for this proposed intervention. First, 
parents were seen as the ones responsible for vaccination decisions and logistics, 
which supports our plan to text the parents of the adolescent. Second, health care 
providers were viewed as a very common source of information; to underscore this 
connection we will tag the messages as coming from the clinic where the family was 
recruited, as in our previous studies. Third, in terms of parent suggested message 
content: (a) protection of their child was a key reason why families continued or 
planned to continue to vaccinate, and also a content area for which they wanted to 
receive messages about; (b) many families felt that once they decided to start the 
series, they should finish it, which relates to their suggested text message content 
regarding the consequences of not being fully vaccinated; (c) side effects and safety 
still remain a concern even after initiation and should be included in the text 
messages. This view relates to the reported parental attitude that they presume 
subsequent doses will be side-effect free if the first dose experience was good. The 
following themes did not differ based on the gender of the vaccine recipient or parent 
language: (1) reason for initiating vaccine, (2) attitude towards continuing the series, 
(3) identifying the parent as responsible for vaccination decisions, and (4) desired 
vaccine health literacy message content. We interpret these findings as a basis for 
not tailoring the message content for Arm 3 by gender or language. Parents will be 
able to self-tailor the content of the messages by requesting more information about 

certain topics, which will be sent to them automatically. Differences in requested 
information can be assessed in the analytic phase. 
Research Aims & Abstracts 
Research Aim and Hypothesis: To compare the effectiveness, in a randomized 
controlled trial, of the addition of interactive, tailored vaccine-health-literacy-promoting 
information to text-message vaccine reminders in improving HPV vaccine series 
completion for minority adolescents. We hypothesize that completion will be higher in 
those whose families receive enhanced text message reminders vs. non-enhanced text 
message reminders. 
Target enrollment: 15,000 
Scientific Abstract: Vaccination is one of the most effective public health interventions, 
yet fundamental problems exist in translating highly effective vaccines into high 
vaccination coverage. Emerging communication technologies, such as text messaging 
offer low-cost, scalable opportunities to improve health literacy and promote healthy 
behaviors, such as vaccination. While we reported the success of text message vaccine 
reminders, effects were limited by their untailored approach. The trans-theoretical model 
of behavior change supports tailoring interventions to an individual’s stage of decision-
making. Human papillomavirus (HPV) is the most prevalent sexually transmitted virus in 
the U.S. and can lead to genital warts, and cervical, anal and penile cancer. The three-
dose vaccine is 90-100% efficacious. Minorities are at greatest risk for such cancers but 
have low HPV vaccine completion rates. Limited health literacy regarding the vaccine 
can affect series completion. We will compare the effects of enhancing text message 
vaccination reminders with interactive, vaccine health literacy-promoting information 
tailored to vaccine decision making-stage on HPV vaccine series completion. The 
effects of these messages represent a new paradigm in interactive health 
communications. 
 
 

Study Design 
Study Sites 
This study will be conducted in four community health clinics affiliated that are part of 
the NewYork-Presbyterian Hospital (NYP) Ambulatory Care Network (ACN) and 
Columbia University. These clinics are in Health Professional Shortage Areas (HPSA) 
and care for many families living in the Washington Heights/Inwood section of 
northernManhattan.53 These practices provide >55,000 visits annually; 87% for publicly 
insured patients, and the vast majority of families are Latino. The Vaccines for Children 
(VFC) Program provides the majority of vaccines given at the practices for free. 
Although the patient population and providers are similar across sites, baseline 
variability between sites will be accounted for in the randomization. All vaccinations 
given at the study sites are documented in the NYP Immunization Registry, EzVac, of 
which Dr. Stockwell is Medical Director. The EzVac database contains over 2.2 million 
vaccinations administered to >275,000patients. EzVac extracts information about 
vaccinations directly from the provider order entry module of the electronic health record 
(EHR) making data accurate for HPV vaccines administered at the clinical sites. EzVac 
as part of its regular practice synchronizes data with the New York Citywide 
Immunization Registry (CIR) which is a population-based registry. NY City Public Health 
Law requires documentation for all vaccinations administered to those18 year- olds be 
submitted to CIR,54 which captures more than 85% of vaccines administered in NY 
City, and 93% of vaccines from the Vaccines for Children Program (VFC).55,56 Almost all 
(91.4%) adolescents at the four study sites are VFC-eligible, i.e. publicly insured or 
uninsured. Therefore, the few vaccines administered outside of the practice network to 
patients from the study sites are also captured in EzVac through this synchronization for 
patient care. Dr. Stockwell’s team has built a secure text messaging platform linked to 
EzVac. This platform allows custom text messages to be created, and sent out 
according to a schedule. Before a set of scheduled message is sent, the system can re-
query the EzVac database to identify if patients have received the appropriate 
vaccination and exclude them from receiving additional reminders. The text messaging 
platform is designed to be interactive; patients can text a reply to stop or resume 

reminders, toggle the language of messages, and ask for assistance. It is also already 
programmed to send and respond to interactive text messages. 
Arm 1: Message Design 
During grant month 1-2, using the information gathered from the preliminary studies 
regarding health literacy-related information affecting completion of the HPV vaccine, 
our previous texting studies, and the relevant literature, we will design the two sets of 
text messages (enhanced vaccine health literacy-promoting and non-enhanced text 
messages). We will follow procedures established in the development and testing of text 
messages, outlined below, used in our recent trial assessing the impact of influenza 
vaccine text message reminders.13 We will first design the message in English paying 
attention to reading level, as assessed by the Flesch–Kincaid readability test.57 Our 
previous text messages have been on a third grade reading level, which is in 
accordance with the American Medical Association health literacy-related guidelines that 
written materials be at no more than a 6th grade reading level.58 We strongly considered 
the need to tailor the vaccine health literacy-promoting messages in Arm 3 differently for 
Spanish- and English-speaking Latino families as well as parents of male and female 
adolescents. Yet, as previously described, our HPV vaccine interview study showed no 
differences by gender or language in elicited themes or desired content. Likewise, in our 
previous studies, we used equivalent English and Spanish content, and found no 
interaction between intervention effects and language of message; there was also no 
interaction with gender. Finally, using the same content, although translated or gender-
specific, allows comparisons to be made across the study population. Thus, the 
messages will be similar for English and Spanish-speaking parents, ensuring proper 
linguistic and cultural equivalency, as well as for males and females, adjusting the 
wording for gender and gender-specific HPV associated disease (e.g. cervical vs. penile 
cancer). Importantly, parents will be able to self-tailor the content of the interactive 
messages by texting back indicators about which items they want more information; the 
text messaging platform then automatically responds with that information. We will be 
able to note if this self-tailoring differed for parents of male vs. female adolescents and 
English vs. Spanish-speaking parents. We will follow best practices based on guidelines 

to translating health services research measures,59 to translate messages to ensure that 
we have linguistically and culturally equivalent messages in Spanish and English. We 
will then pre-test the messages at the study sites with families. The research assistant 
will approach families after the adolescent has received an HPV vaccination, in 
coordination with the nursing staff. After obtaining informed consent, the research 
assistant will text parents in real-time, while meeting with the parent, the proposed 
messages and ask the parent to say back in their own words what the message means 
to them. This adaptation of the “teach-back” method, commonly used health literacy 
tool,58 verifies that the message conveys the desired information. S/he will then ask 
families if there was anything confusing about the message and suggestions for 
change. This process will be iterative until no new changes are made to the messages. 
Preliminarily, we will complete this process with 20 families. If we do not reach 
saturation with 20 families, i.e. changes to the messages are continuing to be made, we 
will continue to recruit using the same sampling plan. 
Study Population 
Eligibility Criteria: 
− Parent of adolescent age 9-17 years 
− Adolescent receives their first HPV vaccine dose at study site within last 2 weeks 
− Has a cell phone number that can receive a text message 
− Parent speaks English or Spanish 
Exclusion Criteria 
− Intends to move away from the NYC area in the study 
Enrollment: Families of adolescents will be enrolled one of two ways. First, they can be 
given an information sheet by their provider or the nurse describing the study. This will 
be attached to an enrollment card on which interested families can write their cell phone 
number. In providing their number they are agreeing to be in the study. This card will be 
placed in a box in the clinic. Those who do not want to enroll can indicate that as well. 
Those who are not given an information sheet or do not have a card in the box, due to 
provider time, will be called and verbal consent obtained from those who are interested. 

We will identify in EzVac those who received their first HPV vaccine in approximately the 
last week for whom we do not have a card. On inclusion, either via return of the 
enrollment card or over the phone, a confirmation text message will be sent to the 
participant family and delivery status assessed to ensure that the family has a working 
cellphone with text message. An undelivered message will be attempted to be delivered 
twice moreover the next week. If there are three undeliverable messages then the 
family will not be considered to have met eligibility criteria and will not be considered 
enrolled in the trial; they will be notified of this. We will enroll only one child per family. If 
more than one is eligible, we will select the one with the next most proximal birthday, 
e.g. a child whose birthday is in 7 days will be selected over a sibling whose is in 75 
days. In the case of twins, whoever was designated at birth as twin A will be enrolled. If 
another adolescent becomes eligible during the trial, the family cannot re-enroll. 
Randomization: Randomization will occur with a 1:1 allocation ratio stratified by (1) 
clinical site; (2) adolescent’s gender; (3) parental language; and adolescent’s age group 
(9-14; 15-17). These factors will be excerpted from the medical record after enrollment. 
Randomization will be done centrally by the programmer using an algorithm based on 
site, gender, language, and age. The statistician and analyst will be kept blinded to 
group assignment. 
Study Sample and Enrollment 
Based on EzVac data, there were 817 adolescents (females and males) 11-17 years old 
who received their first HPV vaccine at the four study sites between October 2012 and 
September 2013. Nearly all of those families (n=805) spoke English or Spanish. A few 
(n=51) were from the same family. Therefore, we expect 1885 adolescents to be eligible 
over the 30-month recruitment period. Across various studies conducted at the study 
sites, we found that between 83%and 89% of parents had text messaging. In one study, 
we assessed those with and without text messages and found no significant differences 
between those who had texting and those who did not in terms of adolescent’s gender, 
parents’ primary language, education level, or parent age. Using the low-end of this 
range, i.e. 83%, we expect 1,564 families to meet age, language and cell phone/texting 
inclusion criteria. To meet our sample size of 956 total (478 per arm), we would need to 

recruit 61.1% of eligible families. In our most recent text message vaccine related study 
in which we completed active recruitment, 92% of eligible patients enrolled. We 
therefore believe our enrollment goal is realistic. At the study sites, 35% of 11–17-year-
olds who initiate HPV vaccination and receive usual care receive 3 doses within 12 
months. In our previous study, we had an effect size of 16% between those receiving 
conventional text messages and usual care. Therefore, based on this data, we expect a 
completion rate of 51% in the conventional group (Arm 2). With the proposed sample 
size in each arm and allocation, we will be powered to detect a minimum of 9% 
difference between arms with 80% power, allowing for a type I error of 5% for the 
primary outcome. In our second dose influenza vaccine study (section D3.1), our effect 
size was 9% between each arm; we expect the effect size in this trial to be higher since 
the messages will be personalized. 
Intervention and Usual Care: The study will include approximately 30 months of 
recruitment. The intervention period for each individual is the 12-month period starting 
from administration of their first dose, as is standard in studies assessing HPV series 
vaccine completion as well as the efficacy trials.37,39,50,60-62 
Arm 1: Randomized to receive additional enhanced text messages with vaccine health 
literacy-promoting information: An interactive query sent before the next dose is due will 
assess whether a parent is in the pre-contemplation stage and unaware their 
adolescent needs another HPV dose or when the dose is due, or if they are in the 
contemplation stage knowing that their adolescent needs further doses, but still 
undecided maybe due to questions regarding the vaccine efficacy, side effects and 
safety, or if they are in the preparation phase and planning to vaccinate but still in need 
of information on where to go or when. These will be based on previously published 
HPV vaccine decisional measures that used the trans-theoretical model.26 Messages 
will then be tailored to their stage. In addition, we will include interactive text messages 
in the series that will allow participants to select to receive more information according 
to their specific needs, allowing self-tailoring of the text messaging. Finally, the current 
health literacy recommendations include “universal precautions” with all patients receive 
health literacy-promoting information regardless of perceived risk for limited health 

literacy.63 Therefore, all participants in this arm will receive some text messages related 
to the most common HPV vaccine related health literacy needs. The text messages 
have been attached to the protocol. 
Arm 2: Randomized to receive non-enhanced text message reminders: Will receive 
non-enhanced text reminders notifying them when the next dose is due. They will 
receive messages in parallel timing to Arm 1. These will not include vaccine health 
literacy-promoting information and will be similar to those used in our previous 
adolescentstudies.20,14 These are also attached to the protocol. Based on our 
preliminary studies, all text messages will be personalized and designed to be from the 
patient’s medical home. In our previous studies, parents indicated that they wanted their 
child’s name on the message as well as that it was being sent on behalf of the clinic.16,19 
The recipient will be the parent based on our previous work. We will text the cell phone 
number listed in the registration system, indicative of the parent who is to be contacted 
regarding medical care. Messages will be sent in English or Spanish based on the 
language preference of the participant. Throughout the study, participants will have the 
option to stop receiving text messages by replying to a text message with the word 
“quit” or “stop” in English or the corresponding commands in Spanish. Participants can 
request help by texting “help”. They can also call the study team. These instructions will 
be given in a text message. After the first message sent to categorize those in Arm 1 
and the parallel-timed message for Arm 2, preliminarily, we will send the first series of 
automated weekly reminders, beginning around d35 after the second HPV vaccine dose 
due date and will end two weeks after the vaccine is first over-due. Two booster doses 
will then be sent 28 and 42 days later, if the child has not been vaccinated. We will 
repeat the intervention beginning approximately d35 after second dose as a reminder 
for the third dose. The messages will be the same but refer to the third dose. The 
hospital’s immunization registry will be queried before the third dose to ascertain if the 
patient received the second dose on time. If not, the message schedule for the third 
HPV vaccine dose will be readjusted for the new due date, based on CDC 
recommendations for minimal and desired intervals. All the study sites allow walk-ins for 
second and third doses of HPV vaccine, so no appointment is needed. 

If the site does not have vaccine, we will send a message to participants due for second 
or third dose letting them know not to come in yet and that we will inform them when the 
sites have vaccine available again. 
Measures 
Outcome Measure: The primary outcome measure is completion of the 3-dose HPV 
vaccine series within 12 months from date of first dose. A secondary outcome is 
timeliness of vaccination, defined as time elapsed in days between dose 1 and 2, and 
dose 1 and 3. 
Process Measures: Individual level information from the log files of the text-messaging 
platform will be collected to demonstrate that the correct messages were sent to each 
recipient for Arms 1 and 2. We will also determine the percentage of (1) text messages 
that sent back as undeliverable; and (2) participants opting out of the reminders during 
the study. We will also determine which specific vaccine health literacy-related 
information families request via text message, providing, in a prospective fashion, an 
indication of the most common parent identified HPV vaccine health literacy needs 
arising at the time that parents are making decisions regarding return for subsequent 
doses. This prospective information does not currently exist. 
Follow-Up: Minimizing attrition and missing outcome data are crucial for valid results. 
We expect there will be little attrition with respect to outcome data. The hospital 
immunization registry, EzVac, automatically collects vaccine administrations from the 
EHR used at the 4 study sites as well as from the New York Citywide Immunization 
Registry (CIR) as part of the usual patient care, thereby allowing capture of vaccines 
administered to clinic patients both at study sites and at practices other than study sites 
in NY City.55,56 Thus, we will have outcome data for subjects who receive HPV 
vaccination at facilities other than a study site. A second important issue is 
documentation of the intervention. In our previous HPV study, no one dropped out of the 
study after enrollment;20 in our influenza text message RCT, only 4.5% of the 9,213 
families declined further messages.13 In addition, we expect only a small number of cell 
phone numbers will change over the course of the intervention. In our previous 
adolescent study, the number of cell phone numbers that changed was <2%.14 

Data Management: Vaccination data will be retrieved from EzVac. All data will be 
accessed using a password-protected computer in a locked office and stored on a 
secure server managed by Columbia University IT, and backed up on a daily basis. All 
data files will be cleaned and edited prior to analysis, under Dr. Stockwell's supervision. 
Data analysis will be performed using SAS (Cary, NC). The programmer will monitor log 
files of text messages sent, replies received, and “undeliverable” messages and provide 
summaries to Dr. Stockwell on a daily basis. 
Statistical Procedures 
Data Analysis: We will first assess the randomization process by comparing baseline 
variables among the three groups. Hypothesis: Completion and timeliness will be higher 
in adolescents in families who receive enhanced text message reminders plus usual 
care vs. non-enhanced text message reminders plus usual care. 
Series Completion: HPV vaccine completion rates in EzVac will be compared for all 
adolescents of participant parents at the end of a 12-month observation period starting 
at receipt of their first HPV vaccine dose. As in our previous RCT of influenza vaccine 
reminders,13 all primary analyses will be done on an intention-to-treat basis, such that all 
randomized participants will be included in the analysis whether or not they at some 
point declined further messages or messages were returned as undeliverable. A 
secondary analysis will compare completion rates for adolescents of participants who 
received the entire set of reminders. Study arms will not be combined. Completion rates 
in the two randomized groups will be compared in the separate combinations delineated 
above using 2-sided 2 tests at a significance level of P<0.05 for each comparison. The 
number of subjects in each cell is expected to be high enough that the Fisher exact test 
will not be needed. Asymptotic confidence limits on the differences and relative rate 
ratios will be calculated. 
Sub-group Analyses: We will perform subgroup analyses in different covariate-defined 
subgroups, for each outcome, to assess the possibility of intervention effects being 
different in these subgroups. This will be tested by including pre-specified covariates 
(adolescent: age, gender, year of entry, clinic site, dichotomized visit count; parent: 
language, previous participation in text message vaccination reminder study),42,43,60,61,64-

68 and their interactions with the intervention in a multiple logistic regression analysis. 
The models will also adjust for the direct effects of these covariates on the outcomes, 
thus increasing power to detect intervention effects. Initial models will be created 
including all of these covariates. Clearly non-significant (p>0.2) covariate terms will be 
removed from the final models. If more than one covariate appears in a final model, two-
way (and, if necessary, multi-way) interactions of those covariates among themselves 
and with the study arm will be tested and included if significant at p>0.2) covariate terms 
will be removed from the final models. If more than one covariate appears in a final 
model, two-way (and, if necessary, multi-way) interactions of those covariates among 
themselves and with the study arm will be tested and included if significant at p<0.05. In 
addition to the covariates listed above, we will include race and insurance to the extent 
possible. Multivariate modeling with these covariates may not be possible depending on 
the degree of variability observed. The visit count will be defined as the number of clinic 
visits per adolescent in the 12-month period post-receipt of the first vaccine dose. Visits 
to the emergency department, hospitalization or specialty visits will not be included. The 
visit count will be dichotomized (the cut point determined by the frequency distribution of 
visit counts). With regard to longitudinal measurement, the outcome is defined as 
success or failure in receipt of 3 HPV vaccine doses within 12 months (no repeat 
measure of outcome). We will include a covariate for year of entry into study. We will 
also adjust for vaccine shortages as needed. 
Timeliness: We define timeliness of vaccination according to the ACIP recommended 
intervals.69 We will construct Kaplan-Meier curves of time from first to second dose and 
separately first to third dose,70 using both intention to treat as well as analyses 
accounting for message deliverability. We will then compare the curves using a log rank 
test. We will perform secondary subgroup analyses using Cox proportional hazards 
modeling to examine subgroup effects and to adjust for the same covariates as 
described above. Interactivity of messages: With regard to analyses pertaining to the 
interactive nature of messages in Arm 1, we will calculate the proportion of families who 
sought information about each individual topic both for the overall arm as well as 
subdivided by gender and parental language and stage of HPV vaccine decision-
making at baseline. We will use Pearson’s chi square to compare completion rates at 12 

months in adolescents in Arm 1 whose families requested further information vs. those 
in Arm 1 who did not. 
Missing data: We will document missing data and classify it by variable and study arm. 
We will then assess if there are relationships between missingness and covariates such 
as demographics, study arm, or year of study entry. In the analysis of the study 
hypotheses, all randomized subjects will be included (intention to treat analysis). 
Missing outcome data will be imputed as failure in all study arms. We anticipate very 
few subjects will be missing outcome data. In the subgroup analyses, covariate data 
may also be missing. We will impute these variables using multiple imputation following 
the methodology of Rubin.71 We will achieve this using the 3-phase SAS/STAT 
procedure that first creates “m” multiply imputed complete data sets using the MI 
procedure, then analyzes the “m” complete datasets using a standard procedure, PROC 
LOGISTIC, and finally generates valid statistical inferences about the missing values by 
combining the results using the MIANALYZE procedure properly reflecting the 
uncertainty due to missing values. In summary, the primary analysis of the study 
hypotheses will be based on intention to treat analysis using all randomized subjects. 
Subjects with missing outcome data will be retained in the analysis; failure to complete 
the 3-dose series will be imputed for those missing these data. We consider this to be 
the most conservative approach to hypothesis testing. We used this approach in our 
influenza-vaccine text message reminder study.13 Multiple imputation will be used for 
the missing covariate data in subgroup analyses. 
In addition as of 10/21/16: Any parents of an adolescent who was <15 years when first 
vaccinated who is already enrolled and has not yet received their second dose, will get 
a text message letting them know that there is an important update, that their child now 
only needs two doses with the second one being 6 months after the first and that we will 
text them when that dose is due. We have included that text message under 
documents. 
Parents of adolescents first vaccinated before 15 years of age who have not yet 
received a second dose will receive an updated series of messages to be consistent 

with the now 2 dose recommendation in both phrasing and timing (second dose now 
due 6 months after the first instead of 2 months and is now the last dose. 
Recruitment And Consent 
Families of adolescents will be enrolled one of two ways. First, they can be given an 
information sheet by their provider or the nurse describing the study. This will be 
attached to an enrollment card on which interested families can write their cell phone 
number. In providing their number they are agreeing to be in the study. This card will be 
placed in a box in the clinic. Those who do not want to enroll can indicate that as well. 
Those who are not given an information sheet or do not have a card in the box, due to 
provider time, will be called and verbal consent obtained from those who are interested. 
We will identify in EzVac those who received their first HPV vaccine in approximately the 
last week for whom we do not have a card. On inclusion, either via return of the 
enrollment card or over the phone, a confirmation text message will be sent to the 
participant family and delivery status assessed to ensure that the family has a working 
cellphone with text message. 
References 
1. Ten great public health achievements—United States, 1900-1999. MMWR Morb Mortal Wkly Rep. Apr 
2 1999;48 (12):241-243.  
2. National, state, and local area vaccination coverage among children aged 19-35 months—United 
States, 2011. MMWR Morb Mortal Wkly Rep. Sep 7 2012;61:689-696.  
3. National and state vaccination coverage among adolescents aged 13-17 years—United States, 2011. 
MMWR Morb Mortal Wkly Rep. Aug 31 2012;61(34):671-677.  
4. Lannon C, Brack V, Stuart J, Caplow M, McNeill A, Bordley WC, et al. What mothers say about why 
poor children fall behind on immunizations. A summary of focus groups in North Carolina. Arch 
Pediatr Adolesc Med. Oct 1995;149(10):1070-1075.  
5. Baker LM, Wilson FL, Nordstrom CK, Legwand C. Mothers' knowledge and information needs relating 
to childhood immunizations. Issues Compr Pediatr Nurs. Jan-Jun 2007;30(1-2):39-53.  
6. Wilson FL, Brown DL, Stephens-Ferris M. Can easy-to-read immunization information increase 
knowledge in urban low-income mothers? J Pediatr Nurs. Feb 2006;21(1):4-12.  
7. Davis TC, Bocchini JA, Jr., Fredrickson D, Arnold C, Mayeaux EJ, Murphy PW, et al. Parent 
comprehension of polio vaccine information pamphlets. Pediatrics. Jun 1996;97(6 Pt 1):804-810.  
8. Cohen B, Ferng YH, Wong-McLoughlin J, Jia H, Morse SS, Larson EL. Predictors of flu vaccination 
among urban Hispanic children and adults. J Epidemiol Community Health. Mar 2012;66(3):204-209.  
9. Scott TL, Gazmararian JA, Williams MV, Baker DW. Health literacy and preventive health care use 
among Medicare enrollees in a managed care organization. Med Care. May 2002;40(5):395-404.  
10. Logan JL. Disparities in influenza immunization among US adults. J Natl Med Assoc. Feb 
2009;101(2):161-166.  

11. Brewer NT, Gottlieb SL, Reiter PL, McRee AL, Liddon N, Markowitz L, et al. Longitudinal Predictors of 
Human Papillomavirus Vaccine Initiation Among Adolescent Girls in a High-Risk Geographic Area. 
Sex Transm Dis. Mar 11 2011;38(3):197-204.  
12. Reiter PL, Brewer NT, Gottlieb SL, McRee AL, Smith JS. Parents' health beliefs and HPV vaccination 
of their adolescent daughters. Soc Sci Med. Aug 2009;69(3):475-480.  
13. Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text 
messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent 
population: a randomized controlled trial. JAMA. Apr 25 2012;307(16):1702-1708.  
14. Stockwell MS, Kharbanda EO, Martinez RA, Lara M, Vawdrey D, Natarajan K, et al. Text4Health: 
impact of text message reminder-recalls for pediatric and adolescent immunizations. Am J Public 
Health. Feb 2012;102(2):e15-21.  
15. Brenner J. Pew Internet: Mobile , Pew Internet and American Life Project, Pew Research Center, 
2013. Available at http://pewinternet.org/Commentary/2012/February/Pew-Internet-Mobile.aspx. 
Accessed on February 22, 2013.  
16. Kharbanda EO, Stockwell MS, Fox HW, Rickert VI. Text4Health: a qualitative evaluation of parental 
readiness for text message immunization reminders. Am J Public Health. Dec 2009;99(12):2176-
2178.  
17. Ahlers-Schmidt CR, Hart T, Chesser A, Paschal A, Nguyen T, Wittler RR. Content of text messaging 
immunization reminders: what low-income parents want to know. Patient Educ Couns. Oct 
2011;85(1):119-121. 10/06/2015  
18. Clark SJ, Butchart A, Kennedy A, Dombkowski KJ. Parents' experiences with and preferences for 
immunization reminder/recall technologies. Pediatrics. Nov 2011;128(5):e1100-1105.  
19. Hofstetter AO, Vargas C, Kitayama K, Stockwell MS. Parental and provider beliefs about text 
message reminder-recalls for early childhood vaccination. Poster presentation at the 2011 mHealth 
Summit (National Harbor, MD) and 2012 Pediatric Academic Societies’ Annual Meeting (Boston, MA).  
20. Kharbanda EO, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to 
promote human papillomavirus vaccination. Vaccine. Mar 21 2011;29(14):2537-2541.  
21. LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination 
coverage in an inner-city population. Arch Pediatr Adolesc Med. Mar 2004;158(3):255-261.  
22. Irigoyen MM, Findley S, Wang D, Chen S, Chimkin F, Pena O, et al. Challenges and successes of 
immunization registry reminders at inner-city practices. Ambul Pediatr. Mar-Apr 2006;6(2):100-104.  
23. Hambidge SJ, Davidson AJ, Phibbs SL, Chandramouli V, Zerbe G, LeBaron CW, et al. Strategies to 
improve immunization rates and well-child care in a disadvantaged population: a cluster randomized 
controlled trial. Arch Pediatr Adolesc Med. Feb 2004;158(2):162-169.  
24. Daley MF, Steiner JF, Brayden RM, Xu S, Morrison S, Kempe A. Immunization registry-based recall 
for a new vaccine. Ambul Pediatr. Nov-Dec 2002;2(6):438-443.  
25. Prochaska JO, DiClemente CC. The Transtheoretical Approach: Crossing Traditional Boundaries of 
Change. Homewood, IL: Dow Jones/Irwin; 1984.  
26. Lipschitz JM, Fernandez AC, Larson HE, Blaney CL, Meier KS, Redding CA, et al. Validation of 
Decisional Balance and Self-Efficacy Measures for HPV Vaccination in College Women. Am J Health 
Promot. Feb 12 2013.  
27. Allen JD, Mohllajee AP, Shelton RC, Othus MK, Fontenot HB, Hanna R. Stage of adoption of the 
human papillomavirus vaccine among college women. Prev Med. May 2009;48(5):420-425.  
28. Marshall RJ, Tetu-Mouradjian LM, Fulton JP. Increasing annual influenza vaccinations among 
healthcare workers in Rhode Island: a social marketing approach. Medicine & Health Rhode Island. 
2010;93(9):271–272, 276–278.  
29. Hariri S, Unger ER, Sternberg M, Dunne EF, Swan D, Patel S, et al. Prevalence of genital human 
papillomavirus among females in the United States, the National Health And Nutrition Examination 
Survey, 2003-2006. JInfect Dis. Aug 15 2011;204(4):566-573.  
30. HPV Vaccine Information for Clinicians - Fact Sheet. Center for Disease Control and Prevention 
(CDC): Sexually Transmitted Diseases (STDs) updated July 12, 2012; Available at 
http://www.cdc.gov/std/hpv/STDFactHPV-vaccinehcp.htm. Accessed on February 28, 2013.  

31. Dunne EF, Nielson CM, Stone KM, Markowitz LE, Giuliano AR. Prevalence of HPV infection among 
men: A systematic review of the literature. J Infect Dis. Oct 15 2006;194(8):1044-1057.  
32. Walboomers JM, Jacobs MV, Manos MM, Bosch FX, Kummer JA, Shah KV, et al. Human 
papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol. Sep 
1999;189(1):12-19.  
33. Grulich AE, Jin F, Conway EL, Stein AN, Hocking J. Cancers attributable to human papillomavirus 
infection. Sex Health. Sep 2010;7(3):244-252.  
34. Human papillomavirus-associated cancers - United States, 2004-2008. MMWR Morb Mortal Wkly 
Rep. Apr 20 2012;61:258-261.  
35. Hu D, Goldie S. The economic burden of noncervical human papillomavirus disease in the United 
States. Am J Obstet Gynecol. May 2008;198(5):500 e501-507.  
36. Hoy T, Singhal PK, Willey VJ, Insinga RP. Assessing incidence and economic burden of genital warts 
with data from a US commercially insured population. Curr Med Res Opin. Oct 2009;25(10):2343-
2351.  
37. Markowitz LE, Dunne EF, Saraiya M, Lawson HW, Chesson H, Unger ER. Quadrivalent Human 
Papillomavirus Vaccine: Recommendations of the Advisory Committee on Immunization Practices 
(ACIP). MMWR Recomm Rep. Mar 23 2007;56(RR-2):1-24.  
38. FDA licensure of bivalent human papillomavirus vaccine (HPV2, Cervarix) for use in females and 
updated HPV vaccination recommendations from the Advisory Committee on Immunization Practices 
(ACIP). MMWR Morb Mortal Wkly Rep. May 28 2010;59(20):626-629.  
39. Recommendations on the use of quadrivalent human papillomavirus vaccine in males—Advisory 
Committee on Immunization Practices (ACIP), 2011. MMWR Morb Mortal Wkly Rep. Dec 23 
2011;60(50):1705-1708. 10/06/2015  
40. Food and Drug Administration. Highlights of prescribing information. Gardasil (human papillomavirus 
quadrivalent [types 6, 11, 16 and 18]). Silver Spring, MD: Food and Drug Administration; 2011. 
Available at http:// 
www.fda.gov/downloads/biologicsbloodvaccines/vaccines/approvedproducts/ucm111263.pdf. 
Accessed on February 28, 2013.  
41. Vaccination coverage among adolescents aged 13-17 years - United States, 2007. MMWR Morb 
Mortal Wkly Rep. Oct 10 2008;57(40):1100-1103.  
42. Neubrand TP, Breitkopf CR, Rupp R, Breitkopf D, Rosenthal SL. Factors associated with completion 
ofthe human papillomavirus vaccine series. Clin Pediatr (Phila). Nov 2009;48(9):966-969.  
43. Chao C, Velicer C, Slezak JM, Jacobsen SJ. Correlates for completion of 3-dose regimen of HPV 
vaccine in female members of a managed care organization. Mayo Clin Proc. Oct 2009;84(10):864-
870.  
44. Moore GR, Crosby RA, Young A, Charnigo R. Low rates of free human papillomavirus vaccine uptake 
among young women. Sex Health. Sep 2010;7(3):287-290.  
45. Hirth JM, Tan A, Wilkinson GS, Berenson AB. Completion of the human papillomavirus vaccine series 
among insured females between 2006 and 2009. Cancer. Apr 27 2012;epub ahead of print.  
46. U.S. Cancer Statistics Working Group. United States Cancer Statistics: 1999–2008 Incidence and 
Mortality Webbased Report. Atlanta: U.S. Department of Health and Human Services, Centers for 
Disease Control and Prevention and National Cancer Institute; Available at: http://www.cdc.gov/uscs. 
Accessed on February 21, 2013.  
47. Niccolai LM, Mehta NR, Hadler JL. Racial/Ethnic and poverty disparities in human papillomavirus 
vaccination completion. Am J Prev Med. Oct 2011;41(4):428-433.  
48. Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, et al. Reviews of evidence 
regarding interventions to improve vaccination coverage in children, adolescents, and adults. The 
Task Force on Community Preventive Services. Am J Prev Med. Jan 2000;18(1 Suppl):97-140.  
49. Garcini LM, Galvan T, Barnack-Tavlaris JL. The study of human papillomavirus (HPV) vaccine uptake 
from a parental perspective: A systematic review of observational studies in the United States. 
Vaccine. Jun 29 2012;30 (31):4588-4595.  

50. Block SL, Nolan T, Sattler C, Barr E, Giacoletti KE, Marchant CD, et al. Comparison of the 
immunogenicity and reactogenicity of a prophylactic quadrivalent human papillomavirus (types 6, 11, 
16, and 18) L1 virus-like particle vaccine in male and female adolescents and young adult women. 
Pediatrics. Nov 2006;118(5):2135-2145.  
51. National and state vaccination coverage among adolescents aged 13 through 17 years—United 
States, 2010. MMWR Morb Mortal Wkly Rep. Aug 26 2011;60(33):1117-1123.  
52. Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. 
Thousand Oaks, CA: Sage; 1988.  
53. Olson EC, Van Wye G, Kerker B, Thorpe L, Frieden TR. Take Care Inwood and Washington Heights. 
NYC Community Health Profiles, Second Edition 2006;19(42):1-16. 
http://www.nyc.gov/html/doh/downloads/pdf/ data/2006chp-301.pdf Accessed on Febrary 28, 2013.  
54. Statewide Immunization Registry, Pub Health Law, Article 21, Title 6, §2168.  
55. Papadouka V, Zucker J, Balter S, Reddy V, Moore K, A. M. New York City Department of Health and 
Mental Hygiene "Impact of Childhood Hepatitis A Vaccination: New York City". 4first National 
Immunization Conference(Kansas City); March 2007. Available at 
cdc.confex.com/cdc/nic2007/recordingredirect.cgi/id/2057; accessed on June 20, 2012.  
56. Metroka AE, Hansen MA, Papadouka V, Zucker JR. Using an immunization information system to 
improve accountability for vaccines distributed through the Vaccines for Children program in New 
York City, 2005- 2008. J Public Health Manag Pract. Sep-Oct 2009;15(5):E13-21.  
57. Kincaid J, Fishburne R, Rogers R, Chissom B. Derivation of New Readability Formulas (Automated 
Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. 
Millington, TN: Naval Technical Training; 1975.  
58. Weiss B. Health literacy and patient safety: Help patients understand. Manual for Clinicians, second 
Edition. Available at http://www.ama-assn.org/ama1/pub/upload/mm/367/healthlitclinicians.pdf. 
Accessd on February 27, 2013.  
59. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life 
measures: literature review and proposed guidelines. J Clin Epidemiol. Dec 1993;46(12):1417-1432.