HPV ECHO: Increasing the Adoption of Evidence-based Communication Strategies for HPV Vaccination in Rural Primary Care Practices
Phase: NA
Conditions: Human Papillomavirus Vaccines, Adolescent Health Services
Sponsor: Milton S. Hershey Medical Center

Page 1 of 25 (V.01/21/2019) 
                                                                                       
HRP-591 - Protocol for 
Human Subject Research
Protocol Title:
HPV ECHO: Increasing the adoption of evidence-based communication strategies for HPV vaccination in rural primary 
care practices.
Principal Investigator:
Name: William A. Calo
Department: Public Health Sciences
Telephone: 717-531-3535
E-mail Address: wcalo@pennstatehealth.psu.edu
Version Date:
1/22/2024
Clinicaltrials.gov Registration #: NCT04587167
Important Instructions for Using This Protocol Template:
This template is provided to help investigators prepare a protocol that includes the necessary information needed by the 
IRB to determine whether a study meets all applicable criteria for approval.
1.
GENERAL INSTRUCTIONS:  

Prior to completing this protocol, ensure that you are using the most recent version by verifying the 
protocol template version date in the footer of this document with the current version provided in the 
CATS IRB Library.

Do not change the protocol template version date located in the footer of this document.

Some of the items may not be applicable to all types of research.  If an item is not applicable, please 
indicate as such or skip question(s) if indicated in any of the instructional text. 

GRAY INSTRUCTIONAL BOXES: 
o
Type your protocol responses below the gray instructional boxes of guidance language.  If the 
section or item is not applicable, indicate not applicable.
o
Penn State College of Medicine/Penn State Health researchers: Delete the instructional boxes 
from the final version of the protocol prior to upload to CATS IRB (http://irb.psu.edu).  
o
Penn State researchers at all other campuses: Do NOT delete the instructional boxes from the 
final version of the protocol.

Add the completed protocol template to your study in CATS IRB (http://irb.psu.edu) on the “Basic 
Information” page.  
2.
CATS IRB LIBRARY: 

Documents referenced in this protocol template (e.g. SOP’s, Worksheets, Checklists, and Templates) can be 
accessed by clicking the Library link in CATS IRB (http://irb.psu.edu).
3.
PROTOCOL REVISIONS: 

When making revisions to this protocol as requested by the IRB, please follow the instructions outlined in 
the Study Submission Guide available in the Help Center in CATS IRB (http://irb.psu.edu) for using track 
changes. 

Update the Version Date on page 1 each time revisions are made.
STUDY00015734
Approval: 1/25/2024

Page 2 of 25 (V.01/21/2019) 
If you need help…
University Park and other campuses:
Office for Research Protections Human Research 
Protection Program
The 330 Building, Suite 205
University Park, PA 16802-7014
Phone: 814-865-1775
Fax: 814-863-8699
Email: irb-orp@psu.edu
College of Medicine and Penn State Health:
Human Subjects Protection Office
90 Hope Drive, Mail Code A115, P.O. Box 855
Hershey, PA 17033
(Physical Office Location: Academic Support Building 
Room 1140)
Phone: 717-531-5687
Email: irb-hspo@psu.edu  
STUDY00015734
Approval: 1/25/2024

Page 3 of 25 (V.01/21/2019) 
Table of Contents
1.0
Objectives
2.0
Background
3.0
Inclusion and Exclusion Criteria
4.0
Recruitment Methods
5.0
Consent Process and Documentation
6.0
HIPAA Research Authorization and/or Waiver or Alteration of Authorization
7.0
Study Design and Procedures
8.0
Subject Numbers and Statistical Plan
9.0
Data and Safety Monitoring Plan
10.0
Risks
11.0
Potential Benefits to Subjects and Others
12.0
Sharing Results with Subjects
13.0
Subject Payment and/or Travel Reimbursements
14.0
Economic Burden to Subjects
15.0
Resources Available
16.0
Other Approvals
17.0
Multi-Site Study
18.0
Adverse Event Reporting
19.0
Study Monitoring, Auditing and Inspecting
20.0
Future Undetermined Research: Data and Specimen Banking
21.0
References
22.0
Confidentiality, Privacy and Data Management
STUDY00015734
Approval: 1/25/2024

Page 4 of 25 (V.01/21/2019) 
1.0
Objectives
1.1
Study Objectives
To evaluate the effect of two Project ECHO-delivered human papillomavirus (HPV) vaccine 
communication interventions in primary care clinics on HPV vaccination rates.  
1.2
Primary Study Endpoints
The primary study endpoint is HPV vaccine initiation (≥1 doses) among adolescents, ages 11-14, 
between baseline and 12-month follow-up at the clinic level. 
We hypothesize that (H1) both ECHO interventions (HPV ECHO and HPV ECHO+) will increase 
vaccination compared to the control but (H2) vaccination rates will be highest for clinics receiving HPV 
ECHO+. 
The three study arms are:
1. HPV ECHO: This intervention will use the Project ECHO model to connect experts on vaccine 
communication and cancer prevention with healthcare providers in rural primary care practices in 
Pennsylvania to improve best-practice communication about HPV vaccine. Participants will participate in 
8 one-hour telementoring sessions and will bring de-identified case studies about parental vaccine 
hesitancy to discuss during sessions with the group of experts. Sessions also include a brief lecture 
focused on the Announcement Approach, a best-practice communication for HPV vaccination.  
2. HPV ECHO+: This intervention will provide the telementoring training as HPV ECHO plus a systems 
strategy, which comprises a recall notice to notify parents that their child is behind for HPV vaccination 
and address their primary reason for not vaccinating.
3. Control: Participants in this arm will be a wait list control and will receive access to the recording from 
all eight ECHO sessions at the end of their wave. 
1.3
Secondary Study Endpoints
Secondary study endpoints #1 (effectiveness) are HPV vaccine initiation by gender, for 15-17 year-olds, 
up-to-date HPV vaccination, concomitant vaccination for Tdap and meningococcal vaccines, and 
vaccinations at interim time points (i.e., 3-, 6-, and 9-month follow-up).
We hypothesize that (H1) both ECHO interventions will increase all vaccination secondary endpoints 
compared to the control but (H2) vaccination rates will be highest for clinics receiving HPV ECHO+. 
Secondary study outcomes #2 (implementation) are cost, acceptability, appropriateness, adoption, 
penetration, and sustainability to healthcare providers. 
We hypothesize that (H3) clinics receiving HPV ECHO+, as the most resource-intensive intervention, will 
incur higher overall costs but lower cost per additional vaccine dose delivered. We also hypothesize that 
(H4) providers in both intervention arms will report equally high satisfaction with (acceptability) and 
perceived fit (appropriateness) of ECHO-delivered interventions, greater use (adoption) of best 
practices, staff engagement (penetration), and maintenance of practice changes (sustainability). These 
outcomes will be evaluated in the two intervention arms only. 
Secondary study outcome #3 (secondary acceptance) is HPV vaccine initiation (≥1 dose) after initially 
declined at the clinic level.
We hypothesize (H5) that clinics receiving HPV ECHO+ will have higher secondary acceptance rates than 
those in the HPV ECHO arm. This outcome will be evaluated in the two intervention arms only.
STUDY00015734
Approval: 1/25/2024

Page 5 of 25 (V.01/21/2019) 
Secondary study outcome #4 (nested study) is to explore the level of exposure to and impact of vaccine 
information from intervention arms versus naturally-occurring sources (e.g., social media). In the nested 
study, we will follow a subset of 211 HPV vaccine-declining parents for up to 12 months. Only Penn State 
Health clinics that are enrolled in the study will be invited to participate in the nested study. The nested 
study is exploratory and has no hypothesis.    
2.0
Background 
2.1
Scientific Background and Gaps
The safe, highly-effective human papillomavirus (HPV) vaccine remains underused in the US; only 51% of 
13-to 17-year-old girls and boys were up-to-date by 2018.1 Rural populations are particularly impacted, 
with 15% lower rates of vaccination.1 The Announcement Approach is effective in increasing HPV 
vaccine uptake during the clinic visit by training providers to make strong vaccine recommendations and 
answer parents’ common questions.2 However, the training is limited to a single 60- to 90-minute, in-
person session delivered by one physician educator and offers standardized scenarios for providers to 
practice their communication skills, mostly by memorizing and reciting messages. Systems 
communication like recall notifications also improve vaccination by reducing missed clinical 
opportunities.3 However, these recall notifications have not been tested in combination with the 
Announcement Approach training to supplement provider communication with parents. Also, rural 
providers, especially family practice physicians, are less likely to strongly recommend HPV vaccine and 
implement systems strategies to improve vaccination.4-6 Past HPV vaccine communication studies tend 
to focus on large urban clinics; in turn leaving rural clinics behind.7 These data prompt the question: 
“How can academic centers support HPV vaccination in rural primary care clinics?” 
Although never tested to support HPV vaccination, the ECHO (Extension for Community Healthcare 
Outcomes) Model is a proven implementation strategy to promote capacity exchange between health 
care experts at academic centers (“the hub”) and primary care providers at the front line of rural 
community health care (“spokes”). Through regularly-scheduled live video sessions, ECHO creates a 
supportive community network where the spokes connect with the hub to discuss a) best practices in 
care and b) complex cases managed within their practice.8 Two recent systematic reviews concluded 
that the ECHO Model is an effective and potentially cost-saving model that increases participant 
knowledge and patient access to quality care in remote and underserved locations.9-10 We propose to 
test the effectiveness of two HPV vaccine communication interventions delivered through project ECHO 
at Penn State College of Medicine: an ECHO-delivered Announcement Approach training (HPV ECHO) 
and HPV ECHO plus systems follow-up communication for parents who initially decline vaccination (HPV 
ECHO+).
2.2
Previous Data
As part of a multi-state RCT with 148 high-volume primary care clinics, Dr. Calo (PI) led a study 
comparing in-person and live-webinar (an ECHO-like intervention) modalities to deliver HPV vaccination 
quality improvement coaching on HPV vaccination intermediate outcomes.11 In both arms, coaching 
improved intermediate outcomes related to providers’ knowledge and self-efficacy to improve HPV 
vaccine delivery (all p<.05). Providers also rated coaching sessions in the two arms equally highly on 
acceptability and satisfaction. Delivery cost per clinic for webinar coaching was two-thirds the cost for 
in-person coaching. These data, from an ECHO-like intervention, strongly suggest that the ECHO Model 
will be an efficient, acceptable, and economically feasible solution for delivering HPV vaccination best 
practices to rural clinics. This RCT study will be the first to rigorously evaluate two ECHO-delivered 
communication interventions on both HPV vaccination and implementation outcomes.
2.3
Study Rationale
The rationale for the project is that ECHO is a robust, highly-accessible platform to deliver best-practice 
communication interventions to rural providers and address the context-specific needs of their patients. 
STUDY00015734
Approval: 1/25/2024

Page 6 of 25 (V.01/21/2019) 
In our study, ECHO will allow us to make use of the Announcement Approach training’s strengths while 
addressing its shortcomings. We will “grow” the Announcement Approach training by having an 
interdisciplinary team of experts offering an expanded best-practice curriculum that includes systems 
strategies to supplement provider communication, through a series of live videoconference sessions. 
The ECHO Model will also allow providers to discuss their own cases of vaccine hesitancy and seek 
solutions that are responsive to their patients’ sociocultural and communication needs. We will also 
provide tailored visual aids to reduce the communication burden of providers and parents during 
(infographic cards) and after (recall notices) healthcare visits. With 218 ECHO hubs in 48 U.S. states, our 
project can exponentially increase national capacity to deliver HPV vaccination best practices across the 
country.
3.0
Inclusion and Exclusion Criteria
3.1
Inclusion Criteria for Primary Care Clinics
Family medicine or pediatric clinic in Pennsylvania
Having at least 100 active patients, ages 11-14
Access to computer or electronic device
Inclusion Criteria for Clinic Staff
Age: ≥18 years
Fluent in written and spoken English
Employed by primary care clinic in Pennsylvania
Involved in vaccine administration, communication, or scheduling 
Inclusion Criteria for Parents in Nested Study
Age: ≥18 years
Fluent in written and spoken English
Parent/guardian of an adolescent ages 11–17, male or female 
Declined initiation of HPV vaccine 
Adolescent children must be a patient in a primary care clinic enrolled in study 
Access to computer or electronic device   
3.2
Exclusion Criteria for Primary Care Clinics
Primary care clinic outside of Pennsylvania
Participated in HPV vaccine communication or quality improvement research either through Penn State 
or another institution in the last 12 months
Exclusion Criteria for Individuals
Not fluent in written and spoken English
Decisional impairment
Not employed by a primary care clinic in PA
Not involved in vaccine administration, communication, or scheduling
Exclusion Criteria for Parents in Nested Study
Age: <18 years
Not fluent in written and spoken English
Not the parent/guardian of an adolescent ages 11–17 
Adolescent already initiated HPV vaccination 
Lack access to computer or electronic device
STUDY00015734
Approval: 1/25/2024

Page 7 of 25 (V.01/21/2019) 
3.3
Early Withdrawal of Subjects
3.3.1
Criteria for removal from study
Participants may voluntarily withdraw from the study at any time. 
If a clinic staff participant leaves the clinic, they will be removed from the study, but another 
eligible staff from the clinic may fill in. Because the material learned through participation in the 
two interventions is meant to be communicated to other staff in the clinic or incorporated into 
clinic’s routine practice, the site will not be removed from the study if they are able to find a 
replacement staff member.
For the nested study, subjects who choose to withdraw will be replaced by another identified 
potential participant from the same clinic. 
3.3.2
Follow-up for withdrawn subjects
There will be no follow-up with any participants (i.e., clinics, clinic staff, parents in nested study) 
who choose to withdraw.
4.0
Recruitment Methods
4.1
Identification of subjects
The study will enroll 45 primary care clinics in Pennsylvania.  Each primary care clinic enrolled in the 
study will be encouraged to invite up to 6 staff to participate in study activities (6 staff x 45 clinics = 270 
staff). Clinic staff include physicians, clinic director or administrator, nurses, physician assistants, other 
healthcare professionals, and other staff involved in the administration, communication or scheduling of 
vaccines. Clinic recruitment will be facilitated through Penn State Health, the Southcentral Pennsylvania 
AHEC (Area Health Education Centers), and the Pennsylvania Association of Community Health Centers 
(PACHC). These three organizations have a combined membership of over 350 primary care clinics that 
might be eligible for this RCT. We do not have direct access to all providers within these clinics, so an 
email, flyer, or mailing will be sent to the clinic office manager, who will share the information with all 
providers in the clinic who can then choose to enroll and participate. 
For the nested study, potential participants will be identified by clinic staff. Using the clinic’s EHR (e.g., 
PowerChart), clinic staff will query the HPV vaccination status of adolescent patients ages 11-17 to 
identify those who have not yet started the first dose of the HPV vaccine and whose parents declined 
vaccination as indicated by provider notes. 
4.2
Recruitment process
4.2.1
How potential subjects will be recruited.
Primary care clinics will be recruited through collaborations with our stakeholder organizations, 
including Penn State Health, the Southcentral Pennsylvania AHEC, and PACHC through email 
communications and phone calls. These organizations have agreed, by providing letters of 
support, to partner with the study team to help with recruitment of primary care clinics. We will 
1) provide study information to their leadership who can relay this information to their member 
clinics or 2) have our study team contacting the clinic director or lead staff (“clinic contact”). An 
email describing the study with a link to a screening form to determine eligibility will be sent to 
that clinic contact. We will also create a marketing website that provides study information and 
links clinics directly to the study screener and the research team’s contact information. 
STUDY00015734
Approval: 1/25/2024

Page 8 of 25 (V.01/21/2019) 
The clinic director or lead staff will be asked to complete a brief screening form in REDCap if 
interested in participating in the study to ensure the clinic is eligible per criteria in section 3.0 of 
this protocol.  Our study team will review screening forms to determine clinic eligibility. Those 
clinics screened eligible for participation in the RCT will be prompted with an online consent 
form and registration form to be completed by the clinic director or lead staff. After completion 
of these forms, study staff will enroll and randomize clinics to study arms. Study staff will then 
send an email communication to each clinic contact indicating which group they have been 
enrolled in. The clinic director or lead staff will be tasked with informing clinic staff about the 
study opportunity, and providing the contact information of the staff (up to 6 per clinic) who are 
interested in participating in study activities.  Participating providers who enroll in the study 
through their clinic will then be invited to participate in a brief 30-minute key informant 
interview via email. Only the 1-2 providers most active participants (attended the most HPV 
ECHO training sessions) from each clinic will be invited to partake in the interview. 
For the nested study, potential participants will be recruited via patient portal and email 
communications. After initial identification in the clinic’s EHR, clinic staff collaborating with the 
research team will send the study flyer to potential subjects via the patient portal. However, this 
method has its faults as only 25% of parents of adolescents utilize the portal (preliminary data 
from two Penn State Health-affiliated primary care clinics). To overcome this barrier, we will 
send the study flyer to the email of potential participants who do not have an active patient 
portal. The email information will be retrieved by the clinic staff as available in the clinic’s EHR. 
The flyer will have a link to connect interested parents to our secure REDCap system where 
parents will find screener questions to confirm eligibly, consent information, and additional 
information about the study. Parents who do not respond to our initial communication will be 
followed up with up to two additional email or patient portal communications.   
4.2.2
Where potential subjects will be recruited.
Clinic recruitment will take place remotely via email, phone call, or virtual meetings to discuss 
specifics related to participation. 
For the nested study, parents will be recruited using a study flyer to be sent via email or the 
patient portal. Phone or virtual calls to discuss specifics related to study participation will be 
available to potential participants per request. All potential subjects will be recruited from 
clinics assigned to the intervention groups. 
4.2.3
When potential subjects will be recruited.
Clinics will be recruited from January 2021 to May 2023 in three annual waves (15 clinics in 
2021; 15 clinics in 2022; and 15 clinics in 2023). Parents in the nested study will be recruited 
from August 2021 to December 2022 (enrolled parents will complete 5 total surveys over a one-
year period, meaning they may be participating until December 2023). 
4.2.4
Describe the eligibility screening process and indicate whether the screening process will 
occur before or after obtaining informed consent. Screening begins when the investigator 
obtains information about or from a prospective participant in order to determine their 
eligibility.  In some studies, these procedures may not take place unless HIPAA Authorization 
is obtained OR a waiver of HIPAA Authorization when applicable for the screening procedures 
is approved by the IRB.  [For FDA regulated studies, consent for any screening activities would 
need to be obtained prior to screening unless specifically waived by the IRB.]
Eligibility will be determined via REDCap screening form. Staff employed by the clinic will not be 
identified until the clinic has been enrolled in the study. Inclusion/exclusion criteria is based 
STUDY00015734
Approval: 1/25/2024

Page 9 of 25 (V.01/21/2019) 
upon primary care clinic characteristics. There will be an explanation of the study, including time 
commitment and description of the study, but consent will not take place until after screening. 
For the nested study, the screening process will begin before obtaining informed consent with 
an electronic pre-screening by a clinic staff. Prior to being asked to participate in the study and 
obtaining informed consent, clinic staff will query the clinic’s EHR to identify adolescent patients 
ages 11-17 who have not yet started the first dose of the HPV vaccine and whose parents 
declined vaccination as indicated by provider notes. Vaccine declination must occur during the 
time the clinic is enrolled in the study. Once a potential participant is identified to meet the 
inclusion criteria, these subjects will receive a study invitation via email or patient portal. They 
will be presented with the SER to inform them of the study requirements so that they may 
choose to consent to participate in the study according to the procedures described. 
Participants will give implied consent by completing a screener form in REDCap in order to enroll 
and participate in this study. Screening will be done electronically through REDCap.
5.0
Consent Process and Documentation 
5.1
Consent Process:
Check all applicable boxes below:
 Informed consent will be sought and documented with a written consent form [Complete Sections 
5.2 and 5.6] 
 Implied or verbal consent will be obtained – subjects will not sign a consent form (waiver of 
written documentation of consent) [Complete Sections 5.2, 5.3 and 5.6] 
  Informed consent will be sought but some of the elements of informed consent will be omitted or 
altered (e.g., deception). [Complete section 5.2, 5.4 and 5.6] 
 Informed consent will not be obtained – request to completely waive the informed consent 
requirement. [Complete Section 5.5]
The following checkbox is for all locations EXCEPT Penn State Health and College of Medicine: 
 
 Exempt Research at all Locations Except Penn State Health and the College of Medicine: If you 
believe that the research activities outlined meet one or more of the criteria outlined in “HRP-312- 
Worksheet- Exemption Determination.” Please verify by checking this box that if conducting an 
exempt research study, the consent process will disclose the following (all of which are included in 
“HRP-590- Consent Guidance for Exempt Research”):
Penn State affiliation; name and contact information for the researcher and advisor (if the 
researcher is a student); the activities involve research; the procedures to be performed; 
participation is voluntary; that there are adequate provisions to maintain the privacy interests of 
subjects and the confidentiality of the data; and subjects may choose not to answer specific 
questions.  
If the research includes the use of student educational records include the following language in 
this section (otherwise delete): The parent or eligible student will provide a signed and dated 
written consent that discloses: the records that may be disclosed; the purpose of the disclosure; the 
party or class of parties to whom the disclosure may be made; if a parent or adult student requests, 
the school will provide him or her with a copy of the records disclosed; if the parent of a student 
who is not an adult so requests, the school will provide the student with a copy of the records 
disclosed.
STUDY00015734
Approval: 1/25/2024

Page 10 of 25 (V.01/21/2019) 
Note: If this box has been checked, skip the remainder of section 5 and proceed to section 6 of this 
protocol. If the investigator’s assessment is inaccurate, an IRB Analyst will request revision to the 
protocol and that an informed consent form be submitted for review and approval. Except for 
exemptions where Limited IRB Review (see “HRP-312- Worksheet- Exemption Determination”) is 
required or where otherwise requested by the IRB, informed consent forms for research activities 
determined to be exempt without Limited IRB Review are generally not required to be submitted 
for review and approval by the University Park IRB. 
5.2
Obtaining Informed Consent 
5.2.1
Timing and Location of Consent 
Clinic sites will be recruited and their explicit permission to recruit their employees will be 
documented. As part of the permission, clinic sites will provide a list of up to 6 clinic staff and 
their emails that are interested in participating in the study. After we receive the name and 
email of interested clinic staff, they will receive an email invite to connect them to the REDCap 
with the SER and baseline survey. To ensure interest, participants will be asked independent of 
their clinic director/lead if they are willing and able to participate. Their responses will not be 
shared with the clinic director/lead. Completion of the baseline survey serves as their implied 
consent (no screener).  Participants who are invited and interested in participating in the key 
informant interview will be presented with an online SER prior to the interview and will give 
their implied consent prior to the scheduled interview. 
The consent process for parents in the nested study will occur at the beginning of the eligibility 
screener in REDCap where potential participants will be prompted with the SER document 
content. Once participant reviews the SER, they will complete the screener, which serves as 
their implied consent.
5.2.2
Coercion or Undue Influence during Consent 
All participants, including those in the nested study, will be prompted with the following text 
prior to start any survey or study activity: “Your participation is voluntary and you do not have 
to answer any questions that you do not want to answer.” The SER also informs subjects that 
their participation is voluntary.
5.3
Waiver of Written Documentation of Consent 
5.3.1
Indicate which of the following conditions applies to this research:
  The research presents no more that minimal risk of harm to subjects and involves no 
procedures for which written consent is normally required outside of the research context.
OR
  The only record linking the subject and the research would be the consent document and the 
principal risk would be potential harm resulting from a breach of confidentiality. Each subject 
will be asked whether the subject wants documentation linking the subject with the research, 
and the subject’s wishes will govern. (Note: This condition is not applicable for FDA-regulated 
research. If this category is chosen, include copies of a consent form and /or parental 
permission form for participants who want written documentation linking them to the 
research.)
OR
  If the subjects or legally authorized representatives are members of a distinct cultural group 
or community in which signing forms is not the norm, that the research presents no more 
STUDY00015734
Approval: 1/25/2024

Page 11 of 25 (V.01/21/2019) 
than minimal risk of harm to subjects and provided there is an appropriate alternative 
mechanism for documenting that informed consent was obtained.  (Note: This condition is not 
applicable for FDA-regulated research.) 
Describe the alternative mechanism for documenting that informed consent was obtained:
Participants will give implied consent. Their completion of the baseline survey (clinic staff) or 
the eligibility screener (parents in nested study) after reviewing the SER indicates their 
consent to participate in the study. 
5.3.2
Indicate what materials, if any, will be used to inform potential subjects about the research 
(e.g., a letter accompanying a questionnaire, verbal script, implied consent form, or summary 
explanation of the research)
The SER will be placed at the beginning of the baseline survey for clinic staff. Clinic staff that also 
participate in the interviews will be preented with another SER at the beginning of a brief 
questionnaire that confirms their willingness to participate and availability for scheduling the 
interview. For the nested study, potential participants will receive the SER prior to completing 
the screener.
5.4
Informed consent will be sought but some of the elements of informed consent will be omitted or 
altered (e.g., deception). 
5.4.1
Indicate the elements of informed consent to be omitted or altered
N/A
5.4.2
Indicate why the research could not practicably be carried out without the omission or 
alteration of consent elements
N/A
5.4.3
Describe why the research involves no more than minimal risk to subjects.
N/A
5.4.4
Describe why the alteration/omission will not adversely affect the rights and welfare of 
subjects.
N/A
5.4.5
If the research involves using identifiable private information or identifiable biospecimens, 
describe why the research could not be practicably be carried out without using such 
information or biospecimens in an identifiable format.
N/A
5.4.6
Debriefing 
N/A
5.5
Informed consent will not be obtained – request to completely waive the informed consent 
requirement
5.5.1
Indicate why the research could not practicably be carried out without the waiver of consent
N/A
STUDY00015734
Approval: 1/25/2024

Page 12 of 25 (V.01/21/2019) 
5.5.2
Describe why the research involves no more than minimal risk to subjects.
N/A
5.5.3
Describe why the alteration/omission will not adversely affect the rights and welfare of 
subjects.
N/A
5.5.4
If the research involves using identifiable private information or identifiable biospecimens, 
describe why the research could not be practicably be carried out without using such 
information or biospecimens in an identifiable format.
N/A
5.5.5
Additional pertinent information after participation
N/A
5.6
Consent – Other Considerations 
5.6.1
Non-English-Speaking Subjects
Non-English speaking is an exclusion criterion.
5.6.2
Cognitively Impaired Adults
5.6.2.1
Capability of Providing Consent
N/A
5.6.2.2
Adults Unable To Consent
N/A
5.6.2.3
Assent of Adults Unable to Consent
N/A
5.6.3
Subjects who are not yet adults (infants, children, teenagers) 
5.6.2.1 Parental Permission
Parental permission is addressed within the SER. By completing the screener 
parents are providing their implied consent for their child’s PHI (HPV vaccine uptake 
status) information to be used as part of this research study. 
5.6.2.2 Assent of subjects who are not yet adults
A waiver of child assent is requested for this study. Children have no active role in 
study activities. Parents are fully informed of and they will complete all study 
activities. Also, as usual care, parents will be the ones providing permission during 
the child’s appointment regarding administration of the HPV vaccine. 
6.0
HIPAA Research Authorization and/or Waiver or Alteration of Authorization
6.1
Authorization and/or Waiver or Alteration of Authorization for the Uses and Disclosures of PHI
Check all that apply:
 
Not applicable, no identifiable protected health information (PHI) is accessed, used or 
disclosed in this study. [Mark all parts of sections 6.2 and 6.3 as not applicable]
STUDY00015734
Approval: 1/25/2024

Page 13 of 25 (V.01/21/2019) 
Authorization will be obtained and documented as part of the consent process. [If this is the 
only box checked, mark sections 6.2 and 6.3 as not applicable]
Partial waiver is requested for recruitment purposes only (Check this box if patients’ medical 
records will be accessed to determine eligibility before consent/authorization has been 
obtained). [Complete all parts of sections 6.2 and 6.3]
Full waiver is requested for entire research study (e.g., medical record review studies). 
[Complete all parts of sections 6.2 and 6.3]
Alteration is requested to waive requirement for written documentation of authorization 
(verbal authorization will be obtained). [Complete all parts of sections 6.2 and 6.3]
6.2
Waiver or Alteration of Authorization for the Uses and Disclosures of PHI
6.2.1
Access, use or disclosure of PHI representing no more than a minimal risk to the privacy of the 
individual
6.2.1.1
Plan to protect PHI from improper use or disclosure
Information is included in the “Confidentiality, Privacy and Data 
Management” section of this protocol.
6.2.1.2
Plan to destroy identifiers or a justification for retaining identifiers 
All identifiers will be destroyed when the study ends.
6.2.2
Explanation for why the research could not practicably be conducted without access to and 
use of PHI
For the nested study, using PHI from clinics’ EHR is the only way to identify potential subjects 
whose children have not yet started HPV vaccination. Assigned clinic staff will access these data 
to review eligibility criteria and send electronic flyer to invite those subjects to participate in the 
study. All other study vaccine data will be received from the clinics aggregated at the clinic level. 
6.2.3
Explanation for why the research could not practicably be conducted without the waiver or 
alteration of authorization
The study’s primary outcome measure and most secondary measures require the use of EHR 
data from participating clinics that contain some PHI. However, these data will be received 
aggregated at the clinic level so getting consent from each patient is impractical because we will 
not obtain individual-level PHI. 
For clinic staff an alteration is requested as the study is conducted virtually and is minimal risk. 
Clinic staff will still be presented with the SER prior to completing the baseline survey. 
Completion of the baseline survey serves as their implied consent to participate in the study. 
For the nested study, in order to identify potential participants for recruitment, there is a need 
to access a limited set of PHI. Clinic staff will access patients’ EHR for the purpose of confirming 
patient gender, age, and HPV immunization status before screening the patients’ parents and 
obtain parental consent. Parent emails will be obtained to distribute our recruitment flyer. A 
partial waiver is requested for recruitment purposes only. We will only use the waiver for 
authorization to the PHI for the recruitment process. Eligible patients will give their implied 
consent, if they refuse to participate in the study, we will no longer use their PHI.
6.3
Waiver or alteration of authorization statements of agreement
Protected health information obtained as part of this research will not be reused or disclosed to any other 
person or entity, except as required by law, for authorized oversight of the research study, or for other 
STUDY00015734
Approval: 1/25/2024

Page 14 of 25 (V.01/21/2019) 
permitted uses and disclosures according to federal regulations. The research team will collect only information 
essential to the study and in accord with the ‘Minimum Necessary’ standard (information reasonably necessary 
to accomplish the objectives of the research) per federal regulations. Access to the information will be limited, 
to the greatest extent possible, within the research team. All disclosures or releases of identifiable information 
granted under this waiver will be accounted for and documented.
7.0
Study Design and Procedures
7.1
Study Design
We will conduct a cluster randomized controlled trial to assess the impact of two ECHO-delivered HPV vaccine 
communication interventions on HPV vaccine initiation in primary care clinics. Using a 3-arm design, 45 rural 
clinics will be randomized in 3 waves (15 clinics per wave) to receive: ECHO-delivered Announcement Approach 
training (HPV ECHO); HPV ECHO plus systems communication (HPV ECHO+); or control. Participating clinic staff 
will be asked to enroll in the 8-session curriculum (only HPV ECHO and HPV ECHO+ arms) and complete two 
online surveys (all arms). Active clinic staff that were randomized to the intervention arms will be invited to 
participate in a brief 30-minute interview.  The study includes a nested study with a subset of 211 parents to 
evaluate the impact of vaccine information on secondary acceptance of HPV vaccination. All study activities will 
be completed by participates using their small media devices like tablets, desktop or laptop computers, and 
smart phones.    
7.2
Study Procedures
7.2.1
Registration/Baseline Survey – Intervention groups and control 
Prior to the start of the study, all participants will be asked questions about their knowledge, 
attitude, confidence, and skills regarding HPV vaccination and vaccine communication practices. 
In addition, participants will be asked general information such as academic credentials, years of 
experience, and role in primary care clinic.  
Please note the baseline survey, case presentation, and final evalutation surveys are used 
regularly in ECHO series that are IRB approved under CATS study ID #10821.
As part of the baseline survey the clinic’s study contact will enter in the immunization data for 
the participating clinic site, these measures are part of our primary outcome.
7.2.2     Case Presentation – HPV ECHO and HPV ECHO+ Intervention Groups 
Case presentation information will be collected prior to each teleECHO clinic session from 
participants in the intervention via REDCap. No patient level data will be collected. Case 
template forms will collect de-identified data on specific patient cases about vaccine hesitancy 
that will be used in case-based learning opportunities.
Please note, this is not a data collection form but used as part of the ECHO educational model to 
drive discussion and information sharing. More information for use of the case forms and the 
ECHO Model can be found under CATS study ID #10821.
7.2.3
Curriculum – HPV ECHO and HPV ECHO+ Intervention Groups
HPV ECHO intervention arm (please note that information regarding specifics of ECHO clinics 
may be found in CATS Study 10281): Clinics in this arm will receive the intervention via real-
time, interactive videoconferencing using Zoom, a user-friendly, HIPAA-compliant, cloud-based 
software application offered at no cost to participants. Sessions will be held twice a month for 4 
months (8 sessions total) at regularly scheduled times (e.g., every 2nd and 4th Wednesday from 
STUDY00015734
Approval: 1/25/2024

Page 15 of 25 (V.01/21/2019) 
1:00–2:00 PM) convenient to providers. Sessions will last 60 minutes each and will share the 
following format (the number, frequency and length of sessions are usual for the ECHO Model8): 

Introductions (5 minutes) – Introductions are intended to provide an inviting atmosphere to 
hub and spoke participants. A full roster and contact information of providers and hub 
experts will also be provided to facilitate networking outside of ECHO sessions.   

Didactic presentations (20 minutes) – Experts will deliver a short PowerPoint presentation 
with question and answer session on the topic. Presentations and session materials will be 
available to participants through our online resource library in Penn State Box, a no-cost, 
cloud storage and collaboration system. 

Case presentations (30 minutes) – Each session includes case-based discussions to ensure 
mastery of the content and skills. Each spoke provider will present at least one case during 
the program. In the assigned session, spoke providers will spend 5 minutes presenting a de-
identified case for one of their current adolescent patients. Other spoke providers are 
encouraged to ask clarifying questions and weigh in on recommendations, then the hub 
experts provide advice on how to address each case using best practices. We will discuss 
two cases per session. To protect patient confidentiality, cases are presented in a de-
identified manner, without disclosure of PHI. There are instructions for presenting a case 
shared with participants prior to the start of the ECHO series in the Welcome Guide. This 
welcome guide is provided to all ECHO participants and describes the use of zoom, how to 
present a case, and provides introductions to the hub team. An assigned, confidential ECHO 
ID# is used to identify presented cases in the order in which they are received. The ECHO ID 
is used to match case presentation material from REDCap to the session in which it was 
presented. If PHI or identifiers are included, they are removed in REDCap prior to sharing 
with participants. Prior to a session, all pertinent information for each patient is entered 
into a standardized REDCap case form by the presenting provider. The hub will then 
generate a case presentation page which is screen-shared during session. Best-practice 
recommendations will be summarized verbally at the conclusion of each case presentation 
and forwarded by email to all spoke providers. 

Close and debrief (5 minutes) – All sessions conclude with a reminder to complete the 
session evaluation, and the hub team encouraging providers to put into practice what they 
have learned.  
All clinics assigned to this arm will participate together during an ECHO session, which 
encourages discussion. All sessions will be recorded. Session topics will focus on training 
providers to use the Announcement Approach to communicate HPV vaccination to parents, as 
well as evidence-based systems strategies they can use to supplement and sustain their 
communication efforts. The curriculum will orient providers with the latest data about HPV-
related cancers and vaccination rates, ACIP guidelines, HPV vaccine effectiveness and safety, the 
rationale for targeting younger adolescents, and to consistently deliver strong recommendations 
(Session 1 [S1]). Even though participants may be familiar with this material, the hub team will 
begin to gain providers’ trust by illustrating their expertise and interest in HPV vaccination. In 
the next sessions, experts will train providers on how to: use a structured approach and 
research-tested messages for answering common parent questions and concerns (S2 and S3), 
and engage in conversations with vaccine-hesitant parents (S4). Providers will then receive 
presentations on systems strategies including how to: conduct immunization quality 
improvement (S5), engage all clinic personnel in vaccination efforts (S6), and access evidence-
based materials and community resources (S7). We will finish discussing how providers can use 
reminder and recall notices and sustain HPV vaccination efforts (S8).   
HPV ECHO+ intervention arm. Clinics in this study arm will receive HPV ECHO sessions, as 
described above, plus systems communications to parents who initially decline vaccination. 
Sessions for intervention arms will be identical in format and curriculum. Case presentations will 
STUDY00015734
Approval: 1/25/2024

Page 16 of 25 (V.01/21/2019) 
be unique to each arm as spoke providers will present and discuss their unique clinical 
challenges related to HPV vaccination. We do not anticipate sessions to diverge across arms 
because cases will address common topics seen in clinics. 
For our systems communication, following, standard practice, participating clinics will 
prospectively identify patients and parents who forgo HPV vaccination at healthcare visits and 
will send recall notices to notify parents that their child is behind for HPV vaccination and 
address their primary reason for not vaccinating. We selected recall notices as our systems 
strategy because studies indicate that they can be delivered through various modalities and be 
equally effective.12-16 Importantly, most EHR programs are capable of creating reports that 
identify specific populations (i.e., children whose parents declined HPV vaccination) for recall 
notices.17 Also, most EHR programs can be programmed to send automatic recall notices that 
are tailored for patients and accompanied by educational messages.17 Our systems strategy will 
take advantage of those features available in most EHR programs. As providers will be trained to 
ask and document the parent’s main concern about HPV vaccination (a step in the 
Announcement Approach), recall notices will include research-tested messages to specifically 
address the reported concern. Following best practices, our recall notices will use 
announcement-style language, state that the provider strongly recommends that the child 
receive the vaccine now, emphasize cancer prevention, and encourage early vaccination to take 
advantage of the 2-dose schedule. When available, these communications will also point out the 
convenience of nurse visits. Notices will go out at the beginning of each calendar month. Clinics 
will send secure electronic notices to parents through the patient’s EHR portal as the primary 
communication modality. Clinics will send mailed recall notices to parents who have opted out 
of the portal messaging, these will be provided by study. Regardless of the modality, notices will 
look similar. Recall notices will be sent to vaccine declining parents every 6 months for 12 
consecutive months starting the sixth month after the start of the intervention (6- and 12-
month). These communications would end early if the adolescent initiated HPV vaccination 
within the 12 months or the parent declined receiving these recall notices. 
7.2.4
Wait List Control Group
Participants in this arm will not receive any HPV ECHO intervention during the study period. 
After the intervention training and outcome evaluations have been completed (12 months), 
clinics in the control group will have full access to the recorded HPV ECHO sessions (intervention 
programming) and study materials (e.g., electronic recall notices). To keep the control group 
engaged, control participants will be invited to participate in other active ECHO sessions that are 
relevant to primary care, but do not overlap with vaccination. Control group participants will be 
able to receive no-cost CME credits for attending these live ECHO sessions. 
7.2.5
Quarterly Check-ins – Intervention and Control Group
All participants in the intervention and control groups will complete 4 quarterly check-ins to 
remind them about their participation in the study and assess if the clinic has engaged in any 
other vaccine communication or quality improvement training or project (4 questions). 
7.2.6
Final Evaluation – Intervention and Control Group 
All participants in the intervention (HPV ECHO and HPV ECHO+) and control groups will 
complete a final evaluation at 4-months  at the completion of the telementoring sessions. The 
final evaluation will repeat measures from baseline. This evaluation will take you approximately 
15-20 minutes to complete. As part of the final evaluation, at 12-months, the clinic’s study 
contact will enter in the immunization data for the participating clinic site (repeated measures 
from the baseline). 
7.2.7
Key Informant Interviews
STUDY00015734
Approval: 1/25/2024

Page 17 of 25 (V.01/21/2019) 
One or two of the most active participants (attended the most HPV ECHO live sessions) from 
each intervention clinic (HPV ECHO and HPV ECHO+) will be invited to participate in a 30-minute 
interview. The interview will occur after participants complete the series and final evaluation. 
Interviews will occur virtually via Zoom and will be audio and video recorded. 
7.2.8 
Nested study
In the exploratory nested study, a total of 211 parents of adolescents, ages 11 to 17, will be 
recruited over a 2-year period and will be followed for up to 12 months. Parents will be eligible 
if they decline the first dose of HPV vaccination for their child after receiving a provider 
recommendation and the provider documents the main reason for declining vaccination in the 
child’s EHR. We will target parents whose children have received at least one of the other 
vaccines recommended for adolescents (Tdap or meningococcal) because they have already 
shown some positive attitude towards vaccination and may be more amenable to getting the 
HPV vaccine at a later visit than those who completely oppose all vaccines. HPV vaccine uptake 
will be tracked using the child’s EHR. 
Recruitment of parents will take place in a subset of 4 PennState Health clinics (~50 parents per 
clinic). Our project manager will train clinic’s nurses or staff on the study protocol, and they will 
identify eligible subjects who will receive an invitation to complete screener and consent. 
Parents will complete a total of 5 online surveys through our secure, web-based REDCap 
platform at baseline (within the first week of declining HPV vaccination) and then, every 3 
months for 12 months (3-, 6-, 9- and 12-month follow-ups). Parents will receive follow-up 
surveys 7 days after the recall notices (HPV ECHO+ arm; Aim 1) so they would have enough time 
to make an appointment or get the vaccine. For each survey, we will send up to two email or 
mail reminders to non-respondents. If the parent gets the HPV vaccine for their child within the 
first few months of study enrollment, they will continue to receive all 5 surveys. The HPV 
vaccine is a multi-dose vaccine, so assessing vaccine communication over the 12-month time 
period will provide more information on vaccine initiation and completion. 
7.3
Duration of Participation
Duration of participation for primary clinic staff, regardless of study arm assignment, includes 15-20 
minutes for the baseline and 15-20 minutes for the follow-up survey. In addition, participants in HPV 
ECHO and HPV ECHO+, will participate in 8 hours of intervention training, and may complete up to eight 
5-minute surveys if they are interested in obtaining continuing education credits. Preparation for case 
discussions in the two intervention groups will take approximately 2 hours. A subset of clinic staff may 
also choose to participate in the key informant interviews. The interviews will last 30 minutes. Total 
participation will span a period of 6 months.  
For the nested study, duration of participation is 15 minutes per online survey (5 in total) through a 12-
month period.
8.0
Subject Numbers and Statistical Plan
8.1
Number of Subjects
We will recruit 45 primary care clinics in Pennsylvania, and each participating clinic can bring up to 6 
staff to participate in the study, including physicians, nurses, physician assistants, medical director and 
other staff involved in the delivery, communication or scheduling of vaccine (n=270). Of those clinic 
staff, only 1-2 per intervention clinic will be invited to participate (n=60). 
For the nested study, we will recruit a total of 211 parents of adolescents, ages 11 to 17. Of note, we 
have recruited 211 parents following the recruitment procedures outlined in this protocol. The final 
invite was sent in September 2023 and the final participant chose to enrol in November 2023.  
STUDY00015734
Approval: 1/25/2024

Page 18 of 25 (V.01/21/2019) 
8.2
Sample size determination
This study will include 45 clinics with 15 randomized to each of the 3 arms. Monte Carlo simulations 
were used to estimate power under the following assumptions: vaccine initiation rate of 40% in the 
control arm, a mean of 425 adolescents/clinic (ages 11-14), and a between-clinic standard deviation of 
0.05. Based on these simulations, we estimate that this study design will yield statistical power of at 
least 0.90, with a one-sided Type 1 error of 0.025, to detect a 5% increase in the proportion of 
adolescents who have initiated HPV vaccination (≥1 dose) in intervention versus control group (H1). The 
0.025 level uses Bonferroni correction to adjust for the two comparisons to be performed (HPV ECHO 
versus control and HPV ECHO+ versus control). The expected number of adolescents/clinic and vaccine 
initiation rate were estimated from available data of a sample of 15 primary care clinics from Penn State 
Health. 
For the key informant interviews, we will conduct rapid qualitative analyses after each cohort and will 
adjust the sample size based on when we reach data saturation. 
The nested study is exploratory; it does not include a sample size determination. 
8.3
Statistical methods
8.3.1. Randomization
Using a 1:1:1 ratio, we will randomly assign clinics to receive either HPV ECHO, HPV ECHO+, or control. 
Each arm will have 15 clinics, for a total of 45 clinics. Covariate-constrained randomization will be used 
to ensure balance among the three arms with respect to clinic size, clinic type (academic vs. non-
academic), rurality/urbanicity using Rural-Urban Continuum Codes from clinic’s physical address (RUCC), 
and historic adolescent HPV vaccination rates. Given a core component of the ECHO Model is spoke 
provider presentations of cases, sessions are usually capped to 25 participants to allow each provider to 
present at least once during the program and actively engage in discussions. In our prior RCT of ECHO-
alike quality improvement coaching, an average of 5 providers per clinic attended live webinars.11 
Accordingly, we will have three waves of 15 clinics each (one wave per year during years 2 to 4) and will 
randomize them to the 3 study arms (5 clinics per arm, per wave); this design will maximize 
accommodation of participants in ECHO sessions. Randomization will ensure groups are balanced among 
the waves.
8.3.2. Effectiveness of ECHO
Our primary outcome is the proportion of patients, ages 11-14, who have initiated their HPV vaccination 
(≥1 dose) in the 12 months after ECHO implementation. We target 11- to 14-year-olds versus older 
adolescents because revised ACIP guidelines may motivate more parents to initiate the 2-dose series at 
younger ages to take advantage of the reduced vaccine schedule. We also target initiation versus 
completion because it is a more amenable indicator of changes in providers’ practices. Our secondary 
outcomes include vaccination by gender, for 15-17-year-olds, up-to-date HPV vaccination, for 
concomitant vaccination (i.e., with Tdap and meningococcal vaccines), and at interim time points (3-, 6-, 
and 9-month follow-up). We will assess Aim 1 outcomes using provider-verified EHR data from 
participating clinics. We will use EHR data because adoption and use of EHR for documenting 
vaccinations in Pennsylvania is very high; contrary to vaccination reporting to the state Immunization 
Information System (IIS), which is voluntary for adolescent vaccines (low IIS capacity) as the majority of 
U.S. states.18
We will also characterize clinics by size (total number of adolescent patients), patient insurance mix 
(proportion of VFC-eligible adolescents), specialty (pediatrics or family medicine), rurality (RUCC), HPV 
vaccination at baseline, and patient demographics (proportion by sex, race/ethnicity). Our experienced 
biostatistician will work with each participating provider or their clinic’s data manager to ensure that 
EHR queries follow American Immunization Registry Association’s guidelines to properly define study 
populations and vaccination status.19 All these data will be aggregated at the clinic level, we will not 
request nor receive any patient-level data. 
STUDY00015734
Approval: 1/25/2024

Page 19 of 25 (V.01/21/2019) 
For the primary outcome, data will be analyzed using a generalized linear mixed-effects model with a 
binomial distribution and logit link. The model will include treatment group as a fixed effect, and wave 
and clinic as random effects. Contrasts will be constructed to test HPV ECHO versus control and HPV 
ECHO+ versus control. Analyses for secondary outcomes will be performed using the same model 
specified for the primary outcome, as well as respective contrasts. For analysis by sex, multiplicative 
interaction terms for sex by treatment will be added to the model.
8.3.3. Implementation outcomes 
Our evaluation of implementation outcomes is informed by both the Damschroder’s Consolidated 
Framework for Implementation Research (CFIR)20 and Proctor’s Implementation Outcomes Framework 
(IOF)21 and will allow us to capture the complexity of implementation in a variety of clinics within their 
unique contexts. Data collection will be done through the following activities: 1) Implementation surveys 
will be administered online through REDCap to clinic staff from enrolled clinics. The surveys will evaluate 
appropriateness and acceptability using adapted items from the Intervention Appropriateness Measure 
and the Acceptability of Intervention Measure scales.22 The survey will also evaluate adoption using 
items from validated instruments measuring the frequency and quality of HPV vaccine 
recommendations and adherence to the Announcement Approach steps. 2) Fidelity checklists will be 
used to document fidelity or the extent to which hub experts (e.g., session format, length) and 
participants (e.g., case discussion) participate in ECHO sessions per protocol. With the permission of 
ECHO participants, we will video-record and watch one-third of sessions for internal quality 
improvement. 3) Key informant interviews will be conducted with active clinic staff that are enrolled in 
the intervention clinics. The interview will evaluate the acceptability of the training using some CFIR 
measures, and evaluate the adoption of measures into practice. These will be conducted after each 
cohort to determine if adaptations to the intervention need to be made to the intervention. 4) 
Recruitment tracking to assess reach to clinics and providers including attendance at each session. For 
clinics that decline participation, we will document the reason. 5) Costs will be estimated from the 
perspective of providers.23 For both intervention arms and control group, we will calculate the 
incremental costs as: (i) any new technology the providers needed to purchase to participate in sessions, 
(ii) providers’ time to participate in sessions, including time to compile case information (costs estimated 
as the number of hours for each provider type multiplied by an average hourly wage from the Bureau of 
Labor Statistics), and (iii) hub staff time calculated similarly to (ii). While hub staff time are not paid by 
each clinic site, we include these costs as representative of incremental costs that would need to be 
considered in future expansions. For the HPV ECHO+ incremental costs, we will also include: (iv) any new 
technology that needs to be purchased to enable communication to parents and (v) additional provider 
or clinic staff time to facilitate the delivery of these communications.
8.3.4. Nested study
All surveys will systematically assess exposure to and impact of vaccine information from study arms or 
naturally-occurring sources (i.e., social media, traditional media, social networks). We will assess HPV 
vaccination intentions using a 3-item scale that addresses the three aspects of intentions (i.e., 
expectation, desire, and planning).24 We will also assess overall HPV vaccination attitudes and beliefs 
using a short validated scale.25 We will also ask parents whether the study communication was adequate 
to address their main concern, whether external information impacted their main concern, and whether 
their main concern changed over time.26 We will assess information’s transport and credibility using 
validated scales conceptualized in narrative persuasion theories.27-28 Exploratory analyses will focus on 
describing the exposure to vaccine information, as well as its impact on attitudes and secondary 
acceptance of HPV vaccination among the subset of parents surveyed. Specifically, we will conduct 
mediation analyses to explore the constructs that could explain our hypothesis; why HPV ECHO+ is 
associated with greater secondary acceptance than HPV ECHO. We will calculate the indirect effects 
based on standardized path coefficients and conduct bootstrapping. 
STUDY00015734
Approval: 1/25/2024

Page 20 of 25 (V.01/21/2019) 
9.0
Data and Safety Monitoring Plan
Not applicable: This study does not involve more than minimal risk to subjects, and the magnitude of 
harm/discomfort is not greater than that ordinarily encountered in daily life.  
9.1
Periodic evaluation of data
N/A
9.2
Data that are reviewed
N/A
9.3
Method of collection of safety information
N/A
9.4
Frequency of data collection
N/A
9.5
Individuals reviewing the data
N/A
9.6
Frequency of review of cumulative data
N/A
9.7
Statistical tests
N/A
9.8
Suspension of research
N/A
10.0
Risks
The risk of participating is minimal, and the magnitude of harm/discomfort is not greater than that ordinarily 
encountered in daily life. In the HPV ECHO+ group and the nested study, the recall notices to be used do not 
contain any graphic or explicit content but may invoke an emotional response due to the nature of the topic 
(i.e., vaccination). However, these are health topics commonly discussed in primary care visits, and recall notices 
are standard practice. The questions asked in the surveys are not sensitive in nature and come from validated 
instruments used in prior studies.   
There is a risk of loss of confidentiality if participant information or identity is obtained by someone other than 
the investigators, but precautions will be taken to prevent this from happening. The confidentiality of 
participants’ electronic data will be maintained to the degree permitted by the technology used. Absolute 
confidentiality cannot be guaranteed. Personal information and survey responses will be labeled with a 
participant code assigned by REDCap and stored in this database. All HPV ECHO sessions will be recorded, 
session recordings will only be available to clinic staff that are enrolled in the study. Employers will not have 
access to the recordings. 
11.0
Potential Benefits to Subjects and Others 
11.1
Potential Benefits to Subjects
There are no direct benefits to participants. However, primary care clinics may include improved rates of 
HPV vaccination. Potential benefits to clinic staff may include improvement in confidence, self-efficacy, 
STUDY00015734
Approval: 1/25/2024

Page 21 of 25 (V.01/21/2019) 
knowledge, and skills to communicate about HPV vaccination. Potential benefits to parents in the 
nested study may include their children getting the HPV vaccine. 
11.2
Potential Benefits to Others
Benefits to others are long term, including improved patient care and may contribute to the 
dissemination if this type of training to help clinics communicate with parents about the importance of 
HPV vaccination and then, increase HPV vaccination rates. 
12.0
Sharing Results with Subjects
For clinics and clinic staff participating in the study, a project brief will be created at the conclusion of the study 
that will be shared with clinics. 
For the nested study, results will not be shared with participants.
13.0
Subject Payment and/or Travel Reimbursements
Clinic staff will receive monetary compensation for their participation in the study, but clinic staff will receive up 
to 8 no-cost CME credits.  One CME credit will be awarded per each hour of participation. Clinics may also elect 
to receive MOC Part 4 credit for their clinic’s participation in the intervention (25 points for general pediatrics, 
20 points for family medicine). Providers will also receive a $50 gift card (Amazon or Target) for completing the 
baseline and follow-up surveys for up to $100 total. Additionally, providers that participate in the 30-minute key 
informant interview will be able to receive an additional $50 gift card as compensation for their time. This 
means that clinic staff that choose to participate in both the surveys and the interviews will be able to receive up 
to $150 for their participation. Clinics such as FQHCs have expressed concern over the financial burden caused 
by the lost clinic time due to their participation. To ensure all clinics have an equal opportunity to participate, 
funds have been made available for clinics to have the option to ask for compensation at $150/session attended 
for up to $1200/clinic, if the clinic has at least one attendant at each of the 8 sessions. 
Participants in the nested study will receive a $25 gift card for each of the first four online survey they complete, 
and $50 for completing the final survey. In total, participants can be compensated up to $150 (5 surveys) for 
their participation in this study.
14.0
Economic Burden to Subjects
14.1
Costs
There are no financial costs associated with participating in this research.
14.2
Compensation for research-related injury
It is the policy of the institution to provide neither financial compensation nor free medical treatment 
for research-related injury. In the event of injury resulting from this research, medical treatment is 
available but will be provided at the usual charge. Costs for the treatment of research-related injuries 
will be charged to subjects or their insurance carriers.
15.0
Resources Available 
15.1
Facilities and locations
The ECHO model utilizes videconferencing technology (Zoom) as a platform for telementoring and case-based 
learning, with a hub-and-spoke structure. Participants may participate in the study from any location. For the 
STUDY00015734
Approval: 1/25/2024

Page 22 of 25 (V.01/21/2019) 
nested study, all study activities will occur at any locations chosen by participants because they can access the 
screener, SER, and surveys through their personal electronic devices.
15.2
Feasibility of recruiting the required number of subjects
We do not anticipate any challenges in recruiting, given the support of Penn State Health and the 
Pennsylvania Association of Community Health Centers to assist with recruitment activities. For the 
nested study, participants will be recruited with the support of four partnering clinics. 
15.3
PI Time devoted to conducting the research
Dr. Calo will devote 25% to oversee study activities.
15.4
Availability of medical or psychological resources
It is not anticipated that medical or psychological resources will be needed, given that study procedures are 
minimal risk. 
15.5
Process for informing Study Team
The investigators and project coordinator/study staff have completed their required Collaborative IRB Training 
Initiative (CITI) in the protection of human research subjects.  The study team will be educated on the 
importance of confidentiality, and proper data handling and storage. Regular team meetings and emails will be 
used to coordinate the study.
16.0
Other Approvals
16.1
Other Approvals from External Entities
N/A
16.2
Internal PSU Committee Approvals
Check all that apply:
  Anatomic Pathology – Penn State Health only – Research involves the collection of tissues or use of 
pathologic specimens. Upload a copy of “HRP-902 - Human Tissue For Research Form” in CATS IRB. 
  Animal Care and Use – All campuses – Human research involves animals and humans or the use of 
human tissues in animals
  Biosafety – All campuses – Research involves biohazardous materials (human biological specimens 
in a PSU research lab, biological toxins, carcinogens, infectious agents, recombinant viruses or DNA 
or gene therapy).
  Clinical Laboratories – Penn State Health only – Collection, processing and/or storage of extra tubes 
of body fluid specimens for research purposes by the Clinical Laboratories; and/or use of body fluids 
that had been collected for clinical purposes but are no longer needed for clinical use. Upload a copy 
of “HRP-901 - Human Body Fluids for Research Form” in CATS IRB. 
STUDY00015734
Approval: 1/25/2024

Page 23 of 25 (V.01/21/2019) 
  Clinical Research Center (CRC) Advisory Committee – All campuses – Research involves the use of 
CRC services in any way.
  Conflict of Interest Review – All campuses – Research has one or more of study team members 
indicated as having a financial interest.
  Radiation Safety – Penn State Health only – Research involves research-related radiation 
procedures. All research involving radiation procedures (standard of care and/or research-related) 
must upload a copy of “HRP-903 - Radiation Review Form” in CATS IRB. 
  IND/IDE Audit – All campuses – Research in which the PSU researcher holds the IND or IDE or 
intends to hold the IND or IDE.
  Scientific Review – Penn State Health only – All investigator-written research studies requiring 
review by the convened IRB must provide documentation of scientific review with the IRB 
submission. The scientific review requirement may be fulfilled by one of the following: (1) external 
peer-review process; (2) department/institute scientific review committee; or (3) scientific review by 
the Clinical Research Center Advisory committee.  NOTE: Review by the Penn State Health Cancer 
Institute (PSCI) Protocol Review Committee or the PSCI Disease Team is required if the study 
involves cancer prevention studies or cancer patients, records and/or tissues. For more information 
about this requirement see the IRB website.
17.0
Multi-Site Study
N/A
18.0
Adverse Event Reporting
18.1
Reporting Adverse Reactions and Unanticipated Problems to the Responsible IRB
In accordance with applicable policies of The Pennsylvania State University Institutional Review Board 
(IRB), the investigator will report, to the IRB, any observed or reported harm (adverse event) experienced 
by a subject or other individual, which in the opinion of the investigator is determined to be (1) 
unexpected; and (2) probably related to the research procedures. Harms (adverse events) will be 
submitted to the IRB in accordance with the IRB policies and procedures.
19.0
Study Monitoring, Auditing and Inspecting
19.1
Auditing and Inspecting
The investigator will permit study-related monitoring, audits, and inspections by the Penn State quality 
assurance program office(s), IRB, the sponsor, and government regulatory bodies, of all study related 
documents (e.g., source documents, regulatory documents, data collection instruments, study data etc.).  
The investigator will ensure the capability for inspections of applicable study-related facilities (e.g., 
pharmacy, diagnostic laboratory, etc.).
20.0
Future Undetermined Research: Data and Specimen Banking
20.1
Data and/or specimens being stored
N/A
STUDY00015734
Approval: 1/25/2024

Page 24 of 25 (V.01/21/2019) 
20.2
Location of storage
N/A
20.3
Duration of storage
N/A
20.4
Access to data and/or specimens
N/A
20.5
Procedures to release data or specimens
N/A
20.6
Process for returning results
N/A
21.0
References
1) Walker TY, Elam-Evans LD, Yankey D, et al. National, regional, state, and selected local area vaccination coverage 
among adolescents aged 13–17 years—United States, 2018. MMWR Morb Mortal Wkly Rep. 2019;68(33):718-723. 
2) Brewer NT, Hall ME, Malo TL, Gilkey MB, Quinn B, Lathren C. Announcements Versus Conversations to Improve HPV 
Vaccination Coverage: A Randomized Trial. Pediatrics. 2017;139(1). pii: e20161764.
3) Walling EB, Benzoni N, Dornfeld J, Bhandari R, Sisk BA, Garbutt J, Colditz G. Interventions to Improve HPV Vaccine 
Uptake: A Systematic Review. Pediatrics. 2016;138(1). pii: e20153863.
4) Finney Rutten LJ, St Sauver JL, Beebe TJ, et al. Association of both consistency and strength of self-reported clinician 
recommendation for HPV vaccination and HPV vaccine uptake among 11- to 12-year-old children. Vaccine. 
2017;35(45):6122-8.
5) Warner EL, Ding Q, Pappas L, et al. Health care providers’ knowledge of HPV vaccination, barriers, and strategies in a 
state with low HPV vaccine receipt: mixed-methods study. JMIR Cancer. 2017;3(2):e12
6) Irving SA, Groom HC, Stokley S, et al. Human papillomavirus vaccine coverage and prevalence of missed opportunities 
for vaccination in an integrated healthcare system. Acad Pediatr. 2018;18(2S):S85-S92.
7) Dempsey AF, Pyrznawoski J, Lockhart S, Barnard J, Campagna EJ, Garrett K, Fisher A, Dickinson LM, O'Leary ST. Effect 
of a Health Care Professional Communication Training Intervention on Adolescent Human Papillomavirus 
Vaccination: A Cluster Randomized Clinical Trial. JAMA Pediatr. 2018;172(5):e180016.
8) Arora S, Thornton K, Komaromy M, Kalishman S, Katzman J, Duhigg D. Demonopolizing medical knowledge. Acad 
Med. 2014;89:30–32.
9) Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The Impact of Project ECHO on Participant and Patient 
Outcomes: A Systematic Review. Acad Med. 2016;91(10):1439-1461.
10) McBain RK, Sousa JL, Rose AJ, Baxi SM, Faherty LJ, Taplin C, Chappel A, Fischer SH. Impact of Project ECHO Models of 
Medical Tele-Education: a Systematic Review. J Gen Intern Med. 2019. doi: 10.1007/s11606-019-05291-1.
11) Calo WA, Gilkey MB, Leeman J, MacKinnon J, Averette C, Sanchez S, Kornides M, Brewer NT. Coaching primary care 
clinics for HPV vaccination quality improvement: Comparing in-person and webinar implementation. Transl Behav 
Med. 2019; 9(1)23-31.
12)  Walling EB, Benzoni N, Dornfeld J, Bhandari R, Sisk BA, Garbutt J, Colditz G. Interventions to Improve HPV Vaccine 
Uptake: A Systematic Review. Pediatrics. 2016;138(1). pii: e20153863.
13) Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient 
immunization navigator program on improving adolescent immunizations and preventive care visit rates. Arch 
Pediatr Adolesc Med. 2011;165(6):547–553.
14) Suh CA, Saville A, Daley MF, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. 
Pediatrics. 2012;129(6).
15) Chao C, Preciado M, Slezak J, Xu L. A randomized intervention of reminder letter for human papillomavirus vaccine 
series completion. J Adolesc Health. 2015;56(1):85–90.
STUDY00015734
Approval: 1/25/2024

Page 25 of 25 (V.01/21/2019) 
16) Szilagyi PG, Albertin C, Humiston SG, et al. A randomized trial of the effect of centralized reminder/recall on 
immunizations and preventive care visits for adolescents. Acad Pediatr. 2013;13(3):204–213
17) American Academy of Pediatrics. AAP Immunization Resources – Immunization Reminder & Recall Systems. Available 
from: https://www.aap.org/en-us/Documents/immunization_reminderrecall.pdf Accessed October 21, 2019.
18) Joint State Government Commission. General Assembly of the Commonwealth of Pennsylvania Youth Vaccinations in 
Pennsylvania: Report of the Work Group on Youth Vaccinations of the Advisory Committee on Public Health Law. 
Available from: http://jsg.legis.state.pa.us/resources/documents/ftp/publications/2016-05-
06%20WEBSITE%20%20SR27%20FINAL%20REPORT%20May%204%202016%20(WB).pdf Accessed October 21, 2019.
19) American Immunization Registry Association (AIRA). AIRA Modeling of Immunization Registry Operations Work 
Group (eds). Management of Patient Active/Inactive Status in Immunization Information Systems: Replacement of 
2005 Guidelines. Atlanta, GA: American Immunization Registry Association. April, 2015. 
20) Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services 
research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 
2009;4:50.
21) Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for 
implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment 
Health. 2011 Mar;38(2):65-76.
22) Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome 
measures. Implementation Science. 2017;12(1):108.
23) Segel JE. Cost-of-Illness Studies- A Primer. RTI International RTI-UNC Center of Excellence in Health Promotion 
Economics; 2006. Accessed from: 
https://pdfs.semanticscholar.org/3bbf/0a03079715556ad816a25ae9bf232b45f2e6.pdf Accessed October 21, 2019.
24) Shah PD, Calo WA, Gilkey MB, Boynton MH, Alton Daily S, Todd KG, Margolis MA, Brewer NT. Questions and 
concerns about HPV vaccine: A communication experiment. Pediatrics. 143(2): e20181872. 
25) Gowda C, Carlos RC, Butchart AT, Singer DC, Davis MM, Clark SJ, Dempsey AF. CHIAS: a standardized measure of 
parental HPV immunization attitudes and beliefs and its associations with vaccine uptake. Sex Transm Dis. 
2012;39(6):475-81.
26) Dempsey AF, Pyrzanowski J, Campagna EJ, Lockhart S, O'Leary ST. Parent report of provider HPV vaccine 
communication strategies used during a randomized, controlled trial of a provider communication intervention. 
Vaccine. 2019;37(10):1307-1312.
27) Green MC & Brock TC. The role of transportation in the persuasiveness of public narratives. Journal of Personality 
and Social Psychology. 2000;79:701–721.
28) Yale RN. Measuring narrative believability: Development and validation of the Narrative Believability Scale (NBS-12). 
Journal of Communication. 2013;63:578–599
22.0
Confidentiality, Privacy and Data Management 
See the Research Data Plan Review Form
STUDY00015734
Approval: 1/25/2024