Web-based High School Media Literacy for Healthy Relationships
Phase: NA
Conditions: Sexual Behavior
Sponsor: Innovation Research & Training

 
 
Study Protocol and Statistical Analysis Plan 
 
Official Title: Web-based High School Media Literacy for Healthy Relationships 
Brief Title: Evaluation Study of the Online High School Media Aware Program 
 
ClinicalTrials.gov ID: NCT04035694 
 
Principal Investigator: Dr. Tracy Scull 
 
Protocol Approved by IRB:  April 29, 2019 
Modification to Protocol Approved by IRB:  March 26, 2020 
 

 
1 
 
Study Protocol Approved by IRB:  April 29, 2019 
 
1. Purpose of the research project (e.g., Specific aims) 
Adolescents are at risk for unhealthy sexual and relationship outcomes. Approximately 41% of 
high school students in the U.S. have had sex, and 43% of sexually active youth did not use a 
condom during their last sexual intercourse. Engaging in risky sexual behaviors can have serious 
consequences for adolescents. For example, approximately half of the 20 million new STI cases 
each year are among people between the ages of 15 and 24, and the U.S. has one of the highest 
teen birth rates among comparable countries. Adolescents are also at risk for sexual assault and 
dating violence. Ten percent of female high school students reported being forced to have 
intercourse against their will, and 16% of female high school students who had dated someone 
in the previous year reported being forced to engage in sexual acts that they did not want to by 
a person they were dating. Additionally, in a study among the almost 70% of high school 
students who had dated in the previous year, 10% reported experiencing physical dating 
violence. These findings suggest that comprehensive sexual health programs that include not 
only discussions of STIs, pregnancy, and contraception, but also healthy relationships, consent, 
and dating violence are needed. 
Media literacy education (MLE) is an effective intervention for changing adolescent risk behavior 
outcomes. MLE has been shown to be an effective intervention for changing risky health 
behavior outcomes including those related to the use of substances, body image issues, and 
sexual health. Furthermore, teachers have reported that MLE is an easier avenue for teaching 
about sexual health than traditional sexual health education methods because it provides an 
engaging and neutral vehicle for discussing attitudes and knowledge about sex. Despite 
evidence regarding the impact of media on adolescent sexual health and the effectiveness of 
MLE, high schools do not systematically teach media literacy and most comprehensive sexual 
health curricula do not address the influence of media on adolescent sexual decision making. 
A web-based MLE program for high school students, Media Aware, was developed and tested in 
a Phase I study. Media Aware is designed to provide students in 9th and 10th grade classrooms 
with sexual health knowledge, media literacy skills, and the skills to make healthy decisions 
about sexual activity. This Phase II RCT will allow us to evaluate the effects of Media Aware 
regarding student sexual health and provide insight into the sustainability of positive attitudinal 
and behavioral outcomes over time. 
The specific aim of this study is to: 
i. Conduct a pretest-posttest RCT with a three-month follow-up to investigate the 
sustainability of positive outcomes in students in the intervention group and to 
examine if behavioral indicators improve over time compared to students in the 
delayed intervention group and to assess the effectiveness of the program 
implementation using teacher feedback and analyses of program process data.  
 
2. Study Procedures (e.g., data collection, intervention implementation) 
a. Approximately 16 health education teachers (one per school) who teach 9th or 10th graders 
will be invited to participate in the study with up to five of their 9th or 10th grade health 
education classrooms.  

 
2 
 
b. Participating teachers will be asked to endorse an online consent form, if they have not 
already.   
c. High schools will be randomly assigned to condition (intervention or delayed intervention) 
after teacher consent and school approval is obtained.  
d. All participating teachers will receive access to teacher resources, the dashboard, and Media 
Aware. Teachers will then be asked to review the Media Aware program and associated 
teacher resources. 
e. Control teachers will be asked to refrain from teaching Media Aware until after the follow-
up.  
f. A member of the project staff or the classroom teacher will introduce the study to students 
in the participating classrooms (intervention and delayed intervention) using a script and 
invite students to participate (complete three questionnaires).  
g. Study permission forms will be distributed to students.  
h. Students will be instructed to review the study forms with their parents, indicate whether 
they wish to participate or not, and return to school with a signed copy of both the forms, 
regardless of their decision to participate.  
i. 
Teachers will collect and store the parental permission and student assent forms in a secure 
place until a member of the project staff is present to collect them.  Alternately, they will 
mail them to the iRT office with a pre-addressed envelope provided by the project staff. 
Teachers will be instructed about storage by a member of the research staff.  
j. 
If over 80% of the eligible classroom students return/submit their parental permission 
forms/youth assent forms (either marked “yes” or “no”), the teacher will receive a $30 
incentive per class. Teachers may receive up to $150 ($30/classroom). 
k. Students whose parents choose not to give permission, or students who choose not to 
assent even though their parents have given permission, or who simply have not returned 
signed forms in time (prior to the administration of the pre-test), will not participate in the 
research study (i.e., complete questionnaires). Students who do not have permission to 
receive sexual health education will also not participate in the research study.  
l. 
Teachers will provide alternative activities for students who are not participating in the 
research study during questionnaire administration. Teachers will follow school procedures 
for students who do not have permission to receive sexual health education while the 
program is being taught.  
m. Project staff member will administer the web-based pre-test questionnaires in each 
classroom to students with parent permission (for sexual health education and study 
participation) and who have assented to participate.  
i. Project staff members will arrange the desks with space between students and 
provide screen shields to increase privacy during the questionnaire. They will also 
maintain quiet in the classroom during data collection and be available to answer 
questions.  
ii. Project staff member will read from a prepared script to instruct students how to 
log-in and complete the questionnaire. Questionnaires will all be completed using 
the online service Survey Monkey.  
iii. Students will be given an envelope with a unique identifying number inside by a 
project staff member. Students will navigate to the questionnaire URL and then 
enter their unique ID. They will use this number in place of their name on the 
questionnaire. After entering the number, they will seal it in the envelope and sign 
their name across the seal. Project staff members will collect the envelopes and 

 
3 
 
keep them secure until they can be stored securely at the iRT office in a locked 
cabinet. 
iv. Students will complete the web-based questionnaires individually. Paper-based 
questionnaires can also be provided, if necessary. If paper-based questionnaires are 
used, they will be collected and stored securely by a project staff member until they 
can be stored securely at the iRT office in a locked cabinet.  
v. The questionnaire should take less than one class period to complete. Students will 
receive a small incentive for their participation.  
n. The intervention group will complete the Media Aware program over approximately the 
next four health class periods.   
o. The delayed-intervention group will receive their regular health education lessons in their 
health classes. Teachers in the delayed-intervention group will be asked to refrain from 
teaching any topics related to sexual health or media literacy from the pre-test through the 
completion of the follow-up.  
p. Approximately four health class periods after the pre-test questionnaire is completed, a 
project staff member will administer the web-based post-test questionnaires in each 
classroom to participating students.  
i. Project staff members will arrange the desks with space between students and 
provide screen shields to increase privacy during the questionnaire. They will also 
maintain quiet in the classroom during data collection and be available to answer 
questions.  
ii. A project staff member will read from a prepared script to instruct students how to 
log-in and complete the questionnaire.   
iii. Project staff members will return the sealed ID envelopes to students. Students will 
open the envelope and enter their number on the questionnaire. They will seal the 
number in a new envelope and sign their name across the seal once again. Project 
staff member will collect the opened envelopes and sealed envelopes and keep 
them secure at the iRT office in a locked cabinet. The opened envelopes will be 
shredded at school or upon return to the iRT office. 
iv. Students will complete the web-based questionnaires individually. Paper-based 
questionnaires can also be provided, if necessary. If paper-based questionnaires are 
used, they will be collected and stored securely by a project staff member until they 
can be stored securely at the iRT office in a locked cabinet.    
v. The questionnaire should take less than one class period to complete. Students will 
receive a small incentive for their participation.  
q. Sometime after students have completed the program, participating intervention teachers 
will be asked to complete the web-based Consumer Satisfaction Questionnaire (CSQ) on 
their own time. The CSQ will allow teachers to provide feedback on their experiences 
implementing the program. Teachers will receive a unique email link to a questionnaire 
using the iRT Data Collection System. Teachers will receive a total of $100 for reviewing the 
program and teacher resources and for the completion of the CSQ. 
r. Approximately three months after the post-test, project staff members will administer the 
final web-based questionnaire to students.  
i. Project staff members will arrange the desks with space between students and 
provide screen shields to increase privacy during the questionnaire. They will also 
maintain quiet in the classroom during data collection and be available to answer 
questions.  

 
4 
 
ii. Project staff member will read from a prepared script to instruct students how to 
log-in and complete the questionnaire. 
iii. Project staff members will return the secret number envelopes to students. 
Students will open the envelope and enter their number on the questionnaire. This 
time, a project staff member will collect the separated numbers and envelops and 
shred them upon returning to the iRT office. The questionnaire should take less than 
one class period to complete.  
iv. Students will complete the web-based questionnaires individually. Paper-based 
questionnaires can also be provided, if necessary. If paper-based questionnaires are 
used, they will be collected and stored securely by a project staff member until they 
can be stored securely at the iRT office in a locked cabinet. 
v. The questionnaire should take less than one class period to complete. Students will 
receive a small incentive for their participation. If a student has completed all three 
questionnaires will also be entered in a $50 gift card classroom drawing, which will 
be conducted at the conclusion of the follow-up questionnaire.  
s. Teachers in the delayed intervention group will enroll their students the Media Aware 
program after the students complete their follow-up questionnaire. Teachers will be asked 
to complete the CSQ after they use the Media Aware program with their students, and 
review the program. Teachers will receive a total of $100 for reviewing the program and 
teacher resources and for the completion of the CSQ. 
 
3. Brief description of proposed analyses 
 
Preliminary Analyses. Psychometric and descriptive analyses will be conducted to study the 
reliability, validity, and distributions of key variables. The impact of random assignment on 
producing equivalent groups between conditions will be evaluated using tests of a series of 
single factor, condition (intervention, delayed-intervention) random effects ANOVA models 
estimated for each variable with a random intercept at the school level. Missing data will be 
examined at each time point and handled in each outcome analysis with an appropriate 
imputation method and estimates and standard errors will be adjusted for imputations, if 
warranted (Schafer, 1997). 
 
Analyses. First, process data obtained from the LMS will be examined. Usability Analyses: The 
effectiveness of program implementation will be evaluated based upon the average percentage 
of lessons successfully completed (completed >80%). The length of time that students take to 
complete the program will be examined to see if the majority of students can complete the 
program in the allotted time frame (i.e., 4 hours spread over approximately 4 days). Satisfaction 
Analyses: Open-ended feedback from teachers and students on their experiences with the 
program will be examined to see if the teacher resources and dashboard impacted program 
implementation. Consumer satisfaction with the program will be achieved if scores average 2.5 
or above on 80% of the 4-pt. Likert scales. Outcome Analyses: The immediate and short-term 
effectiveness of Media Aware for improving student outcomes will be examined. HLM (with 
random intercept at the school level) will be used to account for conditional non-independence 
in outcome measures at post-test and follow-up using SAS MIXED and will examine multiple 
observations over time. The model for the mean of each outcome will contain level 1 fixed 
effects representing the influence of (1) the student’s pretest score and (2) gender and level 2 
fixed effects of condition. Student variables found to be nonequivalent between groups will be 

 
5 
 
included as covariates in these models. The effect sizes will be calculated by dividing the 
appropriate contrast parameter by the sample standard deviation of the outcome. These 
analyses should reveal if there is a significant improvement in students’ sexual health and media 
outcomes after having received the program as compared with the delayed-intervention group. 
Moderator analyses will examine subpopulations, defined by the three categorical variables of 
gender, relationship status, and prior sexual experience. These findings will provide evidence 
regarding how the effectiveness of the program varies as a function of student characteristics. 
Measures of program fidelity will be examined to see if differing quality or quantity of 
implementation can explain the findings. 
 
 
 
 

 
6 
 
Modification to Protocol Approved by IRB:  March 26, 2020 
 
COVID-19 data collection protocol 
In response to major disruptions in daily and school life as a result of COVID-19, and in the interest of 
public safety, we are proposing changes to the data collection procedures for this project that will allow 
for the collection of data without social contact between data collectors and high school students or 
personnel.   
The current data collection and incentive distribution procedures involve project staff members 
administering a web-based questionnaire to students in a classroom setting in high schools and students 
receiving their incentives in person after the data collection is completed. The revised data collection 
procedure will involve students receiving a link to the web-based questionnaire in an email, whereby 
students can complete the questionnaires inside or outside of a classroom setting (e.g., at home) 
without the need for data collectors to be physically present in the classroom. Tangible incentives (i.e., 
water bottles and sticky notes) will be replaced with electronic gift cards that can be distributed 
electronically.  
Below is a summary of the online data collection administration procedure: 
1. Preparation/communication of study protocol change: 
a. Research staff will contact teachers and school administration (e.g., person who gave 
permission for the study to take place at the school) to determine the school’s status related 
to school closure/online instruction.  
b. Research staff will communicate to teacher and school administrator the revisions included 
in this study protocol.  
c. Research staff will send an email to students and parents/guardians communicating the 
revisions in the study protocol. Parent and child emails will stress the importance of allowing 
students to complete the post- or follow-up questionnaire in private. If parents did not 
include an email address on the permission form, teachers will be asked to forward the 
email to parents for them to forward the email to their children.   
2. Data collection procedure: 
a. Research staff will create a new set of temporary unique ID numbers for students that link 
to their original unique ID numbers. Only research staff will have a copy of the linking list 
containing both the temporary and original ID numbers.  
b. Research staff will email students with the link to the online questionnaire along with their 
temporary ID number. 
c. If schools are open or if teachers are holding synchronous online classes, teachers will 
instruct students to open the email and complete the questionnaire simultaneously during 
class.  
d. Otherwise, the students may complete the questionnaires from home on their own time 
over a period of about 2 days.  
e. Research staff will send a reminder via email on the second day. After 2 days, the 
questionnaire links will be closed.  
3. Incentive distribution: 

 
7 
 
a. Once data collection for a classroom is complete, research staff will email a $5 Amazon gift 
card to participating students. 
b. With respect to the classroom follow-up drawing for a $50 Visa gift card, research staff will 
draw names of classroom winners from students who have completed all three 
questionnaires, and send them an online gift card or a physical gift card via signed-mail.  
4. Data download 
a. Data will be downloaded from the online survey collector. A research staff member will use 
the linking list to replace the temporary ID number with the original unique ID number in 
the datafile. The datafile will be merged with the rest of the student data and stored on the 
iRT server. The online survey collector will be cleared and the temporary linking listed 
destroyed after the data are downloaded and verified. At this point, the temporary ID 
numbers are no longer linked to any data. 
5. Additional possible risks associated with this change in study implementation 
a. Parents (or others) may be around during questionnaire completion and could potentially 
see a student’s answers on the questionnaire.  
b. Emails that contain the temporary ID numbers could be breached.  
6. What steps will be implemented to minimize possible risk during study implementation? 
a. The email to parents will ask parents to respect the privacy of students when they are 
completing the online questionnaire. The email to students will ask students to only 
complete the questionnaire if they feel comfortable with their level of privacy. Students will 
be reminded that participation in the study is voluntary; they may choose to not answer any 
question they don’t want to answer; and that they may withdraw from the study at any time 
with no penalty. 
b. The linking list of temporary and original ID numbers will only be kept at iRT. Furthermore, 
once the data are downloaded, the temporary ID numbers will be replaced with the original 
ID number in the file. The online survey collector will be cleared and the temporary linking 
listed destroyed after the data are downloaded and verified. At this point, the temporary 
unique ID numbers are no longer linked to any data.