Interventions to Improve Reproductive Health Among Adolescents
Phase: NA
Conditions: Teacher Training
Sponsor: RAND

1 
 
Identifiers: 
NCT04227236    
 
Unique Protocol ID: 1R41HD094406 
 
Brief Title: 
Interventions to Improve Reproductive Health Among Adolescents 
 
Document date: October 30, 2023 
 

2 
 
STUDY PROTOCOL 
 
To address the need for a facilitator training that was cost-effective, easily accessible, and 
delivers the MPC content with the same effectiveness as the currently available in-person 
training, we developed and tested a prototype of an online simulator-based training for 
facilitators of MPC using a mixed method approach of survey assessments and teacher 
observations. Using a randomized controlled trial for the pilot test of the prototype, we randomly 
assigned a convenience sample of 53 adults to receive either online simulation training or in-
person training for one activity within the MPC program. Using non-inferiority analysis (Schumi 
& Wittes, 2011), a statistical technique that specifically tests for similarity between groups, we 
hypothesized that participants who received the online simulation training would perform no 
worse than the in-person group on knowledge, self-efficacy, fidelity and quality of instruction 
via self-report survey and teacher observations. We also examined the self-reported 
acceptability, usability/applicability, likability and overall impressions among the online 
simulation group. 
 
Making Proud Choices Prototype 
To assess feasibility of using a simulated training environment to train educators to address the 
challenges of a skills-based sexual health curriculum such as MPC, the prototype focused on the content 
from one activity, the “Condom Line-Up”, which is one component of the “Developing Condom Use and 
Negotiation Skills” module. According to the developer, this activity is highly interactive, relying on the 
facilitator’s skill in processing student comments in a sensitive and constructive manner, as well as 
sensitive in nature (i.e., focusing on the use of condoms for adolescents). These qualities made the activity 
a good choice for this “proof of concept” phase. The in-person version of the training lasted 
approximately 25 minutes. The online version was comparable in length and included three elements: a) 
Didactic training: facing the learner, a master trainer avatar provides the rationale, learning objectives and 
goals for the activity; b) demonstration which entailed a virtual facilitator teaching the topic and virtual 
youth having typical reactions and responses, in a side-view of a classroom; and c) simulation teach-back: 
the view switches to the point of view of the learner, with the learner playing the role of the facilitator. 
For instance, a scene starts in which the virtual facilitator begins to teach to the virtual youth. Throughout 
the module, the scene pauses (at approximately 20 spots), and the learner is given at least 4 options on 
what action to take next. Although a best option exists, all options have some elements of better and 
worse responses. After the learner chooses an option, written text appears providing feedback about the 
option chosen. Learners are encouraged to go back and see how the class reacts if they had chosen other 
options as a way to maximize learning. Compared to many traditional online trainings that are unable to 
simulate the true give-and-take of a classroom environment, this innovative technology allows for the 
closest approximation to the feedback received from in-person training. Further, this approach can better 
reinforce information than in-person training in which there is no opportunity to witness the myriad of 
youth reactions.  
 
Participants 
 
In 2017, participants (those who represent potential MPC facilitators) were recruited to be a part 
of the pilot test using a combination of flyers posted at local universities and online advertisements (e.g., 
Craigslist, Facebook). Two-hundred forty-seven individuals expressed interest in the study, 93 of whom 
were excluded because they were younger than 21 years of age or had prior experience with MPC or 
providing other STI or pregnancy prevention education. Out of the 154 eligible individuals, the research 
team reached 123, and using a random number generator, randomly assigned 62 individuals who agreed 
to participate to either the online simulation (31 participants) or in-person (31 participants) training. 
Ultimately, 53 attended the day of pilot test (31 online simulation and 22 in-person). All participants 

3 
 
received one hundred dollars for participation in the pilot study. All participants provided written 
informed consent. All study procedures were reviewed and approved by RAND Corporation’s 
Institutional Review Board (IRB).  
 
Training and Fidelity Observations 
 
All trainings occurred at the same facility, run by staff with extensive experience training MPC 
facilitators from a variety of backgrounds. Both the simulation-based and in-person groups were trained 
on the same Condom Line-Up activity. Participants in the simulation-based training group completed the 
training individually at their own pace as they would in real-world practice (average time = 31 minutes). 
The self-paced approach allowed participants to take as much time as they preferred to review and 
practice the lessons, as well as an opportunity to complete tasks multiple times if desired. Participants in 
the in-person training group received trainings led by a trainer with extensive experience training MPC 
facilitators. In-person trainings were offered to groups of 4-8 participants each; these were 20-minute 
sessions with an additional 5-minutes for Q&A following the training. The in-person approach allowed 
participants the opportunity to hear other participants’ questions and the trainer’s responses, providing the 
opportunity to think about questions that they may otherwise not have considered. All participants 
completed pre- and post-training surveys. Following the post-survey, each participant completed a 
demonstration teaching where they conducted the Condom Line-Up activity from start to finish to a group 
of staff trained to emulate teens and provide scripted responses during the session. Each participant was 
videotaped for the demonstration teaching.  
 
Measures 
Pre- and post-training surveys assessed knowledge, self-efficacy, facilitator skills, and beliefs and 
impressions related to the use of the online prototype. Observation measures focused on fidelity to 
curriculum for the Condom Line-Up activity and quality of the instruction. 
Demographics  
Participants provided demographic information on race/ethnicity, age, gender, education, and 
experience working with youth and teaching HIV/STI programs.  
Facilitator Knowledge 
Knowledge of the curriculum (objective and rationale of the Condom Line-Up) and knowledge of 
the steps of proper condom use was assessed using three items informed by the MPC curriculum (e.g., To 
the best of your ability, please select the rationale for engaging students in an activity that shows them the 
proper steps for using condoms safely.). Responses were coded as correct or incorrect. We examined each 
knowledge item individually in the analyses. 
Facilitator Self-efficacy 
Self-efficacy of teaching sexual health education was assessed using three questions (e.g., How 
confident are you that you can teach teens how to correctly use condoms?) rated from 1 (not at all) to 7 
(extremely). We created a self-efficacy scale score by averaging the scores of the three items for each 
participant ( = .93 for pre-test and  = .94 for post-test). 
Facilitator Skills 
Participants responded to four open ended questions assessing facilitator skills related to handling 
sensitive questions, creating a safe and inclusive environment, engaging students, and handling 
whispering during a presentation on proper condom use. For example, participants responded to the 
following scenario: [You are preparing to teach a class of high school students about safe condom use 
strategies. A student raises their hand and says "Why do we all have to do this? Some people here believe 
in waiting till marriage to have sex." Another student then shouts out "Virgin!" and the class laughs. What 
do you do?] The purpose of these open-ended questions was to present participants with typical scenarios, 
ask for an open-ended answer (as opposed to choosing from a list of possible responses), and then rate 
their answer. Two independent coders (blinded to condition), who were trained by a senior researcher on 
the team, rated the videotaped responses as 2 for "done correctly," 1 for "done partially correct" and 0 for 
“not done correctly.” The intraclass correlation coefficient (Cicchetti, 1994) was .76 for the pre-survey 

4 
 
responses and .79 for post-survey responses. 
 
Beliefs and Impressions of the Prototype 
Participants from the online simulation group also responded to questions assessing their 
perceived acceptability, applicability, liking, and overall impressions of the training. In addition to 
assessing the impact of the simulated training environment on learning the material, it is important to 
determine what potential teachers think about the simulation as this would likely influence its adoption 
into schools (Khlaif, 2018). To assess acceptability, participants rated 19 items on a scale of 1 (strongly 
disagree) to 7 (strongly agree) from the Post-Study System Usability Questionnaire (PSSUQ) (Lewis, 
1995), a well-known, valid and reliable ( =.91 or higher) measure that assesses perceptions of the 
technical aspects of computer-based systems and programs (e.g., system usefulness, information quality, 
and interface quality). To assess usability and applicability to teaching youth about sexual health and 
safety, participants completed the Moore and Benbasat (1991) measure of factors that predict adoption of 
a new technology, specifically, relative advantage (e.g., “Using the online training would make my job 
easier.”), compatibility (e.g., “The online training fits into my work”), ease of use, and result 
demonstrability (e.g., “I could talk to others about the consequences of using the online training”). 
Wording was adapted to be relevant to the online simulation training. Items were rated on a scale ranging 
from 1 (strongly disagree) to 7 (strongly agree) ( = .96). Participants rated likability with one item: [On 
a scale of 1 to 7, with 1 being disliked it very much and 7 being liked it very much, how much did you 
like the appearance of the Condom Line-Up e-training overall—the colors, the pictures, the buttons, etc.?]  
Finally, participants’ overall impressions of the training were assessed using a 4-item measure, 
rated on a 7-point Likert-scale (very difficult to use to very easy to use, very boring to very interesting, 
very amateurish to very professional, and very basic to very informative). Items were averaged to create 
an overall impression score (Cronbach alpha = .90) 
 
Fidelity and Quality of Instruction 
 
Each of the nine components from the Condom Line-Up were rated independently by two coders 
from the videotapes as 2 for "done correctly," 1 for "done partially correct" and 0 for “not done 
correctly.” Coders received training on coding rules from a senior researcher on the team and one of the 
MPC training developers. Coders practiced the coding scheme on a subset of subjects in a group format 
where coders rated alongside a senior researcher on the team. Coder disagreement (>1-point difference) 
was discussed with the measure developer to reach a consensus. After training, two coders rated sub-
activities of all the remaining 45 subjects. Inter-rater reliability using intra-class correlations (ICC) 
(Cicchetti, 1994) ranged from .78 to .98, all within the good-excellent range. Final scores were calculated 
as the average of the two coders’ ratings.  
 
STATISTICAL ANALYSIS PLAN (SAP) 
First, we performed χ2 tests and t tests to assess whether the two groups differed on demographic 
and background variables, and pre-test scores on knowledge, self-efficacy and facilitator skills. Second, 
we examined the non-inferiority of online simulation to in-person by comparing two groups in post-
training knowledge, pre-post change in self-efficacy and facilitator skills, and post-training scores of 
fidelity and quality of instruction. Non-inferiority analyses assess whether a new treatment or approach 
(e.g., online simulation) is acceptably efficacious as an existing approach (i.e., in-person training) 
(Althunian et al., 2017). Or, in other words, the new approach or treatment is not less effective than the 
existing approach by more than a small pre-specified amount. This amount is known as the non-inferiority 
margin.  
To test non-inferiority of the online simulation training, we constructed a two-sided 95% 
confidence interval (CI) for the difference between online simulation and in-person (computed as in-
person minus online simulation) for: (a) change in percent correct for knowledge of the curriculum 
(objective and rationale of the Condom Line-Up) and knowledge of the steps of proper condom use, both 
computed as post-test minus pre-test, using the method described in Farrington and Manning (1990), (b) 

5 
 
change in self-efficacy and facilitator skills (computed as post-test minus pre-test) using t-test statistic, 
and (c) scores of fidelity and quality of instruction based on the role-play activity after the trainings using 
the t-test statistic (Table 5). Non-inferiority of online simulation to in-person (meaning the online 
simulation group was no worse than the in-person group) could be claimed if the upper bound of the CI of 
the difference was less than the margin of non-inferiority. Based on standard practice (e.g., Schoneveld et 
al., 2018) and consensus among the developers of the MPC training on an acceptable difference between 
in-person and online simulation scores, the non-inferiority margin was set at 5% for percent correct of 
knowledge items, 0.5 SD of the in-person change in self-efficacy and facilitator skills, and 0.5 SD of the 
in-person scores of fidelity and quality of instruction. All tests were performed in SAS 9.4. (SAS 
Institute, 2013) 
 
 
 
 

6 
 
INFORMED CONSENT FORM (ICF): E-TRAINING AND SURVEY 
 
What is the purpose of the study? 
Thank you for your consideration to participate in our research study. dfusion, a small woman-run 
business in the San Francisco Bay Area, and the RAND Corporation, a non-profit research organization 
based in Santa Monica, CA are developing a prototype of an online, avatar-based virtual training for 
facilitators of the program Making Proud Choices (MPC), an evidence-based, safe-sex approach to teen 
pregnancy and HIV/STD prevention. Your participation in this study will help inform the development of 
this training for facilitators of Making Proud Choices. 
 
What will I be asked to do? 
As part of the study, you will complete the online training as well as two surveys (one before and one 
after the training) and a performance observation. The training will focus on one module of the larger 
MPC program. The surveys and the training will be completed at your own pace on a computer. The 
surveys will ask about your experience using the training, as well as your knowledge and attitudes related 
to sexual health education. For the performance observation, you will be asked to role play one of the 
skills learned in the training session in front of two of our staff while being videotaped. The videotape 
allows us to review your role playing after the session is complete. The study session including the 
training, surveys and performance observation will take approximately 2.5 hours. 
 
What are my rights during the study? 
Participation in the study is voluntary. You may skip any question on the surveys and the role playing and 
may leave the study at any time without consequence. 
 
Will I receive anything for completing the survey? 
As a thank you for your time, you will receive a $100 gift card, which will be distributed at the end of the 
2.5-hour study session.  
 
How will my privacy be protected? 
Although we have your contact information from the study recruitment period, your contact information 
will not be connected to your survey responses or training sessions. All data collected as part of your 
study participation will be kept confidential. The risk of identification from your data is no greater than 
usual daily activity. Files containing your personal contact information will be password protected and 
only accessible by project staff to protect your confidentiality. 
 
What if I have questions about the study? 
You have the right to ask, and have answered, any questions about the study. If you have questions at any 
time you should contact Regina Firpo-Triplett, MPH, at dfusion at regina.firpo@dfusioninc.com or Matt 
Chinman, Ph.D. at the RAND Corporation at chinman@rand.org. 
 
What if I have questions about my rights as a study participant? 
All research with human volunteers is reviewed by a committee that works to protect your rights and 
welfare. If you have questions or concerns about your rights as a survey participant you may contact, 
anonymously, if you wish, James Tebow, Ph.D., at the RAND Human Subjects Protection Committee: 
310-393-0411 x4173 or tebow@rand.org. 
 
BY SIGNING THIS FORM, I WILLINGLY AGREE TO PARTICIPATE IN THE RESEARCH IT 
DESCRIBES.  
_________________________________________ 
Name of Research Participant (Print) 
_________________________________________ 
Signature of Research Subject 
            Date 
 
 
 

7 
 
INFORMED CONSENT FORM (ICF): IN-PERSON TRAINING AND SURVEY 
 
What is the purpose of the study? 
Thank you for your consideration to participate in our research study. dfusion, a small woman-run 
business in the San Francisco Bay Area, and the RAND Corporation, a non-profit research organization 
based in Santa Monica, CA are developing a prototype of an online, avatar-based virtual training for 
facilitators of the program Making Proud Choices (MPC), an evidence-based, safe-sex approach to teen 
pregnancy and HIV/STD prevention. Your participation in this study will help inform the development of 
this training for facilitators of Making Proud Choices. 
 
What will I be asked to do? 
As part of the study, you will complete the an in-person training as well as two surveys (one before and 
one after the training) and a performance observation. The training will focus on one module of the larger 
MPC program. The surveys will be completed at your own pace on a computer and the trainings will be 
done in a group setting. The surveys will ask about your knowledge and attitudes related to sexual health 
education. For the performance observation, you will be asked to role play one of the skills learned in the 
training session in front of two of our staff while being videotaped. The videotape allows us to review 
your role playing after the session is complete. The study session including the training, surveys and 
performance observation will take approximately 2.5 hours. 
 
What are my rights during the study? 
Participation in the study is voluntary. You may skip any question on the surveys and the role playing and 
may leave the study at any time without consequence. 
 
Will I receive anything for completing the survey? 
As a thank you for your time, you will receive a $100 gift card, which will be distributed at the end of the 
2.5-hour study session.  
 
How will my privacy be protected? 
Although we have your contact information from the study recruitment period, your contact information 
will not be connected to your survey responses or training sessions. All data collected as part of your 
study participation will be kept confidential. The risk of identification from your data is no greater than 
usual daily activity. Files containing your personal contact information will be password protected and 
only accessible by project staff to protect your confidentiality. 
 
What if I have questions about the study? 
You have the right to ask, and have answered, any questions about the study. If you have questions at any 
time you should contact Regina Firpo-Triplett, MPH, at dfusion at regina.firpo@dfusioninc.com or Matt 
Chinman, Ph.D. at the RAND Corporation at chinman@rand.org. 
 
What if I have questions about my rights as a study participant? 
All research with human volunteers is reviewed by a committee that works to protect your rights and 
welfare. If you have questions or concerns about your rights as a survey participant you may contact, 
anonymously, if you wish, James Tebow, Ph.D., at the RAND Human Subjects Protection Committee: 
310-393-0411 x4173 or tebow@rand.org. 
 
BY SIGNING THIS FORM, I WILLINGLY AGREE TO PARTICIPATE IN THE RESEARCH IT 
DESCRIBES.  
______________________________________ 
Name of Research Participant (Print) 
_________________________________________ 
Signature of Research Subject 
            Date 
 
 
 

8 
 
INFORMED CONSENT FORM (ICF): E-TRAINING FOCUS GROUPS 
 
What is the purpose of the study? 
Thank you for your consideration to participate in our focus group. dfusion, a small woman-run business 
in the San Francisco Bay Area, and the RAND Corporation, a non-profit research organization based in 
Santa Monica, CA are conducting this focus group to learn more about your experience using the online 
avatar-based facilitator training for Making Proud Choices. 
 
What will I be asked to do? 
We will be asking the group questions about your experience using the online facilitator training, 
including questions about how easy or difficult it was to use the training, what features you liked and 
didn’t like, and your opinion of the training overall. The focus group session is expected to last 
approximately 90 minutes. 
 
What are my rights during the survey? 
Your participation is your choice, and you can stop or leave at any time without any negative 
consequence. We will ask the group a lot of questions, but you do not have to answer them if you do not 
want to. 
 
Will I receive anything for completing the survey? 
As a thank you for your time, you will receive a $50 gift card for participating in the focus group, which 
will be distributed at the end of the 90-minute session.  
 
How will my privacy be protected? 
Many of the questions ask about your personal experience with the training, but because this is a group 
discussion, please don’t say anything you wouldn't want others to know and talk about. While our study 
team will keep what you say confidential—we are not collecting your names and will not share 
information about you with anyone—we cannot promise you that others in the group will do the same. 
 
What if I have questions about the study? 
You have the right to ask, and have answered, any questions about the study. If you have questions at any 
time you should contact Regina Firpo-Triplett, MPH., at dfusion at regina.firpo@dfusioninc.com or Matt 
Chinman, Ph.D., at the RAND Corporation at chinman@rand.org. 
 
What if I have questions about my rights as a study participant? 
All research with human volunteers is reviewed by a committee that works to protect your rights and 
welfare. If you have questions or concerns about your rights as a survey participant you may contact, 
anonymously, if you wish, James Tebow, Ph.D., at the RAND Human Subjects Protection Committee: 
310-393-0411 x4173 or tebow@rand.org. 
 
 
BY SIGNING THIS FORM, I WILLINGLY AGREE TO PARTICIPATE IN THE RESEARCH IT 
DESCRIBES.  
_________________________________________ 
Name of Research Participant (Print) 
_________________________________________ 
Signature of Research Subject 
            Date