metadata: authors: 0: author: Chong,Alberto
1: author: Gonzalez-Navarro,Marco
2: author: Karlan,Dean S.
3: author: Valdivia,Martin
count: Colombia
docna: 0: docna: Do Information Technologies Improve
            Teenagers' Sexual Education ? Evidence from a
            Randomized Evaluation in Colombia
docty: Policy Research Working Paper
subtopic: Educational Sciences,Health Care Services Industry,Disease Control & Prevention,Social Cohesion
teratopic: Education,Industry,Health, Nutrition and Population,Social Development
abstracts: cdata!: This study reports results from a
            randomized evaluation of a mandatory six-month
            Internet-based sexual education course implemented across
            public junior high schools in 21 Colombian cities. Six
            months after finishing the course, the study finds a 0.4
            standard deviation improvement in knowledge, a 0.2 standard
            deviation improvement in attitudes, and a 55 percent
            increase in the likelihood of redeeming vouchers for condoms
            as a result of taking the course. The data provide no
            evidence of spillovers to control classrooms within
            treatment schools, and it finds that treatment effects are
            enhanced when a larger share of a student's friends
            also takes the course. The low cost of the online course
            along with the effectiveness the study documents suggests
            this technology is a viable alternative for improving sexual
            education in middle-income countries.
geo_reg_key: 517191,119228
repnme: repnme: Do Information Technologies Improve
            Teenagers' Sexual Education ? Evidence from a
            Randomized Evaluation in Colombia
display_title: Do Information Technologies Improve
            Teenagers' Sexual Education Evidence from a Randomized
            Evaluation in Colombia
listing_relative_url: /research/2019/04/30990271/information-technologies-improve-teenagers-sexual-education-evidence-randomized-evaluation-colombia
topicv3: Rape and Sexual Violence m1326262 2443,Education m1327048 470,Health m1504661 3480,Condoms m1504697 3617,Condom Use m1504745 3536
publishtoextweb_dt: 2019-04-17T00:00:00Z
historic_topic: Education,Industry,Health, Nutrition and Population,Social Development
no_of_pages: 29
author_keys: 0: author_key: 659793
1: author_key: 959480
model_extractions: Policy Research Working Paper 8821 Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia _Alberto Chong Marco Gonzalez-Navarro Dean Karlan Martín Valdivia_ Development Economics Vice Presidency Strategy and Operations Team April 2019
Policy Research Working Paper 8821

# **Abstract**

This study reports results from a randomized evaluation of a mandatory six-month Internet-based sexual education course implemented across public junior high schools in 21 Colombian cities. Six months after finishing the course, the study finds a 0.4 standard deviation improvement in knowledge, a 0.2 standard deviation improvement in attitudes, and a 55 percent increase in the likelihood of redeeming vouchers for condoms as a result of taking the course. The data provide no evidence of spillovers to control classrooms within treatment schools, and it finds that treatment effects are enhanced when a larger share of a student’s friends also takes the course. The low cost of the online course along with the effectiveness the study documents suggests this technology is a viable alternative for improving sexual education in middle-income countries..

This paper is a product of the Strategy and Operations Team, Development Economics Vice Presidency. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at achong6@gsu.edu.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia**

## **Alberto Chong, Marco Gonzalez-Navarro, Dean Karlan, and Martín Valdivia**

**JEL classification:** O12, I2, I1

**Keywords:** information technologies, Internet, sex education, teenagers, field experiment

### **1. Introduction**

Providing effective sexual education to teenagers is a pervasive worldwide policy challenge. In many countries, conservative norms lead to restricted sexual education curricula. Deficient sexual education partially explains the high levels of sexually transmitted diseases and teenage pregnancies that are observed in many of the world’s developing countries (WHO 2004). Making matters more consequential for youth, in poor countries there is an acute lack of resources, health system capabilities, and best practices to treat sexually-transmitted diseases (Fortson 2009).

Alberto Chong (corresponding author) is a professor at Georgia State University (email: achong6@gsu.edu) and Universidad del Pacifico; Marco Gonzalez-Navarro is an assistant professor at University of California, Berkeley (email: marcog@berkeley.edu); Dean Karlan is a professor at Northwestern University (email: karlan@northwestern.edu); Martín Valdivia is a senior researcher at Group for the Analysis of Development (GRADE–Peru). (email: jvaldivi@grade.org.pe). This research would not have been possible without the sustained support of the Profamilia staff in charge of implementing and monitoring the intervention, especially German López and Lyda Díaz. The authors also recognize valuable research assistance from Angela García, César Mora, Juan Pablo Ocampo, Martin Sweeney, and project leadership by Beniamino Savonitto and Rachel Strohm. All errors and omissions are ours. The authors thank the Interamerican Development Bank for funding. All opinions are those of the authors, and not of the participating organizations or donors. The authors retained full intellectual freedom to report the results throughout the study. The study received Institutional Review Board (IRB) approval from Innovations for Poverty Action (IPA) under protocol 117.09June-003.
Chong et al.

2 The objective of this research is to test whether, in a predominantly Catholic, middle-income country, information technologies in a school setting can help overcome sexual education–related informational barriers faced by teenagers. Naturally, evaluations of sexual health curricula have been done before. Review papers by Kirby, Laris, and Rolleri (2007); Chin et al. (2012); Fonner et al. (2014), and Goesling et al. (2014) conclude that most comprehensive sexual education programs that have been evaluated rigorously are effective at improving knowledge, attitudes, and self-reported behaviors. Fonner et al. (2014) in particular focus on poor and middle-income country studies and reach the same conclusion. However, this large literature focuses on facilitator-led interventions, which implies that it is difficult to ensure consistent delivery in face-to-face interventions when scaling up. Furthermore, many teachers block this type of education because of dis- comfort discussing sex-related decisions with teens. As a result of these factors, elaborate interventions struggle to translate encouraging results from controlled trials into larger settings (Collins et al. 2002). Information and communication technologies (ICTs) for sexual education hence hold promise to im- prove school-based sexual education along three dimensions: reducing loss of effectiveness when scaling up, reducing costs of implementation, and overcoming educator reluctance to present sexual education material.

The study implements a large randomized evaluation of a comprehensive Internet-based sexual health education course geared to adolescents in Colombian public schools. Assignment to treatment is randomized both across 69 schools as well as within schools at the classroom level (in order to measure information spillovers within the school). The course covers topics that range from sexual rights to the use of contraceptives. It was implemented during a full academic semester in close collaboration with public schools as part of the students’ obligatory curricula.

The sample consisted of 4,599 students enrolled in 138 ninth-grade classrooms from 69 public schools spread across 21 major Colombian cities. The control group received the status quo: brief biology class coverage or sporadic visits by health personnel.<sup>1</sup> To measure differences in outcomes after treatment, the study uses three sources of data: a follow-up survey one week after course completion to measure short-term changes in knowledge and attitudes, a second follow-up survey six months after the course to measure the same outcomes in the longer term; and redemption from local health clinics of a voucher for condoms.

The main focus of the study is on knowledge and attitude indicators since these are the main outcomes of interest in the literature studying young adolescents who, for the most part, have not had sex. Furthermore, these two factors have been shown to be the strongest protective factors in preventing sexually transmitted infections (STIs), hu- man immunodeficiency virus (HIV), and pregnancy among teens (Kirby, Lepore, and Ryan 2007). Recent research has also documented the important role that social norms play in responsible sexual behavior (Munshi and Myaux 2005; Ashraf, Field, and Lee 2014). By changing knowledge and attitudes in youth attending school, sexual education can ultimately play a fundamental role in achieving desirable aggregate changes in sexual behavior.

The condom voucher makes it possible to avoid obvious problems with self-reported sexual behavior data, and follows O’Donnell et al. (1995) and Thornton (2008). Increased condom demand and distribution is a common policy target and tool as part of efforts to reduce sexually transmitted infections and teenage pregnancy. The researchers view this as a key part of the design of the study, given the concern in this literature regarding the relationship between knowledge and attitude outcomes and actual behavior (Ross et al. 2007), and the challenge of gathering accurate responses to sensitive questions on sexual practices.

1 Control schools were asked not to modify their current sexual education because of their participation in the study.
Assignment to the course causes a 0.38 standard deviation improvement in an index of sexual knowledge. The effects are practically identical at completion of the course and six months later, suggesting little decay in knowledge. It also generates a 0.24 improvement in an index of sexual attitudes at course completion, and a 0.17 standard deviation improvement after six months. These aggregate improvements in sexual knowledge and attitudes suggest strong effects over time and across almost all of the underlying subindices.<sup>2</sup> The sexual knowledge and attitude results are corroborated by the redemption rate of condom vouchers. The data show that 28 percent of treatment students redeem condom vouchers, compared to 18 percent of control students, representing a 55 percent increase in redemption. The data do not provide evidence that that the online course increased sexual activity, which together with the increased condom demand result suggests that the course results in improved sexual practice or at least safe-sex preparedness.

The results do not provide any indication of classroom-level spillovers within the same school, which suggests for policy purposes that programs should target entire cohorts, not a subset of classrooms with expectations of spillovers leading indirectly to treatment. The study does, however, find evidence of a reinforcement effect through social networks. Treatment effects are greatest when a large proportion of one’s friends were also treated (as discussed in Manski [2013]).<sup>3</sup> From a policy perspective, an Internet-based course has the major advantage of being low cost compared to human led interventions. In this intervention $1,000 is sufficient to provide a course to 68 students for a whole semester.<sup>4</sup> It is straightforward to use the estimate of change in demand for condoms due to the course to forecast that $1,000 would generate a reduction of 2.2 STIs among the treated students. Given that averting a sexually transmitted disease is estimated in the literature to be valued at $785 this implies a benefit-to-cost ratio of 1.72. Thus, the results point toward substantial social benefits from this course as a method to improve sexual knowledge, attitudes, and future sexual behavior at a relatively low cost.

There is an emerging literature studying the efficacy of computer-based forms of education in developed countries. Noar, Black, and Pierce (2009) review 12 published and unpublished studies using ICTs in sexual education among young adults (average age 22) in the United States (and one study in the Netherlands) and find a positive average effect, albeit with more null results than positive in the individual studies. This study builds on this nascent literature by implementing a large-scale program through the public high school system in Colombia, including analysis of knowledge and attitude spillovers through social networks, and administrative data (through redemption of condom vouchers) rather than self-reported data on condom use.

This research also relates to recent evidence suggesting that computers on their own do not change academic outcomes in a discernible manner (Angrist and Lavy 2002; Krueger and Rouse 2004; BarreraOsorio, and Linden 2009; Fairlie and Robinson 2013). It complements this literature by showing that, once computers have been installed in schools, structured Internet courses—at least for sexual education— can have significant effects.

> 2 The sexual knowledge index is composed of the following subindices: symptoms and causes of STIs, sexual violence, prevention of STIs, pregnancy prevention, and condom use. The sexual attitudes index is composed of the following subindices: condom use, sexual conservativeness and sexual abuse reporting.

> 3 These results complement recent literature such as Fletcher (2007), Richards-Shubick (2015), and Card and Giuliano (2013), who find that peer group norms have a first-order effect in explaining sexual health outcomes.

- 4 All figures in 2012 U.S.  dollars.
Chong et al.

4

# **2. Profamilia’s Internet-Based Sex Education Course**

The online sexual education course was designed by the local NGO Profamilia.<sup>5</sup> Profamilia is Colombia’s largest organization focused on sexual health and reproductive health. With more than 40 years of presence and over 1,800 employees nationwide, Profamilia is well known and used by the local population for sexual health products and services such as contraceptives, HIV testing, and pregnancy tests.<sup>6</sup> Motivated by the stubbornly high level of some important adolescent sexual health indicators nationwide, such as teenage pregnancy rates (Demographic and Health Survey 2005), as well as legal changes, which mandated the introduction of a sexual health curriculum in Colombian public schools, Profamilia embarked on the design of a comprehensive online sexual education course designed for adolescents.

The curriculum aims to shape adolescents’ understanding and perceptions of sexuality, risks, reproductive health, sexual rights, and dating violence. The overarching theme is a human rights approach to pregnancy and teen sexuality. The course focuses on helping the students recognize themselves as endowed with rights, such as the right to say no to sex, to access basic health services, to access family planning services, and to live without sexual violence.<sup>7</sup> Profamilia’s course takes full advantage of Internet connectivity to provide an interactive experience and responsive, anonymous counseling. The modules can be potentially accessed any time of day using a password-protected account, and there is a remote tutor available to answer questions via messages and support the learning process. These features aim to create a safe social environment for adolescents to discuss sensitive topics.

Treatment consisted of five modules. Students worked on the course for a total of 11 weeks. Each group of treated students was initially given three weeks to become acquainted with the platform and complete activities in the first module. After the first three weeks, each group was given two weeks per module to complete activities in the remaining four modules. Each school dedicated one session of 1.5 hours per week to allow the students to complete the course in the school’s computer labs.

In school, each group taking the course worked with the presence of a teacher, who was tasked with helping the students resolve questions about use of and access to the platform, but not questions related to the content of the course. Students were assisted and monitored by an online tutor, who was a trained Profamilia counselor who dedicated part of his or her day to overseeing students during their completion of the course. The tutors had two main roles: answering students’ questions about the course contents and monitoring the students’ performance.<sup>8</sup> At the end of every module, the tutor provided the teacher responsible for the group with a grade for each student, based on the results of a test. To incentivize course completion, each school participating in the course included these grades as a component of the grade of one school subject, typically computer education. Each student had to complete module evaluations individually, which were the basis for his or her individual performance report. Participation in the course was mandatory for students.

# **3. Experimental Design and Estimation Strategies**

## Sample Selection

In Colombia, 13.5 percent of adolescents become sexually active by age 15, and 60 percent have sex before age 18.<sup>9</sup> Profamilia’s course targets 14–15 year olds precisely because they are close to

- 5 The organization’s website can be accessed at www.profamilia.org.co.

- 6 See Miller (2010) for a study of long-term effects of Profamilia family planning services in  Colombia.

- 7 Examples from the course modules can be accessed at www.profamiliaeduca.com/profamilia/index.php.

- 8 Records of the interactions between tutors and students are not preserved and hence were not available for analysis.

- 9 Sexual education courses must ideally be targeted at children of the appropriate age to benefit from them. Very young children may not yet be interested in sexuality issues, which points towards the benefits of targeting an older age range.
becoming sexually active. The sample frame for the study consists of ninth-grade students in Colombian urban public secondary schools. Given the study’s interest in cross-classroom spillovers, it required enrolled schools to have at least two ninth-grade classes. Schools were also required to have at least one computer room with Internet access.<sup>10</sup> All participating administrators of the schools had to consent for their school to participate in the field experiment before knowing the results of the randomization. Schools agreed to facilitate data collection and coordination, to make a computer lab available for the prescribed time every week (if selected to implement the course), and to not substantially modify their sexual and reproductive health education for ninth-graders during the study. A short questionnaire for school principals at baseline revealed that sexual education in the sample was either nonexistent, a topic covered in biology class, or consisted of one or two visits per year by a health professional. Schools in the control group received a sports equipment package as compensation at the end of the study.

The sample consists of 69 public secondary schools recruited in 21 cities across Colombia.<sup>11</sup> From each school, two classrooms of ninth-graders were selected to participate in the study. If the school had more than two classrooms of ninth-graders, a pair was randomly selected by the researchers to partake in the study.

# Data Collection Strategy

Data collection consisted of three rounds of a self-administered in-school pencil and paper survey. These surveys were administered unannounced during class time, so as to minimize data collection costs and attrition. The pencil and paper survey strategy was chosen instead of a computer survey because treatment students would have more familiarity with computers at endline, hence unnecessarily biasing results. The baseline survey was fielded at the beginning of the academic year; the second one at the end of the semester (after taking the course), and the last survey was taken at the end of the academic year, that is, six months after the end of the course. The academic year bracketed the timeline for the final survey data collection since beyond that point there is substantial student attrition due to students switching schools, and more fun- damentally, groups get reshuffled going into the 10<sup>th</sup> grade. This meant that to interview the participants during the 10<sup>th</sup> grade the researchers would have had to either locate the students in their new groups for re-interviews or survey the whole cohort to locate the participants’ responses expost. The first was deemed unfeasible by school administrators while the latter option was beyond the budget for the study. Six months after the end of the course, students were offered a voucher for six condoms with a total market value of about US$5. The offer was made via an email for all students and additionally via an SMS message for those who provided a cell phone number (86 percent of the sample). Data on which students redeemed their vouchers at the local health clinic were obtained by matching the voucher number to the student to whom the voucher was sent. A timeline of the intervention and data collection strategy is presented in fig. 1.

On the other hand, sexual education should in principle be provided before sexual initiation to convey its full benefits. In the United States 15 percent of adolescents have sex before age 15 (Flanigan et al. 2006).

> 10 The study selected schools with a functioning computer lab connected to the Internet with at least one computer for every three students. On average schools had 38 computers with a ratio of approximately 1 participating student per computer.

- 11 The sample excludes rural public schools. In urban settings, it is common for schools in Colombia to have two shifts per day (morning and afternoon). A student is offered a place at a certain shift before the beginning of the school year, and once a school is selected, he/she cannot take classes in other shifts or switch shifts. Given the lack of interaction among children of different shifts, the study treats different shifts in the sample as different schools. Both shifts are used for 13 schools in the sample.
<!-- Start of picture text --> School<br>year End of End of<br>begins semester school year<br>|<br>———————E<br>po<br>Treatment Baseline |ntervention: Online Follow-up Follow-upCondom — condom<br>assignment SUrVey sexualeducationcourse SUrVeY! surveyll Vouchers — vouchers<br>(11 weeks) sentoutviaemailorcell expire(2<br>phone months)<br><!-- End of picture text -->
**Table 2.** Baseline Summary Statistics and Balance

||**Treatment**<br>**students**<br>**(1)**|**Spillover**<br>**students**<br>**(2)**|**Control**<br>**students**<br>**(3)**|**Difference**<br>**(1–3)**|**Difference**<br>**(2–3)**|
|---|---|---|---|---|---|
|**Panel A: Variables available at random assignment**||||||
|School year begins in January (=1)|0.720|0.731|0.699|0.020|0.032|
||(0.01)|(0.01)|(0.01)|(0.12)|(0.12)|
|Single shift school (=1)|0.606|0.623|0.577|0.028|0.046|
||(0.01)|(0.01)|(0.01)|(0.13)|(0.13)|
|Morning shift (=1)|0.637|0.658|0.652|−0.016|0.006|
||(0.01)|(0.01)|(0.01)|(0.12)|(0.12)|
|City with more than 600,000 people (=1)|0.260|0.239|0.251|0.009|−0.011|
||(0.01)|(0.01)|(0.01)|(0.11)|(0.11)|
|Number of 9th grade classrooms in school|3.226|3.258|3.081|0.145|0.177|
||(0.03)|(0.03)|(0.03)|(0.32)|(0.32)|
|Average number of students in each classroom|37.257|37.330|38.296|−1.039|−0.965|
||(0.28)|(0.29)|(0.22)|(2.42)|(2.47)|
|Number of computers in school|37.669|38.246|35.909|1.761|2.337|
||(0.44)|(0.45)|(0.52)|(5.17)|(5.19)|
|School does not teach sexual education (=1)|0.168|0.167|0.135|0.033|0.032|
||(0.01)|(0.01)|(0.01)|(0.09)|(0.09)|
|_p_-value from_F_-test of joint significance on all above var|iables|||0.94|0.89|
|**Panel B: Baseline variables not available at random assi**|**gnment**|||||
|Male (=1)|0.414|0.402|0.490|−0.076|−0.088*|
||(0.01)|(0.01)|(0.01)|(0.05)|(0.05)|
|Not sexually active (=1)|0.617|0.587|0.590|0.026|−0.003|
||(0.01)|(0.01)|(0.01)|(0.04)|(0.04)|
|Age|14.935|15.020|14.977|−0.042|0.043|
||(0.03)|(0.03)|(0.03)|(0.11)|(0.12)|
|Mother’s years of education|12.706|12.641|12.584|0.121|0.056|
||(0.07)|(0.07)|(0.07)|(0.11)|(0.10)|
|Father’s years of education|12.672|12.579|12.503|0.169|0.076|
||(0.08)|(0.08)|(0.08)|(0.13)|(0.13)|
|Socioeconomic level [1, 6]|2.175|2.170|2.162|0.013|0.008|
||(0.03)|(0.03)|(0.03)|(0.13)|(0.13)|
|PC at home (=1)|0.323|0.305|0.326|−0.003|−0.021|
||(0.01)|(0.01)|(0.01)|(0.04)|(0.04)|
|Cellphone (=1)|0.742|0.737|0.716|0.026|0.022|
||(0.01)|(0.01)|(0.01)|(0.03)|(0.03)|
|Does not use Internet in school (=1)|0.447|0.512|0.482|−0.035|0.031|
||(0.01)|(0.01)|(0.01)|(0.09)|(0.09)|
|Does not use Internet (=1)|0.238|0.252|0.252|−0.014|0.000|
||(0.01)|(0.01)|(0.01)|(0.03)|(0.03)|
|Religion is important (=1)|0.619|0.601|0.618|0.001|−0.017|
||(0.01)|(0.01)|(0.01)|(0.03)|(0.03)|
|_p_-value from_F_-test of joint significance on all above var|iables|||0.79|0.62|

_Source_ : Authors’ analysis from student survey. _Note_ : Columns (1)–(3) report means, with standard errors in parentheses. For columns (4) and (5), each row is one regression of the characteristic on treatment and spillover indicator variables, with the coefficient (standard error, clustered at the school level) on treatment and spillover reported. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. Panel A variables were provided by the schools before the baseline survey took place. The study randomized treatment assignment repeatedly until no _t_ -test comparing treatment to control for any covariate was larger than 2.0. Variables in Panel B became available only after assignment to treatment. The last rows in panels A and B report the _p_ -value on an _F_ -test of joint significance for all variables in the panel from a regression where the dependent variable is a treatment dummy (Column 4) or spillover dummy (column 5). Column 4 excludes the spillover group from the analysis, while column 5 excludes the treatment group from the analysis.
Chong et al.

8

# Implementation

The sexual education course was implemented from August through November 2009 in schools that began their school year in January<sup>13</sup> and from November 2009 through March 2010 in schools that began their school year in September _._ As expected for a middle-income country, it was not difficult to recruit schools with computer labs. However, it proved more difficult to recruit schools with workable Internet connections. In fact, in 3 of the 46 groups assigned to treatment, lack of Internet access prevented implementation of the Internet-based course.<sup>14</sup> In some treatment groups, students were unable to complete all five modules due to unforeseen events such as teacher strikes. Grades on the tests at the end of each module were on average 4 out of 10, with a large mass at zero (48 percent). Excluding those students with a score of zero, the average was 8 out of 10, suggesting an acceptable degree of understanding for those actually taking the course and the tests. The high proportion of scores equal to zero highlights the challenges of student compliance associated with Internet-based education. Panel B in table 2 shows summary statistics by treatment condition. The average age is 15 years, 43 percent of the sample is male, and 32 percent have a computer at home.<sup>15</sup>

# Baseline Balance

Panels A and B of table 2 show there are no statistically significant differences across treatment, spillover and control groups except for gender: the control group has 7.6 percentage points and 8.8 percentage points more males than the treatment and spillover groups, respectively. Furthermore, an _F_ -test from a regression of treatment assignment on a full set of baseline characteristics does not reject the null hypothesis that all baseline coefficients are jointly equal to zero (reported in the final row of each panel).

# Attrition

Attrition was 13 percent between baseline and first follow-up, and 10 percent between baseline and second follow-up. Supplementary appendix table S1.2 shows there is no differential attrition between control and treatment, and control and spillover students. The study also analyzes attrition for the condom voucher offer. Because students had to provide a valid cell phone number and/or email in order to be offered the condom voucher, the offer could not be made to every student in the study: 31 percent of students were missing both pieces of information due to nonresponse, misspelled email addresses, or invalid phone numbers, and could not be offered the vouchers. The table shows there was no difference in condom voucher offers between control and treatment groups or between control and spillover groups. In addition, the table also shows that there was no differential attrition by socioeconomic status of the family<sup>16</sup> or parental education.

# Econometric Specification

Randomization allows for identification of reduced form intent-to-treat effects. Let _Y_ denote an out- _ijt_ come of interest at follow-up ( _t_ = 1 or 2) for individual _i_ in classroom _j_ . Treatment and spillover classroom assignment dummies are denoted by _Tj_ and _S j_ respectively. Treatment classrooms were selected for Internet-based sexual health training whereas spillover classrooms were not selected for the training but are in a school that has a treated classroom. Whenever available, the analysis includes the baseline dependent variable as control for precision.

> 13 The school year in some regions of Colombia begins in January (Calendario A), whereas in other regions it begins in September (Calendario B).

> 14 For the statistical analysis, these classrooms are still in the intent-to-treat group.

> 15 Summary statistics for every question used in the survey are reported in Supplementary appendix table S1.1.

> 16 A score from 1 to 6 assigned by the Colombian government to households for targeting all social programs in which 1 is poorest and 6 is richest. This score is well known and self-reported by the students in the baseline survey.
The following regression model is estimated via ordinary least squares as the main specification: where the error term _ε ijt_ is assumed to be uncorrelated across schools but not necessarily within them. Hence, standard errors are clustered at the school level. Because _Tj_ and _S j_ were randomly assigned, the estimated coefficients are unbiased estimators of the intent-to-treat effects of the course, which this paper argues are the policy coefficients of interest. The study has multiple measures of sexual health knowledge and attitudes in the survey. However, testing multiple outcomes using (1) for each measure independently increases the probability of rejecting a true null hypothesis for at least one outcome above the significance level used for each test (Duflo, Glennerster, and Kremer 2008). Hence, the study follows Kling, Liebman, and Katz (2007) and defines a summary measure _Y_<sup>∗</sup> as the unweighted average of all standardized outcomes in a family as follows: Where the mean ( _μ_ ) and variance ( _σ_<sup>2</sup> ) at baseline are used in the standardization of each variable _Yk_ . This allows the estimates _β_ 1and _β_ 2 to be interpreted as the effects of the course in terms of standard deviations of the outcome at baseline.

# **4. Results**

The main results appear in tables 3–5, reporting effects on knowledge (table 3), attitudes (table 4), and individual indicators of sexual behavior and condom redemption (table 5). For each indicator, whenever available the results are included from both follow-ups, the first taken one week after the end of the intervention and the second one taken six months after the end of the intervention. While the study focuses more on the results of the second follow-up, the comparisons of effects between the short- and medium-run provide an indication of the durability of the effects.

## Knowledge

Table 3 presents the impacts on five different standardized knowledge indices measuring: (a) STI symptoms and causes, (b) recognition of instances of sexual violence, (c) STI prevention, (d) pregnancy prevention methods, and (e) proper condom use. Columns (11) and (12) in the table show results for an overall index using all the variables used in the partial indices.<sup>17</sup> The notes in the table report the definition of the individual variables used in the construction of each index.

The aggregate knowledge index shows a 0.37 standard deviation increase in overall knowledge one week after the intervention and a 0.38 standard deviation increase in overall knowledge six months after the intervention compared to students not assigned to the course: Both coefficients are statistically significant at the 1 percent level. Furthermore, the study finds a robust pattern of positive coefficients in all components of the general index as well as statistical significance at least at the 5 percent level six months after the intervention.

The table also shows there is no clear pattern of decay in knowledge outcomes, since some improve while others decrease over time. The second row in the table also shows that there is no clear evidence for cross-classroom spillover effects in terms of sexual knowledge.

> 17 For space reasons, this article does not report results on every individual outcome, but they are available upon request. The general indices calculate the average for all nonmissing outcomes at the individual level. For the partial subindices, which are composed of few variables, the study drops observations, which have missing values for any component of the subindex. This gives the reader information about item nonresponse.
**Table 3.** Knowledge Indicators

||**Knowledge of**<br>**causes of ST**|**symptoms and**<br>**Is subindex**|**Sexual**<br>**knowledge**|**violence**<br>**subindex**|**Preventio**<br>**knowledge**|**n of STIs**<br>**subindex**|**Pregnancy**<br>**knowledge**|**prevention**<br>**subindex**|**Condom use**<br>**subi**|**knowledge**<br>**ndex**|**General k**<br>**in**|**nowledge**<br>**dex**|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**One week**<br>**post**<br>**intervention**|**Six months**<br>**post**<br>**intervention**|**One week**<br>**post inter-**<br>**vention**|**Six months**<br>**post inter-**<br>**vention**|**One week**<br>**post inter-**<br>**vention**|**Six months**<br>**post inter-**<br>**vention**|**One week**<br>**post inter-**<br>**vention**|**Six months**<br>**post inter-**<br>**vention**|**One week**<br>**post inter-**<br>**vention**|**Six months**<br>**post inter-**<br>**vention**|**One week**<br>**post inter-**<br>**vention**|**Six months**<br>**post inter-**<br>**vention**|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|**(6)**|**(7)**|**(8)**|**(9)**|**(10)**|**(11)**|**(12)**|
|Treatment students|0.282***|0.202***|0.254***|0.109**|0.067|0.519***|0.299***|0.335***|0.262***|0.166**|0.372***|0.378***|
||(0.048)|(0.056)|(0.057)|(0.054)|(0.041)|(0.139)|(0.049)|(0.078)|(0.046)|(0.064)|(0.049)|(0.080)|
|Spillover students|0.022|0.064|0.034|−0.025|0.024|0.139|0.043|0.061|0.051|0.031|0.015|0.011|
||(0.044)|(0.053)|(0.054)|(0.059)|(0.044)|(0.147)|(0.050)|(0.082)|(0.054)|(0.064)|(0.050)|(0.085)|
|Control for baseline<br>value of dep. var.|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|
|Observations|4,373|3,867|4,354|3,859|4,353|3,836|4,388|3,874|4,384|3,867|4,388|3,903|

_Source_ : Authors’ analysis from student surveys.

_Note_ : Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. _Knowledge of symptoms and causes of STI subindex_ : Respondent knows STI symptoms include: (a) Abnormal discharges from the penis/vagina; (b) lesions/sores in genitals; and (c) painful urination; respondent knows (d) vomiting and headache are not STI symptoms; (e) HIV can be transmitted by having sexual intercourse without a condom; (f) HIV can be transmitted by a contaminated blood transfusion; (g) HIV transmission does not depend on hygiene; (h) HIV cannot be transmitted via food sharing; (i) clothes sharing; or (j) being in a pool with an HIV-positive person. Respondent knows that (k) HIV is not transmitted if a condom is used while having sexual intercourse with an HIV-positive individual. _Sexual violence knowledge subindex_ : Respondent identifies (a) nonconsensual touching of genitalia, buttocks, breasts, inner thigh as abusive sexual contact; (b) forcible sex by husband on his wife as a form of sexual abuse; (c) having sex with a person who is impaired due to alcohol as a form of rape; (d) if an individual changes his/her mind about sex even at the last minute, sex is nonconsensual and hence a form of sexual abuse; (e) the use of threats to obtain sex is a form of sexual abuse; respondent knows: (f) sexual abuse is more often than not perpetrated by a known person not a stranger. _Prevention of STI knowledge subindex_ : Respondent knows one of the safest methods to prevent an STI is the use of condom whereas the calendar-based methods, hormone injections, and penis withdrawal are not. _Pregnancy prevention knowledge subindex_ : Respondent disagrees with (a) penis withdrawal is a safe method to avoid pregnancy; respondent knows: (b) women can become pregnant in their first sexual relationship; (c) safe methods to prevent a pregnancy include injections and condom; (d) unsafe methods to prevent a pregnancy include calendar-based methods and penis withdrawal; respondent knows that (e) emergency post-coital contraception pills have secondary effects. _Condom use knowledge subindex_ : Respondent knows (a) one of the safest methods to prevent an STI is the use of a condom; (b) condoms can be used only one time; (c) HIV can be transmitted by having sex without a condom; (d) HIV is not transmitted if a condom is used even if the person is HIV positive; (e) one of the safest methods to prevent a pregnancy is by using a condom. _General knowledge index_ : an index of all the variables used in the subindices of the table.
**Table 4.** Attitude Indicators

||**Condom use at**|**titudes subindex**|**Sexually conse**<br>**su**|**rvative attitudes**<br>**bindex**|**Sexual abuse r**<br>**sub**|**eporting attitudes**<br>**index**|**General at**|**titudes index**|
|---|---|---|---|---|---|---|---|---|
||**One week post**<br>**intervention**|**Six months post**<br>**intervention**|**One week post**<br>**intervention**|**Six months post**<br>**intervention**|**One week post**<br>**intervention**|**Six months post**<br>**intervention**|**One week post**<br>**intervention**|**Six months post**<br>**intervention**|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|**(6)**|**(7)**|**(8)**|
|Treatment students|0.170***|0.100*|0.072|0.133**|0.260***|0.112**|0.240***|0.172***|
||(0.051)|(0.051)|(0.046)|(0.058)|(0.048)|(0.054)|(0.053)|(0.056)|
|Spillover students|0.028|−0.024|0.003|0.075|0.035|0.015|0.026|0.022|
||(0.052)|(0.051)|(0.044)|(0.058)|(0.048)|(0.051)|(0.052)|(0.052)|
|Control for baseline|Yes|Yes|Yes|Yes|Yes|Yes|Yes|Yes|
|value of dep. var.|||||||||
|Observations|4,390|3,864|4,389|3,896|4,344|3,854|4,391|3,906|

_Source_ : Authors’ analysis from student surveys.

_Note_ : Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. _Condom use attitudes subindex_ : Respondent disagrees with statements (a) “It’s not right to carry a condom because people may think that I planned to have sex”; (b) “If a woman wants to have sex without a condom, the man must not refuse”; (c) “Only women are responsible for unwanted pregnancies”; respondent is (d) confident if requesting that a condom be used; (e) willing to delay sex if condoms are unavailable; respondent thinks (f) he/she will use a condom in his/her next sexual relationship. _Sexually conservative attitude subindex_ : Respondent thinks that (a) it is not right when people of their age have sex with several partners in the same month; (b) people of their age should wait to have sex; respondent’s answer to (c) age at which men and women should start having sex. Respondent is (d) confident he/she will have sex only when emotionally ready. _Sexual abuse reporting attitudes subindex_ : Respondent thinks that when a teenager is suffering from sexual violence (a) he/she must tell his/her family; (b) he/she must tell the authorities; (c) in case of rape, the afflicted individual must seek medical help; respondent disagrees with the idea that in case of rape the person (d) must not tell anyone. _General attitudes index_ : contains all variables used in the other columns of the table.
**Table 5.** Sexual Activity and Condom Demand

||**Sexually active last**<br>**six months**<br>**Six months post**<br>**intervention**|**Frequency of sex**<br>**last six months**<br>**Six months post**<br>**intervention**|**Number of partners**<br>**last six months**<br>**Six months post**<br>**intervention**|**STI presence**<br>**Six months post**<br>**intervention**|**Pregnancy**<br>**Six months post**<br>**intervention**|**Redeemed Voucher**<br>**for Free Condoms**<sup>**§**</sup><br>**Six months post**<br>**intervention**|
|---|---|---|---|---|---|---|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|**(6)**|
|Treatment students|−0.003|0.212|−0.009|−0.005|0.000|0.099*|
||(0.029)|(0.259)|(0.031)|(0.004)|(0.005)|(0.055)|
|Spillover students|0.023|0.278|0.043|−0.001|0.007|0.048|
||(0.031)|(0.270)|(0.031)|(0.004)|(0.006)|(0.048)|
|Control for baseline value of dep. var.|No|No|Yes|Yes|Yes|No|
|Mean of dep. var. control group|0.26|1.57|0.37|0.01|0.02|0.18|
|Observations|4,364|3,857|3,881|3,774|4,252|3,358|

_Source_ : Authors’ analysis from student surveys.

_Note:_ Dependent variables not standardized. All outcome variables are assessed six months after treatment. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. Column (1) includes students attrited for written survey but later tracked over the phone. Table does not control for baseline value of the dependent variable, because outcome was not measured at baseline (except for column (3)).<sup>§</sup> 3,358 students of the full sample agreed to be contacted for this part of the study. Specification controls for whether individual had a cellphone.
The World Bank Economic Review 13

# Attitudes

Table 4 presents results on attitude indicators. Columns (1) and (2) present results on attitudes towards the use of condoms, (3) and (4) on conservatism with respect to age of initiation of sexual activities, (5) and (6) on attitudes toward denouncing and seeking help in the event of sexual abuse, and columns (7) − and (8) show results for an overall index of attitudes, containing all variables used in columns (1 6).

For the general attitudes index the study finds significant effects of 0.24 standard deviations in terms of attitudes one week after the intervention and 0.17 standard deviations six months after. Significant effects were again found for each subindex six months after the intervention, and the pattern of positive effects on all partial indices is again apparent.

The training was successful in generating more positive attitudes towards the use of condoms at the first follow-up (0.17 standard deviation) and at the second follow-up (0.10 standard deviation). In the sexually conservative attitudes subindex composed of the following variables: (a) indicates that individuals their age should not have multiple sexual partners in the same month; (b) thinks it is too early for individuals of their age to engage in sexual activities; and (c) feels confident he/she will be able to wait to have sex until emotionally prepared to do so, teens scored 0.13 standard deviations higher six months after the intervention. Column (6) shows that treated teens are 0.11 standard deviations more likely to agree with the need to report cases of sexual abuse to the authorities and the need to seek medical attention in such situations. For attitude indicators, there was no consistent evidence of spillovers across classrooms.<sup>18</sup>

# Sexual Activity and Condom Demand

Table 5 reports results on sexual activity and condom voucher redemptions. Specifically, columns 1–3 report effects of the course on self-reported sexual activity six months after the end of the course.<sup>19</sup> Columns 1–3 uniformly show that the course did not increase or decrease self-reported sexual activity among adolescents. Column 1 shows that students were not more likely to be sexually active in the past six months compared to the control group. Similarly, the frequency of sex did not change in the previous six months compared to the control group (column 2) nor in terms of number of partners (column 3). The finding that sexual education does not result in increases in sexual activity is a robust finding in this literature (cf. Kirby, Laris, and Rolleri 2007).

Given that the objective population is teens at high risk of sexual initiation, the setting is not designed to capture other sexual behavior outcomes such as STIs and pregnancy, given their extremely low prevalence (the control group means for these self-reported outcomes are 1 percent and 2 percent; thus there is little room for improvement). The study nevertheless estimated treatment effects for these outcomes and found no statistically significant changes (columns 4 and 5 of table 5).

Of course, the validity of studies using self-reported sexual behavior among adolescents has long been a criticism in this literature (Brener, Billy, and Grady 2003). This challenge was addressed by measuring the percentage of students who redeem vouchers for condoms. This strategy makes it possible to address the possible lack of reliability in self-reported outcomes via an objective safe-sex behavior metric. Condom availability is important for adolescent health given the sporadic nature of adolescent sexual activity. Column 6 in table 5 reports the results of the voucher experiment.

The administrative data from voucher redemption shows statistically significant and important effects. The table shows that 28 percent of treatment students redeem them, compared to 18 percent of control

> 18 Classroom level spillover effects were also statistically undetectable using a dose-response model. Specifically, this was implemented by defining the spillover variable as one over the number of classrooms in grade 9 in the school. Spillover coefficients again followed the same pattern of no significant effects.

> 19 Note that for this table the dependent variables are not an index but a single variable, so the study does not standardize them. This has the benefit of allowing for comparability with other studies. Furthermore, in this table there is no control for baseline value of the dependent variable because it was not available at baseline (except for columns 3-5).
Chong et al.

14

**Table 6.** Friendship Networks Summary Statistics

|||**Cases**|**Percent**|
|---|---|---|---|
|**Treatment students with:**|No friends treated|366|21.2%|
||1 friend treated|277|16.0%|
||2 friends treated|266|15.4%|
||3 friends treated|227|13.1%|
||4 friends treated|286|16.6%|
||5 friends treated|183|10.6%|
||6 friends treated|123|7.1%|
|**Spillover students with:**|No friends treated|1482|88.8%|
||1 friend treated|133|8.0%|
||2 friends treated|11|0.7%|
||3 friends treated|7|0.4%|
||4 friends treated|7|0.4%|
||5 friends treated|13|0.8%|
||6 friends treated|15|0.9%|

_Source_ : Authors’ analysis from student surveys. _Note_ : Friendship link treatment status is established by matching self-reported list of friends with list of names of students answering the survey at (either) follow-up survey. The number of friends treated for students in control schools is equal to zero.

students, a 55 percent increase in redemption ( _p_ = 0.07). The study puts substantial weight on this result as it provides objective evidence of an increase in condom demand. In unreported results, the estimated coefficient is basically unchanged when controlling for distance to the health clinic.

As a robustness check, bounding exercises were performed with differing assumptions on attrition, as in Karlan and Valdivia (2011), and found that the positive effect on condom voucher in table 5 (0.099 percentage points; standard error = 0.055 percentage points) still holds after imputing the mean minus 0.10 standard deviations of the observed treatment distribution to the nonrespondents in the treatment group, and after imputing the mean plus 0.10 standard deviations of the observed control distribution to nonrespondents in the control group.<sup>20</sup>

# Friendship Network Spillovers and Reinforcing Interactions

Next the study takes advantage of the fact that in the surveys students were asked to identify their closest friends by name, and indicate if they were in the same school and/or classroom. This information is used to match each student’s social network to the list of students in the treatment and spillover groups. This makes it possible to analyze treatment and spillover effects differentiating between students for whom a small or a large percentage of friends was also treated.

Table 6 presents summary statistics about the network treatment distribution. For 21 percent of students in a treated classroom their list of friends does not contain anyone from their own classroom. The analysis will consider these students as having a network of friends that is not treated at all. In contrast, 79 percent of students in a treated classroom have at least one friend in their own classroom (who was hence also treated). The table shows substantial variation in the number of friends that are located in the same classroom as a treatment student. The lower panel in the table shows that for students in a spillover classroom, there are few links to students taking the course (in the treatment classroom). Indeed, 89 percent of spillover students have no friends in the treatment classroom—this will affect the precision of the spillover estimates.

20 Results of these simulations are not presented here but are available upon request.
The World Bank Economic Review 15 With this information, a measure of network treatment intensity is naturally defined as the proportion of the student’s network of closest friends who were treated (friends in a treatment classroom/total listed friends).<sup>21</sup> If a student and his or her entire network of close friends were all in the same treatment classroom, then the proportion is equal to one, but if the network of friends includes students from other classrooms or from outside the school, then the proportion is lower. The analysis uses variation in the proportion of close friends that are in the student’s classroom to estimate a heterogeneous treatment effects regression in which the main effects are now interacted with the proportion of friends in the network who were treated ( _Fij_ ). The specification becomes: where in order to control for popularity of a particular student, the regression includes _Nij_ defined as the number of people who mentioned individual _i_ as a friend. Throughout the analysis, standard errors are clustered at the school level. In tables 7, 8, and 9 the interpretation of the main effect ( _Tj_ ) now becomes the effect of assignment to treatment for someone who has zero friends also treated, whereas the coefficient on ( _Fij_ × _Tj_ ) is the additional effect of the course for someone whose full set of friends are also treated (analogously for _S j_ ).

In interpreting these results, the reader should be cognizant that the distribution of a student’s network of friends is not a randomly assigned variable. The identifying variation is coming from whether the student’s friends are in his or her classroom or are rather in another classroom in the school or from outside of the school. This may lead to bias if, for example, more extroverted students have a larger proportion of their best friends in the classroom and this extroversion is related to the outcome beyond the effect stemming from social reinforcement. For this reason, the study conditions on _Nij_ , the number of individuals that mention another student as a best friend.<sup>22</sup> The necessary assumption hence becomes that the proportion of friends in the classroom is related to the treatment response only through the network effects (conditional on the number of friends).

With this assumption in mind, results for network interactions are presented in tables 7, 8, and 9, which report effects six months after finishing the course. Table 7 provides clear evidence of a significant reinforcing interaction effect for students in the treatment group in terms of the overall knowledge index (column 6). The study is able to identify an effect of 0.46 standard deviation in knowledge for wholly treated networks, as opposed to a 0.28 standard deviation effect if the student’s network is not treated. In contrast, there are no statistically significant effects for spillover students, even if their network was fully treated. As noted before, large standard errors are obtained for the spillover estimates due to the small number of spillover students with treated networks. At the bottom of each column, the table reports the _p_ -value from a test of equality of the friendship interaction effects for treatment and spillover students. The reinforcing interaction effect is positive, large, and significant for all five subindices except for those reporting knowledge concerning sexual violence and knowledge about condom use (columns 2 and 5).

Table 8, on attitude indicators, shows an even starker reinforcing interaction effect. In this case, the effects are significant _only_ if the student’s friendship network also took the course. For example, if a student’s full network was treated, the student is predicted to have a 0.24 standard deviation higher attitude index score, whereas the estimated effect is only 0.04 standard deviation if no one in his or her friendship network was treated. Similar outcomes are observed in each of the subcomponents of the index. As in table 7, there is no significant network spillover effect for a student who did not take the course.

> 21 One shortcoming of this network analysis is that the questionnaire did not clearly differentiate between friendship and romantic  relationships.

> 22 When looking at determinants of being more popular in school, the data shows that kids who are more popular are less likely to smoke, drink, and consume drugs at baseline.
**Table 7.** Knowledge: Network Spillover and Reinforcing Interaction Effects

||**Knowledge of**<br>**symptoms and causes**<br>**of STIs subindex**|**Sexual violence**<br>**knowledge**<br>**subindex**|**Prevention of STI**<br>**knowledge**<br>**subindex**|**Pregnancy**<br>**prevention**<br>**knowledge subindex**|**Condom use**<br>**knowledge**<br>**subindex**|**General**<br>**knowledge index**|
|---|---|---|---|---|---|---|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|**(6)**|
|Treatment student|0.132*|0.083|0.377**|0.201**|0.135*|0.278***|
||(0.067)|(0.062)|(0.153)|(0.090)|(0.075)|(0.081)|
|Spillover student|0.082|−0.023|0.137|0.065|0.035|0.022|
||(0.055)|(0.058)|(0.147)|(0.082)|(0.062)|(0.082)|
|Treatment student * % of friends treated|0.136*|0.038|0.258*|0.248**|0.056|0.179*|
||(0.081)|(0.080)|(0.155)|(0.106)|(0.079)|(0.100)|
|Spillover student * % of friends treated|−0.280*|0.170|0.185|−0.154|−0.034|−0.050|
||(0.159)|(0.177)|(0.380)|(0.213)|(0.196)|(0.217)|
|Number of friends|0.034*|0.074***|0.151***|0.080***|0.036*|0.104***|
||(0.018)|(0.017)|(0.043)|(0.019)|(0.019)|(0.020)|
|Control for baseline value of dep. var.|Yes|Yes|Yes|Yes|Yes|Yes|
|_P_-value treatment * (% of friends)<br>=spillover * (% of friends)|0.0387|0.492|0.859|0.0788|0.669|0.334|
|Observations|3,853|3,845|3,828|3,866|3,853|3,888|

_Source_ : Authors’ analysis from student surveys.

_Note_ : Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. _Knowledge of symptoms and causes of STI subindex_ : Respondent knows STI symptoms include (a) abnormal discharges from the penis/vagina; (b) lesions/sores in genitals; and (c) painful urination; respondent knows: (d) vomiting and headache are not STI symptoms; (e) HIV can be transmitted by having sexual intercourse without a condom; (f) HIV can be transmitted by a contaminated blood transfusion; (g) HIV transmission does not depend on hygiene; (h) HIV cannot be transmitted via food sharing; (i) clothes sharing; or (j) being in a pool with an HIV-positive person. Respondent knows that (k) HIV is not transmitted if a condom is used while having sexual intercourse with an HIV-positive individual. _Sexual violence knowledge subindex_ : Respondent identifies (a) nonconsensual touching of genitalia, buttocks, breasts, inner thigh as abusive sexual contact; (b) forcible sex by husband on his wife as a form of sexual abuse; (c) having sex with a person who is impaired due to alcohol as a form of rape; (d) if an individual changes his/her mind about sex even at the last minute, sex is nonconsensual and hence a form of sexual abuse; (e) the use of threats to obtain sex is a form of sexual abuse; respondent knows: (f) sexual abuse is more often than not perpetrated by a known person not a stranger. _Prevention of STI knowledge subinde_ x: Respondent knows one of the safest methods to prevent an STI is the use of condom whereas the calendar-based methods, hormone injections, and penis withdrawal are not. _Pregnancy prevention knowledge subindex_ : Respondent disagrees with (a) penis withdrawal is a safe method to avoid pregnancy; respondent knows: (b) women can become pregnant in their first sexual relationship; (c) safe methods to prevent a pregnancy include injections and condom; (d) unsafe methods to prevent a pregnancy include calendar-based methods and penis withdrawal; respondent knows that (e) emergency postcoital contraception pills have secondary effects. _Condom use knowledge subindex_ : Respondent knows (a) one of the safest methods to prevent an STI is the use of a condom; (b) condoms can be used only one time; (c) HIV can be transmitted by having sex without a condom; (d) HIV is not transmitted if a condom is used even if the person is HIV positive; (e) one of the safest methods to prevent a pregnancy is by using a condom. _General knowledge index_ : an index of all the variables used in the subindices of the table.
The World Bank Economic Review 17

**Table 8.** Attitudes: Network Spillover and Reinforcing Interaction Effects

||**Condom use**<br>**attitudes**<br>**subindex**|**Sexually**<br>**conservative**<br>**attitudes subindex**|**Sexual abuse**<br>**reporting attitudes**<br>**subindex**|**General**<br>**attitudes index**|
|---|---|---|---|---|
||**(1)**|**(2)**|**(3)**|**(4)**|
|Treatment student|−0.020|0.072|0.024|0.043|
||(0.064)|(0.074)|(0.070)|(0.073)|
|Spillover student|−0.021|0.082|0.021|0.032|
||(0.049)|(0.058)|(0.051)|(0.051)|
|Treatment student * % of friends treated|0.213***|0.114|0.166*|0.236***|
||(0.075)|(0.071)|(0.093)|(0.078)|
|Spillover student * % of friends treated|−0.098|−0.059|−0.003|−0.072|
||(0.170)|(0.143)|(0.131)|(0.147)|
|Number of friends|0.068***|0.006|0.044***|0.052***|
||(0.017)|(0.015)|(0.016)|(0.016)|
|Control for baseline value of dep. var.|Yes|Yes|Yes|Yes|
|_P_-value: treatment * (% of friends)|0.129|0.279|0.270|0.061|
|=spillover * (% of friends)|||||
|Observations|3,856|3,882|3,840|3,891|

_Source_ : Authors’ analysis from student surveys. _Note_ : Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. _Condom use attitudes subindex_ : Respondent disagrees with statements (a) “It’s not right to carry a condom because people may think that I planned to have sex”; (b) “If a woman wants to have sex without a condom, the man must not refuse”; (c) “Only women are responsible for unwanted pregnancies”; respondent is (d) confident if requesting that a condom be used; (e) willing to delay sex if condoms are unavailable; respondent thinks (f) he/she will use a condom in his/her next sexual relationship. _Sexually conservative attitude subindex_ : Respondent thinks that (a) it is not right when people of their age have sex with several partners in the same month; (b) people of their age should wait to have sex; respondent’s answer to (c) age at which men and women should start having sex. Respondent is (d) confident he/she will have sex only when emotionally ready. _Sexual abuse reporting attitudes subindex_ : Respondent thinks that when a teenager is suffering from sexual violence (a) he/she must tell his/her family; (b) he/she must tell the authorities; (c) in case of rape, the afflicted individual must seek medical help; respondent disagrees with the idea that in case of rape the person (d) must not tell anyone. _General attitudes index_ : contains all variables used in the other columns of the table.

This provides evidence that the relevant group for a reinforcement effect is the network of friends, as suggested in Sacerdote (2011).

Table 9 presents evidence of reinforcing interaction effects for self-reported sexual behavior: Students with more friends taking the course report significantly lower sexual activity than those with fewer friends taking the course for two out of three indicators.

Column 4 reports results for condom redemption. In contrast to the network interaction results in knowledge and attitudes, the condom redemption results are not significant, although the signs of the coefficients are consistent with tables 8 and 9 in the sense that they point to an improvement the larger the share of friends that is treated. Indeed, treatment students are 8 percentage points more likely to redeem their vouchers than the control group, even if none of their close friends were treated (not statistically significant). The coefficient estimate on the reinforcing interaction suggests a 4.3 percentage point higher condom demand for those with all friends being treated, but again the coefficient is not statistically different from zero.

Column 4 also shows a puzzling result in which spillover students whose entire friend networks were treated are 14 percentage points less likely to redeem their condom vouchers than the control group. Note there is a small number of observations in the spillover group with friendship links to treated students, and the results in all the other indicators do not show a pattern of significant spillovers from classroom to classroom, so the authors do not attach much importance to this coefficient.
Chong et al.

18

**Table 9.** Sexual Activity and Condom Demand: Network Spillover and Reinforcing Interaction Effects

||**Sexually active last**<br>**six months**<br>**(1)**|**Frequency of sex**<br>**last six months**<br>**(2)**|**Number of partners**<br>**last six months**<br>**(3)**|**Redeemed voucher**<br>**for free condoms**<sup>**§**</sup><br>**(4)**|
|---|---|---|---|---|
|Treatment student|0.042|0.340|0.049|0.076|
||(0.036)|(0.313)|(0.039)|(0.058)|
|Spillover student|0.016|0.174|0.032|0.056|
||(0.032)|(0.264)|(0.031)|(0.046)|
|Treatment student * % of friends treated|−0.097**|−0.257|−0.115***|0.043|
||(0.038)|(0.440)|(0.042)|(0.040)|
|Spillover student * % of friends treated|0.110|1.850*|0.186|−0.138*|
||(0.085)|(0.986)|(0.114)|(0.069)|
|Number of friends|−0.005|−0.018|0.008|0.009|
||(0.008)|(0.081)|(0.010)|(0.011)|
|Control for baseline value of dep. var.|No|No|Yes|No|
|Mean of dep. var. control group|0.26|1.58|0.37|0.18|
|_P_-value: treatment * (% of friends)<br>=spillover * (% of friends)|0.0195|0.0662|0.0170|0.0335|
|Observations|4,246|3,843|3,868|3,334|

_Source_ : Authors’ analysis from student surveys. _Note_ : Dependent variables not standardized. All outcome variables are assessed six months after treatment. Standard errors clustered at the school level in parentheses. *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. Column 1 includes students attrited for written survey but later tracked over the phone. Table does not control for baseline value of the dependent variable, because outcome was not measured at baseline (except for column 3).<sup>§</sup> 3,358 students of the full sample agreed to be contacted for this part of the study. Specification controls for whether individual had a cellphone.

Overall, while the anlysis reveals that network effects are clearly identified among treated students, it is not surprising that the study had little power to detect network effects among spillover students given the very few friendship network links that occur across classrooms in the data.

# Cost-Effectiveness and Cost-Benefit Analysis

The marginal cost of the Profamilia course is approximately $14.60 per student. The bulk of this cost ($10) is accounted for by the remote tutor, and the remainder comes from Internet platform costs and computer depreciation. The calculations do not include opportunity costs of the time of the students (such as some alternative educational activity, leisure, or work outside of school).<sup>23</sup> Compared to noncomputer-based sexual health interventions in the United States, which range from $69 to more than $10,000 per student,<sup>24</sup> the Profamilia course is extremely low cost. It is also low cost compared to instructor-led programs in developing countries. Kivela, Ketting, and Baltussen (2013) report costs per student of teacher-led school based sexual education programs of $27 in Kenya and $85 in Indonesia.<sup>25</sup> Table 10 presents the cost effectiveness and cost-benefit calculations. These estimates obviously rest on many assumptions, but they provide much value for policy makers by allowing for comparisons across different interventions.

The key result is that students who take the course are 9.9 percentage points more likely to redeem the condom voucher. For the purposes of table 10, the assumption is made that this indicates a consistent condom user. This makes it possible to link the coefficient to the literature documenting the effect of

> 23 The calculations also exclude the wage cost of the person supervising students in the computer lab because it is unlikely that a school would hire personnel exclusively for the course. This is in line with guidelines by Dhaliwal et al. (2013), who argue that cost-effectiveness should use marginal costs of adding the program, assuming fixed costs are incurred with or without the  program.

> 24 Chin et al. (2012), pp. 280, with inflated estimates to 2012  dollars.

25 Costs refer to one semester of the course and are expressed in 2012 U.S. dollars.
E(ADALY |STD = 1) = ye DALY; - Pr(STD = i|STD = 1), where i represents {Chlamydia, gonorrhea, trichomoniasis, syphilis, other curable STDs, PID, genital herpes, cervical cancer, hepatitis B, hepatitis C, HIV}.
Chong et al.

20 the novelty of this approach is that it provides strong evidence that anonymity and confidentiality of information technologies may be of great use in segments of the society in which keeping such anonymity is difficult to overcome.<sup>28</sup> The results on knowledge and attitudes are important because these two factors have been shown to be the strongest protective factors in preventing STIs, HIV, and pregnancy among teens (Kirby, Lepore, and Ryan 2007). Furthermore, recent research has documented the important role that social norms play in responsible sexual behavior (Munshi and Myaux 2005; Ashraf, Field, and Lee 2014). By changing knowledge and attitudes in youth attending school, sexual education can ultimately play a fundamental role in achieving desirable aggregate changes in sexual behavior.

A second contribution to the sexual health education literature is the focus on spillovers, through a twostage experimental design. The results indicate that spillovers from treated to untreated classrooms in the same school are negligible.

Lastly, the analysis provides strong indications that effects of the course were reinforced when treated individuals had larger percentages of their friend networks in treatment classrooms. The evidence is robust across a large set of sexual health attitude and knowledge indicators. In particular, the analysis found that students whose networks were more intensely treated had significant improvements in knowledge and attitudes, which the study interprets as social reinforcement effects or complementarities. These results demonstrate the positive externalities of the public provision of sex education: When an individual takes a sex education course, this decision has positive effects on sexual health outcomes among his or her close friends. This suggests that without collective action, there is an underprovision of sex education.

These results provide an optimistic assessment of the use of information technologies to generate improved sexual health outcomes among the youth. The cost-benefit analysis suggests that because Internetbased sexual health education programs are extremely low cost, their benefits in terms of STI reductions actually justify the costs.

# **References**

AIDS Control and Prevention Project (AIDSCAP). 1997. _Making Prevention Work: Global Lessons Learned from the AIDS Control and Preventional Project 1991–1997._ Arlington, VA: Family Health International.

Angrist, J., and V. Lavy. 2002. “New Evidence on Classroom Computers and Pupil Learning.” _Economic Journal_ 112 (482): 735–65.

Ashraf, N., E. Field, and J. Lee. 2014. “Household Bargaining and Excess Fertility: An Experimental Study in Zambia.” _American Economic Review_ 104 (7): 2210–37.

Auriol, E., and M. Wartlers. 2012. “The Marginal Cost of Public Funds and Tax Reform in Africa.” _Journal of Development Economics_ 97 (1): 58–72.

Barrera-Osorio, F., and L. L. Linden. 2009. “The Use and Misuse of Computers in Education: Evidence from a Randomized Controlled Trial of a Language Arts Program.” Policy Research Working Paper No. 4836, World Bank, Washington, DC.

Brener, N. D., J. O. G. Billy, and W. R. Grady. 2003. “Assessment of Factors Affecting the Validity of Self-Reported Health-Risk Behavior Among Adolescents: Evidence from the Scientific Literature.” _Journal of Adolescent Health_ 33 (6): 436–57.

Brent, R. J. 2011. “An Implicit Price of a DALY for Use in Cost Benefit Analysis of ARVs.” _Applied Economics_ 43 (11): 1413–21.

Bruhn, M., and D. McKenzie. 2009. “In Pursuit of Balance: Randomization in Practice in Development Field Experiments.” _American Economic Journal: Applied Economics_ 1 (4): 200–32.

- 28 This approach provides an alternative to social marketing campaigns that attempt to increase the use of condoms in developing countries, and may be better suited than the latter for the segments of the population described in this paper. Among others, in Tanzania, a campaign to promote condom use among women was entitled “Talk to Him” and included posters and depicting a variety of confident, empowered, young women (AIDS Control and Prevention Project 1997).
The World Bank Economic Review 21 Card, D., and L. Giuliano. 2013. “Peer Effects and Multiple Equilibria in the Risky Behavior of Friends.” _Review of Economics and Statistics_ 95 (4): 1130–49.

Chin, H. B., T. A. Sipe, R. Elder, S. L. Mercer, S. K. Chattopadhyay, V. Jacob, and H. R. Wethington et al. 2012. “The Effectiveness of Group-Based Comprehensive Risk-Reduction and Abstinence Education Interventions to Prevent or Reduce the Risk of Adolescent Pregnancy, Human Immunodeficiency Virus, and Sexually Transmitted Infections.” _American Journal of Preventive Medicine_ 42 (3): 272–94.

Collins, J., L. Robin, S. Wooley, D. Fenley, P. Hunt, J. Taylor, D. Harber, and L. Kolbe. 2002. “Programs-That-Word: CDC’s Guide to Effective Programs That Reduce Health-Risk Behavior of Youth.” _Journal of School Health_ 72 (3): 93–9.

Demographic and Health Survey (DHS). 2005. _Encuesta Nacional de Demografía y Salud - ENDS Colombia 2005_ . Washington, DC: World Bank.

Dhaliwal, I., E. Duflo, R. Glennerster, and C. Tulloch. 2013. “Comparative Cost-Effectiveness to Inform Policy in Developing Countries: A General Framework with Applications for Education.” In _Education Policy in Developing Countries_ , edited by P. Glewwe, 285–338. Chicago: University of Chicago Press.

Duflo, E., R. Glennerster, and M. Kremer. 2008. “Using Randomization in Development Economics Research: A Toolkit.” _Handbook of Development Economics_ 4 (5): 3895–3962.

Ebrahim, S. H., M. T. McKenna, and J. S. Marks. 2005. “Sexual Behavior: Related Adverse Health Burden in the United States.” _Sexually Transmitted Infections_ 81 (1): 38–40.

Fairlie, R., and J. Robinson. 2013. “Experimental Evidence on the Effects of Home Computers on Academic Achievement among Schoolchildren.” _American Economic Journal: Applied Economics_ 5 (3): 211–40.

Flanigan, C. M., K. Suellentrop, M. M. Whitehead, and J. Smith. 2006. “Teens’ Sexual Experience 1995–2002.” _Science Says_ 22: 1–5. Fletcher, J. M. 2007. “Social Multipliers in Sexual Initiation Decisions Among U.S. High School Students.” _Demography_ 44 (2): 373–88. Fonner, V. A., K. S. Armstrong, C. E. Kennedy, K. R. O’Reilly, and M. D. Sweat. 2014. “School Based Sex Education and HIV Prevention in Low and Middle-Income Countries: A Systematic Review and Meta-Analysis.” _PlosOne_ 9 (3): 1–18. Fortson, J. 2009. “HIV/AIDS and Fertility.” _American Economic Journal: Applied Economics_ 1 (3): 170–94. Gallo, M. F., M. J. Steiner, L. Warner, T. Hylton-Kong, P. Figueroa, M. M. Hobbs, and F. M. Behets. 2007. “SelfReported Condom Use is Associated with Reduced Risk of Chlamydia, Gonorrhea, and Trichomoniasis.” _Sexually Transmitted Diseases_ 34 (10): 829–33. Goesling, B., S. Colman, C. Trenholm, M. A. Terzian, and K. A. Moore. 2014. “Programs to Reduce Teen Pregnancy, Sexually Transmitted Infections, and Associated Sexual Risk Behaviors: A Systematic Review.” _Journal of Adolescent Health_ 54(5): 499–507. Karlan, D., and M. Valdivia. 2011. “Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions.” _Review of Economics and Statistics_ 93 (2): 510–27. Kirby, D., A. B. Laris, and L. A. Rolleri. 2007. “Sex and HIV Education Programs for Youth: Their Impact and Important Characteristics.” _Journal of Adolescent Health_ 40 (3): 206–17. Kirby, D., G. Lepore, and J. Ryan. 2007. “Sexual Risk and Protective Factors: Factors Affecting Teen Sexual Behavior, Pregnancy, Childbearing and Sexually Transmitted Disease: Which Are Important? Which Can You Change?” ETR Associates Working Paper, 1–106. Kivela, J., E. Ketting, and R. Baltussen. 2013. “Cost Analysis of School-Based Sexuality Education Programs in Six Countries.” _Cost Effectiveness and Resource Allocation_ 11 (1): 17. Kling, J. R., J. B. Liebman, and L. F. Katz. 2007. “Experimental Analysis of Neighborhood Effects.” _Econometrica_ 75 (1): 83–119. Krueger, A., and C. Rouse. 2004. “Putting Computerized Instruction to the Test: A Randomized Evaluation of a Scientifically Based Reading Program.” _Economics of Education Review_ 23 (4): 323–38. Manski, C. 2013. “Identification of Treatment Response with Social Interactions.” _Econometrics Journal_ 16 (1): 1–23. Miller, G. 2010. “Contraception as Development? New Evidence from Family Planning in Colombia.” _Economic Journal_ 120 (545): 709–36. Munshi, K., and J. Myaux. 2005. “Social Norms and the Fertility Transition.” _Journal of Development Economics_ 70 (1): 1–38.
Chong et al.

22 Noar, S. M., H. G. Black, and L. B. Pierce. 2009. “Efficacy of Computer Technology-Based HIV Prevention Interventions: A Meta-Analysis.” _AIDS_ 23 (1): 107–15.

O’Donnell, L. N., A. S. Doval, R. Duran, and C. O’Donnell. 1995. “Video-Based Sexually Transmitted Disease Patient Education: Its Impact on Condom Acquisition.” _American Journal of Public Health_ 85 (6): 817–22.

Richards-Shubik, S. 2015. “Peer Effects in Sexual Initiation: Separating Demand and Supply Mechanisms.” _Quantitative Economics_ 6 (3): 663–702.

Ross, D. A., J. Changalucha, A. I. Obasi, J. Todd, M. L. Plummer, B. Cleophas-Mazige, A. Anemona, D. Everett, H. Weiss, D. C. Mabey, H. Grosskurth, and R. J. Hayes. 2007. “Biological and Behavioural Impact of an Adolescent Sexual Health Intervention in Tanzania: A Community Randomized Trial.” _AIDS_ 21(14): 1943–55.

Ruger, J. P., A. B. Abdallah, N. Y. Ng, C. Luekens, and L. Cottler. 2014. “Cost-Effectiveness of Interventions to Prevent HIV and STDs Among Injection Drug-Using Women: A Randomized Controlled Trial.” _AIDS and Behavior_ 18 (10): 1913–23.

Sacerdote, B. 2011. “Peer Effects in Education: How Might They Work, How Big Are They and How Much Do We Know Thus Far?” In _Handbook of Economics of Education_ , Vol. 3, edited by E. Hanushek, S. Machin, and L. Woessmann, 249–77. Amsterdam: North-Holland (Elsevier).

Shaffi, T., K. Stovel, and K. Holmes. 2007. “Association Between Condom Use at Sexual Debut and Subsequent Sexual Trajectories: A Longitudinal Study Using Biomarkers.” _American Journal of Public Health_ 97 (6): 1090–95.

Thornton, R. L. 2008. “The Demand For, and Impact Of, Learning HIV Status.” _American Economic Review_ 98 (5): 1829–63.

World Health Organization (WHO). 2004. _The World Health Report 2004_ . Geneva: World Health Organization.
**Supplementary Appendix Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia Alberto Chong, Marco Gonzalez-Navarro, Dean Karlan, and Martín Valdivia**
# **Supplementary Appendix S1: Summary Statistics for Individual Variables**

**Table S1.1.** Summary Statistics at Baseline

|**INDEX**|**INDIVID**|**UAL variables**|**MEAN**|**SD**|**MIN**|**MAX**|**Q25**|**Q75**|**N**|
|---|---|---|---|---|---|---|---|---|---|
|Knowledge of<br>symptoms and|Respondent knows:|Vomiting is not an STI<br>symptom|0.101|0.301|0|1|0|0|4305|
|causes of STI<br>index variables||Headache is not an STI<br>symptom|0.105|0.307|0|1|0|0|4211|
||Respondent knows STI<br>symptoms include:|Abnormal discharges from<br>the penis/vagina|0.307|0.461|0|1|0|1|4331|
|||<br>Lesions/sores in genitals|0.185|0.388|0|1|0|0|4221|
|||Painful urination|0.320|0.467|0|1|0|1|4334|
||HIV transmission does not|Hygiene|0.665|0.472|0|1|0|1|4512|
||depend on:|Food sharing|0.907|0.291|0|1|1|1|4512|
|||Being in a pool with an<br>HIV-positive person|0.924|0.265|0|1|1|1|4512|
||HIV cannot be transmitted:|If a condom is used while<br>having sexual intercourse<br>with an HIV-positive<br>individual|0.628|0.483|0|1|0|1|4512|
||HIV can be transmitted by:|Having sexual intercourse<br>without a condom|0.791|0.407|0|1|1|1|4512|
|||A contaminated blood<br>transfusion|0.684|0.465|0|1|0|1|4512|
|Sexual violence <br>knowledge|Respondent identifies as<br>abusive sexual contact or|Nonconsensual touching of<br>genitalia, buttocks, breasts,|0.845|0.362|0|1|1|1|4490|
|index variables|abuse:|and inner thigh||||||||
|||<br>Forcible sex by husband on<br>his wife|0.758|0.429|0|1|1|1|4490|
|||Having sex with a person<br>who is impaired due to<br>alcohol|0.759|0.427|0|1|1|1|4490|
|||If an individual changes<br>his/her mind about sex even<br>at the last minute|0.569|0.495|0|1|0|1|4490|
|||The use of threats to obtain<br>sex|0.670|0.470|0|1|0|1|4490|
||Respondent knows sexual a<br>petrated by a known person|buse is more often than not per-<br>, not a stranger|0.181|0.385|0|1|0|0|4343|
|Prevention of|Respondent knows one of|Calendar-based methods|0.929|0.256|0|1|1|1|4504|
|STI knowledge|the safest methods to|Hormone injections|0.795|0.404|0|1|1|1|4504|
|index variables|prevent an STI is not:|Penis withdrawal|0.905|0.293|0|1|1|1|4504|
||Respondent knows one of th<br>STI is the use of condoms|e safest methods to prevent an|0.737|0.440|0|1|0|1|4504|
# **Table S1.1.** Continued.

|**INDEX**|**INDIVID**|**UAL variables**<br>**MEAN**|<br>**SD**|**MIN**|**MAX**|**Q25**|**Q75**|**N**|
|---|---|---|---|---|---|---|---|---|
|Pregnancy<br>prevention<br>knowledge|Respondent disagrees that p<br>to avoid pregnancy|enis withdrawal is a safe method 0.562|0.496|0|1|0|1|4477|
|index variables|||||||||
||Respondent knows women c<br>sexual relationship|an become pregnant in their first 0.723|0.448|0|1|0|1|4506|
||Respondent knows unsafe<br>methods to prevent a|Calendar-based methods<br>0.875|0.330|0|1|1|1|4516|
||pregnancy include:|Penis withdrawal<br>0.791|0.407|0|1|1|1|4516|
||Respondent knows safe|Injections<br>0.471|0.499|0|1|0|1|4516|
||methods to prevent a<br>pregnancy include:|Condoms<br>0.759|0.428|0|1|1|1|4516|
||Respondent knows that emer<br>pills have secondary effects|gency post-coital contraception 0.143|0.351|0|1|0|0|4477|
|Condom use<br>knowledge|Respondent knows condoms|can be used only one time<br>0.608|0.488|0|1|0|1|4485|
|index variables|Respondent knows one of the s<br>STI is the use of a condom|afest methods to prevent an 0.737|0.440|0|1|0|1|4504|
||Respondent knows HIV can b<br>without a condom|e transmitted by having sex 0.791|0.407|0|1|1|1|4512|
||Respondent knows HIV is no<br>used even if the person is HIV|t transmitted if a condom is 0.628<br>positive|0.483|0|1|0|1|4512|
|Condom use<br>attitudes index<br>variables|Respondent knows one of t<br>pregnancy is by using a cond|he safest methods to prevent a 0.759<br>om|0.428|0|1|1|1|4516|
||Respondent disagrees with<br>statements:|think that I planned to have<br>It’s not right to carry a<br>2.894<br>condom because people may<br>sex|1.168|1|4|2|4|4500|
|||If a woman wants to have sex 2.835<br>without a condom, the man<br>must not refuse|1.176|1|4|2|4|4525|
|||Only women are responsible 3.516<br>for unwanted pregnancies|0.931|1|4|3|4|4514|
||Respondent is confident of re|questing that a condom be used 1.552|0.716|0|2|1|2|4533|
||Respondent is willing to dela|y sex if condoms are unavailable 0.678|0.467|0|1|0|1|4518|
||Respondent thinks he/she w<br>sexual relationship.|ill use a condom in his/her next 0.805|0.396|0|1|1|1|4438|
|Sexually<br>conservative|Respondent thinks that:|It is not right when people of 3.683 <br>their age have sex with|0.729|1|4|4|4|4520|
|attitudes index||several partners in the same|||||||
|variables||month|||||||
|||People of their age should<br>3.395 <br>wait to have sex|0.904|1|4|3|4|4544|
**Table S1.1.** Continued.

|**INDEX**|**INDIVIDU**|**AL variables**|**MEAN**|**SD**|**MIN**|**MAX**|**Q25**|**Q75**|**N**|
|---|---|---|---|---|---|---|---|---|---|
||Respondent’s answer to:|Age at which women should<br>start having sex|19.577|3.296|10|30|18|20|4501|
|||Age at which men should<br>start having sex|18.449|3.248|10|30|16|20|4509|
||Respondent is confident he/s<br>tionally ready|he will have sex only when emo|- 1.411|0.776|0|2|1|2|4525|
|Sexual abuse<br>reporting<br>attitudes index|Respondent disagrees with th<br>olence the person must not te|e idea that in case of sexual vi-<br>ll anyone.|0.983|0.131|0|1|1|1|4481|
|variables|Respondent thinks that when|Must tell his/her family|0.713|0.452|0|1|0|1|4502|
||a teenager is suffering from|Must tell the authorities|0.741|0.438|0|1|0|1|4502|
||sexual violence he/she:|In case of rape, must seek<br>medical help|0.596|0.491|0|1|0|1|4502|
|||Must tell someone such as<br>teachers, friends, etc.|0.021|0.144|0|1|0|0|4502|

_Source_ : Authors’ analysis from student surveys.

_Note_ : Variables with range [1–4] are coded as: totally agree, almost agree, sort of disagree, and totally disagree. Variables with range [0–2] are coded as: 0 – not so sure, 1 – almost sure, 2 – totally sure.
**Table S1.2.** Attrition

||**One week**|**Six months**||**One week**|**Six months**||**One week**|**Six months**||**One week**|**Six months**||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
||**post inter-**|**post inter-**|**Condom**|**post inter-**|**post inter-**|**Condom**|**post inter-**|**post inter-**|**Condom**|**post inter-**|**post inter-**|**Condom**|
|**Dep. Var.: Attrited= 1 **|**vention**|**vention**|**voucher**|**vention**|**vention**|**Voucher**|**vention**|**vention**|**Voucher**|**vention**|**vention**|**voucher**|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|**(6)**|**(7)**|**(8)**|**(9)**|**(10)**|**(11)**|**(12)**|
|Treatment students|0.009|0.012|−0.008||||||||||
||(0.020)|(0.019)|(0.040)||||||||||
|Spillover students|0.013|0.024|0.045||||||||||
||(0.024)|(0.018)|(0.046)||||||||||
|Treatment students *||||0.004|0.005|−0.001|||||||
|socioeconomic level||||(0.007)|(0.008)|(0.016)|||||||
|Spillover students *||||0.008|0.008|0.015|||||||
|socioeconomic level||||(0.010)|(0.008)|(0.019)|||||||
|Treatment students * father’s|||||||0.001|0.001|−0.000||||
|education|||||||(0.002)|(0.001)|(0.003)||||
|Spillover students * father’s|||||||0.001|0.001|0.003||||
|education|||||||(0.002)|(0.001)|(0.003)||||
|Treatment students *||||||||||0.001|0.001|−0.001|
|mother’s education||||||||||(0.001)|(0.001)|(0.003)|
|Spillover students * mother’s||||||||||<br>0.001|<br>0.001|<br>0.003|
|education||||||||||(0.002)|(0.001)|(0.003)|
|Constant|0.126***|0.100***|0.313***|0.115***|0.098***|0.304***|0.117***|0.150***|0.333***|0.134***|0.143***|0.393***|
||(0.016)|(0.014)|(0.030)|(0.017)|(0.019)|(0.035)|(0.019)|(0.029)|(0.040)|(0.023)|(0.024)|(0.047)|

_Source_ : Authors’ analysis from student surveys.

_Note_ : *** _p <_ 0.01, ** _p <_ 0.05, * _p <_ 0.1. Attrition = 1 for students observed at baseline but not at first follow-up (column 1), second follow-up (column 2), or without working cellphone or email for voucher offer 6 months after intervention (column 3). Columns 4–6 include socioeconomic status variable, columns 7–9 include father’s education variable, and columns 10–12 include mother’s education variable as controls. Standard errors clustered at the school level in parentheses.