metadata: authors: 0: author: Barrera-Osorio, Felipe
1: author: Bertrand,Marianne
2: author: L. Linden, Leigh
3: author: Perez-Calle, Francisco
count: Colombia
docna: 0: docna: Conditional cash transfers in education :
            design features, peer and sibling effects evidence from a
            randomized experiment in Colombia
docty: Policy Research Working Paper
subtopic: Disability,Educational Sciences,Services & Transfers to Poor,Gender and Development,Economic Assistance,Inflation,Access of Poor to Social Services,Inequality
teratopic: Finance and Financial Sector Development,Education
abstracts: cdata!: This paper presents an evaluation of
            multiple variants of a commonly used intervention to boost
            education in developing countries - the conditional cash
            transfer - with a student level randomization that allows
            the authors to generate intra-family and peer-network
            variation.  The analysis tests three treatments: a basic
            conditional cash transfer treatment based on school
            attendance, a savings treatment that postpones a bulk of the
            cash transfer due to good attendance to just before children
            have to re-enroll, and a tertiary treatment where some of
            the transfers are conditional on students' graduation
            and tertiary enrollment rather than attendance.  On average,
            the combined incentives increase attendance, pass rates,
            enrollment, graduation rates, and matriculation to tertiary
            institutions.  Changing the timing of the payments does not
            change attendance rates relative to the basic treatment but
            does significantly increase enrollment rates at both the
            secondary and tertiary levels.  Incentives for graduation
            and matriculation are particularly effective, increasing
            attendance and enrollment at secondary and tertiary levels
            more than the basic treatment.  There is some evidence that
            the subsidies can cause a reallocation of responsibilities
            within the household.  Siblings (particularly sisters) of
            treated students work more and attend school less than
            students in families that received no treatment.  In
            addition, indirect peer influences are relatively strong in
            attendance decisions with the average magnitude similar to
            that of the direct effect.
geo_reg_key: 119228,517191
repnme: repnme: Conditional cash transfers in education :
            design features, peer and sibling effects evidence from a
            randomized experiment in Colombia
display_title: Conditional cash transfers in
            education : design features, peer and sibling effects
            evidence from a randomized experiment in Colombia
listing_relative_url: /research/2008/03/9362547/conditional-cash-transfers-education-design-features-peer-sibling-effects-evidence-randomized-experiment-colombia
topicv3: Poverty m1326687 695,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Education m1327048 470,Cash Transfers m1327437 1608,Gender m1327621 615
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Finance and Financial Sector Development,Education
no_of_pages: 57
author_keys: 0: author_key: 687244
model_extractions: WPS4580 Policy ReseaRch WoRking PaPeR 4580 imPact evaluation seRies no. 20 Conditional Cash Transfers in Education: Design Features, Peer and Sibling Effects Evidence from a Randomized Experiment in Colombia _Felipe Barrera-Osorio Marianne Bertrand Leigh L. Linden Francisco Perez-Calle_ The World Bank Human Development Network Education Team March 2008
Policy ReseaRch WoRking PaPeR 4580

# **Abstract**

This paper presents an evaluation of multiple variants of a commonly used intervention to boost education in developing countries – the conditional cash transfer – with a student level randomization that allows the authors to generate intra-family and peer-network variation. The analysis tests three treatments: a basic conditional cash transfer treatment based on school attendance, a savings treatment that postpones a bulk of the cash transfer due to good attendance to just before children have to re-enroll, and a tertiary treatment where some of the transfers are conditional on students’ graduation and tertiary enrollment rather than attendance. On average, the combined incentives increase attendance, pass rates, enrollment, graduation rates, and matriculation to tertiary institutions.

Changing the timing of the payments does not change attendance rates relative to the basic treatment but does significantly increase enrollment rates at both the secondary and tertiary levels. Incentives for graduation and matriculation are particularly effective, increasing attendance and enrollment at secondary and tertiary levels more than the basic treatment. There is some evidence that the subsidies can cause a reallocation of responsibilities within the household. Siblings (particularly sisters) of treated students work more and attend school less than students in families that received no treatment. In addition, indirect peer influences are relatively strong in attendance decisions with the average magnitude similar to that of the direct effect.

This paper—a product of the Education Team, Human Development Network—is part of a larger effort in the department to document the impact of educational programs. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at fbarrera@worldbank.org, or (corresponding author), leigh.linden@ columbia.edu _The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank’s Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Conditional Cash Transfers in Education: Design Features, Peer and Sibling Effects Evidence from a Randomized Experiment in Colombia**<sup>**1**</sup>

**Felipe Barrera-Osorio** (World Bank) **Marianne Bertrand** (Chicago GSB) **Leigh L. Linden** (Columbia University) **Francisco Perez-Calle** (Ministry of Education, Colombia)

# **Keywords:** Education, Conditional Cash Transfer, Family, Peer Effects

1 An undertaking of this magnitude requires the assistance of many individuals. We are most indebted to the Secretary of Education of Bogota for cooperating with us in this novel experiment, putting up with the constraints created by the research effort, and, of course, financially supporting the entire project. Fedesarrollo, the think tank for which Barrera-Osorio and Perez were working at the execution of the project, provided financial support as well and helped the SED in the design and implementation of the program. While everyone at the SED has been extremely helpful we are particularly indebted to Abel Rodriguez, Catalina Velasco and Margarita Vega. We are indebted to Silvia Restrepo of Fedesarrollo for the logistical assistance and for the data collection. Camilo Dominguez has done an excellent job as a research assistant during the entire project, and we thank Carlos Ospino and Lucas Higuera for their help at key points in the effort. We thank Sendhil Mullainathan and Mario Sanchez for their comments and assistance, and thank the seminar participants at the World Bank’s Human Development Network, Columbia University’s Department of Economics, NBER Summer Education Meeting, Rutgers University’s Department of Economics, New York University’s Robert F. Wagner School of Public Service, the LACEA Impact Evaluation Network and the CIPREE/BREAD Conference, and the 2008 CIES Conference for their helpful question and comments. The views expressed in this document are solely those of the authors and do not reflect the views of the World Bank or the Colombian Ministry of Education. All errors are of course (and unfortunately) our responsibility. Please send correspondence to Leigh Linden at <u>leigh.linden@columbia.edu.</u>
**JEL Codes:** I22, J13, I28

- 2 -
# **I. Introduction**

Education plays an important role in the development process. At both the macro (for example, Krueger and Lindahl, 2001) and micro level (Angrist and Krueger, 1991; Duflo, 2001, among others), there is strong evidence that education generates higher levels of both income and growth. As a result, developing countries could contribute substantially to future income growth by increasing school attendance and graduation rates. The challenge, however, is getting the kids in school. For example, the net enrollment rate in primary education in 2004 in SubSaharan Africa, Oceania and Western Asia was 64, 80 and 83 percent respectively. Problems are more pronounced with girls, low income families and older children (United Nations, 2006).

Despite the importance of education, we are still far from understanding what determines whether or for how long children are educated. The classic model postulates a simple comparison of the future returns of additional schooling to the short-term direct costs of enrollment and the opportunity costs of the time required to attend. And while it is clear that even this simple relationship is difficult to estimate rigorously, more recent models suggest that liquidity constraints, family dynamics, peer influences, or even commitment issues, can also influence the education decision process among children and their parents.

A large and growing literature has begun to empirically evaluate the various determinants of the schooling decision. For example, one would expect that students should respond to the quality of education, especially in lower income countries where average quality is often quite poor (Pritchett, 2004). However, in the short-term at least, improving quality does not seem to be a major inducement: interventions proven to improve the quality of education generate few changes in participation levels (Banerjee, Cole, Duflo, and Linden, 2007; He, Linden, MacLeod, 2007; Muralidharan and Sundararaman, 2006). On the other hand, interventions that directly change the cost of attending school do seem to work. Families respond to direct reductions in the cost of education either through subsidies to attend private schools (Angrist et al 2002, 2006), reduced user-fees (Barrera, Linden, Urquiola, 2007) or scholarships (Kremer, Miguel, and Thornton, 2007).<sup>2</sup> Families also respond to direct inducements to attend such as school meals and, our focus in this paper, direct cash incentives (Vermeersch and Kremer, 2005; Schultz 2004;

> 2 Health is also a major factor in determining school attendance (Miguel and Kremer, 2006; Bobonis, Miguel, and Sharma, 2006).

- 3 -
Glewwe and Olinto, 2004; Schady and Araujo, 2006; Schultz, 2004; Attanasio et al, 2006 among others).

While direct cash incentive programs have been proven effective, there is much still to learn about best ways to design these programs as well as about their possible indirect effects on siblings and peers. To date, most programs follow a design that is similar to that used in the popular Mexican Conditional Cash Transfer (CCT) program previously known as PROGRESA and now OPORTUNIDADES. Under this program, students are paid on a monthly or bi-monthly basis for meeting a specified attendance (usually 80 percent per month) or enrollment target. In this research, we first consider a variant of that program that aims to relax families’ possible savings constraints, such as those due to commitment problems (see for example, Ashraf, Karlan, and Yin, 2006) or imperfect saving institutions. Such savings constraints might be especially relevant in the education context because of the large expenses families face at the beginning of a new school year. We also consider a second variant of the CCT program that proposes to incentivize directly students’ and their families’ decisions regarding graduation and tertiary enrollment rather than just attendance.

Because our research design generates exogenous variation in exposure to the CCT programs within family and friendship networks, it also allows us to assess whether one child’s exposure to the cash incentives has any indirect effects on his (or her) siblings and peers. Even if not eligible for the CCT program, both siblings and peers might indirectly benefit and increase their demand for schooling because of social network effects. Siblings might also indirectly benefit because of the additional resources the CCT program is bringing in their household. However, if resources are not equally allocated within the household (see for example, Blundell, Chiappori, and Meghir, 2005 or Oster, 2007), the benefits to ineligible siblings might be limited; worse, parents may decide to concentrate most their educational investments in the children that are eligible for the cash incentives, possibly leading to adverse outcomes for those that are not eligible. We view the study of such indirect effects as especially relevant in light of the financial constraints many developing countries face when considering implementing CCT programs, and the likely need to limit eligibility to certain age groups or most at-risk (to drop out) groups.

Specifically, we evaluate 3 conditional cash transfer treatments in a large municipality: Bogota, Colombia. We first use the basic treatment implemented in a manner very similar to PROGRESA. The second treatment, called the “savings treatment,” varies the timing with

- 4 -
which the funds are distributed to families, distributing 2/3 of the funds immediately and the remaining funds at the time the students enroll in school. The third treatment, called the “tertiary treatment,” provides children with the same lower monthly subsidy as the savings treatment, but also pays a large subsidy that incentives both graduation and matriculation to an institution of higher education.

To allocate these treatments, we use an over-subscription model rather than the basic geographic allocation strategy used in most previous studies.<sup>3</sup> We staged a large recruitment drive and randomly allocated about 10,000 treatments to about 17,000 registered children. This model allows us to randomize at the child-level, generating variation within schools, families, and networks of friends. By pairing this randomization with detailed information on children’s siblings and friends, we are able to disentangle how these opportunities change the allocation of work in the household and the activities of the recipients’ peers.

Our research further makes two methodological improvements to previous studies of education-based CCTs. First, we collect attendance data through a series of school visits in order to avoid the self-reporting bias associated with the survey data used in most other studies of CCT models. Concerns about self-reporting biases are particularly important in this context. While subjects’ responses on these surveys have no implications for their participation or standing in the program, the subjects have been conditioned to value attendance and understand that their receipt of the transfers is determined by their rates of attendance. This could lead to a general upward bias in the reporting of attendance and could also lead to a differential bias by those most involved with the program – the treatment families. Second, we directly map students’ friendship networks, allowing us to assess directly the influences of peers on students’ attendance rates. Both of these methodological improvements prove to be important for our empirical analysis.

Our main findings are as follows. Taken together, all of the cash incentive treatments generate significant changes in the behavior of students directly treated by the program. Students are more likely to attend school (2.8 percentage points), more likely to remain enrolled (2.6 percentage points), more likely to matriculate to the next grade (1.6 percentage points), more

> 3 One exception is the system that the national Colombian government used to allocate school vouchers in the Program de Ampliacion de Cobertura de la Educacion Secunderia (PACES) program (see Angrist et al, 2002 and Angrist et al 2006).  While our use of this allocation strategy was a practical solution for conducting a student-level randomization, the intra-family and intra-friendship network variation enabled by this strategy have direct policy implications for this and other allocation mechanisms that partially families or groups of friends.

- 5 -
likely to graduate (4.0 percentage points), and more likely to matriculate to a tertiary institution (23 percentage points). For daily attendance, the effect is much stronger for students who would not have met the attendance target without the program.

However, the form of the incentive matters significantly. Simply changing the timing of the transfer with the savings incentive increases enrollment in both secondary and tertiary institutions over the basic treatment (by 3.6 and 3.3 percentage points respectively) while not reducing the daily attendance rates of students despite the lower monthly transfers. Compared to the basic treatment, the tertiary treatment encourages higher levels of daily attendance (3.5 percentage points more for students least likely to attend) and higher levels of enrollment at the secondary (3.3 percentage points) and tertiary levels (46 percentage points).

We also observe important spillover effects of the program within families and peer networks. Within families, the receipt of a subsidy causes a reallocation of academic opportunities and labor market responsibilities across family members, and this effect seems to interact with the families’ decision about which children to enroll in the lottery. Our findings are consistent with _negative_ spillovers of the program on the education of children that were registered but not selected for treatment by the lottery. For example, comparing households that registered two children, we find evidence of lower school attendance and more labor market work for an untreated child with a treated sibling compared to an untreated child with a similarly untreated sibling. This effect is particularly strong for unregistered girls. The results for unregistered children are more mixed.

In contrast, we find that the treatments generate strong positive externalities across peer groups. Treating friends encourages higher attendance at a similar magnitude as the direct effect. However, the gains in attendance decline sharply in the fraction of friends treated – one treated friend has significant benefits, but an additional friend has almost no additional effect.

The paper is organized as follows. First, we describe the educational system in Bogota, Colombia (Section 2). In Section 3, we describe the research design, including the design of the individual treatments, the allocation process, the various data sets, and the statistical models involved in the process. Section 4 presents verification for external validity, balance of the baseline and measures of attrition. We present the results of the analytical models in Section 5. Finally, we conclude in Section 6.

- 6 -
# **II. Education in Bogota**

Colombia is a relatively typical middle income, Latin American country. Child mortality is relatively low at 21 per 1000 births and individuals can expect to live long lives -- life expectance at birth is 72.6 years. The per capita income of Colombia is US$ 2,020, with only 17.8 percent of the population living on less than two dollars per day (World Bank, 2006).

While the central government maintains control of curriculum as well as of the allocation of teachers and their wages, municipalities are primarily responsible for the administration of public education using national funds. The central government provides resources to municipalities, primarily from income and VAT taxes, and close to 90 percent of these funds are required by law to go toward health and education. With these funds, municipalities must develop, maintain, and run the facilities in their jurisdictions. Municipalities that have greater capacity to collect and administer taxes supplement central resources with local funds, usually from property taxes.

The academic year runs from the end of January until the middle of November. The system is divided into three categories: basic primary (grades one through five), basic secondary (grades six through nine) and middle secondary (grades ten and eleven). After finishing the eleventh grade, children can matriculate to either traditional universities or one of many vocational schools. Students usually start school at five to seven years of age, and children are legally required to attend school through the ninth grade, a period referred to as basic education.

Like in most urban areas in middle-income countries, school attendance is highest for younger children. The enrollment rate for students of age between 5 and 13 are close to 100 percent. After 13 years old, the attendance rate starts to decline. The average attendance rate for individuals aged 15 is 92 percent, 16 is 90 percent and 17 is 80 percent. The drop is faster for low-income individuals. For individuals falling into the bottom two categories of the Colombian poverty index (the SISBEN), the attendance rate for 15 year olds is 84 percent, for 16 year olds is 80 percent and for 17 years olds is 65 percent (Fedesarrollo, 2005). Overall, there were 89,000 between 5 and 18 years of age who were not attending school in 2003. Seventy-four percent of these were classified in the bottom two categories of the SISBEN (Fedesarrollo, 2005).

When surveyed, students claim that the major reason for dropping out is the cost of education. Students have to pay to enroll each year; they also have to pay for required items like

- 7 -
uniforms, books, and supplies. In fact, 64 percent of dropouts claim that the high cost of education is the main reason for leaving school (Fedesarrollo, 2005). Enrollment fees, uniforms, and school materials make up 90 percent of the costs for low-income individuals, and these monthly costs fluctuates between 24,000 and 50,000 pesos depending on the school and grade (US$ 13 to US$ 22), a relatively large expense considering that the poorest families in Bogota earn less than US$ 750 a year.

# **III. Research Design**

In 2005, the city of Bogota established the Conditional Subsidies for School Attendance (“Subsidios Condicionados a la Asistencia Escolar”) program in an effort to improve student retention, lower drop-out rates and reduce child labor. In an effort to improve the program over the basic conditional cash transfer model, the Secretary of Education of the City (Secretaria de Educacion del Distrito, SED) decided to implement a pilot study in two of the twelve localities in the city. The pilot was to run for a year, and then the results would be used to inform the design of the final program that would operate city-wide.

# **A. Design of Treatments**

Ultimately, three interventions were chosen for the pilot. First, operating as a reference is a basic intervention similar to that used in PROGRESA/OPORTUNIDADES. In this basic model, participants would receive 30,000 pesos (approximately US$ 15) as long as the child attended at least 80 percent of the days that month. Based on the responses to our surveys, the total annual value of the transfer (300,000 pesos) is three times more than what students report earning on average and is slightly more than the average 250,000 pesos that families report spending each year on educational expenses. The payments would be made bi-monthly through a dedicated debit card run by one of the major banks in Colombia. Students would be removed from the program if they failed to matriculate to the next grade twice, failed to reach the attendance target in two successive bi-monthly periods, or were expelled from school. Finally, all payments were based on reports provided to the Secretary of Education by the students’ principals.

- 8 -
The two additional treatments were experimental variants of this basic intervention aiming to better reach the goals of the program while keeping the cost of each intervention roughly equivalent to the basic intervention.<sup>4</sup> Based on research that suggests that families may face difficulties saving money for students’ education (either because of intra-household bargaining, personal discounting issues, or simply high costs of savings), the second treatment (savings treatment) varied the timing of the distributions to students’ families. Instead of receiving 30,000 pesos a month for reaching the attendance target, students were paid two thirds of this amount on a bi-monthly basis (20,000 pesos or US$10) and the remaining third was held in account. The accumulated funds were then made available to students families during the period in which students enroll and prepare for the next school year. If students reached the attendance target every month, this treatment would make 100,000 pesos (US$ 50) available to them in December.

Keeping the overall cost of the intervention roughly constant, this treatment differs from the basic intervention with respect to both short-term liquidity constraints and technology to save for longer-term goals. First, because the monthly transfer is reduced, children may attend less often if they face very immediate constraints on school participation (trading off time spent in school with time spent at work, for example). Second, it supplies the accrued funds to families just before they enroll in the next academic year. So, if families’ long-term savings constraints are more significant for children’s academic participation than the more short-term liquidity constraints, the savings treatment could generate both higher attendance and higher re-enrollment rates when compared to the basic treatment.<sup>5</sup> Rather than manipulate the timing of payments, the third treatment (tertiary treatment) changes the outcome students are being incentivized upon. Instead of providing an incentive to attend school, this treatment provides an incentive to graduate and then to matriculate to a higher education institution. Like in the savings treatment, in the short term, the monthly subsidy is reduced from 30,000 pesos per month to 20,000 pesos. However, upon graduating the students earn the right to receive a transfer of 600,000 pesos ($US 300), amounting to 73 percent of the

> 4 The amounts, of course, are not the same because the treatments do not account for inflation.  Making adjustments to account for inflation probably would have been too complicated to explain to potential registrants. However, the inflation rate in 2005 was only 4.85%, and the net effect of this difference is to reduce the value of the savings treatment which, we will show, is more effective than the basic treatment despite the slightly lower value. 5 This effect would be similar to the effect observed in Duflo, Kremer, and Robinson (2006) in which simply offering farmers the option to buy fertilizer at harvest time, when money was available, significantly increased the purchase of fertilizer.

- 9 -
average cost of the first year at a vocational school (823,000 pesos or $US 412). If the student graduates and enrolls in a tertiary institution, they receive the transfer immediately; if they fail to enroll, they can only request the transfer after a year has passed. It is important to note, however, that unlike the savings treatment, the tertiary treatment does more than just change the timing of the payments to families; the total value of the tertiary treatment for students in each grade is greater than the equivalent value of the basic treatment.

Compared to the basic treatment, this tertiary treatment could reduce attendance rate if students’ short-term liquidity constraints are important (because of the lower monthly transfer – as in the savings treatment). However, if short-term liquidity constraints are not binding, the tertiary treatment could stimulate higher graduation rates and possibly attendance rates (if attendance is viewed as a relevant input into graduation), and could also result in higher levels of matriculation to tertiary institutions.

# **B. Structure of Randomization**

As required by the SED, the assessment of the treatments was divided into two separate experiments located in two very similar localities in Bogota, San Cristobal and Suba. Eligible registrants in San Cristobal were randomly assigned between a control group, the basic treatment, and the savings treatment. Eligible registrants in Suba would be assigned either to a control group or to receive only one of the subsidies, with those who had last completed grades six through eight receiving the basic treatment and those who had last completed grades nine through eleven receiving the tertiary treatment. This research design allows us to directly assess the causal impact of each treatment; it also allows us to directly compare the savings and basic treatments. But it requires us to be careful and ensure the comparability of the localities before comparing the effects of the tertiary treatment to the other treatments.

Both experiments were based on an over-subscription model. The city guaranteed enough funds to provide 9,732 students with the subsidies, 6,875 in San Cristobal and 2,857 in Suba, for three years.<sup>6</sup> To participate, a publicly advertised registration process would be held

> 6 Originally, 10,000 subsidies were proposed, but 268 subsidies were used as part of a separate program design to facilitate the reenrollment of students who had previously dropped out of school.

- 10 -
and if there were more interested children than subsidies, then the subsidies would be allocated to children based on a lottery in each locality.<sup>7</sup> During January and the beginning of February, the program was advertised in the two localities through posters, newspapers ads, radio spots, loudspeakers in cars, churches, and community leaders, including principals of schools and priests. Potential candidates for the subsidy were registered during 15 days between the end of February and the beginning of March 2005. The registration was conducted in various schools of the two localities. In order to be included in the program, at least one parent / guardian was required to be present at the registration.

In order to be eligible for the program, children had to meet several criteria. First, the potential candidate had to have finished grade 5 and not yet graduated from grade 11. To focus on lower income families, all children’s families had to have been classified into the bottom two categories on Colombia’s poverty index, the SISBEN.<sup>8</sup> To verify the classification, the student had to present an identification card (which the vast majority of students have). The SISBEN categorization of the household was confirmed online by the SED at the time of registration. In order to eliminate the possibility that families would move to take advantage of the program, only those households that had been classified by the SISBEN system as living in San Cristobal or Suba prior to 2004 were eligible to participate in the program. In total, 17,309 eligible students were registered for the two experiments: 10, 947 in San Cristobal and 6,362 in Suba.

The randomization was publicly conducted on April 4 in each locality. The research team conducted the actual lottery, but in order to ensure transparency of the process, the code was inspected prior the exercise by researchers from the National University. The randomizations were done publicly (projecting the code onto a screen), with representatives of the community, school and local authorities present. The lists of beneficiaries were immediately printed, signed by local officials, and made available to the communities so that parents were able to determine if their children were included.

The randomization was stratified on locality, type of school (public / private), gender, and grade level. Panel A of Table 1 shows the distribution of registrants. In all, 6,875 students from

> 7 The over subscription and recruitment process are similar to the techniques used in the assignment of school vouchers in the PACES program implemented nationally in Colombia.  This process is described in Angrist et al. (2002).

> 8 See Vélez et al (1999) for details for the description of SISBEN.   The SISBEN classified households according to 6 levels, 1 being assigned to the poorest.  Most of the families in these areas were surveyed in 2003 and 2004.

- 11 -
San Cristobal and 2,857 from Suba received one of the treatments. This left 4,072 control students in San Cristobal and 3,505 in Suba, and the students are evenly distributed within gradegender categories.

# **C. Data**

The richness of the available data is one of the major strengths of our study. The data come from six sources. These include general survey data on all eligible families, data collected specifically for the study, and administrative data collected by the SED.

First, we have the data from the original SISBEN surveys from 2003 and 2004 that contain information on all families eligible to register for the lottery. These surveys were conducted as part of the SISBEN national poverty index – in fact, these are subsets of the actual surveys that were used to create the index itself. We have access to all individuals placed into the bottom two SISBEN categories, providing a rich baseline description of all of the eligible families. It also allows us to verify the representativeness of our results by checking that those families who registered for the study were not significantly different from those that did not register. The SISBEN data provide us with several variables at the family level such as schooling level of the household head, physical characteristics of the dwelling, employment status of adults, and family income. It also provides us with individual level variables such as enrollment status at the time of the survey, age, income, and marriage status.<sup>9</sup> The second source of data comes from the program registration process itself. During this process families had to provide some basic information on the students to ensure eligibility. These data include birth date, gender, last grade completed and year in which that grade was completed. Most of this information was verified through the actual SISBEN database and when possible, the SED’s official records.

After the randomization, it became clear that students were spread across a large number of schools, but the density was heavily skewed with the majority of students in a smaller number

> 9 The obvious challenge of using this data is that families knew that they were being surveyed for the purpose of scoring them on a poverty index.  As result, measures of assets and income are probably underestimates of the true values.  However, this bias is almost certainly not correlated with the differences investigated in this paper given the timing and purpose of the survey.  We use this information for two primary purposes.  First we use it to compare registrants to non-registrants, and second we use it as a source of information on the households to which the children in the study belong.

- 12 -
of schools. Based on the available budget, we chose to collect baseline data and the subsequent attendance data in only the 68 schools with the largest number of registered children. This included a total possible sample of 9,768 students. These individuals were chosen from a list of students and the names of the schools that they provided to the SED.

The baseline was conducted between May and July, 2005 and comprised a simple selfadministered survey that the students filled out in class. Of the 9,768 students selected for surveying we were able to locate 9,239 students at the time of the baseline survey in the schools that they claimed to attend. The distribution of these students is provided in Panel B of Table 1. Reassuringly, they have a similar distribution to original registrants and again, are equally distributed within grade-gender categories.

Because the baseline was conducted after the randomization, we were unable to use information on any variables that might have changed immediately as a result of the treatments. The baseline instead allows us to narrow down the sample to those children whose provided information was correct and that we could feasibly track down at the end of the study. From the baseline, we use the following: basic demographic variables, a list of friends the students have of the same grade in school, and most importantly, contact information for tracking students during the follow-up survey.

As a fourth source of data, the research team collected during the last quarter of 2005 data on students’ attendance through direct observation. For this purpose, the team assembled a group of assistants who randomly visited schools and classes. The assistants directly called the roll of all students and students were marked absent if they were not physically present in the classroom. They visited a total of 1,069 classes in the 68 selected schools for 13 weeks, targeting the same 9,768 students originally chosen for the baseline survey. Because we were able to continue looking for all children selected from the 68 schools, this data set is broader than that used for the detailed survey questionnaires as it includes both those students who were found in the baseline and students who, for whatever reason, were not available to be surveyed.

During February and March of 2006 a follow-up survey was conducted. To ensure that the survey did not preferentially treat students still enrolled in school, we conducted the survey at the household level. For the follow up, the research team located the families of 98.14 percent of the baseline individuals – a total of 8,736 students. The survey is a rich source of information, containing data on the participating students (including academic participation, academic effort;

- 13 -
consumption, and labor activities) but also the other children in the household, thereby allowing us to study how the treatments may have affected the allocation of work and resources within families.

Finally, we obtained administrative records from the Secretary of Education that includes the enrollment records of every child in a public school and many private schools in the two localities. This data allows us to assess the effect of the treatments using every student that registered for the randomization, including those not claiming to attend one of the 68 schools selected for attendance data collection and surveying. Combined with the other outcome variables, this provides us with three concentric groups of students: all of the students who registered for the randomization (for which we only have administrative enrollment data), all of the students registered at the 68 schools selected for surveying (for which we have both administrative enrollment data and verified attendance data), and those students at the 68 schools who completed baseline and follow-up surveys (for whom we have administrative enrollment data, verified attendance data, and the information collected in the surveys).

# **D. Analytic Models**

We use three basic models to analyze the data. First, we use a simple difference estimator. Second, we also use a difference estimator that includes controls for individual and family characteristics. Third, we use an instrumental variables model to estimate externalities generated by the treatments within families and students friendship networks.

First, in order to validate the randomization, we use a simple difference model to make simple comparisons between different subsets of the sample without controlling for any covariates. These comparisons are intended to assess the comparability of different groups such as the research groups, registrants and non-registrants, etc. When used to compare a given treatment and the respective control group, for example, the specification takes the following form: To perform this estimate, the data sets containing the treatment group of interest and the respective control group are pooled. The variable<sup>_x_</sup> _ij_<sup>represents a particular characteristic of</sup>

- 14 -
interest for child _i_ in school _j_ . This is regressed on the variable _Treati_ which is an indicator variable for whether or not the individual child is in the respective treatment group. The error variable ε is indexed with both student and school identifiers because the error terms are _ij_ allowed to co-vary up to the school level. Finally, the variable β 1 is the estimated difference.

To estimate the effects of the various treatments we use a difference estimator as well, but also include controls for demographic and school characteristics. This model is specified as follows for San Cristobal:

> The variables from Equation 1 are defined as before.  The variable _yij_ is the outcome variable of interest.  Next, we include two treatment variables that are indicator variables for the specified child receiving the basic and savings treatments, respectively.  The coefficients on these indicator variables are the estimates of the effects of the respective treatment.  The main difference between this specification and Equation 1 is that this includes as control variables

> demographic characteristics _X ijk_ at the child and family ( _k_ ) level as well as fixed effects for each school, φ _j_ .  We again allow the error terms to co-vary up to the school level.  For Suba, we use a similar equation that contains only one treatment dummy and estimate the model for grades 6-8 and 9-11 separately.

In addition to the direct estimates of the programs, we also estimate the external effects of the treatment on students’ family members and peers. For these specifications we are interested in the relationship between the individuals’ behavior and either the fraction of peers treated or the fraction of school-aged family members treated. To do this, we have to account for the fact that the fraction of registered peers or family members is possibly endogenous. As a result, we use an instrumental regression model in which the fraction of treated peers or school-aged family members is instrumented with the fraction of registered peers or family members who receive the treatment. For the friendship networks, the specification takes the following form: 2 _yij_ = β _o_ + β 1 _Frac_ Treat_ + β 2 _Frac_ Treat_ + β 3 _Treat_ + δ _X ijk_ +φ _j_ + ε _ij_ (3) All of the variables are defined as before and β 1 and β 2 are the estimated effects of the fraction of friends treated by the program. For non-treated members of the family, we use a similar

- 15 -
specification except that we omit the school fixed-effects and cluster the standard errors at the family level.

Finally, we use one last specification to estimate what the attendance and enrollment rates of students who received the treatment would have been without the treatment.<sup>10</sup> We estimate these counterfactuals by modeling the behavior or students in our control groups using only the available baseline demographic characteristics. For treated students, we use their baseline characteristics and the coefficients from our regressions on the control group to project what these students’ would have done had they not been treated.

Specifically, we estimate the following model using only the registered children that did not receive the treatment: The model is estimated using ordinary least squares, and the coefficients and variables are the same as in Equation 2. The only exception, of course, is the omission of the treatment dummies. This equation highlights the fact that this proxy measure is only a linear combination of demographic variables, and it contains no new information.

# **IV. External validity, baseline balance and attrition**

We proceed as follows in this section. First, we use the available data from the SISBEN survey to compare the individuals that registered for the program to those who did not and to check comparability between the two localities. Second, for those individuals found at baseline, we compare the students assigned to each research group to ensure that the research groups are balanced at baseline. To make sure that the groups did not become unbalanced due to attrition, we then compare the distribution of students who failed to provide a follow-up survey in each research group. Once we have verified that the groups are indeed still balanced, we then estimate the results of the treatments on the various outcome variables in Section 5.

> 10 Ideally, we would have collected attendance rates of children prior to the randomization.  However, we could not have collected this information ourselves because, until the registration process was complete, we had no way of knowing which of the 515,885 eligible students would register.  We tried to collect historical attendance rates through the teachers’ records, but these records were too often incomplete and when complete, inconsistently kept.

- 16 -
# **A. External Validity**

One of the major problems of randomized evaluations is that, because they often focus on specific group of individuals, it remains unclear whether the results can be extrapolated to other populations. In our case, this is a particular concern given that students self-select into the registration for the program. However, through the SISBEN surveys, we have access to information on all eligible students living in the two localities, and we can directly compare students whose families registered them for the program to those that did not.

This comparison is presented in Table 2. Each row contains estimates for the indicated demographic variable. Columns 1 and 3 provide the average value for all registered children, and columns 2 and 4 provide the simple difference between registrants and non-registrants using Equation 1. While the size of the sample (515,885 children) is sufficiently large that most differences are statistically significant, they are all very small in magnitude except for those concerning school participation. Families have similar numbers of assets, similar household characteristics, and similar scores on the poverty indexes. Figure 1 shows the entire distribution for our income estimate and similar to the mean, the entire distributions of registrants and nonregistrants are comparable.

The main difference is school participation. On average, those registered for the program were more likely to have been attending school when the study was administered (19 and 20 percentage points). There are two reasons for this. First, this particular program targeted students who were already attending school. Second, a primary means of disseminating information about the program was through school principals. This is also born out in Figure 2 where we compare the families using our proxy attendance estimate. Registrants are significantly less likely to be children with similar characteristics to low attending children and much more likely to be similar to those with attendance rates close to 80 percent. The primary implication is that these results are most applicable to the students for which the interventions were targeted through the eligibility requirements: students who are currently enrolled in school and who have completed at least the fifth grade.

Finally, because students are eligible for the tertiary treatment only in Suba, we need to make sure that the students in Suba are similar to those in San Cristobal in order to compare properly the magnitudes of the treatment effects. This is done in columns 5 and 6. Column 5

- 17 -
provides a comparison of all eligible children and column 6 provides a comparison of just those children who registered for the lottery. In all cases, these children are very similar, making it reasonable to perform comparisons across localities.

# **B. Comparison at Baseline**

Given that the students who registered for the lottery are representative of all eligible children in the communities, we turn to checking whether or not the randomization succeeded in creating comparable treatment and control groups. This initial comparability is essential for us to be able to attribute future differences between the research groups to the respective treatments.

The comparisons for students who provided a baseline survey are presented in Table 3.<sup>11</sup> As in Table 2, each row displays the comparisons for the indicated demographic variable. Columns 1-4 compare students in San Cristobal and columns 5-8 compare students in Suba. In both localities, the differences are negligible. For San Cristobal, columns 2-4 display the simple differences (using Equation 1) between the basic treatment and the Control Group, the differences between the savings treatment and the Control Group, and finally, the difference between the two treatments, respectively. Almost all of the differences are statistically insignificant and those that are statistically significant (such as the fact that the basic treatment has 3 percent more girls than the savings treatment) are economically small.

The same is true for Suba. Columns 5 and 7 respectively show the average control group characteristics for the younger (grades 6-8) and older (grades 9-11) children, respectively. The younger children received the basic treatment, and those selected for the basic treatment are very similar to those in the control group (column 6). Similarly, the older children who received the tertiary treatment are similar to the older students who constitute the control group (column 8).

To check for differences in the distribution of children rather than just the mean, we also plotted the distributions. An example is shown in Figure 3. The figure contains a plot of the distribution of household income in the treatment and control groups – as shown in the plot, the distributions are very similar.

> 11 In un-presented results, we also make the same comparison for all students in the sample and for all students claiming to be registered at the 68 schools that were selected for the surveying and attendance collection.  The results for these samples are the same as for the data set of only the students providing a baseline survey.

- 18 -
# **C. Attrition from Baseline**

Comparability at baseline is critical, but even if the two groups are comparable at baseline, it is possible that the treatments might cause different types of students to drop out of the study, making the groups of students that answer our one-year follow up survey incomparable. To check for this, we perform two exercises. First, we check the overall attrition rates in each group. If these are sufficiently low, then compositions of the groups cannot significantly change from baseline to treatment even if significantly different types of students attrit. Second, to asses how different the attritors are, we compare the kinds of students attriting in each group using the baseline characteristics of all of the students.

The first two rows of Table 4 provide the exact number of attritors and their percentage in the research group. Column 1 shows the values for the control group and columns 2-4 show the difference from this value and between the two treatment groups for San Cristobal. Columns 5-8 do the same for Suba. Overall, the attrition rate is very low at just less than 2 percent, and the differences in the number of children who dropped out are mostly in the single digits. Given this extremely low rate of attrition, only very large differences could generate changes in the comparability of the research groups.

Panels B through E then estimate these relative comparisons of background characteristics. The control columns (columns 1, 5, and 7) show the difference in characteristics between those students that attrit and those that remain in the sample at follow-up. The difference columns (columns 2-4, 6, and 8) then display the results of a slight modification of Equation 2 to show the difference between the research groups of the relative differences between attritors and stayers.

Again, these differences are all minor. The vast majority of the differences are extremely small – for example, the differences in the families’ poverty measures are negligible both in economic and statistical terms. The largest differences occur in the age of the head of the household for San Cristobal (3 to 7 years difference), the age of children in San Cristobal (2-3 years), and the years of education of students in Suba grades 9-11 (1.25 years). Overall the distributions are very similar, and especially given the underlying low rates of attrition, the few differences that do exist are arguably too small to generate confounding changes in the measured outcomes.

- 19 -
# **V. Results**

# **A. Academic Participation**

The fact that the research groups are ultimately comparable allows us to causally attribute any changes in the groups at follow-up to the individual treatments. This allows us to assess families’ responses to the various programs by comparing directly the students’ who receive the treatments to the control group and to compare directly the different treatment groups. It is important to note, however, that within secondary school, different aspects of the treatments create separate incentives for attendance and enrollment. Within the academic year, the attendance targets encourage students to attend school regularly. Between academic years, however, the anticipated value of the transfers in the following year, encourage families to reenroll children in school.

Because the program started during the 2005 academic year and required that all students already be enrolled, we first consider attendance rates as measured by our team of attendance monitors during the last few months of the 2005 academic year (Figures 4 and 5 and Table 5). Since all of the treatments incentivize students to achieve the 80 percent attendance target, we first consider the aggregate effects of combined incentive schemes. The overall average effects of the treatments combined was to increase verified attendance at school by 2.8 percentage points which is statistically significant at the one percent level. The pooled effects of the treatment are graphically depicted in Figure 4 which contains a plot of a kernel density estimate of verified attendance for the treatment and control groups. Based on this graph, the treatment effect seems to operate by reducing the number of students who attend none<sup>12</sup> of the time or between 40 and 70 percent of the time and increasing the number of students who attend over 80 percent of the time.

Another way to look at the data is to plot actual attendance rates for each group verse our proxy baseline attendance rates, which is presented in Figure 5. Using a kernel weighted local polynomial estimator, we plot the relationship of actual measured attendance (on the vertical

> 12 It is important to note that students with a verified attendance rate of zero may have actually attended school at some point, but just not frequently enough to be caught during one of the visits (up to 13) conducted during the 2007 academic year.

- 20 -
axis) against the proxy attendance measure (on the horizontal axis). The effect is fairly uniformly distributed across families. Families who would have met the 80 percent target without the intervention seem to respond slightly less strongly than those who would not have met the target, but interestingly the program seems to make a difference even for students who would not seem to have a strong incentive to increase their attendance rates.

We can perform the same exercise using our administrative enrollment rates in 2006. Figure 6 depicts the results. Unlike the results for attendance rates, here the effect of the program seems to be largest for families least likely to have their children re-enroll in school. Students who would have been likely to re-enroll show no effect from the program while those with a probability of re-enrolling of less than 80 percent show an effect that is fairly evenly distributed across students.

Dividing up these effects to test separately for the individual effects of each treatment, we turn to Table 5 in which we investigate the effects of the individual programs on students’ attendance during the end of the 2005 academic year. Panel A contains the results for our primary outcome variable – verified attendance for all students in the 68 schools selected for surveying during the end of 2005. Panel B contains the same verified attendance measure but just for those students who were found in the baseline survey. Columns 1-3 provide the results for the first experiment in San Cristobal with column 1 providing the average for control students and columns 2 and 3 providing the results for the basic and savings treatments. Columns 4-7 provide the results for Suba. Columns 4 and 6 contain the results for the basic (grades 6-8) and tertiary (grades 9-11) treatments while columns 5 and 7 contain the respective controls. Finally, the last column contains the overall average effects by pooling the effects of the individual treatments.

Turning to Panel A, the individual treatments did cause changes in the verified attendance rates. The first row contains the average results for all students. The second row contains students in grades 6-8 and the third row contains grades 9-11. Overall the basic and savings treatments increase attendance in San Cristobal by 3.3 and 2.8 percentage points respectively. The results are evenly distributed across grades. For grades 6-8 in San Cristobal, the savings treatment increases attendance by a similar amount as the basic treatment (3.5 and 2.6 percentage points respectively), despite the lower monthly transfer. Interestingly, the basic transfer has no effect on attendance in Suba, though if we pool the sample the estimated effect is a statistically

- 21 -
significant 2.7 percentage points (p-value < 0.001). And although the point estimate is then lower than the estimated effect of the savings treatment, the difference is not statistically significant. For grades 9-11, the results are different. The results for the basic and savings treatments in San Cristobal are similar, but the results for the Suba experiment (the tertiary treatment) are an increase in attendance by 5.0 percentage points.

The next two rows divide students based on the predicted attendance measure estimated from Equation 3. (In other words, we divide the sample based on whether students’ baseline characteristics are similar to those characteristics of control group students who either met the attendance target or did not.) As shown in Figure 5, the treatment was slightly more effective for students whose projected baseline attendance was below the attendance target – the difference varies by treatment, although the difference is not statistically significant at conventional significance levels for any of the treatments. Only the basic treatment in Suba does not follow this trend, revealing a higher attendance effect for students likely to attend without receiving the treatment.

In Panel B, we move from all of the students in the 68 schools to just focus on those individuals who were found at school in the baseline survey. In the baseline, we attempted to survey every child who was registered as attending one of the 68 schools, and we were able to locate and survey 90 percent of those students at the time. Not surprisingly, the students who took the survey were, on average, higher attending students. In fact, as shown in Panel B, even those students who were projected to attend less than 80 percent of the time based on their household characteristics (the fifth row) would have met the attendance target on average without the subsidies.

As would be expected from the higher average attendance rates, the estimated treatment effects are slightly smaller for this sample of survey participants. Most of the point estimates are still statistically significant at conventional levels, but the average effects are around one percent rather than three percent for the entire sample.<sup>13</sup> The pattern of results, however, is similar to the pattern obtained when all of the students are included. The results are evenly distributed across grades, and those that are projected to attend less than 80 percent of the time show a stronger

> 13 Given the relatively small sample of students who did not give the baseline survey, we have not been able to estimate treatment effects for this sample consistently.  The estimates are very sensitive to the specification used, but the difference in treatment effects between students that did and those that did not give a baseline survey are never statistically significant at conventional levels.

- 22 -
treatment effect than students projected to attend more often. In this sample, the differences between students with high and low baseline projected attendance are statistically significant for the basic and savings treatments (p-value of 0.013).

Table 6 contains the estimated treatment effects for enrollment and school participation in 2006, the subsequent academic year. The first three panels contain estimates of students’ continued enrollment using administrative enrollment data provided by the Secretary of Education. Panel D and E contain self-reported enrollment and attendance data from our followup survey. To keep the outcome measures consistent, we only consider students who were in grades 6-10 in 2005 in Panels A through E. Panel F then contains self-reported enrollment and graduation data for students who were in the 11<sup>th</sup> grade in 2005 and should have matriculated to tertiary institutions in 2006. The columns are organized in the same format as in Table 5.

Panel A contains the estimated treatments effects, using our administrative enrollment measure, for all of the students who participated in the lottery (rather than just the 68 schools selected for surveying). The first row contains estimates for all of the students in the study. Overall, 70 percent of students in the lottery had paid their enrollment fees and enrolled in school in 2006.<sup>14</sup> However, the students who received the savings and tertiary treatments were significantly more likely to have re-enrolled in school than those who did not receive a treatment by 3.6 and 3.3 percent respectively. In contrast to the effects on attendance, only students receiving the basic treatment in grades 6-8 in Suba show a statistically significant effect while those in San Cristobal show no effect. And in San Cristobal where the comparison between the basic and savings treatments is the cleanest, the difference between the treatments is statistically significant at the 5 percent level (p-value 0.024). Thus, shifting the timing of payments does seem to significantly increase the fraction of students that re-enroll in school over the basic conditional cash transfer design.

The distribution of the result by sub-sample is similar to those of the verified attendance results, except that the difference between students projected to be likely or unlikely to reenroll is starker. Rows 2 and 3 show the effects by grade levels, and the effects are evenly distributed. The last two rows then divide the sample based on our estimated probability of re-enrollment.

> 14 This is likely an under-estimate of the number of students who continue to be enrolled both because of matching errors and because the data provided by the SED for some schools (mostly private) were reportedly incomplete. However, given our within school variation in treatment assignment, none of the research groups should be disproportionately affected by these issues.

- 23 -
As graphically exhibited in Figure 6, students with household characteristics correlated with low re-enrollment probabilities are much more likely to enroll due to the treatments. The savings treatment causes a 5.6 percentage point increase in reenrollment and the tertiary treatment causes an 8 percent increase. Those students most likely to re-enroll show no effect at all. This difference is statistically significant at the five percent level (p-value of 0.045) for the savings treatment and the ten percent level (p-value 0.06) for the tertiary treatment.

Because we have data for all of the students in our sample, we can assess the comparability of our sample of 68 schools by re-estimating the treatment effect just on those 68 schools and comparing the estimates to those obtained from the entire sample. The estimates from the students initially enrolled in the 68 schools are provided in Panel B. While the baseline re-enrollment rates are higher in these schools,<sup>15</sup> the estimated treatment effects are very similar to those in Panel A.

Panel C contains the estimated treatment effects for students in the 68 schools who gave a baseline survey, repeating the exercise from Panel B of Table 5 using the enrollment data. While the point estimates for the savings, tertiary, and basic treatments in Suba all have positive point estimates, the magnitudes are again smaller than for the full sample, but not statistically significantly so. Those students least likely to reenroll show the largest response to the treatments. For these students, the savings treatment causes an increase in enrollment of 4.3 percentage points and the tertiary treatment causes an increase of 4.8 percentage points, statistically significant at the five and ten percent levels respectively.

Unlike the previous panels, Panel D contains the same estimates, but with our selfreported enrollment measure from the follow-up survey. The results demonstrate the importance of not relying solely on self-reported data to assess incentive programs. Comparing these estimates to those in Panel C, it seems that families significantly over estimate the enrollment rates of these students relative the administrative measure. This over-estimate is significant enough that almost all of the students report attending school. This then seems to create a ceiling effect that prevents us from detecting a treatment effect using the self-reported data.

Panel E also contains estimates using self-reported data but using the self-reported attendance measure for students’ attendance in 2006. Again, families seem to dramatically

> 15 These schools are all public schools.  As a result, the difference in reenrollment levels may due to the relative completeness of the data than to actual student behavior.

- 24 -
overestimate their children’s attendance rates. Even though some students have dropped out of school, the attendance rates (even the control students) are higher than the observed attendance rates in Panel B of Table 5. As with the self-reported enrollment data, we find no treatment effects. Had we relied on this self-reported measure, we would have significantly underestimated the effects of the program.

The cause of these inflated self-reported measures of enrollment and attendance is unclear. The other self-reported information we collected seems to conform much more closely to estimates obtained through our other data sources. They cannot be due to the incentives provided by the subsidy since these estimates are similarly high for treatment and control students. These results do, however, highlight the caution required when interpreting findings based on self-reported data. They also belie the argument that the exaggeration inherent in selfreported attendance data differences out when comparing treatment and control groups, at least when the reported rates are close to one.

Finally, in Panel F, we present our final academic participation estimates – graduation and tertiary enrollment estimates for students who where in grade 11 in 2005. Given the results in Panel D and E, it is important to exercise caution in interpreting these results. However, unlike the data presented in those panels, the self-reported graduation and tertiary enrollment levels seem much more reasonable. Looking at the control averages for the graduation rates, for example, 88 to 90 percent of eleventh graders report graduating – a rate that is slightly higher than the 85 and 96 percent average reenrollment rates presented in Panel C for students in grades nine and ten. It is also consistent with our estimates for the average number of students reporting that they passed in 2005 (Table 7, Panel A). Finally, the estimates are also consistent with the students reported activities, school fees, and labor market activities presented below in Table 8.

Turning to the estimated treatment effects, all of the estimated treatment effects are positive. However, not all of them are significant, most likely because the sample is much smaller than that used in the other panels. All treatments have positive point estimates for graduation, but only the estimated treatment effect in the pooled regression is statistically significant (10 percent level). For tertiary enrollment, the basic treatment generates a statistically insignificant increase of 4 percentage points, while the savings and tertiary treatments cause increases of 8.8 percentage points and 49.7 percentage points respectively, both of which are statistically significant at the 1 percent level even with the smaller sample.

- 25 -
Despite the numerous studies of programs designed to keep students enrolled in school, comparisons to other research is complicated by differences in methodology and the target populations of the programs. For example, most conditional cash transfer programs provide noneducation forms of assistance along with educational incentives (usually for nutrition or health care) and are targeted at more rural populations of students in primary and lower secondary school (usually grades 1-8). In addition, the majority of studies employs non-experimental evaluation strategies and focuses exclusively on primary and secondary enrollment. We know of no other study that measures the impact of incentive on matriculation to tertiary institutions or that is able to directly verify students’ attendance patterns.

Despite these differences, our estimates of the impact of the savings and tertiary treatments on secondary enrollment are a bit larger but of the same order of magnitude as those measured in other contexts. For example, within the context of the experimental evaluations, our estimates are comparable to those in Honduras (rural students aged 6-13, 1-2 percentage point increase in enrollment, Glewwe and Olinto, 2004), Ecuador (rural students aged 6-17, 3.5 percentage point increase, Schady and Araujo, 2006), and Mexico (rural students grades 1-9, boys 2.5 percentage points, and girls 3.5 percentage points, Schultz 2004). Attanasio et al (2006) assess, with propensity score matching, a similar program to ours (but one that also includes a nutrition component) run by the national Colombian government and find an increase in selfreported enrollment of 4.8 percentage points for urban children aged 14-17. However, these areas are much less developed than Bogota, and students have a baseline enrollment rate of only 65 percent.

The closest study to ours both in methodology and target population is the evaluation of the PACES program that provided vouchers to students entering the sixth grade. Over three years, Angrist et al (2002) find an increase in the percentage of students who completed the eighth grade of 10 percentage points (self-reported data) and over six years (Angrist et al, 2006), an increase in students completing secondary school of 15-20 percentage points (administrative data). Compounded over the relevant time-frame, these results are very close to our one-year enrollment effects for the savings and tertiary treatments assuming that the treatment effects do not decline over time.<sup>16</sup> While a more direct comparison will be possible as the students within

> 16 Unlike the PACES study, we are not yet able to assess the effects of continued enrollment on students’ learning levels.

- 26 -
our cohort age, this similarity is intriguing because at a cost of US$43 per student per year, the savings and tertiary treatments are less than a quarter of the cost of the PACES vouchers (US$190 per student per year).

# **B. Academic Effort, Consumption, and Labor Outcomes**

From students who completed both the baseline and follow-up surveys, we collected other outcome variables which are presented in Table 7. Because the results were relatively similar across grades, we pooled all of the grades, breaking out grade 11 when appropriate. Panel A displays the results for our measures of academic effort. Panel B contains the results for consumption outcomes. And finally, Panel C contains information on labor market activities for students in grades 6-10 and Panel D contains the same estimates for students in grade 11. The columns are defined as in Tables 5 and 6, and all estimates are again made using Equation 2.

In general, the treatments have little effect on our measures of academic effort in Panel A, except for the rate at which students pass. First, the tertiary treatment increases the time spent on homework by a half an hour a week. The other two treatments do not have any affect. Second, the savings treatment causes a 2 percentage point increase in the number of children passing in the last year. The other treatments have a similar effect (1-2 percentage points), but none of them are statistically significant individually. Again, however, when we pool the results, the overall average effect of the treatments is 1.6 percentage points which is statistically significant at the five percent level. Looking at the remaining outcome variables, students’ grades, none of the treatments cause a significant change, and the point estimates are inconsistent.<sup>17</sup> Compared to the pass rates, it may be that grades are a noisier outcome or that teachers consider more than grades when determining who will pass at the end of the year.

Panel B shows our findings for consumption and is divided into food and school expenses. First, we report effects on food consumption (first two rows). All the treatments, except the basic treatment in Suba, increase the number of meals children have eaten over the last three days by about 0.17 to 0.24 of a meal. Similarly for the same treatments, the number of

> 17 It is interesting to note the relative similarity of the effects estimated through self-reported and verified grades. Despite the significant overestimates for enrollment and attendance, students’ reports of their grades are remarkably similar to those that we were able to verify.  Self-reported grades are slightly higher, but measured treatment effects are very similar.

- 27 -
meals with a common source of protein increases by 0.16 to 0.18 meals over 3 days. These effects suggest that the program has a small effect on non-educational household consumption.

The second two rows of Panel B contain the results for school expenses. This variable is the sum of questions that require the family to provide details (in thousands of Colombian pesos) of their educational expenses for the entire year, including administrative fees, school supplies, transportation, and ancillary expenses. The basic, savings, and tertiary treatments seem to have little effect on spending in grades 6-10. However, the tertiary treatment seems to have a large effect on educational expenditures for individuals who are of age to attend a tertiary institution. Not all of the 600,000 pesos are spent on academic activities. Looking at row 4, families would have spent 142,000 pesos anyway on average, but the treatment causes families to spend an extra 246,000 pesos. In total, it seems that 64.7 percent of the subsidy is spent on academic expenses.

Finally, we turn to the labor market outcomes. Panel C contains the results for labor market activities for students in grades 6-10 in 2005, who should all still be in secondary school in 2006. The first three rows show the average for an indicator variable listing the primary activity of the child (studying, work, or taking care of the household); the last two rows show the number of hours worked in the last week that the child worked and the amount of money earned during that week (in 1,000’s of pesos). Generally, there is not much of a change in self-reported primary activities, but there is a relatively large, consistent change in the number of hours worked with treated children working less as a result of the treatments. The basic and savings treatments reduce amount worked by about a third while the tertiary treatment has a much larger effect of reducing time spent at work by more than a half, or 0.8 hours in the last week worked. Earnings are less precisely estimated, but show the same pattern of results.

Panel D has the same format as Panel C and contains the results for only students who were in grade 11 at the time of registration. These students should have graduated and moved onto job or tertiary institutions in 2006. On all outcomes, the basic and savings treatments have almost no effect on students’ labor market activities while, consistent with the academic participation results from Panel F of Table 6, the tertiary treatment has a large effect, shifting students away from work and towards educational institutions. Forty-three percent more students report that their primary activity is studying, and these students seem equally drawn from those who would have otherwise been working or helping around the house. On average,

- 28 -
students work 7 hours less a week and earn 11,000 pesos less, reducing their average level of work and earnings to that of students in grades 6 through 10.

# **C. Heterogeneity in the Treatment Effects**

Another important dimension of these incentive programs is their relative impact on different types of students and families which we investigate in Table 8. We study possible heterogeneity in the effect on treated students by estimating Equation 2 for different subsets of the sample for six outcomes: verified attendance, administrative enrollment, pass rates, meals, protein consumption, and hours worked. We estimate heterogeneity across three dimensions: our projected baseline attendance measure, gender and household income. In order to maintain sufficient sample size within subsets of the data, we pool the data and estimate the average effects across all three treatments, though the pattern of results is generally consistent within treatments as well.

Within these divisions, three general patterns emerge. First, while all of the differences suggest that those families whose children would be projected to attend less than 80 percent of the time respond more strongly to the program than those families who would have already met the attendance target on average, the magnitude of the differences are small and statistically insignificant at conventional levels. Students with low projected attendance who also receive the treatment attend school 1.1 percentage points more often, have a 2.6 percentage point higher enrollment rate, are 1.3 percentage points more likely to pass, eat more, eat more protein, and work a tenth of an hour less a week.

Second, boys seem to respond to the treatment more strongly than girls in their attendance rates and work patterns, though again the differences are not statistically significant. The treatment effect for boys is 4.3 percentage points compared with 1.6 percentage points for girls; however, there is no difference in the enrollment rates. Similarly, boys respond to the treatment by working 0.6 hours less a week while girls work 0.38 hours less a week. This result is consistent with the fact that boys seem to engage in paid work more than girls do. It is possible, for example, that girls reduce the number of hours spent helping in the home by a similar amount, but this outcome was not measured.

- 29 -
Finally, we also divide the sample by income level, revealing that the poorest families show the weakest response to the intervention. Across all interventions, those households at the bottom of the income distribution show almost no response to the treatment (only the 1.8 percentage point increase in attendance is statistically significant), while wealthier families show much stronger responses. All of the households in the sample, of course, are poor. The results suggest, however, that the transfer may just be too small to make a difference in the lives of the poorest families, reporting a total income of less than 3 USD a day. This differential effect highlights the importance of targeting interventions towards individual families.

# **D. Effect on other Household Members**

If families redistribute academic opportunities and labor market responsibilities to reflect available resources and the relative costs of a child’s education, then the receipt of the treatment by one child may affect the activities of other children. How this happens depends on the method of distributing resources and educational opportunities within the family. For example, families may redistribute resources between children so that children have equal opportunities, or, if some children are preferred, families may allocate resources less equally than before. While we do not have sufficient detail to fully unpack the decision process within the family, the intra-family variation in treatment assignment does provide an opportunity test in reduced form whether or not the receipt of the transfers changes the allocation of opportunities within the household.

The first decision that families within our data faced is whether or not they chose to enroll a child in the randomization lottery. Surprisingly, not all children were enrolled. On average, the 6,619 families who completed a follow-up survey had 2.5 kids in the appropriate age range to enroll in the lottery, but only enrolled 1.3 children. This decision already seems to reflect the allocation of activities within the household. In families that received no subsidies, 93 percent of registered children reportedly attend school while only 75 percent of unregistered children attend. Similarly, registered children work 3.1 hours less a week (1.4 hours versus 4.6 hours in the last week worked) than non-registered children.

The most direct test allowed by our research design is to directly compare families who registered more than one child in the lottery. Unfortunately, most families (5,132) only

- 30 -
registered one child, and very few registered more than two (158). Focusing on the 1,329 families who registered two children, however, allows us to conduct three comparisons. First, we can directly test whether the receipt of a subsidy changes the allocation of opportunities in the household by comparing children who received subsidies to those who did not. If families reallocate resources to equalize (unbalance) the allocation of opportunities, then the resulting treatment effect will be smaller (larger) than the overall treatment effect. Second, we can compare treated children in families that receive one treatment to treated children in families that receive two treatments. If resources are shared between children, then treated children in families that only receive one subsidy may experience smaller educational gains than children in families in which both children are treated. Finally, we can also compare the untreated children in a family that receives one treatment to the registered children of families that receive no treatment. If families shared resources between treated children, then we would expect untreated children in households receiving one treatment to be more engaged in school than registered children in households with no treatment. However, if the receipt of a treatment causes families to allocate additional resources to a treated child, then untreated children in households that receive one treatment may in fact be less engaged in school.

The results from these comparisons are contained in Table 9. Because of the smaller sample size, we pool the treatments and then only break out the estimates by locality and then gender. For each comparison, we estimate differences along three variables. First, we estimate the effects on all children using our verified attendance measure. Second, for children who were not in grade 11 in 2005, we estimate the effects using the administrative enrollment data. Finally, we also estimate differences using our measure of labor market hours.

Panel A, for reference, contains a simple estimate of the overall treatment effects by comparing students who received a treatment to those who did not within all families who registered two children. While the sample size is substantially smaller than in Tables 5 and 6, the estimates in families with two registered children are slightly stronger but consistent. Overall, the receipt of a treatment increases attendance by 1.6 percentage points, reenrollment by 2.1 percentage points, and reduces hours worked by 0.9 hours. The results are also distributed as we would expect from the previous results. The increase in attendance is consistent across all groups. The increase in enrollment and reduction in working hours is concentrated in Suba, but evenly distributed by gender.

- 31 -
To test the direct intra-family effects of the program, Panel B directly compares students who received a treatment against those that did not in families who registered two children and received only one treatment. The results suggest that receiving a subsidy did change the allocation of educational opportunities within the family. First, treated students have much higher levels of academic participation than non-treated students. Overall, students attended school 2.9 percentage points more if they received a treatment and worked 1.2 hours less. They were also re-enrolled in school 2.9 percentage points more often, but while this result is consistent with the others, it is not statistically significant at conventional significance levels. Turning to the sub-samples, the effect on attendance is consistent across all groups while the work and enrollment effects are consistent across gender but concentrated in Suba. In fact, the re-enrollment effect of 5.5 percentage points is statistically significant at the 10 percent level in Suba. Second, however, the estimated treatment effects within these families is larger than the overall effects, suggesting that families may allocate resources away from untreated siblings towards treated siblings.

This widening of the gap between treated and untreated students within families that receive one treatment could be the result of additional participation among treated students or lower levels of participation among non-treated students. Panel C compares treated students in households that received one treatment to treated students in households that received two treatments. Within this context, we refer to children within partially treated families as “treatment” students and students from families that receive complete treatment as “control” students. The academic participation of the two samples is remarkably similar. The estimated differences are all very close to zero, even in the sub-samples. The only difference is the reenrollment rates for boys and girls, but this difference is not statistically significant.

Finally, Panel D makes the opposite comparison – comparing the engagement of untreated, registered students in households with one treatment to registered students in households that receive no treatments. As in Panel C, the partially treated households are considered the “treatment” households and the untreated households are considered the “control” households. Unlike Panel C, however, there is some evidence that these students are less engaged than children in untreated households. The results are not as strong as the difference between students within the same household, but in general, students attend less often (1.7 percentage points), are less likely to enroll in school (4.3 percentage points) and work on average

- 32 -
0.7 hours more a week. The difference in enrollment rates is statistically significant at the five percent level. Looking at the subsamples, the signs of the point estimates are consistent with the overall averages, but the magnitude of the effect is stronger in San Cristobal as well as for girls. In fact, the differences in attendance and enrollment rates are statistically significant for both subgroups at the five percent level. These differences may reflect that families have reallocated resources between registered children (and away from girls in particular) due to the treatment.

Estimating the effects on non-registered siblings is more difficult because families can vary both in the number of students they register and the number of children in the household. It is possible, for example, that receiving a subsidy may generate a larger change in a smaller household because educational resources are divided over fewer children. At the same time, it would be incorrect to simply estimate the relationship between the outcomes of unregistered siblings on the number of treated children because the differences between registered and unregistered children suggest that the selection of which children families register is endogenous. To account for both of these possibilities, we estimate a variation of the instrumental variables model provided by Equation 3 in which we model the outcomes of unregistered children as a function of the fraction of treated children in the household and use the fraction of registered children receiving a treatment as an instrument.<sup>18</sup> In total, we include 24,552 school aged siblings in our data set between the ages of 7 and 22.<sup>19</sup> On average, each family has 2.5 school aged children and receives 1.15 subsidies. Thus, the average percentage of treatment in a family is 46 percent, and since the vast majority of families receive only 1 treatment, the variation in treatment density is driven primarily by differences in family size.

The results of these estimates are presented in Table 10. Given the large sample, we use the same format as Tables 5, 6, and 7, estimating the effects of each treatment individually. We also provide the average outcomes for unregistered siblings in families receiving no treatment under the heading “control average” for reference. To facilitate interpretation, the point estimates are also rescaled based on the average treatment density in each sub-sample so the presented results provide the effect experienced by the average household.

> 18 We have also estimated the simple treatment effect on siblings by comparing families that received and did not receive a treatment while registering only one child.  The estimates are similar to those presented in Table 10. 19 We chose the age range for school-aged children by using the range of ages for children who report being enrolled in school in the SISBEN survey.  The results are consistent for other choices of the age range.

- 33 -
Panel A contains the results for all unregistered siblings in our sample first using the selfreported enrollment variable and then the measure of labor market engagement. For no outcome variable or any of the subsamples, is there an effect in San Cristobal. However, in Suba, the two treatments seem to have opposite effects. The basic treatment generally increases the probability that siblings are attending school and reduces the number of hours worked – an effect that is decreasing in the size of the family. There seem to be some gender variation in the size of the effect with boys experiencing the bulk of the reduction in work hours and girls the increase in school enrollment.

The tertiary treatment, however, which sharply reduced the work effort of treated children, seems to increase the work effort of unregistered siblings – primarily boys. The estimates are consistent with point estimates for the enrollment measure, but none of the estimates of the effects on enrollment are statistically significant at conventional levels – though the 4.2 percentage point reduction for boys has a p-value of 0.12. This suggests that the tertiary treatment may be inducing a reallocation of work responsibilities within the household with work being shifted from treated to untreated children.

To check the consistency of our results with the direct estimates of the effects of the treatments on treated children, we break the sample up by income tercile. The treatment effects for the highest two income terciles are presented in Panel B while those for the lowest are presented in Panel C. The results are, in fact, generally consistent with those of Table 8 with families in the highest two income terciles experiencing the largest effects, and those in the lowest tercile experiencing no effect. The only variable that shows a different pattern is the enrollment measure for the tertiary treatment. While the point estimates for the highest two terciles are negative (and statistically significant for boys at the five percent level), the point estimates for the lowest tercile are positive and large but statistically insignificant.

# **E. Peer Effects**

Because our randomization strategy induces intra-school variation in the treatments, we are also able to measure directly the peer effects associated with the program through our explicit

- 34 -
mapping of friendship networks collected in the baseline survey.<sup>20</sup> Our design provides both an explicit mapping of children’s friendship networks and random assignment of the fraction of a student’s friends who are treated. This allows us to compare directly students who have more treated friends to those who have less, using Equation 3. As in the family regressions, the possible endogeneity arising from the fact that registration is likely to have followed friendship ties is controlled by instrumenting for the faction of registered friends actually receiving the treatment.

The friendship question in the baseline survey asked the children to name up to five of their closest friends who were in the same school and grade. All of the children provided at least 1 friend and three quarters provided five. The names of these friends were then matched with the results of the random assignment lottery to determine which friends were treated by the program. Half of the students had no treated friends and about 92 percent had up to 40 percent of their friends treated. On average 15 percent of a students’ friends were treated (about 1 friend). This provides significant variation in treatment density, but also means that our results cannot be generalized to conditions in which more than 40 percent of a child’s friends are treated. All of the results are presented in Table 11.

Column 1 contains the OLS regression between our verified attendance measure and the fraction of a students’ friends that are treated. Already the positive but declining contribution of treated friends is apparent with the coefficient on the linear term of 0.03 and the quadratic term of -0.04. However, both point estimates are statistically insignificant.

The first stage regression is presented in Column 2. The estimates confirm that the outcome of the random assignment process (the fraction of registered friends who are treated) does, in fact, generate significant variance in the overall fraction of a student’s friends who are treated. It in fact, explains 70 percent of the variance in the fraction of the friends treated.

> 20 Such externalities have been previously estimated for conditional cash transfers by exploiting the fact that the original PROGRESA program only provided subsidies for the poorest families in a village.  Both Bobonis and Finan (2006) and Lalive and Cattaneo (2006) compare the school attendance patterns of children whose families were too rich to receive the subsidy between treated and untreated villages.  This comparison allows them to experimentally identify the effects of the externality by looking for changes in the children who would not have been directly treated in both treatment and control villages.  However, the relative percentage of treated and untreated children in each village (the treatment density) is determined by the distribution of wealth in each village, making it difficult to causally associate the magnitude of the effect with the density of treatment in a village.  In unpublished results, we estimate a similar measure (the fraction of treated children within a students’ school-grade-gender cohort) and find no effects suggesting that this indirect measure may underestimate the effects of peers.

- 35 -
The IV model (column 3) shows that the OLS measure seems to be biased downwards as the instrumental variable regressions has larger and more statistically significant coefficients on the density of treatment within a child’s friends. At the average treatment density, the overall magnitude of this effect is 1.4 percentage points, which is of the same magnitude as the direct effect. Adding the direct and indirect effects, the total change in attendance, on average, in this sample of students is 2.5 percentage points. It is important to note, however, that while meaningful at the average treatment density, the indirect effects decline as additional friends are treated. So, there is a large gain to having a single friend treated (roughly the size of the average treatment density), but an additional friend contributes almost nothing.<sup>21</sup> Finally, in column 6, we re-estimate the peer effects model but instead of the density of treatment within a child’s peer group, we use the number of other students treated within a child’s school-grade-gender cohort. While the direct effect is still precisely estimated, this measure of indirect effects does not detect the peer effects that we were able to measure with the explicit friendship mapping. The coefficients are very small in magnitude and statistically insignificant. These results suggest that indirect measure of peer effects may significantly underestimate the magnitude of such effects.

# **VI. Conclusion**

This project demonstrates both that the design of educational incentive programs matter and that experiments can be used to understand how variation in the structure of these programs may affect educational and related outcomes for targeted children and their families. Overall, we find that students that are offered cash incentives more likely to attend school, more likely to remain enrolled, more likely to matriculate to the next grade, more likely to graduate, and more likely to matriculate to a tertiary institution. But we also find that the structure of the cash incentive programs matters. Simply postponing some of the cash transfers to a large lump-sum, at the time of the re-enrollment decision, increases enrollment in both secondary and tertiary institutions without reducing daily attendance. This suggests that there is much to gain in designing CCTs

> 21 In regressions not presented in this draft, we also estimated the effect of peer networks on enrollment using the samples in which we observed direct effects of the subsidies.  In none of these regressions did we find a statistically significant effect of peers on the decision to enroll.  However, our inability to observe an effect may be due low statistical power as well as the fact that peers do not influence the decision to enroll in same way that they influence the decision to attend.

- 36 -
that better take into account the savings constraints (such as a lack of formal savings institutions or commitment problems) many families face. Also, incentivizing on graduation rather than just attendance is shown to be particularly effective, leading to higher levels of daily attendance and higher levels of enrollment at the secondary and tertiary levels. While keeping in mind an important caveat about the larger magnitude of the transfers in this last treatment, this does indicate that more thought should go into selecting the specific behaviors the cash transfers should be conditioned upon.

Our finding of large positive spillover effects through friendship networks offers an optimistic perspective on the likely efficacy of CCTs even in environments where local or national authorities are financially constrained and would have to limit eligibility to targeted subgroups of the population. Treating friends encourages higher attendance at a similar magnitude as the direct effect, and this effect appears to be independent of students’ direct receipt of the incentives, benefiting treated and untreated students equally.

In contrast though, the intriguing within-family dynamics we observe suggest that the CCT spillovers may not all be positive. In particular, we find that children that registered for the program but were not selected for treatment end up attending school less and being more engaged in labor market work if one of their siblings was treated than if that sibling was also untreated. This suggests the possibility that families might redirect some of their prior educational investments towards the program-eligible children, leading to increased inequalities in educational attainment within the household. Of course, the treatments increase the overall resources available to families and we cannot rule out future transfers from educated to noneducated siblings that would more equally distribute the welfare gains. However, these results, particularly those for girls, unambiguously suggest that the ways in which families mediate the allocation of educational opportunities should be an important area for continued research.

- 37 -