metadata: authors: 0: author: Gertler,Paul J.
1: author: Patrinos,Harry Anthony
2: author: Rubio-Codina, Marta
count: Mexico
docna: 0: docna: Empowering parents to improve education :
            evidence from rural Mexico
docty: Policy Research Working Paper
subtopic: Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers
teratopic: Social Protections and Labor,Education
abstracts: cdata!: Mexico's compensatory education
            program provides extra resources to primary schools that
            enroll disadvantaged students in highly disadvantaged rural
            communities. One of the most important components of the
            program is the school-based management intervention known as
            AGEs. The impact of the AGEs is assessed on intermediate
            school quality indicators (failure, repetition and dropout),
            controlling for the presence of the conditional cash
            transfer program. Results prove that school-based management
            is an effective measure for improving outcomes, based on an
            over time difference-in-difference evaluation. Complementary
            qualitative evidence corroborates the veracity of such findings.
geo_reg_key: 119228,517191
repnme: repnme: Empowering parents to improve education :
            evidence from rural Mexico
display_title: Empowering parents to improve
            education : evidence from rural Mexico
listing_relative_url: /research/2008/05/9460676/empowering-parents-improve-education-evidence-rural-mexico
topicv3: School Directors m1326118 2144,Teachers Management m1326119 1523,Teacher Absenteeism m1327035 1528,Education m1327048 470,School-Based Management m1327730 2143
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Social Protections and Labor,Education
no_of_pages: 31
author_keys: 0: author_key: 428566
1: author_key: 95903
model_extractions: WPS3935-IE

## **IMPACT EVALUATION SERIES NO. 4**

# **EMPOWERING PARENTS TO IMPROVE EDUCATION:**

# **EVIDENCE FROM RURAL MEXICO**

## Paul Gertler

Harry Patrinos

## Marta Rubio-Codina<sup>†</sup>

_The authors examine a program that involves parents directly in the management of schools located in highly disadvantaged rural communities. The program, known as AGE, finances parent associations and motivates parental participation by involving them in the management of the school grants. Using a combination of quantitative and qualitative methods, we show that the AGE greatly increased the participation of parents in monitoring school performance and decision-making. Further, the authors find that AGE improved intermediate school quality indicators, namely grade failure and grade repetition, controlling for the presence of a conditional cash transfer program and other educational interventions._

## JEL Codes: I20, I21, I28

## Keywords: School-based management, impact evaluation, Mexico

World Bank Policy Research Working Paper 3935, Revised May 2008 _The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank’s Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent_ .

† Contact information: Paul Gertler, Haas School of Business, University of California at Berkeley; <u>pgertler@worldbank.org, Harry Patrinos, The World Bank; hpatrinos@worldbank.org, Marta Rubio-Codina,</u> University College of London and the Institute for Fiscal Studies; <u>m.rubio-codina@ucl.ac.uk.</u> The opinions expressed herein are those of the authors and not necessarily of the institutions they represent. We thank Thomas Cook, Pierre Dubois, Halsey Rogers and participants at the World Bank AAA decision meetings and at the World Bank School-Based Management workshop for useful comments and suggestions. Previous versions of the paper have also benefit from comments by participants at the LACEA meetings at ITAM, the first Impact Evaluation meetings in Los Andes (Bogotá) and the NEUDC Meetings at Harvard University. For providing data and institutional knowledge, we are grateful to Felipe Cuellar, Narciso Esquivel, José Carlos Flores, Alejandra Macías, Miguel Ángel Vargas and Iliana Yaschine. All errors are our own.
# **1. Introduction**

Improving school performance, especially in poor communities, remains a challenge facing most countries (Filmer et al 2006). One policy being examined by many developing countries is school based-management (SBM), which decentralizes responsibility and decision-making powers to local school management committees (World Bank 2007).<sup>1</sup> SBM takes on many different forms, both in terms of who has the power to make decisions as well as the degree of decision-making. While some programs transfer authority to principals or teachers only, others mandate parental and community participation. SBM devolves authority over one or more of the following: budget allocation, employment and remuneration of teachers and staff, curriculum development, textbook and educational material procurement, infrastructure improvement, school calendar, and monitoring and evaluation of teacher and student performance.

One of the primary reasons proponents support SBM is that decentralizing decision-making to the local level is thought to bring decision-making closer to the people so that their preferences can be better reflected in policy (Oates 1972; Lockwood 2002; Besley and Coate 2003 and Besley and Ghatak 2003). The argument is that local decision-makers are better able to adapt the appropriate mix of inputs and education policies to local preferences, realities, and needs; and are more accountable to their constituencies. However, decentralized decision-making policies such as SBM may not improve school quality (Galiani et al 2008), when parents lack the ability to make their voices heard, when local elites can capture public resources (Bardhan and Mookherjee

> 1 Parental participation in SBM has long been popular in the United States, the United Kingdom, Australia and Canada, and is currently being implemented in a number of countries, including Hong Kong (China), Indonesia, El Salvador, Nicaragua, Kenya, Kyrgyz Republic, Nepal, Paraguay and Mexico.

2
2005, 2006), or when SBM groups are less technically able than higher levels of government to administer schools (Smith 1985).

In this paper, we empirically examine a program that includes parents in school management in a limited way. Parents, especially of younger children, are the principal clients of schools. They represent the interests of their children and, therefore, have the most to gain from better school performance. Participation in management committees provides parents a mechanism for them to assert their preferences over the school’s operational decisions and policies, and make schools more accountable.<sup>2</sup> Their participation allows them to directly monitor principal and teacher effort as well as overall school performance and provides a feedback mechanism for them to voice any concerns.

Specifically, we study the impact of an effort to increase parental participation in school management in rural Mexico. In 1992, Mexico decentralized educational services from the federal to the state level. The federal government complemented school decentralization with the Compensatory Education Program designed to equalize resources and educational standards across all schools with a focus on disadvantaged rural and indigenous schools. The program included a SBM component – the Support to School Management or AGE ( _Apoyo a la Gestión Escolar_ ). AGE provides small monetary grants to parent associations that they can use to invest in infrastructure or in materials they deem important for their schools. Parents also receive training in the management of these funds and in participatory skills to increase their involvement in school activities. Through AGE parents spend more time in the school as well as regular

> 2 Making schools directly accountable to their clients is the primary intervention to improve school quality recommended by the 2004 World Bank’s World Development Report (World Bank 2004).

3
interaction and greater standing with school directors and teachers. As a result, they are better able to monitor the school activities (teacher absenteeism, children attention in class, etc) and to voice their opinions on school matters. AGE was the first program that gave parents any authority over school matters in Mexico.<sup>3</sup> By 2005 more than 46 percent of primary schools in Mexico had an AGE.

We examine whether increased parental participation through AGE helped to create a more conducive learning environment and thereby improved students’ learning outcomes. Through interviews of parents and school directors, we found that they believe that AGE have increased parents’ involvement in school-related activities and facilitated better communication amongst parents, teachers and principals. Both parents and school directors report that the AGE led to an increase in parental participation in school matters. Most parents are interested in teacher effort. They will, for example, complain to principals if teachers are absent. The increased parental presence and oversight in schools make schools more accountable to their end users and, therefore, might ultimately affect student learning.

We test this hypothesis by examining the impact of the AGE on intermediate school quality indicators – school-level grade failure, grade repetition and intra-year drop out rates. We exploit the gradual phasing-in of the AGE intervention over time to identify difference in difference estimates of average treatment effects. Results suggest that the AGE decreased the proportion of students failing and repeating a grade by about 5 percent.

> 3 In 2001, the federal government launched a broader SBM intervention, the Quality Schools Program or PEC _(Programa Escuelas de Calidad_ ).

4
This study contributes to a small literature on SBM in developing countries.<sup>4</sup> Several studies rely on cross-sectional variation, ex-post propensity score matching and exclusion restrictions – either using functional forms or weak instrumental variables – thus leaving their ability to establish causality open to question.<sup>5</sup> Notable quasiexperimental exceptions include Shapiro and Skoufias (2005) and Murnane et al (2006) who use difference in differences models to estimate the impact of Mexico’s PEC (Quality Schools Program) intervention on drop out, repetition and failure rates. Duflo et al (2007) uses a randomized experiment to evaluate the effects of monetary empowerment of local school management committees to monitor and train teachers combined with contract teacher hiring in primary schools in Kenya. They show that combining class size reduction with improved incentives – by either hiring contract teachers (as opposed to civil servants) or increasing parental oversight – leads to significantly larger test scores increases.

The remainder of the paper is organized as follows. The next section describes the AGE intervention in greater detail. In Section 3, we posit the pathways whereby AGE might affect parental participation using descriptive information from the qualitative interviews. In Section 4 we discuss the identification strategy and data used and present the quantitative empirical results. A discussion of potential biases is provided in Section 5. Section 6 concludes.

> 4 Summers and Johnson (1996) review the evidence on the effects of SBM in the United States.

> 5 See for example the works of Jimenez and Sawada (1999, 2003) on El Salvador’s EDUCO; DiGropello and Marshall (2005) on the effects of the Hondura’s PROHECO program; King and Ozler (1998), King et al (1999) and Parker (2005) on Nicaragua’s Autonomous School program; or López-Calva and Espinosa (2006) on the impacts of AGE as well as the other Compensatory Program supports on test scores.

5
# **2. The AGE Program**

AGE is part of a broader school reform designed to improve the supply and quality of education in schools in highly disadvantaged communities. The Compensatory Program consists of: (i) infrastructure improvement, (ii) provision of school equipment , (iii) provision of materials for students (e.g. notebooks, pens, etc), (iv) pedagogical training for teachers, (v) performance based monetary incentives for teachers, and (vi) AGE. Not all of the sub-interventions were introduced at the same time and not all of the schools received all of the sub-interventions.

The Compensatory Program progressively expanded from more to less disadvantaged areas. Between 1992 and 1995, the program was introduced in the poorest municipalities of the poorest 23 states.<sup>6</sup> Coverage was extended to disadvantaged schools in the eight remaining Mexican states in 1998. We use data from schools incorporated starting in 1998 in our analysis. They have lower poverty rates and better educational outcomes than the States incorporated earlier. The worst performing schools were targeted using an index based on (i) a community marginality index (ii) teacher-student ratios (iii) the number of students per school and (iv) repetition and grade failure rates.<sup>7</sup> Each State then decided which sub-interventions would be allocated to which school based on their budget and logistic capacity.

AGE as a sub-intervention was first introduced in the 1996-97 school year. AGE finances and support the schools’ parent associations. The monetary support varies from $500 to $700 per year depending on school size. The use of funds is restricted and subject to annual financial audits for a random sample of schools. Amongst other things,

> 6 Poverty levels are defined according to the marginality index developed by the Population National Council or CONAPO ( _Consejo Nacional de Población_ ).

> 7 CONAFE (2000) provides specific details on the weighting of variables to construct the index.

6
the parents are not allowed to spend money on wages and salaries for teachers. Most of the money goes to infrastructure improvements and small civil works. In return, parents must commit to greater involvement in school activities, participate in the infrastructure work, and attend training sessions delivered by state educational authorities. In these sessions, parents receive training in the management of the funds and in participatory skills to increase their involvement in the school. Parents also receive information on the role of the school as an educator, on the role of the schools’ parent association, on their children educational achievements and on how to help their children learn.

# **3. Did AGE Increase Parental Participation?**

In Mexico, parent associations exist by law but are rather dysfunctional and they typically have little or no access to schools. AGE created a need and a right for parents to have access to schools to decide on the allocation of the grant, manage the funds (establish a feasible budget, record expenses, etc), and participate in infrastructure works directly. Hence, the AGE represent the first time that parents are granted full access to the schools and are given certain – albeit limited – authority over school matters. We argue that this is likely to change parental attitudes towards schooling, attitudes of school directors and teachers to parents, and improve overall school climate. Because parents now spend time in the school, they are better able to monitor school activities (teacher absenteeism, quality of the teaching, kids attention levels, etc) and gather information about school performance. Parents are also better able to voice their opinion over general resource allocation and school policy.

In order to substantiate these arguments, we undertook descriptive qualitative work along two lines. First, we conducted a series of focus groups with parents in three 7
AGE and three non-AGE schools in five communities in the Mexican state of Campeche.<sup>8</sup> In addition, we carried out a larger qualitative survey of school directors’ perceptions in 115 randomly selected AGE schools in the states of Campeche, Guerrero, Michoacán, Sinaloa and Tamaulipas.

The parental focus groups revealed that parents believed that the AGE had indeed improved interaction and communication with school directors and teachers resulting in better educational outcomes. Parents expressed the view that the AGE helped generate and facilitate dialogue between parents, teachers and school directors. Parents in AGE beneficiary schools were pleased with the fact that they were better able to meet with their child’s teacher and to follow their child’s progress more closely. They reported that teacher instructed them on how to improve their child’s performance. They believed that this fostered parental involvement in school and with their children’s education. Parents also perceived that AGE had a positive impact on teacher effort. When asked what impacts they had noticed, parents commented on the fact that teachers stayed longer hours in schools to help students who were falling behind academically. The focus groups established that AGE helped parents be vocal representatives of the school clients as they articulate demands, generate expectations and promote participation. These findings are in line with previous qualitative evidence in the state of Tabasco, which revealed that AGE increased parental participation in school activities, improved parentteacher relations, and reduced teacher absences (World Bank 2000).

The survey of school principal confirmed the perception that the AGE led to an increase in parental participation in school matters. In fact, _all_ school principals believed that this was the case. When asked about the most important change induced by parental

> 8 See Patrinos (2006) for full details.

8
participation, all principals reported positive changes: 40 percent reported increased parental concern about their children’s academic performance; another 30 percent reported increased parental interest in the school overall; and a final 30 percent reported increased interaction between parents and teachers. Indeed, 95 percent of principals reported that AGE increased parental interest in the work of teachers. They reported that parents follow the work of the teachers closely and complain if they do not like what is happening. More than 80 percent of the principals reported that parents complained if teachers were absent. Principals also reported that AGE changed parental attitude towards their children’s performance in school, with 53 percent reporting increased parental help for kids studying and monitoring that the homework was done. Another 42 percent of the principals reported that parents increased going to school to talk to teachers and followed up on their kids learning.

Both parents and principal reported that AGE increased parental participation in school, made parents more demanding in terms of attention to their children’s learning needs and teacher effort, and increased parental involvement with homework. In the following section we test whether AGE improved intermediate schooling outcomes and provide an estimate of the size of the impact.

# **4. Did AGE Reduce Grade Repetition, Grade Failure, and Drop out?**

We estimate the effects of AGE on three educational outcomes: the probability that the student fails an exam, repeats a grade or drops out of school.<sup>9</sup> We use data from a variety of sources including administrative data from the Compensatory Program

> 9 There is very little overlap between the AGE schools and the nation-wide representative sample of schools with standardized test score data. As a consequence, there are too few schools with which to estimate robust effects of AGE on test scores.

9
coverage from 1991 to 2003 to identify which schools receive the AGE and/or any of the other Compensatory Program interventions. Similarly, we use administrative data on other educational interventions to control for their presence in the school. Data on school level grade repetition, failure and drop out as well as other characteristics comes from the Mexican School Census ( _Censo Escolar_ ). We use the 1990 and 2000 Population Census and the 1995 _Conteo_ to construct socioeconomic locality indicators that will help us identify the evaluation sub-sample. The unit of analysis is the school.

# **_4.1. Estimation and Identification_**

In principle, we would like to compare school performance when schools have an AGE to the counterfactual – i.e. quality for the same schools without an AGE at the same time. Since the counterfactual is never observed and we do not have a controlled randomized trial, we are forced to turn to quasi-experimental methods that mimic the counterfactual under reasonable conditions.

We propose to use the phased rollout of the AGE to identify treatment and comparison groups, with the treatment group being schools getting AGE early and the comparison group being those who got AGE later. A major concern is that the late adopters could be different from the early adopters, and that these differences may be correlated with school performance. For example, the schools that received AGE early could be located in poorer rural areas while the ones that received it later could be in wealthier areas. In this case, the correlation between AGE and performance could be confounded with the wealth effect. Alternatively, it could be that schools with the strongest potential for improvement – schools with more engaged parents and motivated 10
school staff – were incorporated at earlier stages. If so, our estimate of treatment would overestimate the true effect of the program.

In principle, many of the types of (unobservable) characteristics that may confound identification vary across schools, but are fixed over time. A common method of controlling for time invariant unobserved heterogeneity is to use panel data and estimate difference in differences models. We use this identification strategy, and hence, compare the change in outcomes in the treatment group to the change in outcomes in the comparison group. By comparing changes, we control for observed and unobserved timeinvariant school characteristics as well as time-varying factors common to both comparison and treatment schools that might be simultaneously correlated with AGE and with indicators of performance. The change in the comparison group is an estimate of the true counterfactual – i.e. what would have happened to the treatment group if there were no intervention. Another way to state this is that the change in outcomes in the treatment group controls for fixed characteristics and the change in outcomes in the comparison group controls for time-varying factors that are common to both comparison and treatment schools.

Formally, we estimate the following regression specification of the difference in difference model for all t =1997-2001:<sup>10</sup> where:

- _Yst_ is the proportion of school _s_ ’s students who fail an exam, repeat a grade or drop out in year _t_ ;

> 10 We take school year 1997-98 as the baseline year.  Evaluation years are from 1998-99 to 2001-02.

11
- _AGEs,t-1_ = 1 if school _s_ had an AGE just before the start at _t-1_ – or early in the school year – of school year _t_ ;

- α _s_ are school fixed effects;

- <sup>η</sup> _t_<sup>are time dummies;</sup>

- ξ _lt_ are state specific year fixed effects introduced to capture state specific common

time effects (State demographic trends, changes in State education policies, changes in State economic conditions, for example) that are correlated with schooling outcomes;

- _Xskt_ is a vector of time varying school characteristics and includes the student-toteacher ratio, the average number of students per class in the school (crowding index), and the presence of other educational interventions coexisting in the school (see Section 5.3).<sup>11</sup>

- ε _st_ = 1 ∑ _N_ ε _ist_ is the school average of individual error terms, which includes _N st i_ = 1

unobserved individual characteristics such as learning ability or disutility from studying. For the time being, we assume unobservables uncorrelated with the explanatory variables.

We compute robust standard errors clustered at the school level to correct for heteroskedasticity and serial correlation.

The coefficient<sup>βˆ</sup> 1 is the difference in difference estimate of the effect of the presence of AGE in the school on the outcome of interest. The specification in (1)

> 11 We have replaced missing values for school characteristics with the municipality average in the school year (or the state average in its default). We have included indicator variables to account for the replacement.

12
assumes that the AGE require at least a full school year to be effective. In a second specification, we decompose the AGE _s,t-1_ dummy in a set of dummies that equal one if the school has had AGE for one year and a second if the school has had AGE for two or more years. This addresses the question of whether the AGE impact on outcomes cumulates over time.

# **_4.2. Treatment and Comparison Groups_**

As argued earlier, we exploit the geographic expansion of AGE over time to construct treatment and comparison groups. Our sample of analysis consists of nonindigenous primary schools in rural areas that did not enroll in the Compensatory Program – and hence did not have AGE – before school year 1998-99 for which the targeting index was constructed.<sup>12</sup> We define the set of AGE treatment schools as the set of schools that first received AGE at the beginning of any school year between 1998-99 and 2001-02, and had AGE continuously ever since. Those that had not received AGE before school year 2002-03 constitute the comparison group.<sup>13</sup> Our final sample consists of a balanced panel of 6,027 rural non-indigenous primary schools that we observe continuously between 1995 and 2003. Of these, 42 percent become AGE beneficiaries over the period.<sup>14</sup> Table 1 shows summary statistics for a few school observable characteristics and for the dependent variables in 1997 (baseline) for AGE treatment and comparison

> 12 We limit on the sample to rural non-indigenous primary schools because the vast majority of AGE beneficiary schools are in rural areas and all indigenous schools were automatically incorporated when AGE first started in 1998 in these States.

> 13 Because we only have AGE coverage data until 2003, we do not know whether schools in the comparison group received AGE at later dates.

> 14 To allow comparison across outcomes, we restrict the sample to schools with complete information on outcomes.  Results are robust to the inclusion of schools with missing information for one or more of the outcomes. We also drop from the sample schools with extremely high numbers of students and/or teachers (top 0.5 percent of each distribution and bottom 1 percent of the distribution of students). We have also trimmed schools with values of the dependent variables in the top 0.5% of each distribution.

13
schools. AGE treatment schools are significantly smaller on average: they have fewer students, teachers, and classrooms. However, treatment schools also seem to have similar learning outcomes prior to the intervention. Average grade failure at baseline is 10.0 percent in treatment schools versus 9.9 in comparison schools. Similarly, baseline grade repetition is 9.5 percent in treatment schools versus 9.1 percent in comparison schools, and the drop out rate is 3.8 percent in treatments versus 4.2 in comparisons. While some of these differences are statistically significant, the order of magnitude of the differences is small.

During the intervention period, schools in the comparison group have a significantly larger proportion of teachers in _Carrera Magisterial_ (pay per performance teacher incentives scheme) but enroll a significantly lower share of students in _Oportunidades_ (conditional cash transfer program). Moreover, comparison schools are significantly less likely to receive any of the other Compensatory interventions. This suggests that there is a positive correlation between receiving AGE and receiving the other Compensatory Program benefits; in particular, the “school supplies” support and the “teacher training” support.

# **_4.3. Average Treatment Effects_**

Table 2 presents the estimates of the average treatment effect from equation (1). For each dependent variable, Model A specifies AGE as a single dummy and Model C specifies AGE as two dummies one for the first year on the program and the second for two or more years. In Model B, we add a treatment time trend. However, the treatment time trend is never significant. All estimations include school and time fixed effects, state specific time trends, and the time varying school characteristics listed above.

14
Results show that AGE is statistically positively associated with improved grade failure and repetition. Specifically, there is a significant 0.5 percentage point reduction in grade failure and a 0.5 percentage point reduction in grade repetition in AGE treatment schools (Table 2, Models A). Given a mean baseline failure and repetition rate of roughly 10 percent, these values imply around a 5 percent decrease in the proportion of students failing and repeating a grade in treatment schools. However, AGE seem to have no impact on the drop out rate. Model B shows that these results are robust to including a treatment time trend. Model C shows that the impact of AGE is achieved in the first year and that impacts do not change with more years on the program. Indeed, we cannot statistically reject the hypothesis that the two AGE coefficients in Model C are equal.

The fact that we find no significant effects of AGE on intra-year drop out rates is not too surprising. Enrolment and completion rates at the primary school level in Mexico are very high – at over 96 percent – hence leaving little scope for improvement. As a result, the drop out is about 60 percent lower than the failure and repetition repetition rates. In addition, students in _Oportunidades_ families need to be enrolled to obtain the cash benefit. We further discuss this issue later.

# **5. Threats to Identification**

The use of difference in differences controls for observed and unobserved timeinvariant school characteristics as well as time-varying factors common to both comparison and treatment schools that might be simultaneously correlated with AGE and with indicators of performance. However, the introduction of treatment in a school might respond to or be correlated with other time varying factors, such as political will, other 15
educational interventions, sorting of students or parental pressure. If these factors also affect outcomes, then our estimates of impact will be biased. In the next subsections, we address each of these potential biases separately. We first test the validity of the key identification assumption of difference in difference models: the equality in the evolution of the outcome variables prior to the intervention.

# **_5.1. Testing for Balance in Pre-Interventions Trends_**

We present two tests of the equality of pre-intervention trends of the outcomes of interest between the groups of treatment and comparison schools. The difference in differences model uses the post intervention trend in the control group as an estimate of the counter-factual, i.e. what would have been the change in the treatment schools’ outcomes if they had not had AGE. If pre-intervention trends are not statistically different, then it is likely that the post intervention trends would have been the same without AGE. The argument is as follows: in the absence of the intervention, the evolution of the dependent variables during the post-intervention period (at _t_ ) should not be significantly different between treatment and comparison schools, had it not been significantly different in the pre-intervention period (at _t’<t_ ).

Hence, we first estimate the following specification on pre-intervention data, i.e. for all _t’_ =1995-1997: where,

- α _s_ are school fixed effects;

- ξ _lt_ are state specific time dummies;

16
- _YRt’_ are yearly dummy variables for all school years in the pre-intervention period;

- _POTAGEs_ is a dichotomous indicator equal to 1 if school _s_ is a potential treatment school; this is to say, if _s_ will receive AGE for some or all of the treatment years ( _t_ =1998-2001);

- _ust’_ is an heteroskedastic disturbance that allows for correlation within schools over time.

In this specification, the test δ _t’ =0_ is equivalent to the test of the equality of the preintervention trends between treatment and comparison schools at each time _t’_ .

In a second specification, we test the equality in pre-intervention trends between comparison schools and schools that entered the program in different years. In this case, the equation to estimate for all _t’_ =1995-1997 is: where _INAGEsj_ is a set of dummies that take on the value 1 if the school _s_ started benefiting from the AGE intervention on year _j_ = 1998,..., 2001. Thus, the coefficients on the interaction with the year dummies, the λ _t’_ ’s, capture differences in pre-intervention trends for schools entering the AGE scheme at different years.

Table 3 reports the test of the difference in pre-intervention trends for the outcomes of interest: grade failure, grade repetition and intra-year drop out. For each dependent variable (reported in columns), the first column (Models A) corresponds to the estimation of specification (2) and the second column (Models B) to the estimation of (3). All specifications include school and time fixed effects and state-time specific trends. As Model A estimates show, there are no significant differences in pre-intervention trends 17
between treatment and comparison schools in any of the outcomes. Similarly, Model B estimates suggest that comparison schools are fairly similar to treatment schools that were phased-in into the AGE scheme at different years. Only one of the 30 estimated coefficients on the interaction terms comes up as significant at standard significance levels.

# **_5.2. Endogenous Program Placement Bias_**

Program placement biases might arise if the state authority decided to allocate programs non-randomly in response to political considerations. For instance, the state government could assign benefits to more disadvantaged schools first given budget constraints. In this case, estimates would be downward biased. Alternatively, the state government could prioritize better performing schools in order to maximize the chances of success and improve its reputation as a good manager. Now, estimates would likely overestimate the true program impact. As shown in Tables 1 and 2, the characteristics of treatment and comparison schools are sufficiently different as to raise such concern. In addition, we know that the state authority had certain discretion over the type and timing of benefits that targeted beneficiary schools would receive.<sup>15</sup> We argue that the inclusion of school fixed effects controls for any time-invariant observed and unobserved heterogeneity across schools. Moreover, the balance in outcome pre-intervention trends between treatment and comparison schools – shown in the previous subsection – rules out any changes in school characteristics that are not a direct consequence of the schools’ acquired treatment status. Finally, the inclusion of separate state specific time dummies should capture any aggregate state time effects that

> 15 The State Government has a final say over the allocation of AGE (and the other Compensatory supports) and this process is unknown to the researcher.

18
might be correlated with the allocation of treatment (shifts in government tastes, for instance) or with outcomes (changes in enrolment due to changes state demographics), and further minimizes the potential for this bias.

# **_5.3. Presence of Other Educational Interventions in the School_**

An additional concern is whether part of the observed effects are driven by other policies also directed to improve schooling quality and accessibility that are simultaneously operating in the school. As noted, the inclusion of state specific time dummies captures any possible change in the allocation of resources to education within states over time. To absorb any remaining bias, we additionally include explicit controls for three other educational interventions.

First, we control for the share of _Oportunidades_ beneficiary students in the school as a measure of the intensity of the program in the school. _Oportunidades_ , formerly known as _Progresa_ , is a conditional cash transfer program that provides financial incentives for families to invest in the human capital of their children. Cash transfers are disbursed conditional on school aged children attending school and on household members engaging in a set of behaviors designed to improve health and nutrition (preventive checkups, prenatal care, and health education).

Second, we include the proportion of teachers under _Carrera Magisterial_ , a voluntary pay per performance scheme targeted to all educators. Principals and teachers are eligible for permanent wage increases if they perform well in an assessment process based on their education and experience and on their school performance (student and teacher test scores).<sup>16</sup>

> 16 See Skoufias (2005) for a review of impact evaluations on the _Oportunidades_ program. McEwan and Santibáñez (2005) provide an evaluation of the _Carrera Magisterial_ scheme.

19
Finally, we control for the presence of other (sporadic) interventions supported by the Compensatory Program in the school: provision of didactic supplies and school equipment, infrastructure building and maintenance, teacher training and performance based incentives to teachers.

Because these programs are likely to be endogenous, interpretation of the estimated effects on the outcomes of interest must be looked at with some reservation. They are also highly correlated with the presence of AGE in the school, as Tables 1 and 2 show. Nonetheless, their inclusion in the regression will provide additional evidence on the robustness of the effect of AGE if they do not substantially alter the value of the estimate of impact.

Table 4 presents results. For each dependent variable, Models A1 and C1 are analogous to those in Table 2 but further introduce the proportion of _Oportunidades_ beneficiaries and teachers in _Carrera Magisterial_ in the school. The estimated effects of AGE on repetition and failure fall slightly after the inclusion of these covariates in the estimation (Model A1, Table 4), but remain significantly different form zero. There still is no estimated impact on drop out. In Models A2 and C2 we additionally control for all other interventions supported by the Compensatory Program. Including these extra variables does not further change the size of the estimated effect.

Concerning the effect of these covariates on outcomes, we observe that the proportion of teachers under _Carrera Magisterial_ and proportion of _Oportunidades_ beneficiary students in the school significantly reduces repetition and failure. The later effect might be due to the fact that the _Oportunidades_ scholarships increase with the grade of enrolment, and are conditional on attendance as well as on not repeating more 20
than twice a grade. _Oportunidades_ may also impact learning outcomes through the improved nutrition and health practices it enforces (reduced morbidity). This is consistent with the growing literature that establishes strong positive effects of health on school performance (Miguel and Kremer 2004; Bobonis et al 2006). Lastly, the other Compensatory program sub-interventions seem to have no impact.

# **_5.4. Changes in the Distribution of Students in the School_**

The error term ε _st_ in (1) includes unobserved student characteristics, θ _ist_ = {skills, ability, motivation}, that we have so far assumed uncorrelated with the observed treatment variables. However, treatment might affect the skill mix of students enrolling in school. For instance, AGE schools might attract higher skill or more motivated students whose parents have a higher preference for education. Alternatively, AGE might enable schools to retain lower skill students who were not doing well academically and would otherwise have withdrawn. If these changes in total enrollment significantly alter the distribution of students’ skills in the school, then the treatment would be correlated with unobserved ability and the estimated effect biased.

Although it is difficult to determine the direction of the bias, we can nonetheless test for its existence by examining changes in enrollment in response to AGE. In Table 5, we estimate (1) on total enrolment to test whether there is any significant difference in the evolution of enrolment between treatment and comparison schools. We find no effect of AGE on total enrollment. We therefore believe that changes in the distribution of student skills are unlikely to account for the observed effects in failure and repetition rates.

21
# **_5.5. Student Learning vs. Parental Pressure_**

Finally, we examine whether the effects observed on schooling outcomes are indeed the result of teachers influencing results in response to increased parental pressure. While, we cannot test this hypothesis with the administrative data available, we did ask principals about this possibility in the May 2006 survey interviews.<sup>17</sup> More specifically, we asked principals about parents’ reactions to the possibility of their children repeating a year or receiving a very poor grade. Less than 3 percent of principals reported that parents demanded that undeserving children be allowed to progress. According to principals, the vast majority of parents (97 percent) accepted that their child was failing or that they received a poor grade. Hence, although AGE makes parents more demanding in terms of teacher attendance and attention to children’s learning needs, they do not seem to make them pressure teachers to change grades for undeserving students.

# **6. Conclusions**

Mexico’s AGE aims to empower parent associations to improve school quality. We have provided quantitative empirical evidence that AGE improved intermediate school outcomes, namely reducing grade repetition and grade failure by 4 to 5 percent. These results are important as Manacorda (2007) and Marshall (2003) show that repetition and failure are associated with poor test performance and a higher probability of subsequent drop out. However, a limitation of the study is that we only have crude measures of school performance, rather than more sensitive measures such as test scores.

> 17 Section 3 describes the random sample of principals used and provides a summary of the main findings stemming from these interviews on parental involvement in the school as a result of AGE.

22
Our qualitative results suggest that the pathways by which AGE improved performance were through increased parental participation in school matters, and improved relations and communication between parents and teachers. Parents in schools with AGE were more likely to observe and complain about teacher absence and poor teaching. And they were more likely to know when their child was not doing well and take corrective action.

These results are consistent with theories laid out in the economics of identity and social exclusion (Akerlof and Kranton, 200 and 2005). This work postulates that one’s identity enters the utility function of both the parent and the school director. Social exclusion occurs when both believe that the parent does not deserve the benefit. The AGE acts to change parental identity and gives them a seat at the table. Indeed, the lack of formal role for parents in the Argentine decentralization of schools may explain why Galiani et al (2008) found positive effects in wealthy communities, but no impact on schools in poorer communities.

Finally, the results have important policy implications. Empowering parents in SBM is likely to strengthen the positive effects of decentralization. However, while the quantitative effects of AGE are strong and consistent, there are albeit modest. The relative small size of the effects should not come as a surprise given that AGE is a very limited intervention. Interventions that greatly increase the power of parents could be considered and tested.

23