# **4.1 DD Approach**

Table 3 presents the DD effects of the PAE program on math learning outcomes using different specifications. The first of these specifications (column 1 in Table 3) estimates the DD effects without controls and using all available data of 311 PAE-eligible public primary schools in Colima. The results show an effect of almost 0.13 standard deviations ( _σ_ hereafter) in 2010, a few months after PAE was launched. According to this simple specification, the DD estimator increased over time although the estimated effects in 2011 and 2012 are not statistically different from that in 2010, as suggested by the learning outcome trends shown in Figure 4. Notice that specification (1) with standard errors clustered at the school level, yields very similar effects to a specification with school fixed effects; that is, a dummy variable for each of the 311 eligible public primary schools (clusters) in Colima. Specification (2) includes the following school-level controls: student - teacher ratio, proportion of teachers that are enrolled in the monetary incentives program _Carrera Magisterial_ , proportion of teachers with a university degree or a post-graduate diploma, and the level of marginalization of the locality where the school is located. Including these controls, which are highly significant, reduces the estimated effects of PAE to close to 0.10 _σ_ in 2010 but still significant at the 99 percent confidence level.

Specifications (3), (4) and (5) are the same as specification (2) but restricting the observations to schools within the optimal bandwidth (OB), half of the OB, and twice the OB. Not surprisingly, under the OB restricted sample, the controls reduce their significance, particularly the proxies for socio-economic status (marginalization levels). However, the effects of PAE remain statistically significant with a point estimator close to 0.11 _σ_ . Notice that under specification (3), the DD estimators across years 2010, 2011 and 2012 are very similar suggesting that, between 2009 and 2010, PAE schools managed to improve 0.11 _σ_ faster vis-a-vis the changes in comparable non-PAE schools as a result of the intervention, capturing the _accountability_ effects of the program and its subsequent diagnosis activities during the “awareness” period. Restricting the schools included in the analysis to half of the OB does not change the DD estimator; however, with only 67 clusters, the estimates lose precision. DD estimations using schools within double the size of the OB (222 clusters) yields very similar results as those with the OB. Albeit marginally smaller, the effects of PAE on math scores are corroborated by the program’s effects on Spanish (see Table AI in Annex). The effects on Spanish scores in 2010 range from close to 0.12 _σ_ when estimating the DD without controls and using all available information, to 0.09 _σ_ when the controls are included and 0.08 _σ_ when restricting the estimation to schools within the OB, always significant at the 95 percent confidence level.

Bertrand et al. (2004) show that, by ignoring the fact that DD focuses on serially correlated outcomes, its estimation can severely underestimate standard errors. They show that a simple correction consisting of collapsing the time series information into a “pre”- and 12