Table 26: Impact on student learning – Intent to Treat estimation - full sample of 348 schools

|Dependent variable|||(1)||(2)||(3)||(4)||(5)|
|---|---|---|---|---|---|---|---|---|---|---|---|
|G<br>Spaece - Mathematics<br>Spaece - Portuguese|rade<br>1<br>1|N<br>59805<br>59806|Results<br>0.12**<br>[ 0.05]<br>0.08*<br>[ 0.05]|N<br>59805<br>59806|Results<br>0.09**<br>[ 0.04]<br>0.06*<br>[ 0.04]|N<br>59805<br>59806|Results<br>0.02**<br>[ 0.01]<br>-0.00<br>[ 0.01]|N<br>59805<br>59806|Results<br>0.03***<br>[ 0.01]<br>0.01*<br>[ 0.01]|N<br>59805<br>59806|Results<br>0.02***<br>[ 0.01]<br>0.00<br>[ 0.01]|
|ENEM - Portuguese|3|45838|0.07*<br>[ 0.04]|45838|0.05<br>[ 0.03]|45838|0.02**<br>[ 0.01]|45838|0.04***<br>[ 0.01]|45838|0.03***<br>[ 0.01]|
|ENEM - Mathematics|3|45838|0.06<br>[ 0.04]|45838|0.04<br>[ 0.03]|45838|0.01<br>[ 0.01]|45838|0.02**<br>[ 0.01]|45838|0.02*<br>[ 0.01]|
|Controls 1*|||||x||x||x||x|
|2013 school-grade<br>average test score<br>2014 school-grade<br>average test score|||||||x||x||x<br>x|

Note: Standard errors in brakets, clustered at the school level . Statistical significance levels * p<0.1 ** p<0.05 *** p<0.01 * Controls 1: age, female, black, brown, students per class, 2013 and 2014 school-grade average pass, failure and drop out rate.

We estimate the following model: ��,� = �� + ���� + ���� + ������ + �� (7) where �� is student achievement averaged at the school-grade i; �� is a dummy variable for the period, which assumes a zero value for the 2013-2014 trend and one for the 2015-2014 trend; �� is an indicator for whether the school was offered participation in the intervention; and �� if the error term, clustered at the school level. In this model, the average pre-treatment trend across the units serves as the counterfactual for the average post-treatment trend. The effect is estimated by comparing the average pre-treatment trend to the average post-treatment trend, given by the coefficient of interest ��.

Table 27: Impact on student learning, difference-in-differences estimation - full sample of 348 schools

|||ITT -|No controls||Dif-Dif|
|---|---|---|---|---|---|
|Dependent variable|||(1)||(2)|
||Grade|N|Results|N|Results|
|% Students who took Spaece|1|348|0.02<br>[ 0.01]|697|-0.01<br>[ 0.01]|
|Spaece - Mathematics|1|348|<br>0.13**<br>[ 0.06]|697|<br>-0.04<br>[ 0.03]|
|Spaece - Portuguese|1|348|0.08<br>[ 0.05]|697|-0.03<br>[ 0.03]|
|% Students who took ENEM|3|348|<br>0.00<br>[ 0.01]|697|<br>-0.04***<br>[ 0.02]|
|ENEM - Portuguese|3|348|<br>0.06*<br>[ 0.03]|697|<br>0.10***<br>[ 0.03]|
|ENEM - Mathematics|3|348|<br>0.06*<br>[ 0.04]|697|<br>0.07***<br>[ 0.03]|

Note: Standard errors in brakets. Statistical significance levels * p<0.1 ** p<0.05 *** p<0.01 * Controls: enrollment, school-grade average for age, female, black, brown and students per class Table 27 shows results for the school-grade level analysis. The first column presents a simple ITT result, with no controls. The second column presents results for the difference-in-differences analysis. Results from the difference-in-differences analysis show no impact on SPAECE scores, but positive effects on ENEM scores, of 0.1 SD in Portuguese and 0.07 SD in Math.

Our last strategy to account for the imbalance in our sample is to use student-level data to control for previous achievement in t-1 for the students we can find in both 2015 and 2014. While we lack data for all students, we are fortunate to be able to match two large sub-samples of students, which give us very strong controls: (a) Students who took SPAECE in 2015 in the 1<sup>st</sup> year of secondary school and in 2014 while in 9<sup>th</sup> grade; (b) Students who took ENEM in 3<sup>rd</sup> grade in 2015 and in 2<sup>nd</sup> grade in 2014. For each of these sub-samples, we run five specifications: (i) To analyze how each sub-sample compares with the total sample, we repeat the ITT results from table 26 (for the full sample, with no controls); (ii) We then show ITT results for the sub-sample, with no controls; (iii) Next, we add a control for students’ previous achievement in t-1; (iv) In addition to prior test scores, we include student demographic and background controls from the SPAECE survey (sex, age, race, parents’ education, domicile on paved street, and recipient of _Bolsa Familia_ cash transfer); (v) To check the robustness of the results from specification (iv), we run a propensity score analysis. We first calculate the propensity score including all covariates used as controls in specification (iv) (previous achievement, sex, age, race, parents’ education, paved street and cash transfer). We then use the method of 32