Table 22 – Sample Balance in Student Characteristics, Pre-treatment, 2013-2015 - SPAECE student survey

||||Control<br>2013|||Control<br>2014|||Control<br>2015||
|---|---|---|---|---|---|---|---|---|---|---|
||Grade|N|Means<br>(T=0)|Diff.<br> (T[1]-T[0])|Sample|Means<br>(T=0)|Difference<br> (T[1]-T[0])|N|Means<br>(T=0)|Diff.<br> (T[1]-T[0])|
|Covariates|||||||||||
|White|1|61909|0.19<br>[ 0.00]|-0.01<br>[ 0.01]|56668|0.18<br>[ 0.00]|-0.00<br>[ 0.01]|64032|0.20<br>[ 0.00]|0.00<br>[ 0.01]|
|White|2|25335|0.18<br>[ 0.00]|0.00<br>[ 0.01]|52383|0.19<br>[ 0.00]|-0.01<br>[ 0.01]|47462|0.18<br>[ 0.00]|-0.00<br>[ 0.01]|
|White|3|12643|0.19|-0.01|24184|0.17|0.01|44422|0.19|-0.01|
||||[ 0.01]|[ 0.01]||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]|
||||0.13|-0.00||0.10|0.00||0.08|-0.01|
|Black|1|61909|[ 0.00]|[ 0.00]|56668|[ 0.00]|[ 0.00]|64032|[ 0.00]|[ 0.00]|
|Black|2|25335|0.11<br>[ 0.00]|-0.01<br>[ 0.01]|52383|0.12<br>[ 0.00]|-0.01**<br>[ 0.00]|47462|0.10<br>[ 0.00]|0.00<br>[ 0.00]|
|Blk|3|12643|0.11|0.01|24184|0.10|-0.00|44422|0.12|-0.01*|
|ac|||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.00]|
||||0.58|0.02*||0.62|0.01||0.57|0.01|
|Brown|1|61909|[ 0.01]|[ 0.01]|56668|[ 0.01]|[ 0.01]|64032|[ 0.00]|[ 0.01]|
|Brown|2|25335|0.64<br>[ 0.01]|0.01<br>[ 0.01]|52383|0.60<br>[ 0.01]|0.02***<br>[ 0.01]|47462|0.62<br>[ 0.01]|0.01<br>[ 0.01]|
|Brown|3|12643|0.62|0.00|24184|0.65|0.00|44422|0.61|0.02**|
||||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
|Elementary School|||0.17|0.00||0.15|0.01||0.17|0.01|
|Incomplete (EF I)|1|61164|[ 0.01]|[ 0.01]|56341|[ 0.01]|[ 0.01]|63320|[ 0.01]|[ 0.01]|
|Elementary School|||0.19|-0.01||0.16|0.00||0.14|0.01*|
|<br>Incomplete (EF I)|2|25101|[ 0.01]|[ 0.01]|51814|[ 0.01]|[ 0.01]|47233|[ 0.01]|[ 0.01]|
|Elementary School|3|12442|0.20|-0.01|23977|0.19|-0.00|43943|0.16|0.01|
|Incomplete (EF I)|||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
|Secondary School|1|61164|0.29|0.00|56341|0.28|0.00|63320|0.25|-0.00|
|Incomplete (EF II)|||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]|
|Secondary School|||0.30|0.00||0.29|0.00||0.28|-0.00|
|Incomplete (EF II)|2|25101|[ 0.01]|[ 0.01]|51814|[ 0.00]|[ 0.01]|47233|[ 0.01]|[ 0.01]|
|Secondary School|3|12442|0.31|0.01|23977|0.30|-0.00|43943|0.28|0.00|
|Incomplete (EF II)|||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
|HihSchoolIncomlete|1|61164|0.16|-0.00|56341|0.17|-0.01*|63320|0.17|-0.01*|
|g p|||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.00]|
||||0.16|-0.00||0.16|0.00||0.17|-0.01|
|High School Incomplete|2|25101|[ 0.00]|[ 0.01]|51814|[ 0.00]|[ 0.01]|47233|[ 0.00]|[ 0.01]|
|HighSchoolIncomplete|3|12442|0.15<br>|-0.00<br>|23977|0.16<br>|-0.00<br>|43943|0.16<br>|0.00<br>|
||||[ 0.01]|[ 0.01]||[ 0.00]|[ 0.01]||[ 0.00]|[ 0.01]|
|HihShlClt|1|61164|0.18|0.00|56341|0.18|-0.00|63320|0.18|0.00|
|g coo ompee|||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
||||0.18|0.01||0.19|-0.00||0.19|-0.00|
|High School Complete|2|25101|<br>[ 0.01]|<br>[ 0.01]|51814|<br>[ 0.01]|<br>[ 0.01]|47233|<br>[ 0.01]|<br>[ 0.01]|
|High School Complete|3|12442|0.18<br>|-0.00<br>|23977|0.18<br>|0.01<br>|43943|0.19<br>|-0.00<br>|
||||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
|d|||0.65|-0.01||0.65|-0.02||0.65|-0.02|
|Pave Street|1|61822|[ 0.01]|[ 0.02]|56231|[ 0.01]|[ 0.02]|63354|[ 0.01]|[ 0.02]|
||||0.64|-0.00||0.65|-0.00||0.65|-0.01|
|Paved Street|2|25316|[ 0.01]|[ 0.02]|52237|[ 0.01]|[ 0.02]|47168|[ 0.01]|[ 0.02]|
|Paved Street|3|12602|0.65<br>[ 0.01]|-0.00<br>[ 0.02]|24137|0.64<br>[ 0.01]|0.00<br>[ 0.02]|44239|0.65<br>[ 0.01]|-0.00<br>[ 0.02]|
|CashTransfer|1|62021|0.70|0.00|56155|0.69|-0.00|63965|0.62|0.00|
||||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]||[ 0.01]|[ 0.01]|
||||0.66|-0.01||0.69|-0.00||0.69|-0.00|
|Cash Transfer|2|25389|[ 0.01]|[ 0.01]|52457|[ 0.01]|[ 0.01]|47063|[ 0.01]|[ 0.02]|
|Cash Transfer|3|12678|0.64<br>[ 0.01]|-0.00<br>[ 0.01]|24193|0.64<br>[ 0.01]|-0.01<br>[ 0.01]|44490|0.66<br>[ 0.01]|-0.00<br>[ 0.01]|
|ote: Standard errors in brak|ets, cluste|red at the s|chool level|. Statistical sig|nificance l|evels * p<0.|1 ** p<0.05 *|** p<0.01||.|

data Note: Standard errors in brakets, clustered at the school level . Statistical significance levels * p<0.1 ** p<0.05 *** p<0.01 Notwithstanding the balance on demographic characteristics and background, there is a marked imbalance in prior test performance. As Table 23 shows, in both years prior to the intervention, 1<sup>st</sup> year students in the treatment schools had significantly higher SPAECE scores in both Math and in Portuguese, of around 0.08 to 0.1 SD. In 2013, third year students in treatment schools also performed better in both subjects on ENEM. In 2014, however, the ENEM performance of students in our treatment and control schools was balanced. Although all of these test scores are for different cohorts of students than the cohort being evaluated, this degree of imbalance raises questions of bias in unobservable characteristics of these schools. Our econometric analysis uses different tests to deal with this imbalance.<sup>17</sup>

> 17 Annex table A16 shows descriptive statistics for SPAECE and ENEM 2015 scores, by treatment group.

29