# 2.9.3 Impact Evaluation Sample

Our impact evaluation sample is comprised of those intervention and control schools which are also part of the sample for the MLSS.

# _MLSS sample_

The intervention and MLSS are run independent of each another. The MLSS sampling was conducted prior to the randomization using the same population frame.<sup>17</sup> The MLSS sampling uses probability proportional to size (PPS) to keep the sample representative not only at a national but at a sub-regional (division) level. Malawi has six education divisions across its three regions;<sup>18</sup> our strata are defined by division, with an additional stratum for all schools within urban locations. The number of schools selected within each stratum depends on the size of that stratum (section A.0.2). The final MLSS sample consists of 924 schools across all districts of Malawi and is representative at division level (see Appendix B).<sup>19</sup>

# _Overlap between treatment/control sample and MLSS_

The scale of our intervention, targeting 1,198 schools across Malawi, enables us to exploit the overlap with the nationally representative MLSS. Of the 924 schools in the MLSS sample, 386 were among the 2396 randomly assigned to treatment or control, of which 202 are treatment schools and 184 are control schools.<sup>20</sup> These schools are used for the evaluation.<sup>21</sup> The full overlap of 386 schools was included in the endline data collection and is exploited for the ANCOVA specification.<sup>22</sup> This overlap is adequately powered to detect an impact on test scores of approximately 0.2 S.D. (see section 2.9.5). This impact evaluation sample is balanced between treatment and control on key variables ( **Table 2** ).<sup>23</sup>

> 17 Like the intervention sampling, the MLSS sampling uses a population frame of all Malawi public primary schools. However, the MLSS sampling does not exclude junior primary schools.

> 18 Divisions are sub-regional administration bodies established for management of education.

> 19 MLSS baseline data (Asim and Casley Gera, 2024) shows that variability in inputs and outcomes is substantial at sub-district level (the lowest unit) and shows limited differences between three regions and six divisions (overall).

> 20 Of these, 258 are from the first cohort, of which 133 are treatment schools and 125 are control schools; and 128 are from the second cohort, of which 69 are treatment schools and 59 are control schools.

- 21 Owing to an error in implementation planning, the initial batch of training included 51 schools which were not part of the treatment group. Eight of these additional schools are in the control group. However, none of these are in the MLSS overlap and therefore not in the IE sample.

- 22 Owing to financial constraints, 337 of these were visited for the baseline round. As we employ a longitudinal sample of students, only this subset of schools is used for analyses including student test scores. There are no significant differences between this subset of schools and the rest of the impact evaluation sample (see Appendix B).

> 23 The impact evaluation sample is also balanced within each cohort (see Appendix B).

18