unlikely that the impacts of the treatment were already fully realized by the end of the term.

Based on these considerations, it was decided to exclusively target call engagement. Ultimately, the implementer valued parental engagement sufficiently to focus on maximizing call response rates, rather than attempting to choose a treatment arm based on very noisy effect estimates of fluency gains and risking inconclusive results. Another way to view this decision is to maximize learning about which arm has the highest call engagement rates, at the expense of learning more precisely which arm has the greatest ORF gains. While this solution may not be optimal, it reflects another reality of policy choice, that policymakers sometimes have to make do with imperfect data.

**Sample and Randomization.** We determined the sample using the phone number on record for the parent.<sup>18</sup> We dropped 2 schools that had fewer than 5 students, and 2 schools with very inflated ORF scores, leaving us with 108 schools with 3,163 unique student-phone number combinations.

We first randomly assigned half of the sample to wave 1 and 2 (1,581 and 1,582 phone numbers, respectively). We did not formally assess the best sample split between first and second wave, but small-sample simulations support equal-sized waves (see supplement of Kasy and Sautmann (2021a)). Before the start of each wave, parents received an introductory call, followed by a text message confirming enrollment and explaining procedures for opt out and for switching phone number. Some parents opted out explicitly and some phone numbers were invalid, leaving a sample of 1,494 in wave 1 and 1,384 in wave 2.

The randomization was stratified at the school level.<sup>19</sup> In wave 1, the assignment shares for the 6 treatment arms were equally 1/7; in wave 2, we used the assignment shares given by exploration sampling, keeping 1/7 of the sample as a control group in each wave. Due to indivisibilities, the total shares are close but not equal to the targeted shares, as shown in Table 2 in section 5.

**Estimating ORF effects.** In many applications, outcomes other than the targeted outcome are of interest to the experimenter. Here, we estimate the effects of the treatments on ORF with reading fluency exam scores obtained after the experiment was completed to learn whether the treatment arm with the highest engagement sees increases in children’s reading performance. We also briefly discuss the possibility that there are differences in how engagement with the calls translates into reading gains, which might imply that the call format with the highest call engagement may not be the format with the highest reading gains.

> 18The implementer has parental consent to use this phone number for school related communications. Based on enrollment data from the start of term 3, we randomly selected one student ID for measurement in the few cases where several student IDs were associated with the same parental phone number (likely siblings). Phone numbers and schools are de-identified by the implementer before sharing with the researchers.

> 19We also stratified assignment on whether the opt-in call or confirmation text message were answered. For example, in wave 1, a large proportion of the sample (796 student IDs) neither opted in nor explicitly opted out. However, the extensive-margin results (Appendix C.4) showed that most numbers answered the phone at least once during the experiment, and so we ignore this in the estimation.

19