progress, and therefore there would be no issue with LAYS conversion. However, in practice studies and interventions vary widely in the test used, making comparability an issue. LAYS can currently be constructed only with the data as they are, so a separate effort is needed to produce comparable underlying learning assessments.<sup>15</sup> Once that effort bears fruit, it will enhance the comparability of LAYS.

For now, therefore, we rely on standardized effect sizes. Standardized effect sizes are used to account for differences across measurement scales and express those effects in relative terms. This should prove useful when comparing effect sizes in education across various assessments and scales. However, standard deviations will not account for whether a given test is either “too hard” or “too easy,” causing floor or ceiling effects. We test for this possibility empirically by comparing standard deviations from tests on nationally representative samples, chosen to ensure that the same underlying population is represented. We focus on primary-level tests for countries that have participated in multiple tests and that have interventions featured in this paper. Appendix Figure B5 compares standard deviations for Tanzania, Malawi, and Indonesia using various assessments: HLO scores derived from EGRA, Progress in International Reading Literacy Study (PIRLS), raw EGRA or Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) tests. We find only small differences of a few points, and as a result, the estimated learning rates per year across assessments are quite similar, as shown in Appendix Figure B6. As a robustness check, Appendix Figure B6 also shows learning rates per year using raw data from the assessments before they are converted to HLO scores.

In terms of standard deviations across samples, it is well known there is high variation across sample populations. As a result, when students answer five questions correctly in a country with less variation, this will appear as a larger standardized effect size than if they had gotten the same questions correct in a country with more variation. This is a feature of standard deviation comparisons rather than a bug, since standardized effect sizes produce relative comparisons to derive a sense of magnitude. Therefore, while LAYS conceptually have the advantage that they do not need to use standard deviations as inputs, in cases where in practice they do use standardized effect sizes as inputs (as in this paper), comparisons of relative performance and rank orders should be prioritized over precise magnitudes of absolute performance.

# **5.4 Status-quo learning**

When we compute LAYS from learning estimates, we first convert learning into equivalent years of school gained. To do this, we express learning relative to status-quo learning trajectories. For

> 15Short of a universal test, a number of efforts currently underway may improve comparability. One approach focuses on a set of common test items that can be inserted into assessments and linked to regional and international assessments, such as TIMSS and PISA. The Jameel Poverty Action Lab has launched an initiative to produce an “item bank” of questions for inclusion in education interventions to produce comparable assessment. The Rosetta Stone initiative of the International Association for the Evaluation of Educational Achievement (IEA), the UNESCO Institute for Statistics (UIS), and other organizations are also working to improve comparability across regional and global assessments by having the same samples of students take both types of assessments (Sandoval Hernandez, 2022).

27