metadata: authors: 0: author: Azevedo,Joao Pedro Wagner De
count: World
docna: 0: docna: Learning Poverty : Measures and Simulations
docty: Policy Research Working Paper
subtopic: Inequality,Educational Sciences,Poverty Lines,Poverty Monitoring & Analysis,Poverty Diagnostics,Poverty Impact Evaluation,Small Area Estimation Poverty Mapping,Poverty Assessment
teratopic: Education,Poverty Reduction
abstracts: cdata!: COVID-19-related school closures are
            pushing countries off track from achieving their learning
            goals. This paper builds on the concept of learning poverty
            and draws on axiomatic properties from social choice
            literature to propose and motivate a distribution-sensitive
            measures of learning poverty. Numerical, empirical, and
            practical reasons for the relevance and usefulness of these
            complementary inequality sensitive aggregations for
            simulating the effects of COVID-19 are presented. In a
            post-COVID-19 scenario of no remediation and low mitigation
            effectiveness for the effects of school closures, the
            simulations show that learning poverty increases from 53 to
            63 percent. Most of this increase seems to occur in
            lower-middle-income and upper-middle-income countries,
            especially in East Asia and the Pacific, Latin America, and
            South Asia. The countries that had the highest levels of
            learning poverty before COVID-19 (predominantly in Africa
            and the low-income country group) might have the smallest
            absolute and relative increases in learning poverty,
            reflecting how great the learning crisis was in those
            countries before the pandemic. Measures of learning poverty
            and learning deprivation sensitive to changes in
            distribution, such as gap and severity measures, show
            differences in learning loss regional rankings. Africa
            stands to lose the most. Countries with higher inequality
            among the learning poor, as captured by the proposed
            learning poverty severity measure, would need far greater
            adaptability to respond to broader differences in student needs.
repnme: repnme: Learning Poverty : Measures and Simulations
display_title: Learning Poverty : Measures and Simulations
listing_relative_url: /research/2020/10/32503844/learning-poverty-measures-simulations
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Poverty m1326687 695,Poverty Measurement and Analysis m1326941 707,Education m1327048 470,Coronavirus (COVID-19) m1939478 4828
publishtoextweb_dt: 2020-10-21T00:00:00Z
historic_topic: Education,Poverty Reduction
origu: Education Global (HEDGE)
no_of_pages: 38
author_keys: 0: author_key: 675142
model_extractions: Policy Research Working Paper 9446 Learning Poverty Measures and Simulations _João Pedro Azevedo_ Education Global Practice October 2020
Policy Research Working Paper 9446

# **Abstract**

COVID-19-related school closures are pushing countries off track from achieving their learning goals. This paper builds on the concept of learning poverty and draws on axiomatic properties from social choice literature to propose and motivate a distribution-sensitive measures of learning poverty. Numerical, empirical, and practical reasons for the relevance and usefulness of these complementary inequality sensitive aggregations for simulating the effects of COVID-19 are presented. In a post-COVID-19 scenario of no remediation and low mitigation effectiveness for the effects of school closures, the simulations show that learning poverty increases from 53 to 63 percent. Most of this increase seems to occur in lower-middle-income and upper-middle-income countries, especially in East Asia and the Pacific, Latin America, and South Asia. The countries that had the highest levels of learning poverty before COVID-19 (predominantly in Africa and the low-income country group) might have the smallest absolute and relative increases in learning poverty, reflecting how great the learning crisis was in those countries before the pandemic. Measures of learning poverty and learning deprivation sensitive to changes in distribution, such as gap and severity measures, show differences in learning loss regional rankings. Africa stands to lose the most. Countries with higher inequality among the learning poor, as captured by the proposed learning poverty severity measure, would need far greater adaptability to respond to broader differences in student needs.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The author may be contacted at jazevedo@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Learning Poverty: Measures and Simulations<sup>*</sup>

## João Pedro Azevedo<sup>♦</sup>

JEL Classification Codes: O12, O15, I21, I24, I25 Keywords: Learning Poverty, Primary Schooling, SDG, Learning Losses, COVID-19

> * I would like to thank Paola Ballon, Paul Corral, Diana Goldemberg, Amer Hassan, Ambar Narayan, Yevgeniya Savchenko, Umar Serajuddin, Shabana Singh for providing comments to earlier drafts of this paper. I would also like to thank Jaime Saavedra, Omar Arias, Christian Aedo, Harry Anthony Patrinos, Silvia Montoya, Kirsty Newman, Koen Geven, Michael Crawford, Reema Nayar, Halsey Rogers, and Syedah Aroob Iqbal for questions and discussions that inspired different parts of this paper. I would like to thank the producers of all the learning data used in this paper—the national governments and the teams at the International Association for the Evaluation of Educational Achievement (Progress in International Reading Literacy Study and Trends in International Mathematics and Science Study), United Nations Educational, Scientific and Cultural Organization (Laboratorio Latinoamericano de Evaluación de la Calidad de la Educación), Conférence des Ministres de l'Education des Etats et Gouvernements de la Francophonie (Programme d'analyse des systèmes éducatifs), and Southern and Eastern Africa Consortium for Monitoring Educational Quality, as well as the Organisation for Economic Co-operation and Development; and the staff from the education management information systems departments of all ministries of education, without whom none of this work would have been possible. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

> ♦ Lead Economist, HEDGE, Education Global Practice, World Bank.
# Highlights

The simulation representing a pessimistic scenario – where schools are closed for 7 months, and mitigation and remediation effectiveness are very low, suggests:

- In the most pessimistic scenario, COVID-19-related school closures could increase the learning poverty rate in the low- and middle-income countries by 10 percentage points, from 53% to 63%. This 10-percentage-point increase in learning poverty implies that an additional 72 million primary-school-age children could fall into learning poverty, of a population of 720 million children of primary-school age.

- Most of the potential increase in learning poverty would take place in regions with a high but still average level of learning poverty in the global context pre-COVID-19, such as South Asia (which had a 63% pre-pandemic rate of learning poverty), Latin America (48%), and East Asia and the Pacific (21%). In Sub-Saharan Africa and low-income countries, where learning poverty was already at 87% and 90% before COVID-19, increases would be relatively small, at 4 and 2 percentage points, respectively. This reflects that most of the learning losses in those regions would impact students who were already failing to achieve the minimum reading proficiency level by the end of primary—that is, those who were already learning-poor.

- To gauge the impacts of the current crisis in Sub-Saharan Africa and the Middle East and North Africa, we need to examine other indicators of learning deprivation. In these two regions, children are on average the furthest below the minimum proficiency level, with a Learning Deprivation Gap (the average distance of a learning deprived child to the minimum reading proficiency level) of approximately 20%. This rate is double the global average (10.5%), four times larger than the East Asia and Pacific Gap (5%), and more than tenfold larger than the Europe and Central Asia average gap (1.3%). The magnitude of the learning deprivation gap suggests that on average, students in those regions are one full academic year behind in terms of learning, or two times behind the global average.

- In the most pessimistic scenario, COVID-19 school closures might increase the learning deprivation gap by approximately 2.5 percentage points in Sub-Saharan Africa, the Middle East and North Africa, and Latin America. However, the same increase in the learning deprivation gap **does not** imply the same impact in qualitative terms. An indicator of the severity of learning deprivation, which captures the inequality among the learning deprived children, reveals that the severity of learning deprivation in the Middle East and Sub-Saharan Africa could increase by approximately 1.5 percentage points, versus an increase of 0.5 percentage point in Latin America. This suggests that the new learning-deprived in Latin America would remain closer to the minimum proficiency level than children in the Middle East and Sub-Saharan Africa.

- The range of policy options required to identify students’ needs and provide learning opportunities will be qualitatively different across regions — simpler in Latin America than in Sub-Saharan Africa and the Middle East.

2
son Ferrer eee rerererere ratererets eeeteretes veomoroers eeeteretet | PRT RTETETF RTRTETETRT
# Introduction

Even before COVID-19 shut schools, the world was in the middle of a learning crisis that threatened efforts to build human capital—the skills and know-how needed for the jobs of the future. More than half of children (53 percent) in low- and middle-income countries either were out of school or failed to learn to read with comprehension by age 10 and so were in learning poverty.<sup>1</sup> The most recent historical trends suggest that it will take 50 years to halve learning poverty in the developing world. COVID-19 is likely to slow that rate by deepening learning gaps and so increasing the number of children in learning poverty in many countries.

<mark>Temporary school closures in more than 180 countries have kept nearly 1.6 billion students out of school, further complicating global efforts to reduce learning poverty. Although most countries have made heroic efforts to put remote and remedial learning strategies in place, learning losses are accumulating rapidly. A recent survey on national education responses to COVID-19 by the United Nations Children’s Fund (UNICEF), the United Nations Educational, Scientific, and Cultural Organization (UNESCO), and the World Bank shows that while countries and regions have responded in various ways, they have found it difficult to reach even half of students.</mark> COVID-19-related school closures are forcing countries even further off track from achieving their learning goals. Students currently in school stand to lose $10 trillion in labor earnings over their working lives.<sup>2</sup> That is one-tenth of global GDP, or half the United States’ annual economic output, or twice the global annual public expenditure on primary and secondary education.

The objectives of this paper are twofold. First it presents some of the properties and limitations of the Learning Poverty measure and proposes complementary aggregations such as the learning poverty gap and the learning poverty severity (or inequality-sensitive-gap), while discussing their main motivations and shortcomings. Second, this paper complements other COVID-19 related simulations by looking at the learning poverty measure. The main drivers of these simulations are school closures, mitigation and remediation effectiveness and the economic shock, which affect two main transmission channels, namely, learning losses and student drop-out at the primary age-group. The rationale and motivation for this broader class of Learning Poverty measures is then used in the simulations, and are quite relevant to nuance the results, especially the regional differences observed between Latina America, South Asia, the Middle East and Africa.

This paper is organized in five sections beyond this introduction. The first section discusses some of the main properties and axioms of the learning poverty measure and proposes a few complementary aggregations, such as a learning poverty gap and a learning poverty distributional-sensitive gap (or _learning poverty severity_ ) which can address some of the discussed short comes. The second section summarizes the COVID-19 simulation methodology and introduces the main scenarios. The third section discuses data coverage and indicators used in the simulations. The fourth section presents and discusses the main results of the simulation work. The last section concludes with a few final remarks.

> 2 Azevedo et al. 2020.

1 World Bank 2019.

4
Moving beyond the mean: Identification, aggregation, properties, and axioms Choices of measurement can significantly shape understanding the size of a problem and its variation over time and space. For example, _GDP per capita_ can be used to assess a population’s prosperity, how it compares with other populations, and how it progresses through time. The _Growth Report_ used _mean income_ to identify countries with high and sustained growth.<sup>3</sup> The Sarkozy Commission, which had Joseph Stiglitz as the president, Amartya Sen as the Advisor, and J. P. Fitoussi as the coordinator, c <mark>h ose the</mark> _median income_ since it corresponds more naturally to the middle of the income distribution.<sup>4</sup> The World Bank, in the context of its twin goals to end extreme poverty and foster income growth in _the bottom 40 percent_ of the population by 2030, focused on that group.<sup>5</sup> So, the arithmetic mean is not the only possible standard for measuring a distribution’s size, and examining several standards at once can clarify the quality of growth—including whether it is shared or not.

Similarly, policy makers and researchers commonly compare educational systems over time or across countries in terms of their _average scores_ on large-scale learning assessments. An average performance can be viewed as how much learning each student could expect to achieve if all students learned the same amount. The limitation of such an approach is evident: it is hard to imagine a situation in which every child in a population achieves the same performance, given the heterogeneity of schools, teachers, peers in a classroom, family backgrounds, and intrinsic motivations. To have a richer diagnostic and a better understanding of differences between systems or of changes in performance, it is critical to move beyond the mean.

This discussion is not new. Much of the literature on measuring poverty and inequality can be seen as a systematic effort to move beyond measures of the distribution’s size (that is, the mean) and look at its spread (inequality) and base (poverty), understanding that size, spread, and base are fully interrelated, jointly determined, and complementary.<sup>6</sup> In October 2019, the World Bank and UNESCO Institute for Statistics (UIS) launched a new synthetic indicator, _learning poverty,_ based on the concept that every child should be in school and be able to read an age-appropriate text by age 10.<sup>7</sup> This formulation reflects the aspiration of Sustainable Development Goal (SDG) 4 that all children must be in school and learning.<sup>8</sup> This paper will introduce and justify other aggregation approaches complementary to the existing learning poverty indicator. This new class of measures is shown to be relevant to understand the simulation results. The new aggregations can also help describe a country’s performance over time and improve comparisons between countries.<sup>9</sup> Measuring poverty has been conceptualized as following two main steps since Sen (1976): the identification of who the poor are and the aggregation of information about poverty across a society. The first problem requires defining a standard, or poverty line, and identifying those who fall below that line and those who do not. The second problem requires a method of combining different individuals’

> 3 Commission on Growth and Development 2008.

> 4 Commission on the Measurement of Economic and Social Progress 2009.

> 5 World Bank 2015.

> 6 For a more detailed discussion, see Foster et al. 2013.

> 7 World Bank 2019.

> 8 SDG 4 makes this commitment: by 2030, signatories will “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.” The goal’s target 4.1 is to “ensure that all girls and boys complete free, equitable, and quality primary and secondary education leading to relevant and effective learning outcomes.”

> 9 Other authors have also applied the welfare measurement concept to education. Denny (2002) used the traditional Foster, Greer, and Thorbecke (FGT) class of poverty measures to compare adult literacy across 12 countries using the International Adult Literacy Survey, and Thomas, Wang, and Fan (2000) estimated education inequality using the Gini index.

5
deprivations to arrive at a population measure. Various indices of aggregation have been proposed in the literature. The Foster, Greer, and Thorbecke (FGT) family of indices is among those most widely used by countries, United Nations agencies, and the World Bank.<sup>10</sup> All these concepts can help understand the World Bank/UIS learning poverty measure.

# Identification

The identification of learning poverty is multivariate, defined through two deprivations—deprivation of schooling and deprivation of learning. Each requires a specific threshold or standard from the education domain. The first deprivation, of schooling, is ordinal, and has enrollment as its threshold. Its measurement is simple, since children attending school are directly observable, and the measure is dichotomous, as a child can only be in one of two states—in school or out of school.

The second deprivation, of learning, is more complicated. It cannot be directly observed and is measured as a cardinal latent variable using large-scale standardized assessments, which are used to derive a measure of minimum proficiency based on a desired and agreed set of competencies, leaving space for ambiguity. By comparison, when investigating deprivation in two societies, two distinct types of exercises are possible:

- **Absolute.** There is less deprivation in community A than in community B in terms of some common standard.

- **Relative.** There is more deprivation in community A than in community B in terms of the minimum needs standard of each, which could be much higher in A than in B.

As argued by Sen (1979), disputing which of the two standards is the “correct” one can be pointless, since both types of questions are of interest. The relevant issue is to note that the questions are distinct and pose different requirements. The global learning poverty measure launched in October 2019 focuses on absolute deprivation.

The exercise requires a common standard. Thanks to UIS-led work in the context of the SDG 4.1, countries and development partners have converged on an agreed definition of a minimum proficiency level (MPL) in literacy through the Global Alliance to Monitor Learning (GAML).<sup>11</sup> The GAML also identified a list of large-scale assessments valid for this process and a threshold for each assessment equivalent to the conceptual standard of minimum proficiency.<sup>12</sup> In addition to identification and aggregation, it is also critical to define the space in which the deprivations and poverty are to be assessed. Given the focus on capturing the conditions of children at about age 10, both schooling and learning are defined at the end-of-primary space. Specifically, learning is defined in the space of reading at the end of primary school.<sup>13</sup> The bivariate definition of this indicator assumes that education systems have two important functions: keeping children in school and making sure that they are learning. The SDGs and the 2018 _World Bank World Development Report_ have shown that a considerable share of children attending school are not

> 10 Foster et al. 2010.

> 11 The GAML aims to improve learning outcomes by supporting national strategies for learning assessments and developing internationally comparable indicators and methodological tools to measure progress toward key targets of SDG 4. It was established by the UNESCO Institute for Statistics, which also hosts GAML’s secretariat.

> 12 For more details on the assessments used in this exercise and their respective thresholds, see <u>https://github.com/worldbank/LearningPoverty/blob/master/00_documentation/001_technical_note/Technical_Note.md#equ ating-across-assessments.</u>

> 13 For a discussion of this choice, see World Bank (2019).

6
learning.<sup>14</sup> But this should not be seen as an indication that children should be learning outside the school system. As the COVID-19 school closures have shown, schools have several other functions to protect and improve children’s health and well-being,<sup>15</sup> such as safety, nutrition,<sup>16</sup> socialization, facilitate parents' labor market participation,<sup>17</sup> and at the macro level can help build social cohesion,<sup>18</sup> democracy and peace,<sup>19</sup> All those complementary functions mean that schooling has an intrinsic value over and above learning and justifies its deprivation as a relevant component of learning poverty.<sup>20</sup> Hence, learning poverty is multivariate and unidimensional, using a nested approach. Table 1 illustrates learning poverty classification using a 2 x 2 contingency table based on the two deprivations, schooling and learning.<sup>21</sup>

**Table 1 Contingency table of schooling and learning deprivations and learning poverty classification**

|||Learning<br>|deprivation<br>|
|---|---|---|---|
|||No|Yes|
|vation|No|_Not School Deprived_and_Not_<br>_Learning Deprived_<br>=|_Not School Deprived_and_Learning_<br>_Deprived_<br>_=_|
|depri||**Not Learning Poor**|**Learning poor**|
|Schooling|Yes|_School Deprived_and_Not Learning_<br>_Deprived_<br>_=_<br>**Learning poor**|_School Deprived_and_Learning Deprived_<br>_=_<br>**Learning poor**|

Note: Given the anchoring on the learning poverty measure on the SDGs, learning deprivation at the End of Primary is measured using schoolbased learning assessments. Hence, this measure requires the combination of two data sources, namely, school-based learning assessments used 14 World Bank 2018.

15 UNICEF 2020; WFP 2020.

16 Bhutta et al (2013) and Adelman et al (2008).

17 Blau and Currier (2006) and Blau and Robins (1988).

18 Easterly et al (2006).

19 Khan (2016).

20 Combing an ordinal (binary) and a cardinal variable always implicitly puts more weight on the ordinal data, since they always get the full score of 1. This is a problem since the measure always puts more weight on the deprivation in the ordinal variable. Alkire and Foster (2011) handle this problem by only using the ordinal form of the cardinal variable as well. The headcount measure of Learning Poverty uses the MPL to transform learning from cardinal to ordinal.

21 This measure combines a direct measure and an indirect measure to identify an underlying learning poverty concept, using two deprivation indicators, schooling and learning, in the space of reading at the end of primary. The direct measure identifies those whose basic need for schooling fails to meet the accepted minimum standard—to be enrolled in school when at the proper age to be at the end of primary. The indirect measure, learning, identifies those who do not to meet a minimum proficiency level as defined by their reading assessment for that age and grade. A learning poor child, on this approach, is one who is not in school or whose reading is not adequate to meet the specified minimum as defined by the GAML process. The indirect measure has the advantage of providing a metric of learning shortfalls through numerical distances from an agreed threshold—something the direct measure does not provide. The indirect method is stringent since it sets three preconditions for the exercise: (1) agreement on a common standard of reference, (2) the availability of large-scale learning assessments of sufficient quality, and (3) the benchmarking of the learning assessment’s competence levels against the standard agreed in the GAML process.

7
to measure the learning deprivation; and EMIS and Population Census or Household Surveys which are required to measure to schooling deprivation. To combine these two data sources, we use a nested measure, in which all out-of-school children are assumed to be learning deprived. As a consequence, we do not observe the joint distribution of learning and schooling.

In the unidimensional and univariate cases, poverty and deprivation are the same, since the identification of someone as poor depends on a single deprivation, and the choice of standard will automatically divide the population into poor and nonpoor.

But the multivariate context is more complicated, since a nested identification criterion determines learning poverty status. In that case, the first observation is whether a child is in school, and the second is whether he or she is learning by a certain threshold. If the first threshold is not satisfied, a child will be considered in learning poverty, regardless if he or she achieves the learning MPL. But if a child is in school, only if the second identification criterion is also satisfied will the child be considered in learning poverty. So, in the nested approach, first schooling is assessed, and then learning. If either of the identification criteria is not met, a child is considered in learning poverty.

# Aggregation

Aggregation in this exercise will implicitly follow the most straightforward procedure, a headcount rate. That is, it will use the number of children below either deprivation threshold divided by the total number of children in the age category. This ratio, learning poverty (LP), is extremely simple and clear for policy makers to interpret, given the observable nature of school enrollment and the use of an agreed common standard of proficiency defined in the context of the SDGs.

Learning poverty can also be expressed in terms of the equations below.

The aggregation step of this deprivation is the simple ratio of the number of students, _G,_ who satisfy the identification criterion for learning deprivation, divided by the total number of students _S._ The number of students who are deprived in this dimension is . 1 B �=qq≤OO = (eq. 2) qq 8
we ~~l +0~~ ~~(-~~ ) ~~(-)~~ LP(x )=((1-Oos¢e ))xBMP(x +) + (O0S(e__+))
**Table 2 Using a headcount ratio to measure the quality of the education system**

|Cumulative distribution||I. Observe|d learning|distribution||II|. Identificat|ion of the|learning po|or|
|---|---|---|---|---|---|---|---|---|---|---|
||A|B|C|D|E|A|B|C|D|E|
||||Mean scor|e|||Le|arning pov|erty||
|Aggregation|366|366|366|366|366|50%|50%|50%|60%|70%|
|10%|258|46|0|117|47|1|1|1|1|1|
|20%|258|58|0|171|132|1|1|1|1|1|
|30%|258|69|247|226|216|1|1|1|1|1|
|40%|258|370|247|280|301|1|1|1|1|1|
|50%|258|381|247|334|385|1|1|1|1|1|
|60%|400|462|493|396|385|0|0|0|1|1|
|70%|440|508|542|477|397|0|0|0|0|1|
|80%|480|554|592|521|564|0|0|0|0|0|
|90%|510|589|629|553|599|0|0|0|0|0|
|100%|535|618|660|580|629|0|0|0|0|0|

Source: Author’s calculations. Deprivation cut-off: school enrollment and learning score below 400 points. The green and red colors denote the deprivation’s intensity, with dark red being the maximum deprivation and dark green being the minimum deprivation.

A known limitation of the headcount ratio is its insensitivity to changes of the cardinal variable. In this case any changes in learning below the MPL will not affect the headcount rate, in other words, if learning among the in-school population increases below the MPL, learning poverty does not fall. Given the ordinal nature of the schooling deprivation, the same consideration does not apply.

# Properties and Axioms

As noted in the previous subsection, Learning Poverty is a cardinal measure nested within an ordinal one. The cardinal nature of one if its dimensions allows an interesting and relevant discussion on how to aggregate it, since its ordinal transformation using the MPL threshold is just one of many possibilities. But how to decide the best aggregation?

Since Sen (1976), the main approach followed has been to use a set of social choice properties and axioms which can help define which desirable characteristics an aggregation method should satisfy.<sup>23</sup> It is worth noting that an axiomatic discussion of education measures is not unprecedented. Basu and Foster (1998), Denny (2002), and Basu and Lee (2009) use a similar approach to discuss adult literacy measures. This paper however, is the first, to the author’s knowledge to apply those criteria for measures of learning in the space of basic education.

> 23 See Zheng (1997) for a review of poverty measure axioms.

10
This paper identifies a few properties and axioms which an education measure might desire, namely, _focus_ , _monotonicity_ , _monotonicity sensitivity, subgroup consistency_ and _additive decomposability_ .

- **Focus axiom:** This axiom requires that if learning of a non-learning poor student in a society changes but does not fall below the MPL, then the level of learning poverty should not change. This property ensures that the measure focuses on the learning deprived only when evaluating learning poverty.

- **Monotonicity axiom** : Other things equal, if learning falls below the MPL, the measure must rise, or at least not decrease. There are two version of this property. One is _weak monotonicity_ , which requires that learning poverty should not fall because of a reduction of learning of a learning deprived student. The other is _monotonicity_ , the stronger version, which requires that learning poverty should rise if learning of a learning deprived student falls.

- **Monotonicity sensitivity axiom:** The poorer the student is (in the sense of learning poverty), the more sensitive a learning poverty measure should be to a drop or increase in that student’s learning.<sup>24</sup>

Traditionally the poverty and inequality measurement literature uses one additional axiom motivated by Pigou-Dolton’s transfer principle from the perspective of welfare economics, namely, _transfer,_ which states that a progressive (regressive) transfer of income between two persons decrease (increase) poverty<sup>25</sup> . To avoid unnecessary confusion in the context of basic education, this paper avoids the wording of the _transfer_ and _transfer sensitivity axiom_ , since learning is not a sum zero game, and it can be shared across students at no one’s detriment.

However, the properties of these axioms can still be relevant and can be motivated in terms of the learning process. There is a shared understanding learning in particular is a progressive and cumulative process.<sup>26</sup> As such, in order to have students or a student population performing above a specific threshold is critical to gradually improve performance, and to take into consideration initial conditions.<sup>27</sup> In this spirit, this

> 24 It should be pointed out that the monotonicity-sensitivity axiom is not independent of Sen's monotonicity and transfer axioms. In fact, monotonicity and monotonicity-sensitivity would necessarily imply in a measure that fulfil the Transfer Axiom. (Kakwani, 1980). Donaldson and Weymark (1986) distinguished four different transfer axioms by incorporating the possible effects and directions of transfers, namely, _Minimal Transfer Axiom_ , _Weak Transfer Axiom_ , _Regressive Transfer Axiom_ , and _Progressive Transfer Axiom_ . The core of these four transfer axioms is that an equalizing transfer (from a richer person to a poor person) should decrease the poverty value, while a disequalizing transfer (from a poor person to a richer person) should increase the poverty value. By definition, _minimal transfer_ is the weakest form among these four axioms while _progressive transfer_ is the strongest form, i.e., _progressive transfer » regressive transfer » weak transfer » minimal transfer_ . The _monotonicity-sensitivity axiom_ is identical to the _minimal transfer axiom_ (Zheng, 1997).

> 25 The transfer axiom does not specify the extent to which the decrease or increase should be.

> 26 From a developmental psychology perspective see Piaget (1936, 1957); from a policy perspective see the Plowden report (1967); from a measurement perspective see von Davier, M. & Lee, Y.S. (2019); and, from an empirical perspective see Williamson (2018), and Welch and Dunbar (2014).

27 From a student perspective, the initial condition argument implies that one needs to be able to differentiate between learning that takes place at different levels of deprivation among the learning poor, and as an “education planner” one might be interested in assigning greater weight among those students who have weaker foundational skills (or are at the very bottom of the cardinal scale). A distributional-sensitive measure allows for that differentiation. From a population perspective, one can argue that countries that want to move their student population above the MPL, given the progressive and cumulative nature of learning, need to pay attention to the foundational skills of the their student population, and need to gradually create targeted interventions to lift those at the bottom of the learning scale. For that again, a measure that is distributional sensitive is very important.

11
paper uses the _monotonicity sensitivity-axiom_ (this is equivalent to the _transfer-sensitivity axiom_ ) as an additional axiom which can be extremely relevant to measure the progress of educational systems.

The properties of _subgroup consistency_ and _additive decomposability_ are quite useful to understand how much of the overall learning poverty can attributed to the learning poverty of a particular group, or in the nested case, to each one of its components, given that learning poverty can be expressed as a collection of population subgroups. This property requires that the overall learning poverty can be expressed as a population-weighted average of subgroup learning poverty levels, or by the different components of the nested measure.

The learning poverty headcount ratio respects the focus axiom, since it is clearly not sensitive to changes in learning above the deprivation cut-offs. But it violates the stronger version of the monotonicity axiom and monotonicity-sensitivity axiom.

Even so, why would the axiomatic properties often discussed in the poverty and inequality literature also matter in education? In the poverty and inequality literature, the properties are often supported in terms of hypothetical income transfers that have no direct analog in education, since one cannot physically transfer a unit of learning from one individual to another.

Since school-age children should be learning in school settings, given the multiple externalities that those settings offer, the axioms are relevant to comparing educational systems and school-age populations, both in regular times and after exogenous shocks such as the COVID-19 pandemic. The case for monotonicity is simple and builds on the fact that a headcount measure does not distinguish between different levels of learning deprivation among those below the chosen threshold.

There are three main arguments to seek a measure that would satisfy the _monotonicity axiom_ in education. First, it seems plausible that children or educational systems with lower scores among the poor are worse off, other things equal, a difficulty that then an aggregation should reflect. Being worse off means that the lower level of learning is associated with an inefficiency and ineffectiveness in the educational system which fails to provide good quality education to all, as stated in the SDGs. Second, the world is in a learning crisis, with 53 percent of children at age 10 unable to read and understand a simple age-appropriate text.<sup>28</sup> In certain regions such as Africa and country groupings such as low-income countries, the learning poverty headcount ratio approaches 90 percent. In that context, it is critical to use a measure that differentiates levels of learning poverty among countries with similar headcount ratios. Third, financial requirements might differ depending on the level of deprivation in a country's school-age population.<sup>29</sup> The relevance of the monotonicity-sensitive axiom in the context of school-age populations is even greater. First, as previously mentioned, learning is a cumulative and progressive process that requires time, and a good measure should be able to capture foundational changes, which take place at lower levels of the scale. Such sensitivity can help policy makers prioritize foundational learning and prevent the understating of the progress that their educational system might be achieving when investing in foundational skills. Second, there is overwhelming evidence that <mark>teaching at a level too high for students’ proficiency has</mark> a detrimental effect on how much they learn, so a good measure should be able to take student’s initial conditions into consideration.<sup>30</sup> Third, a distribution-sensitive measure captures an important dimension of the complexity of the problem—how unequal the distribution of learning among

> 28 World Bank 2019.

> 29 As Denny (2002) suggests, in allocating funds to alleviate the low learning of different subpopulations with the same sample incidence of low learning, it is clearly helpful to know whether the severity of the problem differs across groups, since more resources would typically be needed.

> 30 Banerjee et al. 2016.

12
the learning poor is. The greater the inequality of the learning poverty gap in a population of interest, the more flexible the educational system must be to adapt to students’ needs and effectively <mark>reduce learning poverty, by both identifying the students‘ needs and offering learning opportunities which are adequate to their needs. Understanding such heterogeneity can be of critical importance on the planning stage of structured lesson programs and/or designing Teaching at The Right Level interventions.</mark> <mark>Several additional justifications of u</mark> sing distribution-sensitive measures can be <mark>grounded on egalitarian preferences for society. Such as, given a choice between improving the learning poverty of two children by the same amount, societies preference will be toward the learning poorer one. Several arguments can support this value judgment, such as the diminishing marginal value of levels of literacy and the importance of social cohesion. Even so, such societal preferences are not needed to justify a technical preference for a distribution-sensitive measure of learning poverty.</mark>

# Alternative aggregations and a numerical illustration

Fortunately, several classes of poverty measures fulfill all the proposed axioms. One of them can provide a numerical and empirical illustration of these axioms’ usefulness and relevance. Foster, Greer, and Thorbecke (1984) proposed a broad class of aggregation approaches that are robust to all three axioms, under specific values of a parameter α. The FGT family of measures has the following formulation. Say _x = (x1, x2, ..., xn)_ is a vector from the lowest to highest learning scores. An MPL line _z_ is a threshold such that people whose learning scores are lower than _z_ are defined as deprived in this dimension, but not otherwise. The number of people who are deprived is _q < n_ .

The parameter α measures the index's sensitivity to the degree of “poverty aversion” or the dislike for poverty. This index nests several special cases. If α = 0, equation (2) corresponds to a simple headcount— the proportion of those below the threshold. Setting α = 1 amounts to aggregating the proportionate poverty gaps, whereas setting α = 2 weights each proportionate gap by itself (or the value of α), so a poverty gap of 20 percent contributes four times as much as a gap of 10 percent. As α increases, greater weight is placed on the poorest in society. It can be proven that equation (2) satisfies the monotonicity axiom for α > 0, the weak monotonicity-sensitive axiom for α > 1, and the strong monotonicity-sensitivity axiom for α > 2.

Given that the family of FGT measures has subgroup consistency and is additively decomposable,<sup>31</sup> it is possible to generalize the LP measure to any value of α and combine equations 4 and 5 as:

> 31 Foster, Greer, and Thorbecke 1984.

13
= (a —OoS(x _-)) X BMP(x ;«)) + (OoS(x = ((1 — 00S) x BMP(x ))
**Table 3 The value of the monotonicity axiom (α=1)**

|Cumulative<br>distribution|I. Ob<br>|served<br>|learnin<br>|g distrib<br>|ution<br>|II. Iden<br>|tificatio<br>|n of the<br>|learnin<br>|g poor<br>|III. I<br>|dentificat<br>|ion of lea<br>|rning sho<br>|rtfall<br>|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||A|B|C|D|E|A|B|C|D|E|A|B|C|D|E|
|||M|ean sc|ore|||Lear|ning po|verty||Learn|ing pove|rty gap (a|mong the|poor)|
|Aggregation|366|366|366|366|366|50%|50%|50%|60%|70%|36%|54%|63%|37%|33%|
|10%|258|46|0|117|47|1|1|1|1|1|0.36|0.88|1.00|0.71|0.88|
|20%|258|58|0|171|132|1|1|1|1|1|0.36|0.86|1.00|0.57|0.67|
|30%|258|69|247|226|216|1|1|1|1|1|0.36|0.83|0.38|0.44|0.46|
|40%|258|370|247|280|301|1|1|1|1|1|0.36|0.08|0.38|0.30|0.25|
|50%|258|381|247|334|385|1|1|1|1|1|0.36|0.05|0.38|0.16|0.04|
|60%|400|462|493|396|385|0|0|0|1|1|0|0|0|0|0.04|
|70%|440|508|542|477|397|0|0|0|0|1|0|0|0|0|0|
|80%|480|554|592|521|564|0|0|0|0|0|0|0|0|0|0|
|90%|510|589|629|553|599|0|0|0|0|0|0|0|0|0|0|
|100%|535|618|660|580|629|0|0|0|0|0|0|0|0|0|0|

Source: Author’s calculations. Panels I and II are the same as in table 2. The green and red colors denote the deprivation’s intensity, with dark red being the maximum deprivation and dark green being the minimum deprivation. Learning Poverty Gaps computed using equation 6 when α=1.

The monotonicity sensitivity axiom (α > 1) is a stronger restriction on the class of admissible indices and effectively depends on the distribution (as opposed to the level or distance) of literacy within those who are below the threshold. Thus, a distribution-sensitive learning poverty measure means that reducing learning poverty by a simple increment of learning is sensitive not only to the amount of the increment but also to the initial learning conditions of the students.<sup>33</sup> This is extremely relevant in education, since initial conditions matter and the transition from “learning to read” to “reading to learn” requires the acquisition of a set of foundational skills.

A “learning poverty–averse” education planner would be expected to dislike learning poverty and to desire all learning poor students to be raised to the minimum proficiency. The monotonicity axiom suffices to describe that. But the notion of learning poverty aversion in education goes farther than just disliking the problem, given substantial evidence that to achieve minimum proficiency, students must demonstrate a sequence of abilities with incremental increases in complexity. Education planners know that the types of learning required by students at different skills levels are different, so an ideal measure should distinguish among them.

Table 4 shows five countries with significantly different learning poverty distributions, but the same mean, learning poverty, and learning poverty gap. Imposing only monotonicity (learning poverty gap aggregation) would imply that the situations in all five countries are indistinguishable. That false reading would follow from failing to distinguish which countries’ gaps go up and down among the poor population.

> 33 Zheng 2000.

15
But for the monotonicity-sensitive axiom to be satisfied, gains for the poorest must be weighted more heavily than gains for the less poor. The levels of deprivation of the five countries can only be distinguished if the aggregation respects the monotonicity-sensitive axiom. Then countries A and B would appear very similar, countries C and E slightly worse off, and country D, with 10 percent of children out of school, is almost twice as bad. One interesting aspect of this aggregation class is the much greater sensitivity to the out-of-school problem, something lost in the headcount ratio and not necessarily captured by the learning gap.

**Table 4 The value of the monotonicity sensitivity axiom (α=2)**

|Cumulative<br>distribution|I. Ob<br>|served|learning<br>|distrib|ution|II. Ide<br>|ntificatio|n of the<br>|learning poor|III. Ide<br>|ntificati|on of le<br>|arning sh|ortfall|VI. Id<br>|entificat|ion of le<br>square<br>|arning s<br>d|hortfall|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||A|B|C|D|E|A|B|C|D<br>E|A|B|C|D|E|A|B|C|D|E|
|||M|ean sco|re|||Lear|ning po|verty|Lear|ning po|verty ga<br>poor)|p (amon|g the|Learn|ing pov|erty seve<br>poor|rity am|ong the|
|Aggregation|366|366|366|365|366|50%|50%|50%|50%<br>50%|36%|36%|36%|36%|36%|13%|13%|14%|23%|15%|
|10%|258|208|188|0|200|1|1|1|1<br>1|0.36|0.48|0.53|1.00|0.50|0.13|0.23|0.28|1.00|0.25|
|20%|258|258|258|321|210|1|1|1|1<br>1|0.36|0.36|0.36|0.20|0.48|0.13|0.13|0.13|0.04|0.23|
|30%|258|258|258|321|240|1|1|1|1<br>1|0.36|0.36|0.36|0.20|0.40|0.13|0.13|0.13|0.04|0.16|
|40%|258|258|258|321|280|1|1|1|1<br>1|0.36|0.36|0.36|0.20|0.30|0.13|0.13|0.13|0.04|0.09|
|50%|258|308|328|328|360|1|1|1|1<br>1|0.36|0.23|0.18|0.18|0.10|0.13|0.05|0.03|0.03|0.01|
|60%|400|400|400|400|400|0|0|0|0<br>0|0|0|0|0|0|0|0|0|0|0|
|70%|440|440|440|440|440|0|0|0|0<br>0|0|0|0|0|0|0|0|0|0|0|
|80%|480|480|480|480|480|0|0|0|0<br>0|0|0|0|0|0|0|0|0|0|0|
|90%|510|510|510|510|510|0|0|0|0<br>0|0|0|0|0|0|0|0|0|0|0|
|100%|535|535|535|535|535|0|0|0|0<br>0|0|0|0|0|0|0|0|0|0|0|

Source: Author’s calculations. Panels are not the same as those in tables 2 and 3. The green and red colors denote the deprivation's intensity, with dark red being the maximum deprivation and dark green being the minimum deprivation. Learning Poverty Severity or Gap-Inequality computed using equation 6 when α=2.

In the context <mark>of the FGT class of measures, the index becomes distribution sensitive with α > 1, and the value (α−1) can be interpreted as a measure of learning poverty aversion or for a foundational learning preference. If α took the value 1, the index becomes the learning poverty gap ratio and would exhibit no foundational learning preference. Since helping a learning poor student just below the threshold would be equality impactful as to help a student with significant foundational learning deficits. However, for 1</mark> < α ≤2 , an increase in learning will have a greater impact on children close to the threshold, or an increasing <mark>learning poverty aversion. While, if α</mark> > 2 the measure will present diminishing poverty aversion, meaning as the learning deprivation level incases, the smaller the impact on the measure <mark>. These</mark> 16
<mark>properties can help the “education planner”, who can adjust this parameter to have a measure more or less sensitive to different preferences of the educational system, or policy objectives, which might be to tackle the weakness of students just below the MPL, in which case the</mark> 1 < α ≤ <mark>2 would be preferable, or prioritize students with very week foundational skills, in which case the preference should be for an</mark> α > 2 . Such property can also be useful to diagnose educational systems and identify if they are operating under increasing and diminishing learning poverty aversion practices.

It must be noticed that the choice of framing out-of-school as a necessary and sufficient condition for a child to be in learning poverty, given by the nested structure of the measure, implies that the higher the aversion-to-learning poverty ( <mark>α>0)</mark> , the greater will be the relative contribution of the out-of-school deprivation to the overall measure. This can be easily seen by decomposing the class of Learning Poverty measures, in terms of the schooling and learning contributions. This is possible given the subgroup consistency and additive decomposability of the Learning Poverty extension into an FGT-type class of measure. Table 5 expands the case of country D, presented in Table 5, which has a 10% of the population or 20% of the learning poor population out-of-school. As it can be seen, the higher the “learning poverty aversion”, the higher will be the relative contribution assigned to schooling. In the case of country D, the out-of-school accounts for 20%, 56% and 87% of the overall learning poverty headcount ( <mark>α=0)</mark> , gap ( <mark>α=1)</mark> and severity ( <mark>α=2)</mark> , respectively. However, as the schooling deprivation falls, reflected by the reduction of the share of out-of-school between examples D1 and D10, for a given level of learning deprivation, the relative contribution of the out-of-school deprivation will rapidly accompany the reduction.

When using these alternative measures, one should never lose sight of this property. It is reasonable to assume that different levels of out-of-school population, will have very different root causes, which in turn will require distinct policy responses. In countries with a high levels of out-of-school children, such as D to D5, it might be reasonable to assume they might still have supply problem, which might justify unbundling the two measures and analyzing the learning deprivation component (at different parameters of learning-poverty-aversion) separately from the out-of-school. However, in countries with a low share of out-of-school children, such as D6 to D9, assessing the learning deprivation jointly with the out-of-school might be more informative since the quality of learning might be one of the main drivers of student’s retention and eventual drop-out. In any case, the existence of an analytical solution to decompose the contribution of these two components is a useful property of this class of Learning Poverty measures. The final choice of presentation and emphasis must take into consideration the use case and context in which this measure is being applied. In this paper, Annex A2 presents the results only in terms of learning.

17
**Table 5 Decomposing Learning Poverty in terms of schooling and learning**

|Country|Out-of-Sc<br>Among the|hool<br>Among|Learning<br>Headcount De<br>|Poverty<br>composition<br>|Learning Po<br>Decomp<br>|verty Gap<br>osition<br>|Learning Pov<br>Decomp<br>|erty Severity<br>osition<br>|
|---|---|---|---|---|---|---|---|---|
||<br>Population|<br>the Poor|Schooling|Learning|Schooling|Learning|Schooling|Learning|
|**D**|**10%**|**20%**|**20%**|**80%**|**56%**|**44%**|**87%**|**13%**|
|D1|9%|18%|18%|82%|51%|49%|78%|22%|
|D2|8%|16%|16%|84%|45%|55%|70%|30%|
|D3|7%|14%|14%|86%|39%|61%|61%|39%|
|D4|6%|12%|12%|88%|34%|66%|52%|48%|
|D5|5%|10%|10%|90%|28%|72%|43%|57%|
|D6|4%|8%|8%|92%|23%|77%|35%|65%|
|D7|3%|6%|6%|94%|17%|83%|26%|74%|
|D8|2%|4%|4%|96%|11%|89%|17%|83%|
|D9|1%|2%|2%|98%|6%|94%|9%|91%|
|D10|0%|0%|0%|100%|0%|100%|0%|100%|

Source: Author’s calculations. The example D numbers match the results presented in Table 4, Panels II, Panels III and IV. Examples D to D10, have the same learning deprivation, at the level of D (Table 4, Panel I), however, the share of out-of-school children decline from 10% of the population, or 20% of those in Learning Poverty, to 0%.

As a final note in this section, the FGT is only one of several classes of measures that fulfill these axioms.<sup>34</sup> The index by Clark and associates, as well as special cases such as the Watts index and Chakravarty index, are also valid distribution-sensitive measures.<sup>35</sup> Some of them with different properties, in respect to the FGT. For example, as the learning level increases, the Clark index exhibits a diminishing poverty aversion while the Foster index has an increasing poverty aversion for 1 < α ≤2 . An increase in learning will have a greater impact on children close to the threshold in the case of the FGT class, while the marginal increase will be smaller the closer the student is to the threshold in the Clark index. Note that both indices ( α > 2 for the FGT class) present diminishing poverty aversion.

# COVID-19 simulations: Methodology and scenarios

Simulating the effect of COVID-19-related school closures on learning poverty requires, as previously discussed, simulating the effects on both learning and schooling deprivations. Building on the methodology introduced by Azevedo et al. (2020), these simulations add three main contributions: the focus on learning at the end of primary, the inclusion of school enrollment effects due to the household income shock, and the inclusion of a remediation effectiveness component.

A parameterized Lorenz specification and income–school enrollment elasticities are used to simulate the short-term effects of school closures, mitigation, remediation responses, and economic contraction on the learning poverty headcount ratio, learning poverty gap (population normalized and among the poor), and learning poverty severity (population normalized and among the poor).

Using a parameterized Lorenz allows working with national learning assessments (NLAs) in which only grouped data are available and implementing the simulations in a computationally efficient manner while respecting both the sample and the test design.<sup>36</sup> This procedure relies on simple summary statistics at the country level (15 equally spaced bins with the average test score in reading), computed using sample

> 34 Zheng 2000.

> 35 Clark, Hemming, and Ulph 1981; Chakravarty 1983.

> 36 For a detailed discussion of common errors made by economists while using large-scale international assessments, see Jerrim et al. (2017).

18
weights, replication weights, and the assessment’s plausible values. These data are then used to estimate the Lorenz parameters.<sup>37</sup> The Lorenz curve captures the pattern of _relative learning inequalities_ in the student population. It is independent of any considerations of the absolute learning level. The share of students below a proficiency level captures an absolute standard of the student population. Two functional forms—the Beta Lorenz<sup>38</sup> curve and the General Quadratic (GQ)<sup>39</sup> Lorenz curve—were tested for calculating the Lorenz curve parameters. For this exercise, the General Quadratic (GQ) Lorenz curve was preferred, since it provided better results in terms of both internal and external validation.<sup>40</sup> This exercise computes learners’ share below the end-of-primary minimum proficiency level (MPL), the average learning gap with respect to the MPL, and the average learning gap severity for the same MPL.

Some of the main assumptions of this work are:

- A focus on first-order effects of COVID-19-related school closures in the crisis’s first 12 months. Given the definition of the indicator in terms of children at the end of primary, the work does not look at cumulative effects at other grades.

- Children in high-income countries are learning more than children in low-income countries. The expected learning gains, based on the literature and empirical work, are 0.5 of a standard deviation for high-income countries, 0.4 for upper-middle-income countries, 0.3 for lowermiddle-income countries, and 0.2 for low-income countries.

- Dropout estimates used are based on October 2020 growth projections and consider only children in primary (ages 4–11). The expected dropout rate of primary school age children is substantially smaller than that of secondary school age children (35% of the total Primary and Secondary drop out), given the greater income generating opportunities of secondary school age children and far greater supply constraints in the developing world. The dropout estimates are only affected by income shock, and ignore the potential effects of concerns about school safety concerns and of school disengagement, which are likely to be relevant but are extremely hard to measure at this stage. If anything, these further effects have a clear upward bias, so the out-of-school numbers presented here can be seen as lower-bound estimates. In any case, this crisis is not over, and our understanding of the ramifications to the economy and household welfare are being updated daily.  Since March 2020 global growth projections have been frequently revised, and the recently released Global Economic Prospects indicates that growth projections are likely to continue to go down. In each of these revisions, our expected number of student’s dropping out due to the household income shock is revised upwards. Our initial estimate, based on the March MPO suggested that approximately 1.4 million primary students would drop out of the education system; by October, this number had already been revised to 3.8 million, and is likely to be revised further upward based on revisions to the magnitude of the economic recession (Figure 1).

> 37 For details in the methodology, see Azevedo et al. (2020).

> 38 Kakwani 1980.

> 39 Villasenor and Arnold 1989.

> 40 Lorenz parameters were estimated using the user written Stata ADO function GROUPDATA, available at <u>https://github.com/jpazvd/groupdata.</u>

19
**Figure 1  Estimates of primary student dropouts by 2020 growth projections release**

<!-- Start of picture text --> 0.0 4.5<br>MPO-Mar WEO-Apr MPO-May MPO-Oct<br>4.0<br>-1.0 -0.9 3.8<br>3.5<br>-2.0 3.0<br>2.9<br>-2.4<br>2.5<br>-3.0 2.3<br>-3.4<br>2.0<br>-4.0 1.5<br>1.4<br>1.0<br>-5.0<br>-5.1<br>0.5<br>-6.0 0.0<br>Projection Release Date<br>Growth Projections 2020 (p.p.)* Number of Primary School drop-outs (in millions of students)<br>(in millions of students)<br>Growth Projections 2020 (p.p.)*<br>Number of primary school drop-outs<br><!-- End of picture text --> Note: GDP per capita growth projections in for 2020 from WB-MPO (March, 2020); IMF-WEO (April, 2020); WB-MPO (May, 2020); and WB-MPO (Oct, 2020); (*) Growth projections are weighted by the student cohort of each country, and will different from other global averages reported in the original publications, in are weighted by GDP.

# **Scenarios**

- All scenarios assume that, as of today, remote learning cannot fully match face-to-face delivery. <mark>Although most countries have made heroic efforts to put remote and remedial learning strategies in place, learning losses are accumulating rapidly. A recent survey on national education responses to COVID-19 by UNICEF, UNESCO, and the World Bank shows that while countries and regions have responded in various ways, only half of the initiatives are monitoring usage of remote learning (Figure 2, top panel).  Moreover, where usage is being monitored, the remote learning is being used by less than half of the student population (Figure 2, bottom panel), and most of those cases are online platforms in high- and middle-income countries.</mark>

20
<!-- Start of picture text --> What share ofthe use of different remote learning strategies are being monitored? (%)<br>online wv radio paper<br>100%<br>80%<br>60%<br>40%<br>20%<br>0%<br>HIC UMC LMC uc HIC UMC Lc LIC HIC UMC LMC LIC HIC UMC LMC uc<br>What is the average share (%) of children/youth accessing each remote learning system?<br>80 online wv radio paper<br>73 73<br>70<br>60<br>60 57 58<br>50<br>40 37 41 35 38 38<br>30 29<br>23<br>20 19<br>10 7<br>0 0 | 0<br>HIC UMC LMC uc HIC UMC LMC LIC HIC UMC LMC LIC HIC UMC uMc LIC<br><!-- End of picture text -->
- Remediation: approximately 60 percent of the school loss will be fully remediated.

- Mitigation: approximately 40 percent of the school loss while schools are closed will be fully mitigated in High-income countries, but for the developing world, 30 percent.

## `o` Intermediate

- Remediation: approximately 30 percentage of the 70 percent school loss will be fully remediated.

- Mitigation: approximately 20 percent of the school loss while schools are closed will be fully mitigated in High-income countries, but in the developing world, 15 percent.

- Pessimistic

- Remediation: no remediation.

- Mitigation: approximately 10 percent of the school loss while schools are closed will be fully mitigated in High-income countries, but in the developing world, 7 percent.

# Data: Coverage, description, and indicators

The simulations use the same underlying data used to construct the original learning poverty measures, covering 80 percent of children in low- and middle-income countries.<sup>43</sup> These countries had at least one learning assessment at the end of primary, carried out in the past eight years, that is of sufficient quality<sup>44</sup> to be used for SDG monitoring (table 6).<sup>45</sup> The 80 percent population coverage for the global baseline value of learning poverty is much higher than the population coverage for the global monetary poverty indicator when it was first launched in the 1990s, with data going back to 1981.<sup>46</sup> The main exceptions are the nine countries for which NLA data were used. Only three could be included in the simulations—India, Pakistan and Bangladesh. The NLAs not used are for Afghanistan, Cambodia, China, Ethiopia, the Kyrgyz Republic, Malaysia, Sri Lanka, Uganda, and Vietnam. And three PASEC countries in Africa for which microdata have not yet been made available—the Democratic Republic of Congo, Madagascar, and Mali—were not included in the simulations. The main reason for not including NLAs in these simulations is the unavailability of public use microdata files or even grouped data required to run the simulation models.

The simulation results were rescaled to match our baseline values, given the countries listed above.<sup>47</sup> On average, our simulation baseline result was 1.4 percentage points higher, with the reported baseline of

> 43 World Bank 2019.

> 44 Quality is assessed in this context in terms of design, implementation, comparability, frequency, timeliness, documentation, and data access.

> 45 The number of countries with adequate learning assessments for the learning poverty indicator is considerably lower than this population-weighted figure.

> 46 This level of data scarcity is not unprecedented.  In 1981, when global monetary poverty was first reported at a global scale, only 55 percent of the global population was covered by household surveys, with several regions not even reaching the 40 percent reporting benchmark. For most of the 2000s, thanks to the impetus generated by the Millennium Development Goals process, this number has increased, remaining close to 90 percent. For more information see <u>http://iresearch.worldbank.org/PovcalNet/home.aspx.</u>

> 47 World Bank 2019.

22
learning poverty (LP) at 53 percent and the simulation baseline at 54 percent. For two regions, Latin America and the Caribbean and Middle East and North Africa, the results were the same, since no countries were missing from the simulation. In Africa, despite the countries missing due to unavailable NLAs, the difference was only 2.5 percentage points, with the original LP at 87 percent and simulationbased LP at 84 percent. East Asia and the Pacific is the exception: with China missing from the simulations, the average difference was close to 12 percentage points.

**Table 6 Population and country coverage by country groups**

|||All countries<br>|||Low- and middle<br>Number of<br>|-income countries<br>||
|---|---|---|---|---|---|---|---|
|Country|group|Number of<br>countries with data|Number of<br>countries total|Population<br>coverage (%)|countries with<br>data|Number of<br>countries total|Population<br>coverage (%)|
||Overall|100|217|81.1|62|144|79.7|
||East Asia and Pacific|13|37|86.6|6|23|86.9|
||Europe and Central Asia|33|58|84.0|12|23|74.0|
||Latin American and the<br>Caribbean|16|42|86.8|16|30|88.4|
||Middle East and North<br>Africa|14|21|71.4|6|12|68.8|
||North America|2|3|100.0|N/A|N/A|N/A|
||South Asia|5|8|98.1|5|8|98.1|
|Region|Sub-Saharan Africa|17|48|46.1|17|48|46.1|
||High|44|79|97.7|6|10|99.7|
||Upper middle|27|60|91.8|27|58|92.2|
|level|Lower middle|16|47|75.8|16|46|76.0|
|Income|Low|13|31|63.3|13|30|64.8|
||Part 1|38|73|93.0|N/A|N/A|N/A|
|typeb|IBRD|39|68|91.4|39|68|91.4|
|Lending|IDA / Blend|23|76|56.0|23|76|56.0|

a Low- and middle-Income countries are Part 2 countries eligible to borrow from the World Bank Group and include high-income IBRD clients. b Lending types: Part 1 countries do not borrow from the World Bank Group; International Bank for Reconstruction and Development (IBRD); International Development Association (IDA); and IDA-eligible based on per capita income levels and are also creditworthy for some IBRD borrowing (Blend). Notes: Data include only assessments since 2011. Source: Author’s calculation using the Global Learning Assessment Database, UIS enrollment data, and UN population numbers.

The previous section established the main properties and axioms for the learning poverty headcount measure and discussed how some advantages and weaknesses of different choices of aggregations in terms of the monotonicity axiom and the monotonicity sensitivity axiom. In what follows, this paper 23
summarizes the choice of measures for the simulation results, as well as the methodology and scenarios used. The following criteria are often identified as desirable in the literature on choosing a measure:<sup>48</sup>

- **Simple** . The measure must be understandable and easy to describe.

- **Coherent** . It must conform to a common-sense notion of what is being measured.

- **Motivated** . It must fit the purpose for which it is being developed.

- **Rigorous** . It must be technically solid.

- **Implementable** . It must be operationally viable.

- **Replicable** . It must be easily replicable.

- **Incentive compatible** . It must respect country incentives, meaning that a country will not be penalized when it improves the learning of its student population.

The choice among these characteristics is not always easy. Trade-offs and compromises might be necessary based on the main purpose of the measure. A headcount measure has many of the characteristics, including simplicity, but often at the cost of coherence and incentive compatibility. A measure might be incoherent if it does not conform to a common-sense notion of what is being measured. For example, it might seem plausible to argue that children or educational systems with lower scores among the poor are worse off, other things equal. Incentive compatibility might fail when an index is already too high or when the index misses critical progress at foundational levels of learning. When coherence and incentive compatibility are critical, measures that respect both the monotonicity axiom and the monotonicity sensitivity axiom will be in order. Alternative measures met those criteria at the expense of a very important one, simplicity. This is no trivial choice, and in some cases might not be warranted if only the focus axiom is needed.

For this simulation, initial conditions are radically different, as some regions have much higher levels of learning poverty than others. The simulation needs both simplicity and the ability to generate the correct incentives so different stakeholders will act on the results. Hence, simulation results will be presented based on three measures—the headcount rate, which is suitable for countries and regions with lower levels of learning poverty, and the learning poverty gap and learning poverty severity, which are the most relevant for Africa and the low-income countries aggregate.

However, one important empirical question remains. Are all these complementary different measures empirically relevant? The extend in which will depend if we find that countries with same learning poverty level have different learning poverty gaps (figure 3, panel A), or do countries with the same learning poverty gap have different learning poverty severity (figure 3, panel B). Figure 3 illustrates those points using the latest available data from 99 countries in the learning poverty database for which learning poverty gap and learning poverty severity indicators are available.<sup>49</sup> The figure shows a wide range of

> 48 See for example Alkire et al. (2015) and Cameron et al. (2019).

> 49 All learning assessments used are anchored in a standard deviation of 100 points, this should be sufficient to have the FGTclass of measures to be minimally comparable. Of course, within-country temporal comparisons, assuming temporal comparability of the assessments, are the ideal case. All gap measures are relative to the test-specific MPL. One interesting aspect is that once the gap conversion is made, the measure becomes test independent, and can be presented independently of any scale. One important assumption however, which is shared with the global poverty monitoring, when doing cross-country comparisons, is that the learning (income) marginal sensitivity of the cardinal variable is the same. Meaning, improving one learning point is equally hard (or equally well captured) across all assessments (or different measures of income and consumption).

24
learning poverty gaps among the poor in countries with similar levels of learning poverty (panel A): several countries have around 70 percent learning poverty, but the Arab Republic of Egypt’s learning poverty gap among the poor is almost four times the size of Nicaragua’s. For countries with similar learning poverty gaps among the poor (35 percent), the range of learning poverty severity among the poor is even wider, with Honduras’s almost 10 times Iceland’s. This finding supports the empirical relevance of the distinction between measures and the importance of clarity on which axiomatic properties are needed when choosing one. For policy, the strategies to reduce learning poverty could differ considerably if the levels of learning poverty gap or learning poverty severity are drastically different. Countries with the same level of learning poverty but higher learning poverty severity will need far greater flexibility in learning (and schooling) strategies to adapt to the needs of children with a wider range of learning (and schooling) needs than countries with the same level of learning poverty but a higher learning poverty gap.

Most governments and development partners are working on identifying, protecting, and supporting the learning of the most vulnerable members of the COVID-19 generation. Countries’ initial conditions matter, and the size of the learning distribution does not say much about how vulnerable the youth of different countries are. A headcount measure, such as learning poverty, provides a focus on the base of the distribution, which is critical for prioritizing actions to support those who were suffering the most in the pre-COVID-19 learning crisis. But by itself, that measure does not say how much learning is being lost by the children already experiencing learning poverty—for that, the learning poverty gap is an important measure. Then, as school systems reopen, it will be critical to meet students where their needs are and to monitor changes in the learning distribution among the learning poor—for that, learning poverty severity is the appropriate measure. The use of these complementary measures is supported by both their axiomatic properties and their empirical relevance.

25
<!-- Start of picture text --> 55<br>50<br>-g 45c<br>5aao 40<br>2 @PHL<br>6<br>E35<br>$ OEGY<br>‘a 30<br>S&S<br>5 Dz<br>oi<br>2Sa 20<br>Sso<br>S 15<br>= @NIC<br>=c10<br>a<br>0<br>90<br>s BO<br>@ 70<br>s<br>hes:60<br>€ 50<br>& PHL<br>Fe£ 40<br>S<br>a EGY<br>2 30 DZA<br>:<br>bi<br>= 20<br>NIC<br>10<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>Learning Poverty (%)<br><!-- End of picture text --> <!-- Start of picture text --> 35<br>= 30<br>°og<br>2<br>2 25<br>S<br>&<br>s&<br>fe 35<br>£<br>&Go> 45 EGH<br>2Scg e BLZ<br>=8 10 e@ MAR<br>=<br>2<br>E<br>=§ 5 SO8KD<br>@HND<br>®@ CHL<br>0<br>45<br>a5 40<br>£ 35<br>oa<br>& 30<br>gee 2S<br>g<br>&ie 20 @HND<br>£ ©coG<br>= ae @BLzZ<br>iaBk<br>2 cone<br>—E 10 @QHIKD<br>§<br>=<br>2<br>0<br>0 10 20 30 40 50 60 70 80 90<br>Learning Poverty Gap (%) among the poor<br><!-- End of picture text -->
from 53 percent to 63 percent (figure 4). Sub-Saharan Africa and Europe and Central Asia have the smallest absolute increase of learning poverty, 5 percentage points, while South Asia has the largest (17 percentage points), followed by Latin America (12 percentage points). Africa also has the smallest relative increase, 5 percent, while East Asia and the Pacific and Europe and Central Asia have the largest (more than 30 percent), suggesting that the children in the upper-middle-income and lower-middle-income countries are likely to become the new learning poor. This result only reinforces the understanding that Sub-Saharan Africa was already experiencing a massive learning crisis before COVID-19 in which children were not learning as much when schools were still open. (Table A1 in the annex has the full set of results in terms of learning poverty.)

**Figure 4 Learning poverty simulation results**

<!-- Start of picture text --> 18.0<br>16.5<br>15.0<br>12.0<br>11.5<br>10.5<br>9.0<br>7.6<br>6.0 6.5 6.4<br>4.8 4.6<br>3.0 3.6 3.1<br>2.1 1.5 1.9 1.7<br>0.0<br>WLD EAP ECA LAC MNA SAR SSF<br>52.7 21.2 63.3 58.2 86.7<br>Baseline 13.3 50.8<br>Intermediate Optimistic Pessimistic<br>Change in Learning Poverty<br><!-- End of picture text --> Source: Author's calculation. All underlying numbers can be found in table A.1 in the annex.

As previously discussed, a complementary and relevant set of measures are the gap and severity. In order to avoid the confounding of policies required to improve schooling, in what follows we focus exclusively on the learning deprivation (Table A1 in the annex in the annex has the same results in terms of learning poverty). Sub-Saharan Africa and the Middle East and Northern Africa are the two regions in which students are on average the furthest behind in respect to the minimum proficiency level, with a Learning Deprivation Gap of approximately 20% (figure 5). By comparison, this rate is double the global average (10.5%), four times larger than the East Asia and Pacific Gap (5%), and more than tenfold larger the Europe and Central Asia average gap (1.3%). This average learning gap is equivalent to what students are expected to learn, in the respective regions, in a full academic school. In the pessimistic scenario, the learning gap for the average student in low- and middle-income countries could increase by 30 percent; and East Asia and the Pacific and Latin American being the region with the largest relative increase, close to 40%.

The gap measures are not distribution-sensitive and cannot distinguish between an increase in the learning gap driven by students near the threshold and one driven by those at the very bottom of the learning distribution. The African and Latin American and Caribbean increases in the learning gap might be qualitatively different if, in one set of countries, the pandemic were pushing many children marginally below the deprivation threshold and in the other, it were further deepening the deprivation of those already far below the threshold.

27
# **Figure 5 Learning deprivation gap simulations results**

<!-- Start of picture text --> 3.5<br>3.0<br>2.9<br>2.6 2.7<br>2.5<br>2.4<br>2.0 1.9<br>1.5<br>1.0<br>0.8 0.8 0.8<br>0.5 0.5 0.4 0.6<br>0.0 0.1<br>WLD EAP ECA LAC MNA SAR* SSF<br>10.4 4.4 21.4 9.4 19.2<br>Baseline 1.3 6.2<br>Intermediate Optimistic Pessimistic<br>Change Learning Gap<br><!-- End of picture text --> Source: Author's calculation. All underlying numbers can be found in table A.2 in the annex. The difference between learning poverty and learning deprivation is that learning poverty includes out-of-school children. (*) all countries report LP using NLAs and there for and aggregated gap measures was not computed.

Severity measures can distinguish among these qualitatively different types of impacts. Results for Latin America, the Middle East and Sub-Saharan Africa suggest that on average, sudents in those regions are experiencing a similar increase on the learning gap, of approximatly 2.5 percentage points. However, as discussed in sections 2 and 4, the learning gap fails to take into account the inequality of learning among the learning deprived. This means, that a hypotetical reduction of 20 learning points is fully equivalent, regardless if it happens for students just below the MPL or at the very botoom of the learning distribuiton. This can hide significant differnces on the complexity of the challenge. Figure 6 introduces the results through the lens of the learning severity mesaure introduced in section 2, an indicator which is sensitive to differences in the learning gap-inequality. Two distinct groups emerge from this analysis. Latin America with a learning severity increase of 0.5 percentage point, and the Middle East and Afirca, with an increase greater than one percentage point. The results support the idea that the consequence of this crisis is qualitatively different between Africa and the Middle East, and Latin America. Suggesting that students in the latter group might fall much further, relative to the MPL, than the ones in the former group.

In other words, in absolute terms, Sub-Saharan Africa and the Middle East and North Africa remain the two regions that face the greater challenge to reduce learning deprivation, because of both the magnitude of the problem as well as the heterogeneity of their respective learning deprived students. While, in Latin America, the new learning poor seem to have fallen much closer to the MPL. Moreover, the depth of learning deprivation in Sub-Saharan Africa could increase three times more than the number of new learning deprived children in the region. This is almost three times the global average, and four times more than in Europe and Central Asia. All these results suggest an increase in the complexity and the cost to tackle the learning crisis on the continent.

28
**Figure 6 Learning deprivation severity simulations results**

<!-- Start of picture text -->
1.6<br>1.4<br>1.4<br>1.2 1.2<br>1.0 1.0<br>0.8<br>0.6 0.6<br>0.5<br>0.4 0.4 0.4<br>0.3<br>0.2<br>0.1 0.1 0.1<br>0.0 0.0<br>WLD EAP ECA LAC MNA SAR* SSF<br>3.7 1 11 3.4 6.4<br>Baseline 0.3 1.2<br>Intermediate Optimistic Pessimistic<br>Change Learning Gap Inequality<br><!-- End of picture text -->

Source: Author’s calculation. All underlying numbers can be found in table A.2 in the annex. (*) all countries report LP using NLAs and there for and aggregated gap measures was not computed.

Tackling learning poverty will require strategies to identify and respond to the learning needs of individual students. The greater the learning gap and the inequality among the learning poor, the bigger and more heterogenous the challenge will be. COVID-19 will lead to greater inequality among the learning poor, tackling it will require strategies to Teach to the Right Level and to effectively design and deploy structured lesson programs. Figures 7 and 8 summarize these arguments by presenting the relationship between the _leve_ l of the learning deprivation gap and the learning deprivation severity (or inequality sensitive learning gap) and the _change_ in the respective measures across the different regions. Both Africa and the Middle East seem to have the biggest and more complex challenges to begin with (Figure 7), and those are also the regions in which both the learning deprivation gap and learning deprivation severity could increase the most (Figure 6).

This finding suggests that COVID-19 could qualitatively change the learning crisis on the African continent, since students will come out of this pandemic in a much deeper learning crisis than before, falling further behind the minimum proficiency levels established under SDG 4. This greater depth of the learning crisis in Africa and elsewhere will require qualitatively different policy responses of far greater complexity and cost.

29
**Figure 7 Pre-COVID, the Middle East and SubSaharan Africa had the hardest challenge, as both the gap and gap-inequality of learning are the highest**

**Figure 8 Post-COVID, the learning gap might increase approximately the same in several regions, however, the gap-inequality could increase the most in the Middle East and SubSaharan Africa**

<!-- Start of picture text --> Bigger and more Bigger and more<br>14.5 heterogenous 1.6 heterogenous<br>problem problem<br>12.5 MNA 1.4 MNA<br>1.2 SSF<br>10.5<br>1.0<br>8.5 WLD<br>SSF<br>0.8<br>6.5<br>0.6 LAC<br>4.5 WLD EAP<br>0.4<br>2.5<br>EAP LAC 0.2<br>ECA<br>0.5 ECA<br>0.0<br>Smaller and more -0.5 4.5 9.5 14.5 19.5 24.5 29.5 Smaller and more homogenous 0.0 1.0 2.0 3.0 4.0<br>homogenous -1.5problem Learning gap (p.p.) problem Increase on the learning gap (p.p.)<br>Inequality sensitive learning gap (p.p.) Increase in the inequality sensitive learning gap (p.p.)<br><!-- End of picture text --> <!-- Start of picture text --> problem<br><!-- End of picture text --> Note: variances on types of Arab script poses specific challenges to the teaching and the assessment of learning in the Middle East region. Source: Author’s calculation. All underlying numbers can be found in table A.2 in the annex.

# Final remarks

This paper has expanded the concept of learning poverty, drawing from the poverty measurement literature to present axiomatic properties of poverty measures and building on the FGT family of poverty measures to provide an extension of distribution-sensitive measures of learning poverty. The paper has presented conceptual, numerical, empirical, and practical reasons why this extension can be useful for measuring the consequences of COVID-19 school closures and comparing those consequences over time and across countries.

Three simulation scenarios explored potential learning losses, expressed in terms of the World Bank/UIS learning poverty indicator. In the most pessimistic scenario, learning poverty in low- and middle-income countries was expected to increase 20 percent, or 10 percentage points, going from 53 to 63 percent. Most of that increase might occur in lower-middle-income and upper-middle-income countries, especially in East Asia and the Pacific, Latin America and the Caribbean, and South Asia. Africa, and the group of lowincome countries, both with the highest levels of learning poverty before the COVID-19 pandemic, could have the smallest absolute and relative increases in learning poverty due to COVID-19-related school closures. The small size of those increases simply reflects the magnitude of the learning crisis in those countries before the pandemic.

Building on the FGT class of poverty measures to complement learning poverty measures with measures of a learning poverty gap and learning poverty severity, the discussion showed that Africa is the only region where the learning poverty depth might increase more than the learning poverty breadth.

30
The results projected by these simulations do not have to happen. Parents, teachers, students, governments, and development partners must work together to deploy effective mitigation and remediation strategies to protect the COVID-19 generation’s future. School reopening, when safe, is critical, but it is not enough. The simulation results show major differences in the distribution of learning. The big challenge will be to rapidly identify and respond to each individual student’s learning needs in a flexible and adaptive way and to build back educational systems more resilient to shocks, using technology effectively to enable learning both at school and at home.

31