metadata: authors: 0: author: Azevedo,Joao Pedro Wagner De
1: author: Akmal,Maryam
2: author: Cloutier,Marie-Helene
3: author: Rogers,F. Halsey
4: author: Wong,Yi Ning
count: World
docna: 0: docna: Learning Losses during COVID-19 : Global
            Estimates of an Invisible and Unequal Crisis
docty: Policy Research Working Paper
theme: Education,Mitigation,Human Development and Gender,Data Development and Capacity Building,Access to Education,Environment and Natural Resource Management,Disease Control,Pandemic Response,Public Sector Management,Climate change,Adaptation,Data production, accessibility and use,Education Governance, School-Based Management
abstracts: cdata!: This paper presents updated simulation
            results of the potential effects of COVID-19-related school
            closures on learning outcomes globally. The simulation,
            which updates and extends prior work by Azevedo, Hasan et
            al. (2021) and Azevedo (2020), examines potential learning
            losses as the pandemic moves into the third year. Beyond
            reflecting the longer duration of the crisis, the paper
            extends prior work by using country-specific observed school
            closure information, accounts for the partial reopening of
            some education systems, updates the baseline Learning
            Poverty estimates to reflect its best estimate to date just
            before the pandemic (circa 2019), and uses updated June 2021
            macroeconomic projections to reflect the economic magnitude
            of the crisis. The analysis finds that the overall learning
            levels are likely to fall substantially around the world.
            Under an &#x201c;intermediate&#x201d; scenario,
            school closures could potentially increase the share of
            children in Learning Poverty in low- and middle-income
            countries by 13 percentage points, to 70 percent. Globally,
            learning adjusted years of schooling could fall by 1.1
            years, and the share of youth below minimum proficiency on
            the Programme for International Student Assessment could
            rise by 12.3 percentage points. Furthermore, school
            shutdowns could generate lifetime earning losses of $21
            trillion. These results imply that decisive action is needed
            to recover and accelerate learning.
display_title: Learning Losses during COVID-19 :
            Global Estimates of an Invisible and Unequal Crisis
listing_relative_url: /research/2022/10/33926918/learning-losses-during-covid-19-global-estimates-invisible-unequal-crisis
topicv3: Education Indicators and Statistics
            m1326116 1522,Education Access and Equity m1327050 2127,Learning Assessment m1327283 1555,Labor Earnings m1504041 2738,Coronavirus (COVID-19) m1939478 4828
publishtoextweb_dt: 2022-10-27T00:00:00Z
dois: 10.1596/1813-9450-10218
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 49
author_keys: 0: author_key: 675142
1: author_key: 1985082
2: author_key: 1060969
3: author_key: 123307
4: author_key: 2000639
model_extractions: Policy Research Working Paper 10218 Learning Losses during COVID-19 Global Estimates of an Invisible and Unequal Crisis _João Pedro Azevedo Maryam Akmal Marie-Helene Cloutier Halsey Rogers Yi Ning Wong_ Education Global Practice October 2022
Policy Research Working Paper 10218

# **Abstract**

This paper presents updated simulation results of the potential effects of COVID-19-related school closures on learning outcomes globally. The simulation, which updates and extends prior work by <u>Azevedo, Hasan et al. (2021)</u> and Azevedo (2020), examines potential learning losses as the pandemic moves into the third year. Beyond reflecting the longer duration of the crisis, the paper extends prior work by using country-specific observed school closure information, accounts for the partial reopening of some education systems, updates the baseline Learning Poverty estimates to reflect its best estimate to date just before the pandemic (circa 2019), and uses updated June 2021 macroeconomic projections to reflect the economic magnitude of the crisis.

The analysis finds that the overall learning levels are likely to fall substantially around the world. Under an “intermediate” scenario, school closures could potentially increase the share of children in Learning Poverty in low- and middle-income countries by 13 percentage points, to 70 percent. Globally, learning adjusted years of schooling could fall by 1.1 years, and the share of youth below minimum proficiency on the Programme for International Student Assessment could rise by 12.3 percentage points. Furthermore, school shutdowns could generate lifetime earning losses of $21 trillion. These results imply that decisive action is needed to recover and accelerate learning.

.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at jazevedo@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Learning Losses during COVID-19: Global Estimates of an Invisible and Unequal Crisis<sup>*</sup>

João Pedro Azevedo, Maryam Akmal, Marie-Helene Cloutier, Halsey Rogers, and Yi Ning Wong<sup>†</sup> JEL Classification Codes: O12, O15, I21, I24, I25 Keywords: Foundational learning; basic education; student assessment; SDGs; Learning Poverty; Learning Adjusted Years of Schooling

* The authors would like to thank Ellinore Ahlgren, Elaine Ding, and Ahmed Raza (World Bank); Silvia Montoya (UIS); Borhene Chakroun, Gwang-Chol Chang, and Sonia Guerriero (UNESCO Headquarters); Pragya Dewan, Suguru Mizunoya, Nicholas Reuge, Kenneth Russell, and Haogen Yao (UNICEF Headquarters); Rona Bronwin (FCDO); Joannie Cohen-Mitchell (USAID); and Clio Dintilhac and Izzy Boggild-Jones (BMGF) for multiple conversations and discussions that helped strengthen the narrative around the simulation results.  The work was carried out under the overall guidance of Jaime Saavedra and Omar Arias. The team is extremely grateful to Luis Crouch (RTI International) and to Gabriel Demombynes, Deon Filmer, Laura Gregory, Alaka Holla, and Yevgeniya Savchenko (World Bank) for their comments and suggestions. We apologize for any omissions and express our sincerest thanks to everyone, whether named here or not, who graciously gave their time and expertise. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations or those of the Executive Directors of the World Bank or the governments they represent.

> † All authors are from the World Bank Education Global Practice. Corresponding author: <u>jazevedo@worldbank.org</u>
# **Introduction**

COVID-19-related school closures are pushing countries off track from achieving their learning goals. When children are not able to attend school, they lose an opportunity to acquire new knowledge and they may forget what they learned in the past. Unremediated learning losses may compound over time if children continue to fall further behind the curriculum (Kaffenberger 2020; <u>World Bank, UNESCO, and UNICEF 2021). Simultaneously, the economic shock of the crisis and its impact on income, employment, and</u> government budgets can worsen learning outcomes and reduce the future earning capacity of millions of students.

In many countries around the world, there simply is no nationally representative pre-pandemic baseline estimate of learning to use in assessing COVID-19 learning loss. Many other countries that do measure learning do not do it regularly or in a comparable way, which makes capturing COVID-19 learning loss estimates particularly challenging. Only a handful of countries have empirical estimates of learning losses during COVID-19. Given the lack of actual data, simulations are a useful tool for estimating potential learning losses and their long-term impacts, so that countries can prepare to respond to the adverse consequences of the pandemic.

This work updates and extends the simulation work initially presented in Azevedo, Hasan et al. (2021) and <u>Azevedo (2020). Some of the main updates to the simulation tool include the use of observed school closure</u> information at the country level, inclusion of information on partial closures, and the update of the prepandemic Learning Poverty baseline for certain countries. This work complements the latest high-level results presented in the <u>State of Global Learning Poverty 2022 Update report, and provides detailed information on</u> the updated simulations. We focus on four key outcomes of the simulation model to contextualize learning loss impacts: (i) Learning Poverty, (ii) Learning Adjusted Years of Schooling, (iii) Percent below minimum proficiency in the Programme for International Student Assessment (PISA), and (iv) Lifetime earnings.

Since the publication of the learning loss simulation results in June 2020 in <u>Azevedo, Hasan et al. (2021), the</u> paper has been downloaded more than 17,000<sup>3</sup> times and cited more than 550 times, emphasizing the interest and need to understand learning losses as the pandemic unfolds.<sup>4</sup> To date, the simulation results have helped ministries of education, key national stakeholders, and development partners advocate for and plan evidencebased recovery strategies for mitigating potential learning losses arising from COVID-19 school closures, as well as for tackling the chronic Learning Poverty problem that existed before the crisis.

Given the substantial learning data gaps<sup>5</sup> that accompany the long-lasting learning crisis, these simulation results are often the only source of evidence that can guide policymakers, educators, and researchers towards addressing and analyzing potential learning loss impacts on a global scale. Although they are well aligned with evidence on observed learning losses that has begun to come out, it is important to keep in mind that these results are simulations. They do not replace the value of actual data on learning losses or mitigation effectiveness, which is limited at present, particularly for low-income countries.

> 3 <u>https://openknowledge.worldbank.org/handle/10986/33945</u> 4 Based on Google Scholar statistics here: <u>https://scholar.google.com/citations?view_op=view_citation&hl=en&user=lTKXA78AAAAJ&citation_for_view=lTKXA78AAAA J:8p8iYwVyaVcC</u>

> 5 In July 2021, UNESCO, UNICEF, and the World Bank have joined forces to close the learning gaps that still exist and that preclude many countries to monitor the quality of their education systems and assess if their students are learning. The three organizations have agreed to a Learning Data Compact, a commitment to ensure that all countries, especially low-income countries, have at least one quality measure of learning by 2025, supporting coordinated efforts to strengthen national assessment systems.

2
The paper is structured as follows. The next section provides a brief review of relevant literature, while the following section describes what’s new in the analytical framework and the empirical methodology used in this global update. The two subsequent sections present the results and discuss the main findings, respectively. The final section concludes. The appendices provide methodological details and a detailed description of the main indicators.

# **What Do We Know about School Closures, Mitigation Effectiveness, and Learning Losses?**

This section reviews what we have learned so far about the extent of school closures during COVID-19, the likely effectiveness of remote learning, and the learning losses that resulted. This information all feeds into the data and assumptions used in the simulation model.

## Schools Closures Varied across Contexts

During peak school closures in April 2020, 1.6 billion children, or 94% of the world’s students, were out of school. Over a billion of these children live in low- and middle-income countries. According to the latest results from the UNESCO school closure tracker, documented in Figure A1 in the annex, the length of school closures has varied greatly across regions, with full closures in low- and middle-income countries lasting longer than those in high-income countries. Globally, between February 2020 and February 2022, schools were fully closed for 141 days on average, while partial school closures also lasted for 141 days (Table A1). Partial closures may involve schools offering face-to-face instruction only in certain areas, or for certain grades, or via a hybrid model where only some students receive in-person instruction at a time.

## Remote Learning Provision Varied across Contexts

After COVID-19 prompted school closures in early 2020, many countries pivoted towards providing remote instruction. A 2021 World Bank report documents countries’ experiences with remote learning during the pandemic, and concludes that many countries used multi-modal strategies by combining the use of, for example, internet, TV, radio, as well as printed materials, but there were differences across income groups, with high-income countries more likely to rely on internet-based solutions. Despite provision of remote learning solutions, many countries, especially low-income countries, struggled to ensure take-up of remote learning initiatives. Many countries even found themselves in a “remote learning paradox,” in which they adopted remote learning solutions that were not suited to the needs of most of the student population due to lack of devices or connectivity issues.

## Learning Losses Are Large and Inequitable

The State of the Global Education Crisis report by World Bank, UNESCO, and UNICEF (2021) reviews the literature on learning losses and concludes that learning losses have been large and are unequal, both between and within low-, middle-, and high-income countries. Based on a review of the existing evidence in low- and middle-income countries, the report suggests that the learning losses are roughly proportional to the length of the school closures, but that there is a great deal of heterogeneity across and within countries. Recent analysis by Moscoviz and Evans (2022) suggests that learning losses are concentrated primarily among children from poorer households. Even in high-income countries that had existing technological infrastructure for deployment of remote learning, learning losses were substantial. There is empirical evidence on learning losses from a diverse set of countries, ranging from Belgium and the United Kingdom to India and Brazil. For example, in the Indian state of Karnataka, the share of Grade 3 students in public schools who could perform simple subtraction fell by 8 percentage points between 2018 and 2020. Similarly, students in São Paulo in Brazil learnt only 28% of what they would have in face-to-face classes. In Ethiopia, <mark>children in grade</mark> 3
<mark>4 only learned 31%-40% as much in math as they would have during a normal year (Kim et al. 2021;</mark> <u><mark>W</mark> orld Bank, UNESCO, and UNICEF 2021).</u>

# **Background: What’s New?**

In 2020, the World Bank released COVID-19 learning loss simulations modeling impacts on Learning Poverty (Azevedo 2020), Learning Adjusted Years of Schooling (LAYS),<sup>6</sup> and percent below minimum proficiency in PISA, as well as impacts on future earnings (Azevedo, Hasan et al. 2021). Since then, four major developments have led to revisions of the learning loss estimates and simulations:

- The pandemic has continued. In many countries, school closures have affected two or more school years, and the surge of COVID-19 cases brought about by the Omicron variants indicate we are not out of the woods yet (WHO 2021).

- There is actual data on school closures. At the time of the previous simulations, there was no global data on school closures, and so those simulations relied on uniform assumptions about school closure lengths for all countries under different scenarios. We now have observed country-level data on actual school closures for February 2020 – February 2022 from the <u>UNESCO school closures tracker, which we use for to build the baseline scenario and the different simulation scenarios.</u><sup>7</sup> Therefore, baseline school closure data now differ across countries and regions.

- More up-to-date learning assessment data are available. Internationally comparable data from PASEC 2019, TIMSS 2019, LLECE 2019,<sup>8</sup> and a new regional learning assessment program that took place in East Asia, SEA-PLM 2019, have been released, as have the results from AMPL-b 2021 assessment (in Zambia),<sup>9</sup> and policy linking results of national assessment in Lesotho. This new data has led to a country-level update of the pre-COVID-19 baseline Learning Poverty estimates.

- The Learning Poverty baseline has been updated. For regional and global Learning Poverty estimates, we use the new reporting window of ±4 assessment years around 2019 (the previous window was ±4 years around 2015). Some countries outside of the reporting window were included for temporal comparability with the previous global estimate. The exceptions include Afghanistan, Kyrgyzstan, Lesotho, Pakistan, Tunisia, Uganda, and the Republic of Yemen. We use population estimates for 2019 to calculated populated-weighted averages for global and regional estimates.

This paper presents updated results for the impact of school closures and mitigation effectiveness on the Learning Poverty headcount ratio, LAYS, and percent below minimum proficiency in PISA under three scenarios: optimistic, intermediate, and pessimistic. Since the release of <u>Azevedo, Hasan et al. 2021, we have</u> lowered our expectations regarding mitigation effectiveness based on experiences with remote learning over the past two years of school closures. The previous “intermediate” scenario parameters are now used for the “optimistic” scenario, the previous “pessimistic” scenario replaces the existing “intermediate” scenario, and the previous “very pessimistic” scenario parameters are used for the existing “pessimistic” scenario. In all

> 6 Learning Poverty is defined as the inability to read and understand a simple text by age 10. More information about the Learning Poverty measure can be found here. The World Bank’s Learning Adjusted Years of Schooling (LAYS) concept combines quantity (access) and quality (learning outcomes) of schooling into a single easy-to-understand metric of progress. More information about the LAYS measure can be found here. While both LAYS and Learning Poverty combine schooling and learning, LAYS encompasses all levels of education through secondary school, capturing the educational life of students from 4 to 18 years of age, and represents the learning levels achieved by a schooling system of an entire country (excluding tertiary). Learning Poverty focuses on primary-aged children by combining learning deprivation (share of children at the end of primary below minimum proficiency) and schooling deprivation (share of primary-aged children who are out-of-school) into one multi-dimensional indicator.

> 7 Note that while the previous simulations used school closures as a percentage of one full school year, schools have been closed for almost two full school years to date; we adjust expected learning gains as a product of annual learning gains and observed pandemic school years.

> 8 Learning Poverty reports LLECE results on the SERCE scale.

> 9 Note that the AMPL-b 2021 assessment was conducted during the pandemic.

4
scenarios, schools are closed for the time period based on actual data for February 2020 – February 2022 from the <u>UNESCO school closures tracker,</u><sup>10</sup> assuming no additional future school closures. Differences between the optimistic, intermediate, and pessimistic scenarios are defined as follows:

- Optimistic: Mitigation measures have a medium level of effectiveness.<sup>11</sup> Partial closures are assumed to affect 75% of the student population.

- Intermediate: Mitigation measures have a low level of effectiveness.<sup>12</sup> Partial closures are assumed to affect 85% of the student population.

- Pessimistic: Mitigation measures have a low level of effectiveness. Partial closures are treated as full closures (that is, to affect 100% of the student population).

UNESCO describes partial school closures as follows: “Partially open: Schools are (a) open/closed in certain areas only, and/or (b) open/closed for some grade levels/age groups only; and/or (c) open but with reduced in-person class time, combined with distance learning (hybrid approach). It also includes the countries where national governments have deferred decisions on re-opening to other administrative units (e.g. region, municipality or individual schools), and where a variety of re-opening modalities are being used” (UNESCO <u>2021). Since the information on partial closures from the UNESCO calendar is qualitative in nature, we treat</u> partial closures differently across the simulation scenarios.

Table 1 below outlines the key differences and similarities between the current and previous simulations. It is important to notice that the income<sup>13</sup> and World Bank lending classifications<sup>14</sup> used in this paper are as of July 1, 2021, and differ from the previous papers.

> 10 In the absence of actual data on school closures, the simulations in Azevedo (2020) and Azevedo, Hasan et al. (2021) assumed that schools are closed only for 3-9 months in a 10-month school year. A few regional simulations (for the East Asia and the Pacific and the Middle East and North Africa regions only) relied on World Bank internal data from Task Team Leaders on actual closures in those regions to inform this parameter. 11 Under the optimistic scenario, mitigation effectiveness is assumed to be 10% for low-income, 14% for lower-middle-income, 20% for upper-middle-income, and 30% for high-income countries.

> 12 Under the intermediate and pessimistic scenarios, mitigation effectiveness is assumed to be 5% for low-income, 7% for lowermiddle-income, 10% for upper-middle-income, and 15% for high-income countries.

> 13 See income classifications here.

> 14 See World Bank lending classifications here.

5
**Table 1. Key differences between the current and previous simulations**

||**Current Global Simulations**|**Previous Global Simulations**|
|---|---|---|
|Expected learning gain or<br>school productivity (in HLO<br>points / year) in a regular<br>schoolyear|Varies by countries’ income level. Sa|me as inAzevedo, Hasan et al. 2021.|
|Mitigation effectiveness|Varies by scenario, and by country’s income level.<br>Optimistic scenario: 10% for low-income, 14% for<br>lower-middle-income, 20% for upper-middle-income,<br>and 30% for high-income countries.<br>Intermediate and pessimistic scenarios: 5% for low-<br>income, 7% for lower-middle-income, 10% for upper-<br>middle-income, and 15% for high-income countries.|Varies by scenario, and by country’s income level. See<br>Azevedo, Hasan et al. 2021.<br>Optimistic scenario: 20% for low-income, 28% for<br>lower-middle-income, 40% for upper-middle-income,<br>and 60% for high-income countries.<br>Intermediate scenario: 10% for low-income, 14% for<br>lower-middle-income, 20% for upper-middle-income,<br>and 30% for high-income countries.<br>Pessimistic and very pessimistic scenarios: 5% for low-<br>income, 7% for lower-middle-income, 10% for upper-<br>middle-income, and 15% for high-income countries.|
|Baseline Values|Global Learning Poverty Database from June 2022<br>(different from what was used inAzevedo 2020).<br>Includes recent results for PASEC 2019, TIMSS 2019,<br>LLECE 2019, and from new assessments, such as SEA-<br>PLM 2019 and AMPL-b 2021 (Zambia), as well as NLA<br>from Lesotho that has undergone policy linking. This<br>applies only to Learning Poverty simulations.<br>Global HCI database from 2021 (used inAzevedo,<br>Hasan et al. 2021).|Global Learning Poverty Database from October 2019<br>(used inAzevedo 2020).<br>Global HCI database from 2021 (used inAzevedo,<br>Hasan et al. 2021).|
|Learning Poverty reporting<br>window|The new Learning Poverty window for aggregate<br>estimates is ±4 assessment years around 2019. There are<br>some exceptions: Afghanistan, Kyrgyzstan, Lesotho,<br>Pakistan, Tunisia, Uganda, and Yemen. Population<br>estimates for 2019 are used for aggregate calculations.|The previous Learning Poverty window for aggregate<br>estimates was ±4 assessment years around 2015.<br>Population estimates for 2015 were used for aggregate<br>calculations.|
|Data on actual school closures|UNESCO school closures tracker for February 2020 –<br>February2022.|Not available at the time. The length of school closures<br>was scenario-based.|
|Length of period in school<br>years|2, minus country-specific breaks.|1.|
|Share of the school system<br>closed|Schools can be considered partially closed.|Education system was assumed to be fully open or fully<br>closed.|
|Country-specific school<br>closure information|Yes. UNESCO school closure tracker has country-<br>specific information.<sup>15</sup>|No. All countries were assumed to have the same length<br>of school closures, depending on the simulation<br>scenario.|
|School closure assumptions:<br>Optimistic|Observed country-level school closures, partial closures<br>are assumed to affect 75% of the student population.|Schools assumed to be closed uniformly for 3 months<br>in a 10-month school year.|
|School closure assumptions:<br>Intermediate|Observed country-level school closures, partial closures<br>are assumed to affect 85% of the student population.|Schools assumed to be closed uniformly for 5 months<br>in a 10-month school year.|
|School closure assumptions:<br>Pessimistic|Observed country-level school closures, partial closures<br>are treated as full closures.|Schools assumed to be closed uniformly for 7 months<br>in a 10-month schoolyear.|
|School closure assumptions:<br>Very Pessimistic|N/A|Schools assumed to be closed uniformly for 9 months<br>in a 10-month school year.|

> 15 We impute missing values for share of school system closed by using the regional average by income level for countries with learning data (LAYS, LP, or PISA). The countries are: Belarus; Burundi; the Democratic Republic of Congo; the Republic of Congo; the Arab Republic of Egypt; Hong Kong SAR, China; the Islamic Republic of Iran; the Republic of Korea; Kosovo; Macao SAR, China; Micronesia, Fed Sts.; Nauru; St Kitts and Nevis; St Lucia; St Vincent and the Grenadines; Tajikistan; and the Repbulci of Yemen.

6
|Economic projections|Economic forecasts are from Global Economic<br>Prospects (GEP) June 2021 (for 2020) and January 2022<br>(for 2021).|Economic forecasts were from Macro Poverty Outlook<br>(MPO) October 2020.|
|---|---|---|

A key update in the latest simulations results is that with the school closure data from UNESCO, we no longer assume that closures are uniform across regions or countries, an assumption made in the previous simulation version due to limited data availability on school closures. As shown in Table 2, there are wide differences across regions in the share of the school system that was closed. Taking the intermediate scenario, in which partial closures are assumed to affect 85% of the student population, 70% of the school system was closed in South Asia, compared to 57% in Middle East and North Africa and 34% in Europe and Central Asia.

**Table 2. Share of the school system that is closed by scenario**

||Optimistic|Intermediate|Pessimistic|
|---|---|---|---|
|**Global**|42.9%|45.4%|49.2%|
|**Global(Part 2)**|45.1%|47.6%|51.4%|
|**By Region**||||
|East Asia and Pacific|33.8%|35.9%|39.1%|
|Europe and Central Asia|31.7%|33.7%|36.8%|
|Latin America and Caribbean|67.0%|71.0%|77.1%|
|Middle East and North Africa|53.6%|56.7%|61.2%|
|North America|56.5%|62.6%|71.9%|
|South Asia|66.0%|69.6%|75.0%|
|Sub-Saharan Africa|33.1%|34.7%|37.0%|
|**By Region (Part 2)**||||
|East Asia and Pacific|36.8%|38.9%|42.0%|
|Europe and Central Asia|36.0%|38.1%|41.3%|
|Latin America and Caribbean|72.0%|76.4%|82.9%|
|Middle East and North Africa|44.7%|46.8%|50.0%|
|North America||||
|South Asia|66.0%|69.6%|75.0%|
|Sub-Saharan Africa|33.1%|34.7%|37.0%|
|**By income level**||||
|High-Income|38.1%|40.8%|44.8%|
|Upper-middle-income|52.7%|55.9%|60.7%|
|Lower-middle-income|42.5%|44.7%|48.0%|
|Low-income|34.4%|36.0%|38.4%|
|**By Lending type **||||
|Part 1|37.7%|40.3%|44.2%|
|IBRD|54.7%|57.9%|62.8%|
|IDA/Blend|36.0%|37.7%|40.4%|
|Source: Authors’ calculations using the UNESC<br>learning data (LP, LAYS, or PISA) but no schoo<br>regional average byincome level. The estimates a|O School Closures database coverin<br>l closure data, we impute missing v<br>re countryaverages.|g February 2020-February 2<br>alues for share of school sys|022. For countries with<br>tem closed by using the|

7
The analytical framework underlying the simulations presented in this paper is largely the same as in <u>Azevedo, Hasan et al. (2021), with one important extension – the new model allows for systems to be partially closed,</u> something that the previous model did not account for. As in the conceptual framework underpinning the previous simulations, we can think about the expected learning loss through two key channels: (i) the learning that will not take place while schools are closed, and (ii) the already acquired learning that will be lost or forgotten when students become less engaged with schooling. Additionally, the empirical methodology underlying the simulations is the same as documented in <u>Azevedo, Hasan et al. (2021). Details about the</u> framework and empirical methodology can also be found in Annex B.

Table 3 below describes the key input parameters on school closures, mitigation effectiveness, and school productivity used in the model to simulate the learning and earning outcomes under different scenarios.

# **Table 3. Parameters for simulations by income level**<sup>**16**</sup>

||**Global**|**High-**<br>**Income**|**Upper-**<br>**Middle-**<br>**Income**|**Lower-**<br>**Middle-**<br>**Income**|**Low-Income**|
|---|---|---|---|---|---|
|**A. Learning gains or school productivity (in**<br>**HLOpoints/year)**<sup>**17**</sup>|39|50|40|30|20|
|_Optimistic Scenario_||||||
|B1. Share of the system affected over observed<br>period (24 months)|42.9%|38.1%|52.7%|42.5%|34.4%|
|C1. Mitigation effectiveness (0 to 100%)|21.1%|30.0%|20.0%|14.0%|10.0%|
|D2. HLO decrease (points) =<br>_B1*(A*((Total school weeks/43.3)*(1-C1))_|24.6|24.8|32.1|22.1|12.4|
|_Intermediate Scenario_||||||
|B2. Share of the system affected over observed<br>period (24 months)|45.4%|40.8%|55.9%|44.7%|36.0%|
|C2. Mitigation effectiveness (0 to 100%)|10.5%|15.0%|10.0%|7.0%|5.0%|
|D2. HLO decrease (points) =<br> _B2*(A*((Total school weeks/43.3)*(1-C2))_|29.8|32.3|38.5|25.2|13.6|
|_Pessimistic Scenario_||||||
|B3. Share of the system affected over observed<br>period (24 months)|49.2%|44.8%|60.7%|48.0%|38.3%|
|C3. Mitigation effectiveness (0 to 100%)|10.5%|15.0%|10.0%|7.0%|5.0%|
|D3. HLO decrease (points) =<br>_B3*(A*((Total school weeks/43.3)*(1-C3))_|32.3|35.4|41.7|27.0|14.5|
|**GEP* (GDP per capita growth %) [g]**|3.3|4|5.1|1.5|1.1|

_Note: Values represent country-level averages._

- - Notes: (*) Global Economic Prospects January 11th 2022 update (https://www.worldbank.org/en/publication/macro <u>poverty outlook), with the regional average imputed if no country value was available for 2020 or 2021.</u>

16 The table provides an overview of parameters by income, though simulation parameters are applied at the country level.

> 17 The World Bank’s Harmonized Learning Outcome (HLO) puts learning data from international and regional assessments on a comparable scale. The data can be accessed here. We assume the learning gains will vary from 20 to 50 learning points depending on the country’s income level, as explained in Azevedo, Hasan et al. (2021).

8
# **Results**

We focus on four key outcome indicators for the simulations, with the Outcomes 1-3 being used to illustrate learning losses experienced by children, and Outcome 4 being used to demonstrate the economic loss due to school closures. The three learning-focused indicators measure learning in different ways, as described below, but help illustrate potential learning losses experienced by children. The primary outcome indicators are as follows:

- **Outcome 1: Learning Poverty.** The first simulation uses the UNESCO and World Bank’s Learning Poverty measure. Learning Poverty is defined as the inability to read and understand a simple ageappropriate text by age 10. This indicator combines the share of primary-aged children who are not in school (schooling deprived) and share of children at end of primary below the minimum proficiency level in reading (learning deprived). By combining schooling and learning, the indicator highlights the importance of both more access to schooling and the better learning that is critical to ensuring that schooling leads to acquisition of skills and capabilities.<sup>18</sup>

- **Outcome 2: Learning Adjusted Years of Schooling.** The second simulation focuses on LearningAdjusted Years of Schooling (LAYS). The World Bank’s Learning-Adjusted Years of Schooling (LAYS) concept combines quantity (access) and quality (learning outcomes) of schooling into a single easy-to-understand metric of progress (Filmer, Rogers, Angrist, and Sabarwal 2020). This is one of the components of the World Bank’s Human Capital Index, launched in 2018 and updated in 2020. It encompasses all levels of education before tertiary, since LAYS captures the educational life of students from 4 to 18 years.

- **Outcome 3: Percent below Minimum Proficiency in PISA.** The third simulation uses OECD’s (Organization for Economic Co-operation and Development) PISA, which measures learning outcomes of 15-year-olds in reading, math, science, and other skills such as collective problem-solving. Unlike the Learning Poverty analysis, which focuses on outcomes at the primary level, the analysis of education proficiency based on PISA focuses on student achievement of 15-year-olds, who are generally enrolled at the lower secondary level. Focusing on reading proficiency scores, we simulate how the share of children performing below minimum proficiency (PISA Level 2 or 407.47 points) could potentially change due to school closures and mitigation effectiveness of remote learning.

- **Outcome 4: Lifetime Economic Loss.** The fourth simulation presents the results from LAYS in monetary terms. As done in <u>Azevedo, Hasan et al. (2021), we use expected earnings information</u> from <u><mark>ILO (2020)</mark></u> and <u><mark>World Bank (2020)</mark></u> , together with the expected long-run return to education. We also compute aggregate results by bringing all expected earnings losses to their present value, assuming a work life of 45 years and a 3% discount rate. To make these results more realistic, we also adjust the aggregate loss by the expected adult survival rate (following the World Bank HCI) and the fact that not all workers will always be in gainful employment.

See Annex C for the percentage of population covered for the key outcome variables.

## Outcome 1. Learning Poverty

In the previous release of Learning Poverty results, baseline Learning Poverty was 52.7% for low- and middle-income<sup>19</sup> countries in 2015. In the current release, baseline global Learning Poverty for these

> 18 To learn more about the Learning Poverty indicator update see Azevedo, Montoya et al. (2021). For Learning Poverty, expected learning gains are adjusted according to the standard deviation for the assessment for each country. See annex E1 for all assessments and standard deviations used.

> 19 The estimation for low- and middle-income countries refers to the estimation for World Bank Part 2 countries, which also include the following high-income countries for Learning Poverty calculations: Chile, Croatia, Panama, Poland, and Trinidad and Tobago. Other high-income Part 2 countries include Antigua and Barbuda, Palau, the Seychelles, St. Kitts and Nevis, and Uruguay.

9
countries is expected to be 57% in 2019. This 4.3 percentage point jump in baseline (pre-COVID-19) Learning Poverty is due to the use of updated learning data and reporting windows described in Table 1. In a post-COVID-19 intermediate scenario of low mitigation effectiveness for the effects of school closures, the simulations show that Learning Poverty may increase from its baseline level of 57% to 70% for low- and middle-income (Part 2) countries – an increase of 13 percentage points, as shown in Figure 1, implying an increase of 23% relative to the Learning Poverty baseline value.

**Figure 1. Results of simulation: Learning Poverty**

<!-- Start of picture text --> % Change<br>Baseline 57.0% -<br>Optimistic 68.2% 20%<br>23%<br>Intermediate 70.0%<br>Pessimistic 71.2% 25%<br>0.0% 20.0% 40.0% 60.0% 80.0%<br>Share of Students in Learning Poverty<br><!-- End of picture text --> _Note: Includes low- and middle-income (Part 2) countries only. The estimates are population-weighted averages._ Most of this increase is expected to occur in South Asia and Latin America and the Caribbean, where countries may experience an increase in Learning Poverty of 18.2 and 26.8 percentage points respectively under the intermediate scenario, as shown in Table D1 in the annex. Regions that had the highest levels of Learning Poverty before COVID-19, such as Sub-Saharan Africa, are likely to have smaller absolute and relative increases in Learning Poverty because the learning crisis was already so severe in those countries before the pandemic. This result aligns with the findings in Samiego et al. (2022), who document greater losses in human capital accumulation in developed countries than in developing countries. But children with the weakest foundational literacy before school closures are likely to fall further behind, especially if their families are not literate. Therefore, though the absolute or relative increase in Learning Poverty may not be large in the regions with the highest rates before the pandemic, the many children already below minimum proficiency in those regions may fall further behind. This phenomenon is examined further in simulation results using the Learning Poverty Gap and Severity simulations,<sup>20</sup> described below.

The Learning Poverty estimates treat all students below the minimum proficiency threshold as being equally learning deprived, even though countries with similar Learning Poverty rates have different learning levels among those who are learning poor. Changes in the Learning Poverty rate also do not capture changes in

> 20 The **Learning Poverty Gap** measure captures the depth of learning deprivation and can distinguish students that have different levels of learning deprivation. However, this measure assumes that the effort required to improve an extremely learning-deprived and a moderately learning-deprived student are equivalent. The **Learning Poverty Severity** measure extends the concept of the Learning Poverty Gap by incorporating the student’s initial level of learning and gives greater importance (weight) to students that are in greater learning deprivation. See Azevedo (2020) for a detailed discussion on the concepts of Learning Poverty Gap and Learning Poverty Severity.

10
learning that occur below the minimum proficiency threshold—for example, changes in acquisition of foundational subskills such as mapping sounds to letters, recognizing words, etc., which are critical for developing the foundational reading skills needed to meet the minimum proficiency threshold.

To measure these aspects, the **Learning Poverty Gap** brings together the concepts of Learning Deprivation Gap, which indicates the average effort needed to bring children in school above minimum proficiency, and Schooling Deprivation Gap, which highlights the need to improve access to schooling among those who are out of school. This gap measure provides useful policy-relevant detail that is not captured by the learning poverty measure. For example, as shown in Figure 2 below, there are several countries (such as Morocco and Myanmar) that, under an intermediate scenario, are expected to have similar increases in Learning Poverty but very different expected increases in the Learning Poverty Gap. This finding implies that different levels of effort, resources, and policy focus on children at the bottom or out of school may be required to tackle Learning Poverty.

**Figure 2. Relative Changes in Learning Poverty and Learning Poverty Gap**

<!-- Start of picture text --> 400%<br>350%<br>300%<br>250%<br>HIC<br>200%<br>UMC<br>150% LMC<br>LIC<br>100%<br>MMR<br>50%<br>MAR<br>0%<br>0% 20% 40% 60% 80% 100% 120% 140% 160% 180% 200%<br>Change in Learning Poverty (%)<br>Change in Learning Poverty Gap (%)<br><!-- End of picture text --> _Note: Figure shows relative percent change from the baseline under the intermediate scenario._ Based on the results of the simulation, under the intermediate scenario, the Learning Poverty Gap will rise by 2.1 percentage points from 9.5% to 11.6% for low- and middle-income countries. The increase in Learning Poverty Gap is particularly high in in Europe and Central Asia and East Asia and the Pacific (Table D2). This means that moving forward, these countries will need to focus on children at the bottom, including those out of school (who are assumed not be learning) to ensure that they are not left behind.

While the Learning Poverty Gap measure shows how far students are behind the minimum proficiency level on average, it is not distribution-sensitive, and cannot distinguish between changes in the learning gap driven 11
by students near the minimum proficiency threshold and those driven by students at the very bottom of the learning distribution. Students who are further away from the minimum proficiency threshold likely have different learning needs than those closer to the threshold.

**Learning Poverty Severity** indicates inequality of learning among those below the threshold. It brings together the concepts of Learning Deprivation Severity and Schooling Deprivation Severity. Learning Deprivation Severity measures learning inequality among learning deprived children in school. Compared to the gap measure, the severity measure is more sensitive to changes in learning levels of learning deprived children who are further away from the minimum proficiency threshold and changes in children who are out of school.

For example, as shown in Figure 3, Vietnam and Myanmar are expected to have similar increases in the Learning Poverty Gap under an intermediate scenario, but the two countries have very different expected changes in Learning Poverty Severity. This implies that both countries may need different levels of policy focus on identifying the diverse learning needs among children below minimum proficiency and on providing flexible and tailored learning opportunities. As school systems reopen, it will be critical to meet students at their point of need and to monitor changes in the learning distribution among the learning poor; for that, Learning Poverty Severity is the appropriate measure.

**Figure 3. Relative Changes in Learning Poverty Gap and Learning Poverty Severity**

<!-- Start of picture text --> 400%<br>350%<br>300%<br>250%<br>HIC<br>200%<br>UMC<br>150% LMC<br>LIC<br>100%<br>VNM<br>MMR<br>50%<br>0%<br>0% 100% 200% 300% 400% 500% 600% 700%<br>Change in Learning Poverty Severity (%)<br>Change in Learning Poverty Gap (%)<br><!-- End of picture text --> _Note: Figure shows relative percent change from the baseline under the intermediate scenario._ Based on the simulation results, Learning Poverty Severity will rise by 1.7 percentage points under the intermediate scenario for low- and middle-income countries (Table D3). The increase in Learning Poverty Severity is highest in Europe and Central Asia, followed by East Asia and the Pacific. Tackling Learning Poverty Severity will require a focus on differentiated and targeted learning interventions for children at the bottom based on the learning level of the child.

12
Policies to reduce Learning Poverty could differ across countries and regions depending on the levels of Learning Poverty Gap and Severity. Depending on the country’s Gap and Severity estimates, effectively mitigating learning losses may require different levels of resources and effort targeted at children at the bottom (as captured by Learning Poverty Gap) or a differentiated focus on addressing learning inequality among children at the bottom through tailored learning opportunities (as captured by Learning Poverty Severity).

# Outcome 2. Learning Adjusted Years of Schooling

The overall average baseline for LAYS pre-COVID-19 was 7.8 (for the average value for low- and middleincome countries, LAYS was 6.8). In other words, before COVID-19 children in the average country were completing only 7.8 years of high-quality education. Our simulations results suggest that the average Learning Adjusted Years of School (LAYS) may fall substantially due to COVID-19 school closures. In an intermediate scenario, school closures due to COVID-19 could reduce the country average learning that students achieve during their lifetime from 7.8 to 6.7 Learning Adjusted Years—a drop of 1.1 years, as shown in Figure 4.

## **Figure 4. Results of simulation: Effect on Learning Adjusted Years of Schooling (LAYS)**

<!-- Start of picture text --> % Change<br>Baseline 7.8 - -<br>Optimistic 6.9 -11%<br>Intermediate 6.7<br>-14%<br>Pessimistic 6.7 -15%<br>6.5 7.0 7.5 8.0<br>Learning-Adjusted Years of Schooling (LAYS)<br><!-- End of picture text --> _Note: The estimates are country averages._ Across the globe, the extent of this loss is likely to vary, as shown in Table D4 in the annex. In the average Latin America and the Caribbean country, children were expected to complete 7.8 years of LAYS prior to the pandemic. Simulations suggest that COVID-19 could lower LAYS by 1.8 years in the intermediate scenario. In a typical South Asian country under the same scenario, LAYS could fall by 1.6 years from the baseline of 6.5. At the other end of the spectrum, children in Sub-Saharan African were expected to complete 5 years of LAYS prior to COVID-19, and the simulations suggest that COVID-19 could reduce LAYS by 0.6 years in the intermediate scenario. This variation can be explained by differences in extent of school closures. In Latin America and the Caribbean, schools were fully closed for 225 days and partially closed for 236 days from February 2020 to February 2022. South Asia experienced 273 days of full closures and 256 days of partial closures. In contrast, Sub-Saharan Africa experienced shorter school closures, with schools fully closed for 129 days and partially closed for 94 days on average.

Embedded in the simulation on LAYS is how dropouts will affect the expected years of schooling (EYS), a component of LAYS. Increased dropout rates are one important channel linking emergency school closures 13
and other educational disruptions to losses in average lifetime educational attainment. In general, as child age, the opportunity cost of staying in school increases. This may make it harder for households to justify sending older children back to school after a forced interruption, especially if households are under financial stress. To simulate dropouts, we take the expected shock on income from Global Economic Prospects (GDP per capita growth projections) and estimate the expected effect of this income shock on dropouts using agegroup specific dropout-income elasticities.<sup>21</sup> In 2021, the world economy experienced a strong recovery from the first phase of the pandemic (Global Economic Prospects 2021 and 2022). The GDP per capita is expected to have grown by 4.4 percentage points in 2021, which signals a strong recovery compared to the October 2020 estimate of -0.8 (Macro Poverty Outlook, percentage points). However, growth has been uneven and concentrated in higher-income countries. Thus, while the estimated increase in dropout is lower than in 2021, that improvement is mostly driven by growth in advanced economies.

Income shocks from COVID-19 could increase the out-of-school population by 0.26% for primary school children and 0.23% for secondary school children, or an additional 4.0 million children to drop out from school around the world. Of these children, 3.7 million live in low-income countries, and 1.8 million are between 12 to 17 years of age. Figure 5 shows how dropout relates to changes in income shocks across the different economic projections, during a period in which growth rates have improved under the most recent projections but still underperform pre-pandemic growth.

**Figure 5. Estimates of student dropout by 2020 and 2021 growth projections release**

<!-- Start of picture text --> 12 5<br>10.8 4.4<br>10 4<br>3<br>8<br>6.9 2<br>6 1.5<br>1<br>4 4.1 -0.2 4.0 0<br>2 1.9 -0.2 -0.8 -1<br>0 -2<br>MPO-Mar WEO-Apr MPO-June MPO-Oct GEP<br>2020 2021<br>Student Dropout Estimates by Growth Projection<br>GDPpc Growth Projections for 2020 and 2021 (p.p.)<br>Number of dropouts<br>(in millions of students)<br>GDPpc Growth Projections (p.p.)*<br><!-- End of picture text --> _Note: The macro number used in this paper reflect what was available at the time of the Global Learning Poverty 2022 update in June/2022._

# Outcome 3. Percent below Minimum Proficiency on PISA

Globally, the percentage of 15-year-olds below minimum proficiency on PISA could rise by 12.3 percentage points in the intermediate scenario (see Figure 6 below). These results imply a rise in the share of students who are not able to “identify the main idea in a text of moderate length, find information based on explicit though sometimes complex criteria, and reflect on the purpose and form of texts when explicitly directed to do so” (PISA’s definition of a minimum level of proficiency).

> 21 Figure B2.

14
**Figure 6. Results of simulation: Effect on share Below Minimum Proficiency (BMP) in PISA**

<!-- Start of picture text --> % Change<br>Baseline 40% -<br>25%<br>Optimistic 50%<br>Intermediate 52% 31%<br>Pessimistic 53%<br>34%<br>0% 10% 20% 30% 40% 50% 60%<br>Share of students below minimum proficiency (BMP)<br><!-- End of picture text --> _Note: The estimates are country averages._ The impact is highest in Latin America and the Caribbean countries, where the share of students below minimum proficiency, which was 53.3% at baseline, is projected to increase by 25.1 percentage points to 78.3% under the intermediate scenario, as shown in Table D5 in the annex. In Sub-Saharan African countries, where the share below minimum proficiency in PISA was 77.6% at baseline, is likely to experience a smaller increase of 5.5 percentage points under the intermediate scenario. This is partly because, as observed for Learning Poverty, these regions were already experiencing a more acute learning crisis prior do the pandemic, and most of the losses may be happening for children already below the minimum proficiency threshold. In addition, the share of school system closed was smaller in Sub-Saharan Africa (35% under the intermediate scenario) than in Latin America and the Caribbean and South Asia, where the share of school system closed was around 70% under the intermediate scenario.

# Cross-Country Distribution of Learning Losses as Measured by Learning Poverty, LAYS, and PISA

Figures 7, 8 and 9 show the distribution of learning losses, as measured by Learning Poverty, LAYS, and percent below minimum proficiency in PISA. Two key takeaways emerge. First, the extent of learning losses is heterogenous across countries. In terms of Learning Poverty, the bottom quartile of countries (that is, those with the smallest changes in Learning Poverty) experience an increase of 2.1 percentage points under an intermediate scenario, while countries among the highest quartile of change experience an increase in Learning Poverty of 15.8 percentage points. In terms of LAYS, countries in the bottom quartile experience losses of around 0.5 LAYS under an intermediate scenario, compared with 1.5 LAYS for countries in the top quartile. For the share below PISA minimum proficiency, the estimated increases are 5.0 and 19.6 percentage points, respectively, for the bottom and top quartile of countries.

Second, learning losses are large. Under an intermediate scenario, countries at the 50<sup>th</sup> percentile face learning losses equivalent to a 3.9-percentage-point increase in Learning Poverty, a drop of 1.0 LAYS, and an 11.0percentage-point increase in those below minimum proficiency in PISA.

15
<!-- Start of picture text --> oO<br>+<br>fo)<br>ao<br>g<br>é<br>aN<br>“2 === SSS SS ll port rrrrr rrr rrr ir mate P---<br>ene — ee Men96<br>oO<br>0 20 40 60 80 100<br>Share of countries<br>———— _ Optimistic _———— Intermediate ———— Pessimistic<br><!-- End of picture text -->
<!-- Start of picture text --> Ww)<br>N<br>N<br>Fave)<br>s—<br>2<br>Ss ~~~ eS __- > ~eee<br>s a SE DULL LLL mea ionMeant2 6<br>SS ee _ _ _ _Mean0,90._<br>”<br>oO<br>0 20 40 60 80 100<br>Share of countries<br>————_ Optimistic _———_ Intermediate ———— Pessimistic<br><!-- End of picture text --> <!-- Start of picture text --> ws<br>:<br>—<br>E<br>3<br>8<br>ra<br>< mrrrrrrrrr rk a Sesser rrr rr rrr rome tee r---<br>ee ———— Mea 10<br>oO<br>0 20 40 60 80 100<br>Share of countries<br>————_ Optimistic _———— Intermediate ~~——— Pessimistic<br><!-- End of picture text -->
# **Outcome 4. Present Value Loss of Lifetime Earnings**

Loss of learning (as measured through Learning Adjusted Years of Schooling) is expected to reduce lifetime earnings, and the reduction can be quantified using existing evidence on return to schooling, life expectancy, whether people are able to utilize their human capital through paid employment, and labor market earnings. The previous simulations (pessimistic scenario) projected that approximately $16 trillion of aggregate lifetime earnings (at present value in 2017 PPP) could be lost for this cohort of learners because of their lower levels of learning, their lost months during school closures, and their increased probability of dropping out of school (Azevedo, Hasan et al. 2021). Under the current projections, it is estimated that the lifetime earnings losses add up to approximately $21 trillion under an intermediate scenario (Figure 10). Expressed another way, on average a student could lose $29,162 (2017 PPP $) in lifetime earnings.

As shown in Table D6 in the annex, the economic loss varies by region. Under the pessimistic scenario, the earning losses are highest in East Asia and Pacific, at $5.8 trillion, and lowest in Sub-Saharan Africa, at $0.5 trillion. The lower loss in earnings for Sub-Saharan Africa is driven partly by the already low earning levels (and Learning Adjusted Years of Schooling) in Sub-Saharan Africa and partly by the shorter school closures. However, this finding does not diminish the severity of the setback countries in Sub-Saharan Africa potentially face. With already lower levels of earnings and learning outcomes, these countries cannot afford further any worsening of the prospects for future generations.

## **Figure 10. Aggregate Economic Loss to Lifetime Earnings ($)**

<!-- Start of picture text --> % Change<br>Optimistic 16.6 T<br>8%<br>Intermediate 20.6 T 10%<br>11%<br>Pessimistic 22.8 T<br>-10.0 T 10.0 T 30.0 T<br>Loss income per year per student (2017 PPP $)<br><!-- End of picture text --> _Note: Global aggregate in US dollars at 2017 PPP._ Figure 11 shows the distribution of earning losses. The losses vary substantially, with countries in the bottom quartile expected to experience future earning losses of around $2.9 billion under the intermediate scenario, compared with $64.0 billion for countries in the top quartile. Furthermore, earning losses are substantial overall: under the intermediate scenario, countries at the 50<sup>th</sup> percentile face earning losses equal to $17 billion. The curves also show that with each worsening scenario, more countries experience greater earning losses.

18
<!-- Start of picture text --> Ke)<br>°<br>g<br>=<br>ge —~-\\—~—~->s eeert<br>E i)<br>; ~~ \Se ee<br>2N Mec 0.12 6<br>5 ee<br>&<br>Ww<br>N<br>0 20 40 60 80 100<br>Share of countries<br>——— Optimistic _———— Intermediate ——— Pessimistic<br><!-- End of picture text -->
# **Discussion**

How Do the Simulation Results Compare with Empirical Studies on Learning Losses? <mark>Empirical studies have presented learning losses using two main types of measures of learning: (i) a continuous measure, showing change in standard deviations of test scores or in share of expected acquired learning in a period, and (ii) a discrete measure showing change in share of students achieving certain minimum proficiency levels. Our simulation results are expressed both in terms of a continuous measure (LAYS), and as a share of children below a specific proficiency threshold (Learning Poverty and PISA).</mark> Comparing the simulation results against those from empirical studies highlights two key takeaways. One, as shown by the simulations, a growing number of empirical studies suggest that students have been experiencing learning loss across a range of contexts, from low-income to high-income countries. <mark>In rural Karnataka in India, the share of grade 3 students in government schools able to perform a simple subtraction fell by 8 percentage points (Pratham 2021;</mark> <u>World Bank, UNESCO, and UNICEF 202</u> <u><mark>1). In a few Brazilian</mark></u> <mark>states, the share of second graders who were off-track to become fluent in reading increased by 22 percentage points (World Bank, UNESCO, and UNICEF 2021;</mark> <u><mark>Fundação Lemann, o Instituto Natura e a Associação Bem Comum. 2021). Also in Brazil, the share of second graders at the national level failing to read and write</mark></u> <mark>simple words increased by 18 percentage points between 2019 and 2021 (INEP, 2022). In Uganda, the percent of 6</mark><sup>th</sup> <mark>grade learners rated proficient in English decreased by 4.7 percentage points between 2018 and 2021 (N</mark> <u>APE 202</u> <u><mark>1). In the Russian Federation, 6</mark></u><sup>th</sup> <mark>grade children experienced losses equal to 3-4 months of learning (Zvyagintsev 2021). At the secondary level, close to one-third of children in grade 8 experienced learning losses in Kenya (Whizz Education 2021), measured by knowledge in math topics. In Krasnoyarsk, Russian Federation, 8</mark><sup>th</sup> <mark>grade students experienced an equivalent of 1.5 years of lost learning in more than half a period of full school closures (Stanislavovich 2021). The Monitoring Impacts on Learning Outcomes (MILO) report, looked at COVID-19 end-of-primary learning losses in six countries in Sub-Saharan Africa</mark><sup>22</sup> <mark>(UNESCO 2022). In contrast to the results from most other countries, that study showed that in all six countries, students maintained pre-COVID-19 learning outcomes after the onset of the pandemic, at least until mid-2021. This may indicate that in contexts with a very low initial proportion of students meeting minimum proficiency thresholds, a decline in learning may be difficult to observe, among other factors (see</mark> <u><mark>UNESCO 2022</mark></u> for a full discussion) <mark>. The MILO results are also in alignment with our simulation results which indicate the lowest learning losses in Sub-Saharan Africa for the reasons previously discussed.</mark> Two, the learning loss estimates from some of these empirical studies are roughly in the ballpark of the simulation estimates. <mark>Based on existing empirical studies of learning losses during school closures, the W</mark> <u>orld Bank, UNESCO, and UNICEF (2021)</u> <mark>report suggests that one year of school closures results in learning loss roughly equivalent to a normal year’s worth of learning, though there is heterogeneity across countries. We compare simulated loss in LAYS to actual school closures under different scenarios of the simulations, as shown in Table D12 in the annex. Under an intermediate scenario, roughly one LAYS is lost for one year of school closures, as documented in the analysis using the empirical studies on learning loss. There is heterogeneity across regions: the ratio of loss in LAYS to years of school closures is around 1 in Sub-Saharan Africa, South Asia, Latin America and the Caribbean, and Middle East and North Africa, where LAYS is already low to begin with, and higher than 1 in the remaining regions (see Table D12 in the annex).</mark> <mark>The simulation results highlight the possibility of worsening inequalities in children’s learning and increases in both the Learning Poverty Gap and Severity estimates. Simulation estimates for the Learning Poverty Gap suggest an increase of 2.0 percentage points (intermediate) for low- and middle-income (World Bank Part 2) countries, or an increase of 21% relative to the baseline. Learning Poverty Severity could increase by 1.7 percentage points (intermediate), or 14% relative to the baseline. L</mark> earning loss studies have also presented

> 22 <mark>Burkina Faso, Burundi, Côte d'Ivoire, Kenya, Senegal, and Zambia.</mark>

20
learning inequalities between groups. Recent analysis by <u>Moscoviz and Evans (2022)</u> suggests that learning losses are concentrated primarily among children from poorer households. <mark>A US study found larger declines in math and reading among historically marginalized and economically disadvantaged peers across grade levels 3 to 8 (Lewis et al. 2021). For example, grade 8 students in high-poverty schools showed both lower percentile rank, and a larger decrease in their growth percentile rank during COVID-19 than did peers in lowpoverty schools. In the United Kingdom, the gap in learning loss between disadvantaged secondary pupils and their more affluent peers was around 1.6 months (Education Policy Institute and Renaissance Learning</mark> <u><mark>2021).</mark></u> The <u>MILO report</u> raises important <mark>questions about the proportion of students who were already below the minimum proficiency level pre-pandemic, whose learning losses are hard to observe using measures focusing on children above the minimum proficiency threshold (UNESCO 2022).</mark>

# Planning for School Reopening

Reopening schools and keeping them opened is crucial. Since October 2021, more countries have been fully or partially open, with only a few countries remaining fully closed. School closures have massive effects in terms of not only learning outcomes but also future earnings outcomes of today’s students. For example, the difference in global lifetime earning losses between the “intermediate” and “pessimistic” scenarios is $2.2 trillion. Both scenarios assume the same level of mitigation effectiveness, so the only difference is in the share of the school system assumed to be closed, which demonstrates the high economic cost of school closures. Therefore, reopening schools and keeping them open should be the top priority for countries, as growing evidence indicates that with adequate measures, health risks to children and education staff can be minimized (World Bank 2021).

Beyond their effects on measured learning, school closures have exacerbated the socio-emotional losses experienced by children. Children spend a substantial amount of time interacting with their classmates and teachers when schools are open, and the loss of this interaction has costs. School closures have also affected services essential for children’s development, including school feeding, safety, and psychosocial support. Reopening schools and supporting them to provide comprehensive services promoting the wellbeing and mental health of children should be a priority for countries.

# How Can Education Systems Recover Learning?

Given the large simulated and empirical learning losses across a wide range of contexts, as captured through changes in Learning Poverty, LAYS, and PISA, countries’ education strategies should now focus on learning recovery and acceleration. Education systems need to be able to adapt rapidly to mitigate the long-term effects of school closures and learning losses. Reopening schools will need to be accompanied by learning recovery strategies to help children catch up—for example, by consolidating the curriculum so that teachers prioritize the essential material that students missed during school closures, or by increasing instructional time by extending the school time or modifying the academic calendar. The RAPID framework by UNESCO, <u>UNICEF, and the World Bank (2022)</u> emphasizes five key actions for learning recovery:

- Reach and retain every child

- Assess learning levels

- Prioritize teaching foundational skills

- Increase catch-up learning and increase instructional time

- Develop psychosocial health and well-being

Reducing inequalities should be an important part of this approach. As discussed in <u>World Bank, UNESCO, and UNICEF (2021), COVID-19 has exacerbated inequalities across a variety of dimensions. School closures</u> have lasted longer in low- and middle-income countries compared to high-income countries. Within 21
countries, children from disadvantaged backgrounds were less likely to benefit from remote learning. Recent analysis by <u>Moscoviz and Evans (2022) suggests that learning losses are concentrated primarily among</u> children from poorer households, as documented in the Netherlands, Italy, United States, Mexico, Bangladesh, and Ghana.

Our results show that the Learning Poverty Gap is likely to rise. Countries experiencing an increasing Learning Poverty Gap will need to direct more resources and efforts toward children at the bottom, for example through facilitating structured pedagogy so that teachers can be more effective in closing learning gaps. The simulation results also suggest that Learning Poverty Severity is also likely to increase. Countries experiencing worsening Learning Poverty Severity will need to focus on reducing learning inequality among children at the bottom through tailored and targeted instruction, for example by adopting “teaching at the <u>right level” techniques.</u>

## Address Learning Data Gaps

Finally, to tackle the learning crisis, countries must address crucial learning data gaps by assessing student learning levels. While good learning data has been used to document learning losses in certain countries, grades, and households, many countries lack such data and are therefore flying blind. Policy makers and teachers require granular, timely, and comparable data to implement policies and actions that can help improve learning outcomes. Measuring learning losses through frequent, reliable data is an essential first step in mitigating losses. Such data is vital for the effective implementation of evidence-based strategies, such as targeted instruction, that allow teaching to be aligned with student needs.

## Prioritize Education Financing

Ensuring a safe return to school and facilitating learning recovery will require not only political commitment, but also financial support from governments, given that the vast majority of education comes from domestic government budgets (Hares and Rossiter 2021). The pandemic caused the economies of many countries to reverse a steady upward trend in public spending in education. Forty percent of low- and lower-middleincome countries reduced their education spending in 2020, after the onset of the pandemic, with an average decline of 13.5 percent. Education spending rebounded slightly in 2021 but fell again in 2022, to below 2019 levels (Education Finance Watch 2022). Similarly, the simulation results suggest that without robust action, one of the long-term impacts of school closures will be large economic losses in terms of lifetime earnings. To prevent these losses, education systems need to receive adequate support and financing, so that they can ensure that children are able to recover lost learning.

# **Conclusion**

Simulation results suggest the children’s learning levels are likely to fall substantially because of extended school closures. The share of children in Learning Poverty in low- and middle-income countries could increase by 13 percentage points, Learning Adjusted Years of Schooling could fall by 1.1 years, and the share of youth below minimum proficiency in PISA could rise by 12.3 percentage points. School shutdowns could generate lifetime earning losses of $21 trillion.

Drastic measures are needed to recover and accelerate learning. Recovering lost learning will require reimagining education systems in several important ways. One, as children return to in-person schooling, it is important that education systems use evidence-based strategies to facilitate learning recovery of children. It is particularly important to accelerate the acquisition of foundational skills, such as literacy and numeracy, that children may not have acquired in early grades prior to the pandemic, as later learning outcomes depend on children acquiring these basic skills. To that end, it is critical to address the learning data crisis, and measure the learning levels of children and learning losses experienced. Two, it is important to prioritize financing for education to ensure the availability of resources required for children’s learning recovery and acceleration. It 22
is not possible to do all the above without strong political commitment and domestic coalitions to tackle the learning crisis.

While the pandemic continues to challenge policies and hamper children’s learning, countries have an opportunity to accelerate learning and make schools more efficient, equitable, and resilient by building on investments made and lessons learned during the crisis.

23