inference we draw from these findings is not that global indices have no place; rather, it suggests the need, from the outset, for curating a broad suite of data when anticipating and responding to complex policy challenges, the better to promote real-time learning. Doubling down on what can be readily measured limits the space for eliciting those ‘unobserved’ (and perhaps unobservable) factors that may turn out to be deeply consequential.

3. An emphasis on the easy to quantify inhibits understanding of the foundational problem <mark>An indiscriminate emphasis on aggregated, quantitative data can erode important nuances about the root causes of the problems we want to fix, thereby hampering our ability to craft appropriate solutions and undermining the longer-term problem-solving capabilities of an organization. All too often the designation of indicators and targets has the effect of causing people to become highly simplistic about the problems they are trying to address. In such circumstances, what should be organizational meetings held to promote learning and reflection on what is working and what is not become instead efforts in accounting and compliance. Reporting, rather than learning, is incentivized and management increasingly focuses on meeting the target numbers rather than solving the problem. Our concern here is that, over time, this tendency progressively erodes an organization’s problem-solving capabilities.</mark> <mark>The education sector is perhaps the best illustration of this; time and again practitioners have sought to codify “learning” and time and again this has resulted in an obfuscation of the actual causes underlying the problem.</mark> In a well-intentioned effort to raise academic performance, “the standards movement” in education promoted efforts hinged on quantitative measurement, as reported in the league tables of the Program for International Student Assessment (PISA).<sup>23</sup> PISA runs tests in mathematics, reading, and science every three years with groups of fifteenyear-olds in countries around the world. Testing on such scale requires a level of simplicity and “standardization”, thus the emphasis is on written examinations and extensive use of multiplechoice tests so that students’ answers can be easily codified and processed (Robinson and Aronica 2015). Demonstrating competence on fundamental learning tasks certainly has its place, but critics have increasingly argued that such tests are based on an incorrect assumption that what drives successful career and life outcomes is the kind of learning that is capable of being codified via a standardized test (Khan 2021, Claxton and Lucas 2015).

<mark>In reality, the gap between the skills that children learn and are tested for, and the skills that they need to excel in the 21st century, is becoming more obvious. The World Economic Forum noted in 2016 that the traditional learning captured by standardized tests falls short of equipping students with the knowledge they need to thrive.</mark><sup>24</sup> <mark>Yong Zhao, the presidential chair and director of the Institute for Global and Online Education in the College of Education at the University of</mark>

> 23 These tables are based on student performance in standardized tests in mathematics, reading, and science, which are administered by the Paris-based Organisation for Economic Co-operation and Development (OECD).

> 24 <u><mark>https://www.weforum.org/agenda/2016/03/21st-century-skills-future-jobs-students/</mark></u>

> <u><mark>https://www.weforum.org/agenda/2016/01/the-10-skills-you-need-to-thrive-in-the-fourth-industrial-revolution</mark></u> <mark>“Whereas negotiation and flexibility are high on the list of skills for 2015, in 2020 they will begin to drop from the top 10 as machines, using masses of data, begin to make our decisions for us. A survey done by the World Economic Forum’s Global Agenda Council on the Future of Software and Society shows people expect artificial intelligence machines to be part of a company’s board of directors by 2026. Similarly, active listening, considered a core skill today, will disappear completely from the top 10. Emotional intelligence, which doesn’t feature in the top 10 today, will become one of the top skills needed by all.”</mark>

13