# **I. Introduction**

“What gets measured gets managed; and what gets measured gets done” is one of those ubiquitous (even <mark>cliché</mark> d) management phrases that hardly requires explanation; it seems immediately obvious that the data generated by regular measurement and monitoring is what makes possible the improvement of results. Less well known than the phrase itself is the fact that although it is commonly attributed to the acclaimed management theorist Peter Drucker, Drucker himself never actually said it.<sup>2</sup> In fact, Drucker’s views on the subject were reportedly far more nuanced, along the lines of V. F. Ridgway, who argued over 65 years ago that not everything that matters can be measured and not everything that can be measured matters (Ridgway 1956). Simon Caulkin, a contemporary business management columnist, neatly summarized Ridgway’s argument, in the process expanding the truncated to-measure-is-to-manage phrase to “What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.”<sup>3</sup> Ridgway and Caulkin’s warnings – repeated in various guises by many since<sup>4</sup> – remind us that an indiscriminate usage of quantitative measures and an undue confidence in what they can tell us may turn out to be highly problematic in certain situations, sometimes derailing the very performance improvements that the data was intended to support (Merry et al 2015). We hasten to add, of course, that seeking more and better quantitative data is clearly a worthy aim in public administration (and elsewhere). Many important gains in human welfare (e.g., recognizing and responding to learning disabilities) can be directly attributed to interventions conceived and prioritized on the basis of the empirical documentation of the reality, scale, and consequences of the underlying problem. The wonders of modern insurance are possible because actuaries can quantify all manner of risks over time, space and groups. What we will argue in the following sections, however, is that access to quantitative data alone is not a sufficient condition for achieving many of the objectives that are central to public administration and economic development.

This paper has five sections. Following this Introduction (Section I), we lay out in Section II the ways in which the collection, curation, analysis, and interpretation of data is embedded in contexts: no aspect takes place on a blank slate. On one hand, the institutional embeddedness of the data collection and usage cycle – in rich and poor countries alike – leaves it susceptible to a

> 2 - According to the Drucker Institute; see https://www.drucker.institute/thedx/measurement <u>myopia/</u> (accessed 6 December 2021).

> 3 See https://www.theguardian.com/business/2008/feb/10/businesscomment1 (accessed 6 December 2021). 4 See, for example, former USAID Administrator, Andrew Natsios (2011), citing Lord Wellington in 1812, on the insidious manner in which measures of ‘accountability’ can compromise rather than enable central policy objectives (in Wellington’s case, winning a war). For his part, Stiglitz has argued that “What you measure affects what you do. If you don’t measure the right thing, you don’t do the right thing.” (As quoted in the _New York Times_ , October 4, 2009.) Pritchett (2014), exemplifying this point, notes (at least at the time of his writing) that the Indian state of Tamil Nadu had 817 indicators for measuring the delivery of public education, but none that actually assessed whether students were learning – in this instance, an abundance of “measurement” and “data” was entirely disconnected from (what should have been) the policy’s central objective. In many cases, however, it is not always obvious, especially ex ante, what constitutes the “right thing” to measure – hence the need for alternative methodological entry points to elicit what these might be.

2