metadata: authors: 0: author: Leaver,Clare
1: author: Lemos,Renata Freitas
2: author: Dillenburg Scur,Daniela
count: Brazil,World
docna: 0: docna: Measuring and Explaining Management in
            Schools : New Approaches Using Public Data
docty: Policy Research Working Paper
subtopic: Economics of Education,Education Finance,Educational Sciences,Effective Schools and Teachers,Educational Institutions & Facilities,Public Sector Administrative and Civil
            Service Reform,De Facto Governments,Democratic Government,Administrative & Civil Service Reform,Public Sector Administrative & Civil
            Service Reform
teratopic: Education,Governance,Social Protections and Labor
abstracts: cdata!: Why do some students learn more in some
            schools than others? One consideration receiving growing
            attention is school management. To study this, researchers
            need to be able to measure school management accurately and
            cheaply at scale, and also explain any observed relationship
            between school management and student learning. This paper
            introduces a new approach to measurement using existing
            public data, and applies it to build a management index
            covering 15,000 schools across 65 countries, and another
            index covering nearly all public schools in Brazil. Both
            indices show a strong, positive relationship between school
            management and student learning. The paper then develops a
            simple model that formalizes the intuition that strong
            management practices might be driving learning gains via
            incentive and selection effects among teachers, students and
            parents. The paper shows that the predictions of this model
            hold in public data for Latin America, and draws out
            implications for policy.
geo_reg_key: 119228,517191
repnme: repnme: Measuring and Explaining Management in
            Schools : New Approaches Using Public Data
display_title: Measuring and Explaining Management
            in Schools : New Approaches Using Public Data
listing_relative_url: /research/2019/11/31535876/measuring-explaining-management-schools-new-approaches-using-public-data
topicv3: Teachers Management m1326119 1523,Teacher Pay and Incentives m1326551 1530,Individual Performance Management m1326767 2058,Education m1327048 470,Private Education m1327271 1490
publishtoextweb_dt: 2019-11-06T00:00:00Z
historic_topic: Education,Governance,Social Protections and Labor
no_of_pages: 83
author_keys: 0: author_key: 1351983
1: author_key: 1351982
model_extractions: Policy Research Working Paper 9053

# Measuring and Explaining Management in Schools New Approaches Using Public Data

_Clare Leaver Renata Lemos Daniela Scur_ Education Global Practice November 2019
Policy Research Working Paper 9053

# **Abstract**

Why do some students learn more in some schools than others? One consideration receiving growing attention is school management. To study this, researchers need to be able to measure school management accurately and cheaply at scale, and also explain any observed relationship between school management and student learning. This paper introduces a new approach to measurement using existing public data, and applies it to build a management index covering 15,000 schools across 65 countries, and another index covering nearly all public schools in Brazil. Both indices show a strong, positive relationship between school management and student learning. The paper then develops a simple model that formalizes the intuition that strong management practices might be driving learning gains via incentive and selection effects among teachers, students and parents. The paper shows that the predictions of this model hold in public data for Latin America, and draws out implications for policy.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at rlemos@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Measuring and explaining management in schools:** New approaches using public data

Clare Leaver Renata Lemos Daniela Scur _Blavatnik School of Government, World Bank Dyson School of Applied University of Oxford CEP-LSE Economics and Management, CEPR Cornell University CEPR, CEP-LSE_

**Keywords:** management, teacher selection, teacher incentives, cross-country **JEL codes:** M5, I2, J3

**Acknowledgements:** We thank Melissa Adelman and Justin Sandefur for helpful discussions and comments on earlier drafts. We also thank participants at the World Bank’s Regional Study authors’ workshop, RISE Annual Conference 2018, SIOE 2019, Cornell University Development seminar, Nikki Shure, Chris Barrett, John Hoddinott, Vicente Garcia for useful suggestions. José Mola, Raissa Ebner, Maria José Vargas, Claudia Rivas, and Ildo Lautharte provided excellent research assistance. Leaver is grateful for the hospitality of Toulouse School of Economics, 201819.
# **1 Introduction**

Despite global calls for improvements in education, progress towards learning for all is slow. This deficit is particularly pronounced for poor children and children in low-income countries [Akmal and Pritchett, 2019]. But why do some students learn more in some schools than others? While there are many contributing factors at system, school, and household-level, one consideration receiving growing attention is school management—the processes and practices used by principals day-today as they run their schools [World Bank, 2018]. Academics and practitioners interested in this issue face two challenges: how to _measure_ school management accurately and cost-effectively at scale across schools and countries; and how to _explain_ any observed relationship between school management and learning outcomes in a way that elucidates the underlying mechanisms to guide policy. This paper addresses both of these challenges.

Our first contribution is to develop a new approach to measurement that can, in principle, be used with any existing public dataset containing items about school management. We illustrate using two public datasets as examples: the OECD’s Programme for International Student Assessment (PISA), and the Brazilian school census survey, Prova Brasil. The essence of our approach is to benchmark against the “state of the art”, but expensive, World Management Survey (WMS) in Bloom et al. [2015a]. We show how questions from these public surveys can be classified into WMS topics (53 PISA questions into 14 WMS topics and 33 Prova Brasil questions into 8 WMS topics), how the responses can be coded using the WMS scoring rubric, and finally how these grades can be built into a school management index. Our PISA-based index covers over 15,000 schools across 65 countries, and our Prova Brasil-based index covers nearly all public schools in Brazil. These indices are well-validated and can be used by researchers interested in studying the role of management in education systems across a far wider range of countries and schools than was previously possible.<sup>1</sup> All three indices, WMS, PISA, and Prova Brasil, show a strong, positive (within-country) correlation between school management and student learning outcomes, echoing recent causal evidence from randomized controlled trials in the U.S. [Fryer, 2014, 2017].

Our second contribution is to develop a framework to explore why management matters for schools.

> 1 For example, see Wössmann [2016] for a review of education systems research using large, cross-country surveys.

1
We set out, in general terms, how the impact of school management can be decomposed into learning gains that arise because given actors (teachers, students and parents) become more productive, and learning gains that arise because different actors join the school. To explore why these _incentive_ and _selection_ effects might arise, we turn to a specific model that captures key features of education systems in Latin America.

This model has two main building blocks. The first is the education production function: we assume that student learning depends on teacher ability, teacher effort, and household effort. The second is the impact of management practices where, considering the personnel policy restrictions the public sector faces, we distinguish between operations and people management. Good people management practices enable managers to observe and contract on the performance of their employees, as well as to cultivate the intrinsic motivation of their staff. Good operations management practices enable managers to use resources efficiently and hence offer a higher level of teacher compensation and a more stimulating environment for students.<sup>2</sup> Our framework predicts that good people management practices increase expected test scores through two channels. A teacher with a given ability and intrinsic motivation to teach exerts more effort because these practices provide extrinsic, and cultivate intrinsic, incentives. Compounding this, good people management practices improve selection: a teacher with high ability and high intrinsic motivation prefers a school with performance pay over alternative employments because she anticipates that she will work hard and be rewarded for producing student learning. We focus on Latin American countries and find support for both mechanisms in our PISA data. Principals in schools with higher PISA-based people management scores (predominantly private schools) are less likely to report experiencing teacher shortages and also report higher levels of teacher motivation and effort, compared to principals in schools with lower PISA-based people management scores.

Our framework also predicts that good operations management practices increase expected test scores through two channels. There is no teacher incentive effect but the selection effect remains, now driven by the _level_ rather than _structure_ of compensation. This is reinforced by a household

> 2This assumption echoes the observation made by Baker et al. [1988] that compensation plans featuring explicit rewards seldom account for _all_ of a worker’s rewards.

2
incentive effect that arises because strong operations management practices encourage both students and parents to increase their inputs. We also find evidence of these mechanisms in our PISA data for Latin America. Principals in public schools with higher PISA-based operations management scores are less likely to report experiencing teacher shortages and also report higher levels of teacher motivation, teacher effort and household effort, compared to principals in public schools with lower PISA-based operations management scores.

While this is not definitive causal evidence, this combination of theory and descriptive empirical analysis offers a novel insight into why management matters in schools and we therefore move on to consider policy implications. People management practices such as performance pay, while common in the private sector, may not be possible in public schools. But there would seem to be fewer barriers to conducting assessments to judge teacher effectiveness, and letting such appraisals lead to changes in public recognition, opportunities for professional development, likelihood of career advancement, and/or greater responsibilities. That is, these people management practices help to attract, develop and reward good performers, and, our analysis suggests, should improve both teacher selection and incentives.

There is also substantial variation in the strength of operations management practices _within_ the public sector. This suggest a role for government to encourage principals in public schools with weak operations management to follow best practices. Specific areas suggested by our analysis include processes that facilitate: personalization of learning; dialogue among staff, students and parents focused on continuous improvement; and collection and use of student assessment data.

**Related literature.** Our first contribution—a new approach to measure management practices in schools—relates to two bodies of work. The first is the literature that has evolved since the creation of the WMS dataset first described in Bloom and Van Reenen [2007]. The WMS methodology has been adapted to a range of public sector institutions, including schools and universities [Bloom et al., 2015a, McCormack et al., 2014], healthcare facilities [Bloom et al., 2015b, 2019b], social programs [Delfgaauw et al., 2011, McConnell et al., 2009], and the civil service [Rasul and Rogger, 2016], as well as to low-income settings [Lemos and Scur, 2016]. However, it is expensive and timeconsuming to implement at scale; our approach is a feasible alternative. The second is the literature

3
studying the role of education systems and institutions in determining student performance across countries [Wössmann, 2016]. Many recent papers use PISA data and have looked at this issue through the lens of autonomy [Hanushek et al., 2013, Wössmann et al., 2007], competition [West and Wössmann, 2010], student tracking [Hanushek and Wössmann, 2006, Ruhose and Schwerdt, 2016], external exams [Wössmann, 2005], and instructional time [Lavy, 2015]. Our PISA-based index enables researchers to consider school management in such studies.

Our second contribution—a framework to explain why management matters in schools—relates to the literature in personnel economics exploring incentives and selection. These channels have featured in prior work seeking to explain the performance of private sector employees [Bender et al., 2018, Cornwell et al., 2019, Lazear, 2000], public sector employees [Finan et al., 2017, Prendergast, 2007] and politicians [Besley, 2004, 2006, Gagliarducci and Nannicini, 2013, Martinez-Bravo, 2014]. Most closely related is Lazear [2003], who emphasises the potential selection margin of teacher performance pay, albeit without fully working up a formal model.<sup>3</sup> A selection margin also features in the dynamic occupational model of Rothstein [2015] and the Roy model of Biasi [2019]. We study a wider range of management practices (beyond just performance pay) and provide an intuitive decomposition of the impact of these practices on student learning into incentives and selection.

The remainder of this paper is organized as follows. In Section 2, we set out our approach to measure management practices in schools, illustrating with the construction and validation of PISA-based and Prova Brasil-based management indices. In Section 3, we describe our theoretical framework, its testable predictions, a series of corroborative descriptive analyses from across Latin America, and the policy implications of these results. Section 4 concludes.

> 3See also Dohmen and Falk [2010] who briefly sketch out theoretical reasons why fixed wage contracts and piece rates might be expected to have different impacts on sorting by ability.

4
# **2 How to measure management in schools?**

Until the early 2000s, management was typically viewed as an unmeasurable productivity shifter, to be relegated to the residual in any performance regression [Bloom and Van Reenen, 2007]. Since then, improvements in survey methodology and data access have allowed for advances in measurement. The current “state of the art” approach uses a dedicated survey—the World Management Survey (WMS)—to measure establishments’ adoption of structured management best practices. While the WMS offers uniquely rich information about management practices, it costs approximately USD400 per interview and takes about 4 months to conduct a single country wave [Bloom et al., 2016]. In view of these costs, it may not be well-suited to every context.

In this section, we propose an alternative three-step approach than can, in principle, be used with any existing public dataset containing information on management practices. The first step is to use the original WMS phone survey as a benchmark, and to look for questions in the public survey that elicit information on the management practices already measured by the WMS.<sup>4</sup> The second step is to code answers in line with the WMS methodology. And the final step is to create a management index. In Section 2.1, we provide a brief overview of the WMS questions and coding. In Section 2.2, we describe our approach using two existing public datasets as examples: PISA and the Brazilian school census survey, Prova Brasil. Since Brazil and several other PISA countries are part of the Bloom et al. [2015a] sample, we can compare the (within-country) distribution of each index with the corresponding (within-country) distribution of the WMS index. Both indices are well-validated and can therefore be used by researchers interested in studying management across a wider range of countries and schools than was previously possible.

> 4Our approach follows the spirit of the re-casting of the original phone-based World Management Survey into the US Census Management and Organizational Practices Survey (MOPS) administered to the population of US manufacturing establishments as a self-reported questionnaire [Bloom et al., 2019a]. The MOPS has been replicated in a number of other countries. Its questions follow the WMS topics and look to measure similar practices, but with self-reported answers.

5
# **2.1 Overview of the World Management Survey methodology**

The WMS was developed to measure adoption of structured management best practices in establishments across a range of countries and industries.<sup>5</sup> The rigorous data collection is based on double-blind, semi-structured interviews conducted by highly-trained analysts and monitored by supervisors experienced on the survey methodology. Following its successful implementation in the private sector, the WMS was subsequently extended to public sector organizations [Bloom et al., 2015a, 2019b]; in this paper, we focus on the latter.

The public-sector WMS covers 20 topics across two main areas: _operations management_ and _people management_ . Broadly speaking, operations management in schools covers practices including: whether the school has standardization of instructional processes across classrooms while allowing for within-classroom personalization of learning; whether and how the school uses assessments and data; and whether and how the school sets and uses targets and keeps track of progress. People management covers practices in handling good and bad performance measuring whether there is a systematic approach to identifying good and bad performance, rewarding school teachers proportionately, dealing with underperformers, and promoting and retaining good performers.

For each WMS topic, there is a scoring grid ranging from 1 to 5, which serves as a guide to evaluate answers to questions during the semi-structured interviews. A score between 1 to 2 refers to a school with practically no structured management practices or very weak management practices implemented; a score between 2 to 3 refers to a school with some informal practices implemented, but these practices consist mostly of a reactive approach to managing the school; a score between 3 to 4 refers to a school where a good, formal management process is in place (though not yet consistent enough) and these practices consist mostly of a proactive approach to managing a school; and a score between 4 to 5 refers to well-defined, strong processes in place which are often seen as best practices in education. The overall management index, which measures the level of adoption of structured management best practices, is simply the average of the scores for these 20 topics.

The practices measured by the survey seem to matter: Bloom et al. [2015a] show that their school management score is strongly positively correlated with school-level student outcomes across 6

> 5See Bloom and Van Reenen [2007] for the survey’s inception and Bloom et al. [2016] for a recent review.

6
WMS countries (Brazil, Canada, India, Sweden, UK and US).<sup>6</sup> They find a strong positive correlation for these countries: moving from the bottom to the top quartile of management is associated with a large increase in student learning outcomes, equivalent to approximately 0.4 standard deviations.

# **2.2 A new approach using existing public datasets**

We now describe our approach, illustrating with the examples of PISA and Prova Brasil.<sup>7</sup> .

**Construction.** In 2012, alongside its famous student proficiency tests, PISA ran school principal surveys across 65 countries which included a wide-range of questions on both operations and people management.<sup>8</sup> As a first step, we classified each of the PISA questions that could fall under one of the WMS topics, identifying 53 PISA questions that fit into 14 of the WMS topics.<sup>9</sup> As a second step, we manually assigned scores for each of these PISA questions following the spirit of the scoring grid of the WMS and the US Census Management and Organizational Practices Survey (MOPS). As a final step, we built the overall management index, and the operations and people management sub-indices, following Anderson [2008]. This methodology weights the impact of the included variables by the sum of their row in the inverse variance-covariance matrix, thereby assigning greater weight to questions that carry more “new information”.<sup>10</sup>

PISA data is excellent for cross-country analysis, but it precludes in-depth analyses within countries as the sample of schools per country is typically small and does not include the necessary identifiers. Many countries, however, conduct their own national detailed surveys with school principals, teachers, and students in addition to administering standardized tests across grades. Latin America is particularly prolific: for example, Brazil’s Prova Brasil, Colombia’s SABER, Chile’s SIMCE, and Peru’s ECE are all available to researchers. These questionnaires provide rich information about

> 6We replicate the primary figure from Bloom et al. [2015a] in Figure B.3 in Appendix B. It plots the school-level student learning outcomes by each quartile of school management score.

> 7We provide details to enable replication in Appendix C

> 8Our main focus is on the 2012 data because that survey wave contains a richer set of questions, particularly relating to people management. See Appendix C for a mapping of the 2015 PISA data.

> 9The set of WMS topics that had matching PISA questions is detailed in Appendix C.

> 10We also built the indices using alternative methods (straightforward standardization, factor analysis, and factor analysis with Bartlett correction) which yielded similar results.

7
practices at the school, as reported by a range of actors. In addition, the samples are usually large (often census-based) and contain school identifiers, thereby enabling researchers to explore heterogeneity and answer a wide range of policy-relevant questions. We illustrate how our approach can be applied widely to other national surveys using the example of _Prova Brasil_ . This national survey plays a significant role in Brazil’s education policy because its test results, along with promotion, dropout, and retention rates, are the main inputs to the _Índice de Desenvolvimento da Educação Básica_ (IDEB), a national index representing educational quality at the school, municipality, and state levels.

We followed the same steps to create a Prova Brasil-based management index: we classified 33 questions (14 from the principal questionnaire and 19 from the teacher questionnaire) into 8 WMS topics, coded responses following the same rubric, and used the Anderson [2008] method to build a school management index.

**Potential concerns.** One of the key differences between the WMS survey and PISA or school census surveys is that the WMS is administered and analyzed by an independent interviewer, while the latter surveys are self-reported. There are a number of issues with self-reported data: for example, problems with translation and interpretation, and/or measurement equivalence. To address measurement error of cross-cultural understandings and norms on answering questions in our PISA index, we standardize our PISA-based management index _within countries_ . This has an important implication: since all 65 countries have a mean score of zero, our index cannot be used to construct cross-country rankings of school management. Instead, the value of our PISA-based index lies in enabling academics and practitioners to study the (within-country) correlation between management and other variables for a far wider set of countries than was previously possible.<sup>11</sup> This is not a concern for country-specific national surveys.

Another concern with self-reported data is that it is difficult to assess whether respondents are being accurate and truthful. In the WMS there are several strategies to elicit truthful information

> 11In the 2012 PISA the dataset included a PISA-built ‘leadership and management’ measure. This is distinctively different from ours, as it was based off a section of the questionnaire that was titled ‘management’ and contained only a small subset of questions. This index fails to take advantage of the full questionnaire and the information available elsewhere that also speaks to managerial practices used in the schools. More pertinently, PISA’s measure does not compare well to the (empirically robust) management index derived from the World Management Survey (see Liberto et al. [2015]).

8
during the interview (such as always asking open-ended questions and asking for examples), but these are not available in self-reported questionnaires. We address this issue by focusing on the topics that have a direct equivalent in the WMS to allow for a clear benchmark for our new index. If principals are reporting “good” information in these surveys that allow us to capture similar signals as the WMS scores, we should see similar distributions of scores across the common countries and a similar overall relationship between management and student test scores across countries. For PISA, we compare the distribution of scores and the performance correlations for the common countries as there are no school identifiers available. For Prova Brasil, we use school identifiers to match schools directly and hence provide a one-to-one comparison of the index standardized values.

**Validation of new indices.** As a first validation exercise, we compare the distribution of our PISA-based management index with the distribution of school management as measured by the WMS data in Figure 1 for all countries that the WMS has collected data.<sup>12</sup> The PISA and WMS distributions are reassuringly similar. The Kolmogorov-Smirnov test for equality of distributions rejects in only one of the 9 cases, Italy, where the PISA-index is somewhat more dispersed.<sup>13</sup>

As a second validation exercise, and to ensure we are picking up important variation with our management index, we conduct a basic check of the correlation between our measure and student performance. For each country we separate schools into quartiles of the management measure, and in Figure 2 we show, for each quartile, the average PISA test scores for math, reading and science (in deviations from the global mean). The graph includes all students and schools across the 65 countries available in the 2012 PISA dataset. This simple relationship suggests that students in schools in the bottom quartile of management within their country score are, on average, about 6 points lower than the PISA global mean, while students in schools in the top quartile of management within their country score, on average, about 5.5 points higher than the PISA global mean. To put this into context, 41 PISA points in math are the equivalent of a year of learning. The range of

> 12Independent researchers conducted the WMS in Colombia and Mexico during 2015, with guidance and supervision from the original WMS team. These data were not available to Bloom et al. [2015a]. India is not included in Figure 1 because it did not participate in PISA in 2012.

> 13We show comparisons between the WMS and PISA sub-indices for operations management in Figure B.1 and people management in in Figure B.2 in Appendix B and confirm that the distributions are consistent for both sub-indices.

9
our results mirror how much, for example, the UK average science score changed between 2009 and 2015 (5 points), and how much the Brazilian average science score decreased over the same period (4 points).

Unlike PISA, the data in Prova Brasil includes school identifiers that allow for a one-to-one match with the schools surveyed in the 2013 WMS wave.<sup>14</sup> In total, we have 262 matched schools in the public sector. We use this matched sample in Figure 3 where we show a school-level binned scatter plot of WMS management score against the Prova Brasil-based management score. There is a positive and significant correlation of 0.19, suggesting reasonable internal validation of the Prova Brasil index. As with the PISA index, we repeat the exercise of correlating the new index with student performance in secondary schools and find the same pattern (see Figure 4).

In Table 1 we formalize these relationships by reporting the average correlations between student learning and our management indices. For the student-level PISA dataset, we run OLS regressions via the OECD’s repest Stata command, which uses the five available test score plausible values for each student and subject. We report the standard errors in parentheses and _p_ -values in square brackets. The standard errors are clustered at the school level and use the appropriate survey weights.<sup>15</sup> In the PISA specifications we include country fixed effects, and successively introduce school controls (dummies for school location, student-teacher ratio, log of the number of students, share of government funding relative to total funding the school receives, and ratio of computers connected to the web used as a proxy for school resources) and then student controls (gender, grade, socio-economic status and immigration status). All panels use the same sample but have different subject outcome variables. The R-squared for each set of regressions is reported within each panel, while the common sample characteristics and controls included are reported at the bottom of the table. Column (1) shows the raw relationship between the PISA-based school management index and student performance, only controlling for country fixed effects. The raw relationship ranges from just over 4 to almost 5 points on the PISA scale.<sup>16</sup> Recall that 41 points on the PISA scale for math is equivalent to about one year of learning, and thus the raw correlation is equivalent to about one month of learning for math (similar for the other subjects). Column (2) includes school

> 14Prova Brasil was also administered in 2013.

> 15See Jerrim et al. [2017] for a thorough review of how to best use PISA scores and survey weights.

> 16PISA is standardized across years and countries such that the mean is 500 and the standard deviation is 100.

10
controls, which absorb little of the variation, and Column (3) shows the fully-specified regression including student controls. These controls account for a further point in the student performance. While we refrain from ranking management indices across countries, Figure 5 plots the coefficients of country-level regressions of management on PISA math test scores using the specification of Column (1). The estimation loses precision once we restrict to individual country samples but still broadly supports the positive relationship found in Table 1.

For the Prova Brasil student-level dataset, we run standard OLS regressions, also clustering the standard errors at the school level. Results are reported in standard deviations. In these Prova Brasil specifications, we include state fixed effects, and successively introduce school controls (student-teacher ratio, log of the number of students, dummy variables indicating the presence of an IT lab, science lab, and library, a dummy for male principals, dummies for educational attainment, and dummies for experience as principal), and then student controls (gender, race, socio-economic status, and mothers’ educational attainment). Column (4) shows the raw relationship between the Prova Brasil-based school management index and student performance, only controlling for state fixed effects. One standard deviation higher score in the management index is strongly correlated with 0.068 standard deviations higher Portuguese scores and 0.078 standard deviations higher math scores. Column (5) shows that including school characteristics absorbs very little of the variation, while Column (6) shows that including student characteristics absorbs only slightly more. The fully specified regression supports the general positive relationship between the school management index and student performance.

# **3 Why does management matter in schools?**

There are myriad uses of our new indices. In this paper, we push the frontier of understanding the mechanisms behind the management and performance relationship by focusing on teachers. Our aim is not to provide a theoretical contribution _per se_ , but rather to formalize a policy discussion around teacher incentive and selection mechanisms and their relationship to management practices and student performance. We take wider system-level factors—in particular hiring and firing autonomy, admissions autonomy and competition between schools—as given and assume that teachers and 11
students make choices within the of this environment.

Real-world education systems are diverse, and in particular the dynamics of the public and private sector — and the type of private sector offerings — are different across countries. In some contexts, private schools target affluent households, and jobs in private schools are often seen as more attractive than public sector jobs, typically providing some form of performance-based compensation. In other contexts, there has been a growth of ‘low-cost private schools’ that deliberately cater for the lower end of the income distribution and, in these settings, jobs in the public sector typically confer significant rents relative to the private sector.

In view of this diversity, we focus our model and empirical test on one particular regional system: Latin America. We choose Latin America because its education systems are reasonably homogeneous across countries in terms of the character of public and private schools. Specifically, the private school system caters to the middle (and upper) classes and accounts for about one-fifth of high school students. Private schools tend to be better funded (via costly school fees) and in turn pay higher teacher salaries and offer better facilities. Public schools, on the other hand, are often poorly funded and operate in highly centralized environments. Teacher pay is set on a rigid scale dictated by strong unions. Focusing on a region that has such systems makes the applied theory exercise substantially less complex, and the prevalence of large-scale national surveys in the region opens many possibilities for future empirical work.

Table 2 reports the correlation exercise in Columns (1) to (3) of Table 1 for Latin American countries only, and confirms that the relationship between management and student performance is strong in the region. Further, the coefficient on private schools indicates that students in private schools achieve higher scores, by about 55 points, than students in public schools. This affords a suitable empirical environment to study the channels that we are interested in this section.

# **3.1 Overview of the theoretical framework**

The analysis is built around a student-level education production function. A common, general formulation is _y_ = _A_ ( _L, K_ ) + _ε_ where _y_ is a measure of student learning, _L_ and _K_ are respectively labour and physical capital inputs into the student’s education, _A_ is a (school-specific) productivity 12
parameter, and _ε_ is an error term. Here, we specialize to _y_ = _θ e_ + _a_ + _ε_ , where _θ_ is teacher ability, _e_ is teacher effort, and _a_ is household (student and/or parent) effort. That is, we enrich the specification of labour to allow for (additively separable) teacher and household inputs but abstract from the role of physical capital and school-level productivity.<sup>17</sup> Using a theoretical framework built around this education production function, we show that school management structures can impact student learning outcomes via the following three channels:

1. _Teacher selection_ : schools with high management scores offer compensation packages that select in more able (higher _θ_ ) and more intrinsically motivated (lower effort cost) teacher types.

2. _Teacher incentives_ : schools with high management scores offer compensation packages that extrinsically incentivize, and adopt practices that intrinsically motivate, more effort from any given teacher type that selects in.

3. _Household incentives_ : schools with higher management scores institutionalize a strong work ethic and culture of high achievement among students and encourage greater parental involvement within the school (higher _a_ ).

In Section 3.2, we present a simple model that suffices to make the points above. In Section 3.3 and Section 3.4 we explain, intuitively, how the above selection and incentive effects are driven by people management and operations management. Then, in Section 3.5 we draw together these results and discuss implications for policy. We also briefly comment on how predictions would change in an alternative model featuring ‘low-cost private schools’.

# **3.2 The model**

We focus on a teacher who must decide whether to accept a job offer in her assigned public school, or decline it and apply to a private school or the outside sector.

> 17In principle, parents could play a further role by selecting between schools. Since our PISA data cannot speak to this issue, we leave the analysis of management-induced household selection for future work. Note that we assume management practices change effective labour inputs. In their study of the IT industry, Schivardi and Schmitz [2019] assume that management is an additional input, alongside capital and labour, in an approach that they term “management as a production technology”.

13
**Preferences.** The teacher is risk neutral and cares about her compensation _w_ and effort _e_ . When working in the education sector, the teacher’s preferences are _w −_ ( _e_<sup>2</sup> _− c e_ ). The parameter _c_ captures her _intrinsic motivation_ . This is because for _e < c/_ 2 she derives a marginal benefit from exerting an extra unit of effort in teaching; it is only when _e > c/_ 2 that effort costs kick in. We assume that _c_ = _τ_ + ∆. The first component _τ_ denotes the teacher’s baseline _intrinsic motivation_ . This can be thought of as the realization of a random variable with density function _f_ . The teacher observes this realization perfectly, while (at the time of hiring) employers observe nothing. The second component ∆ is a motivational increment that, as we describe below, is determined by the people management practices in the teacher’s chosen school. When working in the other sector, the teacher’s preferences are simply _w − e_<sup>2</sup> ; intrinsic motivation plays no role. We abstract from differences within classes and focus on a representative household (student plus parents). This household cares only about its effort level _a_ , and has preferences _−_ ( _a_<sup>2</sup> _− γ a_ ). The parameter _γ_ is a motivational increment that is also determined by management practices.

**Performance metrics.** Let _y_ 1 denote a representative student’s learning outcome in a school that hires the teacher, and _y_ 0 denote a representative student’s learning outcome in a school that does not hire the teacher. To the extent that teachers contribute to learning, one would expect _y_ 1 _> y_ 0. We capture this in a simple way by assuming _y_ 1 = _θe_ + _a_ + _ε_ and _y_ 0 = _a_ + _ε_ . If the teacher is not hired by a school but instead chooses to work in the outside sector, her performance is _z_ = _θe_ + _ε_ . The component _θ_ denotes the teacher’s _ability_ . This can be thought of as the realization of a random variable with density function _g_ , and which is drawn independently of _τ_ . The teacher observes this realization perfectly, while (at the time of hiring) employers observe nothing. Draws of the error term _ε_ are independent across employments. We assume throughout that _ε_ is mean zero and distributed _U_ <u>[</u> _<u>ε, ε</u>_ <u>].</u> At times, for the purposes of illustration, we also assume a specific (uniform) distribution for _θ_ , as part of a numerical example that we discuss at the end of this section.<sup>18</sup>

> 18Note that the basic production function is the same across schools; management practices matter by affecting which _θ_ -types are hired, the teacher’s choice of _e_ (which depends on which _τ_ -types are hired), and the household’s choice of _a_ .

14
**Compensation schemes.** Schools offer either a performance-pay contract or a fixed wage contract. Under the former, the teacher receives a base wage of _W_ plus a bonus _B_ if her performance  ̄ exceeds a threshold _y_ . Under the latter, the teacher simply receives a base wage of _G_ . The outside sector offers a performance-pay contract with a low base wage (normalized to zero) and a bonus _β_ if performance exceeds a threshold _z_  ̄.

**The impact of management practices.** We assume that people management has two effects. The first relates to the _structure_ of compensation: good people management practices enable managers to observe, and contract on, the performance of their employees—i.e. to offer a performancepay contract. The second relates to teacher motivation: good people management practices enable managers to cultivate the intrinsic motivation of their staff—i.e. to increase ∆. We assume that operations management also has two effects. The first relates to the _level_ of compensation: good operations management practices free up resources and enable managers to offer a higher level of base pay. The second relates to household effort: good operations management practices help to create a stimulating environment for students and parents —i.e. to increase _γ_ .

We classify schools into three management types: _high_ (strong people and strong operations management), _intermediate_ (weak people but strong operations management), and _low_ (weak people and weak operations management). Performance metrics are indexed accordingly by _i_ = _H, I, L_ . We assume that high management schools are found exclusively in the private sector, while the public sector consists of a mix of intermediate and low management schools. This implies that performance-pay contracts are only offered by private schools (and the outside sector). Figure 6 provides evidence that this key assumption is well-supported in our PISA data. Here, we plot empirical cumulative distribution functions (CDFs) of the PISA-based people management score by sector and find that the private sector CDF (dashed blue plot) clearly first order stochastically dominates the public sector CDF (solid red plot).<sup>19</sup>

**Timing.** The timing of the game is as follows.

> 19We show empirical CDFs by country in Latin America in Figure B.4 in Appendix B. The private sector CDF first order stochastically dominates the public sector CDF in Colombia, Peru, Costa Rica and Mexico.

15
1. Nature chooses the teacher’s two-dimensional type. This realization ( _τ, θ_ ) is observed by the teacher but not by employers.

2. Employers announce management structures and compensation schemes.

3. The teacher is assigned (by government) to a public school and decides whether to accept this post or decline it and apply either to a private school or the outside sector.<sup>20</sup> 4. Having made an occupational choice, the teacher chooses an effort level. Simultaneously, if the teacher is in the education sector, households choose effort levels.

5. A performance metric is realized. The teacher is rewarded in accordance with the compensation scheme announced at Stage 2.

**Numerical example.** At times in the analysis below, we will invoke specific distributional and parameter assumptions. In this numerical example, teacher intrinsic motivation is distributed _τ ∼ U_ [0 _,_ 10], and teacher ability is distributed _θ ∼ U_ [1 _,_ 5]. These random variables are independent of each other and the error term in the production functions. In a _high_ management private school: teacher pay is _W_ + _B_ = 55 if _y_ 1<sup>_H≥_4</sup><sup>_._5 and</sup><sup>_W_= 15 otherwise,and the motivational increments are</sup> ∆= 0 _._ 5 and _γ_ = 2. In an _intermediate_ management public school teacher pay is _G_ = 35, and the motivational increments are ∆= 0 and _γ_ = 2. And in a _low_ management public school: teacher pay is _G_ = 30, and the motivational increments are ∆= 0 and _γ_ = 1. Pay in the outside sector is _β_ = 50 if _z ≥_ 1 and 0 otherwise.

Our interest lies in establishing the impact of management practices on student learning via teacher occupational choice and effort level, and household effort level. We do not model the government’s assignment rule, or the school principal’s choice of management structure, simply treating these as exogenous parameters. The model is straightforward to solve (see Appendix A for details) and yields the insights summarized in the next two sections.

> 20In assuming this timing, we abstract from applicant choice between schools in the public sector. As Table B.4 and B.5 in Appendix B, the degree to which teachers can choose among public schools varies across Latin America, yet our model generally fits the reality in these countries.

16
# **3.3 The impact of good people management**

In this subsection, we use the theoretical framework to give a possible explanation for why schools with good people management may produce better student outcomes. In Section 3.3.1, we decompose the test score gain from people management into two effects: teacher selection and teacher incentives. If this decomposition is correct, then we should see evidence of these mechanisms in intermediate school outcomes. We develop this argument, and present corroborative evidence from our PISA dataset, in Section 3.3.2.

## **3.3.1 Decomposing the test score gain into teacher selection and incentives**

A school of management type _i_ hires the teacher if, given her ( _τ, θ_ ) type, she expects to receive a higher payoff teaching in this school compared to other schools or working in the outside sector. Let the set of ( _τ, θ_ ) types hired by a school of management type _i_ be denoted by _T_<sup>_i_</sup> . The expected learning outcome of a representative student (i.e. _ex ante_ , prior to occupational and effort choices) can therefore be written as where 1 _{_ ( _τ,θ_ ) _∈T i}_ and 1 _{_ [( _τ,θ_ ) _∈T/ i}_ are indicator functions for the hiring and not hiring events. In keeping with the empirical application, we will refer to E [ _y_<sup>_i_]</sup> as the expected test score in school _i_ .

The difference in expected test score across high and intermediate management schools—that is, the impact of people management holding operations management constant—can be written as where the equality follows from the fact that people management only impacts test scores when the teacher is hired (the effect of household effort and the error term difference out). It is helpful 17
to decompose this difference as follows The first term on the RHS of equation (1) captures what we will term the _teacher incentive effect_ of good people management practices. Here, we compare the expected test score outcome in a high management private school with a teacher in the event that the teacher is hired to such a school against the expected test score outcome in an intermediate management public school with a teacher in the counterfactual event that the teacher is hired to a high management private school. In this way, we hold the set of ( _τ, θ_ ) types fixed and just consider how the incentive environment produces test scores.

In Lemma 1 in Appendix A, we derive teacher effort in high and intermediate management schools. Respectively, these are _e_<sup>_H_</sup> = 2( _<u>θε</u>_ _~~−~~_ _<u>Bε</u>_ <u>)</u><sup>+</sup><sup>_<u>τ</u>_</sup><sup><u>+∆</u></sup> 2 and _e_<sup>_I_</sup> =<sup>_<u>τ</u>_</sup> 2<sup>.Substituting, we can write the first incentive</sup> term in (1) as We see from this expression that there are two teacher incentive channels. Part of the reason that test scores are higher in schools with good people management practices is because any given ( _τ, θ_ ) type exerts more effort due to: (i) an _extrinsic_ incentive from the bonus _B_ , and (ii) additional _intrinsic_ motivation arising via the shift term ∆.

The second term in equation (1) captures what we will term the _teacher selection effect_ of good people management practices. Here, we compare the expected test score outcome in an intermediate management public school with a teacher in the event that the teacher is hired to such a school against the expected test score outcome in an intermediate management school with a teacher in the counterfactual event that the teacher is hired to a high management school. In this way, we 18
hold the incentive environment fixed and just consider how the selection of ( _τ, θ_ ) types produces test scores. Substituting for _e_<sup>_I_</sup> , we can write this second selection term as We see from this expression that there are also two selection channels. A further part of the reason that test scores are higher in schools with good people management practices is because: (i) the _τ_ -types selected in are intrinsically motivated to exert more _effort_ , and (ii) the _θ_ -types selected in are of greater _ability_ .

To see this, consider the numerical example illustrated in Figure 7. In the top panel, the grey shaded area depicts the set of ( _τ, θ_ ) types that are hired by a high management private school. The unshaded area depicts the set of ( _τ, θ_ ) types that are hired by an intermediate management public school. It is clear that the intermediate management public school experiences negative selection on both dimensions. More able teachers prefer the performance-contingent compensation schemes available either in private schools or the outside sector. And more intrinsically motivated teachers prefer private schools because they anticipate exerting higher effort (and hence higher pay).

# **3.3.2 Predictions for intermediate school outcomes and evidence from PISA**

Our theoretical framework suggests two mechanisms, teacher selection and teacher incentives, that could explain the positive correlation between people management scores and student learning outcomes apparent in the WMS, PISA and Prova Brasil data. If these mechanisms are correct, then we should see behavioural responses in intermediate school outcomes. In this section, we set out these predictions and then explore empirically whether they hold in our PISA data for Latin America.<sup>21</sup>

> 21These predictions are based on the numerical example illustrated in Figure 7 and are derived (via numerical integration) in Remarks 1 and 2 in Appendix A. We present the main results with our preferred specification, but include additional variations of the main explanatory variable in Table B.3 in Appendix B. The results are robust to alternative specifications.

19
**Teacher shortages.** The probability of hiring the teacher in a high management private school is higher than the probability of hiring the teacher in an intermediate management public school (via teacher selection). In the numerical example shown in the top panel of Figure 7, the area of the grey region is bigger than the area of the unshaded region.

The PISA dataset does not contain objective information on school-level vacancies, so we use a series of 4 questions in the school principal questionnaire that ask the principal whether he/she feels that the school’s capacity is hindered by a lack of qualified teachers in each of math, science, language and ‘other subjects’.<sup>22</sup> It is worth emphasising that these questions are open to considerable interpretation. For instance, a principal might answer ‘a lot’ because he/she feels that the school needs more new posts even if there are few vacancies for existing posts. Conversely, he/she might answer ‘not all’ because of a belief (or desire to say) that the school is coping despite there being vacancies.<sup>23</sup> With this caveat in mind, it can still be instructive to examine the data. Column (1) of Table 3 shows that, consistent with the theory, the teacher shortage index is 0 _._ 535 standard deviations _lower_ among private schools than public schools, significant at the 1 percent level. Column (4) repeats the specification but with the people management index instead of the private school dummy. The relationship is consistently negative, suggesting that a one standard deviation increase in the people management index is correlated with 0 _._ 062 _lower_ teacher shortage in schools, marginally significant at the 10 percent level.

**Teacher motivation.** The expected intrinsic motivation of a teacher hired to a high management private school is higher than the expected intrinsic motivation of a teacher hired to an intermediate management public school (via teacher selection and augmentation of teacher intrinsic motivation). In the numerical example in the top panel of Figure 7, the vertical height of the black point is greater than the vertical height of the blue point.

We explore this prediction by using the school climate section of the school principal questionnaire

> 22We describe how this teacher shortage index, and the other intermediate outcome indices for teacher motivation, teacher effort, and household involvement, are constructed in Appendix C. All indices are standardized.

> 23Consistent with this, by far the most common answer given by principals in both sectors is ‘not at all’, as reflected in the density of the standardized score in Figure B.5 in Appendix B.

20
(questions relating to the perception of teachers’ expectations of their students and meeting student needs, as well as the morale, enthusiasm, pride and valuation of academic achievement) to construct an index of teacher motivation. Column (2) of Table 3 shows that, consistent with the theory, the teacher motivation index is 0 _._ 591 standard deviations _higher_ among private schools than public schools, significant at the 1 percent level. Using the people management index also yields a consistent relationship, shown in Column (5). The coefficient suggests a one standard deviation higher people management score is associated with 0 _._ 238 _higher_ teacher motivation score, also significant at the 1 percent level.

**Teacher effort.** The expected effort level of a teacher hired to a high management private school is higher than the expected effort level of a teacher hired to an intermediate management public school (via teacher selection, extrinsic teacher incentives, and augmentation of teacher intrinsic motivation on-the-job)

We explore this prediction by using the school climate section of the school principal questionnaire (questions relating to how often teachers are absent, late and/or unprepared) to construct an index of teacher effort. Column (3) of Table 3 shows that, consistent with the theory, the teacher effort index is 0 _._ 792 standard deviations _higher_ among private schools than public schools, significant at the 1 percent level. Column (6) reports the same specification for the people management index, suggesting that a one standard deviation increase in the management score is associated with 0 _._ 074 _higher_ teacher effort, significant at the 5 percent level.. For completeness, Figure 8 plots the coefficients for country-level regressions using the same specifications reported in Columns (4) to (6) of Table 3. While there is some variation across countries, the results broadly hold.

# **3.4 The impact of good operations management**

In this subsection, we use the theoretical framework to give a possible explanation for why schools with good operations management may produce better student outcomes. In Section 3.4.1 we 21
decompose the test score gain from operations management into three effects: teacher selection, teacher incentives and household incentives. If this decomposition is correct, then we should see evidence of these mechanisms in intermediate school outcomes. We develop this argument, and present corroborative evidence from our PISA dataset, in Section 3.4.2.

# **3.4.1 Decomposing the test score gain into teacher selection, teacher incentives, and household incentives**

The difference in expected test scores across intermediate and low management public schools—that is, the impact of operations management holding people management constant—is Letting _a_<sup>_I_</sup> and _a_<sup>_L_</sup> respectively denote household effort in these schools, and using the same decomposition as before, we can rewrite this difference as There is no teacher incentive term because both extrinsic teacher incentives and augmentation of teacher intrinsic motivation depend on people management and this is assumed to be constant across these schools. We can write the teacher selection term as: Again, there are two teacher selection channels. Part of the reason that test scores are higher in schools with good operations management practices is because: (i) the _τ_ -types selected in are intrinsically motivated to exert more effort, and (ii) the _θ_ -types selected in are of greater ability.

22
To see this, consider the numerical example illustrated in Figure 7. The unshaded area in the top panel depicts the set of ( _τ, θ_ ) types that are hired by an intermediate management public school, while the unshaded area in the bottom panel depicts the set of ( _τ, θ_ ) types that are hired by a low management public school. It is clear that the intermediate management public school hires both more and better types.

In contrast to the people management case, there is a channel operating via household incentives. A further part of the reason that test scores are higher in schools with good operations management practices is because households (students plus parents) exert more effort due to additional _intrinsic_ motivation arising via the shift term _γ_ .

# **3.4.2 Predictions for intermediate school outcomes and evidence from PISA**

Again, we set out predictions relating to intermediate school outcomes (see Remark 2 in Appendix A), and then explore empirically whether they hold in our PISA data.<sup>24</sup>

**Teacher shortages.** The probability of hiring the teacher in an intermediate management public school is higher than the probability of hiring the teacher in a low management public school (via teacher selection). In the numerical example shown in Figure 7, the unshaded area is larger in the top panel relative to the bottom panel.

We take this prediction to the data using the 4 questions in the PISA school principal questionnaire that ask the principal whether they feel that the school’s capacity is hindered by a lack of qualified teachers (see Appendix C). Column (1) of Table 4 shows a negative and statistically significant correlation between the operations management index and the teacher shortage index. A one standard deviation increase in operation score is associated with a 0 _._ 076 standard deviation _decrease_ in the teacher shortage index, significant at the 10 percent level.

**Teacher motivation.** The expected intrinsic motivation of a teacher hired to an intermediate management public school is higher than the expected intrinsic motivation of a teacher hired to a

> 24We present the main results with our preferred specification, but include additional variations of the main explanatory variable in Tables B.2 in Appendix B. The results are robust to alternative specifications.

23
low management public school (via teacher selection). In the numerical example shown in Figure 7, the vertical height of the blue point in the top panel is greater than the vertical height of the orange point in the bottom panel.

Column (2) of Table 4 shows that, consistent with the theory, the partial effect of operations score on the teacher motivation index is positive and significant at 1 percent; a one standard deviation increase in operation score is associated with a 0 _._ 238 standard deviation _increase_ in the teacher motivation index, significant at the 1 percent level.

**Teacher effort.** The expected effort level of a teacher hired to an intermediate management public school is higher than the expected effort level of a teacher hired to a low management public school (via teacher selection)

Column (3) of Table 4 shows that one standard deviation higher operations score is correlated with 0 _._ 076 standard deviations higher teacher effort, significant at the 5 percent level.

**Household effort** The expected level of household effort in an intermediate management public school is higher than the expected level of household effort in a low management public school (via augmentation of student intrinsic motivation): _a_<sup>_I_</sup> _> a_<sup>_L_</sup> .

We explore this prediction by using the school climate section of the school principal questionnaire to construct an index of household effort, combining student behavior questions (relating to how often students are truant, late, disrespectful and/or disruptive) and parental involvement questions (relating to the extent to which parents: are interested in, and discuss, their child’s progress and behaviour; volunteer for school activities; and participate in school governance or other forms of accountability). Column (4) of Table 4 shows that, consistent with the theory, one standard deviation higher operations management score is correlated with 0 _._ 160 higher household effort, significant at the 1 percent level. For completeness, Figure 9 plots the coefficients for country-level regressions using the same specifications in Columns (1) to (4) in Table 4. Again, the results 24
broadly hold at country-level.

# **3.5 Summary of theoretical prediction**

We developed a simple theoretical framework, built around a student-level education production function, to explore why management practices might matter in schools. Using this framework, we showed that people management practices may be contributing to higher student test scores through two channels: teacher selection and teacher incentives. The predictions that these channels imply for intermediate school outcomes—fewer teacher shortages, higher teacher motivation, and higher teacher effort in schools with strong people management than in schools with weak people management—are all well-supported in our PISA data for Latin America.

We also showed that operations management practices may be contributing to higher student test scores via two channels: teacher selection and household incentives. The empirical support for the predictions that these channels imply for intermediate school outcomes—fewer teacher shortages, higher teacher motivation, higher teacher effort, and higher household effort in schools with strong operations management than in schools with weak operations management—is also strong.

While this does not represent definitive causal evidence, this combination of theory and descriptive empirical analysis offers an insight into why management appears to matter in schools and so we cautiously move on to policy. For example, what type of school management practices might be changed to drive improved student learning? Our analysis suggests that people management is a good place to start. While it may not be feasible (on political or budgetary grounds) for governments to introduce performance pay in public schools, it may be possible to conduct assessments to judge teacher effectiveness, and for these appraisals to lead to changes in public recognition, and to opportunities for professional development, likelihood of career advancement, and greater responsibilities and leadership. Such practices also reward and develop good performers and create a good employee proposition and could improve both teacher selection and incentives.

Beyond the difference in people management across private and public sectors, a striking feature of the PISA data is that there is substantial variation in the strength of operations management practices within the public sector. Some public schools are adopting management practices that 25
appear to be driving student learning, while others _within the same public education system_ are not. This suggests a role for government to encourage schools with weak operations management to follow best practice. As we observe from our mapping exercise, ‘strong’ operations management practices do two things, they: actively promote quality of delivery in the classroom (e.g. via personalization of learning, and encouragement to follow best educational practice); and put processes in place to review school performance and drive change (e.g. dialogue and meetings focused on continuous improvement, collection and use of student assessment data). Such practices could be adopted more widely in the public sector and, our analysis suggests, should improve both teacher selection and household incentives.

To be sure, our goal has not been to produce a global theory; given the diversity of real-world education systems, we have deliberately focused on one region, Latin America, and developed a theoretical framework for that context. One could adapt this framework to study different settings, for instance South Asia and parts of East Africa where there is a preponderance of ‘low-cost private schools’. Such a model would need to allow for the possibility of ‘queues’ for jobs in public schools and, as a result, to explicitly model _demand_ -side selection. To the extent that strong management practices enable public school principals to offer higher levels of compensation and then choose motivated and able teachers from the resulting queue, then qualitatively similar predictions would likely still apply. We hope that our new index will enable fertile ground for further research.

# **4 Conclusion**

Policy makers have begun to set ambitious, universal learning goals. To achieve these targets it will be necessary to understand why, within and across current education systems, some students are learning more in some schools than others. Although there are likely many factors at work, it has been suggested that part of this variation in learning might stem from differences in school management. To explore this issue and develop policy, academics and practitioners need to be able to measure school management accurately and cost-effectively at scale across schools and countries, and be in a position to postulate mechanisms behind any observed relationship between school management and student learning outcomes.

26
This paper has responded to these observations by developing new approaches to measurement, as well as a simple theoretical framework that captures key features of education systems in Latin America. The first application of our new measurement approach used publicly available data from the school principal surveys conducted by PISA to construct a school management index spanning 65 countries. This PISA-based school management index can be well-validated against the more detailed (though also much more expensive) index based on the WMS. As such, it has clear value in settings where cross-country coverage is important, enabling researchers to study and compare the (within-country) correlation between management and student/school-level outcomes for a far wider set of countries than was previously possible.

Our second application used publicly available data from a national administrative survey to construct a school management index spanning all public schools in a single country: Brazil. This Prova Brasil-based index was also well-validated against the WMS. This second application has value in settings where within-country coverage, and the availability to merge with other administrative data sets, is important.

It is striking that both of our new school management indices confirm the strong positive correlation of school management scores with school-level student outcomes first reported in Bloom et al. [2015a]. A positive relationship holds for the global PISA sample, and in the census of schools in Brazil. Our theoretical framework, for the first time, formalizes the possible causal mechanisms in one of these regions: Latin America. We argued that strong people management practices may be improving student learning through a combination of teacher selection and incentive effects, and that schools could be encouraged to adopt practices that reward good performers, develop good performers, and create a good employee value proposition. Looking to operations management, we argued that strong operations practices may be improving student learning through a combination of teacher selection and household incentives, and that schools could be encouraged to adopt practices promoting quality of delivery in the classroom and adopt processes to review school performance and drive change. We also provided a suggestive set of evidence for these channels.

Improvements to management practices present an untapped opportunity for potentially large improvements in educational outcomes, particularly in cash-strapped regions of the world. One pos27
sible way of effecting change is to support existing school principals to introduce stronger people and operations management practices, for instance via training and resources. Fryer [2014, 2017] reports positive results from RCTs injecting best management practices into U.S. public schools. Another possibility is to contract new managers into existing public schools. Romero et al. [forthcoming] report mixed results from an RCT in Liberia in which (non-governmental) management teams were contracted to run public schools: contracting-in raised learning outcomes, but new managers spent more and may have engaged in strategic behaviour. Investigating _how_ to implement strong people and operations management practices to drive learning for all is an important area for future research.

28
Table 1: School management and student performance

|||**PISA**|||**Prova Br**|**asil**|
|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|
|**Panel A:**|**Readi**|**ng PISA**|**points**|**Portug**|**uese scor**|**es (SDs)**|
|Management Index|4.904|3.947|3.019|0.068|0.066|0.059|
||(1.193)|(1.172)|(0.980)|(0.003)|(0.003)|(0.002)|
||[0.000]|[0.000]|[0.002]|[0.000]|[0.000]|[0.000]|
|Private||11.514|2.911||||
|||(2.889)<br>[0.000]|(2.560)<br>[0.255]||||
|_R-squared_|0.24<br>410701|0.29<br>410701|0.42<br>410701|0.03<br>9891822|0.04<br>9891822|0.10<br><br>9891822|
|**Panel B:**|**Mat**|**h PISA **|**points**|**Mat**|**h scores **|**(SDs)**|
|Management Index|4.689|3.937|2.800|0.078|0.075|0.068|
||(1.267)|(1.272)|(1.060)|(0.003)|(0.003)|(0.002)|
||[0.000]|[0.001]|[0.008]|[0.000]|[0.000]|[0.000]|
|Private||11.467<br>(2.874)<br>[0.000]|2.001<br>(2.655)<br>[0.451]||||
|_R-squared_|0.31<br>410701|0.34<br>410701|0.45<br>410701|0.05<br>9891822|0.05<br>9891822|0.10<br><br>989182|
|**Panel C:**|**Science**|**PISA **|**points**||**n.a.**||
|Management Index|4.283|3.601|2.553||||
||(1.187)|(1.217)|(0.982)||||
||[0.000]|[0.003]|[0.009]||||
|Private||10.215|1.245||||
|||(2.751)|(2.377)||||
|||[0.000]|[0.600]||||
|_R-squared_|0.30|0.33|0.43||||
|# Observations<br># Schools|410701<br>15196|410701<br>15196|410701<br>15196|9890704<br>33148|9890704<br>33148|9890704<br>33148|
|Location FE*|Y|Y|Y|Y|Y|Y|
|School controls||Y|Y||Y|Y|
|Student controls|||Y|||Y|

Notes: Standard errors in parentheses, p-values in square brackets. OLS regressions for PISA were run with the student-level PISA dataset using the OECD’s repest Stata command. Standard errors clustered at the school level and use all 5 plausible values for each subject and _student final weights_ . Prova Brasil regressions run with standard OLS. Standard errors clustered at the school level and dependent variables are student learning outcomes on national tests at Grade 9 (the same exercise can be done with primary schools and tests at Grade 5). All specifications include location fixed effects (countries for PISA and states for Prova Brasil). **PISA controls** : School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, and ratio of computers connected to the web as a proxy for school resources. Student controls include gender, grade, socio-economic status and immigration status. **Prova Brasil controls** : School controls include student-teacher ratio, log of the number of students, dummies indicating the presence of an IT lab, science lab, and library as proxies for school resources. Given availability of principal characteristics, school controls also include a dummy for male principals, dummies for educational attainment, and dummies for experience as principal. Student controls include a dummy for male students, a dummy for white students, student households’ consumption index, dummies for mother educational attainment (grades 1-5, grades 6-9, secondary grades 10-12, and college). For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value. All panels use the same sample but have different subject outcome variables. Summary statistics for PISA dependent variables and controls are presented in Table B.1. 29
Table 2: PISA management index and student performance: Latin America

||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|
|---|---|---|---|---|---|---|---|---|---|
||**Readin**|**g PISA **|**points**|**Math**|**PISA p**|**oints**|**Scienc**|**e PISA **|**points**|
|Management Index|8.255|2.681|2.212|7.442|2.432|1.764|7.859|3.092|2.509|
||(1.610)|(1.252)|(1.008)|(1.576)|(1.230)|(1.039)|(1.421)|(1.144)|(0.973)|
||[0.000]|[0.032]|[0.028]|[0.000]|[0.048]|[0.089]|[0.000]|[0.006]|[0.009]|
|Private||56.807|31.921||55.695|32.589|0.000|55.428|33.077|
|||(3.301)|(2.956)||(3.713)|(3.121)||(3.735)|(3.327)|
|||[0.000]|[0.000]||[0.000]|[0.000]||[8.161]|[2.736]|
|_R-squared_|0.032|0.173|0.342|0.041|0.185|0.350|0.040|0.172|0.312|
|# Observations|78144|78144|78144|78144|78144|78144|78144|78144|78144|
|# Schools|3075|3075|3075|3075|3075|3075|3075|3075|3075|
|# Countries|8|8|8|8|8|8|8|8|8|
|Country FE|Y|Y|Y|Y|Y|Y|Y|Y|Y|
|School controls||Y|Y||Y|Y||Y|Y|
|Student controls|||Y|||Y|||Y|

Notes: Standard errors in parentheses, p-values in square brackets. OLS regressions for PISA were run with the student-level PISA dataset for 8 Latin American countries using the OECD’s repest Stata command. Standard errors clustered at the school level and use all 5 plausible values for each subject and _student final weights_ . All specifications include country fixed effects. School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, and ratio of computers connected to the web as a proxy for school resources. Student controls include gender, grade, socio-economic status and immigration status. For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value, for each variable with imputed values.

30
Table 3: People management and intermediate outcomes, public and private schools in Latin America

||(1)<br>z-teacher<br>shortage|(2)<br>z-teacher<br>motivation|(3)<br>z-teacher<br>efort|(4)<br>z-teacher<br>shortage|(5)<br>z-teacher<br>motivation|(6)<br>z-teacher<br>efort|
|---|---|---|---|---|---|---|
|Private School|-0.535<br>(0.122)<br>[0.000]|0.591<br>(0.139)<br>[0.000]|0.792<br>(0.128)<br>[0.000]||||
|People Index||||-0.062|0.238|0.074|
|||||(0.035)<br>[0.077]|(0.040)<br>[0.000]|(0.033)<br>[0.026]|
|_R-squared_|_0.152_|_0.142_|_0.154_|_0.139_|_0.169_|_0.123_|
|Observations|3035|3043|3043|3035|3043|3043|
|School controls|Y|Y|Y|Y|Y|Y|
|Country FE|Y|Y|Y|Y|Y|Y|

Notes: The first row reports the coefficients from regressions of a binary indicator (coded to 1 if the school is a private school, 0 otherwise) on the standardized index of three intermediate school outcomes: teacher shortage, teacher motivation and teacher effort. The second row reports coefficients from regressions of the standardized people management index on each of the intermediate school outcomes. The people management index is built out of the school questionnaire from PISA 2012 using the methodology from Anderson [2008]. All specifications include PISA _school final weights_ and country fixed effects. School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, ratio of computers connected to the web as a proxy for school resources, and average student socio-economic status. For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value, for each variable with imputed values dummies are added to the specifications.

31
Table 4: Operations management and intermediate outcomes, public schools in Latin America

||(1)<br>z-teacher<br>shortage|(2)<br>z-teacher<br>motivation|(3)<br>z-teacher<br>efort|(4)<br>z-household<br>efort|
|---|---|---|---|---|
|Operations Management Index|-0.080|0.238|0.076|0.160|
||(0.043)<br>[0.061]|(0.041)<br>[0.000]|(0.038)<br>[0.044]|(0.054)<br>[0.003]|
|_R-squared_|_0.0787_|_0.171_|_0.154_|_0.242_|
|Observations|2407|2414|2414|2414|
|School controls|Y|Y|Y|Y|
|Country FE|Y|Y|Y|Y|

Notes: All regressions use data from public schools only. The table reports coefficients from regressions of the standardized operations management index on each of the intermediate school outcome. The operations management index is built out of the school questionnaire from PISA 2012 using the methodology from Anderson [2008]. All specifications include PISA _school final weights_ and country fixed effects. School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, ratio of computers connected to the web as a proxy for school resources, and average student socio-economic status. For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value, for each variable with imputed values dummies are added to the specifications.

32
Figure 1: Distribution of overall management scores, PISA vs WMS <!-- Start of picture text --> BRA CAN COL<br>-4 -2 0 2 4 -4 -2 0 2 4 -4 -2 0 2 4 6<br>KS test p-value: 0.181 KS test p-value: 0.887 KS test p-value: 0.116<br>GBR GER ITA<br>-4 -2 0 2 4 -2 0 2 4 -4 -2 0 2 4<br>KS test p-value: 0.965 KS test p-value: 0.700 KS test p-value: 0.008<br>MEX SWE USA<br>-4 -2 0 2 4 -4 -2 0 2 4 -2 0 2 4<br>KS test p-value: 0.350 KS test p-value: 0.870 KS test p-value: 0.956<br>PISA 2012 index WMS index<br>.4 .4 .4<br>.2 .2 .2<br>Density Density Density<br>0 0 0<br>.4 .4 .4<br>.2 .2 .2<br>Density Density Density<br>0 0 0<br>.4 .4 .4<br>.2 .2 .2<br>Density Density Density<br>0 0 0<br><!-- End of picture text --> Note: Data for the World Management Survey index for all countries except for Mexico and Colombia can be found at www.worldmanagementsurvey.org. Distribution of overall management indices standardized within countries. Kernel density curves estimated using WMS sampling weights (calculated as the inverse probability of being interview on log of number of students, public status, and population density by state, province, or NUTS 2 region as a measure of location) for the WMS data and _school final weights_ for the PISA data. Samples include both public and private secondary schools for both datasets, with the exception of Colombia where WMS data is only available to public primary schools. Number of WMS/PISA observations are as follow (WMS/PISA): Brazil = 510/561, Canada = 129/770, Colombia = 468/268, Great Britain = 89/422, Germany = 102/158, Italy = 284/926, Mexico = 157/1327, Sweden = 85/179, United States = 263/136.

33
Figure 2: PISA-based management index by quartile x PISA student outcomes <!-- Start of picture text --> Math Reading Science<br>5.32 5.54 5.32<br>0.91 1.23 0.77<br>0.01<br>-0.25 -0.10<br>-5.91 -5.92<br>-6.71<br>Bottom quartile 2nd quartile 3rd quartile Top quartile<br>PISA-based management score (quartiles)<br>10<br>5<br>0<br>PISA score<br>-5<br>(deviations from the mean)<br>-10<br><!-- End of picture text --> Note: Number of observations: 15,196 schools from 65 countries available in PISA 2012 data. Student outcomes are estimated using five plausible values and collapsed at the school level using PISA’s senate weights. Quartiles of management are built at the country level. Test scores are presented as deviations from the global mean.

34
Figure 3: Prova Brasil-based management index x WMS management index <!-- Start of picture text -->
-2 -1 0 1 2<br>Management Index, WMS<br>1<br>.5<br>0<br>-.5<br>Management Index, Prova Brasil<br>-1<br><!-- End of picture text -->

Note: This graph is a binned scatter plot. Each circle represents the average of 5 schools. The sample contains 262 schools which have data for both Prova Brasil and WMS in 2013. Correlation of 0.19 (p-level:0.00).

35
Figure 4: Prova Brasil-based management index by quartile x student outcomes <!-- Start of picture text --> 0.40<br> Math Reading 0.36<br>0.17<br>0.15<br>-0.01<br>-0.03<br>-0.38<br>-0.40<br>Bottom 2nd 3rd Top<br>Prova Brasil-based management score (quartiles)<br>.4<br>.3<br>.2<br>.1<br>0<br>-.1<br>-.2<br>-.3<br>Prova Brasil score (standardized)<br>-.4<br><!-- End of picture text --> Note: The sample contains 33,148 public secondary schools of Prova Brasil in 2013 for which have available data. For simplicity and to compare the results to the results of PISA and the WMS, we use student learning outcomes on national tests in Portuguese and Math at Grade 9. The same exercise can be repeated with primary schools and national tests in Portuguese and Math at Grade 5.

36
Figure 5: Coefficient plot of PISA-based management index x math PISA points, by country <!-- Start of picture text --> E Europe & Central Asia East Asia & Pacific Latin America & Caribbean<br>Middle East & North Africa Southeast Asia W Europe & N America<br>60<br>40<br>20<br>0<br>Coefficient on management<br>-20<br>LIE TUR TAP PER BEL PRT CHL SRB JPN DEU QRS AUT QCN LUX LTU SGP RUS MNE NLD CHE THA GRC LVA ISL ROU BRA CZE URY COL IDN BGR ESP ITA ARG CAN MEX NZL FIN HKG MYS JOR CRI HRV KOR IRL EST SWE AUS KAZ FRA HUN SVK TUN DNK ALB VNM NOR USA ARE MAC QATPOL ISR GBR SVN<br><!-- End of picture text --> Note: PISA 2012 data. Regressions are estimated using OECD’s repest command in Stata, by country. The specification includes all five plausible values for PISA 2012 and _student final weights_ . Each marker represents the coefficient (and vertical spike represents associated 95% confidence intervals) of the management index on **math** scores.

37
Figure 6: Cumulative distribution of people management, Latin America <!-- Start of picture text -->
-4 -2 0 2 4<br>People Management Index (PISA data)<br>Private Public<br>1<br>.75<br>.5<br>.25<br>Cumulative Probability<br>0<br><!-- End of picture text -->

Notes: Cumulative distribution of the PISA-based people management index for private and public schools for 8 Latin American countries. The people management index is built out of the school questionnaire from PISA 2012 using the methodology from Anderson [2008]. Sample consists of 3075 schools: 2432 in the public sector and 637 in the private sector.

38
<!-- Start of picture text --> T 16<br>8 cH<br>s .<br>» §=1.213,1=1.311<br>4 : @52.870,1=6.222<br>2 Outside sector<br>qT!<br>a<br>| Zz 3 4 5<br>a<br>T 10<br>8 cH<br>é .<br>- 654.106,7=0.602<br>4 - §=2.827,T=6.061<br>2 Outside sector<br>qt .<br>a<br>a|2a45<br>a<br><!-- End of picture text -->
Figure 8: Coefficient plot of PISA-based people management index x intermediate outcomes, by country <!-- Start of picture text --> Teacher Shortage Index Teacher Motivation Index<br>COL CHL BRA URY PER ARG MEX CRI ARG COL URY PER BRA MEX CHL CRI<br>Teacher Effort Index<br>ARG COL URY MEX CRI CHL BRA PER<br>.6 .6<br>.4 .4<br>.2 .2<br>0 0<br>-.2 -.2<br>-.4 -.4<br>Coefficient on people management Coefficient on people management<br>.6<br>.4<br>.2<br>0<br>-.2<br>-.4<br>Coefficient on people management<br><!-- End of picture text --> Notes: Each marker represents the coefficient (and vertical spike represents the associated 95% confidence intervals) from regressions of the people management index on the intermediate school outcome indices for each country in Latin America. The people management index is built out of the school questionnaire from PISA 2012 using the methodology from Anderson [2008]. All specifications include country fixed effects, school controls and PISA _school final weights_ . School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, ratio of computers connected to the web as a proxy for school resources, and average student socio-economic status. For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value, for each variable with imputed values dummies are added to the specifications.

40
Figure 9: Coefficient plot of PISA-based operations management index x intermediate outcomes, by country <!-- Start of picture text --> Teacher Shortage Index Teacher Motivation Index<br>COL CHL ARG PER BRA MEX URY CRI ARG COL BRA URY PER MEX CRI CHL<br>Teacher Effort Index Household Effort Index<br>ARG COL MEX BRA PER CRI URY CHL ARG COL BRA CHL MEX URY PER CRI<br>1 1<br>.5 .5<br>0 0<br>-.5 -.5<br>Coefficient on operations management Coefficient on operations management<br>1 1<br>.5 .5<br>0 0<br>-.5 -.5<br>Coefficient on operations management Coefficient on operations management<br><!-- End of picture text --> Notes: Each marker represents the coefficient (and vertical spike represents the associated 95% confidence intervals) from regressions of the operations management index on the intermediate school outcome indices for each country in Latin America. The operations management index is built out of the school questionnaire from PISA 2012 using the methodology from Anderson [2008]. All specifications include country fixed effects, school controls and PISA _school final weights_ . School controls include school location, student-teacher ratio, log of the number of students, share of government funding relative to total school funding, ratio of computers connected to the web as a proxy for school resources, and average student socio-economic status. For control variables, missing variables are replaced with a value of -99 and we include an indicator variable with a value of 1 for each imputed value, for each variable with imputed values dummies are added to the specifications.

41