metadata: authors: 0: author: Estache,Antonio
1: author: Gonzalez, Marianela
2: author: Trujillo,Lourdes
count: World
docna: 0: docna: Government expenditures on education,
            health, and infrastructure : a naive look at levels,
            outcomes, and efficiency
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Public Sector Administrative & Civil
            Service Reform,Educational Sciences,Energy Demand,Energy and Environment,Energy and Mining,Transport Services,Democratic Government,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Public Sector Development,Macroeconomics and Economic Growth,Poverty Reduction,Transport
abstracts: cdata!: All interested parties seem to agree
            that it is important to be able to monitor public sector
            performance at the sectoral level, but most current work
            based on multi-country databases does not lend itself to
            country-specific conclusions. This is due to a large extent
            to major data limitations both on sectoral expenditures and
            on sectoral outcomes. This paper discusses the related
            issues and shows what we can do with the current data
            inspite of the drastic limitations. The main conclusions of
            the paper are that any efforts to assess country-specific
            performances in relative terms are likely to be difficult in
            view of the data problems. A rough sense of performance
            across sectors can be estimated for groups of countries,
            allowing some modest benchmarking exercises. These estimates
            show that low-income countries generally lag significantly
            behind higher-income countries. Efficiency has improved
            during the 1990s in energy and education but has not
            improved significantly in transport.
repnme: repnme: Government expenditures on education,
            health, and infrastructure : a naive look at levels,
            outcomes, and efficiency
display_title: Government expenditures on
            education, health, and infrastructure : a naive look at
            levels, outcomes, and efficiency
listing_relative_url: /research/2007/05/7550996/government-expenditures-education-health-infrastructure-naive-look-levels-outcomes-efficiency
topicv3: Energy and Extractives m1326595 507,Transport m1326661 135,Education m1327048 470,Public Administration m1327723 723,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Public Sector Development,Macroeconomics and Economic Growth,Poverty Reduction,Transport
no_of_pages: 27
author_keys: 0: author_key: 98672
model_extractions: WPS4219

# **Government Expenditures on Education, Health, and Infrastructure: A Naive Look at Levels, Outcomes, and Efficiency**

## **Antonio Estache Marianela Gonzalez Lourdes Trujillo ***

## **Abstract**

All interested parties seem to agree that it is important to be able to monitor public-sector performance at the sectoral level, but most current work based on multicountry databases does not lend itself to country-specific conclusions. This is due to a large extent to major data limitations both on sectoral expenditures and on sectoral outcomes. This paper discusses the related issues and shows what we can do with the current data in spite of the drastic limitations. The main conclusions of the paper are that any efforts to assess country-specific performances in relative terms are likely to be difficult in view of the data problems. A rough sense of performance across sectors can be estimated for groups of countries, allowing some modest benchmarking exercises. These estimates show that low-income countries generally lag significantly behind higher-income countries although not equally so in all sectors. They also show that efficiency has improved during the 1990s in energy and education but has not improved significantly in transport.

### World Bank Policy Research Working Paper 4219, May 2007

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org._

> * Estache works for the World Bank in Washington, DC.  Gonzalez is with the Universidad Las Palmas de Gran Canaria (ULPGC), Spain. Trujillo is affiliated with ULPGC and with the Center for Regulation and Competition Policy (CRCP), City University, London. We are grateful to Tim Coelli, Ariel Fiszbein, Ana Goicoechea, Pablo Gottfret, Santiago Herrera, Kai Kayzer, Eduardo Ley, Harry Patrinos, Sergio Perelman, Pierre Pestiau, Anand Rajaram, David Saal, Philippe Vanden Eeckaut, and participants at a seminar sponsored by the U.K. Department for International Development for useful discussions and reactions at various stages of the preparation of this paper.
# **1. Introduction**

Fiscal adjustment programs always press governments to increase productivity. That pressure is generally summed up in two questions:

- _How much less could the government spend to achieve current sector-specific outcomes—_ namely _,_ targets for water- or energy-access rates, infant mortality rates, or academic test scores _?_

- _How much better could these outcomes be, given the current levels of sector-specific public expenditures?_

These two questions are probably central to any diagnostic effort to determine the quality of public sector expenditures.<sup>1</sup> Unfortunately, no country, rich or poor, seems to be able to provide a precise quantitative answer to these basic questions at the broad level of aggregation used for sectorwide assessments of needs and performance.<sup>2</sup> Until recently, accountability for the match between sector specific expenditures and the associated education, health and infrastructure service deliveries had not appeared to be a major concern for finance ministers or watchdogs of public sector performance, as long as total expenditure levels were assessed to be sustainable from a macroeconomic perspective.<sup>3</sup> This is obvious in the academic literature. Since the seminal paper by Devarajan, Swaaroop, and Zou (1996), few empirical papers have been published on the sectoral composition of public expenditure in developing countries—in spite of a growing interest among political scientists to document the impact of globalization on the behavior of governments.<sup>4</sup> Things are changing. There is indeed a growing concern for accountability, which in turn is stimulating academic and policy interest in detailed budgeting processes and hence on sectoral resource-allocation decisions.<sup>5</sup> In the last three to five years, various governments 1 See Pradhan (1996), for example.

> 2 The equivalent questions are the bread and butter of a project-level cost-benefit analysis. But even then, the quantitative assessments in most developing countries often fall short of expectations in most of the sectors covered here.

3 This lack of accountability for the level and effectiveness of service delivery also limits the ability of most private companies that are charged with delivering education, health, and infrastructure services (or any other good or service) to raise significant funds from the financial markets.

4 There is, of course, a voluminous literature on sector-specific public expenditure—particularly in heath and education and in some dimensions of infrastructure, mostly telecoms. Although less vast, the literature on the distribution of expenditures between recurrent and capital expenditures is also illuminating. Among the few papers discussing allocation across sectors, the emphasis is on the distinction between productive and social sectors, focusing on some proxy for productive expenditure ignoring the relevance of the heterogeneity of the productive sector.

5 To be fair, this is not the first attempt at change. In the United States, for instance, the Government Performance and Results Act of 1993 featured many aspects of legislation enacted over the past five years in a number of OECD countries. But the Act never really delivered much in terms of measurement or accountability.

1
within the Organisation for Economic Co-operation and Development (OECD)—particularly the Australian, British, Dutch, Finnish, and Norwegian governments—have made significant progress toward linking expenditures and outcomes as part of their efforts to increase publicsector accountability to taxpayers.<sup>6</sup> Most of these countries have produced guidelines for their civil servants with varying amounts of sectoral detail but with enough incentives to begin measuring their activities as any business unit would. The public sector is importing longtested cost-accounting rules from the private sector—at least in some part of the activities of the public sector such as regulated industries.

But progress is slow. Those leading the charge for change recognize quite explicitly (in widely available public documents) the limitations of their efforts.<sup>7</sup> Although it would be overly simplistic, one could nevertheless argue that the two barriers to the implementation of new guidelines are posed, first, by governments that have never properly measured sectoral expenditures, and, second, by governments that have never measured sectoral outcomes well either! The problem is _this_ basic, even in countries trying to produce quantitative assessments of public-sector performance.

This is not to say there are no data. Every government has a budget. In fact, most governments also have “realized” budgets, reporting _ex post_ their actual expenditures. But these budgets are usually intended only to ensure the macroeconomic sustainability of total government expenditures. They are not really designed to track down specific sectoral expenditures—or many other resource-allocation decisions for that matter, which is to say, the distribution between capital and recurrent expenditures. That is a problem for anyone trying to assess the cost-effectiveness of sectoral policies.

The problem is not, however, only with expenditure or cost data. Measuring outcomes is also a serious problem. Once more, it is not that countries do not generate data on outcomes; it is just that the data are not really that helpful to get a sense of the effectiveness of sectoral resource allocation. The coverage of sectoral outcomes is always systematically partial both in terms of indicators of performance or results (e.g., the diversity of transport, health, or education services vs. infrastructure) and in terms of the dimensions of performance (e.g., performance should not only be measured in terms of quantity but also in terms of quality and affordability). For instance, it is surprising to see that about 20 years after the socalled Water Decade was announced, no one is reporting the average quality of water delivered or the number of hours per day that water is available in given countries, or even in a representative sample of cities.

> 6 For an overview of OECD initiatives, see OECD, Country Facts Sheets, GOV/PGC (2006), April 6, JT03207081. The _Atkinson Review_ may be the best documented such initiative, with its own website <u>http: //www.statistics.gov.uk/about/data/methodology/specific/PublicSector/Atkinson/final_report.asp.</u>

> 7 These government websites are the strongest evidence available to the international community.

2
Despite the data limitations, international benchmarking of sectorwide performance based on partial expenditure and outcome indicators is increasingly popular in the literature as a way to increase accountability. Most of the time, such benchmarking takes the form of ranking and country-specific performance assessments at the operator level—utilities, hospitals, schools, and so forth. The approach is quite common in the education and health fields.<sup>8</sup> It is less common in the infrastructure sector, except perhaps for electricity, rail, and telecoms, where the availability of data has stimulated a wide range of publications.<sup>9</sup> The popularity of international benchmarking is, however, not universally supported, in particular when it is used to assess performance at the sectoral level. The critics are indeed strong.<sup>10</sup> Some have very valid conceptual reservations. But most are concerned that the international benchmarking of sector-specific performance leads to the compilation of country-level data that can be used and abused when policies are formed.

In this paper we try to generate some efficiency benchmarks—although at a higher level of aggregation than earlier efforts (which also sought to minimize the risks of misinterpretation identified by the critics).<sup>11</sup> Based on the partial information available, we offer a rough answer to the second of the two core questions posed at the outset—how much more could governments achieve at their current level of expenditure?<sup>12</sup> In the process, we document and discuss in some detail the data problems facing anyone seeking to make broadbrush comparisons across country groups over time in each sector. More specifically, our paper makes three main contributions:

- It provides an overview of sectoral public expenditures and outcomes by sector.

- It gives a sense of the current levels of expenditure and of their efficiency for broad income groups of countries—low, lower-middle, upper-middle—in both developing countries and developed countries.

- And it identifies areas in which government could improve its monitoring of public-sector performance in education, health, and infrastructure.

Our approach is somewhat mechanical. Indeed, our main concern was to be as systematic as possible in collecting and validating all the cross-country data on the level and efficiency of public expenditure in the three sectors over 10–15 years—a relatively long

> 8 This would call for a long bibliography. A good substitute is to go through the Asian Development Bank-ADB- (2006), which provides a very nice overview of both efficiency measures on health and education and of the harsh criticism of these measures.

> 9 See Coelli et al. (2003).

> 10 See Ravallion (2005) for a good and incisive methodological critique of this literature.

> 11 See ADB (2006) and Coelli et a,. (2003) for an overview of the previous research.

> 12 A precise answer is almost impossible to supply, as explained by Ravallion (2005) in his very tough essay on any effort to measure the efficiency of public expenditures.

3
period of time. Because the data on expenditure and outcomes are not generated from the same sources and because they cover different samples of countries, one of the main and most time-consuming challenges was to define expenditure and outcome indicators in a way that would allow us to maximize the number of countries and the number of years represented in the data base. We discuss this in detail later, but it is essential to keep in mind that the binding constraint on the coverage was the availability of data on public expenditure and _matching data_ on outcomes. A country is included in the data base only if those data are available. The meagerness of this basic information is not the only issue. For a number of reasons discussed later, the exercise required some assumptions, in particular when measuring efficiency. The reassuring fact is that once all the data were processed, an interesting story emerged, even if it is a somewhat naïve story in view of the limitations just described.

The paper is organized as follows. Section 2 provides an overview of the data available on expenditure and outcomes. Section 3 discusses the main comparative facts on expenditure levels, including the main information gaps. Section 4 summarizes the information on the outcomes associated with these expenditures. Section 5 presents a rough assessment of the evolution of sector-specific efficiency levels for the four main groups of countries of concern here. The main objective of this section is to get as close as possible to benchmarking cross-country performance given the scanty data. Section 5 concludes with some suggestions for further work and discusses specific information gaps the international community will have to address in its standardized international datasets.

# **2. How much data are there, and what can you do with them?**

This section discusses the information we used to conduct large-scale cross-country and cross-sectoral performance comparisons of public expenditures. We conclude that these crosscountry assessments cannot match the quality of the analysis that could be achieved from country- or sector-specific studies conducted at the level of the business unit—central or local administration, school system, hospital, or water or electricity distribution company. Why? Because the data are simply not good enough to generate much more than “big picture” comparative assessments. The country coverage varies significantly across sectors and across time. The data are, however, good enough to give one a good sense of the evolution of the sectoral allocation of resources across broad, income-based country groupings over time.

The basic data issues can be summarized as follows. In an ideal world, we would have had access to long-term and detailed public and private expenditure data on each subsector, with matching data on all the relevant outcomes as well as all the relevant institutional variables, such as the degree of decentralization, the financing options, and other characteristics of the initial conditions with respect to the outcomes in each subsector. Those data do not exist.

4
In the real world, the immensity of data problems is overwhelming. In fact, it proved to be impossible to produce a simultaneous assessment of multiple categories of public expenditures over a long period of time. First, such an assessment would require data on expenditure that would be consistent across sectors. The IMF Government Finance Statistics database (GFS) proved to be the only data source that achieved this consistency across sectors. GFS is the main data source for most empirical cross-country studies on government expenditures<sup>13</sup> ; it is the only database offering comparable data on public expenditure for all countries in the world, including developing countries.

This does not mean that the data are good. For instance, the GFS ignores a wide range of sectoral expenditures by public enterprises and other alternative public providers. It tends to have much better data on the central government than on subnational governments. For most countries, the data reported in this widely quoted publication tend to underestimate actual expenditure levels. In addition to the gaps in coverage, the GFS does not credit specific sectors for subsidies debited from general government expenditures to support sector-specific activities managed from the sectoral budget. In sum, any study relying on that data will be biased. Yet everyone uses the data, simply because they are the only set available.<sup>14</sup> Second, many of the data are collected sporadically. The time coverage of expenditures and outcomes differs for many countries; in other words, inputs are available for some years and outputs for others. There are hardly any relevant data on sector-specific institutional characteristics. We thus had to pick between (i) maximizing the number of variables used, irrespective of time-coverage compatibility across countries and across subsectors, so as to minimize the loss of sample size, and (ii) getting the cleanest possible dataset, irrespective of the impact on the sample size. We adopted a hybrid solution. We gave up on presenting a number of outcomes available for only a few countries or years in favor of maintaining a reasonable sample size while accepting that the data quality and comparability would not be free from criticism.

Third, because we were unable to generate useful country-specific data given the limited data coverage for each country, we focused on generating _enough_ country coverage in four country-income groups: low, lower-middle, upper-middle, and higher. These classifications turned out to be broad enough not to be influenced by any remaining countryspecific problems. The breadth also gives a sense of the gap that can exist among countries at different stages of development. The availability of time series on these broad groups was essential in the choice of countries and indicators because it also allowed us to track the evolution of this gap among country groups.

> 13 The OECD and European Union have their own databases, which are widely used in the empirical literature but are of limited use here because there are no comparable data for developing countries.

> 14 Note that the World Health Organization has started to generate its own expenditure series on health. It is more detailed and covers almost the whole world, but it does so only since 1998 **.**

5
With these limitations in mind, we were able to whittle the challenges down to the sector-specific issues. Relying on the IMF data for health and education is not a major problem because they explicitly identify these two categories. The problem is more serious for infrastructure, an expenditure category that does not exist in the GFS, which uses instead the categories of transport and “fuel and energy.” The sum of these two categories would be the closest to what is widely known as infrastructure but still would exclude the water and sanitation sectors as well as the telecom sector. The omission of the latter sector in this analysis is not a major problem because in most countries the associated expenditures are now broadly the private sector’s responsibility. The omission of water and sanitation is much more problematic, but it is not the only issue. Fuel and energy combines anything that would have to do with fuel exploration costs and leaves out the very significant role of utilities in the sector. The upshot is that the data allow only for a very partial view of the infrastructure sector.

The time span is 1990 to 2002, although data are not available for every year of this period for every country. The data coverage available for each infrastructure subsector that can be matched with outcome data during that period is shown in table 1. The table distinguishes between general government and central government only to show that any effort to account for all expenditure sources has an overwhelming cost in terms of country coverage. It also shows how much more the coverage would drop if we wanted to cover the same specific countries for the two subsectors. Working with the central government data is a problem because the data on outcomes are poor. These data series are not ideal, but they are the only ones available to maintain a relatively large data sample at this stage.

6
**Table 1 Maximum coverage on expenditure with a matching outcome, 1990–2002**

||Generalgo|vernment|Centralgo|vernment|
|---|---|---|---|---|
||Number of<br>observations|Number of<br>countries|Number of<br>observations|Number of<br>countries|
|Transport|50|21|606|84|
|Energy andfuel|35|21|511|71|
|Health|18|3|147|61|
|Education|9|9|142|45|
||Memoitems:||||
|Energy+ Transport = Infrastructure|24|12|75|13|
|Health + Education =Social|2|2|49|36|
|Social + infrastructure|0|0|5|5|

Source: IMF Government Finance Statistics database.

Looking at table 1, we reached the initial conclusion that we would be better off assessing the partial performance of governments in each subsector for as many of the years for which we have information.<sup>15</sup> For instance, given the outcome data we have, efforts to cover energy and transport in the same country for the same years would limit our sample size to 13 countries. For health and education, the sample size would be limited to 36 countries. Looking at sectors one by one allows us to work with at least 45 countries (education) and up to 84 countries (transport). The specific countries covered for each sector are reported in the appendix. With this approach, there is enough data to generate the “big picture” country groupings (see table 2).

**Table 2 Number of observations available on central government expenditure per sector and per country grouping for the 1990–2002 period**

|Country income<br>category|Transport|Energy and fuel|Health|Education|
|---|---|---|---|---|
|High-income|225|216|244|244|
|Low-income|179|159|142|142|
|Lower-middle-income|198|199|214|214|
|Upper-middle-income|167|162|208|208|
|Total|769|736|808|808|

The problems with the data have by now been well established. Still, we should report additional issues that emerged during our data-cleaning exercise. First, on the outcome side,

> 15 Another way to increase the sample size would be to look at blocks of years or to work with moving averages, recognizing that data are not available for every year. Another option could be to do some more sophisticated smoothing, involving Kalman filters or kernel estimators, but this would be a major project.

7
the international databases provide some of the outcome data in flows of services and some in stocks. Taking the changes in stocks (as in the case of roads) as an indication of services is, however, of limited interest because they do not necessarily reflect the services associated with the stock. Second, matching outcomes with expenditures is not as simple as it sounds. Capital expenditures are typically “oversized” in the short run because they are designed to meet long-term demand. We have no way of knowing, however, the average lead time of demand built in to the existing capital stock in any given year. There is thus a risk of overestimation of the efficiency problems simply because we are not picking up the timing issue properly. Third, the expenditure data are clearly problematic as well. To have a reasonable sample size, we need to work with central government expenditure data. We have no way of assessing correctly how much public-sector expenditure by subnational governments this assumption ignores. It should not be a major problem for infrastructure stocks because such a large portion is generally financed by the central government, not the entire government. It could be a much bigger problem for services, however, because subnational governments often take the responsibility for a fair share of those.

The relevance of the private sector’s role is also ignored by the approach followed here because nobody really monitors this role in any of the sectors.<sup>16</sup> Finally, there are very basic data issues. For many countries, we found data that did not pass some basic commonsense tests, let alone basic statistical tests, whether on expenditure or outcome. We ended up dropping many countries because of this concern about data quality. We dropped a few more after the conversion of all the monetary data into purchasing power parity (PPP) because it also created outliers. The conversion to constant prices of data on energy and fuel was particularly problematic since a large proportion of the changes in expenditure levels can be traced to prices rather than quantities.

# **3. Emerging facts on the evolution of central government expenditure**

Here we summarize the basic information on central government expenditures for the various sectors and the various country groupings. The country coverage for each subsector is shown in table 3. The distribution of countries across income groups is reasonably even, with 20–25 countries for almost each group. The sample size for low-income countries is somewhat smaller (with 16 countries), but it is still statistically significant.

> 16 WHO may have the best set of data on the role of the private sector (in health), and Calderon and Servén (2004) have the best data set, but only for investment in Latin America. There is no other quantitative crosscountry information on the relative importance of the private sector in any of the sectors covered here. There is actually very little qualitative information on this topic.

8
**Table 3 Number of countries for which data are available on central government** **<u>expenditure as percentage of GDP, 1990–2004</u>**

|Countryincome category|Transport|Energy andfuel|Health|Education|
|---|---|---|---|---|
|High-income|25|23|24|23|
|Low-income|16|15|16|16|
|Lower-middle-income|23|22|23|23|
|Upper-middle-income|22|20|23|23|
|Total|86|80|86|85|

The sectoral expenditure levels for each category are presented in figure 1. It is important to keep in mind that these levels are at the lower bounds because they do not pick up any expenses by subnational governments or public enterprises. Figure 1 shows that for all developing-country groups, education represents the largest share of the central government expenditure analyzed here, with an average share of 3.1 percent—and up to 3.6 percent for the lowest-income groups. For high-income countries, health expenditure represents the largest share at 3.9 percent of GDP for our sample of high-income countries—the average for the full sample is 2.6 percent. The transport sector represents a nonnegligible interest of the public sector in all country groups, with an average across countries of about 1.5 percent of GDP.

<!-- Start of picture text --> Figure 1:<br>Central government expenditure<br> on sectors as a share of GDP<br> (1990-2004)<br>5 Lower income<br>4<br>3<br>Lower middle<br>2<br>income<br>1<br>0 Upper middle<br>income<br>High income<br>Transport Energy HealthEducation<br>Share of GDP (%)<br><!-- End of picture text --> Figure 2 provides additional insights on the evolution of these shares over the past 15 years or so. The emerging story is quite heterogeneous across sectors and income groups. Transport has seen its share of central government expenditure decline in all country groups, except upper-middle-income countries, where it has remained fairly stable following a slow upward trend. Fuel has followed a major downward trend except in low-income countries, where rising oil costs have has taken a toll on the budget. Education and health have been relatively stable except in low-income countries, where, after a decline, they have started to recover slowly.

9
**Figure 2 Evolution of central government expenditure (share of GDP at constant 2000 prices)**

<!-- Start of picture text --> Transport Health<br>4.000 5.0000<br>3.500 4.5000<br>4.0000<br>3.000<br>3.5000 High income<br>2.500 High income 3.0000<br>2.000 Low income 2.5000 Low Income<br>Lower middle income 2.0000 Lower middle income<br>1.500 Upper middle income 1.5000 Upper middle income<br>1.000 1.0000<br>0.500 0.5000<br>0.0000<br>0.000<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004<br>time time<br>Energy and Fuel Education<br>2.5000 6.0000<br>5.0000<br>2.0000<br>High income 4.0000<br>1.5000 Low income High incomelow income<br>Lower middle income 3.0000 Low er middle income<br>1.0000 Upper middle income 2.0000 Upper middle income<br>0.5000<br>1.0000<br>0.0000 0.0000<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004<br>time time<br>e a n<br>m ean m<br>m ean m ean<br><!-- End of picture text --> It is clearly difficult to offer much more than a basic description of trends across categories. A much better analysis would consider all the major changes that have taken place in the organization of government in each country but this data is not available although we know it is relevant. Indeed, **c** lear potential explanations for any changes observed over the past 15 years are the increased involvement of the private sector and of subnational governments in all of these sectors. Among national policies that might explain the changes are the deregulation or regulation of the sector and increased (or changed) demand for subsidies directed at the poor. However, very little data on these two dimensions has been collected in a comparable way across sectors.

Supranational changes, too, would seem to be relevant in this regard; shifting priorities among bilateral and multilateral donors would probably contribute as much as shifts in national policies—some would argue that there is a strong correlation between the national and international changes. The supranational changes are particularly important for the poorest small countries, because such a large share of many of their investment programs is financed by donor loans and grants.

The main point to be made here is that any assessment of the performance of governments based on international datasets is going to be limited by the lack of documentation of these sorts of institutional changes. Measuring reforms is as important as 10
measuring expenditures and outcomes and relating the results to a benchmarking exercise that compares the relative performance of governments. Nevertheless, we tried to ignore this limitation to get a sense of what the existing data could reveal in spite of the limitations.

# **4. The emerging facts on outcomes**

To match the analysis of the basic statistics on sectoral expenditures, it may be useful to give a sense of the evolution of the outcome indicators for each of the subsectors. Table 4 offers a brief look at the specific outcome indicators with which we worked. As in the case of public expenditure, the data had great limitations. Ideally, we would have had a long list of outputs associated with each category of expenditure and a good idea of the average time lag between expenditures and expected outcomes. That kind of information is not available.

**Table 4 Outcome indicators**

|**Transport**|**Energy**|**Health**|**Education**|
|---|---|---|---|
|Length of road<br>network (km)|Energy use per PPP<br>GDP (kg oil equivalent<br>perconstantPPP$)|Infant mortality rate<br>(per 1,000 live births)|Primary completion rate<br>(percent of relevant age<br>group)|
|Aircraft<br>departures<br>(number)|Electric power<br>consumption (Kwh/per<br>capita)|Under-5 mortality rate<br>(per 1,000)|School enrollment, primary<br>(percent net)|
|||Immunization, DPT<br>(percent of children<br>ages12-23months)|School enrollment,<br>secondary (percent net)|
||||Repetition rate, primary<br>(percentof totalenrollment)|

Note: PPP = purchasing power parity.

The solution we adopted was again very pragmatic. We looked for outcome or output indicators that could be viewed as relatively useful proxies and for which data were available for the same countries and years for which we had expenditure data. The need to match country coverage with expenditures for all sectors for as many years as possible is what limited the sample sizes the most. Figure 3 provides a visual sense of the evolution of these indicators across income groups.

The figures are clearly intended to give only a very general sense of the evolution of outcomes.<sup>17</sup> In fact, this evolution and the relative position of each income group may be the only two useful contributions of these figures. Both are consistent with expectations—that is, higher-income groups do better than lower-income groups in each sector, and in general outcomes are improving over time across income groups. There are clearly problems with

> 17 An important detail to keep in mind is that some of the outcomes are defined in positive terms (more is better, e.g., completion rates) and some in negative terms (less is better, e.g., mortality).

11
some of the data series in some of the years; in addition, some of the significant increases in coverage that seem to take place in one year are hard to explain unless something is wrong with the datasets for those specific years. These problems are not damaging enough to question the emerging big picture on the evolution of outcomes per income groups.

**Figure 3 Evolution of sectoral outcome indicators by country groups, 1990–2003**

<!-- Start of picture text --> Roads, total network (km) por 1000 inhabitants Aircraft departures<br>450000.00<br>18.00 400000.00<br>16.00 350000.00<br>14.0012.00 High income 300000.00 High income<br>10.00 Low income 250000.00 Low income<br>8.00 Lower middle income200000.00 Lower middle income<br>6.004.00 Upper middle income150000.00 Upper middle income<br>2.00 100000.00<br>0.00 50000.00<br>1990199119921993199419951996199719981999200020012002 0.00<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003<br>Electric power consumption (kwh per capita) Energy use per PPP GDP (kg of oil equivalent per constant 2000 PPP $)<br>9000 0.45<br>8000 0.40<br>700060005000 High incomeLow income 0.350.300.25 High incomeLow income<br>40003000 Lower middle income 0.20 Lower middle income<br>20001000 Upper middle income 0.150.10 Upper middle income<br>0 0.05<br>0.00<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002<br>Mortality rate, infant (per 1,000 live births) Mortality rate, under-5 (per 1,000)<br>180<br>120 160<br>100 140<br>80 LIC 120 LIC<br>100<br>60 LMI 80 LMI<br>40 UMI 60 UMI<br>40<br>20<br>20<br>0 0<br>School enrollment, primary (% net)<br>Primary completion rate, total (% of relevant age group)<br>120.00<br>100.00<br>100.00<br>80.00<br>80.00<br>LIC 60.00 LIC<br>60.00 LMI LMI<br>HMI 40.00 HMI<br>40.00<br>20.00<br>20.00<br>0.00<br>0.00 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 1998 1999 2000 2001 2002<br>1990199119921993199419951996199719981999200020012002<br>1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 199019911992 19931994 19951996199719981999200020012002 2003<br><!-- End of picture text --> 12
# **5. Some rough efficiency measures**

This section provides some illustrative estimates of the efficiency levels achieved by the central government in delivering sectoral services. We report only the results that are reasonably robust. We tried many combinations of sectors. In particular, we tried to assess the multiple output responsibilities of government (education, health, and infrastructure). Unfortunately, the data problems prevented us from producing the multisector analysis we had hoped for.

We ended up focusing on four sector specific panels of data (table 5). For each panel, we estimated two models. The first is the simplest possible distance function. The second is a model corrected to account for institutional variables such as the quality of the bureaucracy and the level of corruption in the country. The models generate a measure of efficiency at the country level, but because we believe that country-specific data often are not reliable, we only report the aggregated data at the country-group level.

**Table 5 Panel used in the estimation for each sector**

|Panel|Sector|Number of<br>countries|Number of<br>observations|Number of<br>outputs/outcomes|Number of<br>inputs <sup>a</sup>|
|---|---|---|---|---|---|
|4|Transport|81|547|2|1(4)|
|5|Energy|66|481|2|1(3)|
|7|Health|60|146|2|1(1)|
|9|Education|36|103|3|1(1)|

_Note_ : Any difference with table 1 is due to the fact that table 1 is based on a matching of outputs and inputs, whereas table 5 adds matching for the availability of data on corruption and bureaucratic quality.

a. The first number is the number of expenditure categories; the second (in parentheses) is the number of control variables used in a second group of regressions (i.e., corruption, quality of bureaucracy, GDP, population, GDP/capita.) These datasets are used to estimate a distance function, which is then used to generate the efficiency estimates for each sector. The empirical application of a distance function requires the specification of an appropriate functional form. We rely on a translogarithmic functional form (hereinafter _translog_ ) because it is flexible, easy to calculate, and allows the imposition of the homogeneity condition for outputs (homogeneity of level 1 in outputs). It offers a flexible functional form providing a local second-order approximation to an unknown functional form. In other words, we made no assumptions regarding _a priori_ restrictions about production technology. Finally, we specify an output orientation in the results reported here. In other words, we try to assess the extent to which with the current level of public expenditures (the input), we can achieve more outputs.

13
Formally, the function can be expressed as follows: Where _y_ is a vector of _M_ outputs, _x_ is a vector of _K_ factors, _i_ relates to the _i_ -th country, _t_ relates to the time trend, _h_ refers to the environmental/institutional variables, Ψt is the coefficient of the environmental/institutional dummy variables _d_ , γ t is the coefficient for the time dummy _f_ , and ε it is an error term. Variables are expressed in relation to their deviation from the geometric mean; therefore, the estimated coefficients can be construed as elasticities at the sample mean.

In order to determine the frontier, _D_ o needs to be equal to 1. When that is the case, the left side of the equation will equal zero (i.e., ln 1=0). This is why we need to assume homogeneity of degree 1. This condition is imposed by normalizing the distance function with one of the outputs.<sup>18</sup> This starts from the assumption that homogeneity implies that: for any w>0. The output chosen does not influence the results (Cuesta and Orea 2002).

If in a translog distance function any output is chosen, say yM, so that w = 1/ yM, then the following expression results: yielding the final expression: The – _ln (DO)_ term can be interpreted as an error term that captures technical inefficiency.

> 18 This approach is quite common now. It has been applied recently by Coelli and Perelman, 1999, 2000 or Orea, 2002, among others.

14
The distance function estimated is stochastic. We decided to rely on an econometric estimation rather than on a nonparametric approach simply because the quality of the data was so poor and the number of outliers so great that we needed to be able to conduct a minimum number of tests to check the robustness of the results. Most of these tests are not available for the nonparametric approaches. Moreover, there is ample evidence to suggest that the nonparametric models are much more sensitive to outliers in the assessment of efficiency levels.

To estimate equation (4), the random disturbance term needs to be determined. The most common method to do so was developed by Battese and Coelli (1988) for panel data. It applies an additive term as suggested by Cuesta and Orea (2002) to account for the fact that we are estimating an output-oriented distance function. The error terms thus have the following form where _vit_ is a symmetrical error term, iid with a zero average (which represents the random variables uncontrollable by the government), and _ui_ is a one-sided negative error term (which measures the technical inefficiency of each operator that is constant over time) and is distributed independently of _vit._ Applied to the distance function, this yields This equation can be estimated by the maximum-likelihood method, which requires distributional assumptions concerning the random shock. This assumes that _vit_ follows a N (0, 2 2 σv ) distribution and _ui_ follows a │N (0, σu )│distribution (Ritter and Simar 1997). We estimated the models with and without the environmental and institutional variables, but neither made a difference to the estimated efficiency levels. In other words, our proxies for the quality of bureaucracy and corruption were not statistically significant. The most interesting result is that while the expenditure levels are significant in all our final models, the associated elasticities are all quite low (below 0.05). The trend variables, however, are statistically significant. The trend variable is surprisingly negative in transport but positive in energy and insignificant in health and education, illustrating the differences in the importance of technological change across sectors.

Table 6 presents a summary of the average efficiency levels estimated for each sector for the 1990–2002 period. It does not make much sense to compare sectors or to try to infer meaning from the efficiency estimates because the specific levels in each sector are largely 15
driven by the quality of the data and the specific measures of outputs and inputs used to estimate the model. In sum, the models are not really that useful in answering the original questions about the scope for improvement in outputs or in costs across sectors.

|**Ta**<br>|**ble 6 Average ef**<br>|**ficiency levels (19**<br>|**90–2002)**<br>||
|---|---|---|---|---|
|**Country income**<br>**categories**|**Transport**|**Energy**|**Health**|**Education**|
|High-income|0.36|0.59|n.a.|0.93|
|Low-income|0.33|0.23|n.a.|0.74|
|Lower-middle-income|0.29|0.30|n.a.|0.92|
|Upper-middle-income|0.37|0.32|n.a.|0.91|

Because we could not really come up with a reasonably good model for health, we do not report the results from the time-series analysis, although we will discuss results from a cross-section of 192 countries at the end of this section.<sup>19</sup> Within each of the other sectors, however, the supporting econometric results are robust enough, and a story can be squeezed out for each subsector. That story differs across the subsectors. A discussion of the evolution of each of these for each country group generates a fuller picture of the global evolution of government expenditure. Figure 4 provides a graphical presentation of this evolution. The main potential use of these results is to get a sense of the relative efficiency levels, even if it is sometimes useful to ignore some of the outlier years.

> 19 The estimated model has the wrong signs on some of the parameters.

16
**Figure 4: Evolution of average levels of efficiency of government expenditure, 1990–2002**

<!-- Start of picture text --> Mean Efficiency Transport Energy<br>0.70 0.80<br>0.60 0.70<br>0.50 High Mean 0.60 High Mean<br>0.50<br>0.40 Low Mean Low Mean<br>0.40<br>0.30 Lower middle Mean 0.30 Lower middle Mean<br>0.200.10 Upper middle Mean 0.200.10 Upper middle Mean<br>0.00 0.00<br>199019911992 1993199419951996199719981999 200020012002 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002<br>years years<br>Education<br>1.00<br>0.95<br>0.90 High Mean<br>0.85<br>Low Mean<br>0.80<br>0.75 Lowerrmiddle Mean<br>0.70 Upper Middle Mean<br>0.65<br>0.60<br>1990 1998 1999 2000 2001 2002<br>Years<br><!-- End of picture text --> In transport, the story is driven by what happens in the road sector. Given that support for the roads sector is one of the largest sources of public expenditure in transport, the general results should not be too far from what would have been generated using a full set of information on public expenditures. The main story emerging is that there are no statistically significant differences across income groups. The richer countries do a little bit better, but this is to be credited mostly to the much higher level of air traffic picked up in our second output for transport.

In energy, the story is very different. The performance levels differ sufficiently across income groups to imply some relevance. On the one hand, the poorer the countries, the more likely it is that the expenditure composition will be dominated by investment. On the other hand, the richer the country, the more likely it is that operation and maintenance expenditures will drive the story. Ideally, we should be able to assess the efficiency of capital expenditures (Capex) and operational expenditures (Opex) separately, as is done for private utilities under a price cap regime, but the data needed to do that do not exist at this level of aggregation. There are other problems as well. For instance, the fact that the cost of fuel is picked up in the assessment may distort the story, in particular given that the expenditure levels are expressed in constant prices. But this should not be sufficient to explain the nearly three times difference between the poorest and richest countries. In spite of the data problems, the upshot of our results is that even if there seems to be a clear scope for improvements in efficiency, the poorest countries are catching up to some degree.

For education, there are differences across income groups, but these are predictable given the indicator used and that quality considerations are not well modeled. The most 17
interesting contribution of this naïve analysis may simply be the visual confirmation that the poorest country group shows signs of catching up. The trend in efficiency is indeed increasing significantly over the 12-year period covered by the sample.

Although the panel data and models for health proved to be too unreliable to deserve much coverage here, we relied on the latest data offered by WHO (2005) to generate a comparison across income groups. We used the general theoretical production function as with the panel data, but focused on different inputs and outputs. We considered two outputs, life expectancy at births for males and for females, expressed in years. For inputs, we relied on one of the following variables: per capita government expenditure on health or total (i.e. government + private) expenditure on health. The data on output were for 2004; on inputs, for 2003. All monetary variables are expressed in U.S. dollars and PPP. The results are not significantly affected by the choice of the input variable, as seen in figure 5.

Once more the average efficiency level estimated for the sector is not that helpful, either because of its specificity to the model or the gaps in relevant information, but the relative position of the various income groups is interesting and has proven quite robust to the various models we tried (as it was with the models relying on panel data). The only problem is that we cannot track the evolution of efficiency. As in energy or education, the results are expected. The highest-income countries have the highest efficiency levels, and the lowestincome countries the worst. Somewhat surprisingly, income level is not a perfect predictor of efficiency as defined here, since the upper-middle-income countries did somewhat worse than those in the lower-middle-income group in 2004. The important gap, however, is that between the lowest income groups and the other groups.

**Figure 5 Efficiency of health expenditure in 2004**

<!-- Start of picture text --> Mean efficiency for per capita government Mean efficiency: of total per capita expenditure on<br>expenditure on health health<br>Efficiency mean 0.96<br>Efficiency Mean<br>1.00 0.94<br>0.92<br>0.95 0.9<br>0.90 0.88<br>0.86<br>0.85<br>0.84<br>0.80 0.82<br>High income Low income Lower middle Upper middle High income Low income Lower middle Upper middle<br>income income income income<br>Country Group Country group<br><!-- End of picture text --> Overall, this section tells us simply that in the poorest countries there is more scope for performance improvements in all sectors than in higher-income countries. This is not really news. Moreover, because the specific performance levels are driven by the choice of models and outputs, they should not be taken at face value. The only thing that comes out strongly enough is that the differences across income groups are significant and that in education and in energy at least, the lower-income countries are not catching up very quickly, at least for the outcome measures with which we worked.

18
# **6. Concluding comments**

It may be useful to conclude by returning to the two questions presented at the outset of this paper. Roughly, they are: How much more can the government do in each sector with its current resource allocation? And how much cheaper could today’s level of intervention be? Based on the data and analytical work reported here, the precise answer to these questions remains: we do not know. A more imprecise, but probably still safe, answer would focus on the wedge between the poorest countries and the others, arguing that there is evidence of significant gains to be achieved considering the differences in efficiency levels across income groups. With the models we used, it was in energy and education that the performance of the low-income countries fell furthest below the benchmark set by the higher-income groups.

By how much could performance improve? Once more, the honest answer is: we do not know—not much help to policy makers. In these days in which quantitative targets of any type seem to be popular—from the Millennium Development Goals to the number of rules and regulations thought to impede business—this is probably not an acceptable reply for the donor community. It may be useful to consider setting a target for efficiency improvements— making the strong assumption that the international community can manage to agree on the inputs and outputs to consider and then to collect the relevant data. A look at the numbers would suggest that a short- to medium term improvement in both sectors of 25 to 35 percent over current levels would not be an unrealistic lower-limit target, at least in terms of the outputs identified here.<sup>20</sup> At the sectoral level, the scope for improvement across income groups seems to be largest in infrastructure sectors. This is particularly clear for transport, where the use of a stock variable (instead of a services variable) is a clear problem when setting targets. Investments in these sectors are amortized in a much slower way than in education or health. Larger initial investments appear to be less efficient early in the life of a project. Because we do not know the average age of the capital stock in question, it is probably safe to stick to a lower limit for any target. This is what justifies a 25 to 35 percent improvement even when current efficiency levels appear to be only around 30 percent in energy and transport and it would seem possible to triple current outcomes.

Ultimately, these numbers are presented as a conservative lower boundary to get a global sense of where country groups are. But they can hardly be used to make countryspecific recommendations. The technical problems underlying these numbers are not minor, and a lot more work would be needed before a more robust “guestimate” could be generated from the current data. In spite of all the good research that has been published, very little has

> 20 This is clearly an arbitrary target but it offers a rough realistic order of magnitude.

19
offered quantitative evidence suitable for use in fiscal diagnostics. A number of technical directions should be explored to refine significantly the results presented here.

The simplest solution would be to test the model on a cross-section dataset made up of the latest data available—as we did for health with the WHO data. By doing this, one would obtain a useful snapshot from relatively large datasets. But it would not say much about the extent to which the poorest are catching up and at what speed. If we want to get a sense of the speed at which things are changing or could change, we need dynamic models. To meet that requirement we could use, for the four sectors, a pure output-expansion model in which the inputs would be used in a second stage to explain differences in efficiency. If a weak or negative relation appeared, we would need to test some alternatives—for example, to group a year in windows analysis (for instance, the mean of year 1 and 2 compared with the mean output of year 2 and 3). If this alternative setting were again inconclusive, it is unlikely that much more could be achieved than what has already been done here. Alternative solutions along the same line would be to look for cost drivers or to work with national accounts data to increase cross-country and intrasector consistency, since the omission biases are likely to be less dramatic with those datasets. The challenge there would be to unbundle the public- from the private-sector providers.

In sum, as evidenced by the long list of papers included in the bibliography, some things can be done better than what we presented here, at least for education and possibly for health. Unfortunately, any recommended use of these sorts of assessments for countryspecific policy purposes will have to be presented with extremely heavy words of caution. Ultimately, policy-relevant work should be conducted at the level of the business unit in each country, but this requires a lot more data than are currently available because, historically, monitoring the performance of the government at the sector level has not really been anyone’s top priority. Until the international community begins to recognize that the sectoral allocation of public expenditure matters, today’s data gaps are unlikely to disappear and the monitoring of the performance of the public sector will continue to be only moderate at the aggregate level and superficial at best at the sectoral level.

20
# **Appendix Countries included in sector samples**

Transport Energy and fuel Health Education Albania Argentina Albania Albania Argentina Australia Argentina Azerbaijan Australia Austria Azerbaijan Bangladesh Azerbaijan Azerbaijan Bangladesh Belarus Bahrain Bahrain Belarus Bolivia Belarus Bangladesh Bolivia Bulgaria Bolivia Belarus Brazil Chile Brazil Bolivia Bulgaria Costa Rica Bulgaria Brazil Burundi Croatia Burundi Bulgaria Cameroon Dominican Republic Cameroon Cameroon Chile El Salvador Canada Canada China Estonia China Costa Rica Costa Rica Georgia Costa Rica Croatia Croatia Hungary Croatia Czech Republic Czech Republic Indonesia Cyprus Denmark Dominican Jamaica Czech Republic Dominican Republic Republic Kazakhstan Denmark Egypt, Arab Rep. Egypt, Arab Rep. Latvia Dominican Republic El Salvador El Salvador Lithuania Egypt, Arab Rep. Estonia Estonia Mauritius El Salvador Finland Gambia, The Mexico Estonia France Georgia Moldova Finland Germany Hungary Mongolia France Hungary India Morocco Georgia Iceland Indonesia Panama Germany Indonesia Iran, Islamic Rep. Peru Hungary Iran, Islamic Rep. Jamaica Poland Iceland Ireland Kazakhstan Romania India Israel Latvia Seychelles Indonesia Jamaica Lebanon Slovak Republic Iran, Islamic Rep. Kazakhstan Lesotho Syrian Arab Ireland Kuwait Lithuania Republic Israel Latvia Madagascar Tajikistan Jamaica Lebanon Malaysia Tunisia Japan Lithuania Mauritius Ukraine Kazakhstan Luxembourg Mexico Uruguay Kuwait Malta Moldova Venezuela, RB Latvia Mexico Mongolia Lebanon Moldova Morocco Lesotho Morocco Nepal Lithuania Netherlands Nicaragua Madagascar Nicaragua Pakistan Malaysia Norway Panama Malta Pakistan Peru 21
Mauritania Panama Poland Mauritius Peru Romania Mexico Poland Russian Federation Moldova Romania Seychelles Mongolia Russian Federation Slovak Republic Morocco Slovak Republic South Africa Nepal Slovenia Syrian Arab Netherlands South Africa Republic Nicaragua Sweden Tajikistan Norway Switzerland Thailand Trinidad and Pakistan Syrian Arab Panama Republic Tobago Peru Thailand Tunisia Trinidad and Turkey Poland Tobago Ukraine Romania Tunisia Uruguay Russian Federation Turkey Vanuatu Seychelles Ukraine Venezuela, RB Singapore United Arab Zimbabwe Slovak Republic Emirates Slovenia United Kingdom South Africa United States Sweden Uruguay Switzerland Venezuela, RB Syrian Arab Zimbabwe Republic Tajikistan Thailand Trinidad and Tobago Tunisia Turkey Ukraine United Arab Emirates United Kingdom United States Uruguay Vanuatu Venezuela, RB Zimbabwe 22