metadata: authors: 0: author: Dollar,David R.
1: author: Levin,Victoria
count: World
docna: 0: docna: Sowing and reaping: institutional quality
            and project outcomes in developing countries
docty: Policy Research Working Paper
subtopic: Social Policy,Legal Products,Judicial System Reform,Common Property Resource Development,Educational Sciences,Legal Reform,Legislation,Regulatory Regimes,Transport Services
teratopic: Health, Nutrition and Population
abstracts: cdata!: Much of the academic debate on the
            effectiveness of foreign aid is centered on the relationship
            between aid and growth. Different aid-growth studies find
            conflicting results: aid promotes growth everywhere; aid has
            a zero or negative impact on growth everywhere; or the
            effect of aid on growth depends on recipient-specific
            characteristics, such as the quality of institutions and
            policies. Although these studies fuel an interesting debate,
            cross-sectional macroeconomic studies cannot be the last
            word on the topic of aid effectiveness. In this paper,
            Dollar and Levin introduce microeconomic evidence on factors
            conducive to the success of aid-funded projects in
            developing countries. The authors use the success rate of
            World Bank-financed projects in the 1990s, as determined by
            the Operations Evaluation Department, as their dependent
            variable. Using instrumental variables estimation, the
            authors find that existence of high-quality institutions in
            a recipient country raises the probability that aid will be
            used effectively. There is also some evidence that geography
            matters, but location in Sub-Saharan Africa is a more robust
            indicator of lower project success rate than tropical
            climate. The authors proceed to disaggregate the success
            rate of World Bank projects by lending instrument type and
            by investment sector, finding that different institutions
            are more important for different types of projects. The
            finding of a strong relationship between institutional
            quality and project success serves to provide further
            support to the hypothesis that aid effectiveness is
            conditional on institutions and policies of the recipient country.
repnme: repnme: Sowing and reaping: institutional quality
            and project outcomes in developing countries
display_title: Sowing and reaping: institutional
            quality and project outcomes in developing countries
listing_relative_url: /research/2005/02/5642288/sowing-reaping-institutional-quality-project-outcomes-developing-countries
topicv3: Justice m1326106 840,Personal Property Rights m1326571 1907,Transport m1326661 135,Education m1327048 470,Health, Nutrition and Population m1327755 621
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Health, Nutrition and Population
no_of_pages: 25
author_keys: 0: author_key: 93908
1: author_key: 687981
model_extractions: # **_Sowing and Reaping:_**

# **_Institutional Quality and Project Outcomes in Developing Countries_**

## **_By David Dollar and Victoria Levin_**

Much of the academic debate on the effectiveness of foreign aid is centered on the relationship between aid and growth. Different aid-growth studies find conflicting results: aid promotes growth everywhere; aid has a zero or negative impact on growth everywhere; or the effect of aid on growth depends on recipient-specific characteristics, such as the quality of institutions and policies. Although these studies fuel an interesting debate, cross-sectional macroeconomic studies cannot be the last word on the topic of aid effectiveness. In this paper, Dollar and Levin introduce microeconomic evidence on factors conducive to the success of aid-funded projects in developing countries. The authors use the success rate of World Bank-financed projects in the 1990s, as determined by the Operations Evaluation Department, as their dependent variable. Using instrumental variables estimation, the authors find that existence of high-quality institutions in a recipient country raises the probability that aid will be used effectively. There is also some evidence that geography matters, but location in Sub-Saharan Africa is a more robust indicator of lower project success rate than tropical climate. The authors proceed to disaggregate the success rate of World Bank projects by lending instrument type and by investment sector, finding that different institutions are more important for different types of projects. The finding of a strong relationship between institutional quality and project success serves to provide further support to the hypothesis that aid effectiveness is conditional on institutions and policies of the recipient country.

World Bank Policy Research Working Paper 3524, February 2005 _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org._
# **_Sowing and Reaping: Institutional Quality and Project Outcomes in Developing Countries_**

# **_1. Introduction_**

In recent years, the international donor community as well as academic researchers have turned their attention to the effectiveness of development assistance. Although studies considering the impact of aid in developing countries were conducted as early as the 1940s and 1950s, a critical mass of literature on aid effectiveness has emerged only in the past decade or so.

Presently, one aspect of aid-related economic literature, the relationship between foreign assistance and economic growth in recipient countries, is the topic of intense debate among academics. Representing the whole spectrum of model specifications and estimation methods, the studies conducted so far support one of three quite different conclusions about the effect of aid on growth.<sup>1</sup> First, there are studies that find an unambiguously positive effect of aid on growth, regardless of any recipient characteristic (Hansen and Tarp 2000, 2001; Dayton-Johnson and Hoddinott 2003; Lensink and Morrissey 1999; Clemens et al. 2004). A contrasting conclusion is reached by Mosley et al. (1987), Boone (1994), and Easterly et al. (2003), who argue that aid, on average, has a zero or negative effect on the economic growth of its recipients, again irrespective of any recipient-specific trait. Finally, still other research on the same topic yields more conditional results, with the sign and size of the effect of aid on growth depending on certain features or characteristics of recipient countries, such as the quality of their institutions and policies or vulnerability to shocks. This third approach is supported by

> 1 For extended reviews of aid-growth literature, see Clemens et al. (2004), Mosley (1980), and Hansen and Tarp (2000).

1
the analysis in World Bank (1998), Burnside and Dollar (2000, 2004), Collier and Dollar (2002), Collier and Dehn (2001), and Guillamont and Chauvet (2001).

The academic discussion above is truly vigorous and useful, but while producing interesting results and raising fascinating questions, cross-sectional macroeconomic studies cannot be the last word on the topic of aid effectiveness. Small changes in specification and country coverage lead to qualitative changes in results in all of these studies, reflecting the fact that there is not enough variation across countries and too much collinearity among determinants of growth. This paper attempts to go beyond the existing rhetoric and introduce microeconomic evidence on factors conducive to project success in developing countries. If aid works the same everywhere, then one would expect to find clear evidence of this in the data on the outcomes of projects funded by development assistance. The finding of a strong relationship between institutional quality and project outcomes, on the other hand, would serve to provide further support to the validity of the third hypothesis of aid effectiveness, with the effect of aid conditional on institutions and policies of the recipient country.

In focusing on project-level evidence, we follow studies performed by World Bank researchers, such as Isham and Kaufmann (1999), Isham et al. (1997), and Kaufmann and Wang (1995). Instead of growth, these three studies use project-level indicators – economic rate of return and project success / failure, provided by World Bank’s Operations and Evaluations department (OED) – as the dependent variable.

Putting different recipient country-specific institutional and policy indicators on the righthand side of their models, the three studies find that the ERRs are higher and probability of project failure lower in countries with better policies and institutions. In the present paper, we extend this analysis and introduce other possible explanations of the rate of project success across countries.

2
Another strand of recent macroeconomic literature has concentrated on the deep determinants of well-being and growth in developing countries. The two most frequently invoked of these underlying influential factors are geographical characteristics and institutional features. Jared Diamond (1997) and Jeffrey Sachs (Gallup, Sachs, and Mellinger 1999; Sachs 2001) are the most vocal proponents of the theory that climate, access to coastline, disease burden, and other geographical aspects determine the developmental path of different countries. On the other hand, Dollar and Kraay (2002), Rodrik et al. (2002), Acemoglu et al. (2001), Engermann and Sokoloff (1997), as well as Knack and Keefer (1995) emphasize the vital importance of stable and effective institutions and policies, or the “social infrastructure” (Hall and Jones 1999), in laying the foundation for successful development. Rodrik et al. (2002) have compared the performance of the two theories (as well as the third one, concerning trade) in the same regression model, concluding that “institutions rule.” In the present paper, we also take a first step in comparing the power of geographical and institutional factors to determine project outcomes in developing countries.

The structure of the paper is as follows. In the next section we introduce the overall success rate of World Bank-financed projects as determined by OED as a measure of aid effectiveness. This success rate varies substantially across countries. We try to explain success rate by measures of institutional quality (a property rights/rule of law measure and Freedom House democracy measure). It turns out that both institutional quality measures are highly correlated with project success rate, property rights/rule of law especially so. Using instrumental variables estimation we provide evidence that this is a causal relationship, with high-quality institutions raising the probability that aid will be used effectively. There is also some evidence that geography matters: but an indicator variable for Sub-Saharan Africa is more robust than an indicator for whether the country 3
is located in the tropics. We also find some modest evidence of diminishing returns to aid: in countries receiving large volumes of aid it is harder to have successful projects.

The third section of the paper disaggregates project success rate in different ways. First, policy-based adjustment loans are separated from investment loans. An interesting result here is that democratic political institutions facilitate successful policy-based loans, whereas property rights/rule of law is more important for investment loans. Given the well-established relationship between property rights and growth, this latter finding makes sense: it is difficult to have a high-return public investment if the institutional framework is not conducive to economic growth. This may appear self-evident with “hard” investments such as transport infrastructure. But it is interesting that there is a similarly strong relationship between property rights and “soft” investments in education and health. It is difficult to have successful projects addressing these important social needs in a weak institutional environment.

The fourth section concludes by making a link between these project-level results and the larger macroeconomic literature on aid effectiveness.

# **_2. Overall Project Success Rate and Institutional Quality_**

We argued in the previous section that cross-country growth regressions alone are unlikely to settle the debate about aid effectiveness and institutional quality because different data-sets, specifications, and methodologies yield different results. We take the view that the cross-country correlations are informative, but need to be complemented by other kinds of information. Here we are going to draw on the extensive database of World Bank-financed projects carried out in nearly 100 different developing countries during the 1990s and examine the factors that influence the success or failure of aidfinanced projects.

4
The World Bank has an independent Operations Evaluation Department (OED) that reports directly to the shareholders (not the management) and that makes an ex post assessment of each project financed by the Bank. OED makes a number of different evaluations; we use its most fundamental “thumbs up, thumbs down” measure of whether a project was satisfactory in meeting its stated objective. We take as our universe all of the countries in which the Bank carried out at least three projects in the 1990s (taking the view that if there were only one or two projects in a country then we do not have a very reliable estimate of the expected success rate of donor-financed projects). For each country, we thus calculate the proportion of World Bank projects, conducted between 1990 and 1999, that were determined to have been successful by the OED. In practice, this success rate varies considerably: in the largest borrowing country, China, 90% of projects succeeded; contrast this with Pakistan’s 67% success rate or Nigeria’s 47%.

We consider a number of hypotheses about why this success rate varies across countries. First, in line with the discussion in the introduction, we consider that the overall quality of the recipient country’s institutions and policies may affect the likelihood of project success. We use several different measures of institutions/policies. From ICRG we take the Rule of Law Index. This variable "reflects the degree to which the citizens of a country are willing to accept the established institutions to make and implement laws and adjudicate disputes" (Knack and Keefer 1995). Higher scores indicate: "sound political institutions, a strong court system, and provisions for an orderly succession of power." Lower scores indicate: "a tradition of depending on physical force or illegal means to settle claims." We take this as a key economic institution that indicates whether agents are secure in making transactions. It has been found in other work to be an important determinant of growth and to be correlated with economic policies such as macroeconomic stability and openness to trade.

5
We also consider a widely used measure of political institutions. Since 1972, Freedom House has published an annual assessment of the state of freedom in all countries (and select territories), now known as “Freedom in the World.” Individual countries are evaluated based on a checklist of questions on political rights and civil liberties that are derived in large measure from the Universal Declaration of Human Rights. Each country is assigned a rating for political rights and a rating for civil liberties based on a scale of 1 to 7, with 1 representing the highest degree of freedom and 7, the lowest level of freedom. The combined average of each country’s political rights and civil liberties ratings determines an overall political freedom measure on a scale of 1-3, with 1 designating “free,” 2 “partly free,” and 3 “not free.” While it is useful to try to distinguish the influence of different types of institutions, there is also a tradeoff in that country coverage for each measure is not complete. Therefore, as a robustness check, we also use a single overall index of institutional quality from Kaufman, Kraay, and Zoido-Lobaton (1999). This measure averages different institutional quality indexes and has very wide country coverage.

Aside from institutional quality, we also include on the right-hand side the log of per capita GDP in 1990 (from Heston et al. 2002) because very poor countries may have limited supporting resources to make development projects succeed. Third, we consider the effect of the overall level of aid relative to GDP on project success rate (aid flows taken from OECD DAC 2003). The influence of total aid could go either way: there could be “increasing returns” in the sense that a lot of aid may create a better environment of supporting services and resources (e.g., an education project may be more likely to meet its objectives if there are also projects in transport, rural development, etc.). On the other hand, one could as easily imagine diminishing returns or absorptive capacity constraints, so that it is hard to have a successful project if the government’s limited 6
capacity is spread over a large number of similar enterprises. Finally, one of the key competing hypotheses is that in countries with geographical constraints to development it is more difficult to have successful projects (implied in Dalgaard et al. 2004). We consider two approaches to including geography: a variable capturing the share of a country’s territory in the tropics (Gallup et al. 1999) and/or regional dummies for each of the major regions of the world.

We start with OLS regressions and with a simple specification in which the project success rate for each country in the 1990s is a function of log per capita GDP, the Rule of Law measure, the Freedom House Index, and the overall quantity of aid relative to GDP (all averaged for the 1990s).

Average project success rate in 1990si = b0 + b1 Log (per capita GDP in 1990i) + b2 Log (average ICRG rule of law in 1990si) + b3 Log (average Freedom House index in 1990si) + b4 Log (average aid to GDP in 1990si), where i is the country in which the WB projects take place.

There is a very strong positive relationship between institutional quality and project success (Table 1, column 1). Interestingly, both political freedom (5 percent level) and the property rights/rule of law measure (1 percent level) enter with high statistical significance, and the joint test on the two is even more significant. Initial income level and aid enter negatively, but neither is statistically significant.

In column 2 we introduce geography in the form of the percent of the country’s territory situated in the tropics. The coefficients on the two institutional variable barely change and remain significant at 5 percent level. Tropical location is associated with failure with a coefficient of –9.6, but the coefficient is not statistically significant.

We can use partial scatters to illustrate this relationship between project success, on the one hand, and quality of institutions or tropical location on the other. Figure 1 shows the partial scatter of success rate on Rule of Law, corresponding to the 7
specification in column 2: it very clear visually that countries that have unexpectedly good institutions for their level of income, have unexpectedly high success rates of projects. East Asian countries such as China, Malaysia, Thailand, and Vietnam are all measured to have strong rule of law for their level of income, and all are countries in which World Bank projects have high success rates. At the other end of the rule of law spectrum, countries such as Pakistan, Haiti, Jamaica, Algeria, and DR Congo all have high failure rates.

Figure 2 shows the corresponding scatter plot for the tropical location variable: there is some relationship, but noticeably less strong than for institutions. One can see visually why tropical location is not a good predictor of success/failure: Malaysia, Thailand, and Vietnam are all tropical countries. African countries such as Ghana, Ethiopia, and Burkina Faso have pretty good success rates as well.

While being in the tropics is not necessarily detrimental to project success, the visual inspection also suggests that there may be regional effects, and that omitting these could lead to an overestimation of the importance of institutions. In column 3 of Table 1 we add regional dummies to the specification (with the excluded region East Asia and Pacific). The regional indicator variable for Sub-Saharan Africa in particular is highly significant; other things equal, success rate is about 25 percentage points lower in this region than in East Asia. Note that the coefficient on rule of law barely changes and remains highly significant. The coefficient on freedom, on the other hand, declines in magnitude and becomes insignificant. In column 4 we drop the tropical variables and all of the regional dummies except Sub-Saharan Africa to provide a more parsimonious specification. The most robust result in these OLS regressions is that rule of law has a strong positive relationship to project success, and that geography does matter in that Sub-Saharan African countries have lower success rates, ceteris paribus.

8
There are a number of reasons to be concerned about the OLS estimates.

Concerning institutions, there is the possibility of reverse causation (successful projects build strong institutions), although, frankly, this seems unlikely to us. More relevant, perhaps, is the so-called “halo effect”: there is inevitably some subjective element in the OED ratings as well as in the institutional quality ratings, and it is possible that visibly successful countries are perceived to have both good institutions and successful projects, creating spurious correlation. For the aid/GDP variable, we did not get very strong results in the OLS regressions, but this may be because of an endogeneity problem: countries receiving unexpected negative shocks may both get higher volumes of aid and have unexpectedly poor project results, again creating spurious negative correlation. For all of the above potential problems, instrumental variables approach can help produce more reliable estimates. So, in the second panel of Table 1 we repeat all of the initial specifications, but now instrument for institutional quality as well as for aid/GDP, using instruments that have been used in other studies exploring the effect of institutions and aid (share of the population speaking English, share speaking a continental European language, distance from the equator, level of population, and each of the above multiplied by population, thus adding up to eight instruments).

In the IV regressions tropical location is not important, and even the regional dummies are insignificant. In Column 4 the coefficient on the indicator for Sub-Saharan Africa is –12.5, but not statistically significant. In this specification both the Freedom and Law variables are statistically significant and the coefficients are larger than in the OLS regressions. The standard deviation of the rule of law variable in this sample is about 1.0 and the standard deviation of freedom is about 0.6. So, one standard deviation higher on rule of law corresponds to 16 percentage points higher success rate, and one standard deviation in the direction of more freedom (a lower number on the index) 9
corresponds to 14 percentage points higher success rate. The coefficient on initial per capita GDP is negative and significant now, so that it is harder to support successful projects in low-income countries. The coefficient on aid/GDP is negative, and significant in two specifications, so that there is some evidence, albeit weak, of diminishing returns or absorptive capacity constraints. Note also that the test of over-identifying restrictions passes very well in all of the specifications.

These initial results are quite encouraging. But we noted that there are nearly 100 countries in which the World Bank has project experience in the 1990s, and yet data availability constrains the sample in Table 1 to 75 countries. So, as a robustness check, we repeat all of these specifications in Table 2, but replace the separate rule of law and freedom indexes with a single overall institutional quality measure (the KKZ index in 1996). While this measure is less precise, it has the advantage of being available for nearly every country, and increases our sample to 90 countries. The main point from Table 2 is that this single institutional quality indicator is always highly significant so we can be confident that the results in Table 1 are not coming from a biased sample. A standard deviation better on the KKZ index corresponds to 24 percentage points higher project success rate (based on the last specification of Table 2 (column 8)).

Thus, a review of project success rates supports the hypothesis that the recipient country’s own institutions and policies are the key determinant of aid effectiveness. Beyond that, there is some evidence of diminishing returns to aid and of a role for geography; notably, it is harder to fund successful projects in Sub-Saharan Africa.

# **_3. Success Rates and Institutional Quality at the Sector Level_**

One potential weakness of the approach taken in the previous section is that we are combining success rates for very different kinds of projects: some are policy-oriented 10
adjustment loans, while others are investment projects. The investment projects in turn cover widely different sectors, from transport investments to rural development to education and health. In this section we explore how robust our results are to disaggregation to the sector level. Disaggregation comes at a cost: in the case of health sector investments, for example, there are only 52 countries that had this type of project, so that the number of observations declines. Given that we found little evidence that endogeneity problems were biasing the qualitative conclusions, here we restrict ourselves to OLS regressions.

First we differentiate the loans between adjustment operations and investment projects (Table 3). Law and democracy are both important for the success of adjustment operations, but it is striking that the coefficient on democracy here is larger than in any other specification. In the case of investment loans, on the other hand, the coefficient on democracy is not significantly different from zero (panel 2). Law is significant at the 1% level for investment projects in two of the four specifications (it is significant at 5% level in the other two). Thus, having a good institutional environment for growth appears to be important for effective investment projects funded by aid. It remains the case that an indicator variable for Sub-Saharan Africa has more explanatory power than a measure of tropical location: for both adjustment loans and investment loans, project success is 26% less likely in Africa than in other continents, other things equal. Note that in seven out of eight specifications per capita GDP is not significant: when low-income countries succeed in creating reasonably good institutions, it is possible for the international community to provide assistance with a high degree of confidence that the results will be positive.

We turn next to a sector-by-sector review. For education projects, property rights/rule of law is the only robust determinant of project success, significant at the 1% 11
level. Economic policy loans consist mainly of adjustment loans, so the results are similar to those earlier findings: democratic institutions are particularly important for success of economic reform support. The results for projects in the health, nutrition, and population sector are similar to those for education: property rights/rule of law is the only robust determinant of success (the smaller number of observations in this sector affected the significance levels in two, less parsimonious, specifications). There is a similar finding for rural projects and for transport investments, though the law variable just misses being significant at 10% level in the latter case.

As before, we can get quite a few more observations if we use a single measure of institutional quality rather than both the law and democracy indicators. Table 4 provides a variety of specifications using the overall institutional quality measure (KKZ). The key point here is that the institutions measure is significant at the 1% level in every sector, except for rural, in which it is significant at the 5% level. In quite a few of the specifications the indicator variable for Sub-Saharan Africa is not large and not statistically significant, nor is the tropical location indicator.

# **_4. Conclusions_**

We would like to conclude by relating these results to the larger debate about aid effectiveness and to the growth-regressions literature. We have documented a stylized fact that is well known to anyone who has practical experience in the aid business: the success of aid-sponsored projects depends primarily on the quality of the institutions in the recipient country. Virtually all donor projects work in China, regardless of sector. In some Sub-Saharan African countries with weak institutions, on the other hand, a majority of efforts fail.

How do we relate this then to the macro literature? As we noted in the introduction, some researchers have found specifications in which aid appears to have the 12
same positive effect on growth in all countries, regardless of their institutions and location. On the face of it, this is an extremely hard result to accept as a robust finding. Since most of the individual investments supported by aid fail in the countries with weak institutions – and we know little or no overall growth is recorded – it is hard to believe that aid has had a positive effect there.

On the other hand, we have to point out that the micro evidence alone cannot be decisive on this question, for the reason that money in general is fungible. When donors fund a project in China, for example, in general the donors are presented with highpriority projects that the government is likely to do anyway. The result of the donor support is that the project is done differently than it would have been without donor involvement and the government has resources freed up to pursue other, less high-priority activities (or to reduce taxes below where they would be otherwise). Thus, it is possible that every single donor-financed project is deemed to be successful, but yet there may be no overall net positive effect if the marginal projects funded by the government are all very bad.

It is for this reason that we find the combination of the micro evidence and the macro evidence most compelling. Our results make it very unlikely that aid works the same everywhere. And the fact that in several different macro data-sets researchers have found that growth is correlated with the interaction of aid and a measure of institutional quality or economic policies increases our confidence that aid is playing a useful role when it is targeted to low-income countries with reasonably sound institutions and policies. In terms of aid policy, our work provides additional support to the view that aid resources have the greatest impact on development when they are channeled to poor countries with sound institutions.

13