metadata: authors: 0: author: Isham,Jonathan T.
1: author: Kaufmann,Daniel
count: World
docna: 0: docna: The forgotten rationale for policy reform :
            the productivity of investment projects
docty: Policy Research Working Paper
subtopic: Economic Theory & Research,Industrial Economics,Educational Sciences,International Trade and Trade Rules,Food Security,Economic Growth,Crime and Society
teratopic: Poverty Reduction
abstracts: cdata!: Using economic rates of return from more
            than 1,200 public and private sector projects implemented in
            61 developing countries, the authors analyze determinants of
            investment productivity. Results from Tobit estimation
            demonstrate that the degree of countrywide policy
            distortions - macroeconomic, exchange rate, trade and
            pricing - critically affects the productivity of
            investments. Countries with undistorted policies are likely
            to be unproductive investments. In countries with distorted
            policies, investments are likely to be unproductive. And
            within a country, investments become more productive when
            economic policymaking improves. The productivity of projects
            in the tradable sectors are also affected (in a nonlinear
            fashion) by the size of a country's public investment
            program. The authors discuss possible selection biases in
            this data set, present tests of robustness, and highlight
            policy implications. In particular, donor financing for
            improvements in the policy climate is likely to pay off. A
            powerful rationale for supporting structual reform is that
            it raises the productivity of both public and private investments.
repnme: repnme: The forgotten rationale for policy reform :
            the productivity of investment projects
display_title: The forgotten rationale for policy
            reform : the productivity of investment projects
listing_relative_url: /research/1995/11/696951/forgotten-rationale-policy-reform-productivity-investment-projects
topicv3: Agriculture and Food Security m1326205 435,Economic Growth m1326241 471,Trade m1326924 698,Education m1327048 470,Black Market m1439229 BLA-3388
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Poverty Reduction
origu: World Bank Office: Kiev (ECCUA)
no_of_pages: 44
author_keys: 0: author_key: 165761
1: author_key: 90114
model_extractions: PoIIcy RESEARCH WORKING PAPER 1549

# Summary findings

Using economic rates of return from more than 1,200 public and private sector projects implemented in 61 developing countries, Isham and Kaufmann analyze determinants of investment productivity.

Results from Tobit estinmation demonstrate that the degree of countrywide policy distortionsmacroecononmic, exchange rate, trade, and pricingcritically affects the productivity of investments. Countries with undistorted policies are likely to end up with highlv productive investments. In countries with distorred policies, investilments are likely _to_ be unproductive. And within a country, investments become more productive when economic policymaking improves.

The productivity of projects in the tradable sectors are also affected (in a nonlinear fashion) by the size of a country's public investment program.

Isham and Kaufmann discuss possible selection biases in this data set, present tests of robustness, and highlight policy implications. In particular, donor financing for improvements in the policy climate is likely to pay off. A powerful rationale for supporting structural reform is that it raises the productivity of both public and private investments.

- - This paper a product of the Regional Mission in Kiev, Europe and Central Asia, Country Department IV was originally prepared as a backgrotu n d paper for _World Development Report 1991._ Copies of this paper are available free from the World Bank, 1818 H Street NW, Washington, D(C 20433. Please contact Sylvia Torres, room H5-169, telephone 202-473-9012, fax 202-522-0010, Internet address dkaufman((@worldbank.org. November 1995. (34 pages)

_The Policy Research WX'orking Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development Issuies._ **_Anr_** _objet tive ol thi series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of tie authors and should be used and cited accordingly. The findings, interpretations, and conclusions are the_ **_authors_** _o n and skouli_ **_not_** _be attributed to the World Bank, its Executive Board of Directors, or any of its member countries._ Produced by the Policy Research Dissemination Center
The Forgotten Rationale for Policy Reform: The Productivity of Investment Projects Jonathan Isham and Daniel Kaufmann
# I. Introduction<sup>1</sup>

The last 35 years of developing country experience should persuade: policies do matter for economic growth. Since 1960, the most successful economies have tended to maintain undistorted domestic prices, a stable macroeconomic framework, open trade regimes, and steady investments in people (World Bank 1991; Sachs and Warner 1995). While pointing in one direction, this experience leaves plenty of room for interpretation--no **t** o mention disagreement--among policy makers and scholars over the importance of economic policies relative to exogenous variables and institutional factors (Mauro 1995, Knack and Keefer 1995), the evidence on causality, and the types of economic policies that matter most (Summers and Pritchett 1993). Theoretical and empirical analyses that specify channels through which national policies may affect long-run growth rates (e.g., Lucas 1988; Romer 1986, 1990; Barro 1990, 1991; King and Rebelo 1990; Mankiw, Romer, and Weil 1992) are often contradictory as well as econometrically problematic (Levine and Renelt 1992).

In many ways, this lively debate on the possible sources of economic growth has obscured one of the key rationales for policy reform: increasing investment productivity. Consider both publicly- and privately-financed investment projects in the developing world. Their aggregate performance, along with the performance of firms, farms, and private entrepreneurs, will over the long-term determine a country's growth rate. Insofar as project investment performance is a contributor to economic growth, the effect of the policy environment on project performance suggests the overall importance of policy reform.

For some years, there has been a limited consensus that the policy environment can affect project-level performance (World Bank 1989). Yet many questions on this linkage have remained unanswered. How important is the policy environment for project-level success? What type of policy distortions are particularly important? How does the importance of policies compare to other factors? Can certain type of projects be designed to be insulated from policy distortions? Could other mechanisms affect both projects and policies?

I Kaufmann: Chief of Resident Mission, Kiev, Ukraine; The World Bank, 1818 H Street, Washington, DC 20433; dkaufmann @worldbank.org. Isham: Research Associate; the IRIS Center at the University of Maryland, Morrill Hall, College Park, MD 20742; isham@econ.uind.edu. This research began during the preparation of _The World Development Report 1991._ The authors benefited from comments and suggestions from many World Bank staff as well as seminar participants at the National Bureau of Economic Research, MIT, and the AEA annual meetings. The authors owe special thanks to Ajay Chibber, Angus Deaton, Paul Duane, Eduardo Engel, Stanley Fischer, Albert Fishlow, Gobind Nankani, Manuel Penalver, Lant Pritchett, Sarath Rajapatirana, Dani Rodrik, Andre Schleifer, Joel Slemnrod, Kenneth Sokoloff, Larry Summers, and Vinod Thomas. In addition, we thank staff of the Operations Evaluation Departmnent of the World Bank and the evaluation unit of the International Financial Corporation for their generosity with data and advice.
In this paper, we use data on economic rates of return (ERRs) from a set of 1,276 public and private investment projects to try to answer these questions. This unique data set has a number of advantages. It has a wide coverage across countries and over time; it was calculated according to a relatively uniform methodology; and it measures productivity from an economic standpoint. Most importantly, by using a microeconomic unit of observation as the dependent variable, the problems of spurious causality and simultaneity often faced by the empirical work in the aggregate growth literature are greatly reduced. In addition, this data provides insights on how policies affect returns to investment at the micro level--insights that are lost in aggregate statistical analysis. This data set also has one major disadvantage: these projects are not a random (nor necessarily representative) sample of typical investment projects in developing countries. This problem must be addressed in the statistical analysis.

This paper is organized as follows. Section Il summarizes the project and country-wide data used in this analysis and the basic statistical results. Sections III and IV present the main econometric results, and section V addresses sample selection biases. Section VI presents a set of proposed mechanisms for the linkage between the policy enviromnent and investment productivity. Section VII concludes with a discussion on project selection in the developing world.

# II. The Data and Basic Results

# A. Project data.

From the World Bank's Operations Evaluation Department (OED) and the evaluation unit of the International Financial Corporation (IFC) , we assembled a data set of public and private sector projects in 61 developing countries, implemented from the late 1960's into the early 1990's. The data include reestimated economic rates of return (ERRs)--as well as other project specific information<sup>2</sup> --from 1,163 investment projects financed by the Bank and implemented by public agencies in developing countries and from 113 private projects financed by the IFC. The analysis in this paper includes all projects in tradeable sectors--agriculture, industry, and tourism--and non-tradeable sectors--transport, infrastructure, energy, water, and urban--for which such ERRs have been calculated and for which a minimum set of country-specific policy indices was available.<sup>3</sup>

> 2 For an analysis on the divergence between _ex-ante_ and _ex-post_ ERRs, see Pohl and Mihaljek (1 992). 3 Kaufmann and Wang (1995) examine the performance of social sector projects--which receive a binary "satisfactory/unsatisfactory' rating from OED but no ERR--as the dependent variable in a Probit specification. They find that the probability of project failure in the social sectors is positively and significantly associated with policy distortion indicators such as the fiscal deficit, the foreign exchange parallel market premnium, and the degree of price distortions.
The reestimated ERR of each project is measured via a standard cost-benefit methodology (Gittinger 1982; Squire and van der Tak 1975)4, about two-to-three years after the completion of World Bank funding. According to this methodology, the discounted stream of project costs and benefits is evaluated at shadow (or border) prices. As such, the rates of return will likely differ from the financial rates of return that a private investor would calculate.

# B. Policy perfornance data

Policy distortions indices were gathered from independent sources.<sup>5</sup> The main indices used in this analysis were:

- (a) black market premia: the average annual mark-up of the parallel market rate for foreign exchange over the official exchange rate;

- (b) fiscal deficit of the central government as a share of GDP,

- (c) index of trade restrictiveness, based upon specific policy criteria such as tariffs and non-tariff

barriers;

- (d) index of pricing distortions in tradable goods, measuring the deviation of the domestic price levels from international price equivalencies for final tradable goods; and

(e) real interest rate.

Quantifying characteristics of macroeconomic, microeconomic and trade regimes can be a delicate and inconclusive exercise (Rodrik 1994, Pritchett 1993). Separately and together, however, these five indicators do capture major policy distortions in each economy. The black market premia reflects distortions in the trade, pricing, and exchange rate regime, as well as macroeconomic instability and capital account restrictions; the fiscal deficit is an indicator of macroeconomic instability. Additional variables incorporated into the investment productivity analysis--including years of education, capital/labor ratio, terms of trade changes, and the degree of project complexity--will be discussed below.

> 4 There are some exceptions to this general use of standard methodology in project evaluation. For example, World Bank and OED staff typically value non-traded output of an agricultural project at economic prices, whereas they typically value non-traded output of a power project at the utility's regulated tariff rates.

> 5 See Appendix 2 for a detailed description of this data.
-4-

Table 1. **Economic policies and the economic rate of** return **(ERR) of projects:** single **policy** distortions

|snge**pocy**|stortons<br>||Averag|e ERR(%<br>|||
|---|---|---|---|---|---|---|
||All<br>|Public<br>||of which||Private<br>|
||projects|projects||||projects|
|._|_________|_________|agricul-<br>ture<br>__________<br>__________|industry<br>___<br>__________sectors|non-<br>tradable<br>||
|Overall averageERR|16.0|16.2|14.3|13.6|18.1|14.0|
|Policy Distortion<br>Index:|||||||
|1. Trade restrictions|||||||
|Highly restrictive|13.2|13.6|12.2|insf|14.6|9.5|
|Somewhat<br>restrictive|15.0|15.4|15.4|insf|16.0|10.7|
|Non-restrictive|19.0|19.3|14.3|insf|24.3|17.1|
|2. Exchange rate<br>overvaluation: black<br>market Premia:|||||||
|High (> 200%)|8.2|7.2|3.2|insf|11.5|insf|
|Medium (20-200%)|14.4|14.9|11.9|13.7|17.2|10.3|
|Low (< 20%)|17.7|18.0|16.6|16.6|19.3|15.2|
|3.Real interest rate|||||||
|Negative|15.0|15.4|12.7|12.7|17.9|11.0|
|Positive|17.3|17.5|17.0|17.8|17.9|15.6|
|4.Fiscal deficit|||||||
|High C 8% of GDP)|13.4|13.7|11.7|10.3|16.6|10.7|
|Medium (4-8%)|14.8|15.1|12.2|21.0|16.8|12.2|
|Low (< 4%)|17.8|18.1|18.6|14.1|18.2|14.3|
|5. Price distortion:<br>index of tradable|||||||
|High distortions<br>|15.6<br>|15.9<br>|13.1<br>|14.0<br>|18.4<br>|11.0<br>|
|Low distortions|17.5|17.5|17.0|16.5|18.1|17.2|

Notes: Average reestimated economic rate of return of public and private projects, classified by single policy distortion.

'Insf' denotes insufficient number of observations (less than 10) to make inferences.

Source: Authors' calculations.
_-5 -_

# C. Basic statistical analysis

Average ERRs, disaggregated by sector and type of single policy distortion, are presented in table 1 6 The differences between investnment efficiency in undistorted and a distorted policy environments can be very large. The (Pearson) correlation coefficients between the policy index and ERRs (not shown) are highly significant, with few exceptions.<sup>7</sup> In most cases, when classifying by a single policy distortion indicator, average ERRs of projects implemented under a distorted policy regime are at least five percentage points lower than those of projects implemented under an undistorted regime.<sup>8</sup> In addition, each of the five policy distortion indices appears to impact significantly across sectors--although to different degrees. Further, the sensitivity of public sector projects to policy distortions is at least as significant as for private sector projects.

A country that mismanages its exchange rate is also likely to exhibit macroeconomic instability as well as trade and pricing distortions: it is therefore relevant to assess the combined effect of policy distortions on ERRs. Average ERRs, disaggregated by various combinations of policy distortions, are presented in appendix table 1. Multiple policy distortions, when compared with an undistorted policy environment, can make a difference of over 10 percentage points. These large differences between investment efficiency in undistorted and distorted environments (measured by multiple indicators) suggest an independent effect of different types of distortions<sup>9</sup> --a result which is explored in the econometric specifications below.

## IV. The Effect of Economic Policy Distortions on ERRs: Econometric Results

The correlations above do not control for other factors nor for mechanisms that mnight affect both ERRs and policies.<sup>10</sup> In order to account for other possible determinants of investment productivity and to explore the relative importance of policies, a set of multivariate econometric specifications was estimated. 1<sup>1</sup>

> 6 Sector-specific disaggregation was possible for the larger public sector data set; the smaller private sector data set, mostly comprising agricultural and industrial projects, was not disaggregated.

> 7 Such as the relationship between both the real interest rate and price of tradeable goods on the one hand, and the ERR of projects in non-tradeables, on the other.

8 Note that these reestimated ERRs are not a true _ex post_ rate of retum: the stream of project benefits is only flowing for a few years by the time of the reestimated calculation. We conducted an analysis of possible measurement biases with the available subsample of seventy public projects with true _ex posi_ evaluations, undertaken five-to-eight years after project completion. The reestimated ERR and the ex post ERR were found to be very highly correlated (r =.8), yet the average _expost_ ERR--I I to 12%--was 3-to-4 percentage points below the average reestimated ERR. Since the _expost_ ERR is a better approximation of the true economic value of the project, this suggests that, on average, a project implemented in a distorted policy framework will have a true ERR lower than 10 percent.

9 The types of policy variable combinations was circumscribed to those where indices measured different types of distortions--thus, for instance, indices of trade openness and of distortions in the price of tradables are not introduced simultaneously, and neither is the fiscal deficit and the real interest rate.

10 Nor, of course, do they control for reverse causation; but it seems quite unlikely that project returns would affect policy distortions.
- 6 -

According to standard evaluation methodology at the World Bank and the IFC, any project with an ERR below - 5% is assigned a value of -5%. About 13 percent of all observations in this data set have ERRs in this range. With such censoring, Tobit estimation is required to generate consistent results.

# A. Specifications with single policy variables.

We tested ten specifications: a pair for each of the five policy variables. The first specification in each pair controls for country- and project-specific inputs that could reasonably affect project performance: the national level of education (measured by average years), terms of trade changes (to account for external shocks) and the degree of institutional complexity of the project (a dummy variable for subsectors regarded by evaluation units as more complex, including integrated rural projects).<sup>12</sup> The second specification in each pair adds two other possible determinant of project performance: the economy-wide capitalllabor ratio (expected to be negatively associated with ERRs) and the average rate of GDP growth during the three years prior to project completion (to control for overall economy-wide dynamism). Since policies may affect capital intensity and overall GDP growth of an economy, the estimated policy coefficients in these specifications indicate the direct impact of policies on ERRs, net of the indirect impact through capital intensity and GDP growth. **1**<sup>**3**</sup> The results are presented in table 2.14 They suggest that policies are critical determinants of project performance even when controlling for these other variables: relatively large changes in single policy indices are associated with statistically significant differences in ERRs of 3-to-7 percentage points. The interpretation of the coefficients on each of the indices are as follows. Lowering the black market premia from 120 percent to 20 percent increases the average ERR by over 5 percentage points **_15;_** moving from a very

**_11_** Since the hypothesis that the structural parameters of the ERR of public and private projects are the same could not be rejected, the data set was pooled.

12 See appendix 2 for a detailed description of these series.

13 When policies affect both the capital/labor ratio and GDP growth, the estimates on policy variables will tend to be overestimated in the first of each pair of specifications in table 3 and underestimated in the second. More precisely, let the set of equations for determining ERRs be: _ERR = P, + 6*X, + a*Z, +&_ _Z,=_ rP,+ u where P = policy variables, X = exogenous country- and project-specif **i** c nputs, and Z = capital/labor ratio and GDP growth. The estimate of the direct impact of policies (O) will be overestimated when Z is omitted; the direct and indirect impact of policies when Z is included is P+a*y.

14 Unless otherwise noted, all continuous independent variables in these and subsequent specifications are three-year averages, including the ERR evaluation year and the two previous years. Altemative specifications with evaluation year data do not alter the results.

15 Two econometric notes. First, the parallel rate premia variable in all specifications is linear up to a premia of 500 percent. To prevent outliers from driving the results, higher values are equated to 500 percent plus a logarithmic transformation of the difference between the real value and 500 percent. Equally robust results were estimated from altemative specifications with differen **t** ransformations of the black market premia, including: (i) any value above 200 percent equaled to 200 percent; (ii) truncating sample for values higher than 200 percent; and (iii) any value above 500
**Table 2. Econometric analysis of ERRs:** Single **policy variable Tobit specifications**

|Independent<br>variables|Parallel rat|e premia|Trade op|enness|Fiscal d|eficit|Distortions<br>pri|in tradable<br>ces|Real interes|t rate dummy|
|---|---|---|---|---|---|---|---|---|---|---|
|Specification|(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)|
|Intercept|19.7|33.0|10.8|24.9|16.9|31.8|-207.0|-158.4|15.3|26.6|
|Policy variable|-0.055<br>(6.1)***|-0.049<br>(5.1)***|2.53<br>(2.8)***|2.18<br>(2.4)**|-0.33<br>(2.7)***|-0.34<br>(2.9)***|-2.26<br> <br>(4.0)***|-1.93<br>(3.3)***|2.48<br>(2.1)**|1.30<br>(1.1)|
|Capital/labor<br>ratio (log)|-|-2.04<br>(3.0)***|-|-2.07<br>(2.7)***|-|-2.37<br>(3.4)***|-<br>|-2.29<br>(3.4)***|-|-1.88<br>(2.6)***|
|Years<br>of|-0.30|-0.18|-0.03|0.43|0.28|0.80|-0.66|-0.09|0.00|0.48|
|education|(1.0)|(0.6)|(0.1)|(1.3)|(0.9)|(2.4)**|(2.1)**|(0.3)|(0. )|(1.4)|
|Project<br>complexity|-2.74<br>(2.0)**|-3.16<br>(2 3)**|-3.50<br>(2.6)***|-3.80<br>(2.8)***|-3.43<br>(2.5)***|-3.68<br>(2.7)***|-3.54<br> <br>(2.6)***|-3.93<br>(2.9)***|-3.82<br>(2.8)***|-3.90<br> <br>(2.8)***|
|Terms of trade|0.02|0.00|0.00|-0.01|0.01|-0.02|0.1|0.08|0.04|0.01|
|improvement|(0.2)|(0.1)|(0.0)|(0.2)|(0.2)|(0.2)|(1.3)|(1.0)|(0.5)|(0. 1)|
|GDP growth|-|0.34<br>(1.6)|-|0.49<br>(2.3)**|-|0.64<br>(3.1)***|-<br>|0.29<br>(1.4)|-|0.69<br>(34)***|
|Log likelihood|-2526|-2519|-2534|-2526|-2540|-2528|-2522|-2515|-2541|-2532|
|No.<br>of<br>observations|656|656|656|656|656|656|656|656<br>_|656|656<br>L|

Notes: Dependent variable is reestimated economic rate of return (ERR) for public and private projects Numbers in parentheses are t-statistics Significance levels: *** = 99 percent; ** = 95 percent; * =90 percent. a/ Each pair of columns includes a different policy variable, as indicated.

Sources: Authors'calculations.
-8 -

restrictive trade regime (1) to a fairly open one (4) increases the average ERR by about 7 percentage points. A difference in the fiscal deficit (as a share of GDP) of eight percentage points--for example, between 2 and 10 percent of GDP--yields an ERR difference of almost 3 percentage points. A large difference in the index of distortion of tradable yields a difference in ERRs of about 3 percentage points. A dummy variable for a positive vs. negative real interest rate yields a difference of 1.3-2.4 percentage points in ERRs (significant in only the first specification of the pair).

As expected, policies distortions are not the only factors that significantly affect the productivity of projects. Across specifications, both the capital/labor intensity and the degree of complexity of the project significantly affect ERRs in the expected direction, while the terms of trade changes variable does not have a substantial impact.<sup>16</sup>

# B. Specifications with multiple policy variables.

If different economic policies have an independent contribution to investment productivity, the overall impact of policy distortions would have been underestimated in these single-policy specifications. We thus introduce a number of policy variables simultaneously into a multivariate specification; this can also suggest which policy indices dominate in their impact on ERRs.

These results are presented in table 3. With the exception of the real interest rate dummy, policy indices do have a significant independent (and possibly additive) effect. For example, in estimations including the capital/labor ratio and GDP growth as independent variables, a one hundred percentage point reduction in the black market premia coupled with a relatively substantial opening up in the trade regime (e.g., from 1 to 3 or from 2 to 4) can be translated into an improvement in ERRs of 9 percentage points, holding other factors constant. These estimated magnitudes are not altered when country fixed effects are included in the specification (column 2). The combination of fiscal deficit and trade variables (columns 3 and 4) also suggest significant and independent effects, although the implied effects of the changes in the policy parameters are not as large a combined impact on ERRs of about 5-6 percentage points for substantial policy changes is indicated by this combination. 17 16 Nor does average years of education of the labor force, measured by the _World Development Report 1991_ data (see appendix 2) or by the total years of education series developed by Barro and Lee (1993). But a sample selection bias may be at play, since Bank/lFC projects in countries with lower skill levels may tend to compensate by allocating additional World Bank staff and extemal consultants in sectoral analysis (World Bank 1995) and in project design and supervision. 17 Specifications including country fixed effects are estimated only where policy indices vary from year to year, which is the case with the black market premia, the trade openness variables, and the fiscal deficit variable but not with the distortion in the price of tradables (one observation per country). Other specifications were estimated as well to test whether year-effects or structural breaks between time periods were apparent (not presented here). Year dummies, as well as segmenting the sample according to different time periods were tried. No significant difference in the behavior of the policy variables was apparent.
- 9 -

**Table 3. Econometric analysis of ERRs: combined policy variable Tobit specifications**

|Specification|(1)|(2)|(3)|(4)|
|---|---|---|---|---|
||Black market<br>premia, trade<br>openness, and<br>interest rate|(1) and country<br>fixed effects|Fiscal deficit<br>and distortion<br>in tradables|Fiscal deficit<br>and trade<br>openness|
|Independent variables|||||
|Intercept|30.2|67.7 a/|-134.2|27.9|
|Black market premia|-0.046<br>(4 9)***|-0.038<br>(2.1)**|-|-|
|Trade openness|2.09<br>(2.3)**|2.34<br>(1.9)*||1.7<br>(1.9)*|
|Real interest rate dumrny|-0.41<br>(0.3)|-1.46<br>(0.9)|-||
|Distortion in tradables|||-1.71<br>(2.9)***|-|
|Fiscal deficit|-|-|-0.22<br>(1.8)*|-0.32<br>(2.7)***|
|Capital/labor ratio (log)<br>i=_______________|-2.09|-5.11<br>(1I6)(<br>1.6|-2.46<br>2.9)***<br>(3.6)***|-2.28<br>(3.3)***|
|Years of education|0.07<br>(0.2)|<br>-0.75<br>(0.4)|<br> <br>0.10<br>(0.3)|0.66<br>(I 9)*|
|hnstitutional complexity|-3.1<br>i2=3_ l|-2.82<br>(2.4)|-3.79<br>** (2.8)***|-3.6<br>(2.6)***|
|Terms of trade improvement|0.02<br>.2)|-0.01<br>(0.2)|0.06<br>(0.7)|-0.04<br>_(0.5)_|
|GDP growth|0.16<br>_________<br>_(0.8)|0.02<br>(0.1)|0.29<br>(1.4)|0.45<br>(2.1)**|
|Country fixed effects|No|Yes***|No|No|
|Log likelihood|-2514|-2481|-2514|-2523|
|Number of observations|656|656|656|656|

Notes: Dependent variable is reestimated economic rate of return (ERR) for public and private projects.

Numbers in parentheses are t-statistics.

Significance levels: *** = 99 percent; ** = 95 percent; ***** = 90 percent.

a/ Intercept maintained by omitting one country dummy.

Source: Authors' calculations.

# C. The relative importance of policy distortions

While the magnitude of the impact of distortions on the efficiency of investments is very large, a substantial share of the variation in ERRs cannot be attributed to pricing policy factors. Even after incorporating a number of policy variables in the analysis, much of the variability in ERRs remains unexplained: without country dummies, the adjusted R-squares in ordinary least squares specifications equivalent to these Tobit specifications do not exceed 15 percent.<sup>18</sup>

**18** The adjusted R-squared increases to 0.65 when country dummies are included.
-10-

Both the relative importance of policies and also their explanatory limits are suggested by the figures in table 4. Under a relatively good policy environment (as measured by a single policy variable), the probability of a 'flop' project--with a negative ERR--is about one-third that under a more distorted regime. As measured by two policy variables (fiscal deficit and trade openness), this probability is about one-eighth that under a more distorted regime. By contrast, the probability of a 'very successful' project--ERR greater than 20 percent--can be between one-and-a-half and twice as high under a relatively undistorted environment, as measured by single or multiple variables. Yet, even under a relatively undistorted policy regime, there is still a 20 to 30 percent probability that the project will not be 'satisfactory'.

These figures suggest that economic policies are important. They also raise an important question: why is it that a significant fraction of projects do not become highly successful--or even satisfactory-in the face of an enabling policy environment? First, not all aspects of an economy's policy framework can be captured by any set of variables. In addition, many other economy-wide factors determine project performance. -- Supportive public investments matter (as discussed in section below); so does the quality of governancel<sup>9</sup> including the legal and regulatory framework. And of course, project-specific inputs are critical, including technical analysis, project management, and--in many sectors--beneficiary participation.<sup>20</sup> Finally, failure of some projects--and by extension of some firms--is part and parcel of decision making under risk and uncertainty and thus of the dynamism of any economy. Risk-taking in undistorted economies will unavoidably also result in some failures, but in many more successes as well.<sup>21</sup>

> 19 An ongoing study (Isham, Kaufinann and Pritchett 1995) under the World Bank's review of aid effectivenesi **s** testing the significance of selected aspects of govemance on project performance. 20 See the recent set of OED Annual Reviews (e.g., World Bank 1 994a) for detailed analyses on other determinants of project performance, and Isham, Narayan, and Pritchett (1995) for an analysis of the effect of beneficiary participation on performance of rural water projects. **21** This point is lost in the growing "industry" of project evaluative reports in intemational organizations, where each failed project is often regarded as an isolated, unambiguously negative occurrence (to be avoided in the future). In these reports, no thought is given to the question of risk-taking under uncertainty and interlinked decisions. In fact, the right strategy can result in a number of successful projects for each failure, so that the "optimal" failure rate is higher than zero.
-11-

**Table 4. Policies and the probability of project success/failure**

||Probability of<br>'flop' project a/|Probability of<br>'unsatisfactory'<br>project|Probability of<br>'satisfactory'<br>project|Probability of<br>'very successful'<br>project|
|---|---|---|---|---|
||Pr(ERR < 0)|Pr(ERR < 10)|Pr(ERR > 10)|Pr(ERR > 20)|
|Policy Variable|||||
|I. Black market premia|||||
|When premia < 30%|7.3%|28.1%|71.9%|29.9%|
|When premia > 30%|18.5%|45.4%|54.6%|16.2%|
|II. Fiscal deficit|||||
|Low deficit (< 4%<br>GDP)|4.9%|24.1%|75.9%|31.6%|
|High deficit (> 4%<br>GDP)|13.3%|36.0%|64.0%|21.6%|
|III. Trade openness|||||
|Few restrictions<br>(index >=3)|4.0%|21.2%|78.8%|30.3%|
|Substantial restrictions<br>index >=3)|13.0%|36.4%|63.6%|20.9%|
|IV.<br>Combined policy<br>distortions: fiscal deficit<br>and tradeopenness|||||
|Low deficit and few<br>restrictions|1.9%|22.6%<br>I|77.4%<br>|35.9%|
|High deficit and<br>substantial restrictions<br>I|16.3%|41.6%|58.4%|17.7%|

Notes: a/ Each cell figure represents the share of 'flop' projects in all projects that were implemented under a given regime. For example, the first cell indicates that in regimes with low black market premia, 6.8 percent of implemented projects are 'flops'.

b/ Includes 'flops' as well as projects whose ERRs were positive but did not exceed 10 percent. The three columns are neither mutually exclusive nor all-inclusive. Source: Authors' calculations.

# D. Changes in policy can make a difference

This analysis has indicated that the quality of the policy framework can make a large difference for project productivity. But this does not necessarily imply that a major policy overhaul will immediately yield
-12-

a vastly improved average ERR. Given the nature of project selection and implementation--and the cost and time of restructuring investments--many benefits of policy reform may not be apparent in the very short term.

Yet the data suggest that within a few years, significant payoffs to policy improvements are possible. Table 5 illustrates that, on average, countries which move from an inappropriate to an adequate policy environment are more likely to end up with much higher ERRs than countries in which policies do not improve. Projects that began preparation when policies were distorted--premia greater than 30 percent--but completed the investment phase when the black market premia was very low, have an average ERR of 17.8 percent.

**Table** 5. The impact of changes in the policy regime on ERRs

||Black market premia|atproject completionb<br>_||
|---|---|---|---|
|Black market premia<br>before project start a/|High premia at project<br>completion<br>(> 30%)|Low premia at project<br>completion<br>(<= 30%)|All projects|
|High initial premia<br>( > 30%)|11.7<br>l|17.8<br>_|14.1|
|Low initial premia<br>(<= 30%)|13.2<br>_<br>__<br>_|17.7<br>=|17.7|
|All projects|12.3|17.7|16.4|

Notes: Average reestimated ERRs from public and private sector projects in each cell.

a/ Initial black market premia (three-year average) at the year of project appraisal. Appraisal takes place toward the end of the project preparation process, usually about a year before implementation begins.

b/ Three-year average of black market premia preceding time of project completion. Source: Authors' calculations.

By contrast, this evidence indicates that countries in which the policy framework deteriorates will experience a substantial drop in investment productivity. Projects that began preparation when policies were not distorted--premia less than 30 percent--but completed when the black market premia was higher have an average ERR of only 13.2 percent.

Of course, this relationship between the black market exchange rate premia and ERRs cannot capture the variety and complexity of policy reform measures that are required to improve investment productivity. The black market premia index is a proxy of many distortions; the exchange rate regime is one of many policy components in an economy. Macroeconomic stability, an appropriate interest rate, relatively open external and domestic trade regimes, and credibility in the reform program are equally important preconditions for an appropriate incentive structure. Nevertheless, the figures in table 5 suggest that policy makers have room for optimism when embarking on economic reforms. When policies improve, high payoffs are expected; but deterioration in the policy framework can be very costly.
-13-

To test econometrically the impact of policy improvement during project implementation, we modified the basic multivariate analysis to control for initial conditions of the black market premia (table 6). Selected specifications which control for fixed country effects confirm the statistical robustness of the relationship between policies and ERRs, controlling for initial policy conditions: economic reforms within a country seem to yield payoffs within a few years.

**Table** 6. ERRs and policy reforms: controlling for initial conditions

||Not controlling for initial policy<br>conditions|Black market premia change<br>during project implementation|
|---|---|---|
|Intercept|87.5|82.4|
|Black market premia at<br>project evaluation|-0.046<br>(2.5)**||
|Black market premia at<br>project appraisal||-0.031<br>(1.0)|
|Premia change since<br>project appraisal||-0.047<br>(2.5)**|
|Capital/labor ratio|-6.8<br>(1.8)*|-6.2<br>(1.6)|
|Education years|-1.6<br>(0.9)|-1.7<br>(1.0)|
|Project complexity|-2.8<br>(2.0)**|-2.7<br>(2.0)*|
|Terms of trade change|0.02<br>(0.2)|0.02<br>(0.2)|
|GDP growth|0.06<br>(0.2)|0.07<br>_(0.5)_|
|Country fixed effects|Yes***|Yes***|
|Log likelihood|-2368|-2369|
|No. of observations|624|624|
|Notes: Dependent variable<br>projects (with black<br>The intercept was n<br>Numbers in parenth|is reestimated economic rate of return (<br>market prenia data available at project<br>ot suppressed in these specifications; a c<br>eses are t-statistics.|ERR) for public and private<br>appraisal and evaluation).<br>ountry dummy was omitted.|

Significance levels: *** = 99 percent; ** = 95 percent; * = 90 percent. Source: Authors' calculations.

V. The Effect of the Overall Public Investments Program on Individual Proiect Productivity In the developing world, governments have been responsible for the provision of basic infrastructure services--in transport, energy, and agriculture (World Bank 1994b).<sup>22</sup> Such public investments 22 The econornic justification for such public investments are familiar. These services enjoy a substantial public good component, and their production and provision are often subject to extemalities and/or large economies of scale. Thus, the private sector will be less likely to provide them or may do so in less than optimal quantities.
-14-

may enhance the productivity of individual projects in the tradable sectors by reducing operating costs, increasing demand for their products, and diminishing downside risks; where these services are absent, the economic efficiency costs can be large.<sup>23</sup> Yet as the public sector extends itself into lower priority areas (where the public good component is nonexistent and/or the private sector can provide these services more effectively), productivity for individual investments may not be enhanced. Maintaining an appropriate balance between the shares of public and private investments in total investment is also important. Public investments in certain priority areas are complementary to the efficiency of individual investments; in other areas, they may supplant private investments.

# A. Basic statistical results

Data for agricultural and industrial projects demonstrate the importance of overall public investments for investment productivity in the tradable sectors. The productivity of private and public tradable projects increases significantly as the share of public investments in GDP grows--but only up to a point. Figure 1 depicts simple range averages from the raw data<sup>24</sup> : the average ERR for investment projects increases by about 5 percentage points as the share of overall public investment in GDP increases from 5 to almost 10 percent. However, as the share of overall public investment increases beyond the 10 percent public investmentlGDP ratio, the average ERR eventually drops.

23 See Lee and Anas (1990) for documentation of the costs of under-provision of public infrastructure services on manufacturing enterprises in Nigeria. 24 For figures I and 2, the points represent ERR averages for each segment.
-15-

**Figure 1: Share of public investment in GDP and the productivity of tradeable projects**

<!-- Start of picture text --> 22<br>2 0 - "' .. -.. . ' . '':. . . ... .. - .'- .' .... .. . . . ' ' .' .- . . . . . ." - ' . . . .- ' " ' . - . - ''': . . . ' . . , , . . .<br>16-~ ~~~~~~~~~~ : :: ~. . . -- .. . . . -. ::<br>.-s~~~~~ .' .. . .. .<br>1 2 - ~ ~ ~ : .: . .: ~ :.: ... . : .: . .. .<br>ERR 124 4 . :~~~~~. ''''- ... - .. .... . ' .0 :' . .. 0 . -. ...... i.... :.: . . .<br>: ~~~~~~~~~~. .. ,. . ': . - -,,-- . :' ,- . . ,. ' '... - . ,- ' . -- - -. --.. -. '-:. '.:-'':<br>10 ~ ~ ~ ~ ~ ~~~. ~ -r::-: :. - ::i . .. .. ..... -: :i:: 1 : . ............<br>10~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5<br><!-- End of picture text -->

**Share of public investmert in GDP**

The data plotted in figure 1 **suggests that** the relationship between overall public **investment** and the productivity of tradable projects is particularly strong for projects implemented in a relatively undistorted policy framework. The ERR of projects implemented under an undistorted environment is on average about 13 percent in countries where the share of public investment in GDP is _5_ percent or less, while the ERR exceeds 19 percent when public investment in GDP is on average 9.5 percent. But as the share of public investment exceeds 10 percent, investment productivity declines--to an average ERR of about 15 percent.<sup>25</sup> These data also demonstrate the importance of maintaining an appropriate balance between public and private investment shares (figure 2). In economies with undistorted policies, the average ERR of tradable projects increases from 14 to 20 percent as the share of public investment rises to about 40 percent. Yet increasing the share of public investments above this range substantially reduces project productivity.

25 This (average) turning point should not be interpreted, however, as a precise benchmark for policy in each country setting; they only suggest that complementary public investments do not increase the ERR of tradable projects after a certain point.
-16-

Figure 2: Share of public investment in total investment and the productivity of tradeable projects <!-- Start of picture text --> 22 - E . . . ............<br>"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. . . ... . . . ""x . .-f -t<br>f~~~~~~~~~~~~~~~. . . . . . 3 .- ;. . -. . .S ".' .E .. '-':'.-'-. .'. . . ......-- '...... .<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. i.f.., .. ...... . ... .. . . ... .. . .. .. . .... -. - . i.-.: Ei. : E. . .. .. . . . .... . . ...<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.: §-.- ... ... ." ..:: - " """"".: . . . . .",<br>1~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .... . .... ....<br>10 .., . -..' '''' . D' '.' ". ' ... .' .'" ''.... '' ...'" ......<br>ERR E:-?:::: ::::<br>1 0 - .. .i .1 P I....<br>20 22.525 27.530 32.535 37.540 42.545 47.550 52.555 57.560 62.5<br>Share df public investment in total Investment<br><!-- End of picture text -->

# B. Econometric Results.

To test the significance of these basic statistical relationships, we conducted restricted Tobit analysis--with spline functions (Green 1990)--for the public investmentlGDP ratio and the public investment/total investments ratio. The results (table 7) indicate the statistical significance of the relationships depicted in figures 1 and 2.

The overall public investmert program of a country appears to affect strongly the productivity of individual projects, especially in settings where the economic policy environment is relatively undistorted. When the policy environment is distorted, the ERR of tradable projects will be low regardless of the relative size or shares of the public investment program (colunmns 2 and 6). By contrast, in an improved economic policy environment, increasing the size of public investment up to about 9.5% of GDP has a statistically significant positive effect; but increasing the size further has a significant negative effect (columns 3 and 4). Likewise, increasing the share of public investments in total investment up to about 40%, has a statistically significant positive effect; but increasing the share further has a significant negative effect (columns 7 and 8).
-17-

**Table 7. Public investments and the ERR of tradable projects**

|||Public inv|estment/G|DP||Public inv|estment/tot|al investme|nt|
|---|---|---|---|---|---|---|---|---|---|
|||All|Low|premia||All|Low|premia|High<br>premia|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|
|Intercept|4.6|12.6|6.9|11.1|7.0|9.1|_5.5_|11.4|7.3|
|Public<br>investment a/|1.22<br>(2.7)***|0.65<br>(3.1)***|1.23<br>(2.4)**|1.12<br>(2.2)**|0.26<br>(2.2)**|0.28<br>(2.4)**|0.35<br>(2.8)***|0.31<br>(2.4)**|-0.08 c/<br>(0.9)|
|High public<br>investment b/|-1.61<br>(2.3)**|-0.76<br>(1.1)|-2.06<br>(2.4)**|-1.95<br>(2.3)**|-0.60<br>(3.5)***|-0.52<br>(3.1)***|-0.69<br>(3.5)***|-0.68<br>(3.5)***||
|Black Market<br>Premia|-|-0.059<br>(5.4)***|-<br>|-|-|-0.057<br>(5.2)***|-|-||
|Terms of trade<br>change|-|0.06<br>(0.5)|-|0.02<br>(0.2)|-|0.06<br>(0.6)|-|0.03<br>(0.2)|0.26<br>(I.1)|
|Project<br>complexity|-|-2.47<br>(1.7)*|-|-2.34<br>(1.4)|-|-2.54<br>(1.7)*|-|-2.73<br>(1.6)|-0.86<br>(0.3)|
|Years of<br>Education|-|-0.09<br>(0.2)|-|_-0.50_<br>(1.2)|-|-0.09<br>(0.2)|-|-0.66<br>_(1.5)_|1.43<br>(1.9)**|
|Log likelihood|-1607|-1588|-1255|-1253|-1601|-1584|-1252|-1249|-336|
|Number of<br>observations|422|422|321|321|422<br>I|422<br>I|321|321|101|

Notes: a/ For public investment/GDP, the segment up to 9.5% of GDP; for public investrnent/total investment, the segment up to 40% of total investment.

b/ For public investment/GDP, the segment exceeding 9.5% of GDP; for public investment/total investment, the segment exceeding 40% of total investment.

c/ This specification (in column 9) is linear, not kinked, since there were no significant breaks in the relationship between public investment and ERRs in regimes with high black market premia sample. Numbers in parentheses are t-statistics.

Significance levels: *** = 99 percent; ** = 95 percent; * = 90 percent.

Source: Authors' calculations.

These results suggest two powerful aspects of policy reform. The best public investment "balance" cannot compensate for poor macroeconomic, trade and pricing policies. In addition, undistorted policies are necessary for high productivity of projects in the tradeable sectors, but **in** themselves they may not always be sufficient: they need to be complemented by appropriate public investments. In a sense, the quality of the public investment program can be regarded as another policy variable--one that is subject to significant distortions. As such, a good policy environment requires more than correct macro-fundamentals and relative pricing: it also requires an appropriate public investment program. 26

> 26 Easterly and Rebelo (1993), using aggregate cross-country data, find that investment in transport and communication is consistently correlated with GDP growth.
-18-

# VI. Possible omitted variable and sample selection bias.

This sample of projects financed by the World Bank group may suffer from selection bias; it is neither random nor necessarily representative of public or private projects in any given country. Consider what may occur in countries where the World Bank project presence is not large relative to overall investment--and where the Bank is not a residual lender. Given its special lending role, the World Bank in such cases may attempt to 'skim-and-insulate': to identify the best possible projects and then try to insulate them from national policy and institutional deficiencies. By contrast, in countries where the Bank project presence is large, one could expect that projects financed by the World Bank Group to have about average performance. In such cases, insulating projects from policy inadequacies would be less feasible.

Thus, we might expect that the World Bank 'project presence' would be inversely related to ERRs. Further, if project presence were negatively related to the quality of the policy framework, the impact of policies on ERRs may have been overestimated in the specifications above. Omitted variable bias--in an upward direction for the estimates of policy effects--would arise if settings where the Bank can skim and-insulate also happen to have better economic policies.

# A. Econometric specifications with World Bank proiect presence.

To test for possible mis-specifications due to such omitted variable bias<sup>27</sup> , we constructed a World Bank 'project presence' variable: the Bank's accumulated project disbursements as a share of the total capital stock. Using this variable (a single observation for each country), we tested for mis-specification in two ways. First, this variable was included as an additional independent variable in the primary set of Tobit estimations (table 8, column 1). Second, the project presence variable was also used to truncate the sample for low and high values of World Bank presence. After removing the outliers, we tested whether the policy coefficients behaved differently for the remaining sample, where the Bank presence was within a more 'normal' range. In all cases (specifications 2-4, for left-, right- and double-tail truncations, respectively), the robustness of the policy coefficients was maintained.<sup>28</sup>

> 27 Instrumental variable estimation was also considered to test for mis-specifications. In this case, however, the challenge of finding an appropriate instrument--correlated with policy variables but not with the error term of the 'second stage' regression _(ERR =p* i8 + a*Xi + a * Z_ + g)--seems insurmountable. Possible candidates, including variables on economic conditions and govemance, for example, must be ruled out because of the likelihood of endogeneity. 28 In specifications including the other main policy variables utilized in our analysis, the results--not reported here--are also very similar.
-19-

Table 8. Selection bias test using World Bank project presence

||Bank presence as<br>independent<br>variable|Truncating data if<br>Bank presence<br><=0.05|Truncating data if<br>Bank presence<br>>0.12|Truncating data if<br>Bank presence<br><=0.05, > 0.12|
|---|---|---|---|---|
|Specification|(1)|(2)|(3)|(4)|
|Intercept|47.5|34.2|34.6|37.3|
|Parallel<br>rate<br>premia|-0.045<br>(4.8)***|-0.053<br>(3.9)***|-0.049<br>(6.1)***|_-0.056_<br>(4.3)***|
|World Bank<br>project presence|-80.55<br>(3 9)***|-|||
|Capital/labor<br>ratio (log)|-3.20<br>(4.3)***|-2.34<br>(2.5)**|-2.18<br>(3.1)***|-2.65<br>(2.6)**|
|Years<br>of<br>education|0.26<br>(0.8)|0.33<br>(0.9)|0.15<br>(0.5)|0.30<br>(0.8)|
|Project<br>complexity|-3.35<br>(2.5)**|-4.09<br>(2.3)**|-2.71<br>(1.9)*|-3.42<br>(1.8)*|
|Terms of trade|0.04|-0.02|-0.01|0.01|
|improvement|(0.5)|(0.2)|(0. 1)|(0.1)|
|GDP growth|0.25<br>(1.2)|0.30<br>(I.0)|0.30<br>(1.4)|0.23<br>(0.8)|
|Log likelihood|-2512|-1495|-2432|-1409|
|No.<br>of<br>observations|656|395|631|370|

___ Notes: Dependent variable is reestimated economic rate of return (ERR) for public and private projects Numbers in parentheses are t-statistics Significance levels: *** = 99 percent; * = 95 percent; * = 90 percent. Source: Authors' calculations.

# B. Econometric specifications with truncated samples.

We carried out a second statistical technique to test for possible biases arising from the Bank's project selection process. Most projects in this data set, in addition to receiving reestimated ERRs, received expected rates of return (AERR) before project implementation. Projects with high AERRs indicate investments where World Bank and borrower country staff anticipated, to some degree, a 'skimming' within the country; projects with low AERR's, by contrast, indicate an anticipated role as lender of last resort.

We truncated the project sample to exclude outlier observations: those with very low AERRs or with very high AERRs. We found (table 9) that the relationship between policies and reestimated ERRs was not found to be statistically different in the truncated samples. This suggests that 'skimming' in some countries and 'lending-as-a-last-resort' in others do not bias the policy parameters.<sup>29</sup> 29 We owe this suggestion to Eduardo Engel.
-20-

Table 9. Selection bias test using appraisal economic rates of return

||Sample trunca|ionbased upon appra|isaleconomic rates of|return(AERR)|
|---|---|---|---|---|
||No truncation|Left truncation:<br>AERR < 15%|Right truncation:<br>AERR > 40%|Double truncation:<br>AERR <15% and<br>>40%|
|Intercept|33.5|35.0|27.0|27.8|
|Black market<br>premia|-0.049<br>(5.0)***|-0.056<br>(4.6)***|-0.050<br>(6.2***|-0.059<br>5.8)***|
|Capital/labor ratio|-2.09<br>(3.0)***|-2.05<br>(2.4)**|-1.29<br>(2.2)**|-1.13<br>(1.6)|
|Education years|0.18<br>_(0.5)_|0.03<br>(0.1)|0.07<br>(0.2)|-0.10<br>(0.3)|
|Project complexity|-3.20<br>(2.3)**|-3.39<br>(2.0)**|-3.62<br>(3.1)***|-4.05<br>(2.9)***|
|Terms of trade<br>change|0.01<br>(0.1)|0.01<br>(0.1)|0.02<br>(0.3)|0.05<br>_(0.5)_|
|GDP growth|0.35<br>(1.6)|0.54<br>(2.0)**|0.20<br>(I.1)|0.33<br>_<br>(16)|
|Log likelihood|-2461|-1933|-2175|-1666|
|No. of<br>observations|640a'|495|597|452|

Notes: Dependent variable is reestimated economic rate of return (ERR) for public and private projects. a! 640 projects with recorded expected rates of return.

t-statistics are in parentheses.

***: 99% confidence level

**: 95% confidence level

*: 90% confidence level Source: Authors' calculations.

VI Exploring the 'black box': the link between economic policies and investment productivity These statistical results on the importance of policy-related factors for investment productivity appear quite robust. What mechanics account for this strong association between the policy environment and project performance? This paper is a first attempt at investigating the empirical relationship between policies and project-level productivity; a definitive answer of the complex linkages within the "black box" cannot be expected here. (Appendix 3 includes three case studies which illustrate how public and private projects can be adversely affected by a distorted policy environment.) In this section, we propose four channels through which the linkage between country-wide policies and microeconomic productivity may be established<sup>30</sup> .

# A. Channels between country-wide distortions and microeconomic productivity

30 In addition, see appendix 4 for a simple model of the relationship between project rates of return and economic policy distortions.
-21-

_Ex post_ evaluations of unsuccessful projects (including the case studies in appendix 3) illustrate how a distorted policy environment can reduce project benefits by delaying and reducing project output. In each case, distorted incentives and weak public investments directly affect project output: even if the correct shadow prices had been used in the _ex ante_ evaluation of project performance, the standard methodology would not capture this likelihood of under-production under policy distortions.<sup>3 1</sup> Thus, while the standard shadow price adjustments are akin to Y-efficiency corrections--i.e., accounting for movement _along_ the production possibilities frontier (PPF)--these reductions of output are similar to X-efficiency considerations--i.e., movement _within_ the PPF.<sup>32</sup> In addition, the choice of outputs is more likely to be incorrect when significant distortions are present. For an agriculture project, for example, inappropriate signals and/or lack of effective demand would make it likely that the wrong trunk road is selected for construction; a solid policy framework, with appropriate agricultural incentives, would promote the selection of a rural feeder road that meets market demand and is economically productive. This problem is compounded in economies where administrative controls in the distribution of inputs are prevalent.

More indirect channels may also be at work. Overvalued exchange rates and unsustainable fiscal deficits lead to two inevitable consequences which are damaging to the return on investment: the 'boom-tobust' cycle during fiscal expansion and foreign-exchange rationing. The cycles lead to periodic under-utilization of capacity, reducing the measured return to capital, while rationing of foreign exchange reduces access to necessary imported inputs.<sup>3 3</sup> Further, even in settings with undistorted policies, ERRs can be very different where basic public investments are lacking, or conversely where the public sector is overextended. Certain projects, particularly agricultural and industry projects, depend on a minimum amount of public infrastructure (e.g., trunk and feeder roads, port facilities, and telecommunications). By contrast, with too much public investment--as **in** 31 We thank numerous seminar participants for exploring this point. There is extensive literature and experience on methodologies to account for deviations of key prices from their equilibrium values (exchange and interest rate, trade taxes, formal wages) which can be applied rigorously and directly to the calculation of net present values and ERRs (Little and Mirrlees 1991). Yet the statistical findings in this paper suggest that the importance of accounting for the impact of the policy framework on the stream of net costs and benefits through capacity underutilization have historically been underemphasized.

32 This hypothesis is supported by a comparison of reestimatedfinancial rates of return (FRR) of private projects under different policy regimes. For example, in our sample, the average FRR under a high fiscal deficit is 8.4 percent, compared to 14.5 percent under a low fiscal deficit. The differences in FRRs under different policy regimes are similar to those in ERRs. These results suggest that it is movernent along the quantity dimension--not adjustments of real or shadow prices-which mainly lower rates of return under a distorted policy environment. **33** We thank Dani Rodrik for this insight.
-22-

the illustrative case of Jamaican sugar processing--the private sector can be crowded out and the productivity of marginal public investments can be very low indeed.

Further research is needed to explore the complex mechanisms whereby the quality of economy-wide policies affect project performance. From this work, we offer a classification of these mechanisms, much of which get translated into capacity underutilization.<sup>34</sup> Poor economic policies adversely affect investments at three crucial stages: i) during project identification and preparation, through the wrong choice of output and scale--and of types of inputs and capital, including import and capital/labor intensities; ii) during project implementation, through access and costs of inputs and capital investments; and iii) during the project's operational life, through lower-than-anticipated demand for output as well as constrained access and higher costs of working capital and foreign exchange for inputs.

# VII. Conclusion

In this paper, we have established a statistical association between a country's policy environment and investment project performance. All types of projects--in the tradable and non-tradable sectors, with public and private financing--are adversely affected by distortions in the macroeconomic, trade, and pricing regimes. Improvements in the policy framework result in improved productivity. Performance of projects in the tradeable sectors is related to the size of public investment. While there are many other factors explaining project success or failure, the quality of overall economic policies is found to be important and statistically very robust. An difference of 10 percent points in ERRs of projects implemented in undistorted economic policy regimes is not uncommon.

Under sensible assumptions regarding incremental capital/output ratios, such differences in ERRs--if economy-wide--can be translated into a 2-to-3 percentage point increase in national income growth-year after year. Standard neoclassical theory predicts a one-time increase in GDP level when certain policies change (e.g., trade openness) but no change in the growth rate; this empirical linkage between economic policies and investment productivity suggests a mechanism through which policies affect the growth rate of economies, not merely a one-time adjustment of levels.<sup>35</sup> The significantly lower productivity of investments under distorted policy environments raises a critical strategic question: should any resources for project preparation and implementation be expended in such countries? Three different justifications are often advanced for project preparation in countries where the policy framework is inadequate at present--and is unlikely to improve in the very near future.

34 See Kaufmann and Wang 1995 for a detailed discussion of the mechanisms linking economic policies and the productivity of investments in the social sectors.

35 See Isham, Kaufmann and Pritchett (1995) for a growth accounting exercise that supports the use of the ERR as an indicator of the economy-wide rate of return.
-23-

In large countries where the Bank presence is relatively small, it can be argued that significantly better-than-average projects can be identified and implemented. Our analysis does suggests that the average ERR is likely to be higher in these countries. But it also shows that poor policies lower ERRs even in 'skimmable' countries: the _ex posl_ ERR in a similar country with good policies would be expected to be significantly higher. The advisability of lending funds to 'distorted-yet-skimmable' settings is weakened further if the overall lending program--including structural adjustment loans--can be used effectively as leverage for improvements in the policy framework (Isham and Kaufmann 1992).

Lending for poverty alleviation and for the social sectors has a clear sociaVequity rationale that transcends efficiency considerations: this could justify project lending even in distorted policy environments. It is important, however, to assess the likelihood of project success under such an approach. The benefits to the target group may be diminished by the overall policy distortions. Implementing social projects may still be justified as long as the benefit, to the poorer segment of the population can be verified. Conceptually, this is equivalent to attaching poverty/distributional weights in project appraisal (Squire and van der Tak 1975). It is then important to make explicit the kind of distributional weights the government and financing agencies are willing to attach to these programs. The results in Kaufmann and Wang (1995) are not very encouraging: the probability of success of social projects is much lower where distortions are present. Hence, the distributional weights may have to be particularly lopsided to justify social sector intervention in countries mismanaging their economies.

Finally, one can argue that investments of longer gestation--in infrastructure, for example-could be initiated under a poor policy framework. The implicit assumption is that by the time the investment is operational, the policy framework would be likely to have improved and thus the benefits would be forthcoming. There are three interrelated risks in this strategy. First, there may be a high probability that the policy framework may not be significantly improved in the future, rendering the project unproductive. Second, the choice of the particular infrastructure project is more likely to be an incorrect one when significant distortions are present. Third, the implementation of a long gestation investment project is likely to be hampered by an inadequate policy environment, particularly when the country is in fiscal crisis, and/or administrative controls in the distribution of inputs are prevalent.

Thus, the pitfalls in these justifications for lending under a distorted policy environment, coupled with the empirical findings of this paper, suggest that an improved (or at least improving) policy framework should be a critical precondition for a significant program of lending assistance for project investments. In fact, the results reported here do suggest that committing to such program of lending assistance being conditional on sound overall economic policies makes sense for lenders and recipients alike. The case for selectivity in country-choice is lent support by the strong association between economic policies and investment project performance found in our research. Yet our findings also suggest room for hope: donor financing for
-24-

improvements in the policy climate is likely to pay off. A powerful rationale for supporting structural reforms is that they raise the productivity of investments--public and private.