metadata: authors: 0: author: Anderson, J.R.
docna: 0: docna: Forecasting, uncertainty, and public project appraisal
docty: Policy Research Working Paper
subtopic: Disability,Educational Sciences,Services & Transfers to Poor,Hydrology,Economic Assistance,Inflation,Access of Poor to Social Services,Energy Demand,Energy and Environment,Energy and Mining
teratopic: Health, Nutrition and Population
abstracts: cdata!: Uncertainty in project planning and
            appraisal is still topical in the World Bank and other
            lending and development agencies although it is certainly
            not a new issue. It is appropriate to reconsider the issue
            now because more than a decade of active research on risk
            analysis has transpired without, however, the seeming
            emergence of agreed procedures and practice. In particular,
            the implications for what information price forecasters
            should provide for risky project appraisers have yet to be
            clarified. In Section 2, theoretical arguments about the
            proper role of uncertainty in appraisal are reviewed, and
            this section is closed by discussion of the various
            'practical' methods that have been proposed, in
            and outside the World Bank. Further procedures for
            quantifying uncertainty in both forecasting and appraisal
            are considered in Section 3. Section 4 presents a set of
            procedures that seem workable and retain some theoretical
            defensibility. These are illustrated through an example.
            Finally, conclusions and implications are drawn out in
            Section 5.
repnme: repnme: Forecasting, uncertainty, and public
            project appraisal
display_title: Forecasting, uncertainty, and public
            project appraisal
listing_relative_url: /research/1989/02/700563/forecasting-uncertainty-public-project-appraisal
topicv3: Water m1326522 137,Energy and Extractives m1326595 507,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Education m1327048 470,Social Assistance m1327201 1466
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Health, Nutrition and Population
origu: Policy Plan & Prog Review Dept
no_of_pages: 61
model_extractions: **Plc, Planning, and Research | Internallonal Comrnodl!y Markets**

Concemed about the most appropriate form of commodity price forecast to give project analysts, the author reviewed the literature on decisionmaking under conditions of uncertainty.

He concluded (in a 1983 report, published here in revised form) that the expected mean forecast is usually the relevant price parameter to use in analyzing public projects under conditions of uncertainty.

# He further concluded that:

and inflation. This requires forecasters of these variables to be explicit about the precision of their forecasts.

The auto jr describes a general procedure for determining approximate magnitudes of risk adjustment expressed as a proportion of expected project retum. The factors used in this approximation are (a) relative risk aversion, (b) relative size of project, (c) relative project risk, and (d) the correlation of project return with national income.

* Public project decisions should not be influenced by the expected variance around the expected mean price.

- Ideally, commodity price forecasts should be conditional forecasts - that is, conditional on forecasts of other variables, such as income

Since this report's publication in 1983, the International Commodity Markets Division has regularly published simple probability distribuiions for its minerals, metaL, and coal price forecasts. It also provides probabilities for its other price forecasts on request.

This paper is a product of the International Commodity Markets Division, International Economics Department. Copies are available free from the World Bank, 1818 H Street NW, Washington DC 20433. Please contact Audrey Kitson-Walters, room S7-053, extension 33712.

The PPR Working Paper Series disseminates the findings of work uider way in the Bank's Policy, Planning, and Research Complex. An objective of the series is to get these fmdings out quickly, even if presentations are less than fully polished. The findings, interpretations, and coniclusions in these papers do not necessarily represent official policy of the Bank.

Produced at the PPR Dissemination Center
# Contents

|Su<br>Pos|mry..c<br>. ....<br>...<br>..<br>. ...<br>, _<br>............<br>*****<br>***<br>**<br>**********<br>tscript 1988...|
|---|---|
|Sec|tion|
|1|Introduction......o<br>l...|
|2|When Uncertainty<br>Matters in Public Project Appraisal..9*ooo*****oo..2|
||2.1 Theoretical perspectives........<br>2|
||2.2Practical considerations....-. 0<br>....|
||2.3 World Bank practice.l.5.......o.........|
|3|Measuring Uncertainty in Project Components......................17|
||3.1 Uncertainty in variables external to the project<br>...............<br>17|
||3.1.1 GDP annual percent change. ....<br>18|
||3.1.2 Conditional forecasting precision...<br>l...........<br>ooeeo19|
||3.2 Uncertainty in project variables......*eo*oo#.e..<br>...<br>.....<br>22|
||3.2.1Quantitiesoo...o..o....o............<br>23<br>3.2.2Prices*e*******9.o..o....o.<br>o....<br>24|
||3.2.3 Other variables...<br>.....................<br>25|
|4|Workable Procedures for Uncertainty<br>Accounting... ....<br>*.........25|
||4.1 Pragmatic methods for computing risk adjustments<br>.......<br>*..o26<br>4.1.1 A 'rough-and-ready'<br>approach...p<br>.. 26<br>4.1.2 A stochastic simulation approacho.. ......... .*#**.29|
||4.2A case study illustration....o.<br>.o . .<br>o.<br>.<br>oo..<br>..<br>.*<br>..<br>. ..<br>.32|
|5|Conclusion......o...o... o..ooo.oo.o...<br>oo..<br>o....**.**....<br>ooe.oo..<br>36|
||5.1Implications for price forecasters..ooo.o.<br>......<br>..<br>....<br>....<br>37|
|6|References<br>...........<br>o.o.............<br>o.oo...40|
|Ap|pendix|
|1|Mean and Variance of Simple Functions of Normal Variables.e.....o..46|
|2|A Monte Carlo Study of Proportional<br>Risk Deductionso.o...6*6.....50|
# Preface

This report by Professor J.R. Anderson of the University of New England, Armidale, Australia, was first published in 1983. It stemmed from the Division's concern over the most appropriate form of commodity price forecast to provide to project analysts within the Bank. Some analysts have argued, for example, that a single point forecast is not adequate and instead suggest a probability distribution forecast. After more than two decades of interest in the subject, a large body of research into the question of decision-making under uncertainty exists. It seemed an appropriate time to try to review the literature and obtain an answer to this question from someone prominent in this field. After reviewing the extensive literature on public decision-making under uncertainty, Anderson's judgment was that the expected mean forecast is the relevant price parameter to use in public project analysis ir. most circumstances. Further, Anderson concluded that public project decisions should not be influenced by the expected variance around the expected mean price. Commodity price forecasts should ideally be conditional forecasts, i.e., conditional on forecaets of other variables such as income and inflation, and this poses demands for forecasters of these variables to be explicit about the precision of their forecasts. In exceptional cases, for example where the project is very large, formal accounting for uncertainty of the price forecast may be desirable and Anderson outlined a fairly simple technique for undertaking this kind of analysis. Anderson saw value in the provision of probabilistic information about commodity price forecasts. Such information gives the user a realistic view of the precision of the forecast and it imposes a useful discipline on the maker of the forecast.

Since the report was published in 1983, the International Commodity Markets Division has regularly published simple probability distributions for its minerals, metals and coal price forecasts. It also provides probabilities for its other price forecasts upon request.

Anderson has revised the original text in light of subsequent comment on the paper and it is published here in its revised form.

Ron Duncan, Chief International Commodity Markets Division International Economics Department
D ii. -

# Postscript 1988

In re-reading this work some five years after the first version was prepared, the general conclusions seem to have withstood the effluxion of time. Some corrections were, however, needed for the main equation for computing proportional risk deductions and for its illustrative applicatioi.s. These corrections arose from the suggestion made by Avinash K. Dixit (Economics Department, Princeton University) that, in general, the extent of variability in the economy at large should play an important role in accounting for the incremental risk associated with a project. The revised results are reported in Anderson (1989) and have been incorporated into this revised Working paper.
- iii -

# Summary

1. Uncertainty, while ubiquitous, should play only a minor role in public project appraisal. This general conclusion serves to support prevalent practice in most agencies. It must be tempered, however, under various exceptional circumstances, including particularly large (relative o a national economy) projects.

2. Other important cases where formal accounting for uncertainty may be important or desirable incluie: (a) projects with socially uninsurable risks to significant disadvantaged groups that would suffer unacceptably in the event of unfavorable uncertain events; and, possibly, (b) projects that are highly correlated with national income.

3. Nearly all projects are risky but, in the public domain, risks of 'small' and 'independent' projects will be effectively shared by the large number of members of society. Hence, while individuals may be averse to risk in their private decision making, from an aggregative perspective society is approximately neutral in its attitude towards risk.

4. Under these circumstances, the relevant criterion for public projects is the maximization of the expected value of social benefits, e.g. the mean or expected present value of net benefits. Aspects of probability distributions of uncertain components of projects (such as measures of dispersion like variance, standard deviation, coefficient of variation, range, etc.) are relevant to project appraisers only insofar as they are required to compute unbiased estimates of expected net benefit. 5. The criterion to be adopted for the exceptional cases of 'large' and/or 'dependent' projects when society, through ineffective sharing arrangements, manifests aversion to risk is intrinsically more complex. Theoretically, if consistent risky decisions are desired, society should seek to maximize the expected value of an intertemporal welfare or utility function, the concavity of which reflects the non-neutral attitude to risk or, equivalently, the diminishing marginal utility of income.

6. The theoretically proper approach seems not to have been used in the practice of public investment appraisal, presumably because of the difficulty of articulating the required function and reconciling it with more traditional social welfare functions oriented to issues in income distribution rather than risk aversion. Practical methods have followed one of two simpler techniques.

7. 'Risk analysis', or the stochastic simulation of a symbolic model of a project's performance over time, has been used to describe the riskiness of a project in the summary form of probability distributions of overall financial performance. The results are usually interpreted in an intuitive or holistic manner although, in principle, an explicit utility function could be embedded in the procedure.
- iv -

8. The alternative simplification has been to assume that the utility function is of a convenient (sometimes approximate) mathematical form with a parsimonious (albeit restrictive) parametric structure (particularly in terms of measures such as the coefficient of relative risk aversion). Such parameters are then given theoretically plausible values so that approximate certainty-equivalent returns can be computed as the guide to decision making. A risk premium (or adjustment, or deduction, or charge) is defined as the difference between the expected return and the certainty-equivalent return.

9. This general approach is exploited herein to provide a simple procedure for determining approximate magnitudes of risk adjustments expressed as a proportion of expected project return. The factors used in the approximation are: (a) relative risk aversion; (b) relative size of project; (c) relative risk of project; and (d) correlation of project return with national income.

10. Unfortunately, recognition of when to worry about uncertainty is not straightforward and, in a sense, can be judged with precision only after a formal risk analysis. Risk analysis is becoming easier and cheaper with the proliferation of microcomputers and facilitating software and, accordingly, project analysts will presumably make greater use of the approach.

11. The implication for project-related commodity forecasters of the general conclusion about the unimportance of uncertainty accounting is that they need concentrate their efforts on estimating expected prices over time. Precision information (e.g. on the standard error of a dated forecast price) will generally not be intrinsically useful in the sense that it should influence a project decision.

12. There do, however, seem to be other virtues in providing more comprehensive probabilistic information to users of forecasts. Communication of judgmental data will be improved and users will know more of the analysts' best assessment of the precision with which they are forecasting. Relatedly, users, including project monitors who may plan strategies and implement contingency arrangements, will be less surprised when eventualities differ from forecast means, as they surely must.

13. Formal statements of probabilistic structures may lead analysts to make better estimates of mean prices, particularly when skewed distributions are involved, and there is a danger that modal (or, worse 'conservative') prices might be reported as forecasts instead of means.

14. If measurements of uncertainty are to be published, users should be provided clear information about what is being measured, especially with regard to any conditionality on forecasts of other uncertain explanatory variables or assumptions.

15. Conditional forecasts, along with their relevant uncertainties or degrees of precision, will generally be the most appropriate to transmit to users. This, in turn, imposes demands on forecasters of exogenous variables that enter commodity forecasting models important to project appraisers to be explicit about the precision of their forecasts.
# 1 Introduction

Uncertainty in project planning and appraisal is still topical (it won't go awayl) in the World Bank and other lending and development agencies (see, e.g., Sarris and Adel ian 1982), although it is certainly not a new issue, given the pioneering ruethodological studies that emerged from the 1960s (Reutlinger 1970, Pouliquen 1970). It is appropriate to reconsider the issue now because more than a decade of active research or. risk analysis has transpired without, however, the seeming emergence of agreed procedures and practice. In particular, the implications for what information price forecasters should provide for risky project appraisers have yet to be clarified. A comprehensive review of decision making under risk and uncertainty is well beyond the scope of this modest study. Fortunately, several apposite reviews are available (Anderson, Dillon and Hardaker 1977, Hey 1979, Jean 1970, Keeney and Raiffa 1977), and these provide entrees to the ever-expanding horizons of literature on risk aversion, uncertainty, consumption and saving, capital budgeting (Weingartner 1963, 1966, Bromwich 1970, Van Horne 1971), information (Bradford and Kelejian 1977, Green 1981, Hilton 1981, Hess 1982) 1982) and stabilization (Newbery and Stiglitz 1981), to mention some of the key related fields. Rather, the purpose here is to explore the most cogent matters with a view to discovering methods that are simple and low-cost enough to implement in the operational environment of project appraisal.

First, in Section 2, theoretical arguments about the proper role of uncertainty in appraisal are reviewed, and this section is closed by a discussion of the various 'practical' methods that have been proposed, in and outside the World Bank. Further procedures for quantifying uncertainty in both
- 2 -

forecasting and appraisal are considered in Section 3. Section 4 presents a set of procedures that seem workable and retain some theoretical defensibility. These are illustrated through an example. Finally, conclusions and implications are drawn out in Section 5.

2 When Uncertainty Matters in Public Project Appraisal

# 2.1 Theoretical persipectives

For the past 20 years a vigorous, and probably still unfinished, debate has raged over the importance, or otherwise, of allowing for uncertainty in appraisal of public investments. Early polar positions were that public investments should be discounted at (risky) market rates, so that investment patterns are not distorted (Hirshleifer 1965, 1966, Pauly 1970) or at the riskless rate, because government can effectively pool risks into unimportance through its large and diversified portfolio of investments (Samuelson 1964, Vickrey 1964) and its multi-generation time horizon.

The argument was advanced significantly through introducing the notion of sharing of public risks by members of society, in the signal contribution of Arrow and Lind (1970). In a stylized world of (statistically) independent risky projects, they demonstrated that, when the risks are publicly borne (i.e. shared) the total cost of risk bearing is insignificant and, accordingly, goverments should ignore uncertainty in appraising public investments. Therefore the appropriate discount rate is independent of considerations of risk.

The controversy thus fueled has yet to run its course. Although the central result of Arrow and Lind (1970) has not been successfully challenged (see Gardner (1979) and Bird (1982) and the recent assault of Rustagi and Price (1983)), the setting and its relevance, and the interpretations that
- 3 -

should be made, have often been questioned. Mishan (1972) took Arrow and Lind to task for what he saw as questionable use of the Kaldor-Hicks criterion of social improvement, and argued that, when public investment is possible .n the private sector, the relevant opportunity cost of public funds is the 'full actuarial rate of return' (p. 163). Use of a (lower) riskless rate of return in public appraisals would, he contended, deny potentially larger growth.

McKean and Moore (1972) quibbled, seemingly erroneously, with how large the number of people sharing risks needs to be for the Arrow and Lind result to hold. The criticism of Nichols (1972) was more cogent. Echoing Mishan (1972), he emphasized the dependence of the opportunity cost (and thus the rate of discount) of public funds on the size and disposition of such funds. This theme was again taken up (somewhat more formally) by Sandmo (1972). In concluding (along with Hirshleifer 1965) that public-sector discount rates should always include a margin for risk corresponding to that for comparable private investments, he stressed that the difference from the Arrow and Lind conclusion centered on the different assumption made about the (non-) independence of returns from public projects. He, along subsequently with Fisher (1973), also addressed the irrelevance of the Arrow and Lind result for projects producing pure public goods, whose risks do not get spread into obscurity.

One useful clarification of the Arrow and Lind idea is that of James (1975). She noted potential inconsistencies between piecemeal and global appraisal of projects using the risk-spreading theorem unless, in appraising a group of projects as a group, the risk pooling effects (Samuelson 1964, Vickrey 1964) provide sufficient gains in risk reduction through diversification.
-4-

With Arrow and Lind (1972) unrepentant, and in spite of some refinements elaborating the nature of income taxation as a social risk-sharing mechanism (Mayshar 1977, but see also Stewart 1979 and Mayshar 1979) and more realistic specification of the fiscal system of an economy (Foldes and Rees 1977), this is essentially where the issues are becalmed in the controversy over discount rates in public projects. Meantime, however, authors with the more overtly practical purpose of providing guidance to project appraisers had been developing, largely independently it seems, procedures for g &-niing with uncertainty in appraisal. The topic was explored by Reutlinger ' : 3, pp. 523), but was taken up more comprehensively in the works now to be reviewed.

The authors of the UNIDO (1972) Guidelines (notably P. Dasgupta in this instance) kept the argument simple in defending as normal practice the use of expected net present (if appropriate social) value, E[PV], evaluated at the (riskless) social rate of discount. They did note some 'exceptional cases' (p. 111) which were resolved by introducing a concave (risk-averse) albeit arbitrarily specified utility function for national consumption: (a) an unusually large project, where benefits are a substantial fraction of national income; and (b) where national income is uncertain, and project benefit is correlated with (i.e. not independent of) national income. Both these exceptions, of course, depart from the key assumptions underlying the Arrow and Lind (1970) results. Their illustrations pointed to tt'e likely small deduction from E[PVJ occasioned by large-project effects but to the potentially significant adjustments involved in acr- :iting for correlation effects. These can be in either direction. For instance, a project with a strong negative correlation with national income (such as, say, a major flood-control and irrigation project in an agrarian economy) may have a certainty-equivalent benefit in
-5-

excess of E[PVJ. Conversely, projects positively correlated with national income will be analogously discounted for uncertainty.

Lit"'e and Mirrlees (1974) offer advice remarkably similar to that of UNIDO (1972), generally in the spirit of Arrow and Lind (1970), and catalog several 'more difficult cases' (p. 316) when social E[PVJ may be inadequate as a criterion. Briefly, these are: (a) projects where (downward sloping) demand effects may not be properly accounted for when prices or quantities are uncertcrn (Here E(PV] may still be a satisfactory criterion providing that nonlinearities are allowed for in computing E[PV], although this will minimally require knowledge of the appropriate joint distribution of the component random variables.);

- (b) projects with benefits (X) correlated with (i.e. not independent of) national income (Y);

- (c) projects with future public relations sensitive to uncertain outcomes, argued not to be very important;

- (d) projects with uncertain X large relative to Y;

- (e) locally 'important' projects where benefits are not widely spread and are, perhaps, concentrated in seriously disadvantaged group,, whereupon this is a special (local) case of (d);

- (f) projects with uncertain benefits and relatively high 'irreversible' costs, perhaps to the environment.
_-6-_ Concave utility functions are also used by Little and Mirrlees to develop useful pragmatic approximations to assist planners in computing riskadjusted (approximate certainty-equivalent, X ) values for project benefits. Their formulae feature a dimensionless coefficient of relative risk aversion, A, that is intuitively reasoned to be in the range 0 to 4, probably about 2. It would be unity if the utility function were logarithmic. Some sample evidence from farmers in Nepal suggests that higher values (say about 4) may be more appropriate for low-income groups (Hamal and Anderson 1982).

The two key approximations are based on severely truncated Taylor series representations and are presented here in a form that highlights the coefficient of variation. The first is a second-order approximation for the 'large project' case:

- **A** ~~~~~~2

- (1) X = E[X]{l - _(A;2)_ C[XJ E[X]/E[Y]I},

- where: X is the certainty-equivalent value of the randorm benefit X, El ] is, again, the expected value operator, V[ is the variance operator, and

C[ ] is the coefficient of variation operator C[XJ = _V[X]P_<sup>_5_</sup> _/E[X]_ The second is a first-order approximation for a project mutually dependent with income: (2) X [ E[X]{l - Ap C[X]C[Y]}, xy where P xy is the simple correlation between X and Y and, as for equation (1), the risk deduction A X (the second term in the curly brackets) is expressed as a fraction of E[X], namely, P.

A similar approach was taken by Scandizzo (1980), also exploiting the popular constant relative risk aversion function U = (l/(l-A))YWA in his attempt to synthesize risk accounting into the Squire and van der Tak (1975)
- 7 -

framework of social weights to account for distributional impacts of projects. Since he dealt with the case of a closed economy, he also emphas zed the relevance of 'unit revenue' price forecasts that embody the negative correlation between output and price brought about by the conjunction of downward sloping demand curves, producers' expectations and the likelihood that uncertainty enters 'multiplicatively' (via yields) in typical agricultural markets (Hazell and Scandizzo 1975). Consider the additional (project) output of good i as Xi, for which the unit revenue version of price is Ri = Pi ui/ui, where price Pi and yield ui in the present notation, are stochastic, then Scandizzo's version (his equation (39)) of equation (1), in which it is assumed that there are no distributional impacts, is (3) X = E[X] - AE£ oPi C[Ri]C[Rj]E[Xi]/E[Y]p where the summations are over all additional goods. When income groups are introduced, indexed by k, distributional impacts are inextricably interrelated with the risk attitudes implied by the social welfare or utility function in expressions like (his equation (46)) (4) X = E[XJ + E(wk -)E[XkJ - AELe ikpijC[Ri]C[R ]E[X ik]/E[YI, k kij where wk < 1 is the kth social weight based on the standardized curvature of the welfare function and eik is the ratio of demand to supply own-price elasticities for the ith good and kth group of consumers. Formulae such as the latter two have yet to find a place in operational practice of project appraisal. It may be asking too much of simple formulae to accomodate income distribution and risk aversion considerations simultaneously and, indeed, this may not be necessary in other than exceptionial cases. As noted earlier, social accounting may be involved in assessing present values, of which the expected value is taken as the criterion for choice.
Pragmatic procedures such as these are unpretentious simplifications to assist in the (implied rare) more difficult cases. Naturally, more elegant and precise procedures are available but these come at quite some cost in terms of additional specification of: (a) the nature of the probability distributions through which uncertainty is encoded; and (b) the nature of utility functions through which individual and societal risk attitudes are encoded (e.g. Anderson, Dillon and Hardaker 1977).

The most explicit treatment of a variety of 'simple' cases is provided by Wilson (1977). His models show clearly the interrelationships betwE.n individual and aggregate risk and risk aversion and, inter alia, deal with the issue of the efficient allocation of risk in an economy, among individuals and over time, especially through capital and insurance markets. He emphasizes the point made by Little and Mirrlees (1974) and many others that ordinarily it is inaccurate (and perhaps quite misleading in biasing against long-lived investments) to use a risk-adjusted discount rate. Usually an adjustment (a risk 'charge' or deduction) should be made to E[PVJ, along the lines of equations (1) and (2) above. Wilson's models focus on a measure of risk aversion he calls a 'risk tolerance', ri for the ith individual where, in terms of the above notation, ri = Wi/Ai where Wi is the individual's wealth.

The style of his results can be introduced by considering the static case with (negative) exponential (constant risk-aversion) utility and normally distributed uncertainty. Aggregate risk tolerance over n individuals is r = Er. = nr. The risk charge for an individual with uncertain income yi with mean mi and variance a2 is (5a) a(yi) = l/(2r) a ,2 and this charge can be aggregated once any dependence among the Yi is
- 9 -

specified. In the extreme case of independence (his equation (1.48)), the aggregate charge is the same as the average individual charge (5b) a (y) = (1/(21))a2 which, if n is large, is insignificant, relatively speaking (as proved also in the Arrow and Lind (1970) theorem) but in the other extreme case of perfect positive correlation (his equation (1.49)), it is (5c) A(y) = n(l/(2r))a2, namely the full sum of all individual risk charges. An intermediate case is provided by adding a project with random return z whence the incremental risk charge due to the project (his equation (1.50) based on bivariate normality) is (6) A(z) = (l/(2r))[c 2+ 2a a p1 **_0_** z z yyz which clearly shows the benefits of negatively correlated projects. Indeed, the incremental charge for any project depends on the correlation of its benefits with all other projects adopted so that, where uncertainty is important, a project should not be appraised in isolation but rather all possible combinations should be considered.

Wilson (1977) generalizes these static results to several intertemporal cases with a concomitant increase in complexity. For instance, analogous to the A(y) static results above, a recursive formula (his equation (2.33)) is required to compute the corresponding dynamic charge applicable to the planning moment t = 0: where 8 t is the price at t-l of a bond paying $1 at date t and _8_ t is the price of the same bond at date 0, a Ot is the intertemporal correlation between incomes in periods 0 and t, and A1 is the risk charge from date l
- 10 -

forward which will depend on the conditional variance of incomes given yo subsequent to the uncertainty of this first period having been resolved, and so on. Perhaps the most important point to note here is the pervasive role evident for the intertemporal correlations, as well as the already-noted correlations with other sources of uncertain income. Also, the gains from early resolution of uncertainty may well influence the timing of some information-gathering endeavors.

In spite of the confessed simplicity of such models for project appraisal under uncertainty, the demands implied for several categories of rather sophisticated information are considerable. Evidently, from the paucity of applications (except within the Inter-American Development Bank), most analysts have decided that the informational and analytical costs of such disaggregated risk analysis outweigh the benefits to be gained in terms of. 'better' decisions about uncertain projects. This generalization seems to hold also for the even-more-ambitious attempts to represent utility (preferences about risk, time and whatever) in a multi-attribute utility theory (MAUT) setting (e.g. Bell, Keeney and Raiffa 1977, Dillon and Perry 1977). Analysts have seemingly left the theorists to get on with the largely unfinished (and possibly unfinishable) task of sorting out a cohesive theory of risky investment appraisal (see Anderson, Dillon and Hardaker (1977, Ch. 8), Meyer (1977), and Drynan (1981) for some reviews of attempts to do this), and have resorted to a diversity of further pragmatic procedures to which attention is now turned.

# 2.2 Practical considerations

Several alternative methods can be used to analyze uncertainty in projects and, broadly following Bonini (1975), those considered here are (a)
- 11 -

certainty models, (b) Hillier models, (c) stochastic simulation rnodels and (d) decision analytical models. These are not mutually exclusive approaches in several respects. For instance, a stochastic simulation model run with _all_ random variables collapsed to their means will be a certainty model and, if it is linear, will have identical expected criteria, such as E[PV] or E[ERR] where ERR denotes 'economic rate of return', the World Bank's term for the initernal rate of return.

The certainty model is the standard riskless approach of investment appraisal that is used routinely in private and public agencies. In the context of public projects, the point to emphasize (following the review of section 2.1) is that, if risk is being ignored or assumed away, expected or mean values must be used for all truly random elements that are subject to linear operations (like summation) in computations of PV and ERR. It seems that there is still a tendency for analysts to use 'most likely' or modal values (e.g. Gittinger 1982, p. 9), in spite of the careful advice of authors such as Reutlinger (1970, pp. 25-7), Little and Mirrlees (1974, p. 322) and Ray and van der Tak (1977). Of course, under many distributional assumptions, mode and mean will be identical, or nearly so.

These remarks concerning linear operations on expected values do not absolve the certainty modeler from bias in E[PV] estimation when components enter nonlinearly, such as project life (Solomon 1966, Greer 1970) and the discount rate (Kaplai and Barish 1967, Griffiths and Dillon i976) in which case Jensen's inequality holds, or, multiplicatively and nonindependently (Wagle 1967), in which case (see Appendix 1) for z = xy (8) Etz] = E[x]E[yl + cov[x,y]. Jensen's inequality states that, if z = f(x), E[zJ < f(E[x]) as f'(x)<0.
- 12 -

A variant of the certainty model in which some attempt is made to grapple with uncertainty is 'sensitivity analysis', wherein assumptions about components of a project are systematically perturbed in order to discover the sensitivity of the criterion, say E[PV], to values of presumably uncertain elements. In this way project designers can learn something useful of the reality they are modeling, as argued by Ray and van der Tak (1977), but the dissenting position of Little and Mirrlees (1974, p. 309) on sensitivity analysis is surely appropriate when uncertainty can be dismissed, and sensitivity analysis per se is surely inadequate to the task when it cannot be dismissed from consideration.

The Hillier (1963, 1969) model for estimating the probability distribution of PV by E[PV] and V[PV], along with its extensions to the distribution of ERR (Fairley and Jacoby 1975), relies on appeal to the Central Limit Theorem for approximate normality of PV. Then, only estimates of mean and variance of PV are required and, in turn, only, apart from a complete discrete distribution of project life, if this is uncertain, means and (co)variances of all components are required to be specified. Further simplifying assumptions are advocated by Hillier for practical implementation in order to reduce the demand for estimates of all the potential correlations involved. Box-Jenkins type models have also been suggested as convenient methods for modeling dependencies, especially over time (Bussey and Stevens 1972).

The Hillier model is subject to the same biases noted for the certainty model for nonlinear operations and multiplicative uncertainty. The potentially important statistical dependencies are clearly highlighted, although analysts may shy from the explicit challenges of specifying relevant contemporaneous and intertemporal correlations among the random variables.
- 13 -

Perhaps this, along with thL difficulties of estimating the unconditional variances (Bonini 1975), is why exploitation of the Hillier model has seemingly been so slight. A further virtue is its convenienice for determining efficient portfolios of interdependent projects, in which application it is deserving of much greater attention than it has received. However, when the portfolio/capital budgeting aspects are played down, as they tend to be in public project appraisals such as conducted by the World Bank, this virtue is of little advantage, and more flexible ad hoc methods are of greater appeal.

Stochastic simulation models have been the most widely used models for 'risk analysis' (Hertz 1964) or project appraisal under uncertainty. Indeed, this approach of Monte Carlo sampling of random elements to compute empirical distributions ('risk profiles') of criteria such as PV and ERR was examined, exposited (Reutlinger 1970) and applied (Pouliquen 1970) by the World Bank at an early stage of the innovation cycle. Its key virtue is its flexibility and ease of incorporating virtually any stochastic (random) consideration or other relationship that may be desired, including the perhaps critically important uncertainties associated with streaming of projects. The proliferation of low-cost computers has facilitated adoption of the technique in a diversity of research and commercial applications (see e.g. Anderson (1974) for a review of agriculturally oriented applications)).

Several persistent difficulties, however, continue to constrain use of such simulation models in project appraisal. Part of the 'cost' of the inherent flexibility is a relatively large cost of analyst time in getting started and in refining a model. An application seemingly always takes longer and absorbs more resources than was naively anticipated (Dillon 1971, Anderson 1974). The major practical difficulty centers on the specification of stochastic dependencies within a simulation model, a difficulty noted already for the
- 14 -

Hillier model. Reutlinger (1970, pp. 24, 41) addressed this issue and noted the significant bias (especially in estimates of V(PV]), that can be incurred through misspecification of (particularly) intertemporal or serial correlations. In the context of a multivariate normal representation of uncertainty, Harrison and Cassidy (1977) have illustrated such biases for estimated distributions of PV, and also pointed to underspecification of autocorrelation as an important culprit ir. underestimation of project risk (as measured, say, by V[PV]). Hull (1980) has additionally explored how correlated normal variables can be transformed to represent non-normal risks.

Decision analytical models are the final category in this brief catalog of approaches to project appraisal under uncertainty. The distinctions from the categories discussed above are often blurred but the essence of such models is to optimize a sequence of decisions over time using the principles of dynamic programming, perhaps aided by representing the sequence as a decision tree (Raiffa 1968, Anderson, Dillon and Hardaker 1977). As noted at the close of section 2.1, modern developments in investment analysis under uncertainty have been concentrated on MAUT representations of intertemporal and risk preferences of decision makers. In principle, decision analysis could be embedded in a simulation model in place of the usually exogenously supplied decisions and rules. The certainty model might be thought of as a very special case of decision analysis with a risk-neutral utility function defined over PV. What with the difficulties of specifying applicable intertemporal utility functions and the complexity of decision-analysis computations in the typical absence of pertinent software, decision analytical models per se do not seem destined for much application in public project appraisal in the medium term.
- 15 -

# 2.3 World Bank practice

In termq of the categories sketched, consideration of uncertainty in project appraisal has been confined mostly to the certainty model, with much emphasis on sensitivity analysis (e.g. Gittinger 1982), and (rarely) to stochastic simulation (Reutlinger 1970, Pouliquen 1970). Experience and developments since 1977, if there have been any, have seemingly not been documented, although agencies such as IDB do continue routine application of risk analysis of very risky projects such as international tourism, petroleum and mineral exploration and some industrial projects (Powers, 197x, 1982).

The most recent 'official' position on such matters is that of Ray and van der Tak (1977) in CPN 2.02. They (a) highlighs the general relevance of E[PV] and E[ERRI and clarify that 'best estimates' should be of these expectations; (b) note the inadequacy of such criteria under exceptional circumstances (detailed by Little and Mirrlees (1974) and summarized in section 2.1 above)) but without providing 'how to do it' guidance; (c) argue for the usefulness of sensitivity analysis and computation of switching values; and (d) suggest some scope for more use of 'quantitative risk analysis' as a vehicle for informing appraisers about risk, providing that assumptions and qualifications are well documented. Assuming this is where things stand as of 1983, is this the most desirable state of affairs?

Stochastic simulation models (e.g as reviewed in the Appendix to CPN 2.02 and illustrated by Reutlinger (1970) and Pouliquen (1970)) do indeed provide analysts with a (hopefully) coherent review of the risk involved in a project, but is this intrinsically useful in the Bank environment, in the sense that it might influence decision making? Following section 2.1, the answer to this must be "No", unless some of the exceptional circumstances
- 16 -

apply. Under such circumstances, say with 'large' or 'dependent' projects, the question may again be posed, and this time the answer must be "Maybe but", given the way they have been done, "Probably No"I This negative answer arises because the linkages to the risky project from the national or local economy (which constitute the exceptional circumstances) have not, as far as I can ascertain, been explicitly modeled and accounted for. This might be expressed alternatively as follows. The exceptional circumstances emount to cases where society is properly viewed as risk averse in its attitude to a project. The impact of such risk aversion has not, it seems, been formally measured by computing a risk adjustment (e.g. to the E[PVJ performance criterion), through presumption of an explicit utility function. It may be (hence the "Maybe" above) that such an accounting has been done informally on the basis of intuitive consideration of estimated risk profiles or even perhaps more formally by sorting projects according to, say, second-degree stochastic dominance rules (Hadar and Russell 1969, Anderson, Dillon and Hardaker 1977, Ch. 9). S. Reutlinger (personal communication 1983) believes that public project appraisers often do manifest (personal?) risk aversion and multiple objectives (although perhaps in a project-dependent manner). In this case, the informational transmission on uncertainty through risk analysis may have been of more use in informal interpretation and decision making than is apparent to the present writer.

To be more positive, it seems that the most straightforward method of social risk accounting, albeit doubtless controversial in terms of its informational demands, is to embody social risk aversion explicitly in any risk analysis. To the extent that dependence with national income 'or, more generally, with other selectable projects) is important, such dependence,
17 -

-

along, of course, with specification of the uncertainty of national income, needs also be incorporated in the model for risk analysis.

Lest this seems like more work to dubious advantage, there are some useful potential spinoffs from incorporating such variables that are usually regarded as external to a particular project. For instance, if one or more prices important in a project are mutually influenced by (stochastically dependent on) national income (or, say, growth of gross domestic product), modeling this will not only provide the capability of computing the risk adjustment and certainty equivalent PV but may also greatly simplify proper accounting of the dependencies among the prices. That is, correlation effects will be built into the logical structure of the model rather than having to be specified as part of an arbitrary multivariate probability distribution. Analogously, similar problems of serial dependence may also be swept up, at least in part, by incorporation of aggregative 'driving functions'. Another such example would be modeling an index of seasonal experience which, in turn, conditions 'automatically' several mutually dependent crop and pasture yields in an agricultural project.

# 3 Measuring Uncertainty in Project Componeu,ts

Almost every component of a projec. is, in principle, subject to uncertainty, whether it be starting date, li.e of the project, costs, and benefits and their component prices and quantities. Given the emphasis herein on price forecasting, attention is directed first to quantifying uncertainty about variables that enter forecasting equations exogenously.

3.1 Uncertainty in variables external to the project This first case of uncertainty is introduced by means cf a univariate example of an important exogenous variable and more general issues are then taken up and illustrated.
- 18 -

# 3.1.1 GDP annual percent change

The growth rate of gross domestic product (CDP), g, is a key variable in many of the structural equations for commodities and is a worthy subject for quantification of inherent uncertainty. GDP must always manifest variability since it is the aggregation of many individually variable components and, since these are not perfectly predictable, the variability will properly be interpreted as uncertainty (Quiggin and Anderson 1979, p. 194).

The uncertainty in g might be modeled in several different ways. Perhaps the most appealing approach would be to represent gt (in turn, derived from the first differences of the logarithm of GDP) as an autoregressive integrated moving average (ARIMA) process whereby the stochastic structure could be specified, estimated and thus explicitly and comprehensively described. For instance, gt might prove to be adequately described as a firstorder autoregressive AR(1) process, (9) gt = 4lgt-l +6 + et' where 6 is trend or 'drift', 01 is the autoregressive parameter and et is 'white noise' with zero mean and constant variance 0 2 The variance of gt is then found (Pindyck and Rubinfeld 1976, p. 521) as

**y0=** a2,(l, 12) For instance, if a= 3.5, 1 = *4, then yo 4.2 with standard deviation yo A 2. For the 8 years 1975 to 1982 (see the second column of Table 1 below), gt for the OECD countries as a group had a mean of 2.3 percent and standard deviation 2.0 percent (cv = .9).

It might be argued that such a crude description of variability, abstracting as it does from any prediction of the series, overstates the extent of uncertainty. To explore the matter further, consider the data presented in Table 1. Predictions of gt are published at various times. Those
-19 -

identified with an asterisk constitute a somewhat arbitrarily selected set of 'predictions' made one and two years ahead. Subtracting the actual gt from these gives the column called 'errors' for which there is a mean for the time series of 1.9 percent and standard deviation 1.3 percent (cv = .7). Thus short-term forecasting does produce a seemingly less uncertain series. However, for long-term forecasting, the relevant dispersion is that of the series itself.

Attention is now turned to the impact of such uncertain exogenous variables in derived forecasts.

TABLE 1: REAL ANNUAL GDP GROWTH RATES IN OECD

|----------------|g<br>Actual<br>Reported by<br>(latest<br>Ghose (1978)<br>estimates)<br>--------------------------------------------------|P<br> <br>1978<br>Report 8|g<br>redicte<br>1980<br>14|d<br>1982|g-g<br>Errors<br>using*****<br>__---------|g_g<br> <br>g<br>_|
|---|---|---|---|---|---|---|
|1975|-0.7<br>3.0*||||+3.7|-5.30|
|1976|5.2<br>5.4*||||+0.2|0.04|
|1977|3.6<br>5.0*||||+1.4|0.39|
|1978|3.9<br>4.6*|3.8|||+0.7|0.18|
|1979|3.4<br>3.3|4.2*|||+0.8|0.23|
|1980|1.4|4.2*|1.7||+2.8|2.00|
|1981|1.2|4.2|3.5*||+2.3|1.92|
|1982|0.2 (est.)|4.2|3.5*|0.2|+3.3|16.5|

* Selected as 'the' short-term forecast.

# 3.1.2 Conditional forecasting precision

The main paper on conditional forecasting accuracy is by Feldstein (1971). His key simplifying assumption is to introduce stochastic forecast exogenous variables xF but to assume that these are independent of estimates of the regression coefficients. Other important restrictions are serially independent disturbances, no lagged endogenous variables and linearity in the
- 20 -

models. His equation (4) for the variance of forecast error is: (10) a 2F = - n + BAB + tr(nA) + o2 yF -;aF -F u where a is the covariance matrix of the regressors and A is the covariance matrix of the forecasts.

To simplify the notation somewhat, write this generalization of Appendix 1 in summation form, and omit inessential subscripts as: (11) Y Z biX+ u

- (12) E[Y] = E E[b JE[X i

(13) V[Y] = Z £ X1Xj cov[bi, bji + V[u]

- E Z bibj cov(Xi, Xj] + £ V[bi] V[XiJ

where E, V and cov denote expectation, variance and covariance operators, and all summations are over i = 1...., k. Note that the final two terms in (13) are additional to the first two which are the (traditional) expression for the analogous unconditional variance for a forecast. Feldstein argues that (expressed in terms of relative (to squared means) variance), the 'traditional' might plausibly be half or less of the magnitude of this 'conditional'.

To see this model more transparently, consider the special case where b= a, b2 = b,, X1 = 1, X2 = x so (11') Y = a + bx + u (12') E[Y] = a + **bE[x]** (13'] V[Y] = V[(a + 2x cov(a,b) + x<sup>2</sup> V[b] + 92 + b<sup>2</sup> V[xJ + V[b]V[xJ because the intercept 'variable' X1 is nonstochastic and V[X1j = 0, or equivalently, in terms that are perhaps more familiar, (13'") V[Y] a2 [1 + l/T + (x - X) 2/ (X - X)2] + b2V[x] + V[b]V[x] t t 2 (13..'') V[YJ (Z=unconditional variance) **+** b V(x] + V[b]V(xJ.
- 21 -

For further concreteness, consider the concocted 'regression': Y = 46 + 50x, a =2083 and (20) where forecast x = 2, V[x] = 2 and where unconditional variance (SE) is 2341 (48.8) but conditional variance (SE) is 8141 (90.2), illustrating that the standard error of the conditional forecast is nearly double that reported traditionally once uncertainty in X is accounted. The present example is based on the 1982 forecast for cocoa price in 1990 (World Bank 1982).

= In conditional variance terms, unconditionally this was 48.8/246 .20, combined with the cv(x) = 2o5/2 = .71 results in a conditional cv = 90.2/246 = .37.

The practicability of such an approach hinges on the estimation of cov(Xi,X;), since it has already been demonstrated that all the other elements of the computation can be handled (see Annex D of World Bank, 1982, Volume I of Report No. 814/82).

To address some further possibilities, consider first the univariate case of x in the 'regression' above. The (mean) forecast value of x is 2. One might simply 'have' a subjective cv of .71 as illustrated. Alternatively, one might resort to a distributional representation, such as the convenient triangular distribution.

The triangular distribution is defined by three parameters, A, M, B, (range and mode). For the present, suppose that these pararmeters are 0, 2, 4--
- 22 -

i.e. the analyst believed the most likely x is 2, the lowest possible is 0 and the highest possible is 4. This might be, say, growth rate of GDP in 1990. The first two moments of the distribution are then found as: (14) E[X] = (A + M + B)/3 = 2 (15) V[X] = (1/18) [(B - A)<sup>2</sup> + (M-A)(M-)J1 = .67 so that S[X] = V[X]*<sup>5</sup> = .82 and cv = .41, for example.

Alternatively, other somewhat more cumbersome subjective elicitation procedures could be used to translate analyst's feelings of uncertainty into summary statistics. These are described by Anderson, Dillon and Hardaker (1977, ch. 2, pp. 23-6) and are taken up further in section 3.2. One very convenient special case is that of normality.

When it comes to the more general case of eliciting the set of covariances for several elements Xi, i = 1,...,k, the resort to normality is especially useful as the elicitation procedures become very cumbersome (Anderson, Dillon and Hardaker 1977, pp. 28-37). In that case, it is likely that pragmatists may prefer to rely on historical sample estimates of such covariances--and then assume that these will persist into the future. This embodies the dubious principle that 'the future will be like the past because in the past the future was like the past'! The example of the previous section illustrates simple procedures for extrapolation of such historical patterns of variability and forecast inaccuracy. Knowledge of any structural change (for example, of changed stockholding and intervention policies in the international market for grains) should, of course, be included in the estimation of future trends and variabilities of prices and quantities.

3.2 Uncertainty in project variables The considerations involved in representing uncertainty in project
- 23 -

variables differ little from those noted for external variables. A few representative cases are explored for any generalizations that can be made. 3.2.1 Quantities The archetypical quantities subject to uncertainty in project appraisal are agricultural yields, such as of crop and livestock enterprises, expressed at any apposite level of aggregation. The methods for describing uncertainty in such random variables are, of course, just as applicable to other uncertain quantities such as commencement lead time, population growth rate, labor productivity, cost overrun, supply of factors such as irrigation water, rate of technical change, etc.

Subjective probability (Savage 1954, Raiffa 1968, de Finnetti 1974) is the natural language for describing or encoding all uncertainty. Such probabilities are judgemental expressions of degrees of belief that are subject to the classical calculus of probability. People differ in their probability judgments, as they do in other personal characteristics. To the extent that judgments are influenced by a common core of experience, and perhaps historical data, assessors will, however, tend to converge in their probability assessments. Sometimes, when past observations are judged to be of ongoing (unchanging) relevance, these may be processed directly (objectively) into probability distributions that encode future uncertainty as subjective probabilities.

In any such description of uncertainty, analysts must make several choices as to method of elicitation or estimation, type of distribution (discrete, continuous or mixed; univariate or multivariate), family of distribution (arbitrary empirical or some theoretical distribution such as normal, beta, triangular, rectangular, etc.) and style of description (e.g.,
- 24 -

graphically, parametrically, or by several moments). These considerations, which are not readily susceptible to 'cookbook' treatment because of their essential subjectivity, are detailed variously by Raiffa (1968), Schlaifer (1969), Reutlinger (1970), Stael von Holstein (1970), Winkler (1972), and Anderson, Dillon and Hardaker (1977, Ch.2 ), among many others.

Choices concerning these aspects are not independent. As observed in section 3.1, a popular choice for univariate continuous distributions, because of its ease and flexibility, is the directly elicited triangular distribution with its three parameters and easily sketched PDF. In many other cases, perhaps through appeal to Central Limit theorem reasoning, the two-parameter (mean and standard deviation) normal distribution may be chosen, especially if the distribution is multivariate (in which case parameters consist of k means, k standard deviations and k(k-l)/2 correlations).

The main driving functions in agriculture, such as climate, pestilence etc. tend to be statistically independent from year to year so that the need to specify autocorrelations among quantities is probably slight. Contemporaneous effects, however, may be much more common but, as noted at the end of section 2.3, it may prove most convenient to model directly joint casual random features such as rainfall, and thence to condition the variables subject to the joint random effects. Where this is not possible, such jointly distributed quantities will have to be specified directly as such. 3.2.2 Prices In principle, prices can be handled in the same manner noted for quantities. Again continuous probability distributions will be those most frequently relevant. It may be more feasible, however, to contemplate modeling an economic structure in which random prices are generated. Random variation
- 25 -

may arise, for example, from the aggregative effects of stochastic yields and perhaps also random demand influences.

Contemporaneous dependencies among prices may be modeled relatively simply by relating prices to other random variables such as GDP (perhaps in major importing countries) that, for instance, cause similar demand shifts across commodities. Such an approach may also lead to simple accounting for serial correlations. These are likely to be rather more common among prices than quantities, and, accordingly, ARIMA modeling (Box and Jenkins 1970) of such time series (perhaps complemented by other information on markets) may prove an expedient modeling approach. It is assumed here, of course, that the best available information on trends of expected prices is already embodied in any project appraisal.

# 3.2.3 Other variables

The procedures that best suit modeling of other uncertain components will depend on the particular circumstances perceived. For example, uncertain project life will probably best be represented as an arbitrary discrete probability distribution. In a sense, any elicitation or estimation of probabilities is an arbitrary, judgmental exercise that can always be criticized and thus minimally requires full explication and preferably a reported rationalization.

# 4 Workable procedures for uncertainty accounting

Now that the main issues have been canvassed, and prevalent procedures reviewed, it is opportune to advance some suggestions for methods that feature the key aspects of social risk aversion when it should be accounted for, yet which do not involve infeasible analytical costs in the process. Needless to say, these desiderata severely constrain the options, and it is
- 26 -

just possible that the optimal set of procedures is as empty as recent practice would implyl

# 4.1 Pragmatic methods for computing risk adjustments

A first step is the inherently difficult one of deciding whether any sort of accounting for risk is worthwhile. The subjectivity here is overt and inescapable, since an answer cannot be given with any precision until some risk analysis has actually been completed. Sensitivity analysis of a deterministic or certainty model, for example, just cannot address the question. Unless the project appraiser has some strong intuition that uncertainty will be important in decision making (Reutlinger 1970), the general guidelines of Little and Mirrlees (1974) are probably useful in this decision, namely ignore uncertainty unless the project is 'large' (say expected return > 10 percent of GDP) or 'significantly' correlated with GDP (presumably in some intuitive sense).

# 4.1.1 A 'rough-and-ready' approach

A first extremely simplified approach might be used as a screening device to provide a hint as to the virtue of a more thoroughgoing analysis of the impact of uncertainty. Several possibilities suggest themselves but first consider the quintessence of the crude approach in its simplest guise consisting of the following steps that might succeed a conventional (certainty) appraisal:

1. Choose a 'representative' early period (year) in the life of a project when returns and costs should have 'settled down' (t*);

2. Estimate the ratio (R) of mean project return to mean GDP (or other more local measure of aggregate income or economic performance judged to be most relevant) for this period;
- 27 -

3. Elicit (i.e. subjectively formulate) the simple correlation between project return and the aggregate income (p) and estimate the coefficient of variation of aggregate income (detrended), namely c y ; 4. Assess the mean and standard deviation (or coefficient of variation) of all major uncertain variables (prices and quantities) for this period; 5. Compute a rough estimate of the coefficient of variation of net project return (cx) for this period. (The mean return will have already been computed in the first-round appraisal.); 6. Compute, by means of simple reference formulae or tables, the proportional risk adjustment for the period Pt* (i.e. the risk deduction expressed as a proportion of mean project return); 7. Decide if this is 'significant' (say, >.Ol) and:

- (a) if so, adjust (multiply) estimated E[PVJ by the factor

- (l-Pt*) to give a crude risk-corrected or certainty equivalent PV; or

- (b) if not, conclude that, in this instance, uncertainty has no worrying impact on the appraisal and, accordingly,

- proceed to ignore it and base the decision on the certainty appraisal.

The gross simplifications embodied in this sequence are all too obvious. The idea of a 'representative period' greatly simplifies the process but at the cost of ignoring (a) uncertainties in the developmental phases early in the life of the project, (b) uncertainty about the life of the project, (c) serial dependencies among the uncertain variables (bias from this
- 28 -

omission is probably in the direction opposite to that inherent in ignoring (a) and (b)), and (d) of representing so crudely the interdependence with the rest of the economy.

Yet this rough and ready method is not as costless as may be apparent at first blush. Step 4 may involve considerable new data gathering (e.g. on probability distributions for forecast prices) and/or subjective elicitation (along the lines sketched in section 3.2). Step 5 is not too difficult if not too many of the project components are uncertain (whence the simplifying formulae of Appendix 1 can be used) but can be a little more cumbersome if several mutually dependent variables are involved and a Monte Carlo approach must be used (Anderson (1976) provides such a program).

The heart of the method is Step 6 which is now explained more fully. In reviewing (in section 2.1) the Taylor-series approximations presented by Little and Mirrlees (1974), it was noted that the separate exceptional cases of 'large' projects (equations (1)), and 'small dependent' projects (equation (2)) were catered for and, depending on whether the response to Steps 2 or 3 is approximately zero, respectively, such formulae can be used directly in Step 6. Of course, if both are effectively zero, one should proceed to Step 7(b) forthwith without incurring any costs of risk analysis.

Potentially, however, there are many 'interesting' cases for which the answers to both Steps 2 and 3 are non-zero, and a new approximation procedure is then called for. To this end, a small Monte Carlo analysis was run for a diverse range of values of key summary attributes of a project in relation to an economy (Appendix 2). The results can be summarized conveniently, albeit with the loss of some precision, by means of a variant of equation (6)
- 29 -

expressed in a form analogous to that used in equations (1) and (2): (16) P = Acx (cx R/2 + pcY) Equation (16) can be entered for the computation in Step 6 by substituting the values determined in Steps 2, 3 and 5. This is a rough mechanical approximation, probably as 'good' as those suggested by Little and Mirrlees (1974), but its 'goodness' (as does their's) depends on the level of risk aversion that is really appropriate and, in this rough-and-ready approach, this issue can be dodged by presuming, in equation (16), that relative risk aversion A is two.

The immediate extensions to this simplest version of the present approach are still fairly 'rough' but the 'ready' advantage diminishes rapidly. There is clear scope for honing the estimation in Step 3. Extending the temporal coverage beyond the representative single period in Step 1 has obvious consequences for additional information on n periods (i.e. at least n times the one-period case) but, in addition, has the less obvious requirement of explicating interperiod (e.g. intertemporal correlation) effects which may be both demanding of specification and important in consequence. More comprehensive stochastic specification in Step 4 c- lead to 'better' probablistic description, but at possibly considerable informational cost. To go beyond the pragmatism implicit in Step 6 requires a rather more expensive form of analysis, perhaps along the lines to be elaborated in the next section.

4.1.2 A stochastic simulation approach It was presumed in section 4.1.1 that a conventional or certainty analysis of a project has been done as a prelude to any consideration of risk. As was observed in section 2.2, however, a certainty model minimally provides th0 basic structure of a more wide-ranging stochastic simulation of the project investment phenomena. The general procedures for such simulation modeling
- 30 -

are outlined by Reutlinger (1970), Fishman (1971), Naylor (1971), Mihram (1972), Anderson (1974), Kleijnen (1974-75) amongst others, and need not detain the present discussion unnecessarily.

Accordingly, the focus of attention should be on aspects of the method that have particular implications and cconsequences for project appraisal. Most of these aspects have been mentioned in the critical reviews of previous procedures. Two aspects of special significance are the correlation or dependence structures, and the related question of linkages beyond the project itself, including correlations with macroeconomic aggregates and the associated feedbacks to the performance of the project in both costs and returns.

The further point of almost unique significance is the embedded utility function. Once the analyst has elaborated the logical and stochastic structure of the project, the completing assumption must be the explication of an intertemporal preference structure. As mentioned elsewhere herein, the possibilities available are diverse in terms of both theoretical defensibility and operational convenience. Some of the simpler possibilities are:

- (a) utility of present value (Hillier 1969)

- (17a) U = U(PV)

for example, the constant relative risk aversion utility,

- (17b) U = (l/(l-A))(PV)l1A,

where A * 1 is relative risk aversion, or, if A = 1,

- (17c) U = ln(PV),

and present value is defined conventionally as

- T

- (18) PV= C/(l+r ) t t=Ot t
- 31 -

where Ct and rt are the period t net cash flow and interest rate, respectively; (b) additively separable utility (Jean 1970) T (19) U = ZktUt(Ct) t=ot where period utilities Ut might, for example, be of the form (17b, c) and the scaling constants kt are either determined on the basis of preferential and utility independence assumptions (Keeney and Raiffa 1977, Ch. 6) or, much less defensibly, set arbitrarily at the riskless discount factor, kt = l/(l+r)t, so that a present value of utility is computed; (c) multiplicative utility (Keeney and Raiffa 1977) (20) U = {nt[KktUt (Ct)+lJ-l1/K where the sum of the period scaling constants Zk * 1 tt and thus the new scaling constant K * 0; (d) multiplicative benchmark utility (Anderson, Dillon and Hardaker 1977) where a multiplicative ordinal function (21) Q = t(Yt + Ct) is used to convert each sequence of aggregate income plus project return, (Y0+C0, Y +C ,..., Y +C ) to a benchmark equivalent (Y0+C0 _',_ Y +C1 ,.... Y +C ), with Y +C t = 1,..., T equal to some minimal target level and then a T T t t utility function, again perhaps of the form (17b, c), defined for (22) U n U(YNC0+ IYl+C1+..., YT+CT+) The latter may be described as a 'rough-and-ready' way of circumnavigating the complexities of assessment and modeling described by Meyer (1977), 'taking the line that analysis using a rough but easily made approximation is better than either having none at all or the expense of a detailed appraisal...' (Anderson, Dillon and Hardaker 1977, p. 265). The benchmark Yt+Ct **+** might, for instance, be set at E[Y1]. Suppose a sequence is (900, 1200, 1250) and the
- 32 -

benchnmark is 1000, then the benchmark sequence is (1350, 1000, 1000) and, if the benchmark utility (equation (22)) is l/(l-A)(Y +C0 )1-A where A is 2, then the utility would be evaluated as -13501 = -7.407 x 10-4.

In a stochastic simulation, each sequence could be thus evaluated as a utility, averaged over replications (repeated pseudorandom encounters) and thus an expected utility computed. This could be interpreted conveniently as a certainty equivalent by solving equation (22) for the certainty equivalent benchmark and, in turn, the difference between this and the (computed) expected benchmark E[Y +C I ] gives the risk adjustment which might be expressed in proportional terms analogous to the previous Step 6. The decision as to the need for risk accounting can then be taken as before. All this sounds a little tedious, and it surely will be. Such, howevar, are the challenges minimally faced in social risk accounting in project appraisall 4.2 A case study illustration For simplicity, a simple hypothetical project is considered to illustrate the methods proposed in section 4.1. This is first examined with the rough-and-ready method. As it is hypothetical, there is little need to dwell on the context and assumptions except in so far as they have implications for the methods being described.

The assumptions about the economy are based on the recent experience of the Dominican Republic. This country is typical of several efficient producers and exporters of sugar (from cane) in that sugar is a major source of foreign exchange (here about 35 percent of exports) but, since the traded sugar market is so volatile, this source is rather unstable and contributes to significant macroeconomic fluctuations. Gross value of sugar production constitutes about 10 percent of GDP but this varies considerably (e.g. from 27 percent in 1974 to 4 percent in 1978). Another measure of this dependence
- 33 -

between the sugar industry and national income is the simple correlation, between the residuals from constant growth rate trends of (a) real GDP and (b) sugar output valued at the real international price (i.e. this valuation abstracts from domestic sugar pricing and the price rcealized on priviledged sales to USA and other importers). This correlation for the 2i years to 1981 is .32.

The hypothetical project involves a major new sugar estate and associated infrastructure of mills, roads and other handling facilities. When fully on stream there will be a (hypotheticall) additional 30,000 ha of cane harvested annually which, when processed, will have to be sold on the international market but, it is assumed, within the limits agreed under the International Sugar Agreement. Following a conventional appraisal, the expected cash flows are as now tabulated. As with many other sugar projects under the assumed depressed prices, it is not highly profitable (E[ERR] = 2.30 percent).

# Expected cash flow

<!-- Start of picture text --> Year 0 1 2 3 4 5* to 20<br>$106 -340 -240 -140 25 50 52.8<br>Expected GDP<br>$109 7.0 7.2 7.4 7.6 7.8 8 (increasing<br>at 2.8% p.a.)<br><!-- End of picture text --> The several steps involved in the crude risk accounting are now

# illustrated.

Step 1 Year 5 is a 'settled down' year with the project fully on stream. Returns and GDP are really stochastic, e.g. cv[detrended GDP] = .09, stochastic return from the project in the 'settled' year 5 is given
- 34 -

- by X = a{y(p-u) v}

- where a = project size (i.e. area harvested, ha)

- y = centrifugal sugar yield (presumed normal E[yJ = 8 t/ha, Sly] =.8, cv[y] = .1)

- p = export sugar price in year 5 (forecast as the mean or trend, E(p] = US$350/t)

- u = costs varying with y (harvesting and processing, net of byproduct sales, $/t)

v = other costs, varying with a ($/ha), and where capital charges associated with the initial investment of $720m are not double counted.

It is assumed initially that a = 30,000, u = 30 and v = 800 are known with certainty, and that uncertainty enters via the agronomic uncertainty about y which is assumed to be independent of the uncertainty inherent in the eventual market price p.

- Step 2 R = 52.8 x 106/(8 x 109) = .0066.

- Step 3 Say, p = .4, since the economy is very dependent on sugar exports and (after linear detrending), cy = .09.

- Step 4 Suppose the forecaster believes that, with due regard to all the possible sources of error, the distribution pf p in year 5 is approximately triangular with parameters (180, 300, 570), so that, using equations (14) and (15), E(p] = 350, S[p] = 81.6, cv[p] = .233.

- Step 5 Then, using the Appendix 1 formulae,

E[X] = a[E[y](E[pJ - u)-v} = 52.8 x 106 and, if p is regarded as approximately normal and independent of y,
- 35 -

from Appendix 1 equation (1.7b), V[X] = a {Vtyp] + u Vty]} = 30,000 {(82 x 81.62) + (3502 x .82) + (81.62 x .82) + (302 x 8 2)} S[X] = 30,000 x 713.7 = 21.4 x 106 cv[X] = .406.

> Step 6 Substitute these values for R, p, cy, and CX (and, say, A = 2) in equation (16),

P = 2(.406) ((.406)(.0066)/2 + (.4)(.09)} = 2(.406) (.00134 + .036)

**=** .030.

Step 7 So, the risk adjustment here is somewhat trivial and the appraiser, in retrospect, was not really assisted (or hindered) in the appraisal task through this consideration.

Other perspectives on the size of the adjustment can be gained by reference to the period returns. The absolute value of the year 5 adjustment is 52.8 x 106 x .03 = 1.58 x 106, and the certainty equivalent return is thus 51.22 x 106. Using this certainty equivalent in place of the expected return in years 5 through 20 yields a 'risk corrected' ERR of 1.99 percent which is about 100 x (2.25 - 1.99)/2.25 = 11.6 percent less than the expected ERR of 2.25 percent.

This minor adjustment magnitude is probably representative of the great majority of public projects, especially those reviewed by the World Bank. Naturally, the results can be made more 'interesting' (i.e., the economic consequences of risk can be made to seem more important) by enlarging some of the terms, especially CX but also R.
- 36 -

It could be argued, with some conviction, that the riskiness of the project is rather understated by the procedure. In particular, costs may also be properly regarded as uncertain and fairly highly correlated with GDP. Also, there are inevitably uncertainties in the streaming of the project, especially when it is planned for implementation at a time when prices prove to be very depressed. There is, further, the question of representing the uncertainty in the price forecasts over time in a way that reflects the statistical dependencies through time. These more vexing issues of risk can be grappled with only in a more comprehensive stochastic model.

# 5 Conclusion

Uncertainty, while ubiquitous, should play only a minor role in public project appraisal. This general conclusion thus serves to rationalize and reinforce prevalent practice in most agencies engaged in such work. It must be tempered a little, however, under various exceptional circumstances, including particularly large (relative to a national economy) projects. Other important cases where formal accounting for uncertainty may be important or desirable, include projects with socially uninsurable risks to significant disadvantaged groups that would suffer unacceptably in the event of untoward uncertain events, and, possibly, projects that are highly (especially positively) correlated with national income.

Unfortunately, recognition of when to worry about uncertainty is not straightforward and, in a sense, can be judged with precision only after a formal risk analysis. Risk analysis is becoming easier and cheaper with the proliferation of microcomputers and facilitating software and, accordingly, project analysts will presumably make greater use of the approach in the near future than they have in the past decade or so.
- 37 -

A position on the judgment intermediate between naked intuition and formal analysis is offered under the guise of a rough-and-ready approach to assessing risk adjustments (inevitably deductions) expressed as a proportion of an expected period return from a project. The approach features modest informational demands and a simple computational procedure. It can be reflected on further by interpreting the approximating equation (16) (i.e. equation (2.6)) as a basically constant elasticity adjustment function. The response of proportional risk deduction: to relative risk aversion A is unit elastic; to relative size of project R is unit elastic, ceteris paribus; to relative risk of the economy cY is unit elastic, for a given nonzero correlation p ; to relative risk of project cy is elnvtic (two), ceteris paribus; and to correlation p is unit elastic, ceteris paribus, respectively.

To get 'much' adjustment out of the crude equation, something must be 'large' and the most obvious candidate is tiie coefficient of variation of project return cX, the summary measure of the relative riskiness of the project viewed in isolation.

Cases leading to 'significant' adjustments will indeed be rare in practice, so that the existing policy of essentially ignoring uncertainty in most Bank project work is probably appropriate, and may even be optimal. The exceptional cases that are likely to be most important are very large risky projects in undiversified small economies (e.g,, copper in PNG) or risky projects that are large relative to an isolated target community to which they are directed (e.g., tea in the PNG Southern Highlands).
- 38 -

# 5.1 Implications for price forecasters

If the general conclusion is correct (namely that uncertainty needn't be accounted for in most public project appraisals), the happy message for commodity forecasters is that they need concentrate their efforts only on estimating expected prices over time. Precision information (e.g. on the standard error of a dated forecast price) will generally not be intrinsically useful in the sense that it should influence a project decision.

Other considerations, however, may serve to soften this 'hard line' conclusion. For the present purpose, it is assumed that the additional costs incurred in providing probability distributions (or some simple summary thereof like a coefficient of variation, standard error, high-density range, etc.) rather than just expected values (means) of prices are rather trivial (approximately zern). With this qualification. there must be virtue (albeit essentially unquantifiable) in providing more comprehensive probabilistic information to users of forecasts:

- (a) communication of judgmental data will be improved (users will know more of analysts' best assessment of the precision with which they think they are forecasting, thereby revealing the fundamental stochastic nature of the forecasting process);

- (b) users will be less 'surprised' when eventualities differ from forecast means as they surely must (this may be especially useful for monitors of balance of payments, as well as project monitors and evaluators); and

- (c) explication of probabilistic structures may lead analysts to make better estimates of mean prices, particularly when skewed distributions are involved, and there is a danger that modal (or, worse, 'conservative' or 'pessimistic') prices might be
- 39 -

issued in lieu of means).

Lest this enthusiasm for probabilistic forecasting get too unbridled, some warnings are in order:

- (a) if measurement of uncertainty is to be transmitted, all parties should be clear about what is being measured, especially with regard to any conditionality on forecasts of other uncertain explanatory variables or assumptions;

- (b) conditional forecasting precision will generally be the most appropriate to transmit to users, but this poses demands for forecasters of variables (like population, income, economic activity indexes and energy prices) that enter other commodity forecasting models to be explicit about the precision of their forecasts;

(c) in turn, all such intertemporal error modeling hinges crucially on 'adequate' representation of serially dependent time series and, since this is a subjective and imperfect art, there will always be a background of 'estimational uncertainty'; and (d) finally, a 'credibility gap' may develop over interpretation of forecast means when standard errors are very high (confidence intervals very wide), although this might be moderated through a sympathetic educational program. Relatedly, users should be encouraged not to 'misuse' probabilistic information when only best estimates of means are appropriate.