metadata: authors: 0: author: Maloney,William F.
docna: 0: docna: Evaluating emergency programs
docty: Policy Research Working Paper
subtopic: Social Analysis,Quality of Life & Leisure,National Governance,Government Policies,Health Care Services Industry,Labor Markets,Labor Policies,Public Sector Administrative & Civil
            Service Reform,Youth and Governance,Inequality,Democratic Government,Rural Labor Markets,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Poverty Reduction
abstracts: cdata!: Emergency programs are designed to
            soften the impact of economic crises-income shocks
            experienced by an entire community or country-on consumption
            and human capital accumulation. Of particular concern are
            poor people: as a result of inadequate savings or inadequate
            access to credit or insurance markets, the poor are unable
            to draw on resources from better times to offset a loss in
            income today. Further, the systemic nature of the shocks
            means that risk cannot be effectively pooled through local
            informal insurance mechanisms. Emergency interventions have
            included social funds, workfare programs, training programs,
            conditional transfers (linked to health center visits or
            children's school attendance, for example), and
            traditional direct, unconditional transfers in kind (such as
            communal tables or targeted food handouts). The author
            highlights some conceptual problems in choosing among these
            options and evaluating one program of a certain type
            relative to another. It argues that most such interventions
            can be thought of as containing both a transfer and an
            investment component and that their evaluation as emergency
            programs needs to more explicitly incorporate the
            intertemporal nature of their design. More specifically, the
            mandated investments in physical or human capital will
            benefit the poor, but only in the future-after the
            crisis-and their implementation diverts resources from
            alleviating present hardship. This needs to be reflected in
            the discount factor used to evaluate these investments.
            Maloney argues that the way emergency programs are financed,
            particularly the way the burden is shared between central
            and municipal governments, also has important implications
            for the criteria for evaluation. The analysis suggests that
            most conventional means of evaluating projects-net present
            value at market discount rates, labor intensity, cost per
            job created-may not be relevant or are at least ambiguous in
            the context of emergency programs. As a result, policymakers
            are left with few "hard" indicators with which to
            evaluate such programs. Maloney argues for an approach in
            which the policymaker weighs the appropriateness of
            deviations from the theoretically "ideal"
            benchmark program, which delivers a "smart"
            transfer costlessly to the target beneficiary, and discusses
            the arguments for or against these deviations. The modest
            goal of the proposed approach is to clarify the key issues
            and provide more solid grounding for the necessarily
            subjective judgment calls that policymakers will inevitably
            have to make.
repnme: repnme: Evaluating emergency programs
display_title: Evaluating emergency programs
listing_relative_url: /research/2001/12/1660271/evaluating-emergency-programs
topicv3: Political Economy of Reform m1326263 836,Poverty m1326687 695,Workfare Programs m1327197 1666,Public Administration m1327723 723,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Poverty Reduction
origu: Poverty Reduction & Econ Mngmt (LCSPR)
no_of_pages: 30
author_keys: 0: author_key: 93000
model_extractions: I 1iN **ICY** RrsVSARCH WORKING PAPER 2728

# Summary findings

Emergency programs are designed to soften the impact of economic crises-income shocks experienced by an entire comimunitv or country-on consumption and human capital accumulation. Of particular concern are poor people: as a result of inadequate savings or inadequiate access to credit or insurance markets, the poor are unable to draw on resources from better times to offset a loss in income today. Further, the systemic nature of the shocks means that risk cannot be effectively pooled through local informal insurance mechanisms.

Emergency interventions have included social funds, workfare programs, training programs, conditional transfers (linked to health center visits or children's school attendance, for example), and traditional direct, uLlconditionial transfers in kind (such as communal tables or targeted food handouts).

Maloney highlights some conceptual problems in choosing among these options and evaluating one program of a certain type relative to another. It argues that most such interventions can be thought of as containiing both a transfer and an investment component and that their evaluation as _emergency_ programs needs to more explicitly incorporate the intertemporal nature of their design. More specifically, the mandated investments in physical or human capital will benefit the poor, but only in the future-after the crisis-and their implementation diverts resources from alleviating present hardship. This needs to be reflected in the discount factor used to evaluate these investments. Maloney argues that the way emergency programs are financed, particularly the way the burden is shared between central and municipal governments, also has important implications for the criteria for evaluation.

The analysis suggests that most conventional means of evaluating projects-net present value at market discount rates, labor intensity, cost per job created-may _not_ be relevant or are at least ambiguous in the context of emergency programs. As a result, policymakers are left with few "hard" indicators with which to evaluate such programs. Maloney argues for an approach in which the policymaker weighs the appropriateness of deviations from the theoretically "ideal" benchmark program, which delivers a "smart" transfer costlessly to the target beneficiary, and discusses the arguments for or against these deviations. The modest goal of the proposed approach is to clarify the key issues and provide more solid grounding for the necessarily subjective judgment calls that policyinakers will inevitably have to make.

Tlhis paper-a product of the Poverty Reduction and Eco nomic Management Sector Unit, Latin America and the Caribbean Region-is part of a larger effort in the region to improve the design of safety nets. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Anne Pillay, room 18-104, telephone 202-458-8046, fax 202-522-2119, email address apillay@worldbank.org. PolicyResearch WorkingPapers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at wmaloney@worldbank.org. December 2001. (24 pages) _rhe Policy Research WVorking Paper Series disseminates the findings of work in progress to encourage the exchange of ideas ahout development issues. An obhective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the itames of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper Ire entirely those of the authors. The) do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent._ Produced by the Policy Research Dissemination Center
**Evaluating Emergency Programs: Intertemporal and Financing Considerations***

William F. Maloney LCSPR

> *I'm grateful to Omar Arias, Judy Baker, Wendy Cunningham, Norman Hicks, Polly Jones, Vicente Paqueo, Martin Ravallion, Norbert Schady, Miguel Szekely, Judith Tendler, Quentin Wodon, and Carlos Eduardo Velez for helpful discussions
# 1. **Introduction**

Emergency programs seek to mitigate the impact of economic crises- income shocks experienced by an entire community or country- on consumption and human capital accumulation. Of particular concern are the poor who, due to inadequate savings, or inadequate access to credit or insurance markets, are unable to draw on resources from better times to offset a loss in income today. Further, the systemic nature of the shocks means that risk cannot be effectively pooled through local informal insurance mechanisms. Emergency interventions to date have included workfare programs, certain varieties of social funds, conditional transfers (eg. conditional on child school or health center attendance), training programs as well as traditional direct unconditional transfers in kind (communal tables, targeted food hand-outs, etc. ). 1 This article highlights some conceptual problems in choosing among these options, and evaluating one program within a genre vs another. It first argues that most can be thought of as containing both a transfer and an investment component, and that their evaluation as _emergency_ programs needs to more explicitly incorporate the intertemporal nature of their design. More specifically, the mandated investments in either physical or human capital will benefit the poor, but in the future, after the crisis, and their implementation diverts resources away from alleviating present hardship. Second, it argues the way emergency programs are financed, in particular how the burden is shared between central and municipal governments, also has important impacts on the criteria for evaluation.

The analysis suggests that most conventional means of evaluating projects generally, and emergency programs in particular- net present value at market discount rates, labor intensity, cost per job created- may not be relevant or are at least ambiguous

> ' For brief review of some programs see Khadiagala (1995), Lustig (1997), Subbarao (1997), Hicks and Griffin (1998), Klugman (1999), Baker (2000), Wodon and Hicks (2000). Social Investment Funds frequently started as emergency programs but also exist as programs to develop infrastructure in non-crisis

2
in this context. As a result, policy makers are left with few "hard" indicators with which to perform the nonetheless essential evaluations of their programs, and this article will notably fail to provide any new ones. It does, however, argue for an approach where the policy maker weighs the appropriateness of deviations from the theoretically "ideal" benchmark program, and discusses the various arguments in favor or against these deviations. The necessarily modest goal is to clarify the key issues and provide more solid grounding for the necessarily subjective judgement calls that policy makers will inevitably have to make.

## **What is a Crisis?**

Emergency programs or safety nets more generally seek to mitigate the impact of sudden falls in income below their normal or "permanent" level. There is a large literature on the importance of the ability to smooth consumption across good and bad periods in LDCs (see, for example, Case, 1995, Townsend 1995, Morduch 1995, 1999, Besley 1995) and the intuition can easily be captured in a standard intertemporal model. For simplicity sake, we imagine an individual or community that receives income each year and makes consumption decisions based on comparing utility or welfare across two periods of time according to a linear function where total welfare across time is the discounted sum of welfare in each period.<sup>2</sup>

# _U = U(CI) + U(C2)_

U has the usual properties that welfare increases as consumption increases and that it increases at a decreasing rate. For now, we also assume that the individual or community can either borrow or invest, either at the interest rate R. Total wealth available to be distributed across both periods is therefore the discounted value of income across both periods periods. We are concerned only with their role in an emergency context. See Stewart and van der Geest (1995) Lustig (1997) Comia (1999) and Tendler (2000), and for more detailed evaluations.

3
_U = U(C,) + U((Yi - Ci)(l + R) + Y2)_ In this case where the poor can easily smooth their consumption, welfare across the entire period is maximized where If the interest rate R=O, an additional dollar spent on consumption in each period should yield exactly equal additional utility. This is the central insight behind the traditional permanent income hypothesis -- people will smooth their consumption over lottery winnings and job loss so as to maintain the welfare arising from an additional unit of consumption constant over time.

Adverse income shocks are thought particularly damaging to the poor because they lack this ability to smooth consumption across time. That is, their present consumption exactly equals their reduced present income: a fall in income during a crisis in period 1 leads to an equivalent fall in CI. Analytically, this fall in consumption pushes the marginal utility of an additional dollar today far in excess of that in the future. Put more concretely, tomorrow they may be poor, but today they are hungry and hence an additional calorie is valued much more now.<sup>3</sup> This effect is magnified during crises which can be thought of as adverse shocks that affect many members of a community simultaneously. Not only are workers likely to be less able to find new employment or recover a temporary income fall, but whatever

> **2** For simplicity we assume the discount factor to be zero.

> _3_ Morduch (1995) calculated that, under the assumption of moderate risk aversion of households and a coefficient of variation of income is 40%, a not unusually high number, then households would be willing to give up 16% of their income to achieve perfect smoothing, a large amount for households living near substistance levels.

4
informal safety nets exist-community lending or grants- are vastly weakened by both the reduced income of the community and the simultaneous demands put on it.<sup>4</sup> From a purely theoretical view, the gains in social welfare arising from redressing the impacts of crises are not distinguishable from those of economic growth-in both cases the ability to smooth by borrowing allows a shifting out of the social utility curve. The fact that in the former case we borrow from the future to raise present utility and in the latter we may raise both is immaterial from the point of view of raising total discounted social welfare. It is as reasonable for governments and the World Bank to lend for safety net programs that mitigate falls in present income as for gains in future productivity.

# _Crisis-alleviation Programs_

Government's role can be seen as redressing the effects of the market failures that lead to missing financial or insurance markets necessary for this smoothing. States could simply lend money to the affected individuals or regions, but in general they have chosen to give grants that serve to increase the resources available today. This may be due to all the complexities that led to the absence of credit and insurance markets in the first placecostly monitoring and absence of collateral for example. Or it may be that society believes that some share of the socially optimal transfers to augment the permanent income of the poor are best earmarked for times when the necessary smoothing is impossible and hence the welfare gain from the transfer is highest.

Whatever the reason, if the amount of the grants does not fully offset the fall in income, the simple view sketched above suggests that the largest effect on welfare would be gained from dedicating them entirely to consumption today. Following this logic, emergency programs should be thought of as vehicles for transfers and the central measure of their success is their transfer efficiency -how much of resources earmarked for the emergency program actually arrive at the target population during the crisis. As

> 4For example, Korean households may have been less able smooth systemic shocks (the crisis) as well as idiosynchratic shocks. World Bank (2000).

_5_
an idealized benchmark, we might imagine the government deploying "smart" cash transfers that would perfectly and costlessly target those most suffering from a fall in incomes. In this case, the transfer efficiency would be 100%.

However, most of the programs envisaged today require that some fraction be dedicated to investment projects in either physical or human capital that will yield benefits in the future: _Workfare programs_ use part of the government transfer for salaries, and part to finance materials for physical investment projects. The Chilean Programa de Empleo Minimo (1975-1987) and Programa Ocupacional para Jefes de Hogares (1982-1987) employed unskilled labor in projects ranging from food preparation, drainage and road maintenance to well digging with an emphasis on high labor intensity. The Argentine Trabajar and the Colombian Manos a la Obra (2000 to present) program envisioned more permanent investment projects serving the poor target populations program. Perhaps intermediate between the two, the Brazilian Drought project (1998) covered small-scale irrigation projects, water and sanitation projects, building and maintaining community facilities (schools, health clinics, parks), and rural road construction projects; and the Mexican Program de Empleo Temporal (1995-present) covers irrigation and clearing of land, paving roads, and installing water and sewerage systems.<sup>5</sup> We can also include in this category SIFs specifically designed as emergency programs.<sup>6</sup> In all cases, some fraction of available government resources is diverted to pay for non-labor inputs.

_Conditional transfer programs_ can be thought of as payments for the labor of both mother and child, and associated costs (books, transport, uniforms etc.) necessary to an investment project in human capital with potential returns being future income to the family and well-known social externalities. Programs like Progresa in Mexico have, to date, been thought of primarily as promoting human capital accumulation. But they may

> _5_ I am grateful to Joachim Von Amsberg for discussions on the Brazilian program. See Wodon "Government Programs and Poverty in Mexico" (1999).

> 6See Comia (1999) for a discussion of Social Emergency Funds (SEFs) vs Social Investment Funds (SIFs) vs Social Action Programmes (SAPs).

6
also be expanded or made more generous in crises situations to serve as a means of transferring income as is currently contemplated in Colombia.

_Training programs_ that offer a living subsidy to the worker and cover training costs can be thought of as fundamentally similar to conditional transfers. These appear to some degree in the Brazilian Drought program, the Mexican Training Scholarships for the Unemployed (Probecat), and potentially in the Colombian emergency program. Again, the number of participants can be expanded during crises.

Such transfer cum investment programs are, on the surface, appealing since they provide both an emergency transfer and a project yielding future benefits to the target population. Yet, from a purely theoretical point of view, they compare very unfavorably to our benchmark and, in fact, the investment component is probably rarely justifiable. This follows from the fact that for emergency programs the correct discount factor to make future benefits from the investment component commensurate with those now is not the market rate of interest, but rather the "consumption rate of interest"-how much we value additional consumption in the future vs today. 7 Given the traumatic, but generally transitory nature of crises, the answer is likely to be "very little" and as a result, very little of potential transfers for today's consumption should be reduced to finance investment projects that will yield benefits in the future.

More formally, this effect can be captured in the first two arguments on the right hand side of equation 2a which argue that the social benefits arising from an additional dollar of government spending (G) on the emergency project are comprised of the marginal utility arising from the share of the an additional dollar of G that is transferred to families in the crisis period (t), plus the marginal utility arising from return to investment (1 +R)(1-t). The subscripts

> ' See Squire and van der Tak (1995) p 27-28, 69.

7
denote marginal utility during the crisis period, c, and marginal utility in normal times, n. The third term we will return to in a moment. Equation 2b rewrites this as marginal social benefit measured in terms of "units of crisis alleviation"- the marginal benefit of a dollar during the crisis period.

What is now clear is that in calculating the value of the program to the target population, we cannot simply add the income flows from the investment project discounted at the market rate to the cash transfer. This would imply that the marginal utility of a dollar in crisis and non-crisis periods is equal which the intertemporal analysis above suggests is implausible.

In fact, the investment components represent the opposite of smoothing: every dollar spent on materials for a project that will yield future benefits takes a dollar away from present consumption. The effect is very large. In the case of workfare programs and some SIFs, often the share of the budget that goes to wages, and hence current consumption, is below **50%.8** At a very basic level the government is saying: "I know you're hungry today, so here's $100 dollars, half of which you can consume today, but the other half (augmented by the gains of investing) you will consume tomorrow when you're less hungry." For the poor to have decided to make this allocation on their own, equation (1) tells exactly what the rate of return would have to be. The project must give a return equal to the value of an additional dollar of consumption when the worker is hungry relative to the benefit of an additional dollar in normal times. The ratio of utilities preceding the returns to investment in the second term of equation 2b is arguably the "consumption" discount factor that should be used in evaluating these projects and it is likely to be extremely high. Middle class consumers in good times in the US are

> **8** IDB (1997) evaluations put the labor share in SIFs at 30%. Cited in Tendler (2000).

8
comfortable paying rates of interest of above 20% on credit card debt. Borrowing rates from moneylenders in rural areas can easily reach 5% a month or 80% yearly. A consumption rate of discount of 30% seems plausible and will render all but the best physical investment projects unattractive. This is probably also the case with human capital investments envisaged by Progresa or similar programs which would yield returns of perhaps 7-13%.9 In fact, by virtue of being conditional, such transfers imply that households would not have invested in human capital were they unconstrained.

Since future benefits of the investment components are heavily discounted, the primary measure of program quality again, is the transfer efficiency, which, as we've seen, can be far below 50%. In sum, it is probably impossible, within our simple framework, to justify any of the transfer cum investment options relative to a straight unconditional transfer as emergency measures.

# _Why Might Policy Makers Want an Investment Component?_

Why then, do rational governments and international organizations elect these strategies over unconditional transfers? More generally, when designing emergency programs, what factors may lead us or force us to depart from the Platonic "smart" transfers described earlier? There are several possibilities, many legitimate, yet virtually all of which are problematic from an evaluation point of view because first, the benefits may not be easily measurable or second, they represent objectives that imply tradeoffs with emergency relief. As a result, no a priori evaluation of the importance of these considerations is possible and policy makers are left to make some educated, if heroic efforts in that direction. What follows is not an exhaustive review of justifications for investment components. However, it may provide something of a check list to see which "ring true" in a particular context as reasons to depart from a pure transfer.

_1. Anti-poverty vs. crisis programs:_ Most obviously, if, in fact, the situation is really one of ongoing poverty rather than a short lived- crisis, then the appropriate discount 9 Wodon (1999).

9
factor is likely to be closer to the market rate of interest and investment programs can be justified on their own merits. An infrastructure development SIF with volunteered labor would be more appropriate in this case than a high labor intensity emergency program.

2. _Unmeasurable returns to investment:_ Unmeasurable benefits may raise the social return to a level where investment based programs are optimal, even if evaluation at the consumption rate of discount suggests they are not. As an example, income transfers without accompanying effort by recipients may be socially damaging or cause social values to "depreciate" over the long run. The shift from welfare to workfare in the US as well as Colombia's concern with "asistencialismo" or dependency creating programs suggest that these effects are non-negligible. Having the transfer associated with an infrastructure project, training program, or human capital investment program may mitigate these concerns. On the other hand, they maybe less likely to merit attention across the short time horizon of a crisis.

There may also be developmental synergies and accumulation of social capital that emerge from the process of putting together projects with NGOs that would yield dividends in the future. (Isham et al.1995, Zyl, et al 1995, Bain and Hicks 1998, Fox and Gershman, 1999). Again, this is an issue of building long run social capital which, as any other investment, must be evaluated at the consumption rate of discount. Further, the existing literature is not unanimous that such synergies do emerge (see Tendler 2000).

3. _The poor are short sighted and/or externalities exist:_ Arguments for conditional transfers that seek to prevent the depreciation of human capital implicitly assume that families in crisis wouldn't do so on their own. This might be because families are irrational. It may also be the case that there are social externalities from education that cause the private return to deviate from the social. This, again, is an ongoing 10
empirical question and the evidence to date suggests that families may be more protective of their human capital than first imagined.'<sup>0</sup> _4. Raising welfare through investment is superior to raising welfare through smoothing._ It may be that governments are far more willing to come up with funding for a project that actually creates an asset rather than one that simply transfers or pays for work on recurrent projects." As noted above, from an theoretical point of view this is probably indefensible: to maximize discounted social utility, it is desirable to minimize the diversion of resources to the future. But governments may want "something" to come from their spending or perhaps from a political point of view, it may be more advantageous to deliver 5 jobs and an infrastructure project rather than 10 jobs. Or donors may believe that investments may make it unlikely that there will be recourse to their resources for emergency purposes in the future.

Ravallion (1991) has compellingly argued that such political economy considerations may be central to justifying workfare programs. Only half of government resources may go to current transfers, but if the investment component leads to overall program being more than double in size, then this would justify a transfer cum investment vs. a direct transfer program. More specifically, he argues that it may be precisely that the created assets offer benefits beyond the target poor population that generates broader support for the schemes, as was the case in the Maharashtra Employment Guarantee Scheme. In this case total transfers as a fraction of the resources likely without the workfare component, rather than as a fraction of total resources probably captures the quality of the project better. The general importance of these effects, remains an open empirical question, however. While it is clear that Colombia and Argentina put heavy weight on the investment component, Chile and Mexico put far less in their nonetheless substantial programs.

> **10** De Ferranti et al. (2000), summarizing recent studies, argue that in fact, the poor try to protect human capital during downturns far more than is usually thought.

> **'** My thanks for this thought to Martin Ravallion, DEC (1999). See also Gelbach and Pritchett (1997).

11
Further, recurrent concerns with low project quality both in workfare programs and SIFs (Khadiagala 1995, Tendler 2000) and the generally disheartening evaluations of training programs12 suggest that governments and donors may need to take a harder look at what they are getting. And overall program quality may not be enhanced by the very political economy considerations that are being considered here. Tendler has argued that distributive programs with many spatially dispersed small projects are an excellent vehicle for patronage as she argues was the case in Argentina, Mexico and Peru. As Schady (2000) argues for Peru, demonstrable distortion of SIFs for electoral purposes does not completely undermine their distributive impact, or, we might guess, project quality, but it must adversely affect both. The good news is that, even if the project is of low quality, the high consumption rate of discount implies that this is largely irrelevant to the overall value of the emergency program- we can think of the investment costs as simply part of the cost of the transfer. On the other hand, these costs are so high as to warrant very careful evaluation of political economy arguments, and a search for alternative vehicles.

Another consideration that was of concern to Asian governments during the crisis is that transfer programs may generate constituencies during the crisis who then fight for their preservation after. Tying the transfer to a project which has a very visible ends may politically facilitate scaling back the program when it is less necessary (World Bank 2000). However, Tendler (2000) argues that the reverse dynamic has occurred in Latin America. The autonomous and rule-free units set up under SIF emergency programs often sought to perpetuate themselves arguing, counter to her findings, that they were better agents for delivering small scale rural project than traditional government bureaucracies.

_5. Investment projects generate employment._ In the cases where workfare programs are advocated, several reasons are put forward for why this is an important goal.

**12** Probecat in Mexico probably should be seen largely as a transfer program with limited impact on human capital accumulation. Wodon and Minowa (1999) control for endogeneity of program participation and

12
_i. The dignity of labor._ Conventional economic theory sees consumption as the bottom line of welfare calculations and work as a "bad" suffered by humanity in order to consume. However, some have argued that employment in itself is important, both by providing self-esteem for the worker or by avoiding family violence, alcoholism, and loss of motivation resulting from having unoccupied workers sitting around the house.

Theoretically this adds the third term in equation 2 which captures the gain in social utility arising from moving the crisis level of employment toward full employment. Present welfare is affected both by consumption per se and by employment. Unfortunately, an employment position is generally a more costly vehicle for transferring income than a direct transfer and this implies that there is also a tradeoff between the first and third arguments. Theory suggests that the relative amounts of transferred income vs. employment created should be determined by the relative marginal utilities arising from each and their relative cost. The problem is, of course, that it is impossible to quantify the social benefit arising from the creation of one additional job relative to transferring an additional dollar of consumption. One strategy might be to verify that the target population would rather work for half the transfer than receive all of it while not working. That this is true remains to be verified by careful empirical work in the field.

There is also something odd about the fact that discussions of safety nets for better-off or formal workers revolve around personal accounts (forced savings so workers may smooth across time) or insurance schemes. Neither envisages work per se as a necessary concomitant of smoothing consumption. In fact, we can think of transfers as the limiting case of the increasingly popular personal accounts where the worker's contribution goes to zero. Here again, we get into the moral hazard type questions implicit in (1) above. But it is also the case that, find, contrary to earlier evaluations, no impact on unemployment or wages.

13
by definition, emergency programs are dealing with a temporary absence of jobs and, again, the long run incentives may not be central.

Ravallion (1991) again introduces a potentially important political economy issue related to the moral hazard issue above. Donors may think that their beneficiaries should work and require reassurance that the recipients will no be made so much better off that they would be discouraged from taking the necessary steps to escape poverty in the future.

_ii. Job creation may be important symbolically._ At a national level, an "employment" program may have symbolic value in attacking the most obvious indicator of crisis and may ease acceptance of macro-stabilization programs, which may have longer-term positive impacts on the poor. We could, in theory, also treat this as part of the unmeasured return to the investment. Several analysts, citing the very limited coverage and impact of the SIFs on employment and the poor, suggest that the more symbolic motivation is weighted heavily by sponsors of SIFs<sup>13</sup> and that these programs may deflect attention away from more effective interventions or adjustment programs with fewer negative impacts.

_iii. Targeting:_ It may, in fact, be difficult to target pure transfers. Progresa type schemes are very intensive in information and their focus on permanent poverty may mean that they may miss those thrown into poverty by the crisis.'<sup>4</sup> The selfselection that arises out of the sub-market wage policy of the employment schemes provides an alternative means of targeting that is superior on both counts. In this case, the overhead costs of a workfare scheme may thus be seen as part of the cost of targeting. However, it is important to stress that investment components per se are not necessary to get the targeting benefits of workfare and, given the discount factor, really only increase the costs of targeting. Keynes' See Stewart and van der Geest (1995) Comia (1999) and Tendler (2000).

14 As Ravallion (1991) and others have stressed, the imperfect measures of living standards of potential recipients lead to costs that, in a world without these constraints, would be deadweight losses.

14
digging and filling of holes, or any other recurrent project-gardening, repainting, - will target and will cost less than an investment project per dollar transferred.

## **II. Problems in Evaluating Emergency Programs**

# **_1. The Problem_** _of_ **_Multiple_** _Objectives:_ **_Resisting the Temptation_** _of_ **_Conventional Evaluation Measures_**

Equation 2b succinctly demonstrates the underlying problem of evaluating emergency programs against a pure transfer. First, each of the three possible objectives, transfers to affected groups, investment project development, and employment generation has an essentially unquantifiable weighting in the benefits function. To sum up total benefits requires knowing the consumption discount factor, any unobserved returns to investment, and the social benefit of an additional unit of employment. Without these, we are largely lost and the temptation is to revert to conventional measures of success of the individual components. These, however, are _faux amis_ and alone tell us little about the quality of the project: a. Comparison to a pure cash transfer program is not reasonable since project and to a lesser degree, employment generation divert resources away from immediate transfers. Further, it is difficult to capture the political economy issue of what the total size of the project would be, and hence the magnitude of the transfer, in the absence of a project.

**Table 1: Impact of Labor Intensity of** b. Comparison to pure job **Projects on Cost per Job-Day** Labor Share in Costs $/day-job creation programs is not reasonable since investments 0.3 16.7 projects raise the cost per job 0.5 10.0 relative to "make work" type 0.7 7.6 programs. Table 1 shows that 1 _5_ Note: Based on a daily wage of $5 how "costly" a program iS per -_-______

15
worker depends greatly on how much capital or other materials is combined to construct infrastructure projects.

c. Comparison to pure infrastructure projects is unreasonable since the emphasis on employment generation means least-cost production methods may not be used.

Evaluated against a pure transfer, a pure job creation program, or human or physical capital investment projects, emergency programs with investment or job creation components will likely come up short. Whether this type of hybrid program is deemed a success depends fundamentally on the weights put on each objective in the social welfare function. _15 Since by nature these weights will be highly subjective, virtually any program can bejustified_ _An example. Trabajar in Argentina._ Ravallion's evaluation of the Trabajar workfare program in Argentina provides a concrete example of the difficulty of evaluating emergency programs.

1. His emphasis on high labor intensity projects to increase present- transfers, to the detriment of total project value implies a consumption discount factor above the usual market rate. This implies that a Trabajar infrastructure project will appear inefficient when compared to those without the transfer goal.

2. His calculation of the total benefit (transfer + investment) to the poor/per $1 (-.4) is double his calculation of just the wage transfer componantl$1 (-.2). Compared to the cash transfer programs in Eastern Europe he cites (.19-. 58) or targeted food subsidies (.19-. 69), Trabajar does not do particularly well as a transfer program.

> _15_ The judgemen **t** hat workfare programs are an essential element of an emergency programn fundamentally is a judgement on the value of labor generated-it is easy to imagine a value that would lead to a corner solution with no labor component.

16
3. On a cost per job basis, Trabajar at a GDP adjusted 1.2 $/day/job is substantially more expensive than Chile's Programa de Empleo Minimo at .4. But as table 1 shows cost per job is inversely related to the labor intensity of the project and this may only reflect that the Chilean program was fundamentally a job creation program and did not heavily value the creation of durable investment projects. Again, both cost per job and labor intensity may be misleading criteria.

The lesson is that, despite the fact that Trabajar may look like a poor performer when measured against programs that have only one of its three possible goals as its objective, it is impossible without a knowledge of the unobservables in equation 2 to conclude that it is a bad or good emergency program. For the same reason, it is hard to assert with confidence that it is "best practice".16 _So, what do we do?_ With the traditional "hard" measures of project quality shown to be misleading, how can we evaluate emergency programs? Although there are no easy or air-tight alternatives, policy makers should probably begin by hewing as closely as possible to our platonic ideal of the smart transfer and view the program as the minimum deviation necessary to transfer the most resources effectively today. Although crude, working through the considerations listed above and weighing their relevance to the situation at hand may help in designing the least cost vehicle for transferring resources. For example, if the government is not set on creating infrastructure projects and no appreciable additional mobilization of resources is likely to occur by undertaking them, then transfer efficiency can be raised by avoiding them. If information problems are large and targeting is a problem, then the costs of a workfare scheme may be unavoidable.

> 16 See for example IBRD 1999, Klugman 1999, Hicks and Griffin 1998.

17
# **_2. The Importance of Financing Structure_**

If we knew exactly how to weight the elements of equations 2a and 2b and the programs were faced with a fixed budget, the socially optimal combination of transfers, investment and job creation could be calculated. In practice, however, emergency programs have been designed in such a way that budget is to a large degree endogenous to the design of the program. This further complicates the process of evaluation in significant ways.

_1. How are costs shared between local and central governments? Is labor intensity relevant?_ In both and Trabajar and in some of the many varieties of Social Investment Funds (SIF), higher labor intensity and hence lower cost per job created is considered better (Goodman et al. 1997, Ravallion 1998). However, in both types of programs, there is often system of implicit or explicit counterpart funding by the local municipalities/communities of resources contributed by the central government. In some SIFs (Peru for example), the central government provided the financing for materials and the municipality/community the financing for labor. As discussed earlier, since the labor is often volunteered, this structure has little value as an emergency program and in Trabajar and now Manos a la Obra, the financing is reversed. In both funding schemes, the total resources available vary with the labor intensity of the project, and this has important implications for evaluation. Consider three possible financing designs:

1. If the total financing is fixed, perhaps by a constant rate of cofinancing between central and municipal governments/communities, we maximize equation 2a subject to a fixed financing constraint. Abstracting from the value of employment (the third argument) for now, the optimal labor intensity depends primarily on the relative value placed on present transfers to labor vs the value of the projects as discussed in the first part of the paper. The correct discount factor to use in

18
project evaluation is the consumption discount factor above and this will lead, as a rule of thumb, to an emphasis on high labor intensity projects.

2. If the central government provides materials and municipalities/communities provide _paid_ labor, as in some SIF, then higher labor intensity means simultaneously more resources devoted to the program and to the total value of the investment projects, and a larger share of total resources devoted to labor presently. High labor intensity is, without question, good.

3. If the central government provides labor, and municipalities/communities provide materials as in the case of Trabajar in Argentina and Manos a la Obra in Colombia, then the logic is reversed. The level of transfers to workers is fixed by the contribution of the central government. Increasing the labor intensity _decreases_ the contribution of the municipalities for each central dollar contributed and hence the total value of investment projects and future benefit to the poor. Since _there is no impact on present transfers,_ the correct discount factor may well be the standard economy wide interest rate. Under this financing scheme, it is not clear why we would prefer more labor intensive projects.

In sum, the criteria for evaluation, and in particular, the discount factor appropriate to the evaluation of investment components depends on the structure of financing.

The financing problem for conditional transfers can be framed similarly to the workfare program. The households (municipalities) provide the complementary factorsschool books, uniforms etc. However, because these costs offset the transfer and do not augment it as is the case with workfare programs, we should think of the central government as purchasing both "materials" and labor which would be the same as if we make no distinction between municipal and central resources (case 1 above). In the event that there is not excess capacity in schools, health clinics or training centers during the crisis, it is also necessary to include all additional supply side costs when comparing them to workfare programs.

19
# _2. Additionality_

The traditional issues of additionality are important in evaluating emergency programs. Any central government subsidy to the municipal governments can lead to a "crowding out" of existing resources. If we suppose, for example, that the municipal governments had planned to implement a number of projects generating a certain amount of employment, it is difficult to be sure that an employment subsidy as envisaged in Trabajar or Manos a la Obra doesn't merely free up municipal resources for other purposes with no net gain in employment. In Colombia, for example, municipal investment programs are approved yearly at the central government level and, for the first year of the emergency program, additionality could be guaranteed. However, in subsequent years, municipal governments can redirect investment resources and offset the impact of the program.<sup>17</sup> Theoretically, this is potentially a very serious issue and there is little existing evidence that would assure that it isn't important in practice. Nor does Bank experience with additionality issues suggest ready contracting mechanisms that would eliminate the problem. One impractical solution is careful monitoring of all municipal budgeting and investment decisions. A second is simply that that projects be kept effectively separate from municipal finances, perhaps through complete financing by the central government. This, however, eliminates much of the leverage arising from municipal contributions.

# **Conclusion:**

Most emergency programs- workfare programs, conditional transfers, or training programs- employed to date include an investment component, either an infrastructure

> " As a particularly perverse twist, if the municipality cuts funding to direct or in-kind transfers (comedores populares etc) in order to pay for intermediate inputs for investment projects, we may be taking away the safety net during the crisis to finance future consumption. My thanks for this thought to David Rosenblatt, LCSPR (1999)

20
project, or the protection or generation of human capital. This implies that their evaluation needs to incorporate their intertemporal dimension and, particularly, the very high rate of discount that families in crisis are likely to have.

Though on purely economic grounds, most of the investment components are likely to be unjustifiable when evaluated with the appropriate crisis discount factor, there may be unmeasurable returns to investment, social externalities, political economy effects, or in the case of workfare programs, an intrinsic value to employment itself that may make them justifiable. However, the difficulty of measuring the incidence and social importance of these considerations, and the trade-offs they often imply with emergency relief, makes disciplined evaluation of these projects elusive. Evaluation is further complicated by the fact that the way emergency programs are financed, and in particular, how the burden is shared between central and municipal governments. The goal of this article has therefore been to attempt to highlight and discuss the factors that policy makers might bear in mind while designing such programs in the hope of making the unavoidable subjective judgement calls somewhat better grounded.

21
# **References:**

Bain, Katherine and Norman Hicks (1998) "Building Social Capital and Reaching out to Excluded Groups: The challenge of Partnerships" Poverty Reduction and Economic Management Group, IBRD.

Besley, Timothy (1995) "Nonmarket Institutions for Credit and Risk Sharing in LowIncome Countries," _The Journal of Economic Perspectives,_ Vol. 9, Number 3, Summer 1995.

Case, Anne (1995) "Symposium on Consumption Smoothing in Developing Countries," _The Journal of Economic Perspectives,_ Vol. 9, Number 3, Summer 1995.

Cornia, Giovanni Andrea (1999) "Social Funds in Stabilization and Adjustment Programmes, Research for Action working paper 48, The United Nations University.

De Ferranti, David, Guillermo Perry, Indermit S. Gill and Luis Serven (2000), _Securing our Future in a Global Economy,_ World Bank Latin American and Caribbean Studies Viewpoints, Washington.

Fox, Jonathan, and John Gershman (1999), "Investing In Social Capital? Comparative Lessons from Ten World Bank Rural Development Projects in Mexico and the Philippines," mimeo, UC Santa Cruz.

Gelbach, Jonah B. and Lant Pritchett (1997), "More for the poor is less for the poor: the politics of targeting," Policy research working paper 1799, The World Bank.

Goodman, Margaret, Samuel Morley, Gabriel Siri and Elain Zuckerman (1997) "Social Investment Funds in Latin America: Past Performance and Future Role.

Hicks, Norman and Charles Griffin (1998) "Responding to Social Protection Needs during an Economic Crisis: A Note on Possible Actions for LAC." Mimeo IBRD.

Inter-American Development Bank (1997) Social Invesment Funds in Latin America: Past Performance and Future Role, by Margaet Goodman et al. Evaluation Office, Social Programs and Sustainable Development Deparment, March, Washington, DC.

Isham, Jonathan, Deepa Narayan, and Lant Pritchett (1995), "Does Participation Improve Performance? Establishing Causality with Subjective Data," _World Bank Economic Review_ 9:175-200.

Jalan, Jyotsna and Martin Ravallion (1999) "Estimating the Benefit Incidence of an Antipoverty Program by Propensity Score Matching." mimeo, IBRD.

22
Jorgensen, Steen, Margaret Grosch, Mark Schacter (1992), _Bolivia's Answer to Poverty, Economic Crisis, and Adjustment: the Emergency Social Fund,_ IBRD Regional and Sectoral Studies.

Khadiagala, Lynn S. (1995) "Social Funds: Strengths, Weaknesses, and Conditions for Success." World Bank Education and Social Policy Discussion Paper #52.

# Klugman, Jeni, (1999) "Social Safety Nets and Crises." Mimeo, IBRD PREMPOV

IBRD (1999) "Managing the Social Dimensions of Crises: Good Practices in Social Policy" mimeo.

Lustig, Nora (1997), The Safety Nets which are not Safety Nets: Social Investment Funds in Latin America, mimeo, IDB.

Ministerio de Trabajo y Seguridad Social (1999) "Evaluacion Ex-Post de los Proyectos Financiados por el Programa Trabajar II," Unidad Coordinadora Central del Programa Trabajar II, Buenos Aires, Argentina.

Morduch, Jonathan (1995) "Income Smoothing and Consumption Smoothing," _The Journal of Economic Perspectives,_ Vol. 9, Number 3, Summer 1995.

Ravallion (1991) "Reaching the Rural Poor Through Public Employment: Arguments, Evidence, and Lessons from South Asia." _World Bank Research Observer_ 6:2 153-175.

# Ravallion, Martin (1998) "Appraising Workfare Programs" mimeo, IBRD

Ravallion, Martin and Quentin Wodon (forthcoming), "Does Child Labor displace Schooling? Evidence on Behavioral Responses to an Enrollment Subsidy, _Economic Journal._ Ravallion, Martin (2000), "Are the Poor Protected from Budget Cuts: Theory and Evidence for Argentina." Mimeo, Regional Study on Risk, LAC.

Schady, Norbert R. (2000) "The Political Economy of Expenditures by the Peruvian Social Fund (FONCODES), 1991-1995, _American Political Science Review_ 94:2 289304.

Squire, Lyn and Herman G. van der Tak (1995), _Economic Analysis of Projects,_ Baltimore: Johns Hopkins University Press.

Stewart, Frances and Willem van der Geest (1995) "Adjustment and Social Funds: Political Panacea or Effective Poverty Reduction? Employment Papers 2, Geneva: ILO (employment department).

23
Townsend, Robert M. (1995), "Consumption Insurance: An Evaluation of Risk-Bearing Systems in Low-Income Economies" _The Journal of Economic Perspectives,_ Vol. 9, Number 3, Summer 1995.

Wodon, Quentin and Mari Minowa (1999), "Training for the Urban Unemployed: A Reevaluation of Mexico's Training Program, Probecat" in World Bank, _Government Programs and Poverty in Mexico._ Washington.

Wodon, Quentin and Norman Hicks (2000), "Protecting the Poor During Crises through Public Spending? Framework and Application to Argentina and Mexico," mimeo, Poverty Group, Latin and the Caribbean Region, World Bank.

World Bank (2000), _East Asia, Recovery and Beyond,_ Washington, D.C.

Zyl, Johan van, Tulio Barbosa, Andrew N. Parker and Loretta Sonn (1995), "Decentralized Rural Development and Enhanced Community Participation" 24
**Policy Research Working Paper Series**

|**Title**|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2706 Household Income Dynamics<br>in Rural China|Jyotsna Jalan<br>Martin Ravallion|November 2001|C. Cunanan<br>32301|
|WPS2707 Financial Intermediary Development<br>and Growth Volatility: Do<br>Intermediaries Dampen or Magnify<br>Shocks?|Thorsten Beck<br>Mattias Lundberg<br>Giovanni Majnoni|November 2001|A. Yaptenco<br>38526|
|WPS2708 Accountability and Corruption:<br>Political Institutions Matter|Daniel Lederman<br>Norman Loayza<br>Rodrigo Reis Soares|November 2001|P. Soto<br>37892|
|WPS2709 Explaining Leakage of Public Funds|Ritva Reinikka<br>Jakob Svensson|November 2001|H. Sladovich<br>37698|
|WPS2710 Breaking up the Collective Farm:<br>Welfare Outcomes of Vietnam's<br>Massive Land Privatization|Martin Ravallion<br>Dominique van de Walle|November 2001|C. Cunanan<br>32301|
|WPS2711 Identifying Class Size Effects in<br>Developing Countries: Evidence from<br>Rural Schools in Bolivia|Miguel Urquiola<br>|November 2001|H. Sladovich<br>37698|
|WPS2712 Contract Risks and Credit Spread<br>Determinants in the International<br>Project Bond Market|Mansoor Dailami<br>Robert Hauswald|November 2001|W. Nedrow<br>31585|
|WPS2713 Causes of Inequality in Health:<br>Who You Are? Where You Live?<br>Or Who Your Parents Were?|Adam Wagstaff<br>Pierella Paci<br>Heather Joshi|November 2001|H. Sladovich<br>37698|
|WPS2714 On Decomposing the Causes of<br>Health Sector Inequalities with an<br>Application to Malnutrition Inequalitie<br>in Vietnam|Adam Wagstaff<br>Eddy van Doorslaer<br>s Naoko Watanabe|November 2001|H. Sladovich<br>37698|
|WPS2715 Paying for Health Care: Quantifying<br>Fairness, Catastrophe, and<br>Impoverishment, with Applications to<br>Vietnam, 1993-98|Adam Wagstaff<br>Eddy van Doorslaer|November 2001|H. Sladovich<br>37698|
|WPS2716 Does Foreign Bank Penetration<br>Reduce Access to Credit in<br>Developing Countries? Evidence<br>from Asking Borrowers|George R. G. Clarke<br>Robert Cull<br>Maria Soledad Martinez P|November 2001<br>eria|P. Sintim-Aboagye<br>37644|
**Policy Research Working Paper Series**

|**Title**|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2717 Bridging the Economic Divide within<br>Nations: A Scorecard on the<br>Performance of Regional Developme<br>Policies in Reducing Regional Income<br>Disparities|Raja Shankar<br>Anwar Shah<br>nt<br>|November 2001|A. Santos<br>31675|
|WPS2718 Liberalizing Basic<br>Telecommunications:<br>The Asian<br>Experience|Carsten Fink<br>Aaditya Mattoo<br>Randeep Rathindran|November 2001|R. Simms<br>37156|
|WPS2719 Is There a Positive Incentive Effect<br>from Privatizing Social Security?<br>Evidence from Latin America|Truman G. Packard|November 2001|T. Packard<br>89078|
|WPS2720 International Migration and the Global<br>Economic Order: An Overview|Andres Solimano|November 2001|A. Bonfield<br>31248|
|WPS2721 Implications for South Asian Countrie<br>of Abolishing the Multifibre<br>Arrangement|s Sanjay Kathuria<br>Will Martin<br>Anjali Bhardwaj|November 2001|M. Kasilag<br>39081|
|WPS2722 Japan's Official Development<br>Assistance: Recent Issues and<br>Future Directions|Masahiro Kawai<br>Shinji Takagi|November 2001|J. Mendrofa<br>81885|
|WPS2723 Using Development-Orineted<br>Equity<br>Investment as a Tool for Restructurin<br>Transition Banking Sectors|Helo Meigas<br>g|November 2001|S. Torres<br>39012|
|WPS2724 Tropical Bubbles: Asset prices in<br>Latin America, 1980-2001|Santiago Herrera<br>Guillermo Perry|November 2001|R. lzquierdo<br>84161|
|WPS2725 Bank Regulation and Supervision:<br>What Works Best?|James R. Barth<br>Gerard Caprio Jr.<br>Ross Levine|November 2001|A. Yaptenco<br>38526|
|WPS2726 Applying the Decision Rights<br>Approach to a Case of Hospital<br>Institutional Design|Florence Eid|November 2001|A. Santos<br>31675|
|WPS2727 Hospital Governance and Incentive<br>Design: The Case of Corporatized<br>Public Hospitals in Lebanon|Florence Eid|November 2001|A. Santos<br>31675|