metadata: authors: 0: author: Abou-Ali, Hala
1: author: El-Azony, Hesham
2: author: El-Laithy, Heba
3: author: Haughton, Jonathan
4: author: Khandker, Shahidur R.
count: Egypt, Arab Republic of
docna: 0: docna: Evaluating the impact of Egyptian social
            fund for development programs
docty: Policy Research Working Paper
theme: Other rural development,Social analysis and monitoring,Rural services and infrastructure
majtheme: Rural development
subtopic: Health Care Services Industry,Sanitation and Sewerage,Engineering,State Owned Enterprise Reform,Public Sector Administrative & Civil
            Service Reform,Water and Human Health,Hydrology,Water Supply and Sanitation Economics,Sanitary Environmental Engineering,Small Private Water Supply Providers,Environmental Engineering,Health and Sanitation,Inequality,Town Water Supply and Sanitation,Democratic Government,Economics and Finance of Public
            Institution Development,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Health, Nutrition and Population,Poverty Reduction,Communities and Human Settlements
abstracts: cdata!: The Egyptian Social Fund for Development
            was established in 1991 with a mandate to reduce poverty.
            Since its inception, it has disbursed about $2.5 billion, of
            which nearly two-fifths was devoted to supporting
            microcredit and financing community development and
            infrastructure. This paper investigates the size of the
            impact of the Funds interventions, whether the benefits
            have been commensurate with the costs, and whether the
            programs have been targeted successfully to the poor. The
            core of the impact evaluation applies propensity-score
            matching to data from the 2004/2005 national Household
            Income, Expenditure and Consumption Survey. The authors find
            that Egypts Social Fund for Development programs have had
            clear and measurable effects, in the expected direction, for
            all of the programs considered: educational interventions
            have reduced illiteracy, health and potable water programs
            have lowered household spending on health, sanitation
            interventions have cut household spending on sanitation and
            lowered poverty, and road projects have reduced household
            transportation costs by 20 percent. Microcredit is
            associated with higher household expenditures in
            metropolitan areas and urban Upper Egypt, but not elsewhere.
            The Social Fund for Developments road projects generate
            benefits that, by some estimates, exceed the costs, as do
            health and potable water interventions; this is less evident
            for interventions in education and sanitation. The Fund
            argues that its mission is primarily social, and so should
            not be judged using a cost-benefit analysis. The Fund
            support for microcredit is strongly pro-poor; the other
            programs analyzed have a more modest pro-poor orientation.
geo_reg_key: 80475,107234,80464
repnme: repnme: Evaluating the impact of Egyptian social
            fund for development programs
display_title: Evaluating the impact of Egyptian
            social fund for development programs
listing_relative_url: /research/2009/07/10814164/evaluating-impact-egyptian-social-fund-development-programs
topicv3: State Owned Enterprises m1326323 721,Sanitation m1326453 2000,Water m1326522 137,Poverty m1326687 695,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Health, Nutrition and Population,Poverty Reduction,Communities and Human Settlements
no_of_pages: 38
model_extractions: WPS4993 Policy Research Working Paper 4993 Impact Evaluation Series No. 31 Evaluating the Impact of Egyptian Social Fund for Development Programs _Hala Abou-Ali Hesham El-Azony Heba El-Laithy Jonathan Haughton Shahidur R. Khandker_ The World Bank Development Research Group Sustainable Rural and Urban Development Team July 2009
Policy Research Working Paper 4993

# **Abstract**

The Egyptian Social Fund for Development was established in 1991 with a mandate to reduce poverty. Since its inception, it has disbursed about $2.5 billion, of which nearly two-fifths was devoted to supporting microcredit and financing community development and infrastructure. This paper investigates the size of the impact of the Fund’s interventions, whether the benefits have been commensurate with the costs, and whether the programs have been targeted successfully to the poor. The core of the impact evaluation applies propensityscore matching to data from the 2004/2005 national Household Income, Expenditure and Consumption Survey.

The authors find that Egypt’s Social Fund for Development programs have had clear and measurable effects, in the expected direction, for all of the programs considered: educational interventions have reduced illiteracy, health and potable water programs have lowered household spending on health, sanitation interventions have cut household spending on sanitation and lowered poverty, and road projects have reduced household transportation costs by 20 percent. Microcredit is associated with higher household expenditures in metropolitan areas and urban Upper Egypt, but not elsewhere.

The Social Fund for Development’s road projects generate benefits that, by some estimates, exceed the costs, as do health and potable water interventions; this is less evident for interventions in education and sanitation. The Fund argues that its mission is primarily social, and so should not be judged using a cost-benefit analysis. The Fund support for microcredit is strongly pro-poor; the other programs analyzed have a more modest pro-poor orientation.

This paper—a product of the a product of the Sustainable Rural and Urban Development Team, Development Research Group—is part of a larger effort in the department to understand the cost-effectiveness of rural development interventions such as social fund of Egypt. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at skhandker@worldbank.org.

_The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank’s Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Evaluating the Impact of Egyptian Social Fund for Development Programs**

Hala Abou-Ali Cairo University Hesham El-Azony Egyptian Social Fund for Development Heba El-Laithy Cairo University Jonathan Haughton Suffolk University, Boston Shahidur R. Khandker<sup>1</sup> World Bank, Washington DC Disclaimer: The findings and opinions in this paper do not necessarily reflect those of the institutions with which the authors are affiliated.

> 1 Please send correspondence to skhandker@worldbank.org .
# **Introduction**

Social funds have been defined as “agencies that finance small projects in several sectors targeted to benefit a country’s poor and vulnerable groups based on a participatory manner of demand” (Jorgensen and Van Domelen 1999). After the first social fund was set up in Bolivia in 1987, the idea attracted immediate support, and by May 2001 the World Bank had provided financing to more than 98 social funds in 58 countries.

This is the context in which the Egyptian Social Fund for Development (SFD) was established in 1991, when the government sought to widen the social safety net in the wake of an economic reform and social adjustment program. Run by a semi-autonomous agency that reports to the office of the Prime Minister, the SFD is one of the three main safety net programs in Egypt, along with the system of subsidies of basic foodstuffs, and the Ministry of Social Solidarity’s cash transfer program.

The mandate of the SFD is to (i) reduce poverty by supporting community-level initiatives, (ii) increase employment opportunities, and (iii) encourage small-enterprise development. In doing this, it is tasked with mobilizing national and international resources, and cooperating with governmental bodies, NGOs, and community and private sector groups. Since its inception, the SFD had disbursed a total of LE 11.4 billion as of the end of the first quarter of 2008, equivalent to approximately US$2.5 billion; this represents about US$2.36 per person per year<sup>2</sup> , or about 0.15 percent of GDP. An estimated 32% of Egyptians are potential beneficiaries from SFD projects, in the sense that they live in villages or wards where there is at least one SFD intervention.

Of its total spending, the SFD devoted 62 percent to supporting the development of small enterprises, 9 percent to microcredit, and the remaining 28 percent to financing community infrastructure. The SFD acts as a source of funding for projects, which are implemented by other agencies.

The focus of this paper is on SFD’s activities in community infrastructure and microcredit. Broadly, the question of interest is whether the SFD has had an impact on living standards. More specifically, our purpose is to measure the effects of SFD interventions on the immediate outputs (such as the number of households connected to sewers), on the short-term direct outcomes (for instance, the under-five mortality rate), and on the long-term impact on poverty, illiteracy, and employment.

> 2 Source for population information: UN World Population Prospects: The 2006 Revision.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 2 of 36_
We also address a number of other interesting issues, such as whether SFD interventions are well targeted to the poor – a subject explored in more detail in ESFD (2006) – or cost effective (see Rawlings et al. 2004, p.v), and whether the benefits of SFD interventions exceed the costs. The cost-benefit analysis should not, however, be viewed as an evaluation of the SFD, which was tasked with pursuing a number of social objectives rather than ensuring that benefits exceeded costs.

The usual justification for undertaking an impact evaluation applies here: as a matter of public policy we would like to know how effective the SFD has been, and hence if its efforts should be expanded or reduced, or its programs reoriented or redesigned. There is a dearth of such evaluations; a recent multidonor review mission wrote that “SFD needs to ... above all – deepen its attention to practical policy analysis and impact assessment.” In what follows we first set out the essential features of the SFD’s activities in microcredit and community infrastructure (comprising public works, and community development), outline the methodology used to measure the impact of these programs, discuss the viability of the data, present the results, weigh the costs against the benefits of SFD programs, and evaluate the extent to which the interventions were targeted to the poor.

# **The Social Fund for Development**

The main programs of the SFD are set out in Table 1. In this paper we seek to measure the impact of interventions in education, health, potable water, sanitation, road maintenance, and microcredit. Collectively these represent disbursements of LE 3 billion over the period 1991 – 2007, or just over a quarter of all SFD spending. The remaining spending was either on minor programs or administration, or went to the Small Enterprise Development Organization, which we are not in a position to evaluate.

The _Community Development Program_ is intended to promote projects that alleviate poverty. It is targeted geographically to relatively poor parts of the country, where it funds literacy classes, supports primary health care through the renovation of health units and training, and finances small environmental projects, including the digging of latrines, garbage collection, and tree planting.

In targeting poor areas, the SFD uses its network of 27 regional offices, which help identify local needs, mobilize community participation, and monitor the progress of projects. The Community Development _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 3 of 36_
Program activities work with long-established partners that include the Ministry of Health and Population, and the General Authority for Literacy and Adult Education.

The _Public Works Program_ has provided substantial funding for the extension of public potable water networks, sewage treatment and connections, and the construction and maintenance of paved roads. It also finances the restoration and construction of some public buildings such as health units and youth centers, and environmental projects that include covering watercourses, stabilizing river banks, and collecting garbage. This program works closely with existing government units both for project identification and implementation, although much of the actual work on the projects is done by local private contractors.

**Table 1: Principal Programs and Activities of the Social Fund for Development, 1991-2007**

|**Main Program**|**Main activity /**<br>Disbursements,<br>1991-2007|**Activities, 1991 – 2007**|**Evaluated in**<br>**this paper?**|
|---|---|---|---|
|**Community**<br>**Development**|**Health**<br>LE 390 million|Renovated health units (840)<br>Health awareness campaigns<br>Training of health personnel|Yes|
||**Education**<br>LE 200 million|Illiteracy classes (1.5 million people enrolled)<br>Single classroom schools<br>Community schools|Yes|
|||Training of teachers for literacy classes||
||**Other**<br>LE 210 million|Public buildings (rehabilitation; youth and community<br>centers)<br>Environment (covering waterways, weeding, river bank<br>protection)|No|
|**Public Works**|**Potable Water**<br>LE 620 million|Public networks (8,480 km)<br>Ground tanks (375)<br>Artesian wells (440)<br>Shelter houses (200)|Yes|
||**Sanitation**|Public networks (459)|Yes|
||LE 220 million|Treatment plants (16)<br>Pumping stations (38)||
|||House connections (19,500)||
||**Roads**<br>LE 500 million|Construction and maintenance of rural roads (2,500 km)|Yes|
||**Other**<br>LE 660 million|NGO capacity building<br>Environment (latrines, garbage collection, draining, pest<br>spraying, tree planting)<br>Training|No|
|**Microcredit**|LE 1,062<br>million|Loans (575,000)|Yes|
|**Small**<br>**Enterprises**|LE 7,100<br>million|Support for small enterprises, including loans and technical<br>support.|No|

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 4 of 36_
The _micro-credit program_ seeks to encourage the supply of loans to low-income households. It provides working capital to some financial intermediaries, such as Community Development Associations, Productive Family Associations, and the Principal Bank for Development and Agricultural Credit; in other cases it gives grants to financial institutions, on condition that they commit to more microlending.

The government has been increasing the pressure on the SFD to operate in a way that is financially selfsustaining – for instance, by charging fees, or putting more emphasis on microlending at appropriate interest rates. It recently refused a loan from the World Bank that would have provided funds to the SFD, because it was concerned that the return on the spending would not be sufficient to justify the cost of repaying the loan. On the other hand, the government might be more inclined to help the SFD obtain such funding if it found that SFD projects have a strong impact, which returns us again to the importance of evaluating the effects of SFD spending.

# **Measuring Impacts**

In seeking to measure the impact of SFD programs, we first need to clarify what impacts need to be measured, and then set out an appropriate methodology whereby these impacts may be identified and quantified.

There are two broad types of program effect that are of interest to us: short-term outcomes, and long-term impacts. The variables that we use to measure these effects are set out in more detail in Table 2, for each of the six sectors of interest. So, for instance, in evaluating projects under the education program, we might be interested in the proportion of participants in literacy classes who obtained diplomas (an outcome), and the effect on the illiteracy rate or employment rate (impacts).

We would like to think that SFD interventions would have unambiguously positive effects: more education would reduce the illiteracy rate; health projects should reduce mortality rates and poverty, as should more potable water and improved sanitation; better roads, and more microcredit, should increase incomes and lower poverty.

However, experience from other evaluations of social funds suggests that improvements such as these cannot be taken for granted. For instance, Rawlings et al. (2004), in a comparative study of social funds in six countries, found that while interventions led to greater access to and use of basic services, tangible improvements in welfare were sometimes elusive. For instance, educational projects did not raise test _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 5 of 36_
scores or primary school enrollment rates in rural Bolivia, although they did raise enrollment rates in Nicaragua and Zambia. Similarly, spending on primary health centers reduced infant mortality in Bolivia, but had small or no discernible effects on health outcomes in other countries. And sanitation projects led to health gains in Peru and Bolivia but not in Honduras.

**<u>Table 2.  Measures of Inputs, Outputs, Outcomes, and Impacts</u>**

||**Inputs**|**Outputs**|**Outcomes**<br>**(short-term)**|**Impacts**<br>**(long-run)**|
|---|---|---|---|---|
|Education|Money<br># of individuals trained to teach|# of class rooms established<br># of beneficiaries<br># of nurseryclasses|<br># (percentage) of the<br>participants getting the<br>intervention|Illiteracy rate<br>Employment rate|
|Health|Renovation of health clinics<br># of health personal trained<br># of health awareness campaigns|# of beneficiaries<br># of health clinics renovated<br>as percentage of total<br>health clinics<br># of mothers that received<br>pre-natal care|<br>Rate of immunization<br>Under 5 mortality rate<br>Average expenditure on<br>medicine||
|Potable water|Length of pipe lines constructed /<br>rehabilitated<br># of elevated or ground tanks|# of household connected as<br>a % of total of household<br>in the community<br>Expenditure on water<br>procurement|<br> <br># under 5 mortality rate<br>Incidence of diarrhea<br>Incidence of illnesses due to<br>water related disease<br>Expenditure on medicine||
|Sewerage<br>connection|Length of sewerage network<br># of waste water treatment plant|# of household connected<br>Expenditure on sanitation/<br>septic tanks|Mortality<br>Morbidity<br>Diarrhea<br>Expenditure on health||
|Roads|Length of rural roads constructed/<br>rehabilitated|Transportation expenses<br>reduction<br>Mode of transport<br>Travel time to reach<br>clinic/school|Employment<br>Farm income/ non farm<br>income<br>Under 5 mortality<br>School enrolment rate||
|Microcredit|# of NGOs receiving loans from<br>the SFD<br>Amount of money received from<br>the SFD|Average amount loans<br>extended (received) by<br>SFD NGOs<br># of beneficiaries receiving<br>loans|Diversification of income<br>Average income<br>Employment level<br>Diversification of<br>employment|Poverty|

Where projects have no discernible overall impact, the explanation is typically that the money was poorly spent (e.g. on a shiny new school to replace an adequate older building, or on a road to nowhere), or crowded out existing services (e.g. a new health center might change where one goes for care, but may not raise the level of care appreciably).

The simplest approach to measuring the effects of a program is to compare the outcomes for those who were covered by the program with those who were not. Unfortunately, this does not work, because of the problem of endogenous program placement. For example, the rate of gastroenteritis is 6.9% in villages with an SFD health program but just 2.7% for communities without one; yet it would not be credible to claim that an SFD health program actually causes more gastroenteritis.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 6 of 36_
The difficulty is that the SFD has, for some of its programs, targeted poorer areas, where many of the outcomes of interest (literacy rates, mortality rates, poverty rates, expenditure levels) are inherently worse. This also shows up in Table 3, which shows expenditure levels and poverty rates for villages with and without the program interventions of interest. Thus we see that household expenditure per capita levels are lower in localities with an SFD program; and the poverty rate is 30% in villages where the SFD had a program to provide potable water, compared with a poverty rate of 18% elsewhere in Egypt.

**<u>Table 3.  Poverty Rates and Expenditure Levels for Areas With, and Without, SFD Programs</u>**

||Without any SFD<br>program|With at least one<br>SFDprogram|All|
|---|---|---|---|
|Per capita annual expenditure|2,680|2,292|2,556|
|Per capita annual food expenditure|1,165|1,076|1,127|
|Headcount poverty rate (%):||||
|Overall|18.6|21.6|19.6|
|If there was an SFD intervention in||||
|Education|19.8|15.0|19.6|
|Health|19.5|22.1|19.6|
|Microcredit|18.8|23.1|19.6|
|Paving roads|19.5|21.8|19.6|
|Potable water|18.2|30.2|19.6|
|Waste water|19.6|12.8|19.6|
||Ranked|by expenditure per capita<br>||
||Poorest 40%|Middle 40%|Top20%|
|% having any SFD intervention||||
|Urban|10.8|10.4|7.6|
|Rural|9.9|9.9|10.5|

_Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

There are a number of possible solutions to the problem of endogenous program placement. In most of what follows we use propensity score matching, but we check the robustness of our results using a common impact model. Some further explanation of these approaches is in order.

# _Propensity Score Matching_

Even if the placement of SFD programs (“treatments”) has not been done randomly, it may be possible to measure the impact of the program by using matched comparisons. The idea is to match each participant, in this case a village, with an otherwise identical non-participant (the comparator) – based on observed pre-treatment or exogenous characteristics – and then to measure the average difference in the outcome variable between them.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 7 of 36_
Units that cannot be matched are discarded – this is central to good matching – because they “cannot support causal inferences about missing potential outcomes” (Diamond 2005, p.9). The hope is that this allows one to mimic the effects of randomization. Of course, the resulting measure of impact is only compelling to the extent that one believes that the matching has been done well and the treatment assignment is ignorable; in other words, we know that the treatment was not assigned randomly, but we believe that we may proceed as if it were.

To match treatment with non-treatment units it is helpful to create a summary measure of similarity in the form of a _propensity score_ . Let p( _Xi_ ) be the probability that unit _i_ be assigned to the treatment group, conditional on _Xi_ , and define This probability of participation – the propensity score – can be estimated using an _assignment model_ . Given survey and census information, we first pool the two samples (i.e. the participants and nonparticipants) and estimate a probit model of program participation as a function of pre-treatment and exogenous variables that might influence participation.

The computation of propensity scores is only the first step in the process. Rosenbaum and Rubin (1983) prove that treatment cases may be matched with comparison cases using just the propensity score rather than the entire set of pre-determined covariates _Xi_ . In other words, to find the non-participant that is most closely matched to the participant, one only needs to find the non-participants with the propensity scores closest to that of the participant. They also show that where _Gi_ | _T_ = ,1 _p_ ( _x_ ) is the difference between the treatment outcome<sup>_Y_</sup> _iT_ for treated unit _i_ and the (control) outcome for the non-treated unit closest in propensity score to _i._ In other words, the average treatment effect (the gain for the treated, here denoted by _G_<sup>_TT_</sup> ) may be obtained by computing the expected value of the difference in the outcome variable between each treated household and the perfectly matched comparison household as matched using the propensity score.

Perfect matching is not possible in reality, so in practice one needs to compute where _Yi_ is the observed outcome for the _i_ th individual who is treated and _Ji_ is the set of comparators for _i_ , and _N_ is the set of units for which the set of comparators is non-empty (the “common support”, discussed _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 8 of 36_
in more detail below). With nearest neighbor matching one chooses the _m_ closest comparators; we follow the common practice of using _m_ =1. As a robustness check, we also use kernel matching, which puts more weight on closer comparators than those that are more distant. Dehejia and Wahba (2002) argue that the choice of matching mechanism is not as crucial as the proper estimation of the propensity scores, but this is not a settled issue.

In practice, the plausibility of propensity score matching depends on ensuring “common support” and “balancing.” Only in the area of common support is it possible to make comparisons that allow us to make inferences about causality (Rubin and Waterman 2006), so our comparisons need to be confined to this area, and an impact evaluation is not possible unless there is an area of common support (Imbens 2004, p.7). The area of common support occurs where the densities of the estimated propensity scores for participants and for non-participants overlap, as shown in Fig. 1. Above point B there are no comparators for borrowers, so matching is not possible in this zone; below point A there are no borrowers that need to be matched. In all the results reported below there was an acceptably broad region of common support.

<!-- Start of picture text --> Region of common support<br>Density<br>for non-<br>participants<br>Density for<br>participants<br>0 A .2 .4 .6 B .8 1<br>3<br>2<br>1<br>0<br><!-- End of picture text -->

**Fig. 1. Illustrating the Region of Common Support.**

In addition, for propensity score matching to work, the treatment and comparison groups need to be “balanced.” A treated unit and matched comparator might both have essentially the same propensity scores, but this does not guarantee that they are similar in a relevant way: one village might be well educated but ageing, while another (with a similar propensity score) might be poorly educated but young.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 9 of 36_
It is not necessary for every individual match to be close, but it is important for the distributions of covariates for the treated and the comparators to be similar, and this is what is meant by balance. More formally, in order to verify balance we need to check whether _p_ (ˆ _X_ | _T_ = )1 = _p_ (ˆ _X_ | _T_ = 0) (4)

where _p_ ˆ gives the empirical (rather than population) density of the data.

In line with common practice, the algorithm we use is to estimate a propensity score, match treated with non-treated units, and check for balance; if balance is not achieved, we revise the assignment model, and repeat this process until (hopefully) balance is achieved. In checking for balance, we divide the data into strata, based on the estimated propensity score, and then use a series of t-tests or chi-square tests to check that, within each stratum, the values of each covariate (age, literacy rates, and so on) are on average the same. This allows one to “check the adequacy of the statistical model” of assignment (Imbens 2004, p.18).

# _Impact Models_

There is another way to deal with the problem of endogenous program placement. It may be possible to assume that SFD programs have been assigned randomly, _conditional on some observable variables_ , _X_ , that might include measures such as village size, or literacy rates, or the structure of the local economy. This would constitute partial randomization, and is a maintained assumption rather than a testable proposition.

If we are willing to accept the conditional exogeneity of program placement, it may be possible to estimate the impact of a treatment such as an SFD program using a parametric model with controls. Suppose that we may assume where the error terms are normally distributed with zero means and constant variances, and the superscripts refer to the treatment group (T) and the comparison group (C). These two equations are often estimated together in the form of a switching regression using the pooled data from both the treatment and comparison samples, giving _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 10 of 36_
where _Ti_ takes on the values of one or zero and the error term now takes the form ε _i_ = _Ti_ ( ν _iT_ −ν _iC_ ) +ν _iC_ . If we may assume that the error term (the “latent effects”) has zero mean conditional on the _X_ covariates and treatment – a reasonable assumption if there is even partial randomization – then we have _E_ ( ν _iT_ | _X_ , _T_ = _t_ ) = _E_ ( ν _iC_ | _X_ , _T_ = _t_ ) = ,0 _t_ = ,1,0 and we can get consistent estimates of the average treatment effects by applying ordinary least squares (OLS) to Eqn. 6, and noting that

_G ATE_ = _E_ [ α _T_ − α _C_ + _X i_ ( β _T_ − β _C_ )]. (7) If we are also willing to assume (more problematically) that β<sup>T</sup> =β<sup>C</sup> then we obtain the _common-impact model,_ where the average treatment effect reduces to α _T_ − α _C_ .

In what we report below we mainly emphasize the results of propensity score matching, but we include the results of the common-impact model both as a robustness check, and as legitimate estimates in their own right.

# **Choosing a Counterfactual**

In all impact evaluations one is trying to compare the actual impact with a counterfactual, where the latter represents our view of what would have happened in the absence of the project. In the case of SFD projects, a number of possible comparisons are possible, because of the presence of non-SFD interventions by ministries and NGOs. The situation is illustrated in Figure 2, and suggests the following useful comparisons:

- (i) **A vs. B.** Any given village either has an SFD intervention or it does not.  This comparison of A vs. B provides one possible baseline.  The implicit assumption here is that in the absence of an SFD intervention, there is still a substantial probability that there would have been some other intervention – most villages without an SFD project have an intervention by some other organization.  If, in this comparison, SFD villages (A) perform better than non-SFD villages (B), then it is largely because the SFD projects are more effective than non-SFD projects.  This is a relatively high bar, and probably provides a lower bound to the impact of the SFD.

- (ii) **D vs. F.** This compares villages that have just an SFD project with villages that have no interventions at all.  It is the relevant comparison if, in the absence of an SFD project, there would have been no other project.  In that sense it measures the pure effect of the SFD.  However, it implicitly assumes that, in the absence of an SFD project, there would have be no comparable

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 11 of 36_
project so, for instance, if the SFD had not financed a project to pave the local road, then the road would have remained unpaved. This is an extreme assumption, and is not entirely plausible, because other organizations – NGOs, ministries, local self-help groups – might well have stepped in if the SFD were not present. Thus the comparison between D and F may be expected to provide an upper bound to the effect of the SFD.

- (iii) **C vs. E.** In this case the comparison measures the incremental impact of SFD interventions, given that there are already other interventions.  If there are synergies between SFD projects and other interventions – for instance, an SFD sewerage project might work best if some other entity is already at work providing piped water – then this comparison should pick up this effect.

<!-- Start of picture text --> Is there an SFD intervention?<br>Yes: A No: B<br>With other “SFD only”: With other No<br>interventions D interventions: interventions<br>(“mixed”): C E at all: F<br><!-- End of picture text -->

**Figure 2.  Creating a Counterfactual for SFD interventions**

In the results reported below, we measure the impact using all three comparisons. But before doing this, some discussion of the data is warranted.

# **Data Considerations**

Most SFD programs operate at the level of a village (or sub-village), and this is the level at which the analysis is conducted. Although the basic unit of observation is the village, many of the variables – per capita income and expenditure levels, for instance – are based on household-level data. The main source of such data is the 2004/2005 edition of the national Household Income, Expenditure and Consumption _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 12 of 36_
Survey (HIECS), conducted by the Central Agency for Public Mobilization and Statistics (CAPMAS), and which interviewed 47,095 households between July 1, 2004 and June 30, 2005. The sampling is stratified (by governorate), and clustered, with 40 households being interviewed in every sampled village or ward (the primary sampling units). The questionnaire collected basic socioeconomic data, including enough information to establish consumption-based measures of poverty.

Three changes were made to the survey to ensure that the data could be used for an impact evaluation. First, a module was added to the survey instrument that included questions about SFD activities and community organizations and initiatives. This supplementary module was administered to all of the 12,000 households surveyed in the fourth quarter of the HIECS survey (i.e. from April-June 2005) – each of the four rounds of the survey are nationally representative – as well as to all 10,000 sampled households living in areas with SFD interventions. This module gathered information on illiteracy and health programs, the extent of SFD interventions, and the providers and impact of microcredit.

Second, an additional 3,760 households in 94 primary sampling units (PSUs) were surveyed in areas where the SFD was active, in order to ensure enough coverage of areas in which the SFD operated; the sampling followed the HIECS protocol, and the questionnaire included the supplementary module, but omitted some of the irrelevant parts of the HIECS questionnaire. The intention was to ensure that strictly comparable survey data would be available for at least 80 treatment villages (PSUs) for each of the seven sub-sectors (education, health, potable water, wastewater, road maintenance, the environment, and microcredit) covered by SFD interventions.

The third modification was that a questionnaire on microcredit was administered to a randomly-selected sample drawn from all present borrowers under the SFD program, within PSUs in the HIECS master sample.

It should be noted that panel data were not available, so powerful techniques of impact evaluation such as double differencing were not feasible. This is a major, if inescapable, limitation of the current study.

The household-level data were supplemented with information from a community survey that was integrated with the HIECS survey, and provided information on the availability, accessibility, and quality of water and sewerage systems, health posts, and primary schools at the level of individual villages and wards (i.e. PSUs). It also sought to measure perceptions of the SFD interventions, and the extent of community participation in the projects. The community survey was also administered in the 94 areas _Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009_ _Page 13 of 36_
covered by the supplemental survey. At the community level, information was also available from the 2000 census, and this was an important source of pre-determined variables for use in estimating the propensity score equation.

# **Results 1: The Example of Potable Water**

To illustrate the approach that we take to measuring the impacts of SFD programs, we first present in some detail the results for potable water projects, comparing villages that have both SFD and other (“nonSFD”) potable water interventions with villages that only have non-SFD interventions; this is the C vs. E comparison, to use our terminology from the previous section. This allows us to discuss both the methodological and practical issues involved in our estimations. A complete summary of the results for the other SFD programs and comparisons is given in the subsequent section of the paper.

**_Propensity Score Matching._** The first step here is to estimate a propensity score equation that satisfies the balancing property.  Our procedure was to estimate a probit  assignment model, check for balance, and then adjust the model until balance was achieved.  The model whose estimates are shown in Table 4 is “balanced”, and it is also broadly reasonable: SFD potable water projects were more likely to occur in areas where families were larger, the poverty rate higher, and access to sewers was lower.  The region of common support stretches from 0.012 to 0.725, which includes all the SFD project areas and 1,044 of the 1,089 non-SFD areas, and should provide an adequate basis for subsequent matching.

**Table 4.  Estimate of Propensity Score Equation, Potable Water Projects, Comparing “Mixed” SFD and** **<u>Other Interventions with Non-SFD Interventions</u>**

||Coefficient|Standard Error|
|---|---|---|
|Dependent variable: =1 if village has SFD piped water project, =0 otherwise|||
|Households with access to a sewage system in 2004/05 (%)|-0.169|0.013|
|Average household size in 2004/05|0.251|0.007|
|Female-headed households in 2004/05 (% of all households)|0.577|0.051|
|PSU had one or more training programs (yes=1) in 2004/05|-0.304|0.014|
|Illiteracy rate in 2004/05 (%)|-0.243|0.007|
|Headcount poverty rate in 2000 (%)|0.0015|0.0003|
|Female-headed households in 2000 (% of all households)|0.033|0.001|
|Intercept|-2.548|0.040|

_Notes._ Probit equation; pseudo R<sup>~~2~~</sup> = 0.082. Data come from 1,291 PSUs (villages and wards), of which 202 have SFD and other interventions, and 1,089 have non-SFD interventions only. The regression was weighted by population size. The region of common support is [0.012, 0.725] and includes all the PSUs with SFD and other interventions and 1,044 of the PSUs with nonSFD interventions. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

In Tables 5-7 we present a number of estimates of the effect of SFD interventions on the headcount poverty rate, which is one of the long-term impacts of most interest. The first measure is based on a regression equation in which the headcount poverty rate in a village is the dependent variable, and the _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 14 of 36_
presence of an SFD intervention is captured as a binary covariate. The regression is confined to the sample of PSUs (villages and wards) in which there are non-SFD projects to supply potable water, and so the measure of impact here should pick up the incremental effect of SFD interventions, given that there are already other interventions.

The regression results in Table 5 use only those observations that are in the region of common support, and they weight the observations by the inverse of the propensity scores, as suggested by Imbens (2004, p.16). By this measure, the presence of an SFD piped water project reduces the poverty rate by 0.8 percentage points, an estimate that is significant at the 1% level.

**Table 5.  Estimates of Regression Showing Impact of SFD projects on the Headcount Poverty Rate, Comparing PSUs Having “Mixed” SFD and Other Interventions with PSUs Having only Non-SFD Interventions**

||Coefficient|p-value|
|---|---|---|
|Dependent variable: Headcount poverty rate (%)|||
|**SFD piped water project in place (1=yes, 0=no)**|**-0.81**|**0.00**|
|Households with access to a sewage system in 2004/05 (%)|-1.53|0.00|
|Average household size in 2004/05|0.26|0.07|
|Female-headed households in 2004/05 (% of all households)|1.65|0.08|
|PSU had one or more training programs (yes=1) in 2004/05|-0.16|0.64|
|Illiteracyrate in 2004/05(% illiterate amongthose older than 10)|-0.45|0.01|
|Headcount poverty rate in 2000 (%)|0.07|0.00|
|Female-headed households in 2000 (% of all households)|0.06|0.02|
|Intercept|1.08|0.17|

_Notes._ Observations are weighted by the square root of the inverse of the propensity score (if SFD project is in place) or by square root of the inverse of 1-propensity score (if SFD project is not in place). Adjusted R<sup>2</sup> = 0.168. Data come from 1,290 PSUs (villages and wards) for which data were available and fall within the region of common support (as determined by the propensity score equation – see Table 4). _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

The results of the propensity score matching are shown in Table 6. The upper panel uses nearestneighbor matching, and also finds that the poverty rate is 0.8 percentage points lower in villages with SFD projects than in the comparison villages, although the difference is not quite statistically significant. Kernel matching finds that the under-5 mortality rate is about 0.4 percentage points lower in the treatment villages, but here again the difference is not quite statistically significant.

Table 7 displays the results of estimating straightforward impact equations. These use observations from the full data set, and not just for villages in the area of common support, and they include a large number of other variables (which reduces the sample size somewhat, because some values are missing). The results of the common-impact model – given in the middle columns of Table 7 – show that the presence of a SFD piped water project is associated with a 0.5 percentage point reduction in the poverty rate, after controlling for other influences, but the result is not statistically significant. The general-impact model is based on the same variables as used in the common-impact model, but also includes interactions between _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 15 of 36_
the presence of the SFD project and these variables; it finds an implausibly large reduction (of 2.2 percentage points) in the poverty rate.

**Table 6.  Results of Propensity Score Matching to Determine the Impact of SFD projects on the Headcount Poverty Rate, Comparing PSUs Having “Mixed” SFD and Other Interventions with PSUs Having only Non-** **<u>SFD Interventions</u>**

||Number<br>of PSUs|Headcount<br>Povertyrate(%)|Analytical Standard<br>Error /(t-statistic)|Bootstrapped Standard<br>Error /(t-statistic)|
|---|---|---|---|---|
|Nearest neighbor matching|||||
|Matched treated cases|202|4.07|||
|Matched comparison cases|170|4.83|||
|Treated – Comparison||**-0.76**|0.56(t=-1.37)|0.53(t=-1.43)|
|Kernel matching|||||
|Matched treated cases|202|4.08|||
|Matched comparison cases|1,044|4.48|||
|Treated – Comparison||**-0.39**|n.a.|0.32(t=-1.22)|

_Notes:_ Comparisons use propensity score based on equation in Table 4, and confined to the region of common support, where the propensity score is between 0.020 and 0.725. “Treated cases” refers to PSUs where there are both SFD and non-SFD potable water projects, and these constitute 16% of all 1,246 PSUs in the region of common support. “Comparison cases” refers to PSUs where there are only non-SFD potable water projects. Kernel matching uses Epanechnikov kernel functions with a bandwidth of 0.06. Bootstrapped results are based on 100 draws. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

A more complete set of estimates of the effects of potable water provision are shown in Table 8; in each case, we are comparing villages where there are both SFD and non-SFD water projects with villages where there are only non-SFD water projects. Surprisingly, the presence of an SFD water supply project does not appear to increase the number of households that have access to piped water, suggesting that for this type of project, the SFD substitutes for, rather than supplements, other potable water projects. On the other hand, when there is an SFD project, households spend less on water than they otherwise would; one possibility is that the SFD may not require as much cost recovery as other suppliers of water. The effects of SFD water projects on health are mixed; they appear to reduce spending on health care, and may reduce the child mortality rate, but the incidence of illness, especially diarrhea and renal disease but perhaps also malaria and bilharzia, actually rises. This merits further investigation; the presence of additional water, if poorly treated or disposed of improperly, could indeed increase the incidence of water-borne disease; if this is the case, then the provision of potable water would need to include provisions that deal with water quality and disposal.

# **Results 2: Summarizing the Impacts of SFD Projects**

The measured impacts of SFD projects are summarized in Tables 9-14. In each case we only present the relevant measures of impact, and not the complete (and voluminous) regression and other details. The _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 16 of 36_
**Table 7.  Estimates of Impact Equations for SFD Potable Water Projects (In the Presence of Other Potable** **<u>Water Interventions) on the Headcount Poverty Rate</u>**

||Common-im|pact model|General imp|act model|
|---|---|---|---|---|
||Coefficient|p-value|Coefficient|p-value|
|Dependent variable: Headcount Poverty Rate (%)|||||
|**SFD piped water project in place (1=yes, 0=no)**|**-0.48**|**0.13**|-19.82|0.04|
|<br>Average value of durable goods|-0.65|0.00|-0.60|0.00|
|Children asproportion of household members|-3.71|0.20|-5.95|0.06|
|Adult males as proportion of household members|-1.36|0.68|-5.48|0.12|
|<br>Adult females as proportion of household members|-5.03|0.16|-6.95|0.07|
|<br>Proportion of households with access to sewerage(%)|-0.83|0.03|-0.73|0.07|
|Average household size|0.33|0.04|0.40|0.03|
|Female-headed households (% of households)|1.34|0.18|0.96|0.39|
|<br>Proportion of children who are working (%)|-5.56|0.00|-4.89|0.00|
|Illiteracy rate (%)|0.89|0.35|0.49|0.65|
|Prevailing economic activity in PSU is services (yes=1)|0.97|0.12|0.64|0.32|
|<br>PSU has one or more training programs(yes=1)|0.42|0.16|0.45|0.14|
|PSU has an illiteracy eradication program (yes=1)|-0.59|0.71|0.03|0.88|
|<br>2000: Employment rate (%)|0.13|0.01|0.02|0.01|
|<br>2000: Illiteracyrate(%)|-0.03|0.17|-0.03|0.18|
|2000: Headcount poverty rate (%)|0.05|0.00|0.06|0.00|
|2000: Per capita expenditure|0.001|0.00|0.001|0.02|
|2000: Per capita food expenditure|-0.004|0.02|-0.002|0.15|
|2000: Female-headed households (% of households)|0.05|0.08|0.01|0.78|
|2000: Access to piped water (% of households)|0.002†|0.62|-0.002†|0.64|
|<br>2000: Access to sanitation (% of households)|<br>0.002|0.65|<br>0.002|0.67|
|<br>Intercept|8.17|0.01|9.72|0.00|
|Interaction terms (of variables with presence of SFD piped|No||Yes||
|water project):|||||
|Adjusted R<sup>2</sup>|0.25||0.26||
|<br>Number of observations(PSUs)|1,267||1,267||

_Notes:_ Treatment effect for interactive equation, averaged over all observations: -2.21. † Coefficient shown here has been multiplied by 10<sup>6</sup> . _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

**Table 8.  Estimates of Effects of SFD Potable Water Interventions (in the Presence of Other Potable Water Interventions)**

||Propensity <br>|Score Match<br>|ingModel<br>|Impact<br>|Model<br>|
|---|---|---|---|---|---|
||Regression|Nearest<br>Neighbor|Kernel<br>Matching|Common<br>Impact|General<br>Impact|
|**Outputs**||||||
|Households with access to piped water (%)|-0.02|-0.12|-0.02|-0.02|-0.03|
|Average household spendingonpotable water|-1.02**|-1.92*|-1.39**|-1.21*|-1.57|
|**Outcomes (short-term)**||||||
|Under-5 mortality rate (per thousand)|-5.94|4.85|-4.07|-5.55|-35.19|
|Prevalence of diarrhea (%)|2.63**|2.93**|2.37**|2.49**|3.43|
|Prevalence of malaria (%)|0.04*|-0.00|0.04|0.04|0.11|
|Prevalence of renal disease (%)|0.42**|0.46**|0.46**|0.54**|0.15|
|Prevalence of bilharzia (schistosomiasis) (%)|0.12*|0.16|0.01|0.14*|-0.16|
|Per capita expenditure on health care|-12.13**|-17.78*|-12.71**|-6.34|22.44|
|**Impact (long-run)**||||||
|Povertyrate(%)|-0.81**|-0.76|-0.39|-0.48|-2.21|

_Notes:_ * denotes significant at 10% level, ** denotes significant at 5% level. Significance levels for General Impact Model are not shown here. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 17 of 36_
computations begin by estimating a propensity score equation, and then match the treated with the control villages (or households, in the case of the microcredit results) in the area of common support. As discussed above, the “Regression” columns report the results of estimating the impact equation in the area of common support, weighting the observations with the inverse of the propensity score.

The single largest component of the SFD interventions in education has been the provision of literacy classes, which have enrolled almost 1.5 million people. It may be seen from Table 9 that this has had a measurable effect in reducing the proportion of people who are illiterate (by about 0.04; the mean value is 0.29) and providing literacy certificates (up by about 0.08; the mean value is 0.37). These results are robust, in the sense that they apply to all three comparisons. However, the efforts have not had any clear effect on the employment rate, and they are associated with higher, rather than lower, poverty rates, which is unexpected. One possibility is that SFD educational projects equip people to leave their villages and find better jobs elsewhere; this could actually increase the local poverty rate, if this process allows the less-poor to leave.

**Table** **<u>9. Summary of Effects of SFD Interventions in Education</u>**

|||SFD inte<br>inter|rvention vs<br>vention[A v|. no SFD<br>s. B]|SFD int<br>interve|ervention on<br>ntion at all[|ly vs. no<br>D vs. F]|SFD inter<br>other inter|vention with<br>ventions on|others vs.<br>ly [C vs. E]|
|---|---|---|---|---|---|---|---|---|---|---|
||Mean|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|
|**Outcome:**|||||||||||
|% who got|0.37|0.08***|0.01|0.07**|0.08**|0.02|0.09*|0.08**|0.05|0.10|
|<br>certificate†||(2.72)|(0.18)|(2.01)|(1.99)|(0.25)|(1.93)|(1.96)|(0.53)|(1.62)|
|**Impacts:**|||||||||||
|Illiteracy rate|0.29|-0.04***|-0.02|-0.01|-0.04***|-0.02|-0.02|-0.03**|-0.02|-0.004|
|||(-4.8)|(-0.08)|(-1.08)|(-3.55)|(-0.76)|(-1.10)|(-2.20)|(-0.19)|(-0.19)|
|Employment|0.95|-0.005|-0.01|-0.003|0.002|-0.0004|0.003|-0.01**|-0.01|-0.01|
|<br>rate||(-1.52)|(-0.81)|(-0.53)|(0.58)|(-0.05)|(0.63)|(-2.38)|(-0.59)|(-1.11)|
|Poverty rate|4.1|0.04*|1.40**|0.95**|0.26|0.71|0.63|0.51|1.08|1.22|
|||(1.71)|(2.34)|(2.39)|(0.88)|(0.99)|(1.30)|(1.45)|(1.21)|(1.60)|

_Notes:_ † Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. † % of participants who obtained a certificate. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

SFD health projects mainly consist of renovating health centers, training personnel, and conducting health awareness campaigns. The clearest and most consistent effect of this spending is that it reduces annual spending on health, by about LE 9 per household (the mean spending is LE 51). There is also a clear and strong effect on the under-5 mortality rate, which now stands at 19.9 per thousand, but is reduced by about 6 per thousand as a result of SFD interventions.

However, the effect of health interventions on the poverty rate is less clear. When SFD interventions occur in addition to other interventions (the C vs. E comparison), they appear to lower the poverty rate, _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 18 of 36_
but if one compares areas that just have SFD health projects with areas that have no health projects, the SFD areas are poorer. This is not compelling, and suggests that our propensity score matching may not be controlling adequately for other influences in these cases.

**Table 10.** **<u>Summary of Effects of SFD Interventions in Health</u>**

|||SFD inte<br>inter<br>|rvention v<br>vention[A<br>|s. no SFD<br>vs. B]<br>|SFD int<br>interve<br>|ervention on<br>ntion at all[<br>|ly vs. no<br>D vs. F]<br>|SFD inter<br>other inter<br>|vention wit<br>ventions on<br>|h others vs.<br>ly [C vs. E]<br>|
|---|---|---|---|---|---|---|---|---|---|---|
||Mean|Regres-<br>sion|Nearest<br>Neighbor|<br>Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|
|Under-5<br>mortalityrate|19.9|-6.19*<br>(-1.92)|-7.29<br>(-0.96)|-5.92<br>(-1.55)|-4.41<br>(-1.00)|-16.52<br>(-0.74)|-4.32<br>(-0.90)|-8.71*<br>(-1.84)|-11.75<br>(-0.97)|-9.25*<br>(-1.65)|
|Expenditure on|50.9|-8.69***|-3.79|-10.18|-10.00***|-8.91*|-12.70***|-7.82**|-0.79|-6.88*|
|Health||(-3.75)|(-0.88)|(-3.25)|(-3.32)|(-1.77)|(0.63)|(-2.17)|(-0.12)|(-1.69)|
|Poverty Rate|4.5|0.15|-0.20|0.44|2.16***|3.43***|2.68***|-1.57***|-2.07*|-1.48***|
|||(0.59)|(0.28)|(0.86)|(6.37)|(3.03)|(3.26)|(-4.37)|(1.95)|(-2.85)|

_Notes:_ Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

The SFD has spent LE 620 million on providing potable water, including a public network of 8,480 kilometers. As Table 11 shows, this has led to sharp drops in spending on water and on health care. It is noteworthy that the reduction in health spending here is twice as large as the reduction associated with SFD health projects. There is also some modest evidence in Table 11 that the provision of potable water has led to a lower child mortality rate.

The effects on poverty of providing potable water are more mixed; where SFD projects are in addition to other interventions, they help reduce poverty, but not otherwise.

**Table 11.** **<u>Summary of Effects of SFD Interventions in Potable Water Provision</u>**

|||SFD inte<br>interv<br>|rvention vs<br>ention[A v<br>|. no SFD<br>s. B]<br>|SFD inte<br>interven<br>|rvention on<br>tion at all[ <br>|ly vs. no<br>D vs. F]<br>|SFD interv<br>other interv<br>|ention with<br>entions on<br>|others vs.<br>ly [C vs. E]<br>|
|---|---|---|---|---|---|---|---|---|---|---|
||Mean|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|
|Expenditure on<br>water|17.2|-1.53***<br>(-3.61)|-1.72*<br>(-1.88)|-1.42***<br>(-2.65)|-2.83***<br>(-3.48)|-0.90<br>(-0.64)|-1.76*<br>(-1.70)|-1.02**<br>(-2.03)|-1.92*<br>(-1.88)|-1.39**<br>(-2.36)|
|Under-5<br>mortalityrate|19.9|-3.87<br>(-1.25)|-0.93<br>(-0.11)|-4.19<br>(-1.10)|2.30<br>(0.39)|2.59<br>(0.16)|1.71<br>(0.24)|-5.94*<br>(-1.16)|4.85<br>(0.84)|-4.07<br>(-0.84)|
|Expenditure on|50.9|-15.36***|-23.87***|-13.85**|-21.42***|-16.22*|-17.00***|-12.13***|-17.78**|-12.72***|
|Health||(-4.16)|(-3.63)|(-4.05)|(-3.47)|(-1.76)|(-3.31)|(-2.69)|(-2.20)|(-3.87)|
|Poverty Rate|3.9|-0.36*<br>(-1.66)|-0.57<br>(-1.15)|-0.15<br>(-0.41)|0.64<br>(1.61)|0.20<br>(0.23)|0.40<br>(0.53)|-0.81***<br>(-3.22)|-0.76<br> (-1.4)|-0.40<br>(-1.06)|

_Notes:_ Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

The sanitation projects of the SFD consist mainly of constructing public networks and providing house connections. These activities have had the expected effects – increasing access to sewerage networks, halving household spending on sanitation, and lowering the poverty rate. These effects are especially _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 19 of 36_
clear where SFD projects are undertaken in conjunction with other interventions (comparing C vs. E); when SFD-only areas are compared with areas lacking any project, the effects of SFD projects are either not statistically significant or counterintuitive (in the case of renal disease). It is curious that sanitation projects are not associated with lower household spending on health; one might have expected better sanitation facilities to improve health and so reduce such spending. Better health might then be associated with higher productivity and incomes, and a lower poverty rate.

**Table 12.** **<u>Summary of Effects of SFD Interventions in Sanitation</u>**

|||SFD int<br>inter<br>|ervention vs<br>vention[A<br>|. no SFD<br>vs. B]<br>|SFD int<br>interve<br>|ervention on<br>ntion at all[<br>|ly vs. no<br>D vs. F]<br>|SFD inter<br>other inter<br>|vention wit<br>ventions on<br>|h others vs.<br>ly [C vs. E]<br>|
|---|---|---|---|---|---|---|---|---|---|---|
||Mean|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|<br>Kernel|
|Access: sewer-<br>age network|0.52|0.04<br>(1.57)|0.12<br>(1.02)|0.11<br>(1.47)|0.01<br>(0.25)|0.11<br>(0.70)|0.07<br>(0.66)|0.06*<br>(1.81)|0.27*<br>(1.94)|0.16*<br>(1.80)|
|Spending on<br>sanitation|4.41|-1.47**<br>(-2.49)|-2.26<br>(-1.50)|-1.82<br>(-1.41)|0.003<br>(0.00)|-0.048<br>(-0.01)|-0.53<br>(-0.29)|-2.30***<br>(-2.81)|-1.60<br>(-0.67)|-3.07***<br>(-3.28)|
|Prevalence: renal|0.54|0.22*|0.11|0.20|0.99***|0.71|0.88**|-0.39**|-0.23|-0.44***|
|disease||(1.96)|(0.30)|(0.82)|(6.59)|(0.94)|(2.17)|(-2.45)|(-0.44)|(-4.01)|
|Prevalence:|0.09|-0.04|0.003|-0.04|-0.09|-0.51|-0.11***|0.09|0.19|0.03|
|bilharzia||(-0.62)|(0.03)|(-0.50)|(-1.57)|(-1.21)|(-4.52)|(0.83)|(1.03)|(0.17)|
|Spending on|99.1|0.79|24.25|9.83|3.20|19.96|10.09|6.59|8.72|9.55|
|health||(0.14)|(1.30)|(0.66)|(0.42)|(0.82)|(0.63)|(0.74)|(0.22)|(0.40)|
|Poverty rate|3.6|-0.80***<br>(-2.62)|-1.86<br>(-1.52)|-1.21*<br>(-1.66)|-0.15<br>(-0.34)|-1.43<br>(-1.02)|-0.97<br>(-0.86)|-1.07***<br>(-2.57)|-0.91<br>(-0.64)|-1.54*<br>(-1.80)|

_Notes:_ Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

Over the period 1991-2007, the SFD spent over LE 500 million on building and maintaining rural roads. As expected, these projects were associated with substantial reductions in household spending on transportation, by about LE 13 (compared to a mean of LE 66). The economic effects of these projects are quite strong: they raised the employment rate, and they increased farm income by almost half, presumably by lowering the cost of acquiring inputs and of delivering farm output to market. Only in the case where SFD-only areas are compared with no-project areas do roads also raise non-farm income; in the other comparisons non-farm income fell, perhaps as a result of greater competition from outsiders as areas become more accessible, or possibly because the increased attractiveness of farming led some people to cut back on non-farm activities. More surprisingly, there was no clear impact of the road projects on the poverty rate. A plausible explanation is that opening up villages to the outside world may raise incomes on average, but expose some vulnerable members of society to new economic pressures.

There is an interesting pattern that runs through Tables 9 to 13: as a general rule, SFD interventions have a stronger measurable effect when the comparison is between areas where there are SFD interventions alongside other projects (case C) and areas where there are only other projects (case E). A reasonable _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 20 of 36_
interpretation is that the SFD has the strongest impact when its efforts complement other projects, rather than when it operates alone. It is not entirely clear why this should be so, but it is possible that there are threshold effects at work: for instance, providing potable water to half a village may be helpful, but if the SFD then steps in and supplements such efforts, thereby ensuring everyone has potable water, it may be that only then are the effects on health appreciable.

**Table 13.** **<u>Summary of Effects of SFD Interventions in Road Maintenance and Improvement</u>**

|||SFD inte<br>inter<br>|rvention vs<br>vention[A<br>|. no SFD<br>vs. B]<br>|SFD int<br>interve<br>|ervention on<br>ntion at all[<br>|ly vs. no<br>D vs. F]<br>|SFD inter<br>other inter<br>|vention wit<br>ventions on<br>|h others vs.<br>ly [C vs. E]<br>|
|---|---|---|---|---|---|---|---|---|---|---|
||Mean|Regres-<br>sion|Nearest<br>Neighbor|<br>Kernel|Regres-<br>sion|Nearest<br>Neighbor|Kernel|Regres-<br>sion|Nearest<br>Neighbor|<br>Kernel|
|Spending on<br>transportation|65.8|-12.94***<br>(-6.76)|-5.93*<br>(-1.90)|-11.57***<br>(-4.20)|2.92<br>(0.86)|-4.40<br>(-0.60)|1.34<br>(0.25)|-17.79**<br>(-7.78)|-12.41**<br>(-2.38)|-16.68***<br>(-6.32)|
|% road utilization|0.15|-0.001<br>(-0.10)|-0.006<br> (-0.28)|0.01<br>(0.95)|-0.30**<br>(-2.08)|0.06<br>(0.20)|0.002<br>(0.14)|0.008<br>(0.82)|0.005<br>(0.21)|0.01<br>(0.74)|
|Employment rate|0.95|0.01***<br>(2.99)|0.01<br>(1.32)|0.01***<br>(2.70)|-0.01<br>(-1.38)|-0.01<br>(-0.85)|-0.002<br>(-0.20)|0.02***<br>(4.17)|0.01<br>(1.58)|0.02***<br>(2.90)|
|Farm-income†|68.5|27.8***<br>(6.68)|29.7***<br>(2.93)|20.9***<br>(2.84)|36.2***<br>(4.85)|35.3<br>(1.56)|37.9**<br>(2.30)|23.8***<br>(4.72)|14.0<br>(1.01)|15.2*<br>(1.82)|
|Non-farm<br>income†|54.6|-10.0*<br>-1.89|-6.1<br>-0.65|-11.4**<br>-2.08|22.48**<br>2.12|35.62<br>1.62|16.06<br>0.93|-20.4***<br>-3.48|-19.8**<br>-2.22|-22.1***<br>-3.63|
|Under-5<br>mortalityrate|19.9|-3.68<br>(-1.17)|-16.22<br>(-1.44)|-4.80<br>(-1.20)|-1.38<br>(-0.24)|2.74<br>(0.27)|-1.54<br>(-0.24)|-4.97<br>(-1.31)|-13.08<br>(-1.07)|-5.81<br>(-1.21)|
|Enrollment rate,<br>school|0.91|0.01<br>(1.24)|0.02<br>(1.21)|0.01<br>(0.92)|-0.002<br>(-0.16)|-0.02<br>(-0.56)|-0.004<br>(-0.18)|0.01<br>(1.42)|0.01<br>(0.39)|0.01<br>(1.02)|
|Poverty rate|3.8|0.01<br>(0.05)|-1.07<br>(-1.46)|-0.15<br>(-0.33)|0.66*<br>(1.67)|1.79<br>(1.29)|0.84<br>(0.96)|-0.36<br>(-1.23)|-0.68<br>(-0.79)|-0.38<br>(-0.77)|

_Notes:_ Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. † All coefficients shown here should be multiplied by 1,000. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

# _Microcredit_

The SFD does not lend directly to households, but since 1991 it has spent LE 1,062 million to support microcredit, channeling both working capital and grants to financial intermediaries that in turn do provide microcredit. SFD support for microcredit has gone to productive family associations (33%), NGOs (31%), and the PBDAC (20%), as well as to village banks (9%), and occasionally to commercial banks (2%) or other lenders (5%).

However, it is not possible to link individual loans with the activities of the SFD, so the approach that we take in this section is to estimate the impact of microcredit in general. Our working assumption is that if microcredit has an impact, then SFD support for microcredit has an impact. This is not a trivial assumption; if SFD support for microlending simply displaces other support for microlending, then SFD _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 21 of 36_
interventions in this area might have no net effect. Unfortunately, there is no way to determine whether this is the case.

In the Egyptian context, microcredit consists of extending modest amounts of credit to individuals. Microloans may not exceed LE 5,000 (about US$900), and most are much smaller: 48% of the microloans made in 2005 were for no more than LE 1,000, and only 7% were for more than LE 3,000.

The expectation is that such loans provide a way for households to generate income, and in the case of poor households, even rise out of poverty. In 2005, 44% of those who took out microloans worked in agriculture, and a further 30% were traders. Fully 55% of such loans went to women, and 42% of borrowers said they were illiterate. On the other hand, most of the loans are short-term; in 2005, 70% of microloans had a duration of less than a year, and 22% were repayable over 1-2 years. When asked (in the 2004/05 HIECS survey) whether microloans increased their income, 80% of households said yes, while the remaining 20% said no.

Our analysis of the impact of microcredit is disaggregated to the level of five regions – metropolitan areas, urban and rural Lower Egypt, and urban and rural Upper Egypt. This is done because of the belief that the effects of microcredit may vary substantially, depending on the underlying economic structure. We measure the impacts using propensity score matching; separate propensity score equations are estimated for each region, using household-level data – unlike our earlier estimates, which are based on village-level aggregates – and the models adapted until they were balanced. We employ the propensity scores to match “treated” (i.e. borrower) individuals in the region of common support with comparators, using kernel matching, nearest-neighbor matching, and a regression model. The results of the kernel matching are shown in Table 14, and in almost all cases were substantially the same as those found using the other two techniques; the two exceptions are noted in Table 14.

One clear pattern does emerge from Table 14: microlending appears to lead to substantially higher levels of non-farm income per capita, especially in urban areas. It is also associated with widespread selfemployment and, more modestly, with lower literacy rates.

Most of the other effects are not consistent from one region to the next. The sharpest distinction is between the metropolitan areas and urban Lower Egypt on the one hand, and the rest of the country on the other. In the former, microcredit is associated with higher levels of expenditure, food expenditure, and _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 22 of 36_
income, and lower levels of poverty. Elsewhere, microcredit comes with lower expenditure and income per capita and higher poverty rates.

**Table 14.** **<u>Summary of the Effects of Microcredit</u>**

||Metrop|olitan|Lower<br>Urb|Egypt:<br>an|Lower<br>Rur|Egypt:<br>al|Upper<br>Urb|Egypt:<br>an|Upper<br>Ru|Egypt:<br>ral|
|---|---|---|---|---|---|---|---|---|---|---|
||Coeff-||Coeff-||Coeff-||Coeff-||Coeff-||
||icient|t-stat|icient|t-stat|icient|t-stat|icient|t-stat|icient|t-stat|
|Farm income per capita|398.5|_5.99_|-61.8|**_1.82†_**|-141.4|_-3.73_|118.8|_2.80_|-257.1|_-7.05_|
|Non-farm income per capita|896.9|_4.70_|573.9|_4.44_|113.5|_2.93_|604.5|_5.02_|121.9|_3.79_|
|ln(expenditureper capita)|0.10|_3.00_|0.10|_2.42_|-0.01|_-0.71_|-0.25|_-8.91_|-0.14|_-9.10_|
|ln(income per capita)|0.12|_2.96_|-0.03|_-0.66_|-0.04|_-2.25_|-0.21|_-6.70_|-0.11|_-6.65_|
|ln(food expenditure per capita)|0.21|_6.49_|0.18|_4.22_|-0.01|_-0.64_|-0.11|**_-3.19†_**|-0.09|_-5.34_|
|Food as share of expenditure|0.05|_5.67_|0.04|_3.40_|0.001|_0.25_|0.06|_10.40_|0.02|_5.95_|
|Unemployment rate (in area)|0.03|_1.85_|0.01|_0.33_|0.04|_5.67_|-0.01|_-0.93_|0.01|_1.92_|
|% in area working for a wage|1.97|_0.85_|12.07|_2.95_|18.08|_11.35_|-3.82|_-1.36_|20.36|_15.36_|
|% in area self-employed|38.02|_10.29_|32.98|_7.35_|16.80|_11.60_|33.21|_14.63_|11.69|_8.79_|
|Illiteracy rate (in area)|-0.03|_-2.19_|-0.04|_-1.10_|-0.08|_-7.48_|-0.02|_-1.06_|-0.09|_-7.80_|
|Poverty gap rate (P1)|-0.59|_-6.58_|-0.76|_-2.65_|0.55|_2.00_|4.37|_5.76_|4.52|_8.38_|
|Headcountpovertyrate(P0)|-3.47|_-4.24_|-4.17|_-2.29_|3.15|_2.34_|16.75|_6.55_|16.99|_8.93_|
|Memo items:|||||||||||
|# of households in sample|2961||2514||7772||2558||6675||
|Number of treatment cases|235||116||659||269||827||
|Region of common support:|||||||||||
|Lr bound propensity score|0.01||0.01||0.01||0.02||0.05||
|Upr boundpropensityscore|0.33||0.13||0.46||0.29||0.30||
|Mean values of variables:|||||||||||
|Farm income per capita|55.37||124.09||735.54||120.99||709.71||
|Non-farm income per capita|955.29||887.50||354.76||722.19||245.28||
|ln(expenditureper capita)|8.15||7.87||7.68||7.80||7.43||
|ln(income per capita)|8.28||8.06||7.85||7.96||7.61||
|ln(food expend. per capita)|7.27||7.06||6.97||6.97||6.75||
|Food as share of expenditure|0.43||0.46||0.50||0.45||0.52||
|Unemployment rate (in area)|0.05||0.06||0.04||0.06||0.03||
|% in area working for a wage|76.15||61.07||45.46||62.81||41.45||
|% in area self-employed|20.92||31.02||40.37||31.57||42.79||
|Illiteracy rate (in area)|0.18||0.22||0.33||0.24||0.46||
|Poverty gap rate (P1)|0.59||1.22||1.65||3.26||6.88||
|Headcountpovertyrate(P0)|4.36||7.32||11.94||16.30||34.19||

_Notes:_ Results shown here are based on propensity score matching using kernel matching. Only treatment cases in the area of common support are matched. † Nearest-neighbor matching and regression model do not find a statistically significant effect here. Coefficients highlighted in bright yellow are statistically significant at the 5% level or higher; those highlighted in pale yellow are statistically significant at the 10% level.

_Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

On explanation for this surprising result is that matching techniques work well when the variables in the propensity score equation capture, and hence control for, all the relevant factors that distinguish one _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 23 of 36_
borrower from another. Inevitably, there are also unobservable influences that cannot be captured in survey data, and it is entirely possible that we have not been able to control satisfactorily for the fact that borrowers of microcredit have little education and are inherently poor, especially in the less-affluent regions of the country.

# **Costs Compared with Benefits**

Even if a program has an impact, it may not be worth undertaking if it is too costly. The results of a first pass at measuring the cost effectiveness of SFD programs (except for microcredit) are shown in Table 15. The procedure that we followed involves measuring the total impacts of each program, using only the statistically significant coefficients from Tables 8-13, and dividing into the total cost. The numbers are in Egyptian pounds, and are for 2004. For instance, in order to measure the cost per one fewer illiterate person, using the regression method and comparing A vs. B, we divided the cost of the program (LE 200m) by (-0.038 × population in A × adults as a share of the population). This gives a value of LE 1,842, which is the cost of pulling one person out of illiteracy, on the (not very reasonable) assumption that the eradication of illiteracy was the only goal of SFD educational spending.

The available information on the cost of each program is approximate, and represents a raw total of spending from 1991 to 2007; in the absence of annual numbers it was not possible to inflate earlier figures to reflect inflation, or to cut off the spending totals in 2004, the year for which the household and community survey data are available. Nonetheless, the spending totals do reflect the correct orders of magnitude involved.

As with the measures of the impacts themselves, the estimates of cost-effectiveness are not completely consistent. For instance, the provision of potable water or sanitation is associated with a reduction in poverty, at a cost (per one fewer poor) that averages somewhere between LE 24,684 (kernel estimates of A vs. B for sanitation) and LE 63,161 (regression estimates of A vs. B for potable water). Yet road schemes and literacy programs are associated with _more_ poverty – either the econometric results are not adequately controlling for program placement, or roads and reading allow the less-poor to leave, reducing the poverty rate of those who remain – and the effects of health interventions on poverty are either positive or negative, depending on the comparison that one favors.

The cost-effectiveness analysis, however interesting, is incomplete, because it does not try to balance benefits with costs, and it does not provide a mechanism for adding up the multiple benefits for any given _Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009_ _Page 24 of 36_
program, such as roads, or sanitation. The solution to undertake a cost-benefit analysis, which puts a monetary value on the benefits generated by the programs, and compares these values with the costs.

**Table 15.  Cost Effectiveness of Social Fund for Development Programs (in LE per outcome)**

||Regres-<br>sion|A vs. B<br>Nearest<br>Neigh-<br>bor|Kernel|Regres-<br>sion|D vs. F<br>Nearest<br>Neigh-<br>bor|Kernel|Regres-<br>sion|C vs. E<br>Nearest<br>Neigh-<br>bor|Kernel|
|---|---|---|---|---|---|---|---|---|---|
|**Education (LE 200m)**||||||||||
|Per certificate obtained|909||1,000|1,866||1,555|1,667|||
|Per one less illiterate|1,842|||3,682|||5,385|||
|Per one more job|||||||-9,723|||
|Per one fewerpoor|-89,952|-2,508|-3,683|||||||
|**Health (LE 390m)**||||||||||
|Per one less U-5 death|183,176||||||196,170||184,762|
|Per LE1 saved on health spending|13.1|||33.7|37.8|26.5|21.8||24.8|
|Per one fewerpoor||||15,594|9,814|12,568|-10,866|-8,241|-11,525|
|**Potable water (LE 620m)**||||||||||
|Per LE1 saved on water|146.2|130.4|157.9|250.6||403.4|322.5|170.3|236.6|
|Per one less U-5 death|||||||551,786|||
|Per LE1 saved on health spending|14.6|9.4|16.2|33.2|43.8|41.8|27.0|18.4|25.8|
|Per one fewerpoor|63,161||||||40,458|||
|**Sanitation (LE 220m)**||||||||||
|Per person getting access to sanitation|||||||24,399|5,741|9,946|
|Per LE1 saved on sanitation spending|203.6||||||678.9||508.0|
|Per person fewer with renal disease|-1,387|||-374||-421|4,045||3,541|
|Per person fewer with bilharzia||||||3,371||||
|Per one fewerpoor|37,552||24,684||||145,668||101,268|
|**Roads (LE 500m)**||||||||||
|Per LE1 of transport spending saved|27.8|60.7|31.1||||47.3|67.8|50.5|
|Road utilization||||0.002||||||
|Per one more job|59,903||54,457||||87,674||93,519|
|Per LE1 of additional farm income|12.9|12.1|17.2|17.3||16.6|35.4|||
|Per LE1 of additional non-farm inc|-36||-32|28|||-41|-43|-38|
|Per one fewerpoor||||-95,188||||||

_Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

There are two challenges in undertaking the cost-benefit analysis: first, one has to measure the total impact of the programs, which we have done above; and second, it is necessary to value each output. For the latter, we drew on secondary sources, using the values that are set out in Table 16. We suppose that the benefit of pulling someone out of illiteracy is the avoided cost of schooling per child that completes fifth grade. This treats literacy as a right that needs to be provided, rather than valuing it solely in terms of labor market outcomes. We value an additional job by measuring the economic rent that accrues when someone who was not working now has full-time employment. This may be measured as the wage _Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009_ _Page 25 of 36_
actually received minus the opportunity cost of supplying that labor (see Boardman et al. 2006, chapter 11). To measure the wage appropriate to the type of jobs that SFD interventions are likely to help create, we used the private sector monthly wages received by those with an elementary school education or less in 2004 (Said 2007, Table A2), adjusted to the prices of 2004; in the absence of further information, we assume that the labor rent constitutes half of this total, and we also assume that the jobs created last for, on average, five years. This gives a total value per job created of LE 10,700 (in 2004). This is substantially less than the “cost of creating a job,” which Ghoneim (2006) puts at LE 40,000 in the relatively labor-intensive garment sector and MEMRI (2002) pegs at LE 50,000 in agriculture.

Some SFD programs reduce poverty. One way to put a value on this is to measure the increase in expenditure that would be required to raise a poor person, whose expenditure is at the mean level for the poor, out of poverty. The World Bank (2007) shows that the annual poverty deficit per poor person is LE 266 per year, or LE 1,333 over a period of five years, which we take to be the effective life of an SFD poverty-reducing intervention.

**<u>Table 16: Benefits per unit of output produced by SFD projects</u>**

|Impact to be valued|LE|Comments and Sources|
|---|---|---|
|Per one less illiterate|1,157|The benefit is that we save the cost of educating a child to the fifth grade. The cost<br>per fifth-grade completer was LE 911 in 1998-99 (USAID 2006, based on Zaalouk<br>2006); this was grossed up by 27% to reflect consumer price inflation between<br>1998-99 and 2004 (World Bank,_World Development Indicators_, online, accessed<br>March 7,2009).|
|Per one more job|10,700|Measures the economic rent obtained by working; assumed to be half of the wage<br>earned in private sector employment by those with an elementary school education<br>or less in 2006 (adjusted to the prices of 2004); the wage data come from Said<br>(2007).|
|Per one fewer poor|1,333|The poverty line is LE 1,423 per person per year. Using this line, a poverty rate of<br>19.6%, and data on income distribution in 2004 from PovcalNet, we fitted a<br>quadratic approximation to the Lorenz curve (see Datt 1998) and estimated the<br>average expenditure of the poor to be LE 1,175. We assume that when someone is<br>brought out of poverty, their spending rises from LE 1,175 to LE 1,423 (i.e. by LE<br>248),and maintains this improvement for fiveyears.|
|Per one less U-5 death|796,000|Boardman et al. (2006, chapter 15) conclude that a value of US$4 m ± USD$2 m,<br>in 2002 prices, is appropriate for the US; Viscusi and Aldy (2003) in their meta-<br>analysis use a value of US$6.7m (in 2000 prices). We assume that the value of a<br>statistical life in Egypt is the same multiple of GDP/capita as in the US.|
|Per LE1 saved on medicines (or water,<br>or sanitation,or health,or transport)|5|When a project saves LE 1 in spending, we assume that this lasts for five years.|
|Per person less with renal disease|75,222|The Disability-Adjusted Life Year (DALY) of nephritis is 0.091 and of nephrosis is<br>0.098, which means that one less case of renal disease saves the equivalent of<br>0.0945 of the value of a life(Mathers et al. 2001).|
|Per person less with bilharzia|4,776|The Disability-Adjusted Life Year (DALY) of a bilharzia infection is 0.006, which<br>means it is equivalent to 0.006 of the value of a life(Mathers et al. 2001).|
|Per LE1 of additional farm income, or<br>additional non-farm income|5|We assume that this effect lasts for five years.|

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 26 of 36_
Where SFD interventions reduce mortality, it is necessary to use a value of a statistical life. To the best of our knowledge, this has not been estimated in the context of Egypt. Based on their review of studies undertaken in the United States, Boardman et al. (2006) believe that in the United States a statistical life may be valued at US$4m ± 2m. We adjust this number to reflect the lower levels of income in Egypt than in the United States, giving a value of LE 796,000 in 2004 prices. Sanitation projects also reduce the incidence of disease, and these too need to be valued. One approach is to estimate the Disability-Adjusted Life Years (DALYs); for each medical condition, the DALY measures the estimated reduction in life expectancy plus a measure of the handicap faced by sufferers. For an incident of bilharzia, the DALY is 0.006 (Mathers et al. 2001), which means that each case of bilharzia that is avoided leads to a saving equivalent to 0.006 of the value of a life. The DALY for renal disease is 0.0945.

The combined impact of these effects is shown in Table 17. Consider, for example, the regression estimates of the effects of SFD road projects, comparing areas with SFD projects (A) with areas that do not have SFD projects (B). The total benefit of these projects come from savings in transport costs (LE 90m), job creation (LE 334m), higher farm income (LE 193m), adjusted for lower non-farm income (-LE 70m), for a net total benefit of LE 548 m. This exceeds the cost of the road projects (LE 500m), indicating that these projects appear to be economically worthwhile. Since some of these results are sensitive to the valuation of a statistical life, we also report the totals under the assumption that a statistical life has onefifth of the valuation discussed above.

The most consistent results are for SFD road projects, where the measured benefits are generally positive, and in the case of the A vs. B comparison, are arguably higher than the costs. The net benefits are not as large in the D vs. F and C vs. E comparisons, where they typically do not exceed half of the cost of the program. For SFD potable water projects, the benefits are also generally positive, but in only one case (the regression estimate based on C vs. E, assuming a high value of a statistical life) do the benefits exceed the cost. The SFD health projects generate modest benefits in raising people out of poverty and reducing private spending on health care, but by some of the econometric measures they lower child mortality. If the value that we use for a statistical life is appropriate, then the main benefit of these health projects occurs by reducing child deaths.

The benefits of the education projects are rather modest, relative to the cost; and the effects of the sanitation projects cannot be determined with much reliability; they are driven by the effects on renal disease, which in turn are highly sensitive to the value put on a statistical life; in this case some of the _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 27 of 36_
econometric estimates have an unexpected sign, implying, if taken at face value, that some of the projects may have made some health outcomes worse.

**Table 17.  Cost-Benefit Analysis of Social Fund for Development Programs**

|||A vs. B|||D vs. F|||C vs. E||
|---|---|---|---|---|---|---|---|---|---|
||Regres-<br>sion|Nearest<br>Neigh-<br>bor|Kernel|Regres-<br>sion|Nearest<br>Neigh-<br>bor|Kernel|Regres-<br>sion|Nearest<br>Neigh-<br>bor|Kernel|
|**Education (LE 200m)**||||||||||
|Per one less illiterate (LE 1,157)|126|||63|||43|||
|Per one more job (LE 10,700)|||||||-220|||
|Per one fewerpoor(LE 248/yr × 5)|-3|-106|-72|||||||
|**Total **|**123**|**-106**|**-72**|**63**|||**-177**|||
|**Total, with lower value of life**|**123**|**-106**|**-72**|**63**|||**-177**|||
|**Health (LE 390m)**||||||||||
|Per one less U-5 death (LE 0.796m)|1,695||||||1,583||1,680|
|Per LE1 saved on health spending (× 5)|149|||58|52|74|89||79|
|Per one fewerpoor(LE 248/yr × 5)||||33|53|41|-48|-63|-45|
|**Total**|**1,844**|||**91**|**104**|**115**|**1,624**|**-63**|**1,714**|
|**Total, with lower value of life**|**319**|||**91**|**104**|**115**|**200**|**-63**|**202**|
|**Potable water (LE 620m)**||||||||||
|Per LE1 saved on water (× 5)|21|24|20|12||8|10|18|13|
|Per one less U-5 death (LE 0.796m)|||||||894|||
|Per LE1 saved on health spending (× 5)|212|330|191|93|71|74|115|168|120|
|Per one fewerpoor(LE 248/yr × 5)|13||||||20|||
|**Total **|**247**|**354 **|**211**|**106**|**71**|**82 **|**1,039**|**186**|**133**|
|**Total, with lower value of life**|**247**|**354**|**211**|**106**|**71**|**82**|**234**|**186**|**133**|
|**Sanitation (LE 220m)**||||||||||
|Per LE1 saved on sanitation spending (× 5)<br>Per person fewer with renal disease<br>(0.0945 × life)<br>Per person fewer with bilharzia (0.006 ×<br>life)|5<br>-11,928|||-44,269||-39,271<br>312|2<br>4,091||2<br>4,674|
|Per one fewerpoor(LE 248/yr × 5)|8||12||||2||3|
|**Total **|**-11.915**||**12**|**-44,269**||**-38,960**|**4,094 **||**4.679**|
|**Total, with lower value of life**|**-1,180**||**12**|**-4,427**||**-3,896**|**413**||**472**|
|**Roads (LE 500m)**||||||||||
|Per LE1 of transport spending saved (× 5)|90|41|80||||53|37|50|
|Per one more job (LE 10,700)|334||367||||228||214|
|Per LE1 of additional farm income (× 5)|193|207|145|144||151|71|||
|Per LE1 of additional non-farm inc (× 5)|-70||-79|90|||-61|-59|-66|
|Per one fewer poor (LE 248/yr × 5)||||-7||||||
|**Total**|**548**|**248**|**514**|**227**||**151**|**291**|**-22**|**198**|
|**Total, with lower value of life**|**548**|**248**|**514**|**227**||**151**|**291**|**-22**|**198**|

_Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 28 of 36_
This cost-benefit exercise rests on fairly strong assumptions, and so is somewhat tenuous; better measurement might help alleviate some of the problems. However, even these rudimentary numbers are useful enough to help concentrate minds for the future: the SFD education and sanitation projects do not look compelling, and even the other projects (health, potable water, and roads) have benefits that do not generally exceed the costs. There are certainly other worthwhile justifications for SFD projects – equity, the satisfaction of human rights, political imperatives – but on strictly economic terms none of them look particularly compelling based on the available evidence.

# **Targeting**

Even when the costs of a program exceed its benefits, the program might be justifiable on distributional grounds; for example, although minimum wage laws are inefficient, they may be one of the more effective ways of channeling resources to the poor (Gramlich 1997). This naturally leads us to ask whether the SFD programs were successful in targeting the poor. During the first phase of its work (19921996) the SFD functioned primarily as an emergency fund and safety net to support implementation of the Economic Reform and Structural Adjustment Program (ESAP), and it sought to target its assistance geographically based on poverty and unemployment rates at the level of the 27 governorates, broken down where possible into urban and rural divisions.<sup>3</sup> The SFD has since transitioned into a permanent development organization with the twin goals of supporting and developing micro- and small enterprises, and enhancing human capital for the poor, and so retains its original focus on combating poverty. A poverty-targeting unit has now been established within the SFD to help it identify more clearly those areas of the country where interventions are likely to have the greatest impact on the poor.

The first question to ask is how effectively the SFD has targeted its spending _at the governorate level_ . Given its stated commitment to targeting the poor, one might expect SFD spending to go disproportionately to governorates with a higher level of poverty. We measure this visually in Figure 3: the horizontal axis shows the cumulative percentage of the population, obtained by sorting governorates from those with the highest to those with the lowest level of poverty. The vertical axis shows cumulative percentages; the smooth regular curve tracks the incidence of poverty and shows, for instance, that 80% of the poor in Egypt live in the poorest half of the governorates (as measured by population).

If SFD spending were distributed to governorates strictly in proportion to population, then it would follow the diagonal in Figure 3. This is close to being the case for microcredit projects (Figure 3, bottom panel).

> 3 As of 2004 there were 27 governorates; two more (Helwan, and 6<sup>th</sup> of October) were created in April 2008.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 29 of 36_
On the other hand, spending on health projects comes close to tracking the number of poor people by governorate, at least in the governorates with the highest incidence of poverty. It is clear from Figure 3 that despite SFD claims that its programs are allocated largely based on the incidence of poverty at the governorate level, all of them fall short of allocating spending (at the governorate level) in proportion to the number of poor.

<!-- Start of picture text --> 100<br>90<br>Poverty<br>80<br>70<br>Community<br>Development<br>60<br>50<br>40<br>Education<br>30<br>20<br>10<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>Cumulative % of population (by governorate, from poorest to least poor)<br>100<br>90<br>Poverty<br>80<br>70<br>Public Works<br>60<br>Health<br>50<br>40<br>30<br>Microcredit<br>20<br>10<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>Cumulative % of population (by governorate, from poorest to least poor)<br>Cumulative % of poverty, etc<br>Cumulative % of poverty, etc<br><!-- End of picture text -->

**Figure 3. The incidence of poverty, and of SFD spending, at the governorate level** _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

Even if SFD programs are not allocated to governorates in proportion to poverty, they could be well targeted to the poor _at the village or ward level_ – for instance, by going disproportionately to poor villages, even in rich governorates. With the exception of microcredit, we do not have household-level _Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 30 of 36_
information on the targeting of SFD programs, but we do have information from the HIECS 2004/2005 at the village level, and so can determine whether SFD spending is directed mainly towards poor villages (even if this is no guarantee that the poor, rather than the non-poor, are the ultimate beneficiaries). Figure 4 displays the relevant information; it shows the cumulative percentage of the population from the poorest to the richest villages and wards on the horizontal axis, while the vertical axis shows the cumulative percentage of SFD spending on the various programs; for each program there is a different “concentration curves”. Although it is not strictly comparable, since it is based on household-level data, Figure 4 also shows a Lorenz curve – the heavy line to the lower right – which tracks the distribution of per capita income in 2004.

<!-- Start of picture text --> 100<br>90<br>80<br>Roads<br>70<br>Microcredit<br>60<br>50<br>Education Income Distribution<br>40 (Lorenz Curve)<br>30<br>20<br>10<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>Cumulative % of population (poor to rich)<br>100<br>90<br>80<br>Potable Water<br>70<br>Health<br>60<br>Environment<br>50<br>40<br>Waste Water Income Distribution<br>30 (Lorenz Curve)<br>20<br>10<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>Cumulative % of population (poor to rich)<br>Cumulative % of SFD spending<br>Cumulative % of SFD spending<br><!-- End of picture text -->

**Figure 4. Distribution of Spending on SFD Programs, Based on Village/Ward-Level Data, 2004-05.** _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 31 of 36_
The thin diagonal in Figure 4 shows the line of “perfect equality” and serves as a useful reference. For two of the SFD programs – education, and waste water – the cumulative percentage of spending is below the line of perfect equality. This means, for instance, that the poorest 20% of villages (by population) receive less than 20% of SFD spending on education, or on waste water treatment. Spending on these programs thus favors the better-off over the poor.<sup>4</sup> All the other SFD programs have concentration curves above the line of perfect equality, meaning that spending is geared more towards the poor than the rich (at the village level). Spending on potable water projects, and to a lesser extent on road and health interventions, is relatively strongly geared toward poorer villages. While these effects are clear, they are not overwhelming; the poorest 30% of the population lives in villages that received 43% of SFD potable water spending, and 35% of SFD health and road spending. These reflect a pro-poor bias, but not a strong one.

Table 18 provides some further information on SFD targeting. For each SFD program, it shows the headcount poverty rate in villages with, and without, an SFD intervention. For instance, the poverty rate in villages and wards with SFD road projects was 21.8%, compared to 19.5% in areas without an SFD road project, a difference that is statistically significant (t=-5.03); the national poverty rate in 2004-05 was 19.6%. The numbers confirm that SFD education and wastewater treatment projects were not well targeted to villages with higher poverty rates, while road, health, and (especially) potable water projects were more successfully targeted to the poor. The poverty rate in villages and wards that received any SFD intervention was 21.6%, which is significantly higher than the 18.6% poverty rate in areas with no SFD project.

The numbers in Table 18 allow us to check the robustness of these findings. Let us suppose that there are six basic needs, as follows:

- (a) Access to drinking water on the property;

- (b) Having a toilet other than a simple pit latrine;

- (c) Primary-age children are enrolled in school;

- (d) The household head has at least three years of education and is employed; or at least one in every three household members is employed;

- (e) There are no more than three persons per room (excluding bathrooms); and

> 4 Public spending is “regressive” if it represents a smaller percentage of household income (or spending) for the poor than for the rich; this would be reflected in a concentration curve below the Lorenz curve, which is not the case even for SFD spending on education or waste water treatment.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 32 of 36_
(f) The dwelling is not improvised, and the walls are made of brick (or better).

Based on the HIECS 2004/2005 data, 6.2% of the Egyptian population met all of these basic needs, while 14.2% had three or more unmet basic needs. Defining the poor as those with at least three unmet basic needs, we see that only SFD health and potable water projects were targeted to villages with disproportionately high levels of this type of poverty.

**<u>Table 18.  Poverty Rates in Areas Served by SFD Projects</u>**

||Headcount po|verty rate (%)|t-statistic for<br>difference in|% of household<br>unmet ba|s with at least 3<br>sic needs|
|---|---|---|---|---|---|
||No SFD<br>project|With SFD<br>project|poverty rates|No SFD<br>project|With SFD<br>project|
|Education|19.8|15.0|14.2|14.5|10.1|
|Wastewater treatment|19.6|12.8|9.9|14.2|13.2|
|Environment|19.5|20.1|-1.7|14.2|13.8|
|Roads|19.5|21.8|-5.0|14.3|13.2|
|Health|19.5|22.1|-6.0|14.2|14.7|
|Microcredit|18.8|23.1|-18.1|14.1|14.9|
|Potable water|18.2|30.2|-38.6|13.5|19.5|
|Microcredit*||45.6||||
|Anyintervention|18.6|21.6|-15.6|14.2|14.3|

Note: * Based on household-level data. All other figures are based on village/ward-level data from HIECS 2004/2005. _Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

These numbers likely understate the extent to which SFD programs reach poor households if, as is plausible (although not certain), the actual beneficiaries of SFD programs in a given village are disproportionately poor. In the case of microcredit it is possible to identify the households that avail of the credit, using the HIECS 2005/2005 household data. The relevant numbers are set out in Table 19, and they show that the poorest 10% of the population account for 24.9% of all microcredit loans; furthermore, the headcount poverty rate among microcredit users was 45.6%, or far above the national rate of 19.6% (in 2004-05). Without further information on the size of loans, we cannot tell what proportion of the _value_ of microcredit goes to the poorest households, but the indications are that microcredit is very highly targeted to the poor. However, it must be noted that these numbers refer to all microcredit, and not specifically to SFD-supported microcredit, which individuals cannot identify separately, mainly because SFD support for microcredit operates indirectly via NGOs and financial intermediaries.

# **Conclusions**

In our examination of the Egyptian Social Fund for Development, we have focused on three key questions: How large is the impact of the SFD interventions? Have the benefits been commensurate with the costs? And have the programs been targeted successfully to the poor?

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 33 of 36_
**Table 19.  Percentage** **<u>of direct beneficiaries of microcredit lending, by decile, 2004-05.</u>**

||Urban Areas|Rural Areas|All Egypt|
|---|---|---|---|
|**Deciles**||||
|Poorest|23.8|25.2|24.9|
|2|9.9|18.1|16.2|
|3|8.9|15.6|14.1|
|4|7.8|11.8|10.8|
|5|6.1|9.1|8.4|
|6|7.8|6.5|6.8|
|7|11.3|5.8|7.1|
|8|8.0|3.7|4.7|
|9|8.9|3.0|4.4|
|Richest|7.7|1.3|2.8|
|**_Memo:_**||||
|Poverty rate, beneficiaries|33.1|49.4|45.6|
|Povertyrate, population|10.1|26.8|19.6|

_Source:_ Authors’ calculations based on HIECS and community survey 2004/2005.

We have sought to measure the impact of six separate SFD programs; in every case there are several measurable outcomes, each estimated in at least three distinct ways for three types of comparisons. This gives well over 200 measures of impact. Inevitably, with such a large body of estimates, some of the results do not appear to be entirely consistent, and a few are either surprising or implausible. That said, the general thrust of the estimates is that SFD programs have had clear and measurable effects, in the expected direction, for all of the programs considered here.

Educational interventions have reduced illiteracy, but have had no clear effect on employment; health programs have brought down household spending on health by almost a fifth, and markedly reduced under-5 mortality; potable water provision has lowered household spending both on water and health, and has also reduced under-5 mortality; sanitation interventions have cut household spending on sanitation and lowered poverty; and road projects are associated with a 20 percent drop in household transportation costs, and a higher employment rate. As a general rule, SFD interventions have had the strongest identifiable impact when they have complemented other projects, rather than operating alone.

The SFD supports microcredit, but does not lend directly to households. Our analysis of the impact of microlending in general finds that it is associated with higher levels of non-farm income per capita. In the metropolitan areas and in urban Upper Egypt, microcredit goes together with higher household expenditures and less poverty, although this association is not evident elsewhere.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 34 of 36_
The SFD has a mandate to reduce poverty, increase employment opportunities, and encourage smallenterprise development. This focus on the social dimension means that the SFD has not traditionally needed to trade off costs with economic benefits, and so it cannot be judged by this measure. An economic cost-benefit analysis does have a role, however, if only in identifying the most easily-justifiable SFD activities.

We find that SFD road projects generate benefits that, by some estimates, exceed the costs. This is also true of health and potable water interventions, at least if the value put on a statistical life is not too low. The strictly economic benefits of educational projects are more modest relative to the costs; and the benefits from sanitation projects are highly sensitive to the econometric measures used to quantify their impact, and so the measures are unreliable. It should be emphasized that the valuation of unit benefits is based on secondary sources, and so requires strong assumptions that are difficulty to verify independently.

Even where SFD interventions may not pass a cost-benefit test, they may well be justified on other grounds – as an efficient way to achieve social equity or to further human rights, or to satisfy political needs. With the exception of microcredit, it is not possible to associate program benefits with individual households, so in most cases we have only been able to measure the effectiveness with which the SFD has targeted its interventions to poor areas, whether poor governorates, or poor villages/wards.

The SFD interventions that we have analyzed go to areas that have a higher headcount poverty rate than the national average (21.6% vs. 19.6%), and a lower level of average per capita annual household expenditure (LE 2,292 vs. LE 2,556). In this broad sense, SFD interventions are pro-poor, although at first sight only modestly so. A household-level analysis of targeting was only possible for microcredit, and in this case the poverty rate among borrowers was 45.6%, or more than twice the national average. Projects related to roads, health, and (especially) potable water were also targeted to poor areas, but spending on education and on wastewater treatment went disproportionately to areas with lower-thanaverage village-level poverty rates. If, within these villages, the beneficiaries were generally poor, then even these interventions may have been pro-poor.

There is unfinished business. The data were not sufficiently rich to allow us to do a complete cost-benefit analysis of the SFD microcredit programs, or of the impact, costs, or benefits of the largest single SFD program, which provides support to small and medium enterprises.

_Impact Evaluation: Egyptian Social Fund for Development_ _Version of June 29, 2009_ _Page 35 of 36_