36 fundamental role in this process. This section briefly presents the method, summarizes some results from case studies directly related to capital formation, and finally presents some caveats when drawing lessons for policy from these evaluations.

# **1. Method**<sup>**19**</sup>

An impact evaluation assesses changes in an outcome indicator (education attainment, health, income level, or well-being) of individuals, households, communities, or firms, that can be attributed to a particular project, program or policy. The central impact evaluation question is what would have happened to those affected by the intervention if they had not in fact been affected by it. Since this is unobservable, the key challenge is to specify a counterfactual – that is, to select a group which is as similar as possible (in observable and unobservable dimensions) to those affected by the intervention. This comparison allows assessment of causality – attributing observed changes in the outcome indicator to the program, while removing confounding factors.

There are alternative methods to specify the counterfactual, that is, the selection of the comparison group that will be used to measure the impact of the intervention. Which method is chosen will depend on numerous considerations, such as the type of project, the evaluation question, the time constraint, data availability. The methods have different advantages and disadvantages that have been reviewed and presented in detail elsewhere (Baker, 2000 and Ravallion, 2001, 2005). Here, just an intuitive summary follows.

The core of impact evaluation is to select a comparison group, which should be identical (or as similar as possible) to the treatment group, with the key difference that the comparison group did not benefit from the program. The design of the comparison group is oriented to control for any bias due to differences in observable characteristics (schooling, geographic location, participation in the labor market, etc.) or unobservable ones (skill, motivation, connections, etc.) between the two groups. There are alternative methods to assign individuals into the treatment (beneficiary or affected by the intervention) and control (unaffected by the intervention) groups.

The first method, randomization, allocates the intervention randomly among eligible beneficiaries. The random selection of treatment and control groups ensures there is no bias, and hence, it is considered the most robust evaluation method. The disadvantages of the method emerge from the likelihood of its applicability due to the political circumstances that enable to select randomly beneficiaries for a program, or to ethical considerations that allow the policymaker to deny treatment to somebody that actually needs it. The method is also inapplicable in projects that have national coverage due to the impossibility of selecting a comparison (no-treatment group). This method is also known as an experimental design of the evaluation.

Other methods, known as non experimental, generate the comparison group that resembles the treatment group through econometric techniques. The selection into the 19 This section draws on material presented in more detail in the impact evaluation website of the World Bank, on the Handbook on this topic (Baker, 2000), and several papers by M. Ravallion.