from a very low baseline, IE came to cover about 10 percent of the Bank’s projects and establish, through counterfactual analysis, the causal pathways of what policies financed by these projects work and why.<sup>6</sup> The Bank’s impact evaluations involve research and field staff working with the project and government teams to introduce structured and iterative learning in project operations. They aim to inform policy design, identify policy-relevant research questions, run experiments, and collect and analyze data, to feed evidence back into policy decisions during implementation. At their best, impact evaluations are a framework of collaboration for the generation and real-time use of policy-relevant evidence.

This study investigates the role of these impact evaluations on the quality of projects. For this purpose, we assume that we can summarize the quality of projects by three conditions: (i) the problem/goal is correctly identified, (ii) an appropriate course of action and financing is agreed to, and (iii) project activities are implemented as planned. While impact evaluation can potentially influence (i-iii), it is hard either to measure or find good proxies for (i) and (ii). We thus restrict our analysis to (iii), which is measurable and a necessary condition for projects to achieve their impact.

While measuring the link between IE and development outcomes of projects is of great interest to the development community, it is at best challenging. There is plenty of evidence established on a case-by-case basis,<sup>7</sup> but estimating the impact of IEs in general requires a valid counterfactual and a common impact metric across the policies being evaluated. Constructing such a counterfactual seems unfeasible.

Our second-best, but tractable, approach is to investigate the link between IE and project implementation. To do this, we construct a panel using administrative data on the whole portfolio of World Bank impact evaluations and projects approved during 2005–11. Linking information on projects’ disbursements with information on impact evaluation allows us to explore the relationship between IE and project implementation using 100 impact evaluations and 1,135 projects implemented in 114 countries in 11 sectors during 28 quarters.

We use project disbursements as a measure of implementation. This is because disbursements are made against evidence that the agreed activities are being implemented by the relevant government and supervised by the Bank project staff. We use three specific quarterly measures: (i) cumulative disbursements relative to loan size, (ii) the gap between planned and actual disbursements relative to loan size, and (iii) the ratio of project to average disbursements for all projects in the relevant sector. To our knowledge, this is the first quantitative study aiming at identifying the effect of IE on the implementation of development projects.

> 6 In addition to IE, all Bank projects must complete an implementation completion report. About a third of them are also subject to project evaluation by IEG. These assessments are descriptive in nature, not causal.

> 7 Examples from World Bank IE work are Bruhn et al. (2013), Brune et al. (2011), Friedman et al. (2010), Kondylis, Jones, and Stein (2014).

3