may arguably face problems of serial under-reporting in project completion reports. Insofar as selfassessments of overall performance can be relied upon to accurately reflect practitioner experiences, the delivery challenge framework can serve as a useful analytical tool for assessing project success and Bank performance. This assumption is in line with previous literature, which similarly relies on such data (Kilby and Michaelowa, 2019). Testing delivery challenges against subsequently-assigned evaluation ratings provides an additional benefit, highlighting which challenges actually end up substantively impacting service delivery and performance. This in itself serves a useful purpose, conditioning prior expectations of overall performance against independently-assigned posterior evaluations.

Delivery challenges emulate measures such as Country Policy and Institutional Assessment (CPIA) scores in this regard, offering a comprehensive diagnosis of project weaknesses. But where CPIA scores cover 16 criteria over four clusters, delivery challenges disaggregate issue-areas across a more comprehensive set of issues. In addition to covering dimensions that are simply not covered by the aforementioned (including contextual determinants, specific project design attributes, and stakeholder effects), the delivery challenge taxonomy offers more granularity than other instruments, something that has been noted in qualitative applications of the data in previous analytic work.<sup>7</sup> This allows us to take a holistic view of the factors that impact project outcomes and Bank performance.

# **3.2 Outcome Measures**

Following previous literature, we employ the Independent Evaluation Group’s (IEG) rating of project outcomes as the main dependent variable in our analysis. Outcomes are benchmarked against the achievement of stated project objectives, standards, and expectations, as laid out in the project design.<sup>8</sup> Though the measure is conditioned on the scope of development objectives laid out at the time of approval, it provides several advantages for our analysis. First, IEG’s ratings have become a standard measure in a growing body of literature that use them as a heuristic for project success.<sup>9</sup> Second, IEG’s operationalization of project outcomes provides a degree of external validity to the general concept under exploration. All projects are subjected to a desk review, and approximately 25% of the projects completed each year undergo a more detailed evaluation. IEG evaluates World Bank projects according to a six-point scale ranging from highly unsatisfactory to highly satisfactory. Though not a perfect measure of project performance, using IEG evaluation scores allows for a consistent analysis of a large sample of projects, offering similar advantages as the delivery challenge taxonomy in terms of both replicability and generalization.

Furthermore, the models are tested against IEG’s performance and project supervision variables. The Bank’s overall performance is defined as a function of two indicators, measuring the ability to ensure the quality of the project at entry and maintaining quality of supervision during implementation. Analogous to the outcome measure, the performance variable more explicitly focuses on how successful the Bank was in implementing a project from inception to conclusion. Testing the

> 7For an illustrative narrative of delivery challenges affecting water service delivery projects, see: http://globaldeliveryinitiative.org/library/case-studies/how-shape-enabling-environment-sustainable-water-servicedelivery-nigeria. Another example of the delivery challenge concept can be found here, this time in relation to the Health, Nutrition, and Population sector: http://globaldeliveryinitiative.org/library/case-studies/reducing-riskhivaids-among-female-sex-workers-india.

> 8For more on data sources, see https://ieg.worldbankgroup.org/sites/default/files/Data/reports/ieg-wb-projectperformance-ratings-codebook.pdf

> 9See Denizer, Kaufmann, and Kraay (2013) for a more in-depth discussion of this norm.

9