The analysis takes advantage of a novel dataset examining the challenges that are faced throughout the project life cycle, from preparation to completion. Using data mining techniques and automated content analysis of over 5,000 World Bank project completion reports, we compile a taxonomy of 42 “delivery challenges,” mapping problems faced by Task Team Leaders across time, region, and sector. The challenges are clustered by project, stakeholder, and context, with the latter group serving as structural controls that plausibly impact delivery and effectiveness. These data are then analyzed using Bayesian model averaging, addressing variations in parameter estimates induced by variable specification effects. Posterior probabilities for each delivery challenge are conditioned on the overall likelihood of having a substantive effect across all possible model configurations. These are further tested alongside a battery of controls and fixed effects, offering a sense of their overall impact on the aggregate subset of World Bank projects tested. Delivery challenges are assessed with respect to their impact on three outcome measures: the achievement of stated development objectives (outcomes), the World Bank’s effectiveness in carrying out the project (performance), and the proactive identification of potential risks to overall project success (supervision).

Analytically, delivery challenges serve as powerful measures, standardizing the set of obstacles encountered throughout the project life cycle and offering a cross-cutting assessment of overall performance. Paired with the model averaging procedure described above, this approach offers two advantages. First, it provides a more granular measure of impediments than previous work, looking at the potential correlates of project success at every node along the development and execution life cycle. This is of value to scholars and practitioners alike, as it offers both a more nuanced exploration of outcomes as well as a diagnostic tool to detect _ex-ante_ the potential challenges that the World Bank may face in preparing and supervising projects. With reference to the latter, the practical utility of the DeCODE framework has already been demonstrated within country-<sup>1</sup> and sector-specific challenges.<sup>2</sup> This paper contributes to our understanding of the impact these challenges have relative to project outcomes and World Bank performance overall.

Second, the identification of delivery challenges provides a way to capture variation in project performance across a broad set of logged project case histories. By drawing on implementation completion reports, we strive to aggregate and analyze the impact of obstacles that are organically defined by those who supervise disbursement projects in the first place. Put together, this allows for a holistic analysis of success and failure across the project’s lifespan. By using a cross-cutting measure of delivery challenges, this paper identifies the most likely issues projects face, as well as the magnitude of effects posed by said challenges. Matching these factors to the sequence in which delivery challenges are likely to be expected, we develop a typology for structured and focused analysis. This is especially helpful in addressing issues related to model selection and diagnosis, as it provides a means of simultaneously assessing the various dimensions along which projects have historically under-performed.

The data and methods proposed above serve to generate standardized coefficients for the most impactful challenges faced in the implementation of World Bank projects. Though some of the measures are comparatively less informative, the examination of the full corpus of delivery challenges allows us to weigh the relative impact of each. The paper is organized as follows. Section 2 offers an overview of the relevant literature on project effectiveness. Section 3 summarizes the conceptual framework, including a description of the delivery challenge taxonomy, the outcome measures tested, and the methodology. Section 4 presents the results of the analysis, Section 5 presents findings on World Bank performance and outcomes, and Section 6 concludes.

> 1Link: How to Overcome Delivery Gaps: Some Examples from Korea

> 2Link: Addressing Delivery Challenges in Education: Lessons from Millions of Learning Case Studies

2