appears to serve as an informative heuristic for weaker projects in need of readjustment.

Taken together, this output appears to confirm pre-existing expectations of what ought to impact performance ratings, benchmarking the impact of relative challenges in order to provide an overview of how these features behave relative to one another. In order to complete the analysis, the next section will focus explicitly on the supervision feature of the performance measure.

## **4.4 Posterior Probabilities: Project Supervision**

Figure 7 shows cumulative model probabilities for a model averaging run of delivery challenges on project supervision ratings. Given that the outcome measure constituted one of the two components in the Bank performance indicator, results across the two runs should be roughly comparable. This appears to be the case. Project Finance and environmental challenges are less significant a measures on supervision, replaced by challenges impacting H.R. & Organizational Capacity. This appears to follow our prior expectations for understanding the determinants of success in supervision. Examining the posterior coefficients in Table 10, we see that Project Data & Monitoring challenges expectedly have the highest negative posterior coefficients on supervision (-0.109), followed by Project Design challenges (-0.067). On the other hand, Macroeconomic challenges as well as problems related to Coordination & Engagement have the highest positive impact on supervision quality, with both showing a mean coefficient of 0.048.

Disaggregating the results as before, we find that a similar subset of delivery challenges impact project supervision ratings. This is illustrated in Figure 8. The breakdown of posterior coefficients in Table 11 shows that Overambitious Objectives (-0.105), absence of effective indicators (-0.069), Organizational Capacity (-0.061), and inadequate Reporting & Supervision (-0.092) all contribute to lower supervision ratings. Once again, the indicators related to higher average supervision ratings appear to link to those areas where project leadership was able to proactively identify and resolve risks to efficient implementation. Output from the models testing supervision quality shows that effective treatment of stakeholder engagement challenges, socio-cultural idiosyncrasies, and macroeconomic risks all partially contribute to better evaluations of Bank supervision. These results remain roughly consistent when control variables are added to the model space (see Table 12): once again, TTL quality (0.080), approval-duration timespan (-0.118), and the presence of a financial management specialist (-0.064) also appear to serve as substantively significant correlates.

Though not discussed at length here, the supplementary appendix also analyzes the impact of delivery challenges on IEG’s rating of project quality at entry.<sup>24</sup> Overambitious Objectives, Reporting & Supervision, Time Allocation, absence of Indicators, and Organizational Capacity are all shown to drive down ratings on this measure. On the other hand, socio-cultural challenges identified with gender relations and macroeconomic issues related to financial instability and forex volatility both appeared to be associated with higher quality at entry.

# **5** Discussion of Results

This paper used the DeCODE taxonomy to systematically test the ways in which different delivery challenges impact project outcome ratings, Bank performance, and the quality of supervision. Bayesian model averaging was used to measure the impact of different project challenges on the

> 24Please reach out to the authors for a copy of the appendix.

19