The first component of the Bank performance measure focuses on project design, including the “extent to which the Bank identified, facilitated preparation of, and appraised the operation such that it was most likely to achieve planned development outcomes and was consistent with the Bank’s fiduciary role.”<sup>22</sup> The second component evaluates the quality of supervision, measuring whether and to what extent adequate steps were taken to proactively identify and resolve threats to efficient implementation.<sup>23</sup> As such, positive coefficients on contextual challenges may represent areas in which adequate preparation and supervision were conducted; in other words, those challenges appear to be areas in which project leaders took pains to account for imminent obstacles to project delivery and the achievement of development objectives. This may also explain why the identification of Socio-Cultural and Macroeconomic factors was associated with higher Bank performance ratings, once again reflecting the project quality and monitoring dimensions of the instrument.

Similar to the tests on outcomes, delivery challenges are further disaggregated at the subcategory level in order to better understand which subcomponents are most closely related to variation in Bank performance. These are shown in Figure 6. Coefficients from the model averaging run on performance using subcategory-level delivery challenges are summarized in Table 8. Both components of the project design category show a negative impact on outcome, with overambitious objectives (-0.095) weighing somewhat more than time allocation issues (-0.068). Likewise, challenges relating to the availability of indicators (-0.074) and reporting (-0.075) negatively impacted Bank performance, a feature that clearly relates to the supervision component of the measure. Turning to the H.R. category, a curious phenomenon can be seen. While that challenge has a relatively low PIP and substantive coefficient at the category level, disaggregation of the effect at the subcategory level shows that this is caused by the presence of two challenges with opposing effects, which appear to negate one another at the category level. Thanks to the more granular data, we see that while skill transfer challenges have a moderate positive impact, the subcategory assessing organizational capacity actually has a relatively pronounced negative impact on Bank performance ratings (-0.078). Finally, disaggregation of challenges shows that the macroeconomic challenges most relevant for understanding Bank performance relate to financial instability (0.061) and foreign-exchange volatility (0.048). These and the rest of the context-cluster challenges have a positive impact on performance ratings, with two exceptions: challenges related to civil unrest and post-conflict climates continue to reduce performance ratings, but their substantive impact appears to be relatively moderate.

The results remain roughly consistent when external controls are included in model averaging. This output is shown in Table 9. The six delivery challenges highlighted above retain their impact, in addition to several structural variables that now show relatively high posterior coefficients. As before, approval to closing time remains an informative determinant of overall performance, showing a strong negative coefficient (-0.118). Added focus on performance (as opposed to the achievement of project indicators) accounts for why this variable has a stronger impact on performance relative to outcome. While TTL quality yields a net-positive impact on performance (0.087), the presence of a financial management specialist decreases performance ratings overall (-0.084), as was the case with outcome ratings. The presence of a specialist is not itself indicative of effectiveness; rather, it

> 22International Bank for Reconstruction and Development, and Independent Evaluation Group International Bank for Reconstruction and Development. Results and performance of the World Bank Group. World Bank, 2010.

> 23In cases where a large difference exists between the ratings assigned to the two dimensions, evaluators are asked to use the project’s outcome rating as a means of appraising performance overall, thereby speaking to the issues of endogeneity raised earlier. However, this should not be of concern for the analysis attempted here, particularly since model averaging was also conducted on the supervision subcomponent of the measure.

18