# **4.2 Posterior Probabilities: Project Outcomes**

Posterior Inclusion Probabilities (PIPs) from the Bayesian model averaging run on delivery challenge categories are presented in Figure 3. Model averaging was conducted using a random model prior, which (per Ley and Steel, 2012) implements a random-theta prior and binomial-beta hyperprior on the prior inclusion probability. The random prior represents the default model prior choice in the BMS package. The model employs the default setting for the hyperparameter on Zellner’s _g_ prior, using the Uniform Information Prior (UIP), which sets _g_ = _N_ . The horizontal axis proportionally graphs the Cumulative Model Probabilities (CMPs) for the most likely model specifications: for instance, the top model specification has a posterior model probability of 0.346. Colloquially put, this translates into a 35% likelihood that the combination of variables represents the “true” model specification, given the data. The vertical axis indexes the variables included in this model averaging run (in this case, the 14 delivery challenges), with blue bars representing positive coefficients and the red bars negative. The substantive effects for the delivery challenge categories tested in this model averaging run are summarized in Table 4.

Note that nine of the delivery challenges show posterior inclusion probabilities (PIPs) higher than 0.95, meaning that over 95% of the model mass rests on specifications that include those covariates. In addition, most of the tested variables take a binary value on the conditional positive column, indicating that their effect is either uniformly positive (1) or negative (0). Stability across conditional posterior direction and PIP is suggestive of model convergence and a good test that a covariate is a stable estimator on outcome. Given the size of the model space (2<sup>14</sup> or 16,384 possible combinations), all specifications were enumerated, meaning that the posterior inclusion probabilities in this run are not estimates on a sample of the model space but rather an expression of the total share of possible models in which those variables were included. Examining posterior means, we can see that the delivery challenges’ PIPs positively correspond to their substantive significance, with Data & Monitoring (-0.07) and Commitment & Leadership challenges (-0.06) showing the strongest impact on outcome ratings.<sup>16</sup> Cumulative model probabilities for the first two specifications account for over half of the posterior model likelihood. These include the first nine delivery challenges with either Legislation & Regulation or (in the second specification) Business Environment challenges.

The results highlight an intriguing attribute of the delivery challenges visualized in Figure 3: of the covariates that show high posterior probabilities of inclusion in the top specifications, five of them (Project Finance, Coordination & Engagement, Environment & Geography, Disasters & Emergency Response, and Business Environment) show a positive impact on outcome ratings. Challenges pertaining to the project’s environment and emergency response have the strongest substantive impact among those named, and are both in the context cluster of delivery challenges. Why might this be the case? Understanding the positive impact of some challenges involves going back to the way the taxonomy was coded in the first place. Delivery challenges are situations and phenomena that test the capacity of the implementation team (and the Bank more generally) to successfully bring a project to fruition. TTLs identify both challenges that hindered the implementation of a loan as well as those that arose but were ultimately overcome. The positive results are therefore an intuitive reflection of the way in which Implementation Completion Reports (ICRs) are compiled, showing diversity in the magnitude and valence of the various obstacles that were faced (and potentially overcome) by the implementation team. In this case, context-based references to emer-

> 16Results show standardized coefficients, with explanatory and outcome variables normalized to mean zero and standard deviation one in order to present all data on the same order of magnitude.

14