such as the CPIA ratings. The paper addresses such concerns of reverse causality by controlling for context factors at baseline, that is, in the year of project approval.<sup>43</sup> The major external validity threat is _sample selection bias._ One potential purpose of this review is to provide Bank teams with risk predictors for _potential_ PSM reform projects. Risk predictors matter at the project design stage, when Bank teams face the questions: Should a PSM project be designed in a given country?—and how? Relative to the set of potential projects that Bank teams consider, those that are completed and evaluated is much smaller. Sample selection bias would occur if the “actual” project portfolio were nonrepresentative of “potential” projects in ways that affect project performance.<sup>44</sup> In this case, sample selection bias seems very likely because projects that have been approved and implemented likely had better chances of success than projects considered but not chosen. The broadest consequence of such selection bias is that project success rates are biased upwards.<sup>45</sup> They draw an overly optimistic picture of the odds of success and failure when considered at the design stage.

# **6 DESCRIPTIVE STATISTICS: COMPARING PSM AND NON-PSM PROJECTS**

This section compares the performance, country targeting, and process and management characteristics (part of research question 2) of PSM and non-PSM projects.<sup>46</sup>

## **6.1 Comparing Project Performance**

PSM projects on average perform worse than non-PSM projects in achieving at least “moderately satisfactory” (MS+) IEG outcome ratings. As figure 5 illustrates, the share of PSM projects (broadly defined) with MS+ IEG outcome ratings (at 69.4 percent) is 5.2 percentage points lower than for nonPSM projects (74.6 percent). This difference is statistically significant at the 1 percent significance level (see annex table A.1). For “upstream” PSM projects (narrowly defined), the success rate is very similar to broadly defined PSM projects, at 68.54 percent.<sup>47</sup>

> 43 Bias due to reverse causality could still occur if a government eager to obtain a World Bank investment loan for PSM reforms undertakes PSM reform efforts prior to receiving the loan to demonstrate reform commitment. If these prior reforms led to improvements in the CPIA scores at baseline, these improvements would (in part) be caused by the prospect of the Bank’s investment project. But this variant of reverse causality due to expectations is unlikely to play a major role.

> 44 In other words, the conditional independence assumption—that is, that “PSM reforms” are selected in the sample independent of their potential outcome (and are thus exogenous)—is unlikely to hold.

> 45 Bias would in particular occur if the selection were correlated with other project covariates, such as country context. If, for example, Bank teams filtered out more potential projects with poor potential outcomes in nondemocratic environments than in democratic environments, the regression estimates would underestimate the negative effect of nondemocratic environments on project success rates.

> 46  Not all statistics underlying this section are reported in the data annex. In particular, a discussion of time trends in the IEG ratings of PSM and non-PSM projects between 1990 and 2013 are omitted, which could be driven by factors such as changes in the worldwide political context or the Bank’s lending strategy. For details on this and other descriptive statistics, please refer to Blum (2014).

> 47  Interestingly, when considering the IEG institutional impact ratings, no equivalent performance difference is observable. With an average of 48.05 percent of PSM projects (broadly defined) performing at least “moderately satisfactorily” and 46.74 percent of non-PSM projects, the performance difference is negligible and not statistically significant. This also holds for “upstream” PSM projects (see Blum 2014).

25