# **4.  Estimating the Prediction Model**

We begin by estimating the prediction model using all IPF operations evaluated between 1995 and 2005.<sup>2</sup> This will allow us to assess the out-of-sample predictive power of the model by how well it predicts actual project outcomes between 2006 and 2012. The basic prediction model includes the following correlates of project outcomes: project size (as measured by the logarithm of the original commitment in dollars), preparation time (months from PCN review to project approval date), elapsed time between approval and effectiveness (measured in months), initially planned project length (months from approval date to initial estimated closing date), TTL track record (average IEG rating on other projects managed by the same TTL)<sup>3</sup> , and country policy performance (average country-level CPIA score during life of project). The sample consists of 1561 IPF operations for which these variables are all available.

The main significant correlates of project outcomes are country policy performance, and TTL track record, which both enter positively and highly significantly. This is consistent with the findings of [6] which documents a strong relationship between project outcomes and both of these variables in a larger set of World Bank projects. In addition, projects that are initially planned to take a long time are significantly less likely to be rated satisfactory. Preparation time and time elapsed between approval and effectiveness are negatively correlated with project outcomes, but not significantly so. Unlike in some of our earlier work, project size is not significantly correlated with outcomes. In the prediction 2 We exclude World Bank projects providing general budget support rather than specific project financing from our analysis as the lifecycle of these projects is very different from the bulk of the World Bank’s IPF operations. 3 Calculating this measure of TTL track record is complicated by the fact that projects often have multiple TTLs over their lifespan. We work with project-level data reflecting the identity of the TTL as of the end of each fiscal year for the life of each project in our sample. We then construct the TTL record variable as a weighted average of the IEG ratings of all of the projects the TTL managed, with weights proportional to the number of fiscal years for which the TTL was responsible for each project. For example, if a TTL managed one project for 4 years and it was rated satisfactory by IEG, and another project for just one year of its life and this project was rated unsatisfactory by IEG, then the TTL record variable assigns a weight of 80 percent to the satisfactory outcome and 20 percent to the unsatisfactory outcome. To avoid a mechanical correlation between TTL track record and project outcomes, we drop the project in question from this calculation in the historical portfolio data. This means that the TTL record variable is available only for projects managed by TTLs who show up as TTLs for at least two projects in the dataset. Projects managed by “first-time” TTLs are therefore dropped from the analysis. In the prediction exercise in the active portfolio, we base the TTL record variable on all available projects in the historical data, even if there is just one past evaluated project for a given TTL. Finally it is important to note that the TTL track record variable is based only on the identity of the TTL during the implementation period of the project, and not from the project identification and preparation stages. This is because we draw TTL data from the ISR sequences for projects, and these begin only during the implementation phase.

7