# Recent studies of risk prediction identified significant predictors of project outcomes – but they still only explain a small percentage of the downgrades

Two recent studies attempted to identify factors contributing to the success and failure of investment projects, and to build prediction models for the outcome ratings of those projects. In the first, Denizer, Kaufmann, and Kraay (2011) analyzed correlations between project outcomes and project characteristics, such as size and duration, as well as country variables and project variables corresponding to the 12 original warning indicators used in the implementation monitoring systems. In the second study, Geli, Kraay, and Nobakht (2014) built on the findings of the previous study and designed an outcome prediction model using a combination of country and project variables, and comparing it with the prediction value of the IP and DO ratings as the project matures.

Denizer, Kaufmann, and Kraay used data on more than 6,000 World Bank-financed projects undertaken in 130 countries since the 1970s to develop their model. Their major findings were:

- Eighty percent of the variation in project outcomes occurs _within_ countries rather than _between_ countries, and is therefore accounted for by _project_ variables rather than _country_ variables.

- Country Policy and Institutional Assessment (CPIA) scores account for 40 percent of the betweencountry variations and therefore for 8 percent of the variation in outcome. It is among the factors most strongly correlated with project outcome.

- Project variables used in the regression analysis, comprising both characteristics (such as size, duration, and preparation costs) and early flags, overall explain only 6 percent of the 80 percent variation, or 5 percent in the project outcome.

- The quality of the task team leader is strongly correlated to the project outcome, with a contribution about equal to that of the CPIA.

- Among the most significant results of individual partial correlation of outcome with project variables were project size, duration, and preparation and supervision costs, which were negatively correlated with outcome; early IP/DO flags, which were negatively and very significantly correlated with the ultimate outcome; and the early monitoring and evaluation (M&E) flag and sometimes the project management flag, which were also significantly and negatively correlated with outcome.

- No evidence was found that disbursement delays are significantly correlated with outcomes.

Geli, Kraay, and Nobakht, for their prediction model, used all investment operations between 1995 and 2005,<sup>4</sup> and included correlates for size, preparation time, elapsed time between approval and effectiveness, initially planned project length, task team leader track record, and CPIA score. Their major findings were:

- The main correlates with outcome were task team leader track record and CPIA score; preparation time and effectiveness delay were negatively correlated but not significantly so; project size not significantly so.

- Application of the full model with all the variables correctly predicted about 40 percent of the outcomes, but the same result was obtained with only the CPIA and the task team leader track record variables.

4 After defining the model using the sample of 1,561 projects, with closing dates between 1995 and 2005, the model was applied to a second “out of sample” set of 1,168 projects closing from 2005 to 2012. Results between the two sets were consistent.

5