metadata: authors: 0: author: Ralston,Laura R.
count: World
docna: 0: docna: Success in difficult environments : a
            portfolio analysis of fragile and conflict-affected states
docty: Policy Research Working Paper
subtopic: Private Sector Economics,Public Sector Management and Reform,Hydrology,Private Sector Development Law,Marketing,Inequality
teratopic: Public Sector Development,Conflict and Development
abstracts: cdata!: The World Bank Group has identified
            support to fragile and conflict-affected states as a
            strategic priority. This paper provides a systematic
            portfolio review of the International Development
            Association-funded projects in fragile and conflict-affected
            states during 2001 to 2013 and a detailed empirical analysis
            of the correlations between project and country-level
            characteristics with project outcome ratings. The portfolio
            review identifies a decline in the proportional amount of
            resources directed to fragile and conflict-affected states
            and a decline in the number of internationally recruited
            staff based in these countries. The empirical analysis finds
            no statistical difference in whether projects obtain at
            least a moderately satisfactory outcome rating between
            countries that are fragile and conflict-affected states and
            those that are not. Examination of the distribution of
            project outcome ratings indicates that projects in fragile
            and conflict-affected states obtain slightly lower ratings
            conditional on being unsatisfactory or satisfactory.
            Detailed cross-section regression analysis finds that
            indicators of project complexity, such as supervision costs,
            staff time, preparation time, and financing, are correlated
            with lower outcome ratings. Project leader characteristics
            are correlated with project outcome ratings, but to a lesser
            degree in fragile and conflict-affected states, potentially
            indicating that it is more difficult for project leaders to
            influence project outcomes in these environments. Last, a
            new approach to control for unobservable project
            characteristics, such as inherent complexity or ambition,
            shows preliminary evidence that changes in the project
            leader and increases in the supervision budget are
            correlated with improvements in project performance.
repnme: repnme: Success in difficult environments : a
            portfolio analysis of fragile and conflict-affected states
display_title: Success in difficult environments :
            a portfolio analysis of fragile and conflict-affected states
listing_relative_url: /research/2014/11/20366317/success-difficult-environments-portfolio-analysis-fragile-conflict-affected-states
topicv3: Private Sector Development m1326060 1921,Human Resources m1326304 1211,Water m1326522 137,Fragility, Conflict and Violence m1326586 2432,Poverty m1326687 695
publishtoextweb_dt: 2014-11-01T00:00:00Z
historic_topic: Public Sector Development,Conflict and Development
no_of_pages: 47
author_keys: 0: author_key: 1107821
model_extractions: WPS7098 Policy Research Working Paper 7098 Success in Difficult Environments A Portfolio Analysis of Fragile and Conflict-affected States _Laura Ralston_ Fragility, Conflict and Violence Group November 2014
Policy Research Working Paper 7098

# **Abstract**

The World Bank Group has identified support to fragile and conflict-affected states as a strategic priority. This paper provides a systematic portfolio review of the International Development Association–funded projects in fragile and conflict-affected states during 2001 to 2013 and a detailed empirical analysis of the correlations between project and country-level characteristics with project outcome ratings. The portfolio review identifies a decline in the proportional amount of resources directed to fragile and conflict-affected states and a decline in the number of internationally recruited staff based in these countries. The empirical analysis finds no statistical difference in whether projects obtain at least a moderately satisfactory outcome rating between countries that are fragile and conflict-affected states and those that are not. Examination of the distribution of project outcome ratings indicates that projects in fragile and conflict-affected states obtain slightly lower ratings conditional on being unsatisfactory or satisfactory. Detailed cross-section regression analysis finds that indicators of project complexity, such as supervision costs, staff time, preparation time, and financing, are correlated with lower outcome ratings. Project leader characteristics are correlated with project outcome ratings, but to a lesser degree in fragile and conflict-affected states, potentially indicating that it is more difficult for project leaders to influence project outcomes in these environments. Last, a new approach to control for unobservable project characteristics, such as inherent complexity or ambition, shows preliminary evidence that changes in the project leader and increases in the supervision budget are correlated with improvements in project performance.

This paper is a product of the Fragility, Conflict and Violence Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at lralston@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Success in Difficult Environments: A Portfolio Analysis of Fragile and<sup>_∗_</sup> Laura Ralston

> _∗_ **JEL Codes: O2, O19, O22**

_I thank Aart Kraay and Patricia Geli for helpful comments and Come Dechery, Kwadwo Kusi, Dana Cazacu and Sandra Cardozo for assistance with data extractions. All errors are my own. Email: lralston@worldbank.org_ 1
# **Introduction**

The World Bank Group (WBG) has identified support to Fragile and Conflict-affected States (FCS) as a strategic priority (World Bank 2011 [1]). With about 200 million people living in extreme poverty in FCS and a projection of this increasing to 230 million by 2030, while the same metrics are projected to decline 1,047 million to 289 million in non-fragile states (Burt et al 2014 [2]), understanding what contributes to successful development outcomes in FCS is of paramount importance. While there is anecdotal evidence of what works and what does not work, there is little comprehensive empirical analysis of the WBG’s operations in these countries.

This paper contributes a systematic portfolio review of the International Development Association (IDA) funded projects to FCS over the period 2001 to 2013 as well as detailed project-level empirical analysis on the correlates of outcome ratings provided by the World Bank’s Independent Evaluation Group (IEG). The portfolio review examines how the IDA portfolio has evolved since 2001, in terms of the number of projects, size of the portfolio, the flow of resources directed to FCS countries relative to other IDA borrowing countries and portfolio growth across regions and sectors. Detailed time-series analysis on project-specific trends, such as project size, project length, preparation and supervision costs and human resources is conducted, as well as an analysis of average project performance, between 2001 to 2013. The empirical project-level analysis is undertaken to examine the correlates of project success in FCS. Factors considered include project length, size, preparation and supervision costs, staff time spent on the project, number of mission days, staff experience, design of project in terms of stated project development objectives, as well as macro level factors, such as GDP growth and country policy and institutional assessment (CPIA) ratings.

The portfolio analysis indicates that proportionally the level of resources directed to FCS countries has declined in recent years relative to other IDA-only borrowing countries. The proportion of projects in FCS countries peaked in FY2010 relative to non-FCS countries and the proportion of total net commitments associated with active projects peaked in FY2009. The recent decline is due to a decrease in the number of approvals per year to FCS countries relative to non-FCS countries since FY2008. The Africa region has the largest share of the FCS projects and net commitments, followed by the South Asia region. A comparison of the portfolio breakdown by sector board by FCS IDA borrowing countries, non-FCS IDA borrowing countries and all other countries finds no systematic differences.

The results from the project-level time-series analysis shows that the size of new projects approved between 2
FY2009 to FY2013 decreased slightly in FCS countries, while increasing substantially in non-FCS countries. This provides further evidence to explain the proportional decline in IDA resources directed to FCS countries in recent years - there has not only been a decrease in the number of new projects but also in their size. Total preparation costs for FCS countries have decreased gradually since FY2006, while total supervision costs have gradually increased since FY2001 apart from in FY2013 when a drop in supervision costs can be observed. As projects in FCS countries have smaller net commitments than those in non-FCS countries, relative preparation and supervision costs tend to be higher in FCS countries. The analysis on the human resources directed to FCS countries shows that projects in FCS countries have only 25% of staff time for supervision and preparation provided by staff based in these countries, while about 40% of staff time is provided from the country office in non-FCS countries. There has also been a recent decline in the amount of GH+ staff time spent on projects in FCS countries relative to non-FCS countries and a decline in the total number of internationally recruited staff based in FCS country offices.

Regression analysis comparing IEG outcome ratings of projects in FCS and non-FCS countries shows that there is no significant difference for whether they obtain a moderately satisfactory or higher outcome rating and there is also no significant differential time-series trend for this measure of outcome success. This means that statistically projects in FCS countries are no different from projects in non-FCS countries in whether they obtained a satisfactory rating or not over the period FY2001 to FY2013. However, using the more disaggregated measure of outcome ratings, where 1 corresponds to a highly unsatisfactory outcome and 6 to a highly satisfactory outcome, there is a small significant difference between the outcomes of projects in FCS versus non-FCS countries, with projects in FCS countries obtaining 0.2 points lower on average. Examination of the distribution of outcome ratings indicates that this is due to FCS projects obtaining relatively worse ratings when they are unsatisfactory and lower satisfactory outcomes when they are satisfactory. These results are robust to non-linear model specifications.

The empirical analysis of the correlates of project outcome ratings finds that projects that are larger and require less preparation time and resources tend to obtain higher outcome ratings, while projects that involve larger supervision costs and more staff time tend to obtain lower outcome ratings. This correlation is likely driven by unobservable project characteristics such as project complexity such that more difficult projects are implemented on a smaller scale and require more inputs, like supervision and preparation time. These findings are consistent with the results reported in Denizer et al 2013 [3].

Interestingly, however, other inputs such as the representation of internationally recruited staff or the experience 3
or grade level of staff providing time to supervise and prepare projects, are not significantly correlated with project outcome ratings. This may be fortuitous given the reduction in internationally recruited staff in FCS country offices in recent years. One other input that does seem to matter is office size, but only at a threshold level. An office size of 2-5 GF+ level staff is positively correlated with outcome ratings, while an office size of more or less does not correlate with any change in outcome ratings.

Comparing the correlation between project characteristics with outcome ratings in FCS and non-FCS countries shows that project ratings are more significantly positively correlated with project size (measured by net commitment) in FCS countries, while TTL characteristics seem to be less correlated with project outcome ratings in FCS countries. The first finding suggests that in FCS countries there may be more stringency over large investments and so that project size is more strongly correlated with project success. The latter finding indicates that project outcomes are harder to control by TTLs in FCS countries. This is an important new result that extends the recent research provided by Denizer et al 2013 [3].

As discussed earlier, it is important to recognize that underlying project characteristics, such as the ambitiousness or inherent complexity of the project, may drive the correlations that are observed. For example, for ambitious projects it may be more difficult to obtain satisfactory outcome ratings and this may lead to more preparation and supervision time and finances. In an attempt to control for such project fixed effects a set of panel regressions are estimated that consider the performance of the project at the midway and end point. This enables estimation of the correlation between changes to project inputs, such as supervision costs or the TTL leading the project, with changes to project outcomes, while controlling for time-constant project characteristics. These results find that increasing supervision costs and changing the TTL are correlated with improvements in the projects outcome rating. This is a new area of research that warrants further exploration given the potential gains to development outcomes this analysis may provide.

The next section of the paper discusses the methodology adopted in the analysis. Section 2 provides an overview of the evolution of the IDA and FCS portfolio between FY2001 to FY2013. Section 3 discusses time series trends in project-level characteristics. Section 4 provides simple regression analysis to compare project outcome ratings in FCS countries to non-FCS countries. Section 5 provides detailed empirical analysis on the correlates of project outcome ratings.

4
# **1 Methodology**

This analysis uses a fixed sample of FCS countries for the period of analysis, 2001-2013. While the FCS list can change from year to year, with some countries exiting the list and others re-entering the list, it is difficult to interpret trend changes and correlations between sets of variables, when the composition of a sample varies from year-to-year. Keeping a constant sample enables time series trends to be more meaningfully interpreted as developments for a specific set of countries, rather than possible composition affects, which could not be ruled out if the sample varied from year to year. Similarly, understanding the correlation between the likelihood a project will obtain a satisfactory outcome rating and whether the project is in an FCS country is difficult to interpret when the sample changes from year to year, but has a more straight-forward interpretation when a simple, time-constant categorization of countries is adopted.

Thus, in line with the recent Independent Evaluation Group’s (IEG) study on WBG assistance to FCS (World Bank 2013 [5]), a fixed sample of FCS countries is used. Adopting a similar approach to the IEG paper, the analysis focuses on IDA-only countries and considers Bank operations in 34 fragile and conflict-affected states against that of countries that were never classified as FCS as a benchmark for measuring portfolio trends and outcomes. The 34 FCS countries include 22 always on the FCS list and 12 on the list for part of the review period (Table 1). The categorization is derived from the FCS lists for FY06 to FY13.<sup>1</sup> Partial FCS countries are those classified as FCS in at least two years during this period. Myanmar was left off the list of always FCS as it received no IDA financing during FY01-12. South Sudan is not covered in the IEG report but is included in the analysis in the sample of always FCS.

The underlying data in all sections of this paper is built up from project by fiscal year observations rather than portfolio aggregates. This means that in section 2 where an overview of the FCS and IDA portfolio is provided, the data is compiled by summing across the total number of projects for which a record exists in each fiscal year. It is important to note that the data does not come from earmarked portfolio amounts to specific countries, regions, sectors or networks. Rather it focuses on lending operations based on information recorded in the World Bank’s lending operations database in each fiscal year.

In sections 2 and 3 all operations that are IDA financed are considered. This includes development policy operations. This enables a complete mapping of IDA resources to FCS and other IDA-only countries since FY01.

> 1The FCS lists can be found on the FCV Group website: http://go.worldbank.org/BNFOS8V3S0

5
For section 4 and 5 where the project outcome ratings are analyzed controls are included in the regression analysis for development policy operations since development policy operations are significantly different in design and implementation. In addition robustness checks on the regression results have been carried out where development lending operations are dropped from the analysis. The results presented are robust to this check.

Table 1: Categorization of IDA-Only Countries

|**Always FC**|**S**|**Partial FCS**|**Ne**|**ver FCS**|
|---|---|---|---|---|
|Afghanistan<br>Angola<br>Burundi|Guinea-Bissau<br>Haiti<br>Kosovo|Cambodia<br>Cameroon<br>Djibouti|Bangladesh<br>Benin<br>Bhutan|Mauritania<br>Micronesia, Fed. States<br>Moldova|
|Central African Republic<br>Chad<br>Comoros<br>Congo, Dem. Rep.<br>Congo, Rep.<br>Cote d’Ivoire<br>Eritrea<br>Guinea|Liberia<br>Sierra Leone<br>Solomon Islands<br>Somalia<br>South Sudan<br>Sudan<br>Timor-Leste<br>Togo|Gambia, The<br>Kiribati<br>Lao PDR<br>Nepal<br>Sao Tome and Principe<br>Tajikistan<br>Tonga<br>Vanuatu|Burkina Faso<br>Ethiopia<br>Ghana<br>Guyana<br>Honduras<br>Kenya<br>Kyrgyz Republic<br>Lesotho|Mongolia<br>Mozambique<br>Nicaragua<br>Niger<br>Rwanda<br>Samoa<br>Senegal<br>Sri Lanka|
|||Yemen, Rep.|Madagascar<br>Malawi|Tanzania<br>Tuvalu|
||||Maldives<br>Mali<br>Marshall Islands|Uganda<br>Zambia|

6
# **2 Overview of the FCS Project Portfolio FY01-13**

**The number of active projects among IDA-only borrowing countries peaked in FY2011, with a total of 1192 projects, from 653 in FY2001 (see Figure 1).** Since FY2011 the number of active projects has fallen to 1119 in FY2013. Across the categorization of IDA-only borrowing countries, the number of active projects in Always FCS countries, also peaked at 348 in FY2011. However, in relative terms, the proportion of projects in Always FCS countries peaked in FY2009 at 31.7%, from 17.8% in FY2001, and was 26.5% in FY2013. The proportion of projects in Partial FCS countries has gradually increased from about 15% in FY2001 to 17% in FY2013, with absolute numbers increasing from 97 to 194.

**The total net commitments associated with active projects has consistently risen and was $48.6bn in FY2013.** While the proportion of net commitments associated with projects in FCS countries has risen from 21.8% in FY2001, it peaked in FY2009 at 34.8% of the portfolio and has since decreased to 30.3% in FY2013. As the proportion of net commitments associated with projects in Partial FCS countries has stayed approximately constant, this growth and decline has mainly come from proportion of net commitments associated with projects in Always FCS countries.

**The trends in disbursements to IDA-only borrowing countries reflect the trends in net commitments.** Disbursements have consistently risen through to FY2013, although the proportion flowing to FCS countries peaked in FY2009.

**Data on the number and size of new project approvals help explain the plateauing of the portfolio (see Figure 2).** The number of approvals per year has decreased since FY2011 and the commitments of approvals peaked in FY2008. Between FCS and non-FCS countries the proportion of approvals of projects (by number) in FCS countries has decreased since FY2008 and the size of the projects approved has proportionally declined substantially.

**The Africa region has the highest number of projects and largest level of net commitments, of all regions, in the FCS IDA portfolio (see Figure 3).** This was the case over the period FY2001 to FY2006, and has continued through to FY2013, with on average 251 projects per year and $7.2bn in net commitments. The South Asia region has the next largest portfolio and with on average 67 projects per year and $2.9bn in net commitments. The other regions have smaller FCS portfolios with between with 21 to 42 projects per year, and $350m to $1.1bn.

**The Sustainable Development Network (SDN) has by far the highest number of projects and largest level**

7
**of net commitments, of all networks, in the FCS IDA portfolio (see Figure 4).**<sup>2</sup> SDN had about 240 projects per year and over $7bn in net commitments between FY2007 to FY2013. The Human Development Network has the next largest portfolio with around 120 projects per year and $3.5bn in net commitments between FY2007 to FY2013. The Poverty Reduction and Economic Management Network (PREM) and the Financial and Private Sector Development Network (FPD) have much smaller portfolios in FCS countries.

**Figures 5 and 6 compare the regional and network decomposition in the FCS portfolio to the same decomposition in other non-FCS IDA-only borrowing countries and other non-IDA borrowing countries.** These figures demonstrate that across the IDA portfolio, Africa is the dominate region with the highest number of projects and largest level of net commitments, both among FCS and non-FCS countries. However, when non-IDA borrowing countries are considered, much larger portfolios can be observed for other regions relative to their IDA lending portfolio. With regards to the network decomposition SDN has the largest portfolio among non-FCS IDA and non-IDA borrowing countries, followed by HDN.

**Figures 7 shows a more detailed decomposition of the FCS, non-FCS and non-IDA portfolios between FY2007 and FY2013.** These figures show that across the portfolios the relative number of projects and proportion of resources going to different sector boards is reasonably similar. Discrepancies include that the size of Social Protection (SP) projects in FCS countries tend to be smaller than in non-FCS and non-IDA countries, PREM Economic Policy (EP) projects are largest in non-IDA countries, while Water (WAT), Urban Development (UD) and Transport (TR) projects are largest in FCS countries.

> 2Given that this analysis uses historical data between FY2001 to the end of FY2013 the previous network and sector board structure is used.

8
<!-- Start of picture text --> No. of IDA-only Active Projects per FY<br>3<br>~<br>o<br>o<br>=<br>3<br>oO 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br><!-- End of picture text --> <!-- Start of picture text --> ~ Net Commitment of IDA-only Active Projects per FY ($ m)<br>=F<br>8<br>oOo<br>i—JSoo<br>wv<br>o<br>oO<br>o<br>So<br>oa<br>oO<br>N<br>o<br>So<br>:2<br>=<br>oO 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br><!-- End of picture text --> <!-- Start of picture text --> o Disbursements of IDA-only Active Projects per FY ($ m<br>Ss<br>a<br>oOo<br>oao<br>=<br>8ry-<br>o<br>So<br>a<br>o~<br>o<br>o<br>a<br>°<br>=<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br><!-- End of picture text --> <!-- Start of picture text --> IDA-only Project Portfolio<br>by Number of Active Projects per FY<br>S<br>Oo<br>o<br>Ee<br>2<br>@o<br>a<br>oO<br>wT<br>8<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>HN AlwaysFCS [EM Partial FCS<br>f Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> IDA-only Project Portfolio<br>by Net Commitments of Active Projects per FY ($ m)<br>8<br>Sooo<br>2<br>Ooa<br>wToO<br>oO<br>N<br>:oOIMI<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>HN AlwaysFCS [I Partial FCS<br>f Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Total IDA-only Project Portfolio<br>by Disbursements of Active Projects per FY ($ m)<br>8<br>Q o<br>ge2Oo<br>a<br>wvoO<br>&<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>HN AlwaysFCS [EM Partial FCS<br>MM Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> No. of IDA-only Project Approvals per FY<br>Ss<br>wv<br>So<br>oO°<br>o<br>oOo<br>N<br>o-<br>oOi=]vl lt<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br><!-- End of picture text --> <!-- Start of picture text --> IDA-only Project Portfolio<br>by Number of Project Approvals per FY<br>8<br>Soo<br>58<br>Oo<br>a<br>wvo<br>oON<br>Oo<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EEE AwaysF-CS «(I Partial FCS<br>ME Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Net Commitment of IDA-only Project Approvals per FY ($ m)<br>So<br>Sooa<br>oO<br>ooS o<br>o<br>=<br>o<br>o<br>So<br>=]<br>So<br>oS<br>N<br>oa 2O ll<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br><!-- End of picture text --> <!-- Start of picture text --> IDA-only Project Portfolio<br>by Net Commitments of Project Approvals per FY ($ m)<br>8<br>Soo<br>58<br>Oo<br>a<br>owv<br>oON<br>Oo<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EEE AwaysF-CS «(I Partial FCS<br>ME Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> FCS Portfolio by Region<br>Number of Active Projects per FY<br>RAR FY01-06<br>FY07-13<br>EAP FY0 71 - 1306<br>ECA FY01-06<br>FY07-13<br>LCR FY01-06<br>FY07-13<br>MNA FY01-06<br> Fy07-13<br>SAR FY01-06<br>FY07-13<br>0 50 100 150 200 250<br><!-- End of picture text --> <!-- Start of picture text --> FCS Portfolio by Network<br>Number of Active Projects per FY<br>FPD FY01-06<br>FY07-13<br>HDN FY01-06<br>FY07-13<br>PREM FY01-06<br>FY07-13<br>SDN FY01-06<br>FY07-13<br>0 50 100 150 200 250<br><!-- End of picture text --> <!-- Start of picture text --> FCS Portfolio by Region<br>Net Commitment of Active Projects per FY ($ m)<br>FY01-06<br>RAR FY07-13<br>EAP FF Y0 71 - 1306<br>FY01-06<br>ECA FY07-13<br>FY01-06<br>LCR FY07-13<br>FY01-06<br>MNA Fy07-13<br>FY01-06<br>SAR FY07-13<br>0 2,000 4,000 6,000 8,000<br><!-- End of picture text --> <!-- Start of picture text --> FCS Portfolio by Network<br>Net Commitment of Active Projects per FY ($ m)<br>FY01-06<br>FPD FY07-13<br>HDN FY01-06<br>FY07-13<br>FY01-06<br>PREM FY07-13<br>SDN FY01-06<br>FY07-13<br>0 2,000 4,000 6,000 8,000<br><!-- End of picture text -->
<!-- Start of picture text --> WB Portfolio by Region<br>Number of Active Active Projects per FY<br>AFR FY01-06Fy07-13Fy07-13<br>EAP FY01-06Fy07-13Fy07-13<br>ECA FY01-06Fy07-13Fy07-13<br>LOR FY01-06Fy07-13Fy07-13<br>MNA FY01-06 Fy07-13 Fy07-13<br>SAR FY01-06Fy07-13Fy07-13<br>) 200 400 600 800<br>MEE FCSIDA «=NNNN non-FCSS IDA-FCSS IDA IDA<br>MEE Other<br><!-- End of picture text --> <!-- Start of picture text --> WB Portfolio by Region<br>Net Commitment Commitment of Active Active Projects per FY FY ($ m) m)<br>AFR FY01-06Fy07-13Fy07-13<br>EAP FY01-06Fy07-13Fy07-13<br>ECA FY01-06Fy07-13Fy07-13<br>LOR FY01-06Fy07-13Fy07-13<br>MNA FY01-06 Fy07-13 Fy07-13<br>SAR FY01-06Fy07-13Fy07-13<br>) 10,000 20,000 30,000 40,000<br>MEE FCSIDA «=NNNN non-FCSS IDA-FCSS IDA IDA<br>MEE Other<br><!-- End of picture text --> <!-- Start of picture text --> WB Portfolio by Region WB Portfolio by Region<br>Number of Active Active Projects per FY Net Commitment Commitment of Active Active Projects per FY FY ($ m) m)<br>AFR FY01-06Fy07-13Fy07-13 AFR FY01-06Fy07-13Fy07-13<br>EAP FY01-06Fy07-13Fy07-13 EAP FY01-06Fy07-13Fy07-13<br>ECA FY01-06Fy07-13Fy07-13 ECA FY01-06Fy07-13Fy07-13<br>LOR FY01-06Fy07-13Fy07-13 LOR FY01-06Fy07-13Fy07-13<br>MNA FY01-06 Fy07-13 Fy07-13 MNA FY01-06 Fy07-13 Fy07-13<br>SAR FY01-06Fy07-13Fy07-13 SAR FY01-06Fy07-13Fy07-13<br>) 200 400 600 800 ) 10,000 20,000 30,000<br>MEE FCSIDA «=NNNN non-FCSS IDA-FCSS IDA IDA MEE FCSIDA «=NNNN non-FCSS IDA-FCSS IDA IDA<br>MEE Other MEE Other<br>WB Portfolio by Network WB Portfolio by Network<br>Number of Active Active Projects per FY FY Net Commitment Commitment of Active Active Projects per FY FY ($ m)<br>FPD FY01-06 FY01-06<br> ev07-13 FPD ev07-13<br>HDN FY01-06 FY01-06<br> Ey07-13 HDN Ey07-13<br>PREM FY01-06 FY01-06<br>507-13 PREM 507-13<br>SDN FY01-06 FY01-06<br>5y07-13y07-13 SDN 5 y07-13<br>0 500 1,000 1,500 0 50,000 100000<br>HN «&FCS'|DA)«=- HJ nn -FCS -FCS IDA HN «&FCS'|DA)«=- HJ nn -FCS -FCS IDA<br>ME Other ME Other<br><!-- End of picture text -->

<!-- Start of picture text --> WB Portfolio by Network<br>Number of Active Active Projects per FY FY<br>FPD FY01-06<br> ev07-13<br>HDN FY01-06<br> Ey07-13<br>PREM FY01-06<br>507-13<br>SDN FY01-06<br>5y07-13y07-13<br>0 500 1,000 1,500<br>HN «&FCS'|DA)«=- HJ nn -FCS -FCS IDA<br>ME Other<br><!-- End of picture text -->

<!-- Start of picture text --> WB Portfolio by Network<br>Net Commitment Commitment of Active Active Projects per FY FY ($ m)<br>FY01-06<br>FPD ev07-13<br>FY01-06<br>HDN Ey07-13<br>FY01-06<br>PREM 507-13<br>FY01-06<br>SDN 5 y07-13<br>0 50,000 100000<br>HN «&FCS'|DA)«=- HJ nn -FCS -FCS IDA<br>ME Other<br><!-- End of picture text -->
<!-- Start of picture text --> FCS Portfolio by Sectorboard<br>Number of Active Projects 2007-2013<br>sD SDN-W, PD<br>D<br>SDI<br>SDN-<br>SDN HE<br>SDN<br>SDN HSP<br>N-EP<br>SDN- PPS FPOIGE<br><!-- End of picture text --> <!-- Start of picture text --> NonFCS Portfolio by Sectorboard<br>Number of Active Projects 2007-2013<br>SDN-WAT PD<br>SDI<br>sD -ED<br>SDN.<br>SDN.<br>HE<br>SDN<br>-SP<br>SDN. M-EP<br>+PO<br>SDN HPS<br><!-- End of picture text --> <!-- Start of picture text --> NonlIDA Portfolio by Sectorboard<br>Number of Active Projects 2007-2013<br>SDN-WAT PD<br>SDN. “ED<br>-HE<br>SDN -SP<br>M-EP<br>SDN- MPO<br>SDN.<br>M-PS<br>SDN<br>son -ARD<br><!-- End of picture text --> <!-- Start of picture text --> FCS Portfolio by Sectorboard<br>Net Commitment of Active Projects 2007-2013 ($ m)<br>SDN-WAT a<br>LED<br>SDN, LHE<br>_sp<br>NLEP<br>SDN<br>-PO/GE<br>M-PS<br>sow SD —, HARD<br><!-- End of picture text --> <!-- Start of picture text --> NonFCS Portfolio by Sectorboard<br>Net Commitment of Active Projects 2007-2013 ($ m)<br>SDN-WAT SEPD LED<br>SDN. “HE<br>_SP<br>SDN<br>SDN. M-EP<br>SON<br>sDN4 M-PO<br>SDN: aml -PS<br><!-- End of picture text --> <!-- Start of picture text --> NonlIDA Portfolio by Sectorboard<br>Net Commitment of Active Projects 2007-2013 ($ m)<br>SDN-WAT PD<br>SDI LED<br>HE<br>SDN.<br>-SP<br>SDN<br>con<br>SDN<br>N-EP<br>SDN<br>‘sD relied<br><!-- End of picture text -->
# **3 Time-series Trends of FCS and non-FCS Projects’ FY01-13**

**The size of projects, measured through net commitments and total disbursements, varies from year to year (see Figure 8).** When examining the size of projects by the fiscal year in which they closed, no obvious trends emerge for either FCS or non-FCS projects. However, using the more forward looking measure of net commitment in the year a project is approved, it appears that FCS projects are on average getting marginally smaller, while non-FCS projects are getting significantly larger.

**Projects in both FCS and non-FCS countries declined in average length from FY2001 to FY2010 from about 6 years to a little over 4 years and are now increasing in length.** In general projects in FCS countries tend to be around 6 months shorter than those in non-FCS countries.

**Total and relative preparation costs vary from year to year but both show a declining trend in FCS and non-FCS countries (see Figure 9).** FCS countries on average have lower preparation costs per project but as projects in FCS countries are generally smaller, this corresponds to larger preparation costs relative to net commitment.

**Total supervision costs in FCS and non-FCS countries are similar and increased between FY2001 to FY2012 but dropped in FY2013.** Relative supervision costs tend to be larger in FCS, given their smaller net commitments, and grew more substantially through to FY2012. However in the most recent year where data was available, FY2013, relative supervision costs had dropped substantially.

**Less staff time is spent supervising and preparing projects in FCS countries than in non-FCS countries (see Figure 10).** Almost 40% of staff time is provided by staff located in country offices for projects in non-FCS countries, while about 25% is provided by in country staff for FCS countries. The proportion of time spent on projects by GH or above level staff has experienced a small recent decline in FCS countries relative to equivalent measure in non-FCS countries.

**Mission travel to projects in FCS and non-FCS countries has stayed approximately constant since FY2005**

**(see Figure 11).** Consultant time spent on projects in FCS and non-FCS countries grew substantially between FY2001 to FY2007, but has since declined substantially. Consultant time has generally been slightly higher in FCS countries. Total time spent on projects, measured using staff time and consultant time, has declined in both countries, and has always been lower in FCS countries.

**The number of staff based in FCS countries has plateaued in recent years due to a decline in the number**

14
<!-- Start of picture text --> ooO Project Size by Net Commitment in IDA-only Portfolio<br>8<br>oO~<br>Ooo<br>Eo<br>el<br>oO<br>wv<br>8<br>Oo<br>2<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Closing FY<br>HE NeverFCS MERE Always+Partial FCS<br><!-- End of picture text --> <!-- Start of picture text --> o Project Size by Total Disbursed in IDA-only Portfolio<br>oO<br>oO<br>a<br>So<br>o<br>=<br>8<br>Eo<br>ae?<br>°owv<br>=<br>Oo<br>N<br>=<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Closing FY<br>ME NeverFCS MEN Always+Partial FCS<br><!-- End of picture text --> <!-- Start of picture text --> ooO Project Size by Net Commitment in IDA-only Portfolio<br>8<br>oO~<br>i=]o<br>°<br>wo<br>oO<br>wv<br>8<br>oO<br>2<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Approval FY<br>HE NeverFCS [EE Aways+Partial FCS<br><!-- End of picture text --> <!-- Start of picture text --> Project Length in IDA-only Portfolio<br>o<br>wo<br>vw<br>g<br>2<br>>a<br>N<br>=<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Closing FY<br>ME NeverFCS MEN Always+Partial FCS<br><!-- End of picture text -->
<!-- Start of picture text --> o Project Preparation Costs in IDA-only Portfolio<br>Ss<br>8<br>o<br>o<br>s3<br>S<br>7+<br>N<br>:ooO |<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Approval FY<br>HE NeverFCS [EN Aways+Partial FCS<br><!-- End of picture text -->

<!-- Start of picture text --> o Supervision Costs in IDA-only Portfolio - BB + TF<br>S<br>°<br>2<br>8<br>g8<br>e<br>wo<br>oO<br>2001 2002 2003 2004 2005 2006 edi 2008 2009 2010 2011 2012 2013<br>GE sNeverFCS [EN Always+Partial FCS<br>TFIEG data 2013included Report FCSwhere countryavailablecategorizationfrom BW. adopted adding South Sudan;<br><!-- End of picture text --> <!-- Start of picture text --> Relative Preparation Costs in IDA-only Portfolio<br>8<br>= oOo<br>f=<br>°<br>8<br>oy<br>ot<br>e<br>|aoo il i<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>Approval FY<br>HE NeverFCS [EN Aways+Partial FCS<br><!-- End of picture text -->

<!-- Start of picture text --> Relative Supervision Costs in IDA-only Portfolio - BB + TF<br>S<br>£:<br>f=<br>°<br>%<br>ge<br>8<br>a<br>oO<br>2001 2002 2003 2004 2005 2006 edi 2008 2009 2010 2011 2012 2013<br>GE sNeverFCS [EN Always+Partial FCS<br>TFIEG data 2013included Report FCSwhere countryavailablecategorizationfrom BW. adopted adding South Sudan;<br><!-- End of picture text -->
<!-- Start of picture text --> Staff weeks per year - IDA project<br>3<br>=<br>28<br>5<br>gA<br>Bose<br>g<br>=<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>ME NeverFCS [ME Aways+Partial FCS<br>Source: TRS Reports<br><!-- End of picture text --> <!-- Start of picture text --> GF+ Staff time - IDA project<br>8<br>— oS<br>3°<br>=<br>eoao<br>By<br>s<br>Ro<br>N<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>MES NeverFCS ME AwaystPartial FCS<br>Source: TRS Reports<br><!-- End of picture text --> <!-- Start of picture text --> In country staff time - IDA project<br>8<br>=<br>&<br>BS<br>5<br>=<br>=~o<br>s<br>©<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>ME NeverFCS [ME Aways+Partial FCS<br>Source: TRS Reports<br><!-- End of picture text --> <!-- Start of picture text --> GH+ Staff time - IDA project<br>g<br>—<br>3<br>=<br>ea<br>Bo<br>s<br>2<br>oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>MES NeverFCS ME AwaystPartial FCS<br>Source: TRS Reports<br><!-- End of picture text -->
<!-- Start of picture text --> Mission Days per year - IDA project<br>s<br>o<br>3<br>S<br>5<br>co]<br>g<br>oO<br>2005 2006 2007 2008 2009 2010 2011 2012 2013<br>ME NeverFCS [ME Aways+Partial FCS<br>Source: BI Travel RM Reports<br><!-- End of picture text --> <!-- Start of picture text --> Consultant time - IDA project<br>3<br>Bo<br>et<br>a<br>3<br>=<br>oO ll<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>MES NeverFCS ME AwaystPartial FCS<br>Source: Business Warehouse<br><!-- End of picture text --> <!-- Start of picture text --> GH+ Mission Days per year - IDA project<br>&<br>&<br>s<br>a<br>ze<br>x<br>6R<br>oO<br>2005 2006 2007 2008 2009 2010 2011 2012 2013<br>ME NeverFCS [ME Aways+Partial FCS<br>Source: BI Travel RM Reports<br><!-- End of picture text --> <!-- Start of picture text --> Total time (days) - IDA project<br>sN<br>a<br>><br>a<br>a<br>=82<br>=<br>a<br>; oO<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>MES NeverFCS ME AwaystPartial FCS<br>Sources: TRS Reports, BW, BI<br><!-- End of picture text -->
<!-- Start of picture text --> Duty Station Staff in FCS countries<br>=<br>S<br>o<br>re)<br>5 ~<br>é<br>re)<br>28<br>zs"<br>2<br>oOo<br>re)<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EN RS) 6sMR LS<br>Source: HR PeopleSoft database covering all WBG EC2+ and GF+ staff<br><!-- End of picture text --> <!-- Start of picture text --> Duty Station Staff in non-FCS countries<br>3<br>=<br>5 oO<br>3$8<br>3%<br>F<br>°<br>o<br>oOo<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EN RS) MR. LS<br>Source: HR PeopleSoft database covering all WBG EC2+ and GF+ staff<br><!-- End of picture text --> <!-- Start of picture text --> Duty Station Staff per FCS project<br>°=<br>o<br>3<br>a<br>=a<br>a<br>&<br>=<br>gt<br>=<br>“<br>o<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EN RS) 6s MR LS<br>Source: HR PeopleSoft database covering all WBG EC2+ and GF+ staff<br><!-- End of picture text --> <!-- Start of picture text --> Duty Station Staff per non-FCS project<br>g<br>»<br>37=<br>3<br>See<br>g<br>=<br>wo<br>oOo<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>EN RS) MR. LS<br>Source: HR PeopleSoft database covering all WBG EC2+ and GF+ staff<br><!-- End of picture text -->
# **4 Performance of FCS and non-FCS Projects’ FY01-13**

**Project performance in FCS countries remained on par with non-FCS countries throughout the period FY2001 to FY2013 (see Figures 13 and 14).** Regression analysis shows that there is no significant difference between projects in FCS and non-FCS countries for whether they obtain a moderately satisfactory or higher outcome rating from IEG (see Table 2). There are also no differential time series trends for this measure of outcome success. This finding is robust to changing the model specification from an OLS linear model to a probit non-linear model.

Regression analysis on the more detailed 6-point measure, where 1 corresponds to a highly unsatisfactory outcome and 6 corresponds to a highly satisfactory outcome, shows that FCS projects score approximately 0.2 points lower, with this difference mostly contributed by projects in Always FCS countries (see Table 3). This difference in performance looks to be contributed by proportionately fewer projects in FCS countries obtaining moderately unsatisfactory ratings relative to projects in non-FCS countries (see Figure 15). There are also proportionately fewer projects in FCS countries obtaining satisfactory and highly satisfactory ratings relative to projects in nonFCS countries. In other words, when FCS projects are unsatisfactory they are more likely to be more significantly unsatisfactory than in non-FCS countries, and when they are satisfactory they are rated less highly compared to projects in non-FCS countries.

This finding is also robust to changing the model specification to a non linear multinomial logit model, which finds that projects in FCS countries are relatively less likely to obtain ratings of 3, 4 and 5 compared to projects in non-FCS countries.<sup>3</sup>

> 3These results are available on request from the author.

20
<!-- Start of picture text --> IEG Outcome Ratings<br>a. By number of projects<br>se<br>2oo ae eet awn,<br>o oe me arm ee tea<br>vo<br>Eo<br>=<br>So<br>OF<br>Pan<br>g<br>£8<br>2<br>:<br>°<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>FY of project completion<br>—_— Never FCS ——— Always+Partial FCS<br>—---- Bankwide<br><!-- End of picture text --> <!-- Start of picture text --> IEG Outcome Ratings<br>a. By number of projects<br>wo<br>ao<br>= — etwe eee<br>geELoio |e> ~SotsSS<br>°<br>w<br>e°<br>5<br>22<br>N<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>FY of project completion<br>—_— Never FCS ——— Always+Partial FCS<br>—---- Bankwide<br><!-- End of picture text -->

<!-- Start of picture text --> IEG Outcome Ratings<br>a. By number of projects w/ 95% Confidence Interval<br>ss<br>Fi ® 4] -2+=-=7a —~R-e SSSBNTT ca seJl rtseoae<br>fia<br>Eo aon see Nea ~ a<br>S Soon Lea x Ne eee<br>2 ~ a SS es po weerrT7 ~<br>6s ae eae wn<br>><br>2°<br>8a<br>5<br>=<br>be<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>FY of project completion<br>__ __<br>—_— Never FCS ——— Always+Partial FCS<br>----- —aaee<br><!-- End of picture text --> <!-- Start of picture text --> IEG Outcome Ratings<br>a. By number of projects w/ 95% Confidence Interval<br>wo<br>ey Uoese<br>~ | ctire<t>-*i, WOT Sees ae<br>&SeBioos |eet ~via ae- . wrCS Posner a ogaillo ---<br>©=ow etal “7 a ~ NU- “ oN xf“ ’<br>§<br>ge<br>N<br>N<br>2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013<br>FY of project completion<br>__ __<br>—_— Never FCS ——— Always+Partial FCS<br>----- —aaee<br><!-- End of picture text -->
<!-- Start of picture text --> IEG Outcome Ratings by FCS status 2001-12<br>4 2 3 4 By 6<br>IEG Outcome Rating<br><!-- End of picture text -->
Table 3: OLS Dependent Variable Satisfactory Rating (1-6)

|I(FCS=1)|-0.20|-0.22|-0.34||
|---|---|---|---|---|
||[0.06]***|[0.06]***|[0.23]||
|t||-0.01|||
|||[0.01]|||
|I(FCS=1)*t||0.02|||
|||[0.02]|||
|I(AlwaysFCS=1)||||-0.24|
|||||[0.07]***|
|I(PartialFCS=1)||||-0.11|
|||||[0.09]|
|Constant|3.91|3.91|3.98|3.93|
||[0.04]***|[0.04]***|[0.14]***|[0.11]***|
|Year FE||||X|
|Year FE x I(FCS=1)|||X||
|_R_<sup>2</sup>|0.01|0.01|0.03|0.02|
|_N_|1,292|1,282|1,292|1,292|

23
# **5 Correlates of Project Success in FCS and Non-FCS Projects**

**This section of the analysis focuses on understanding more about correlates of project outcome ratings.** It is important to be careful in interpreting these results to distinguish between correlation and causation. Out of a set of observable project characteristics, such as project length, size, costs, staff time, mission days, etc., it is possible to compare the mean level of these characteristics in projects that obtain satisfactory outcome ratings to those that do not, and thus ascertain the correlation between success, as measured through outcome ratings, and project characteristics. This, however, does not mean that these characteristics determine project success. It could be that other unmeasurable factors influence both project success and project characteristics, such as the complexity or difficulty of a project, and this leads to the correlation that we observe.

**Figure 16 shows the breakdown of outcome ratings for projects in FCS and non-FCS countries, as well as the breakdown of satisfactory outcome ratings by sector board.** As discussed in the previous section there are proportionately fewer projects in FCS countries obtaining moderately unsatisfactory ratings (3) and proportionately more obtaining unsatisfactory ratings (2), relative to projects in non-FCS countries. Across sector boards there are not many systematic differences apart from projects under the FPD and Water (WAT) sector boards obtain slightly weaker outcomes in FCS countries, while Health (HE) projects do relatively less well in non-FCS countries.

**Figure 17 considers the country-level covariates, such as the country’s policy and institutional assement (CPIA), GDP per capita growth and the size of the World Bank country office.** CPIA scores are lower in FCS countries as is expected given the criteria for the FCS list but within FCS and non-FCS countries CPIA is not correlated with project success. However, these samples of countries contain very little variation in CPIA scores, meaning that it is difficult to expect variation in the CPIA scores to have significant correlation with project outcome ratings. Where GDP per capita growth data have been recorded, it appears that FCS countries have stronger growth trends than non-FCS countries and in both groups of countries growth is correlated with stronger outcome ratings. As discussed in the previous section, non-FCS countries have larger country offices than FCS countries. However, country office size does not appear to be linearly correlated with project success in FCS countries and is negatively correlated with project success in non-FCS countries.

Figure 18 considers the project-level covariates related to the design and preparation of projects such as size, length, preparation time and costs. **Project size is positively correlated with project success, while project length, preparation time and preparation costs are negatively correlated with project success in both FCS** 24
**and non-FCS countries.** This may indicate that more resources are directed to less risky projects, that may be quicker to implement and less complex in their design, requiring less preparation time and financing. Other explanations could also be consistent with these results.

**Figure 19 shows that staff time and supervision costs are negatively correlated with project outcome ratings,** indicating that projects that obtain unsatisfactory outcome ratings on average involve a greater amount of staff time and greater supervision costs. This is true for projects in both FCS and non-FCS countries. Similarly mission days and consultant days are negatively correlated with outcome ratings.

**Table 4 demonstrates that projects in FCS countries are no more or less correlated with satisfactory outcome ratings than projects in non-FCS countries.** This remains true once controls for CPIA, GDP growth, sector boards and network boards are included in the regressions. Interestingly, as was suggested in figure 17, once a control is included for whether a project occurs in an FCS country or not, a country’s CPIA is no longer significantly correlated with project success.<sup>4</sup> Including sector board controls increases the coefficient of determination (statistical fit) of the regression as measured through R-squared. Examination of the correlation between outcome ratings and network boards indicates that projects under the FPD, HDN and PREM network boards perform relatively less well compared to projects under the SDN network board. This is true in non-FCS and FCS countries. Table 5 replicates the regression results shown in table 4 using the 6-point IEG outcome rating as the dependent variable. With limited controls, projects in FCS countries appear to be correlated with slightly lower outcome ratings. However, this result is not robust to the inclusion of sector or network boards as controls.

More extensive regression analysis finds that **development policy operations (DPOs) and projects with a greater focus in one sector are negatively correlated with satisfactory outcomes (see table 6). Project size** , measured through net commitments, **is positively correlated with satisfactory outcome ratings, and this correlation is stronger in FCS countries. Project complexity** , in so far as it can be measured through **supervision costs, staff weeks working on the project, and preparation time** is **negatively correlated with outcome ratings.** Interestingly, this correlation is not intensified for projects occurring in FCS countries, as one might have anticipated if complex projects are more difficult to implement in these environments. There appears to be a threshold for country office size, whereby having **an office with 2-5 staff members is positively correlated with project outcome ratings,** while having an office size of only one or greater than five does not correlate with any change in

> 4Separate analysis, not shown here, found that CPIA ratings are positively correlated with a project’s outcome rating when no controls for an FCS grouping are included.

25
project outcome ratings. In this sample, controls for **the representation of internationally recruited staff** or **the experience or grade level of staff composition in country offices are not correlated with project outcome ratings. Other correlates of project success include early warning indicators,** such as problem project flags in the first half of a project, that are raised when a project obtains unsatisfactory outcome ratings during implementation status reports. A potential problem flag that is automatically raised when a project obtains at least 3 unsatisfactory outcomes across a range project implementation measures<sup>5</sup> during the first half of the project is also negatively correlated with final outcome ratings for projects in FCS countries.

**The characteristics of task team leaders (TTLs) are correlated with project success, although this is more so in non-FCS countries than FCS countries.** Included in the analysis is a measure of a TTL’s propensity to implement satisfactory projects, referred to as TTL quality in the regressions. This is constructed as the average outcome rating a TTL obtains on projects in which s/he leads excluding the project being analyzed as the dependent variable.<sup>6</sup> Among non-FCS countries this measure is positively correlated with project outcome ratings. However, among FCS countries this correlation disappears, indicating that in the environment of FCS countries the characteristics of TTLs may have a less dominant role, possibly since other external factors have greater influence on project outcomes. In both FCS and non-FCS countries, the experience of the TTL, measured through the number of years a TTL has previously led projects, is negatively correlated with outcome ratings. While this result seems counter-intuitive, it has several possible explanations. One: less experienced TTLs may have more recently acquired technical knowledge that translates into improved project outcomes. Two: less experienced TTLs have stronger reputational incentives to implement a satisfactory project and may increase their effort. Three: there may be directed matching by managers to put more experienced TTLs on more difficult projects, which would lead to a spurious negative correlation between project success and TTL experience.

The robustness of the regression results presented in table 6 has been checked by (i) performing separate regression analysis for FCS and non-FCS countries, and (ii) omitting DPOs from the regression analysis. The first robustness check is verify that no systematic collinearity between explanatory variables and explanatory variables interacted with the FCS country indicator is leading to spurious regression results. The second robustness check is

> 5These implementation measures include effectiveness delays, counterpart funding, financial management, safeguards, monitoring and evaluation, legal coverage, procurement, project management, country portfolio record, long term risk and slow disbursements. The measure for country environment has been deliberately excluded from the evaluation criteria for the potential problem flag, since all FCS countries trigger this measure automatically.

> 6This variable was constructed using the methodology introduced in Geli, Kraay and Nobakht (2014) [4]: http://go.worldbank.org/5FH9W1X620.

26
to verify that the inclusion of DPOs in the sample of projects is not leading to spurious correlations. The regression results presented are robust to both these checks.

**Table 7 further examines the contribution of variables that control for TTL fixed effects for explaining the variation in project outcome ratings.** As a benchmark, column (1) demonstrates that 8% of the variation in project outcomes can be explained by using a set of country indicator controls. This means that country-specific factors or fixed effects, such as quality of institutions, economic growth trends, size of WB country office, only explains 8% of the variation in project outcomes. Including the measure of TTL quality defined above as the propensity of TTLs to implement satisfactory projects only leads to an additional 1% increase in the coefficient of determination (see column (2)). Introducing a full set of TTL indicators that control for TTL-specific factors or fixed effects explains about 46% of the variation in project outcomes (see column (3)) and this is true for projects in both FCS countries (see column (4)) and non-FCS countries (see column (5)). However, these regressions are close to saturation with many regressors compared to the number of data points, so it is hardly surprising that they achieve high statistical fit. Columns (6) and (7) estimate the correlation of TTL quality with project outcome ratings in FCS and non-FCS countries while controlling for country fixed effects. This shows that TTL quality remains positively correlated with project success, even when country environment is taken into account, although the correlation is still weaker in FCS countries.

As discussed earlier, it is important to distinguish between causation and correlation in interpreting these results and recognize that underlying project characteristics, such as the ambitiousness or inherent complexity of the project, may drive the correlations that are observed. For example, for ambitious projects it may be more difficult to obtain satisfactory outcome ratings and this may lead to more preparation and supervision time and finances. In an attempt to control for such project fixed effects a set of panel regressions are estimated in tables 10 to 12. For each project two data points are collected as a measure of satisfactory progress or outcome. In the first half of a project, satisfactory progress is defined as the absence of any problem project flags, unsatisfactory implementation or development objective rating reports<sup>7</sup> , and potential problem flags. In the second half of a project, the IEG outcome rating is used to evaluate whether the project reached a satisfactory outcome. The fixed effects regressions can be interpreted as estimating the correlation between changes to project inputs, such as supervision costs or the TTL leading the project, with changes to project outcomes, while controlling for time-constant project characteristics.

Tables 8 and 9 show summary statistics for projects that obtain unsatisfactory and satisfactory outcomes, re-

> 7For example that would be detected through the implementation status reports (ISRs).

27
spectively, in their first half. These results are consistent with the earlier results and show that time-constant characteristics such as preparation costs and time are negatively correlated with outcome ratings.

**Table 10 shows that controlling for project fixed effects TTL quality and CPIA are positively correlated with outcome ratings, while GDP growth is negatively correlated with outcome ratings.** This means that increases in a country’s CPIA and changing the lead on the project to a TTL with a greater propensity for success are correlated with improvements in a projects outcome ratings between the half way and completion points. On the other hand increases in GDP growth are correlated with decreases in a projects outcome ratings between the half way and completion points.

**Table 11 enables project varying characteristics to have a differential impact between FCS and non-FCS countries.** This shows that changes to the TTL leads to improvements in a projects outcome ratings in both sets of countries but the results are weaker in FCS countries.

Table 12 shows the coefficient estimates conditional on projects with unsatisfactory outcomes at the half way point. **While the results are weaker, there is some evidence to suggest that increasing supervision costs, increasing consultant time and increasing the size of a country office can lead to improvements in these projects final outcome ratings.** 28
<!-- Start of picture text --> Number of Projects by IEG Rating & FCS status 2001-12<br>So<br>8wm<br>sN<br>re<br>58<br>Bo<br>28Eas<br>ire)red<br>:<br>woN Po i |<br>°<br>1 2 3 4 5 6<br>IEG Outcome Rating<br>HEE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->

<!-- Start of picture text --> FCS IEG Outcome Ratings by Sector Boards<br>£aD<br>ised<br>o& 2<br>6<br>ot<br>2<br>3°<br>®<br>a Oo<br>€ € €& & € © @ S$ FF SY SK<br>Sector Boards<br>ARD - Agriculture & Rural Dev., ED - Education, EMT - Energy & Mining<br>ENVPS - Environment, FPD - Financial & Private Sector Dev., HE - Health, Nutrition & Pop.,<br>UD-- PublicUrban SectorDev., WAT Gov.,- SDV Water - Soc. Dev., SP - Soc. Protection TR - Transport,<br><!-- End of picture text --> <!-- Start of picture text --> IEG Outcome Ratings by Sector Boards<br>Z8ED<br>EMT &&<br>ENV os<br>* <2<br>FPD 8%<br>GIcHE oe<br>PS ¢$ e<br>:TRsP z&&¢<br>up oe<br>wat S%<br>&<br>4% Stlstactory Outcorne Rating<br><!-- End of picture text --> <!-- Start of picture text --> Non-FCS IEG Outcome Ratings by Sector Boards<br>£oT<br>is<br>@oE<br>robe<br>re<br>2<br>ot<br>On<br>cS<br>oO<br>€ € €& & € © @ S$ FF SY SK<br>Sector Boards<br>ARDENV-- Environment, Agriculture & FPDRural - FinancialDev., ED - Education,& Private SectorEMT Dev.,- Energy HE & - Health,Mining Nutrition & Pop.,<br>PSUD -- PublicUrban SectorDev., WAT Gov.,- SDV Water - Soc. Dev., SP - Soc. Protection TR - Transport,<br><!-- End of picture text -->
<!-- Start of picture text --> CPIA by IEG Rating and FCS status<br>wo<br>wv<br>Lo<br>®<br>=N<br>-<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> In Country Staff by IEG Rating and FCS status<br>2oO<br>50<br>On<br>£<br>ici)<br>o<br>roy<br>°<br>zo?<br>c<br>5<br>=a<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> GDP per capita growth by IEG Rating and FCS status<br>wo<br>§2ao wv<br>foa<br>D<br>ao<br>a<br>oO<br>c<br>§~<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS + [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> In Country GF+ Staff by IEG Rating and FCS status<br>a<br>é<br>8=<br>£<br>b=<br>as<br>ri<br>oO<br>5 oO<br>°=<br>F<br>ov<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> Net Commitment by IEG Rating and FCS status<br>8<br>ge<br>£<br>E<br>Eo<br>orE:<br>=a<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Preparation Time by IEG Rating & FCS status<br>+<br>¢<br>—<br>oe<br>Ex<br>e<br>a2<br>§ =<br>oO<br>=°<br>S<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Project Length by IEG Rating and FCS status<br>cy<br>2<br>5<br>go<br>2<br>Ss<br>2we<br>&<br>N<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Preparation Costs by IEG Rating & FCS status<br>Oo3<br>2<br>we<br>=.<br>83<br>a*<br>a)<br>ag<br>§ So<br>28<br>Oo<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> Staff Weeks by IEG Rating and FCS status<br>8<br>E<br>7?<br>oO=<br>2<br>83<br>a<br>Be<br>=<br>Ze°<br>= -<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Mission Days by IEG Rating and FCS status<br>3<br>=°<br>>Bo<br>ao<br>=<br>a<br>§<br>3S<br>°o<br>N<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Supervision Costs by IEG Rating & FCS status<br>8<br>8<br>=<br>°<br>S<br>ea<br>&<br>ae<br>c<br>oa<br>oeOo<br>wv<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Consultant Days by IEG Rating and FCS status<br>8<br>s<br>=<br>=88<br>2<br>&,<br>eo<br>5<br>soO<br>7<br>=<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> PDO word count by IEG Rating and FCS status<br>a<br>fe) 8<br>o~<br>sO<br>aw<br>2<br>23<br>5<br>39<br>-<br>=o<br>oO<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> PDO Intermediate Indicators by IEG Rating and FCS status<br>2<br>[2<br>£<br>Se<br>&Be2<br>£<br>2,<br>%<br>:<br>oN<br>BSFyoO<br>Unsat Sat<br>IEG Outcome Rating<br>HEE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> PDO Indicators by IEG Rating and FCS status<br>©<br>ow<br>£oO<br>3 v<br>£<br>fe)<br>fo<br>=<br>Zn<br>:<br>=_<br>oO<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Slow Disbursement Flags by IEG Rating & FCS status<br>a<br>oo<br>Fy<br>a<br>2 2<br>En<br>3 o<br>Bw<br>Oxo<br>&<br>Be<br>° N<br>Z_<br>oO<br>===oO k=<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->
<!-- Start of picture text --> ISR TTL Onsite by IEG Rating & FCS status<br>22<br>fz<br>fo)Exa<br>aw<br>o<br>%<br>san<br>o _<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> TTL Quality by IEG Rating & FCS status<br>-<br>3<br>ZooO<br>a<br>3<br>8 +<br>2<br>oO<br>x<br>o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> TTL Project Experience by IEG Rating & FCS status<br>= o<br>yo<br>E<br>Fa£ nNiN<br>3<br>en<br>e<br>o<br>~o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [I Never FCS<br><!-- End of picture text --> <!-- Start of picture text --> Problem Project Flags by IEG Rating & FCS status<br>=<br>a”<br>so<br>a wo<br>5-<br>3s +<br>Boo<br>o<br>so<br>-<br>= o<br>Unsat Sat<br>IEG Outcome Rating<br>HE Always+Partial FCS [EE Never FCS<br><!-- End of picture text -->
Table 4: OLS Dependent Variable Satisfactory Index (0/1)

|Dummy for FCS Country|-0.04|-0.00|-0.01|-0.01|-0.00|0.28|
|---|---|---|---|---|---|---|
||[0.03]|[0.04]|[0.04]|[0.08]|[0.04]|[0.47]|
|CPIA Rating||0.05|0.06|0.06|0.04|0.04|
|||[0.06]|[0.06]|[0.06]|[0.06]|[0.06]|
|Log(GDP per capita growth)||0.03|0.02|0.02|0.02|0.02|
|||[0.02]|[0.02]|[0.02]|[0.02]|[0.02]|
|FPD|||||-0.15|-0.11|
||||||[0.05]***|[0.06]*|
|HDN|||||-0.10|-0.14|
||||||[0.03]***|[0.04]***|
|PREM|||||-0.08|-0.10|
||||||[0.04]**|[0.05]**|
|FPD*FCS||||||-0.53|
|||||||[0.48]|
|HDN*FCS||||||-0.20|
|||||||[0.47]|
|PREM*FCS||||||-0.25|
|||||||[0.47]|
|SDN*FCS||||||-0.32|
|||||||[0.47]|
|Constant|0.69|0.50|0.47|0.45|0.57|0.57|
||[0.02]***|[0.19]**|[0.20]**|[0.20]**|[0.19]***|[0.19]***|
|_R_<sup>2</sup>|0.00|0.00|0.04|0.06|0.02|0.02|
|_N_|1,291|1,130|1,127|1,127|1,130|1,130|
|Sector Dummies|||Y||||
|Sector*FCS Dummies||||Y|||
|Network Dummies|||||Y||
|Network*FCS Dummies||||||Y|

35
Table 5: OLS Dependent Variable Satisfactory Rating (1-6)

|Dummy for FCS Country|-0.20|-0.15|-0.17|-0.27|-0.15|0.98|
|---|---|---|---|---|---|---|
||[0.06]***|[0.10]|[0.10]*|[0.18]|[0.10]|[1.07]|
|CPIA Rating||0.06|0.08|0.11|0.05|0.06|
|||[0.13]|[0.13]|[0.13]|[0.13]|[0.13]|
|Log(GDP per capita growth)||0.05|0.05|0.04|0.05|0.05|
|||[0.04]|[0.04]|[0.04]|[0.04]|[0.04]|
|FPD|||||-0.48|-0.37|
||||||[0.12]***|[0.14]***|
|HDN|||||-0.20|-0.30|
||||||[0.08]**|[0.10]***|
|PREM|||||-0.23|-0.31|
||||||[0.08]***|[0.10]***|
|FPD*FCS||||||-1.73|
|||||||[1.10]|
|HDN*FCS||||||-0.94|
|||||||[1.08]|
|PREM*FCS||||||-1.02|
|||||||[1.08]|
|SDN*FCS||||||-1.24|
|||||||[1.07]|
|Constant|3.91|3.65|3.56|3.52|3.81|3.81|
||[0.04]***|[0.44]***|[0.44]***|[0.45]***|[0.44]***|[0.44]***|
|_R_<sup>2</sup>|0.01|0.01|0.07|0.09|0.03|0.03|
|_N_|1,291|1,130|1,127|1,127|1,130|1,130|
|Sector Dummies|||Y||||
|Sector*FCS Dummies||||Y|||
|Network Dummies|||||Y||
|Network*FCS Dummies||||||Y|

36
Table 6: OLS Dependent Variable Satisfactory Index (0/1)

|Dummy for FCS Country|-0.08<br>[011]|-0.32<br>[025]|-0.31<br>[026]|-0.16<br>[028]|-0.24<br>[028]|-0.18<br>[032]|-0.05<br>[034]|
|---|---|---|---|---|---|---|---|
|CPIA Rating|.<br>0.06<br>[006]|.<br>0.03<br>[006]|.<br>0.04<br>[006]|.<br>0.09<br>[007]|.<br>0.04<br>[007]|.<br>0.05<br>[007]|.<br>0.06<br>[008]|
|Log(GDP per capita growth)|.<br>0.02|.<br>0.01|.<br>0.02|.<br>0.03|.<br>0.01|.<br>0.01|.<br>0.00|
||[0.02]|[0.02]|[0.02]|[0.02]|[0.02]|[0.02]|[0.02]|
|Dummy for DPO|-0.09<br>|-0.19<br>**|-0.20<br>**|-0.19<br>**|-0.14<br>*|-0.19<br>**|-0.21<br>**|
|Share of project in largest sector|[0.07]<br>-0.07|[0.08]<br>-0.13|[0.08]<br>-0.13|[0.08]<br>-0.12|[0.08]<br>-0.12|[0.09]<br>-0.13|[0.10]<br>-0.11|
|<br>|[0.07]<br>|[0.07]*<br>|[0.07]*<br>|[0.07]<br>|[0.07]*<br>|[0.07]*<br>|[0.08]<br>|
|log(Project Size - $ m)|-0.00|0.02|0.04|0.05|0.04|0.05|0.05|
||[002]|[002]|[002]*|[002]**|[002]*|[002]**|[002]**|
|log(Project Size - $ m)*FCS|.<br>0.06|.<br>0.04|.<br>0.03|.<br>0.04|.<br>0.04|.<br>0.06|.<br>0.07|
||[0.03]**|[0.03]|[0.03]|[0.03]|[0.03]|[0.03]*|[0.04]*|
|Project length (yrs)|-0.03<br>|-0.02<br>|-0.02<br>|-0.02<br>|-0.01<br>|-0.01<br>|-0.00<br>|
||[0.01]***|[0.01]|[0.01]*|[0.01]**|[0.01]|[0.01]|[0.01]|
|(Project length (yrs))*FCS|-0.02|-0.02|-0.02|-0.01|-0.01|-0.01|-0.02|
|<br>|[0.01]*|[0.01]<br>|[0.01]<br>|[0.02]<br>|[0.02]<br>|[0.02]<br>|[0.02]<br>|
|log(Supervision Costs per Yr - $’000s)||-0.09<br>[003]***|-0.05<br>[003]*|-0.05<br>[003]*|-0.05<br>[003]*|-0.04<br>[003]|-0.06<br>[003]*|
|log(Supervision Costs per Yr- $’000s)*FCS||.<br>0.01|.<br>-0.00|.<br>-0.01|.<br>0.00|.<br>-0.01|.<br>-0.00|
|||[0.04]|[0.04]|[0.05]|[0.05]|[0.05]|[0.06]|
|log(Preparation Costs - $’000s)||0.01<br>|0.03<br>|0.01<br>|0.02<br>|0.01<br>|0.01<br>|
|log(Preparation Costs - $’000s)*FCS||[0.02]<br>0.06<br>|[0.02]<br>0.05<br>|[0.03]<br>0.05<br>|[0.03]<br>0.03<br>|[0.03]<br>0.04<br>|[0.03]<br>0.04<br>|
|||[0.04]<br>|[0.05]<br>|[0.05]<br>|[0.05]<br>|[0.05]<br>|[0.05]<br>|
|log(Preparation time - mths)||-0.06<br>003**|-0.06<br>003**|-0.05<br>003*|-0.05<br>003*|-0.06<br>003**|-0.05<br>003*|
|log(Preparation time - mths)*FCS||[.]<br>-0.05<br>[005]|[.]<br>-0.04<br>[005]|[.]<br>-0.06<br>[005]|[.]<br>-0.01<br>[005]|[.]<br>-0.01<br>[005]|[.]<br>-0.04<br>[005]|
|lTl Sff Wk Y||.|.<br>011|.<br>010|.<br>010|.<br>011|.<br>008|
|og(ota ta ees per r)|||-.<br>|-.<br>|-.<br>|-.<br>|-.<br>|
|log(Total Staff Weeks per Yr)*FCS|||[0.04]***<br>0.04<br>|[0.05]**<br>0.01<br>|[0.04]**<br>0.03<br>|[0.05]**<br>0.03<br>|[0.05]<br>-0.00<br>|
|Country Offce Size = 1 Staff|||[0.07]|[0.08]<br>-0.15|[0.08]<br>-0.16|[0.09]<br>-0.17|[0.10]<br>-0.21|
|<br>||||[0.14]<br>|[0.13]<br>|[0.14]<br>|[0.16]<br>|
|Country Offce Size = 2-5 Staff||||0.16<br>[008]**|0.14<br>[008]*|0.15<br>[008]*|0.14<br>[009]|
|Country Offce Size = 6-10 Staff||||.<br>0.03<br>|.<br>0.03<br>|.<br>0.05<br>|.<br>0.00<br>|
|||||[0.08]|[0.08]|[0.09]|[0.10]|
|Country Offce Size = 10+ Staff||||-0.00<br>|0.01<br>|0.04<br>|0.01<br>|
|IRS Ratio of Country Offce||||[0.08]<br>-0.07<br>[011]|[0.08]<br>-0.05<br>[011]|[0.09]<br>-0.04<br>[012]|[0.10]<br>-0.01<br>[013]|
|(IRS Ratio of Country Offce)*FCS||||.<br>-0.10<br>[0.16]|.<br>-0.07<br>[0.16]|.<br>0.00<br>[0.17]|.<br>0.07<br>[0.19]|
|D f Pbl Pt Fl i H1|||||013|014|013|
|ummy or roem rojec ag n|||||-.<br>[0.05]***|-.<br>[0.05]***|-.<br>[0.06]**|
|(Dummy for Problem Project Flag in H1)*FCS|||||0.01<br>|0.01<br>|0.01<br>|
|Dummy for Potential Problem Flag in H1|||||[0.09]<br>-0.04<br>[005]|[0.09]<br>-0.02<br>[005]|[0.10]<br>-0.03<br>[006]|
|(Dummy for Potential Problem Flag in H1)*FCS|||||.<br>-0.21|.<br>-0.24|.<br>-0.23|
|TTL Ei|||||[0.10]**|[0.10]**<br>001|[0.11]**<br>001|
|xperence (yrs)||||||-.<br>[0.00]*|-.<br>[0.00]*|
|(TTL Experience (yrs))*FCS||||||0.00<br>|0.00<br>|
|||||||[0.01]<br>|[0.01]<br>|
|TTL Quality||||||0.16<br>[007]**|0.22<br>[007]***|
|(TTL Quality)*FCS||||||.<br>-0.23|.<br>-0.29|
|Dummy for TTL onsite||||||[0.14]*<br>-0.09<br>|[0.15]*<br>-0.10<br>|
|(Dummy for TTL onsite)*FCS||||||[0.07]<br>0.16<br>013|[0.08]<br>0.16<br>014|
|Constant|0.70|1.21|1.25|1.03|1.14|[.]<br>1.09|[.]<br>0.87|
||[0.22]***<br>|[0.27]***<br>|[0.27]***<br>|[0.29]***<br>|[0.29]***<br>|[0.31]***<br>|[0.34]**<br>|
|_R_<sup>2</sup>|0.07|0.10|0.10|0.12|0.16|0.19|0.19|
|_N_<br>Sector Dummies|1,126<br>Y|1,008<br>Y<br>37|1,006<br>Y|921<br>Y|921<br>Y|862<br>Y|752<br>Y|
|PDO Controls|||||||Y|
Table 7: OLS Dependent Variable Satisfactory Index (0/1)

|TTL Quality||0.22||||0.21|0.15|
|---|---|---|---|---|---|---|---|
|||[0.05]***||||[0.06]***|[0.07]**|
|(TTL Quality)*FCS||||||0.03|-0.08|
|||||||[0.10]|[0.13]|
|_R_<sup>2</sup>|0.08|0.09|0.46|0.49|0.53|0.09|0.21|
|_N_|1,291|1,186|770|214|439|1,186|918|
|Country Dummies|Y|Y||||Y|Y|
|TTL Dummies|||Y|Y|Y|||
|FCS countries only||||Y||||
|Non-FCS countries only|||||Y|||
|Full Controls|||||||Y|

38
Table 8: Summary Statistics for Unsatisfactory Projects in First Half

|**Variable**|**Mean**|**(Std. Dev.)**|**N**|
|---|---|---|---|
|Dummy for FCS Country|0.395|(0.49)|392|
|CPIA Rating|3.173|(0.426)|382|
|GDP per capita growth|1.476|(3.912)|372|
|Dummy for DPO|0.028|(0.165)|392|
|Share of project in largest sector|0.668|(0.236)|391|
|Project Size - $ m|36.341|(46.046)|392|
|Project length (yrs)|7.141|(1.997)|392|
|Dummy FPD project|0.099|(0.3)|392|
|Dummy HDN project|0.321|(0.468)|392|
|Dummy PREM project|0.089|(0.286)|392|
|Dummy SDN project|0.487|(0.5)|392|
|Supervision Costs per Yr - $’000s|116.39|(104.19)|281|
|Preparation Costs - $’000s|576.542|(457.602)|366|
|Preparation time - mths|23.445|(17.119)|392|
|Total Staff Weeks per Yr|30.288|(17.378)|238|
|No. of Mission Days per Yr|60.884|(42.949)|130|
|No. of Consultant Days per Yr|32.867|(65.907)|238|
|Staff + Consultant Days per Yr|184.305|(124.335)|238|
|Size of Country Offce|9.367|(9.377)|264|
|IRS Ratio of Country Offce|0.469|(0.312)|258|
|No. of Risk Flags per Yr|1.806|(0.991)|392|
|Approval-Effectiveness time (mths)|7.322|(5.647)|392|
|TTL Experience (yrs)|6.278|(5.567)|392|
|TTL Quality|0.622|(0.295)|372|
|<br>Dummy for TTL onsite|0.092|(0.225)|392|
|No. of words in PDO|55.636|(58.207)|349|
|No. of PDO Indicators|1.407|(2.504)|349|
|No. of Intermediate Indicators|4.493|(7.733)|349|

39
Table 9: Summary Statistics for Satisfactory Projects in First Half

|**Variable**|**Mean**|**(Std. Dev.)**|**N**|
|---|---|---|---|
|Dummy for FCS Country|0.362|(0.481)|724|
|CPIA Rating|3.292|(0.392)|684|
|GDP per capita growth|2.292|(3.643)|679|
|Dummy for DPO|0.109|(0.312)|724|
|Share of project in largest sector|0.644|(0.242)|722|
|Project Size - $ m|39.87|(53.924)|724|
|Project length (yrs)|5.572|(2.237)|724|
|Dummy FPD project|0.083|(0.276)|724|
|Dummy HDN project|0.298|(0.458)|724|
|Dummy PREM project|0.173|(0.378)|724|
|Dummy SDN project|0.443|(0.497)|724|
|Supervision Costs per Yr - $’000s|104.424|(115.974)|561|
|Preparation Costs - $’000s|469.115|(407.175)|658|
|Preparation time - mths|16.657|(14.869)|724|
|Total Staff Weeks per Yr|30.227|(31.248)|539|
|No. of Mission Days per Yr|64.295|(62.011)|305|
|No. of Consultant Days per Yr|41.972|(75.571)|539|
|Staff + Consultant Days per Yr|193.106|(184.887)|539|
|Size of Country Offce|8.949|(9.444)|559|
|IRS Ratio of Country Offce|0.437|(0.3)|541|
|No. of Risk Flags per Yr|0.662|(0.523)|724|
|Approval-Effectiveness time (mths)|5.270|(3.364)|723|
|TTL Experience (yrs)|6.528|(6.399)|724|
|TTL Quality|0.682|(0.296)|634|
|Dummy for TTL onsite|0.102|(0.263)|724|
|No. of words in PDO|57.658|(49.232)|623|
|No. of PDO Indicators|1.681|(3.111)|623|
|No. of Intermediate Indicators|4.658|(8.137)|623|

40
Table 10: Fixed Effects OLS Dependent Variable Satisfactory Index (0/1)

|log(GDP per capita growth)|-0.04|-0.04|-0.04|-0.04|-0.05|
|---|---|---|---|---|---|
||[0.02]*|[0.02]|[0.03]*|[0.03]*|[0.03]*|
|CPIA Rating|0.23|0.20|0.20|0.19|0.16|
||[0.09]**|[0.09]**|[0.14]|[0.15]|[0.16]|
|TTL Quality||0.17|0.17|0.17|0.17|
|||[0.08]**|[0.09]*|[0.09]*|[0.10]*|
|log(Supervision Costs per Yr - $’000s)|||-0.05|-0.05|-0.04|
||||[0.04]|[0.04]|[0.04]|
|log(Total Staff Weeks per Yr)|||0.03|0.04|0.04|
||||[0.05]|[0.05]|[0.05]|
|log(No. of Consultant Days per Yr)||||0.00|0.00|
|||||[0.01]|[0.01]|
|log(Size of Country Offce)|||||-0.00|
||||||[0.05]|
|IRS Ratio of Country Offce|||||-0.04|
||||||[0.14]|
|Constant|-0.05|-0.10|0.05|0.08|0.13|
||[0.30]|[0.31]|[0.49]|[0.51]|[0.52]|
|_R_<sup>2</sup>|0.01|0.02|0.02|0.02|0.02|
|_N_|1,864|1,709|1,454|1,454|1,332|

41
Table 11: Fixed Effects OLS Dependent Variable Satisfactory Index (0/1)

|log(GDP per capita growth)|-0.03|-0.04|-0.04|-0.04|-0.03|
|---|---|---|---|---|---|
||[0.03]|[0.03]|[0.03]|[0.03]|[0.03]|
|log(GDP per capita growth)*FCS|-0.02|0.02|-0.01|-0.01|-0.07|
||[0.05]|[0.05]|[0.06]|[0.06]|[0.06]|
|CPIA Rating|0.27|0.21|0.24|0.22|0.27|
||[0.11]**|[0.11]*|[0.20]|[0.21]|[0.22]|
|(CPIA Rating)*FCS|-0.11|-0.05|-0.07|-0.06|-0.09|
||[0.19]|[0.20]|[0.30]|[0.31]|[0.34]|
|TTL Quality||0.25|0.24|0.24|0.20|
|||[0.09]***|[0.12]**|[0.12]**|[0.12]*|
|(TTL Quality)*FCS||-0.26|-0.20|-0.20|-0.06|
|||[0.17]|[0.20]|[0.20]|[0.21]|
|log(Supervision Costs per Yr - $’000s)|||-0.06|-0.06|-0.05|
||||[0.05]|[0.05]|[0.05]|
|log(Supervision Costs per Yr- $’000s)*FCS|||0.02|0.02|0.03|
||||[0.10]|[0.10]|[0.11]|
|log(Total Staff Weeks per Yr)|||0.06|0.06|0.04|
||||[0.06]|[0.06]|[0.06]|
|log(Total Staff Weeks per Yr)*FCS|||-0.05|-0.05|-0.03|
||||[0.09]|[0.10]|[0.10]|
|log(No. of Consultant Days per Yr)||||0.01|0.01|
|||||[0.01]|[0.02]|
|log(No. of Consultant Days per Yr)*FCS||||-0.00|-0.00|
|||||[0.03]|[0.03]|
|log(Size of Country Offce)|||||-0.04|
||||||[0.06]|
|log(Size of Country Offce)*FCS|||||0.12|
||||||[0.12]|
|IRS Ratio of Country Offce|||||-0.29|
||||||[0.19]|
|(IRS Ratio of Country Offce)*FCS|||||0.58|
||||||[0.30]*|
|Constant|-0.08|-0.08|-0.06|-0.01|-0.10|
||[0.30]|[0.31]|[0.55]|[0.56]|[0.59]|
|_R_<sup>2</sup>|0.01|0.02|0.02|0.02|0.03|
|_N_|1,864|1,709|1,454|1,454|1,332|

42
Table 12: Fixed Effects OLS Dep. Var. Sat. Index (0/1) - Unsatisfactory Starters

|log(GDP per capita growth)|-0.05|-0.05|-0.02|-0.01|-0.02|
|---|---|---|---|---|---|
||[0.06]|[0.06]|[0.08]|[0.08]|[0.07]|
|log(GDP per capita growth)*FCS|0.05|0.07|-0.02|-0.02|-0.02|
||[0.10]|[0.10]|[0.11]|[0.11]|[0.10]|
|CPIA Rating|1.12|0.99|1.40|1.07|0.51|
||[0.19]***|[0.20]***|[0.45]***|[0.45]**|[0.43]|
|(CPIA Rating)*FCS|-0.67|-0.46|-0.77|-0.56|-0.40|
||[0.33]**|[0.34]|[0.60]|[0.60]|[0.57]|
|TTL Quality||0.50|0.16|0.15|0.13|
|||[0.19]***|[0.25]|[0.24]|[0.21]|
|(TTL Quality)*FCS||-0.61|-0.21|-0.19|-0.11|
|||[0.29]**|[0.36]|[0.36]|[0.32]|
|log(Supervision Costs per Yr - $’000s)|||0.41|0.30|0.24|
||||[0.15]***|[0.15]**|[0.13]*|
|log(Supervision Costs per Yr- $’000s)*FCS|||-0.15|-0.06|-0.29|
||||[0.23]|[0.23]|[0.21]|
|log(Total Staff Weeks per Yr)|||-0.15|-0.12|-0.10|
||||[0.19]|[0.19]|[0.16]|
|log(Total Staff Weeks per Yr)*FCS|||-0.06|-0.07|0.20|
||||[0.27]|[0.26]|[0.23]|
|log(No. of Consultant Days per Yr)||||0.08|0.01|
|||||[0.03]**|[0.03]|
|log(No. of Consultant Days per Yr)*FCS||||-0.01|-0.00|
|||||[0.05]|[0.05]|
|log(Size of Country Offce)|||||0.48|
||||||[0.11]***|
|log(Size of Country Offce)*FCS|||||0.25|
||||||[0.19]|
|IRS Ratio of Country Offce|||||-0.18|
||||||[0.36]|
|(IRS Ratio of Country Offce)*FCS|||||0.71|
||||||[0.51]|
|Constant|-2.68|-2.63|-4.53|-3.62|-2.84|
||[0.50]***|[0.51]***|[1.13]***|[1.15]***|[1.06]***|
|_R_<sup>2</sup>|0.11|0.14|0.16|0.21|0.41|
|_N_|655|621|512|512|469|

43
# **6 Conclusions**

This paper contributes a systematic portfolio review of the International Development Association (IDA) funded projects to FCS over the period 2001 to 2013 and a detailed empirical analysis of the correlations between project and country-level characteristics with project outcome ratings. The portfolio review identifies a decline in proportional amount of IDA resources directed to FCS countries as well as a decline in the number of internationally recruited staff based in these countries.

The empirical analysis finds there is no statistical difference in whether projects obtain at least a moderately satisfactory outcome ratings between FCS and non-FCS countries. This finding is robust to nonlinear model specification checks. In addition some project characteristics that are often thought to be important, such as the experience level of project leaders or the time spent on projects by internationally recruited staff or higher grade level staff, are not significantly correlated with outcome ratings for projects in FCS and non-FCS countries. Factors that are correlated with outcome ratings include project supervision and preparation time and project early warning indicators, such as implementation flags raised during the first half of the project by project leaders. These are all negatively correlated with outcome ratings most likely indicating that these are complicated and difficult projects to implement. This latter finding is consistent with the results presented in Denizer et al [3], which used a larger sample of projects and included IBRD funded projects as well as IDA funded projects. Some factors within managerial control that are positively correlated with project outcomes are office size - it appears important to have a country office of at least 2-5 GF+ level staff, and the project leader appointed to the project - TTLs that have a greater propensity to lead successful projects continue to obtain better ratings on average. This is true even when controlling for country fixed effects and in FCS countries, although to a lesser degree. This finding further demonstrates that project outcome ratings can be difficult to influence in FCS environments. Last, a new approach to control for unobservable project characteristics, such as inherent complexity or ambitious, shows preliminary evidence that changes to the project leader and increases to supervision budget are correlated with improvements in project performance in both FCS and non-FCS countries.

44