metadata: authors: 0: author: Marcelo Gordillo,Darwin
1: author: House,Ruth Schuyler
count: World
docna: 0: docna: Effects of multilateral support on
            infrastructure PPP contract cancellation
docty: Policy Research Working Paper
subtopic: City to City Alliances,Private Sector Economics,Private Participation in Infrastructure,National Urban Development Policies & Strategies,Regional Urban Development,Urban Economics,Hydrology,Urban Economic Development,Transport Services,Urban Communities
teratopic: Informatics,Infrastructure Economics and Finance
abstracts: cdata!: This paper examines the relationship
            between multilateral support and contract cancellation in
            long-term infrastructure public-private partnerships. The
            analysis draws on a large data set and employs a multi-level
            econometric model to define propensity scores and matching
            estimators to compare rates of cancellation between projects
            with multilateral support and a comparison group of
            public-private partnership projects without multilateral
            support. The results suggest that multilateral support has a
            positive effect on the survival of long-term public-private
            partnership infrastructure contracts. Whereas observed the
            data suggest that multilateral support has no effect on
            cancellation rates, a quasi-experimental approach shows that
            the cancellation rate for projects with multilateral (6
            percent) would have been about 48 percent higher without
            support from multilateral development organizations.
repnme: repnme: Effects of multilateral support on
            infrastructure PPP contract cancellation
display_title: Effects of multilateral support on
            infrastructure PPP contract cancellation
listing_relative_url: /research/2016/07/26581810/effects-multilateral-support-infrastructure-ppp-contract-cancellation
topicv3: Private Participation in Infrastructure
            m1326086 414,Public-Private Partnerships m1326334 2206,Water m1326522 137,Transport m1326661 135,Urban Development m1327371 699
publishtoextweb_dt: 2016-07-19T00:00:00Z
historic_topic: Informatics,Infrastructure Economics and Finance
no_of_pages: 18
author_keys: 0: author_key: 937656
1: author_key: 1365991
model_extractions: WPS7751 Policy Research Working Paper 7751 Effects of Multilateral Support on Infrastructure PPP Contract Cancellation _Darwin Marcelo Schuyler House_ Public-Private Partnerships Cross-Cutting Solutions Area & Singapore Infrastructure and Urban Development Hub July 2016
Policy Research Working Paper 7751

# **Abstract**

This paper examines the relationship between multilateral support and contract cancellation in long-term infrastructure public-private partnerships. The analysis draws on a large data set and employs a multi-level econometric model to define propensity scores and matching estimators to compare rates of cancellation between projects with multilateral support and a comparison group of publicprivate partnership projects without multilateral support.

The results suggest that multilateral support has a positive effect on the survival of long-term public-private partnership infrastructure contracts. Whereas observed the data suggest that multilateral support has no effect on cancellation rates, a quasi-experimental approach shows that the cancellation rate for projects with multilateral (6 percent) would have been about 48 percent higher without support from multilateral development organizations.

This paper is a product of the Public-Private Partnerships Cross-Cutting Solutions Area and the Singapore Infrastructure and Urban Development Hub. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at dmarcelo@worldbank.org, and shouse@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**_Effects of Multilateral Support on Infrastructure PPP Contract Cancellation_** Darwin Marcelo and Schuyler House<sup>1</sup> June 30, 2016

_Keywords:_ Public-Private Partnership, contract cancellation, multilateral support, multilevel model, propensity score matching.

JEL classification codes: H54, H76, R42, C21, C52, C55

> 1. The authors may be contacted at <u>dmarcelo@worldbank.org</u> and shouse@worldbank.org.
# **_Introduction_**

A growing body of quantitative research on public-private partnership (PPP) performance, particularly for long-term infrastructure contracts, attends to factors that influence the level of investments in PPP and PPP performance over time. With respect to performance, one line of enquiry questions how project-level and contextual factors affect the likelihood that a PPP will survive to its intended contract term. While PPP ‘success’ involves multiple aspects of performance (e.g., service outcomes, attainment of sector goals, profitability, etc.), contract survival is undoubtedly important due to the high costs of early termination and renegotiation. Moreover, cancellation is a clear sign that significant and insurmountable problems between parties to the contract could not be overcome via intra-contractual adjustments, renegotiation, or at extremis, arbitration.

Because the transaction costs of contract cancellation and renegotiation are high (Guasch, Laffont, & Straub, 2003; Bitran, Nieto-Parra, & Robledo, 2013), the question of how PPPs can be sustained is important. In addition to legal, regulatory, and contractual factors (i.e., the realm of institutions), the role of agency in PPP success is also of interest. Key participants include contract-granting government units, private operators and sponsors, regulators, and financiers. Each of these parties plays a different role in the PPP and brings a different set of interests, resources, and strategies to the management, oversight, financing, and overall execution of the PPP. In the context of developing regions, another important set of stakeholders includes the multilateral development institutions that are engaged in supporting PPPs through sector reform, technical assistance and financial support.

The survival of a contract to its intended term implicitly recognizes that all parties to the contract are sufficiently satisfied with the PPP outcomes, such that they wish to remain in the contractual relationship enshrined within the PPP arrangement. In this study, the analysis aims to offer clarity to whether projects with multilateral support (MLS) have greater likelihood of succeeding than those without.<sup>2</sup> Some qualitative PPP studies argue that multilaterals have supported PPPs as a global solution with too little regard for local context, thus promoting its uptake in inappropriate circumstances and leading to costly contract cancellations (e.g., Bayer, 2009). On the other hand, other studies have suggested that multilaterals can play important convening and knowledge-sharing roles that increase a PPP’s likelihood of improved performance and contract sustenance (Jandhyala, 2015; House, 2014).

> 2. Multilateral support refers to financial support from international financial institutions including Asian Development Bank (ADB), African Development Bank (AfDb), Central American Bank for Economic Integration (BCIE), West African Development Bank (BOAD), Black Sea Trade and Development Bank (BSTDB), Development Bank of Latin America (CAF), Caribbean Development Bank (Caribank), East African Development Bank (EADB), European Bank for Reconstruction and Development (EBRD), European Investment Bank (EIB), Inter-American Development Bank (IADB), Inter American Investment Corporation (IAIC), International Bank of Reconstruction and Development (IBRD), International Development Association (IDA), International Finance Corporation (IFC), Multilateral Investment Guarantee Agency (MIGA), and North American Development Bank (NADB).

2
This working paper presents the results of a quantitative analysis focused on the influence of direct multilateral financial support on PPP contract cancellation. The World Bank’s Private Participation in Infrastructure (PPI) Database<sup>3</sup> defines canceled projects as those “from which the private sector has exited in one of the following ways: selling or transferring its economic interest back to the government before fulfilling the contract terms; removing all management and personnel from the concern; or ceasing operation, service provision, or construction for 15 percent or more of the license or concession period, following the revocation of the license or repudiation of the contract.” Multi-level probabilistic models and matching estimators across a large data set of PPPs are applied to examine the relationship between MLS and rates of contract cancellation. Results show that PPP projects that benefit from MLS have lower cancellation rates.

# **_Multilateral Support for Infrastructure PPPs_**

Following the definition adopted by the PPI Database, a project is considered to have multilateral support when it receives financial support including lending, equity contributions, or issuances of financial guarantee products. Upstream policy support, project preparation assistance, and other kinds of technical assistance not linked to a financial commitment on the part of the multilateral do not meet this definition of MLS, though these kinds of support are likely to play a role in the closure and successful application of infrastructure PPP.

The propensity of a PPP to receive MLS is hypothesized to be contingent on a number of project characteristics, market conditions, and strategies of multilaterals, reflecting both “donor interest” and “recipient need” models of aid flows (Basìlio, 2014). The population and level of development within a country may affect MLS lending generally, and for infrastructure PPPs specifically. Other analyses of multilateral aid suggest that multilateral financial flows are biased towards less populous countries with lower per capita GDPs (Basìlio, 2014; Neumayer, 2003).

Sector and region are also included to account for the organization of multilaterals’ lending programs, which are often segmented by sector and/or region, and whose management units may have different interests and ideas about private participation in infrastructure. Depending on experiences, practices, and beliefs adopted in the organizations and their sub-units, PPP participation may be pursued more or less actively. Moreover, since some development banks are organized to serve particular regions only (e.g., ADB, IADB, AFIDB, etc.), differences between the organizations with respect to general support of PPP may affect the likelihood of a project receiving MLS in each region. Lastly, contract type and size of investment is also proposed to affect the likelihood of MLS, since multilateral development banks

> 3. The Private Participation in Infrastructure (PPI) Project Database has data on infrastructure projects being tracked from 1990 in 139 low- and middle-income countries. The database is the leading source of PPI trends in the developing world, covering projects in the energy, telecommunications, transport, and water and sewerage sectors. Projects include management or lease contracts, concessions, Greenfield projects, and divestitures. The data are presented in a variety of ways useful to researchers, policy makers, journalists and others.

3
(MDBs) may systematically prefer certain contractual forms or projects with larger required investments.

In this paper, the probability of receiving MLS at the project level is estimated to serve as a metric –a _means_ by which to compare PPP projects with and without MLS. To this end, project-level and aggregated country- and regional-level factors are taken into consideration to identify most-similar sets of PPPs with and without MLS for later comparison with respect to cancellation rates.

# **_Data_**

The statistics and probabilistic models in this study follow the definition of PPP adopted by the PPI Database. According this source, infrastructure projects with private participation (PPI) that have reached financial closure excluding divestitures, telecom projects, or merchant<sup>4</sup> projects are considered PPPs. Using this definition, of the full PPI data set of 7,042 projects, 5,054 are PPP projects. Of these, 637 involve MLS and 4,417 do not (Figure 1). This PPP project sample is used to estimate the likelihood of receiving multilateral support.

Figure 1. Project data set for econometric models: PPPs from 1990-2015 <!-- Start of picture text --> PPIs<br>(7,042 projects)<br>PPPs<br>5,054 projects (71%)<br>With MLS Without MLS<br>637 PPPs (12.6%) 4,417 PPPs (87.4%)<br><!-- End of picture text --> Source: PPI Database, Feb 2016 The project outcome analyzed in this paper –cancellation rate– is based only on PPP projects in that are currently classified as operational, concluded, or canceled; and have reached financial closure between 1990 and 2010 (i.e., PPPs that have survived five or more years). Of the 5,054 PPPs in the PPI Database, 3,552 projects have survived more than 5 years, of which 2,777 are operational or concluded, whereas 173 are canceled (Table 1).

The main reason not to consider relatively young PPPs (less than six years of potential operability) is simple: a project’s outcome (e.g., conclusion, cancellation, survival) can only be reasonably measured when projects have been operational for enough time to allow their outcomes to become observable.<sup>5</sup> For example, PPP

> 4. Merchant projects are those wherein “a private sponsor builds a new facility in a liberalized market in which the government provides no revenue guarantees. The private developer assumes construction, operating, and market risk for the project.”

> 5. According to the PPI Database, canceled projects have an average duration (between the financial closure year and its cancellation) of 5.89 years. Moreover, only one of the PPP projects that reached financial closure after 2012 has been canceled.

4
cancellation rates could be underestimated if one-year operational projects were included, as these projects are likely too young to reveal inherent problems that could lead to distress or cancellation.<sup>6</sup> Comparing the percentage of projects with multilateral support that are canceled (6.1%) to the rate of cancellation of non-MLS projects (5.8%), the rates are not statistically different (see Table 1 and Figure 2).

Table 1. Summary statistics of project data set, PPPs from 1990-2010

|PPP Status|Multilate<br>With MLS|ral Support<br>Without MLS|Total|
|---|---|---|---|
|Canceled PPPs|24|149|173|
|Total operational or concluded|368|2,409|2,777|
|Total(canceled,operational,or concluded)|392|2,558|2,950|
|Cancellation rate|6.1%|5.8%||

Source: PPI Database, Feb 2016 Figure 2. Relative cancellation rates of MLS and non-MLS projects, 1990-2010 <!-- Start of picture text --> Cancellation rate<br>0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%<br>With MLS Without MLS<br><!-- End of picture text --> Source: PPI Database, Feb 2016 These observed patterns could lead one to conclude that the cancellation rate of PPP projects is not reduced with financing from multilateral organizations. However, the effect of ‘not having MLS’ is not directly observable in the group of PPPs that do not have MLS. To this end, a proper non-MLS comparison group (counterfactual) for projects _with_ MLS needs to be established.

To do this, a probabilistic model was first estimated to calculate the probability of receiving multilateral support for the full set of PPP projects. Second, a group of projects without MLS but with a very similar (statistically identical) probability of receiving MLS to the group that actually received MLS was identified. Details of the methodology and the results of comparing projects with MLS with their counterfactual are presented in the following sections.

# **_Methodology_**

In randomized controlled experiments, a randomly assigned intervention divides the individuals under study into two groups: those with intervention (treatment group) and those without intervention (control group). Because of the randomization, both groups have exactly the same likelihood of being treated and

> 6. To this point, it is important to recognize also that younger projects which are canceled are less likely to be recognized in the PPI Database due to lags in project status updates. As a robustness check, analysis was also performed on sets excluding PPPs with less than seven and less than five years of potential operability. See sensitivity analysis section and Annex 4 for more detail.

5
are statistically identical in terms of their observable and unobservable characteristics. The effect of the intervention can then be tested by directly comparing outcomes between the treatment and control groups.

In observational studies, however, interventions typically cannot be randomly assigned. In the case of this study, for example, the participation of a multilateral in a PPP is _not_ random. Moreover, a number of factors drive the decision of multilateral banks to support some projects and not others (e.g., multilaterals may focus on certain types of PPP projects or tend towards specific sectors or countries). In other words, projects with and without MLS do not have the same probability of receiving multilateral support and may be systematically different. More importantly, a direct comparison of the outcomes between projects that do and do not involve MLS can lead to the wrong conclusions on the real effects of multilateral support.

To deal with MLS not being randomly assigned, we employ a quasi-experimental design to “identify a comparison group that is as similar as possible to the treatment group in terms of baseline (pre-intervention) characteristics” (White & Sabarwal, 2014). In our case, the comparison group consists of PPP projects without MLS, but with the same likelihood of getting funding from multilateral banks as the treatment group (projects with MLS). The difference in outcomes between treatment and comparison groups provides an estimate of the ‘net effect’ or impact of multilateral support. The cancellation rate is the outcome subject to analysis.

To control for treatment selection bias; that is, projects with MLS systematically different from other PPP projects, a propensity score matching<sup>7</sup> strategy was applied to match the treatment group to a very similar set of projects without multilateral funding. Then, outcomes between the two groups were compared to estimate the effect of MLS. The process consisted of three steps:

1. Defining a metric to compare projects;

2. Selecting a matching estimator and identifying a comparison group; and 3. Analyzing the matched data set to examine patterns of cancellation.

_1. Defining a metric to compare projects_ A first step to compare observations consists of defining a metric or standard of measurement that is common to all projects. This metric can correspond to one criterion or several synthesized in the form of an index. For this analysis, a continuous latent variable, the propensity to receive MLS, was calculated for all PPP projects. As mentioned above, multilaterals may be more prone to participate in certain kinds of projects, sectors or locations. These factors increase or reduce the probability –or propensity– of a project receiving MLS. At the project level, this

> 7. The propensity score matching approach is nicely summarized by Stuart and Rubin (2007), who explain that "propensity scores are balancing scores: At each value of the propensity score, the distribution of the covariates that define the propensity score is the same in the treated and control groups. In other words, within a small range of propensity score values, the treated and control groups’ observed covariate distributions are only randomly different from each other, thus replicating a mini-randomized experiment, at least with respect to these covariates."

6
propensity is simply a score –a number between 0 and 1– that measures its probability of receiving MLS.

The propensity scores at the project level were calculated using a _multilevel mixed effects probit regression_ approach. As described by Guo and Zhao, “multilevel modeling corrects for the biases in parameter estimates resulting from clustering. In contrast to the popular belief, ignoring multilevel structure can result in biases in parameter estimates as well as biases in their standard errors. The more highly correlated the observations are within clusters, the more likely that ignoring clustering would result in biases in parameter estimates” (2000). The results section shows that PPP projects are, indeed, correlated or clustered at the country level. This is unsurprising, given that projects within a country are subject to the same macroeconomic and legal environments and typically fall under a common national infrastructure and investment policy. Moreover, the influence of PPPrelated actors and organizations (e.g., multilateral development banks, operators and sponsors, private financiers, etc.) on the PPP environment may have an equalizing effect on the national PPP environment.

# _2. Selecting a matching estimator and identifying a comparison group_

Propensity scores are then used to match the set of treatment projects (with MLS) to a comparison group. Note that the difference in the propensity scores between projects with and without MLS provides a measure of proximity or closeness between them. Using this information, weights are assigned to each project without MLS following a simple rule: the farther away the comparison unit is from the treated unit, the lower the weight (Essama-Nssah, 2006). The resulting comparison group contains projects without MLS that are weighted according to their average proximity to the ones with MLS.

Following Sianesi (2001), the weight given to a non-treated project is in proportion to the closeness of the propensity scores between projects with and without MLS, where the proportion is defined as: Where K(. ) is a Kernel weighting function, p is the propensity score (probability of receiving MLS), _h_ is a smoothing parameter called the bandwidth,<sup>8</sup> and i and j denote projects with and without MLS, respectively.

# _3. Analyzing the matched data set to examine patterns of cancellation_

Lastly, the cancellation rates associated with the matched treatment and comparison groups are compared to examine any significant differences between contract survival in projects with and without MLS.

> 8. The smoothing parameter is used to approximate a function that best captures the patterns in the data (Simonoff, 2012). In this case, each data point is a pairwise distance between observations pi and pj .

7
<!-- Start of picture text --> Selected countries<br>.<br>+<br>yo +<br>=w<br>Dcr, + +<br>aBu2 + . + i ++<br>Oo 4<br>> +<br>co<br>So<br>o fe LA} 5 oO oO Cc 12<br>#¢ $8 # 3 38 & 2 &<br>@ = £ G 5 a i ie<br>oO > e uw se) a ec<br><= is Oo &<br><!-- End of picture text -->
PPP projects in Africa, Eastern Europe and Central Asia, and Latin America and the Caribbean are more likely to receive MLS. The model shows that compared to Greenfield projects, Brownfield concessions and management and lease contracts are less likely to involve multilateral participation. Transport projects tend to receive less MLS than power projects (the base comparator). Lastly, projects reaching financial closure in earlier years were likelier to receive MLS.

Finally, PPP projects with funding from multilateral banks have an average estimated probability of getting MLS (i.e., the propensity for those projects as estimated by the regression model) almost three times higher than the probability of PPPs without MLS (10.7%). This result supports the point made before –that projects with and without MLS may be systematically different, in turn making a direct comparison in their outcomes misleading regarding the role of multilaterals in the performance of PPPs.

<mark>Table 2. Probability of a PPP project to receive multilateral support (MLS) Mixed-effects probit regression Group variable: country</mark>

|Group variable: country<br>MLS(1=With Multilateral Support)<br>Empty|Basic|Full|
|---|---|---|
|Total investment|0.634<br>***|0.849<br>***|
|Square: Total Investment|-0.090<br>**|-0.120<br>***|
|Sector|||
|Transport|-0.290<br>***|-0.311<br>**|
|Water and sewerage|-0.045|-0.055|
|Type of PPP|||
|Concession|-0.276<br>***|-0.311<br>***|
|Management and lease contract|-0.375<br>***|-0.337<br>**|
|Country|||
|GDP per capita (constant 2005 US$)||-0.0001<br>***|
|Population(millions)||-0.006<br>***|
|Region|||
|AFR||0.559<br>**|
|EAP||-0.290<br>*|
|ECA||0.675<br>***|
|LAC||0.794<br>***|
|MENA||0.225|
|Financial Closure Year|-0.018<br>**|-0.007|
|Constant<br>-0.749<br>***|36.095<br>***|13.479|
|_Country level variance_|||
|_var(investment)_||_0.120_<br>_*_|
|_var(constant)_<br>_0.379_<br>_***_|_0.456_<br>***|_0.147_<br>_***_|
|_LR test vs. probit regression: chi2(2)_<br>_521.39_|_500.88_|_123.74_|
|_Prob > chi2(2)_<br>_0.000_|_0.000_|_0.000_|
|_Wald chi2()_|_88.9_|_340.4_|
|_Prob > chi2_|_0.000_|_0.000_|
|_Multilevel Structure_|||
|_Intraclass Correlation_<br>_0.275_<br>_***_|||
|_# Obs (PPP projects)_<br>_5054_|_5054_|_4927_|
|_# Groups (countries)_<br>_120_|_120_|_117_|
|_Pseudo-R2_<br>_0.136_<br>* p<.1; ** p<.05; *** p<.01<br>Robust Std. Err. adjusted for clustering on country|_0.172_|_0.211_|

Source: Author’s calculations 9
# _Matching propensity scores for multilateral support_

Weights for projects without MLS were calculated in proportion to their proximity to projects with MLS.<sup>9</sup> These weights make projects without MLS suitable for comparison with projects receiving multilateral funding. When applied, the weighted average of the probability of receiving MLS for projects without multilateral funding is statistically identical to the average probability of getting MLS for projects that actually receive it. The bias in the propensity to receive MLS between groups is reduced by 99% (see Annex 2).

# _Cancellation rates and the role of multilateral funding_

The comparison of cancellation rates between the matched sets suggests that multilateral development organizations have a positive effect on the survival of long-term PPP infrastructure contracts _._ Table 3 shows that, without matching, the cancellation rates for PPP projects with and without MLS are statistically the same (6.1% versus 5.8%). However, as shown above, the sets of projects with and without MLS do not have the same probability of receiving multilateral support (28.5% versus 10.7%, respectively), suggesting that the groups are systematically different.<sup>10</sup> As such, a simple direct comparison of the two groups is misleading when trying to understand the real effect of MLS on cancellation.

When projects with MLS are instead matched to a set of PPP projects with the _same probability_ of having MLS but _no actual_ multilateral funding, a different result is revealed: the mean cancellation rate for the counterfactual set of non-MLS projects is 8.6%, as compared to a much lower 5.8% cancellation rate for projects with multilateral support. In other words, it is estimated that the cancellation rate of projects with MLS would have been 2.8 percentage points (about 48%) higher without funding from multilaterals.<sup>11</sup> Ta <mark>ble 3.</mark> Propensity score matching summary, mean rates of cancellation<sup>12</sup>

|Averag|e Treatment Effect on Cancellation Ra|tes of PPPs|with MLS. M|atching metho|d: Kerne|l|
|---|---|---|---|---|---|---|
|Variable|Sample|Treated|Controls|Difference|S.E.|T-stat|
|PPP Status:|Unmatched|0.061|0.058|0.00|0.01|0.22|
|Canceled|Matched using propensity scores|0.058|0.086|-0.03|0.02|-1.83|
||||P(T|< |t|) = 0.074|9 after m|atching|

Source: Authors’ calculations To put this result in perspective, the 24 canceled PPP contracts with MLS identified between 1990 and 2010 (Table 1) represent total committed inve <mark>stments of US$19.3 billion (Annex 5). Without MLS, that number could have risen to 36</mark>

> 9. An Epanechnikov Kernel density function was applied to smooth the calculated distances. Distances are equal to the difference in the propensities between projects with and without MLS. 10. The balance tests in Annex 2 show that projects with and without MLS are different, not just in terms of their probability of receiving MLS, but also in terms of several other project-, country–, and regional– level characteristics.

> 11. For only 11 PPP projects with estimated probabilities of receiving MLS above 67%, it was not possible to find a match. These projects represent 2.8% of the total number of PPP with MLS. See Annex 2 for more details.

> 12. This comparison includes canceled, operational and concluded PPP projects.

10
<mark>canceled PPPs, involving committed investments close to US$28.6 billion (19% of the investments associated with projects with MLS). Note that PPP projects with MLS are generally bigger in terms of committed investment amounts than other PPPs (US$389 million with MLS versus US$196 million without MLS).</mark>

# _<mark>Sensitivity Analysis</mark>_

In addition to the kernel weighting method shown in Table 3, two other matching methods –Nearest-Neighbor and Radius matching– were applied (Caliendo and Kopeinig, 2005). First, a nearest-neighbor matching algorithm was used to match projects with MLS to the closest ones in terms of propensity to receive MLS, but which did not actually receive MLS. These comparisons matched projects with MLS to the 1<sup>st</sup> , 5<sup>th</sup> , 9<sup>th</sup> , 13<sup>th</sup> , 17<sup>th</sup> , 21<sup>st</sup> , and 25<sup>th</sup> closest neighbor projects without MLS. To avoid weak matches (i.e., matching to projects too far away), a ‘caliper’ or radius of 0.02 percentage points was imposed around the calculated propensity to receive MLS. For example, if a project with MLS has an estimated probability of receiving MLS of 0.65, the n<sup>th</sup> closest neighbor(s) would be the one(s) without MLS, but with probabilities of receiving MLS in the range of 0.63 to 0.67.

Second, since there is no rule to choose a specific caliper, a radius matching algorithm was estimated for six different radii ranging from 0.005 to 0.03. Following the example, for a caliper of 0.03, the radius algorithm would match a PPP with MLS and estimated probability of receiving MLS of 0.65 to all projects without MLS and propensities to receive MLS in the range of 0.62 to 0.68. Comparisons through these matching methods (18 counterfactual scenarios in total) showed that PPP projects without MLS have a higher chance to be canceled confirming the kernel-based results (in average, 9.3% for nearest-neighbor-based and 8.8% for radius-based comparisons).

Finally, kernel matched comparisons were also performed on data sets excluding PPPs with less than seven years of potential operability (i.e., financial closure after 2009) and on data sets excluding PPPs with less than five years of potential operability (i.e., financial closure after 2011). This range covers about two standard deviations around the average duration of canceled PPP projects (between 5.3 and 6.4 years) (see Annex 4 for more detail). In both cases, the counterfactual set of projects without MLS showed significantly higher rates of cancellation than the set of PPPs with MLS: 8.7% versus 6% excluding PPPs with less than 7 years, and 8.2% versus 5.3% excluding PPPs with less than 5 years).

Figure 4 represents cancellation rate differences between MLS projects and the counterfactual (the set of projects with the same average probability of MLS, but that did not actually receive it), showing that projects financed by multilateral organizations perform significantly better with respect to contract survival.

11
Figure 4. Relative cancellation rates of MLS and non-MLS projects, based on propensity matched sample <!-- Start of picture text --> Cancellation rate<br>0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%<br>With MLS Counterfactual Without MLS<br><!-- End of picture text --> Source: PPI Database, Feb 2016

# **_Conclusions and Next Steps_**

The analysis identifying counterfactual scenarios suggests a positive impact of multilateral support on PPP contract performance. The main result is that the observed cancellation rate for projects with MLS would have been 48% higher without support from multilateral development organizations. The dynamics that lead to this outcome are yet to be fully explored, but present important questions about the dynamics of project support associated with multilateral involvement in PPPs. Multilateral funding often involves policy advice, capacity building, oversight and risk mitigation, project preparation assistance, assistance in mediation or renegotiation, and other forms of support that could create ‘halo effects’ over and above direct funding. To what extent these kinds of support help mitigate problems that lead to project cancellation is an important area of further study, particularly considering that some kinds of multilateral interventions could potentially be replicated and extended to other projects. Moreover, some types of project support may be more positively impactful than others, and better understanding could help prioritize the oversight and support activities of both lenders and governments.

These findings may also be further expanded by (a) analyzing the effects of multilaterals by source of support (i.e., the specific institutions) and size of investment; (b) analyzing the effects of specific forms of multilateral financial support (e.g., guarantees, loans, etc.) on contract success; and (c) evaluating the impact of multilateral support on project performance outcomes (e.g., coverage expansion, improvements to service quality, efficiency and productivity gains, tariff reductions, etc.).

The latter two proposed analyses require gathering of additional project data. Once additional data is available to capture variations in project status over time, it will be possible to study the role of project and institutional factors on paths of contract performance (e.g., paths to distress and recovery from distress). An important question amenable to study with more historical data is what role multilaterals play in facilitating recovery from distress and reversion to operational status versus cancellation.

12