metadata: authors: 0: author: Marcelo Gordillo,Darwin
1: author: House,Schuyler
2: author: Raina,Aditi
count: Chile
docna: 0: docna: Prioritizing Infrastructure Investments : A
            Comparative Review of Applications in Chile
docty: Policy Research Working Paper
subtopic: Hydrology,Global Environment,Transport Services,Multi Modal Transport,Social Development & Poverty
teratopic: Poverty Reduction,Water Resources,Transport
abstracts: cdata!: Governments worldwide face the difficult
            challenge of deciding which infrastructure projects to
            prioritize and select for implementation, given the limits
            of available funding and the need to attain their
            developmental goals. The key objective of this report is to
            conduct a comparative exercise between the World Bank's
            Infrastructure Prioritization Framework, a multicriteria
            analysis–based methodology to project prioritization, and a
            more complex cost-benefit analysis–based approach. The
            report focuses on Chile, which has a well-institutionalized
            evaluation process that uses cost-benefit analysis to assess
            projects on their quality and ability to generate value for
            money. The analysis compares the results of the
            Infrastructure Prioritization Framework alongside
            Chile's current cost-benefit analysis–based and
            multicriteria analysis approaches to the same subsets of
            projects in the road transport and water reservoir
            subsectors, respectively. The results show that the
            Infrastructure Prioritization Framework has application
            beyond its original proposition and can complement a
            traditional cost-benefit analysis by directly considering
            social and environmental policy goals that are otherwise
            difficult to quantify in a cost-benefit analysis. The
            analysis also finds that in Chile there is a discrepancy
            between the stated goals and objectives of the appraisal
            system and the actual implementation. In the case of
            transport sector projects, there is an evident deviation
            between cost-benefit analysis–based selection policy and
            actual decisions made for project implementation. In the
            case of water catchment selection, there is a bias toward
            projects with higher financial-economic performance as
            compared to social-environmental performance, despite policy
            intentions to afford consideration to environmental and
            social development goals.
geo_reg_key: 119228,517191
repnme: repnme: Prioritizing Infrastructure Investments : A
            Comparative Review of Applications in Chile
display_title: Prioritizing Infrastructure
            Investments : A Comparative Review of Applications in Chile
listing_relative_url: /research/2018/10/30472848/prioritizing-infrastructure-investments-comparative-review-applications-chile
topicv3: Water m1326522 137,Transport m1326661 135,Environment, Natural Resources and Blue
            Economy m1327391 566,Road Transport m1327629 1801,Social Development m1327660 134
publishtoextweb_dt: 2018-10-04T00:00:00Z
historic_topic: Poverty Reduction,Water Resources,Transport
no_of_pages: 60
author_keys: 0: author_key: 937656
1: author_key: 1555573
model_extractions: WPS8602 Policy Research Working Paper 8602 Prioritizing Infrastructure Investments A Comparative Review of Applications in Chile _Darwin Marcelo Schuyler House Aditi Raina_ Infrastructure, PPPs & Guarantees Global Practice October 2018
Policy Research Working Paper 8602

# **Abstract**

Governments worldwide face the difficult challenge of deciding which infrastructure projects to prioritize and select for implementation, given the limits of available funding and the need to attain their developmental goals. The key objective of this report is to conduct a comparative exercise between the World Bank’s Infrastructure Prioritization Framework, a multicriteria analysis–based methodology to project prioritization, and a more complex cost-benefit analysis–based approach. The report focuses on Chile, which has a well-institutionalized evaluation process that uses cost-benefit analysis to assess projects on their quality and ability to generate value for money. The analysis compares the results of the Infrastructure Prioritization Framework alongside Chile’s current cost-benefit analysis–based and multicriteria analysis approaches to the same subsets of projects in the road transport and water reservoir subsectors, respectively. The results show that the Infrastructure Prioritization Framework has application beyond its original proposition and can complement a traditional cost-benefit analysis by directly considering social and environmental policy goals that are otherwise difficult to quantify in a cost-benefit analysis. The analysis also finds that in Chile there is a discrepancy between the stated goals and objectives of the appraisal system and the actual implementation. In the case of transport sector projects, there is an evident deviation between cost-benefit analysis– based selection policy and actual decisions made for project implementation. In the case of water catchment selection, there is a bias toward projects with higher financial-economic performance as compared to social-environmental performance, despite policy intentions to afford consideration to environmental and social development goals.

This paper is a product of the Infrastructure, PPPs & Guarantees Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at dmarcelo@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Prioritizing Infrastructure Investments: A Comparative Review of Applications in Chile

Darwin Marcelo, Schuyler House and Aditi Raina Keywords : Infrastructure prioritization, infrastructure planning, public investment, principal component analysis, multi-criteria analysis, transport, water JEL Classification Codes : R42, O18, O21, O22, H54, C38
# Table of Contents

|Table of Contents ................................................................................................................................................ 2|
|---|
|Abbreviations ................................................................................................................................................... 4|
|Acknowledgements ........................................................................................................................................ 5|
|Chapter 1. Introduction ...................................................................................................................................... 6|
|Approaches to Infrastructure Appraisal and Selection ......................................................................... 6|
|Comparing Approaches: A Case Study of Chile ....................................................................................... 8|
|Chapter 2. Infrastructure Appraisal and Selection in Chile ..................................................................... 10|
|Evolution of Project Appraisal and Selection in Chile ........................................................................... 10|
|Chile’s Project Investment Cycle ................................................................................................................ 11|
|Project Appraisal and Selection in Chile .................................................................................................. 13|
|Extending Investment Decision Support in Chile .................................................................................. 14|
|Multi-Criteria Analysis to Support Investment Decision-Making .................................................... 15|
|Chapter 3. Infrastructure Prioritization Framework: An Alternative Approach ................................. 17|
|The IPF Process ............................................................................................................................................... 17|
|Step 1. Select Criteria ................................................................................................................................... 18|
|Step 2. Prepare Data .................................................................................................................................... 19|
|Step 3. Constructing Performance Indices – SEI and FEI .................................................................... 19|
|Step 4. Creating the Visual Interface: The Investment Prioritization Matrix ................................ 20|
|Evolution of the IPF ....................................................................................................................................... 21|
|Pre-Analytical Steps .................................................................................................................................... 21|
|Technical Improvements: Variable Specification and PCA Restrictions ........................................ 21|
|Sensitivity Analysis and Criteria Weighting .......................................................................................... 22|
|Organizational and Capacity Issues ......................................................................................................... 23|
|Chapter 4. Applying IPF to Chile Infrastructure Project Proposals ....................................................... 24|
|Water Catchment Projects in Chile .......................................................................................................... 24|
|Water Catchment Project Sample ........................................................................................................... 24|

2
|Water Catchment Project Indicators ...................................................................................................... 25|
|---|
|IPF Results: Water Catchment ................................................................................................................. 29|
|Water Catchment IPF Matrix .................................................................................................................... 33|
|Comparing IPF to Selection of Water Catchment Projects ............................................................... 34|
|Applying IPF to Road Transport ................................................................................................................36|
|Transport Policy Goals and Road Project Criteria ................................................................................36|
|Road Transport Project Sample ...............................................................................................................36|
|Transport Project Indicators ..................................................................................................................... 37|
|IPF Results: Road Transport ..................................................................................................................... 38|
|Road Transport IPF Matrix ........................................................................................................................ 39|
|Comparing IPF to Funding of Transport Projects ................................................................................ 39|
|Chapter 5. Conclusion ..................................................................................................................................... 40|
|References .......................................................................................................................................................... 42|
|Annex 1. Chilean Law Relevant to Project Appraisal and SNI ............................................................. 44|
|Annex 2. Water Catchment Raw Project Data ...................................................................................... 45|
|Annex 3. Water Catchment Project SEI Calculations, by Region ...................................................... 47|
|Annex 4. Water Catchment Project FEI Calculations, by Region...................................................... 50|
|Annex 5. Road Transport Raw Project Data ........................................................................................... 53|
|Annex 6. Road Transport Project SEI Calculations, by Region (with standard poverty rate) .... 54|
|Annex 7. Road Transport Project SEI Calculations, by Region (with multidimensional index|
|poverty rate) ...................................................................................................................................................56|
|Annex 8. Road Transport Project FEI Calculations, by Region .......................................................... 58|

3
# Abbreviations

|BIP|Integrated Project Bank|
|---|---|
|CBA|Cost-Benefit Analysis|
|CFA|Centralized Finance Agency|
|CORFO|National Development Corporation|
|DIPRES|Budget Office|
|ESP|Social Project Evaluation|
|IDI|Investment Initiative|
|IPF|Infrastructure Prioritization Framework|
|IRR|Internal Rate of Return|
|MDS|Ministry of Social Development|
|MIDEPLAN|Ministry of Planning and Cooperation|
|MCA|Multi Criteria Analysis|
|MCEM|Multi Criteria Evaluation Methodology|
|NPV|Net Present Value|
|ODEPLAN|National Planning Office|
|RATE|Result of Technical-Economic Analysis|
|RS|Recommended Favorably (according to RATE)|
|SCBA|Social Cost-Benefit Analysis|
|SNI|National Investment System|

4
# Acknowledgments

This report was prepared by a team of experts from the World Bank's Infrastructure, PPPs and Guarantees Group with inputs and guidance from Pilar Contreras García, Head of Unit Public Investments and Non-Financial Assets (DIPRES) at Ministry of Finance, and Eduardo Koffman, Coordinator of the Planning and Development Department at Ministry of Transport and Telecommunications. We would also like to thank the Department of Irrigation Project at Ministry of Public Works for providing all the project-relevant data information for the water reservoirs analysis. The World Bank team included Darwin Marcelo (Task Team Leader), Schuyler House and Aditi Raina. Cledan Mandri-Perrott and Jordan Schwartz provided essential guidance and oversight. The team would also like to thank the World Bank Singapore Infrastructure Hub for its support.

5
# Chapter 1. Introduction

Infrastructure services are significant determinants of economic and social development and are typically prominent components of national development plans. While national governments and their central finance agencies (CFAs) often consider numerous project proposals from agencies, line ministries, and various sub-national government units, financial resources are often insufficient to fund the full set of proposals, particularly in the short-term. Global estimates of infrastructure investments required to support economic growth and human development lie in the range of US$65 trillion to US$70 trillion by 2030, while the estimated pool of available funds is limited to approximately US$45 trillion.

Governments worldwide face a two-pronged challenge; to increase the pool of funding available for infrastructure development and to make difficult decisions about which projects to select for implementation, given the real limits of available funding. This paper deals with the latter challenge –namely the need for CFAs, ministries, and other relevant agencies to prioritize potential infrastructure projects aligning needs with fiscal constraints while attaining their respective economic and social development goals.

The key objective of this report is to conduct a comparative exercise between the World Bank’s Infrastructure Prioritization Framework (IPF), a multicriteria-based methodology to project prioritization, and a more complex Cost Benefit Analysis (CBA) based approach (i.e. more data and analytically intensive). To this end, the report focuses on the infrastructure prioritization process in Chile, a country that is recognized for the strength of its institutions and capacity of its public administration. Chile has a well-institutionalized evaluation process that uses CBA to assess projects on their quality and ability to generate value for money. The Ministry of Public Works (Ministerio de Obras Públicas – MOP) has been recognized for its capacity to prepare and implement high-quality infrastructure projects (OECD, 2017). This report explores the theoretical and practical challenges of prioritization; the robustness and integrity of current approaches; and the comparative outcomes of these approaches and their alternatives. This exercise is intended to serve practical ends.

The purpose is to progress discourse on project prioritization and selection in order to validate useful and productive public administration guidance, on the one hand, and create space for alternative approaches to support sound investment decision-making and responsiveness to nonmonetizable policy aims, on the other.

## Approaches to Infrastructure Appraisal and Selection

In addition to a growing infrastructure gap, the past 20 years have also seen a shift towards decentralized infrastructure planning. Many subnational governments, regional entities, and sector agencies have been delegated responsibility for infrastructure planning to promote local responsiveness. Moreover, while spending ceilings are defined by the centralized finance agency (CFA), allocation of funds for implementation remains at the line ministry level. At the national level, decision-makers must deal with numerous project proposals, each with varying amounts of attendant project information. These projects must be ideally appraised, compared, and selectively allocated funds for implementation.

The framework on Public Investment Management (PIM), proposed by Rajaram et al. (2014), is useful for guiding governments through the processes of infrastructure planning, appraisal, investment, and implementation, with an eye to increase the effectiveness of infrastructure 6
<!-- Start of picture text --> Maintain asset<br>register,<br>Consistency in Authority to operate and _ Evaluation to<br>project screen and maintain improve<br>preparation reject projects asset guidance<br>147<br>3o 3a 64<br>Linktoa Key to An effective budget and<br>development credible procurement process to support<br>strategy selection implementation and operation<br><!-- End of picture text -->
accountability. The World Bank’s Infrastructure Prioritization Framework (IPF) is one such multicriteria analysis (MCA) approach that condenses government-selected project indicators into composite financial-economic and social-environmental indices. The analysis may incorporate the results of financial or partial social cost-benefit analysis, but does not require full SCBA.

# Comparing Approaches: A Case Study of Chile

The IPF has been piloted in Vietnam, Panama, Argentina, and Sri Lanka. These pilots imparted methodological and practical lessons that have been used to adjust and improve the IPF. An important unanswered question remained, however, as to how effectively the IPF can substitute for the best practice of project appraisal and selection based on SCBA. Moreover, while IPF was designed as a ‘next-best’ prioritization approach based on ‘less-than-SCBA’ appraisal, IPF may nevertheless have something to offer countries where SCBA/CBA approaches are institutionalized.

For these reasons, the IPF was additionally piloted in Chile, where CBA-based analysis is a standard input to project acceptance for economic infrastructure such as roads, transfer ports, dams, railroads, etc. Chile stands apart from much of the world with respect to systematic, institutionalized project appraisal and evidence-based project selection. The Government of Chile (GoC) has a centrally-managed Public Investment System (SNI) that separates project proposal (initiated by line agencies and sub-national units) from appraisal, selection, and budget allocation performed by the Ministry of Social Development and the Ministry of Finance. The SNI is used to consolidate project information, subject proposals to policy filters, and appraise projects before inclusion in sector plans and budget requests. The Chilean SNI is likely the most systematically managed and consolidated investment appraisal system in Latin America (de Rus Mendoza, 2014) and is generally seen as a good example of a “structured and coherent framework for identifying, coordinating, evaluating and implementing public investments” (OECD, 2016, p. 93).

The ready availability of CBA appraisals in Chile proffered a valuable opportunity to compare IPF outcomes with CBA-based project selection. Moreover, the IPF is relevant in the Chilean context for other reasons. For one, the government recognizes the value of additional policy considerations alongside the results of CBA and has implemented a multi-criteria approach to project selection in the water sector, indicating recognition of the value of MCA even where cost-benefit analysis is widely applied. Second, the CBAs employed for sector-level project selection deviate somewhat from the academic policy approach to SCBA, due primarily to the realities of time and resource demands. Most appraisals – particularly for small- to medium-size projects – are partial (financial) CBAs that rely on highly standardized assumptions and often yield results with limited variance across projects. This exposed the possibility for MCA approaches to help fill in the missing considerations in partial CBAs.

This report presents an overview of the current system of project appraisal and selection in Chile, a summary of the IPF methodology and its evolution, and the comparative results from applying IPF alongside Chile’s current CBA and MCA approaches to the same subsets of projects in the road transport and water catchment subsectors. The report follows with a discussion of the findings of the exercise.

The results of this exercise show that the IPF has application beyond its original proposition of being a stop-gap measure until more sophisticated project appraisal methods can be implemented. This is because it can complement a traditional CBA by directly considering social 8
and environmental policy goals that are otherwise difficult to quantify in a CBA analysis. In addition, this exercise led to two deeper findings that went beyond the initial aim of merely comparing the IPF and CBA results. The first was that CBA analyses are not used as the basis of prioritization in Chile. There was an evident deviation between CBA-based selection policy and actual decisions made for project implementation, in the case of transport sector projects. The second was the fairly consistent alignment of IPF- and MCA-based prioritization in the case of water catchment selection, but with a surprising inclination towards projects with higher financial-economic performance as compared to social-environmental performance, despite policy intentions to afford key consideration to security, environmental, and social development goals. Therefore, in both cases, there is a discrepancy between stated goals and objectives of the appraisal system and the actual implementation.

9
# Chapter 2. Infrastructure Appraisal and Selection in Chile

The SNI is essentially a set of processes, data collection mechanisms, and appraisal functions that support project selection across multiple sectors. By design, and with the overarching policy goal of promoting economic growth, project selection is generally based on social net present value. Because SCBA does not consider distributional effects and regional or territorial inequalities, the SNI is complemented by a cost efficiency analysis when ‘desired but non-quantifiable’ social or environmental outcomes are deemed significant enough to justify project costs.

Chile has developed a CBA-based system for investment decisions (Candia et al., 2015). Over time, the investment system has evolved, most recently by extending the appraisal approach to consider additional factors via multi-criteria analysis (MCA). The following section provides an overview of the evolution of Chile’s infrastructure investment system and its current technical and institutional aspects.

## Evolution of Project Appraisal and Selection in Chile

Chile’s Sistema Nacional de Inversiones (National Investment System) (SNI) is a centralized public investment system jointly administered by the Ministerio de Desarrollo Social (Ministry of Social Development) (MDS) and the Ministerio de Hacienda (Ministry of Finance), via the Dirección de Presupuestos (Budget Office) (DIPRES). MDS is responsible for ex-ante project appraisal and expost evaluation, as well as systematic data collection and reporting, while the Ministerio de Hacienda (through DIPRES) sets the public budget.

The SNI is the latest organizational arrangement in an extended history of formalized project appraisal and selection. The genesis of Chile’s investment system was the Corporación Nacional de Fomento (National Development Corporation) (CORFO) established in the 1950s, created to evaluate the financial and social impacts of national projects and units, with a strong emphasis on state enterprises. This agency’s role in investment decision-making was assumed in the 1960s by the Oficina de Planificación Nacional (National Planning Office) (ODEPLAN), which gave rise to the first formal project appraisal system. ODEPLAN also served as a platform for developing government capacity for project appraisal, leading to the creation of a specialized Social Project Evaluation unit (ESP). Through the 1970s, Chile developed extensive guidance, processes and methodologies for project appraisal and developed an integrated investment decision-making system that specified the roles and relationships among the ministries and other governmental units. Institutionally, this would be consolidated in the 1990s with the creation of the Ministry of Planning and Cooperation (MIDEPLAN), later renamed the Ministry of Planning in 2005, and replaced by the Ministry of Social Development (MDS) in 2011.

The national appraisal system employed cost-benefit analysis from its earliest days to appraise proposed projects. During the 1970s, however, it was recognized that some projects – particularly in social sectors like health and education – involve social benefits that are difficult to estimate, but which may be assumed high enough to outweigh project costs. This led to the adoption of a ‘cost-efficiency’ approach to appraisal in some sectors, wherein effort is concentrated on minimizing costs to attain the desired outcome (often the provision of basic services), with qualitative assessment of expected benefits. The cost efficiency approach also allowed the government to deal with distributional effects and regional inequalities.

The government employed hybrid approaches to investment appraisal, including cost-efficiency and multi-criteria approaches, to support project prioritization. The cost-efficiency approach was 10
often applied by combining expected benefits into a single composite indicator to be compared to project costs.<sup>3</sup> While CBA remained the standard project appraisal method, cost-efficiency approaches were used to appraise 71% of projects proposed between 2000 and 2015, accounting for 47% of proposed investments (Agostini and Razmilic, 2015). Therefore, a spectrum of appraisal techniques has been in use since the 1970s. Over the past few decades, the CBA and MCA methodologies have developed with respect to valuation approaches, applied assumptions, and analytical sophistication, but the overall methodologies and institutional frameworks have remained quite the same.

# Chile’s Project Investment Cycle

By law, except the armed forces (which have their own systems), any public-sector institution wishing to develop an investment project must do so via the SNI. Law pertinent to this process is described in Annex 1. The proponent unit initiates the process by submitting background project information to the SNI. This information is immediately available to the public via an open digital registry called the Banco Integrado de Proyectos (Integrated Project Bank) (BIP). The BIP provides a record of all project proposals in standardized format and tracks project development from initial proposal through ex-post project evaluation.

The proponent agency engages in an iterative process of submissions and approvals with MDS, via the SNI, that involves increasing levels of detail with respect to project appraisal as the project progresses through the system. Upon initial submission to the SNI, a project is assigned a unique identification code within the BIP, which can be used to track the projects’ progress through these stages. Except for some project types (e.g., projects with pre-approved designs), project proponents must submit project preparation information to the SNI, via the BIP platform, at the following stages:

- Profile (concept): the policy problem is described, along with the purpose and context of the project, alternative solutions under consideration, and an assessment of the feasibility and impacts of various alternatives to inform the selection of the most viable alternative;

- Prefeasibility: prefeasibility studies include additional project details, including tentative schedules, budgets, and more extensive information on expected benefits;

- Feasibility: full feasibility studies, including CBA or cost-effectiveness analysis are provided;

- Design: technical architectural, engineering, and construction studies are done, and the timings of investments and detailed budget are specified. Project execution plans are required to be based on specific estimates of the costs of equipment, personnel and supplies, as well as a realistic schedule to estimate the duration of the various activities required; and

- Execution: the project is approved to seek funding. Some projects can apply to the design and execution stages simultaneously when the main sources of risk are known.

Simple projects may not include detailed information at every step, however, and may move directly to project execution from the project profiling stage.

At each stage, MDS assesses the project and approves or rejects its progression to further development, depending on whether it meets the requirements of each stage. It then issues a

> 3 Pilar Contreras, an economist with a long career in public investment in Chile (ODEPLAN) and currently serving as Chief of Investment (DIPRES), reported in an interview that, where CBA was not feasible for health and education projects, analysts used relevant variables common to all projects (e.g., malnutrition, education, infant mortality, etc.) to construct a weighted, combined single indicator reflecting each project’s projected impact.

11
Resultado del Análisis Técnico Económico (Economic Technical Analysis Result) (RATE), which results in one of the following RATE results:

- a) Recommended Favorably (RS);

- b) Missing Information (FI), to specify that records lack required information necessary to secure favorable recommendation;

- c) Technical Objection (OT), reflecting a negative assessment;

- d) Reassessment (RE), wherein the project is recommended for additional analysis; or

- e) Breach of Regulations (IN), when spending is executed without the support of MDS.

Projects must attain a favorable RATE (RS) at each stage to move to the next stage of development.

For typical projects, the information required to pass each stage is summarized in Table 1.

Table 1. SNI Informational Requirements for Investment Project Assessment

|Stage Transition|Submission Requirements|
|---|---|
|Profile to Prefeasibility / Feasibility|Pre‐investment study containing:<br><br>Definition of the problem<br><br>Analysis of supply and demand<br><br>Study of solution alternatives<br><br>Initial cost estimates<br><br>Preliminary strategic and economic evaluation<br><br>Further specification of best solution|
|Prefeasibility / Feasibility to Design|<br>Detailed budget<br><br>Feasibility study<br><br>Detailed line‐item budget|
|Design to Execution|<br>Full engineering design<br><br>Draft bidding proposal|

Sources: Government of Chile (2017). _Standards, Instructions, and Procedures for the Public Investment Process (PIN);_ Presentation: _Public Investment Management Conference: The Chilean Experience_ Once a project moves to the Execution stage, the proponent may seek funding for the project (in the SNI, the designation ‘Execution’ simply reflects authorization to seek funding, but does not necessarily mean that the project is funded or under implementation).

Projects are typically funded from the proponent unit’s annual budget allocated by the Ministerio de Hacienda. Depending on the agency, funding may also come from other sources. For example, projects formulated by municipalities are funded by the regional government through the National Fund for Regional Development (FNDR). Projects funded solely by the national government do not have to necessarily have an RS from SNI, though projects that are regionally funded do require an RS RATE by law.<sup>4</sup> Within the limits of their respective budget allocations, public entities (ministries, government agencies, etc.) then apply their own approaches to prioritize and select projects. In the case of nationally-funded projects, agencies must select from among projects that have been positively recommended by MDS for funding (assigned a RATE of RS). In the transport sector, projects are given ‘high’, ‘medium’, or ‘low’ priority by a largely qualitative consideration of the project’s

> 4 Organic Constitutional Law on Government and Regional Administration, Law 19175, Article 75.

12
alignment with sectoral and national strategies, whereas in the case of developing small reservoirs, appraised projects are subject to a multi-criteria process to prioritize.

# Project Appraisal and Selection in Chile

Project selection in Chile is notionally based on social cost-benefit analysis (SCBA), with an overarching goal of maximizing societal benefits using scarce public resources. SCBA requires the quantification and monetization of societal costs and benefits, including potential positive and negative externalities as well as social and environmental benefits and costs that may be difficult to quantify and monetize. All projected costs and benefits are discounted to determine the net present values (discounted benefits minus discounted costs) of proposed projects, from the societal point of view.<sup>5</sup> Generally speaking, SCBA can be used either to eliminate projects whose costs outweigh benefits or that do not meet minimum internal rates of return (IRR), or to rank projects by highest net present value (NPV), benefit-cost ratio (BCR), or NPV index (the ratio of the net present value of benefits minus costs to the value of the initial investment). The great strength of SCBA is the ability to compare projects across sectors and regions based on a common metric of monetized value.

In Chilean practice, prior to considering CBA outcomes, projects must first pass initial screenings for legality and strategic alignment and meet the informational requirements for every stage of the SNI. SCBA is, thereafter, used to filter projects based on a minimum internal rate of return rather than to rank projects. In other words, CBA results help decide eligibility for further development but are not necessarily used to prioritize from among RS-rated projects.

Chile developed its capacity and processes for CBA appraisal involving sophisticated estimation techniques, including the use of shadow pricing, the application of various estimation assumptions and methods for various kinds of projects, and standardized use of social discount rates and conversions for values of various expenses and profits in analyses. Some recent advances in project appraisal that have been mentioned are:

- Consideration of the benefits associated with decreased road traffic accidents;

- Consideration of the benefits associated with the reduction of greenhouse gas effects;

- Evaluation of multipurpose projects (dams) or project networks (bike paths);

- Consideration of increased traffic generated by transport projects (as a benefit);

- Use of hedonic pricing in the evaluation of urban parks; and

- Improved measurement of social prices (e.g., fuels, carbon dioxide, travel and leisure time).

Proposed investments in the transport; forestry, agricultural, and fisheries; and water sectors must be subjected to CBA to generate economic indicators such as Internal Rate of Return (IRR), Net Present Value (NPV), and Net Present Value Index (IVAN). These metrics are to be included in SNI project documentation and are used to guide investment decisions.

One of the most important calculated values – IRR – is used to filter projects. Following initial project appraisal and pre-feasibility, investments that do not meet a minimum IRR of 6% are eliminated from consideration (i.e., they do not receive RATEs of RS). Exceptions to this filtering rule are made for projects with low IRRs that are nevertheless deemed strategically significant (in

> 5 In Chile, the discount rate is approved by law and encoded in guidance on application of CBA to projects included in the SNI. See SNI instruction here (link).

13
the case of water security, for example) and/or when it is recognized that CBAs are missing key information or are unlikely to capture important social benefits (e.g., closing regional income disparities, assuring future environmental quality, etc.). In these cases, a cost efficiency analysis or MCA are used to augment the appraisal (see discussion on MCA to follow).

# Extending Investment Decision Support in Chile

While Chile has an extensive record in systematic project appraisal, there remain some technical, policy-oriented, and procedural shortfalls related to its current use as the basis of investment decision-making. These are also recognized internally and have served as an impetus for recent government efforts to extend the processes of project selection to include additional approaches to appraisal and comparison.

In the transport sector, for example, the Highway Design and Maintenance Model (HDM-III/4) has been applied since the late 1980s to extensively estimate expected full life-cycle costs (including construction and maintenance) and benefits (e.g., maintenance, fuel, and travel time savings) associated with road projects. Nevertheless, due to the time and resource demands inherent to SCBA, appraisers must employ extensive assumptions. This can reduce the variation of results across sets of similar projects, tempering the comparative power of estimated metrics.

Typically, it is also only feasible to account for select costs and benefits. Some key considerations may be excluded from analysis, especially costs and benefits that are strategic, environmental, or distributional in nature. Rooted in economic optimization and efficiency, CBA inherently favors projects that generate higher revenues and, therefore, cannot account for strategic or distributional issues. CBA also does not give weight to future-oriented goals such as national security or environmental preservation and privileges more profitable<sup>9</sup> projects in metropolitan areas and low-cost regions (such as coastal metropolitan areas). As such, infrastructure funding in Chile is often concentrated in regions that are already more developed, exacerbating territorial inequalities (Ahmad & Viscarra, 2016).

It is increasingly recognized that infrastructure development must consider an extended set of goals beyond economic efficiency. As a recent OECD report on ‘Gaps and Governance Standards’ in Chile’s infrastructure development system states, “The project evaluation and prioritization system will need to accommodate transversal issues and multiple policy goals,” including sustainability commitments. The report points out that “the current system offers limited scope for incorporating transversal issues and other political objectives into the decision-making process in a transparent way. Nevertheless, changes to project evaluation methodologies and selection criteria must not come at the expense of value for money and efficiency considerations” (2017). For this reason, multi-criteria analysis can be helpful to incorporate multiple considerations in addition to maximization of economic benefits, including climate change, cost efficiency, and regional inequality, in a transparent manner (OECD, 2017).

Lastly, putting issues of methodological robustness aside, the outcomes of CBA analysis (e.g., IRR, NPV, IVAN) do not, in fact, strictly guide project selection or the order of fund allocation. As mentioned earlier, CBA is used to filter projects (e.g., removing those with IRRs of less than 6%) through to ‘Execution’ status, which explicitly confers SNI approval and allows the proponent to seek funding. The capital budget submitted to Congress by the Ministry of Finance may only include projects approved by SNI. Therefore, passing the CBA filter is a necessary condition of funding and implementation. But beyond this, calculated IRRs and other SCBA metrics (NPV, IVAN, BCR) are not necessarily used to prioritize within the set of projects that attain ‘Execution’ 14
status. Rather, proponent units may use any number of approaches (which are often undocumented or based on loosely-defined qualitative criteria) to select projects for implementation from the SNI-approved set. Moreover, a budget decree issued for a sector or unit does not bind the unit to developing the specific projects included in the budget proposal to the Ministerio de Hacienda.

# Multi-Criteria Analysis to Support Investment Decision-Making

Some efforts have been made to extend the Chilean approach to project appraisal and investment decision-making to deal with the technical, policy-related, and implementation issues discussed above. In addition to the cost efficiency approach (which assumes that project benefits will be sufficient to justify estimated costs), Chile has also institutionalized the use of multi-criteria analysis (MCA) to support investment decisions for some sectors.

Specifically, MCA has been applied in the rural water sector to deal with an observed mismatch between the methodological outcomes of CBA for water catchments and strategic goals of the sector. More specifically, water security warrants the development of water catchments to ensure the long-term availability of water for agricultural, residential, and industrial use, but CBA appraisals of water catchments typically yield IRRs of less than 6%, which would result in the filtering out of most catchment projects under the prevailing SNI process. As such, a 2014 Decree on the Use of a Multi-Criteria Evaluation Methodology (MCEM) for Small Reservoirs was issued by the government, based on a set of criteria and weights approved by the National Irrigation Commission.

The MCEM applies the Analytic Hierarchy Process (AHP) to aggregate five criteria and 20 subcriteria into an overall score associated with each water catchment project.<sup>6</sup> The methodology is applied in the final stage of progression through the SNI, which starts with an initial filtering to eliminate projects that require resettlement, impose environmental threats, or exhibit various technical difficulties. Thereafter, projects pass through the pre-feasibility, feasibility, and design stages in SNI as in other infrastructure sectors. At any time, projects may be filtered out if major environmental, technical, legal, or political difficulties arise. In the last stage, catchment projects that pass filtering are subject to the small-reservoir MCEM analysis for final selection.

The criteria and sub-indicators applied to select reservoir projects are detailed in Table 2, along with weights used to combine criteria into a single score. Values associated with the sub-criteria are not measured as continuous variables. Rather, sub-criteria are scored ordinally. Most are given an ordinal score across a range (often 0, 5, or 10), though some are simple binaries [0,1].

An additional criterion – technical complexity – is used in the ultimate selection of projects, though this is not weighted along with the MCE criteria in Table 2. This additional consideration covers technical issues such as soil mechanics, location of the catchment with respect to natural channels, proximity of materials earthworks and dumping sites for excavated soils. The degree to which technical complexity influences the ultimate selection of projects, relative to the MCE scoring, is not specified.

> 6 For technical details of AHP, see Saaty, Thomas L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9-26.

15
Table 2. Reservoir MCEM Criteria and Sub-Criteria

|Criteria / Weight|Indicators / Sub‐criteria|
|---|---|
|Economic<br>18.4%|Social VAN ($ million)<br>Investment ($ million) / hectare<br>Investment ($ million) / land plots|
|Social<br>34.1%|% households under poverty line<br>Surface area of subsistence farms / small farms <12 ha<br>Number of beneficiaries (population in irrigable zone)<br>Indigenous communities in the territory [0,1]<br>% growth of rural population during last inter‐census period<br>Extreme zone, border region, or undeveloped area [0,1]|
|Strategic<br>22.6%|Number of water shortage decrees in past five years<br>Number of jobs generated (landowners and relatives)<br>Number of irrigation association systems that can be connected<sup>7</sup><br>Electricity generation capacity (MWh/year)|
|Environmental /<br>Territorial<br>9.2%|Number of people required to relocate<br>Number of archaeological site affected<br>Hectares of native forest in flood zone|
|Management<br>15.7%|Interest/support of beneficiaries<br>Economic contribution of regional government (regional government<br>contribution / total project investment)<br>Organization (1‐4 indicating degree of incorporation / legal standing)<br>Number of land parcels required to expropriate|

Source: Ministerio de Hacienda, 2014. Minuta Matriz Multicriterio Plan de Pequeños Embalses

> 7 This measures the number of ‘asociación pequeños regantes’ (APRs) or small irrigation associations that can be linked.

16
# Chapter 3. Infrastructure Prioritization Framework: An Alternative Approach

The Infrastructure Prioritization Framework is a quantitative multi-criteria approach to withinsector infrastructure project comparison. The IPF condenses project-level indicators (selection criteria) into two composite indices – a financial-economic index (FEI) and a social-environmental index (SEI) – and considers these alongside the budget constraint for a particular sector. Results are displayed graphically to map the projects’ relative expected performance along these two dimensions.

While the IPF is quantitative in nature, it is also policy-responsive, since the government specifies the set of project selection criteria that reflect the sectoral and developmental policy goals. These criteria may include social, strategic, and environmental considerations alongside traditional financial and economic factors. In fact, the IPF was developed by an infrastructure team within the World Bank in response to government demand for alternative decision support approaches that could directly consider key policy goals; be feasibly applied across large projects sets within the resource means of the government; and remain systematic and evidence-based (Marcelo et al., 2016).

IPF is designed to employ quantitative measures to the greatest extent possible to systematize project comparison and limit subjectivity in selection. While the IPF is a multi-criteria approach, it can utilize the results of CBA analysis as a key decision factor. While the approach was initially envisaged for low-capacity governments, its relevance to the Chilean context is demonstrated in the results of this report. In this section, we summarize the IPF prior to presenting the results of its application to the transport and water catchment sectors in Chile.

## The IPF Process

Implementing the IPF is relatively straightforward and follows five steps: (1) selecting decision criteria; (2) gathering related project indicator data; (3) calculating social-environmental and financial-economic indices; (4) plotting projects and budget limits; and (5) comparing projects (see Figure 2). In this section, we summarize IPF application in terms of these steps.<sup>8</sup>

> 8 An extensive technical description of the IPF methodology is detailed in Marcelo et al., 2016, and Marcelo et al., 2015.

17
<!-- Start of picture text --> Decided with decision makers, experts, and and key<br>stakeholders<br>[| | available)Source project data (CBASource project data (CBA project data (CBA data (CBA (CBA elements incorporated incorporated if<br>Using statistical and and mathematical techniques techniques<br><!-- End of picture text -->

Decided with decision makers, experts, and and key stakeholders [| | available)Source project data (CBASource project data (CBA project data (CBA data (CBA (CBA elements incorporated incorporated if Using statistical and and mathematical techniques techniques PerformanceSerformance IndicesIndices | (05 combinere selected variablesto create SEI [ fie se kta, | Combine SEI, FEI, and budget constraint SEI, FEI, and budget constraint FEI, and budget constraint and budget constraint constraint se alltel rine to create a four-quadrant four-quadrant matrix boos Based on informed deliberation on informed deliberation informed deliberation deliberation

<!-- Start of picture text --> Indices 5 re<br>[ fie se kta, | Combine SEI, FEI, and budget constraint SEI, FEI, and budget constraint FEI, and budget constraint and budget constraint constraint<br>se alltel rine to create a four-quadrant four-quadrant matrix<br>boos Based on informed deliberation on informed deliberation informed deliberation deliberation<br><!-- End of picture text -->
# Step 2. Prepare Data

The second IPF step is to gather and transform raw data so that they are usable for calculating SEI and FEI project scores. A simple Excel environment can be constructed to populate a prioritization database with raw data. Because selection criteria variables may have different units of measurement (i.e., they are not all dollarized costs and benefits) but are combined in additive models, two types of data transformation are required. First, any qualitative data must be transformed into either ordinal or scalar data.<sup>9</sup> Second, observations are standardized to deal with disparate units of measure, transforming all measurements to have a transformed value between [-1] and [1], with the set having a zero mean and unit variance.<sup>10</sup>

# Step 3. Constructing Performance Indices – SEI and FEI

Indices are used to combine information from multiple variables into composite indicators. In IPF, variables are organized into two classes to construct the social-environmental index (SEI) and the financial-economic index (FEI). The FEI may be the standardized value of a single component derived from CBA (e.g., NPV or BCR) or a combination of several factors, but the SEI is typically constructed by combining a number of key social and environmental variables. This is done via an additive model, wherein each indicator’s contribution to the overall index score is determined by weights (the coefficients associated with each variable).

For example, if the SEI variables selected include (a) number of beneficiaries (BEN), (b) number of poor served (POOR), and (c) number of jobs created, the function may be expressed as

## SEI� w� ��BEN�� w� ��POOR�� w� ��JOBS�,

where weights, w�, are associated with each social-environmental indicator.

The weights used to combine variables can be set subjectively or objectively, such as using some form of Principal Component Analysis (PCA). PCA is an information reduction procedure that seeks redundancies in sets of variables. These redundancies can be expressed as linear combinations or ‘principal components’ of the variables comprising the set. One key characteristic of PCA is the ability to calculate coefficients (weights) based solely on the statistical relationship between variables. While other weighting schemes may be used, PCA is particularly useful when there is a preference to objectively assign weights.

Some significant advances have been made with respect to using PCA to determine weights. These changes have been made to deal with policy preferences regarding the relative importance of criteria. Over the course of the Sri Lanka and Chile pilots, for example, calculation methods were developed to add restrictions to PCA that can attain the following: require a particular coefficient sign (+/-) associated with specified variables; require that criteria are weighted to reflect a pre-set order of importance (a weighting order); and require a minimum weight for a variable.

> 9 This can be done with either scaling methods or via approaches like ALSOS. The transformation of categorical and ordinal qualitative and quantitative data into usable numerical data may be done using the Alternating Least Squares Optimal Scaling (ALSOS) algorithm, a widely-accepted transformation approach. Within a quantified ordinal variable, the numbers assigned by the ALSOS algorithm to each category reflect the distance between categories, revealing the implicit metric of the variable (Perreault & Young, 1976).

> 10 Numerical values are standardized via a standardization formula that can be coded into Excel. The standard score z of a raw score x is z_ij= (x_ij-μ_j)/ _j, where μ is the sample mean and  is the standard deviation of the variable j.

19
After SEI and FEI variables are combined in the additive model, resulting values are normalized and rescaled to generate SEI and FEI scores between 0 and 100 for each project. The rescaled score z� can be expressed as z� � �z� �Z���� / �Z��� �Z��� � � 100, where Z��� is the minimum value for variable z and Z��� is the maximum value. These rescaled scores are used as the SEI and FEI scores for plotting in Step 4.

# Step 4. Creating the Visual Interface: The Investment Prioritization Matrix

To create a visual comparison, projects are plotted on a two-dimensional Cartesian plane, with axes representing the SEI and FEI. The budget limit for the sector is also imposed along each axis (intercepting the axis where funds are exhausted). First, however, the budget must be hypothetically allocated separately to the SEI- and FEI-ranked project lists to determine the fundable sets in each. In other words, the budget limit is hypothetically allocated to the top-ranked projects on each list (as if selection were based only on SEI or only on FEI) until resources are exhausted. The resulting fundable sets are compared simultaneously on the investment matrix.

A ‘good’ project in terms of financial and economic performance may nevertheless be undesirable from a social and environmental perspective, and vice versa. As such, decision-makers must consider projects along both dimensions. Projects can be compared by their respective SEI and FEI scores on a visual interface called the Infrastructure Prioritization Matrix (Figure 3). Once projects are plotted, the budget limit is imposed onto the plane – perpendicular to each axis – at the point where the budget would be exhausted if funding were determined solely by each index. The plane is intersected by the dually-imposed budget limit, creating four quadrants.

Figure 3. Example Investment Prioritization Matrix, Panama Water and Sanitation Pilot, 2015 <!-- Start of picture text --> 50<br>45 P15<br>P3<br>40 P23<br>P10<br>P1<br>P24<br>35<br>30<br>P27<br>25 P17 P29 P4 P16<br>P12<br>P22 P28<br>20 P7<br>P35<br>P19<br>P21 15 P18 P13<br>P26 P25 P31 P5<br>P11 P34<br>10 P32 P6<br>P20<br>P30 P33P14<br>5<br>0<br>0 10 20 30 40 50 60 70 80 90 100<br>FEI<br>SEI<br><!-- End of picture text --> Source: Prioritizing Infrastructure Investments in Panama: Pilot Application of the World Bank Infrastructure Prioritization Framework (April 2016) 20
Projects that fall inside the budget constraint along each axis represent the ‘Investment Possibilities Set’ for each dimension. Projects in the upper right quadrant fall in the Investment Possibilities Set for both SEI and FEI and are then categorized as ‘High Priority’ projects.

# Evolution of the IPF

Through pilot applications of the IPF in Vietnam, Panama, Argentina, Sri Lanka, and Chile, the framework has been refined. This section discusses key areas of progression, including lessons learned on important pre-analytical steps and capacity and institutional requirements, technical aspects of variable specification and weighting, and the improved use of sensitivity analysis.

## Pre-Analytical Steps

Pre-analytical processes can help filter projects to reduce the analytical burden of prioritization and ensure sufficient comparability of data during project comparison. One of the challenges of early pilots was that some data, even from within feasibility studies, was either opaquely determined or had limited comparability across projects (Mandri-Perrott, Marcelo, and Haddon, 2015). Feasibility studies should follow clear rules, guidelines, and standards of appraisal to ensure quality and comparability of data (particularly financial estimations) across projects.

Additionally, filters are helpful to ensure that projects meet basic informational, policy, or strategic requirements and/or align with key sector goals. Filters may also be useful when there are inherent biases observed in the set of projects proposed or where the government aims to break regressive patterns. In Vietnam, for example, it was observed that projects in poorer regions tended to score lower on inputs to the FEI or SEI. This observation justified use of an initial filter to target areas with higher poverty rates (Mandri-Perrott, Marcelo, & Haddon, 2015).

## Technical Improvements: Variable Specification and PCA Restrictions

A second set of lessons is that special consideration should be given to the selection and definition of variables as well as the weights assigned via PCA. For one, metrics must be carefully specified to deal with regressive biases. As in Vietnam, Panama revealed an inherent bias towards infrastructure projects in wealthier urban regions due to better scoring on project indicators. If development plans aim to improve rural areas, however, this can yield adverse results. Alternative to using a filter, this problem can be overcome by careful indicator specification and/or the inclusion of additional indicators to capture development goals.

Variable specification is also important to balancing considerations of efficiency and efficacy. For example, one could use the absolute number of beneficiaries as an input to the SEI to consider policy effectiveness where service expansion is a priority. On the other hand, ‘beneficiaries per dollar spent’ may be more appropriate if the key goal is fiscal efficiency. In the case of Panama, where development of rural services is an important policy goal, the decision was made not to control indicators by project size to avoid privileging urban projects with greater economies of scale (Marcelo, Mandri-Perrott, & House, 2015). Another lesson on variable specification relates to the appropriate use of financial and economic indicators under conditions of low information, particularly regarding project benefits. If only project costs can be estimated, additional variables must be considered to construct the FEI (i.e., FEI should not be based on cost only).

Another technical issue arose regarding the use of PCA for weighting. Since PCA synthetizes information based on correlations between variables (which may yield positive or negative 21
coefficient signs), it is important to make sure that weights reflect the desired relationship between a variable and the composite indicator. A problem arises, for example, if PCA assigns a negative weight to a variable that should be positively rewarded in selection. In some cases, this can be resolved by alternative specifications of a component variable, but an important methodological development has been the imposition of a coefficient sign restriction in PCA. This development allows the user to restrict PCA results to ensure that variables that should positively contribute to the SEI are assigned positive-signed coefficients, and vice-versa for variables that should be scored negatively.

# Sensitivity Analysis and Criteria Weighting

Another important improvement has been the addition of sensitivity analysis to test the robustness of results with different variable specifications and criteria weightings. In the Chile pilot, for example, there was an expressed goal of focusing transport investments in areas with higher poverty. Since poverty rates may be measured in several ways, however, a sensitivity analysis applied two alternative poverty rate approaches to compare IPF results.

Since criteria weighting is one of the most important methodological decisions for building indices, it is another important area of sensitivity analysis. The results of IPF are determined, in part, by the weights associated with each indicator. Though a significant lesson from Panama was that composite indices were far more sensitive to indicator values than to the weights used to combine them.<sup>11</sup> While this must be further tested, this suggests that PCA may be a useful way to weight variables if time and objectivity are important factors in selection.

In practice, the use of subjective weighting can give rise to several problems, including lack of transparency, manipulation of weights to privilege ‘pet projects’, or index scores with low variation (and then, limited value for comparison). On the other hand, subjective weighting is more intuitive and directly responsive to policy preferences. Developments have focused on finding a compromise between the responsiveness of subjective weighting and the objectivity of PCA.

In addition to the sign restrictions discussed above, the need arose to adjust the mechanics of PCA to better capture policy preferences. The use of PCA was further refined to allow several additional restrictions on coefficients. These include the ability to specify a minimum value for a coefficient and the ability to specify the order of weighting (order of importance to the overall score). These restrictions define a spectrum from purely objective to purely subjective weighting (Figure 4).

Figure 4. Spectrum of Index Weighting Approaches

> 11 A sensitivity analysis was performed to compare PCA indices against composite indices using subjectively established weights. Two subjective weighting schemes (equal weighting and hypothetical policy-determined) were tested to calculate alternative SEI composite indices. The categorization of projects changed only minimally when using policydetermined or equal weights (Marcelo, Mandri-Perrott, & House, 2015).

22
<!-- Start of picture text --> Positive<br>Objective<br><!-- End of picture text --> <!-- Start of picture text --> PCA<br><!-- End of picture text --> <!-- Start of picture text --> Neutral<br><!-- End of picture text --> <!-- Start of picture text --> Sign-constrained<br>PCA<br><!-- End of picture text --> <!-- Start of picture text --> Normative<br>Subjective<br><!-- End of picture text --> <!-- Start of picture text --> Subjective<br>weighting<br><!-- End of picture text --> <!-- Start of picture text --> Minimum-<br>value PCA<br><!-- End of picture text --> <!-- Start of picture text --> Preference-<br>ordered PCA<br><!-- End of picture text --> <!-- Start of picture text --> Responsive<br><!-- End of picture text -->

# Organizational and Capacity Issues

To improve robustness of results and foster concurrent application with other supportive analytical tools (including CBA and expert assessment), users must have sufficient technical capacity to understand the mechanics and implications of key decisions regarding the use of IPF, including decisions about criteria selection, indicator specification, and weighting. Further, to extoll the benefits of the responsiveness inherent to the tool, the proposed methodology should not be a one-off exercise. Rather, it should be utilized as a progressive approach, intended to ‘live and grow’ with the country's infrastructure needs and policy objectives. As such, the prioritization program should involve continuous refinement of the decision-support tool, based on informed deliberation regarding criteria selection and any pre-decisions of a policy nature (Mandri-Perrott, Marcelo, & Haddon, 2015). Last, planning offices and decision makers must be familiarized with the multi-criteria approach to build credibility of the decision support tool itself, establish familiarity with its use, and legitimize the results of analysis.

23
# Chapter 4. Applying IPF to Chile’s Infrastructure Project Proposals

This section documents the results of the IPF-based project prioritization for water catchment and road transport projects and compares these results with actual project selection outcomes. The greatest value of the Chile pilot is the opportunity to compare IPF results to project selection informed by CBA to test the analytical demands of IPF inputs, the robustness of outputs, and degree of alignment of IPF results with the outcomes of other approaches. The comparative results show that prioritization outcomes are affected not only by the methodologies in use, but also by the practices and policies of project selection – i.e., how the results of analyses are applied in decision-making.

In this chapter, we first present the IPF mechanics and comparative results of IPF- and MCE-based prioritization for water catchments and follow with IPF construction and comparative results of IPF- and SCBA-based prioritization for road transport projects.

## Water Catchment Projects in Chile

<mark>Water resource management is an important policy priority in Chile with direct impacts on rural and agricultural development and environmental sustainability. Despite an abundance of water resources (overall availability of around 50,000 m</mark><sup>3</sup> <mark>per capita per year), Chile faces water stress due to geographic distribution patterns. Most of Chile’s population lives in arid and semi-arid areas where water availability is low (less than 1,000 m</mark><sup>3</sup> <mark>per capita year) and demand exceeds surface water supply. An increasing need to offset unmet demand by groundwater extraction has led to a significant increase in annual freshwater withdrawals, which has become a key sustainability concern.</mark><sup>12</sup> <mark>Moreover, Chile is projected to move from a level of medium water stress to extremely</mark> high stress in 2040 due to the impacts of climate change.<sup>13</sup> <mark>The erosion and desertification of soils also present a recognized sustainability challenge related to development of the Chilean forestry and agriculture sectors. Deforestation, overgrazing, inadequate crop management, and irrigation practices have resulted in soil degradation affecting nearly half of the territory and 75 percent of productive soils. This increased water stress and soil degradation disproportionally affects the poorest populations, who rely on small-scale agriculture as a critical income source; depend on natural resources for food, fuel, and building materials; and</mark> are typically located in arid rural regions most affected by climate variability and drought.

<mark>As such, there is increased pressure to better manage water resources to ensure water security, meet growing agricultural and industrial demand on water resources, support adequate provision to the poorest communities, and deal with increased water stress associated with the effects of climate change.</mark>

### Water Catchment Project Sample

Projects are typically prioritized within regions (as opposed to the national level) to ensure the disbursement of funding across regions. Since projects are allocated funds region by region, the IPF was also applied at the regional level in four areas: Biobio, Maule, O’Higgins, and Valparaiso.

> 12 Chile’s annual freshwater withdrawals as percentage of internal resources went from 2.3 percent in 1992 to 4.0 percent in 2014.

> 13 Maddocks, Young, & Reig (2015).

24
The regions and samples of proposed projects considered via MCE and IPF are summarized in Table 3.

Table 3. Water Catchment Project Sample

|Region|Number of Projects in|
|---|---|
||IPF Sample|
|Biobio|14|
|Maule|13|
|O’Higgins|22|
|Valparaiso|12|

To compare IPF and MCE prioritization outcomes, projects were first assigned scores and ranked in regional groups, as in practice. They are also presented in ranked order as a pooled (all regions combined) group in the IPF analysis.

# Water Catchment Project Indicators

<mark>The selection of indicators used to construct the SEI reflects some of the key policy goals of the government with respect to developing water catchments. As discussed in Chapter 2, many of the intended benefits associated with improving water resources are strategic, environmental, and social – and these are often difficult to quantify and monetize. Therefore, economic analyses often result in calculated internal rates of return (IRRs) below the 6% threshold required for favorable recommendation. These low IRRs are likely due to undervaluation of some long-term benefits. Recognizing the importance of water resource development, however, the government</mark> implemented the Multi-Criteria Evaluation approach to assess these projects.

<mark>The IPF draws from the MCE’s components to select the sets of input indicators for the SEI and FEI. Data previously gathered for the MCE were simply re-organized to fit the format of the IPF approach, with three important changes. First, the selection of input variables required paring down the extended list of MCE indicators. The resulting IPF indicator set excluded some MCE variables either because (a) they brought redundant information to the analysis or (b) exhibited no variation across projects. Second, some indicators used in IPF drew directly on project data in natural units rather than the ordinal score (e.g., 0, 5, 10) assigned to projects in the MCE approach. Third, some values maintain their ordinal scores, but are transformed so that more positive scores are recorded with higher values (in the MCE, lower scores are attributed to better performance on sub-indicators). The resulting indicators are described in Table 4, and relevant transformations are described in Table 5.</mark> 25
|Table 4: Variables Includ<br>Type<br>Indicator|ed as SEI and FEI Indicator<br>Included Variables|s<br>English|Use / Relevant Calculations|
|---|---|---|---|
|<br>Investment ($m)|Inversión inicial|Initial Investment|Direct from MCE data|
|Other<br>Inputs<br>Predios (u)|Número de predios|# Land Plots|Direct from MCE data|
|<br>Surface (ha)|Superficie equivalente|Surface|Direct from MCE data|
|Poverty|Porcentaje de hogares en<br>pobreza comunal|Poverty|Drew from MCE data, but<br>directly used the percentage<br>value rather than an assigned<br>ordinal score|
|Beneficiaries|# Predios / superficie|# Land Plots / Surface<br>Area|Direct from MCE data|
|SEI<br>|Comuna extrema, fronteriza<br>o rezagada|Underdeveloped<br>Community||
|Jobs|Generación de empleo<br>agrícola|Generation of<br>Agricultural<br>Employment|See Table 5|
||Relocalización de vivienda|Household Relocation||
|Territorial|Afecta bosque nativo|Native forest affectation|See Table 5|
|i_NPV|VAN social / inversión inicial|NPV / Investment|Direct from project data|
|Expropriations|Expropiaciones|Expropriations|See Table 5|
|FEI<br>Endorsement|Interés de beneficiarios|Community<br>Endorsement|See Table 5|
|Legal|Desarrollo organizacional|Legal Standing of<br>Organizations|See Table 5|

26
Table 5. Calculation of Select SEI and FEI Indicators

|Indicator|Calculation / Indexation Rule|
|---|---|
|Jobs|(a) For Comuna Extrema (underdeveloped community 'UC')<br>(i) If UC = Yesassign 1<br>(ii) If UC = Noassign 0<br>(b) Generación de Empleo Agrícola (Agricultural employment 'AE')<br>(i) If AE = Yesassign 1<br>(ii) If AE = Noassign 0<br>(c) Jobs Index = a + b<br> <br>Converted|
||Original MCE Score<br>Underdeveloped Community<br>Agricultural Employment|
||Yes<br>1<br>1<br>1|
||No<br>10<br>0<br>0|
|Territorial|(a) For Relocalizacion de vivienda (household relocation 'HR')<br>(i) If HR = Yesassign 0<br>(ii) If HR = Noassign 1<br>(b) Afecta Bosque Nativo (native forest affected 'FA')<br>(i) If FA = Yesassign 0<br>(ii) If FA = Noassign 1<br>(c) Environmental = a + b<br> <br>Converted|
||Original MCE Score<br>Relocation Required<br>Affects Native Forests|
||Yes<br>1<br>0<br>0|
||No<br>10<br>1<br>1|
||(a) The more expropriations 'EX', the higher the risk to the project.<br>(b) We use the following formula for expropriations:|
|Expropriations|EX Index � ���EX � max�/�EX′s range��<br>Therefore, the larger the number of expropriation in original data, the smaller value taken in the<br>expropriations index.|
|Interest<br>|(a) This stands for interest of beneficiaries 'IB' in the project|
|(community<br>endorsement of|<br>(i) If IB = YES (the community endorse the project)Assign 1<br>|
|<br>|(ii) If IB = No (the community doesn't endorse the project)assign 0|
|project)||
||(a) This variable stands for the soundness of legal standing of organizations<br>|
||(i) If original value = 1, legally constituted organization (best)assign 3<br>(ii) If original value = 5, "de facto" constituted organizationassign 2|
||(iii)If original value = 10,non‐constituted/non‐existingorganizationassign 1.|
||Legal Standingof Organizations|
|Legal|(in English)<br>Original<br>Converted|
||Oiióliddídi<br>Llltittditi<br>1<br>3|
||rganzacn conpersonaajurca<br>egayconsue organzaon<br> <br>|
||Organización de hecho constituida<br>"De Facto" constituted<br>5<br>2|
||Organización de hecho no constituida<br>"De Facto" non‐constituted<br>10<br>1|
||Inexistencia de organización<br>Non‐existingorganization<br>10<br>1|

# Water Catchment Project Indicator Weighting

While projects were ranked region by region, the weights associated with criteria were calculated via PCA based on the full (combined) sample of projects. This decision was made due to the low degree of variation of many variables within each region. In other words, criteria weights were 27
SEI = w, X Poverty + wz X Beneficiary + w3 X Jobs + w, X Territorial FEI =ws X IVAN + We X Expropriations + w7 xX Endorsement + wg X Legal, 18 18 SEI = .316 xX Poverty + .642 x Beneficiaries + .400 x Jobs + .573 x Territorial (standardized) FEI = .316 x IVAN + .316 x Expropriations + .635 x Endorsement + .630 x Legal (standardized)
# IPF Results: Water Catchment

After calculating the SEI and FEI scores for each catchment project according to the weighted additive models above, the projects are ranked by each score, by region. The calculations for SEI and FEI scores are detailed in Annexes 3 and 4, while the graphical results of rankings by each index are summarized in Figures 5, 6, 7, and 8. Figures 5 (SEI) and 7 (FEI) display results region by region, whereas Figures 6 (SEI) and 8 (FEI) display the full set of projects pooled into a single ranked group according to each index.

In each ranking chart, the projects that were actually approved for funding by the government are marked with a checkmark above the project’s score. Generally speaking, irrigation projects with higher SEI and FEI scores tended to be those approved for funding, though some projects with outlying low SEI scores were among the approved set.

## Water Catchment SEI Results

Project rankings by SEI for each region are detailed in Figure 5. In each ranking chart, the projects actually selected for funding are marked with a checkmark.

Figure 5. Water Catchment Project SEI Scores and Ranking, by Region <!-- Start of picture text --> Biobio SEI Maule SEI<br>100 100<br>90 90 ✔ ✔<br>80 80<br>70 70 ✔ ✔<br>60 60 ✔<br>50 50<br>✔ ✔<br>40 40<br>30 ✔ ✔ 30<br>20 20<br>10 10<br>0 0<br>Project ID Project ID<br>O'Higgins SEI ✔✔ Valparaiso SEI<br>100 100<br>90 90<br>8070 ✔✔ ✔ 8070<br>60 60<br>50 50 ✔ ✔✔<br>40 40<br>30 30 ✔<br>20<br>20<br>10<br>10 ✔<br>0<br>0<br>Project ID<br>Project ID<br>5 3 4 2 10 9 7 11 13 12 8 1 6 14 20 22 15 24 21 17 16 26 23 25 27 19 18<br>36 46 35 29 30 31 37 41 42 43 32 33 34 28 47 48 39 40 38 44 45 49 56 58 57 51 50 55 54 60 61 52 53 59<br><!-- End of picture text --> ✔ - Project approved for funding by GoC Source: Authors’ calculations 29
# Pooled SEI Results

Figure 6. Water Catchment Project SEI Scores and Ranking, Pooled Regions <!-- Start of picture text --> Pooled Region SEI<br>100<br>90<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Project ID<br>Biobio 14 projects O’Higgins 22 projects<br>Maule 13 projects Valparaiso 12 projects<br>Source: Authors’ calculations<br>20 22 23 5 56 36 46 58 57 15 16 47 48 26 51 24 21 4 44 45 49 50 27 2 10 9 7 11 13 35 17 19 55 3 54 25 29 30 31 37 41 42 43 39 40 60 12 8 61 32 33 34 28 18 1 52 53 38 6 14 59<br><!-- End of picture text --> 30
# Water Catchment FEI

Figure 7. Water Catchment Project FEI Scores and Ranking, by Region <!-- Start of picture text --> Biobio FEI ✔ Maule FEI<br>100 100 ✔ ✔<br>90 90 ✔<br>80 ✔ 80 ✔<br>70 70<br>✔<br>60 60<br>50 50<br>40 ✔✔ 40<br>30 30<br>20 20<br>10 10<br>0 0<br>Project ID Project ID<br>O'Higgins FEI<br>✔✔✔ ✔<br>100<br>90<br>80 ✔<br>70<br>✔<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Project ID<br>Valparaiso FEI<br>✔✔<br>100<br>90<br>✔ ✔<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Project ID<br>5 7 13 4 14 12 11 1 9 2 10 6 8 3 24 20 21 18 16 15 22 25 27 23 17 19 26<br>35 33 36 32 34 42 43 44 31 29 30 39 40 41 37 38 48 49 46 45 47 28<br>50 59 51 56 53 52 57 55 61 58 54 60<br><!-- End of picture text --> ✔ - Project approved for funding by GoC Source: Authors’ calculations 31
Figure 8. Water Catchment Project FEI Scores and Ranking, Pooled Regions <!-- Start of picture text --> Financial and Economic Indicator (FEI)<br>100<br>90<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Project ID<br>Biobio 14 projects O’Higgins 22 projects<br>Maule 13 projects Valparaiso 12 projects<br>5 50 59 35 33 36 32 34 51 56 53 52 24 7 44 14 13 4 21 12 11 20 1 9 2 10 6 18 16 15 42 43 22 25 31 29 30 57 55 8 39 40 41 3 27 23 37 38 48 49 17 61 58 46 45 47 54 28 26 19 60<br><!-- End of picture text --> Source: Authors’ calculations 32
# Water Catchment IPF Matrix

Figure 9 shows the results of plotting projects (by their SEI and FEI scores as x and y coordinates) onto a water catchment prioritization matrix, region by region. Yellow project points denote projects that were actually approved for funding. Interestingly, these results suggest different patterns with respect to funding preferences. In Maule and Valparaiso, for example, approved projects are those that had the highest FEIs, whereas project selection in O’Higgins and Biobio does not directly reflect high FEI or SEI.

Figure 9. Water Catchment IPF Matrices, by Region <!-- Start of picture text --> Biobio Maule<br>100 P14 100 P18<br>90 90<br>80 80 P19<br>P25 P27<br>70 70<br>P23<br>60 P6 60 P16 P26<br>50 50<br>P17<br>40 P1 40 P24 P21P15<br>P8<br>30 P12 30<br>P13<br>20 P7 P4 P11P10P2 P3 20 P20 P22<br>10 10<br>P5 P9<br>0 0<br>FEI FEI<br>O'Higgins Valparaiso<br>P49<br>100 P44 P45 100 P59<br>90 P38 90<br>80 P40 P39 P48 80<br>P47<br>70 P32 P42 P30 70<br>P33 P41 P28<br>60 60 P53<br>P34 P37<br>50 P43 P29 P31 50 P52<br>40 40 P61 P60<br>30 P35 30 P50 P55 P54<br>P51<br>20 20 P57<br>P56 P58<br>10 10<br>P36 P46<br>0 0<br>FEI FEI<br>SEI SEI<br>0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100<br>SEI SEI<br>0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100<br><!-- End of picture text --> Source: Authors’ calculations 33
Comparing IPF to Selection of Water Catchment Projects Table 8 offers more detail on the comparative outcomes of IPF versus MCE ranking in each region. The MCE-ranked projects marked with an asterisk are those that were not recommended for implementation due to location in an area where environmental restrictions were in place. Highlighted projects are those that were recommended for funding.

Table 8. Comparing IPF Maps and MCE Project Rankings, by region

|||Biobio||||
|---|---|---|---|---|---|
||IPF Ma|pping||MCE Ranki|ng|
||Biob|io|ID|Name|Score|
||||1|Kaiser|3.0|
|100|P14||9|Quidico 1|3.3|
|90|||3|Ranquil|3.5|
|80|||8|Mirihue|3.5|
|70|||14|Leoneras|3.5|
|60<br>EI|P6||2|Las Puentes|4.1|
|50<br>S|||10|Quidico 2|4.3|
|40|P1|P8|6|Pichi Bureo*|4.4|
|30|P13<br>P12||5|Laguna El Pillo*|4.6|
|20|P4<br>P2<br>P10<br>P7<br>P11|P3|12|Vegas de Itata|4.6|
|10|P5<br>P9||7|Rumena|5.0|
|0<br>0|10<br>20<br>30<br>40|50<br>60<br>70<br>80<br>90<br>0|13|Tauco|5.3|
|||10<br>FEI|4|Tranaquepe*|5.4|
||||11|Perales|5.9|
|||Maule||||
||IPF Ma|pping||MCE Ranki|ng|
||Mau|le|ID|Name|Score|
||||19|La Bruja|2.38|
|100|P18||17|Peralito*|3.12|
|90|||18|Peralito 2|3.17|
|80|P2|P27<br>P19|25|El Molino|3.22|
|70|5|P23<br>|26|Derivado Porvenir 2|<br>3.31|
|60<br>EI|P16|P26<br>|21|Sauzal|4.50|
|50<br>S|P24<br>P21|P17|24|Manantiales|5.01|
|40|P15<br>||27|Limávida|5.11|
|30|||23|Vaquería|5.45|
|20|P20<br>P22||20|El Guindo|5.55|
|10|||22|Botacura|6.20|
|0<br>0|10<br>20<br>30<br>40|50<br>60<br>70<br>80<br>90<br>100|15|Huencuecho I|6.65|
|||FEI|16|Huencuecho II|6.65|

* Not recommended due to environmental restriction

34
O’Higgins <!-- Start of picture text --> IPF Mapping MCE Ranking<br>ID Name Score<br>O'Higgins<br>29 San Francisco 1 (Zonada) 6.2<br>P49<br>100 P44 P45 30 San Francisco 2 (Zonada) 6.2<br>90 P38 31 San Francisco 3 (Zonada) 6.2<br>80 P40 P39 28 Codegua (CFGD) 6.4<br>P48<br>70 P32 P42 P30 P47 37 Manquehue 1 (Zonada) 6.5<br>P33 P41 P28 38 Manquehue 2 (Zonada) 6.5<br>60<br>50 P34 P43 P29 P31 P37 41 Estero Seco (CFGD) 6.7<br>36 Ucúquer (CFRD) 7.6<br>40<br>32 El Maiten 1 (Zonada)* 7.8<br>30 P35<br>33 El Maiten 2 (Zonada)* 7.8<br>20<br>34 El Maiten 3 (Zonada)* 7.8<br>10<br>P36 P46 35 Huehuinco (CFRD)* 7.8<br>0<br>40 Las Palmas 2 (RCC) 8.2<br>42 Cementerio 1 (Hormigón) -<br>FEI<br>43 Cementerio 2 (Hormigón) -<br>Valparaiso<br>IPF Mapping MCE Ranking<br>ID Name Score<br>Valparaiso<br>55 Pullally 3.04<br>100 P59<br>54 Santa Marta 3.26<br>90<br>80 60 Santa Julia 4.48<br>70<br>59 Valle Hermoso* 4.85<br>60 P53<br>56 Cuncumen 1 5.11<br>50 P52<br>40 P61 P60 58 El Zaino 2* 5.18<br>30 P50 P55 P54 52 Chalaco* 5.60<br>P51<br>20 P56 P57 P58 57 Lo Zárate 2 5.62<br>10<br>53 Las Carditas 2* 6.18<br>0<br>50 Vitahue* 6.31<br>FEI<br>51 Paihuen* 6.50<br>SEI<br>0 10 20 30 40 50 60 70 80 90 100<br>SEI<br>0 10 20 30 40 50 60 70 80 90 100<br><!-- End of picture text -->

- Not recommended due to environmental restriction

Note: Some projects that entered the database later do not have a corresponding MCE ranking, such as Project 39, 44-49 in O’ Higgins and Project 61 in Valparaiso 35
# Applying IPF to Road Transport

Regional proposals for transport projects are initiated with the submission of draft regional investment pre-project proposals (pre-ARI), which are developed at the regional level with the participation of representatives of the Ministerio de Obras Públicas (Ministry of Public Works) (MOP) subnational offices, Secretarías Regionales Ministeriales (SEREMI). These initial proposals are considered jointly by regional and national planners. SEREMI sends several regional projects on for consideration to MDS to follow the process of ongoing project appraisal and RATE. While the national government may decide to fund projects with a RATE other than RS, regional projects require an RS designation.<sup>14</sup> The MOP determines which of the projects that received an MDS favorable recommendation (RS) will be sent further for funding to the Ministry of Finance. In the MOP's prioritization methodology, projects are classified as medium, high or low priority based on considerations of strategic relevance, completeness of information, and alignment with regional development plans. Projects selected by the MOP are sent to the Ministry of Finance for review. On approval, a decree is issued by the Ministry of Finance to commit financial resources and initiate project execution. To modify the list of projects, MOP must send its revised prioritized list to the Ministry of Finance to be approved and included in a new decree.

## Transport Policy Goals and Road Project Criteria

The processes of transport project prioritization described above hinge on two primary inputs: project cost-benefit analysis and qualitative assessment of project alignment with medium- and long-term regional and national transport development plans. The former – CBA – is based on a sophisticated approach to assessing road project lifecycle costs, namely the Highway Development and Management (HDM-III/4) tool.

The latter involves consideration of Chile’s transport policy goals. Road density in Chile lags behind other Latin America and OECD countries, partly due to low overall population density but high concentration in Santiago and Valparaiso. The road network’s size (approximately 78,000 kms in 2010) has not significantly changed since 1990, though the share of paved roads has increased from 13.8 to 23.3 percent. This pilot application of IPF considers inter-urban road projects intended to extend the road network and link urban areas. Specifically, the pilot was applied to 50 projects in the BIP that met basic requirements (mentioned below) to be included in the IPF pilot, including HDM-3 data and the existence of a CBA.<sup>15</sup> Road repositioning projects were excluded from the pilot.

## Road Transport Project Sample

The sample of 50 interurban road transport projects to which IPF was applied includes only a set that has reached Execution status in the SNI. Additionally, this set has passed the basic profitability test of having IRRs of at least 6% and have been judged to align with transport policy goals.

> 14 The Organic Law on Financial Administration of the State, Article 19, requires only that a project’s profitability be analyzed, but does not require project profitability. The Organic Constitutional Law on Government and Regional Administration (19175), Article 75 states that projects financed with regional funds require an RS RATE.

> 15 Special classifications exempt some projects from requiring CBA (e.g., projects in extreme zones, projects with expected volumes of less than 400 vehicles per day).

36
# Transport Project Indicators

SEI indicators were selected to consider beneficiaries, job creation, extension of service to areas with higher poverty levels, and savings associated with reduced gas consumption.<sup>16</sup> Each of these input indicators is controlled for project size, yielding the following criteria inputs: beneficiaries per dollar invested (Ben_inv), jobs per dollar invested (Jobs_inv), and gas savings per dollar invested (Savings_inv). Beneficiaries is calculated by estimating traffic volumes, considering the occupancy rates for different types of vehicles. This information is taken directly from project proposal data included in the BIP. Data on the creation of jobs directly associated with construction and operation are also calculated based on BIP data, which were in turn derived from an estimation model based on the macroeconomic input-output matrix. Gas savings is an estimate of the annual savings in gas, measured in liters. The initial set of indicators also considered the extension of service to indigenous populations, but this criterion was later dropped due to limited variation.

Lastly, the SEI includes the poverty rate and tests the use of two alternative specifications to measure poverty. One takes a simple poverty rate (Pov_c) reflecting the percentage of people in the service area classified as “income poor”. It is important to note that it is not necessary that the users of the road are the same as the population that lives nearby the road that is under appraisal. It is possible that the locals could have a low vehicle per capita ratio, and the usage of the road is composed by transportation trucks, tourists, drive through or long-haul trips, etc. The other applies a multi-dimensional composite poverty index (MPI-CL) developed by MDS (Pov_multi) that accounts for five poverty dimensions: education, health, employment and social security, housing, and networks and social cohesion.

The FEI takes into account only one measure – the net present value index (IVAN) – derived from project CBAs. The IVAN is equivalent to a ratio of the net present value of future project costs and benefits to the project’s initial investment cost.

The formulas for SEI and FEI are as follows: SEI�w� �Ben_inv�w� �Jobs_inv�w� �Savings_ben�w� �Pov_c/multi �standardised� FEI� IVAN �standardized� where w�, ..., w� are the weights associated with each SEI criterion. Input project data are detailed in Annex 5.

## Transport Project Indicator Weighting

The criteria weighting of SEI variables was done using PCA, with the two specifications for measuring poverty. Table 9 gives the weights associated with each variable using each approach.

Table 9. Criteria Weighting, Transport SEI

|Simple poverty rate|Ben_inv<br>|Jobs_inv<br>|Savings_inv<br>|Pov_c<br>|
|---|---|---|---|---|
||0.614|0.224|0.724|0.224|
|Multi‐dimensional|Ben_inv|Jobs_inv|Savings_inv|Pov_multi|
|poverty index|0.442|0.224|0.839|0.224|

Note: The weights have been determined by PCA with the restriction that all weights must be positive and have a minimum contribution of 10%

> 16 In economic ($) terms, this would likely be considered in the FEI; however, here, our interest is to reduce consumption for environmental reasons. Therefore, the savings is measured in liters and included in the SEI.

37
# IPF Results: Road Transport

As with water catchment projects, transport projects are ranked by SEI and FEI scores separately. The calculations for SEI and FEI scores are detailed in Annexes 6, 7, and 8, while the graphical results by each index are summarized in Figures 10, 11 and 12. Figures 10 and 11 show the SEI ranking based on the simple poverty indicator and the multi-criteria poverty index, respectively.

## Road Transport SEI and FEI Rankings

Figure 10. Road Transport SEI Ranking Results from Input and Weighting with Pov_c <!-- Start of picture text --> 100<br>90 Social and Environmental Indicator (SEI) - Pov_c<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Figure 11. Road Transport SEI Ranking Results from Input and Weighting with Pov_multi<br>100<br>90 Social and Environmental Indicator (SEI) - Pov_multi<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>Figure 12. Road Transport Project FEI Ranking Results<br>100 Financial and Economical Indicator (FEI)<br>90<br>80<br>70<br>60<br>50<br>40<br>30<br>20<br>10<br>0<br>P31 P12 P10 P26 P34 P4 P46 P19 P23 P13 P39 P28 P14 P17 P44 P5 P21 P25 P18 P29 P8 P16 P30 P6 P9 P43 P22 P20 P35 P49 P11 P32 P42 P48 P3 P33 P2 P41 P47 P1 P50 P15 P38 P36 P27 P37 P40 P45 P7 P24<br>P31 P19 P12 P10 P34 P26 P5 P18 P14 P46 P28 P39 P23 P44 P17 P4 P25 P21 P1 P13 P20 P11 P35 P6 P30 P22 P9 P48 P29 P49 P50 P3 P42 P2 P8 P16 P41 P32 P33 P47 P43 P15 P27 P40 P37 P36 P38 P45 P7 P24<br>P21 P46 P42 P8 P14 P3 P39 P26 P27 P5 P25 P16 P10 P22 P31 P17 P12 P50 P18 P48 P7 P23 P49 P44 P19 P9 P32 P45 P40 P35 P2 P6 P4 P33 P28 P15 P34 P41 P30 P47 P11 P29 P43 P13 P20 P37 P24 P1 P38 P36<br><!-- End of picture text --> Source: Authors’ calculations 38
# Road Transport IPF Matrix

Figure 13 shows projects by their SEI and FEI scores. Projects are color coded according to the duration between a project reaching Execution status in the SNI and the allocation of project execution funding. This indication of urgency with respect to funding is taken as a loose proxy of project prioritization. Projects coded in yellow were allocated funds within a year of reaching Execution status, and projects coded in orange were funded between one and two years of reaching Execution status. Green projects were either funded two or more years after reaching Execution status or are still awaiting funding, indicating lowest priority.

Figure 13. Road Transport Project Prioritization Matrix <!-- Start of picture text --> 100<br>P24<br>90<br>80 P7<br>P45<br>70<br>60 P40<br>P27 P37<br>50 P38 P36<br>P15<br>40<br>P2<br>P3 P32<br>P42P16P25 30 P22 P50 P9P35 P47 P43 P1<br>P8 P49 P33 P41 P11<br>P5 20 P48 P20<br>P6 P30<br>P21 P18 P29<br>P14 P44 P28<br>P39P46 10 P17 P23 P19 P4 P34 P13<br>P10P26 0 P31 P12<br>0 20 40 FEI 60 80 100<br>SEI<br><!-- End of picture text --> <mark>Funded <1 year ⏺ ; Funded within 1-2 years ⏺ ; Funded in 2+ years or not yet funded ⏺</mark>

# Comparing IPF to Funding of Transport Projects

Figure 13 demonstrates no clear pattern with respect to the link between project prioritization and the outcomes of CBA analysis. If prioritization of projects that met the basic requirements were decided according to highest profitability, this would have resulted in high-FEI projects being funded first and, therefore, a concentration of yellow- and orange-coded projects towards the upper limits of the x (FEI) axis. Nor does the figure demonstrate prioritization of projects’ funding order by joint consideration of CBA with other social and environmental goals. Prioritization based on CBA plus alignment with social and environmental goals would have resulted in a concentration of yellow and orange projects in the northeast quadrant of the matrix.

Interestingly, three projects –24, 36, and 38– with the highest SEI and FEI scores were not allocated funds for implementation until 2, 9, and 9 years, respectively, after having reached Execution status in the SNI.

These results suggest that, while CBA is an important filtering mechanism for road transport, it is not necessarily the basis of prioritizing investments for projects that pass the profitability test. Moreover, while the government does engage in consultation to discuss the social, environmental, and strategic implications of various projects to inform selection, much of this discussion and consultation is informal and largely unstructured.

39
# Chapter 5. Conclusion

The pilot of the IPF in Chile produced valuable overarching findings beyond the outcomes of the actual prioritization exercise. First, the outputs of CBA were found to be useful inputs to the FEI – inputs that can also be effectively complemented by the simultaneous and direct consideration of social and environmental policy goals that are otherwise difficult to quantify and value. The extensive economic appraisals (partial SCBAs) in place are useful to generate important economic measures associated with projects, including IRRs and NPV index (IVAN) scores, which should be used as key inputs to project prioritization and, in the case of IPF, inputs to the FEI. In this case, the IPF complements rather than substitutes traditional CBA appraisal.

The Chile pilot has shown that IPF has application beyond the 'stepping stone' approach previously proposed (Marcelo, et al, 2016), which couched IPF as a stop-gap measure until more sophisticated project appraisal methods could be implemented. Such CBA outputs as benefit-cost ratios, internal rates of return, and net present value indexes can be used to construct the FEI – either as the sole input or in combination with other relevant financial and economic factors, such as risk. As such, the IPF has relevance to a wide array of government capacity levels. It can serve to systematize consideration of key factors, where only limited project information is available, or complement full economic appraisals with additional policy considerations.

Moreover, CBA analyses are not necessarily used as the basis of prioritization. In Chile, for example, CBA analysis is used as a filter to eliminate projects with IRRs below a profitability threshold (6%). Once projects pass this filter and are given a positive endorsement for development (RS) within the SNI, they may be prioritized in a number of ways (often unspecified) by the proponent agency. The reservoir MCE directly responds to the inability of the CBA to fully account for critical yet undervalued strategic issues, which resulted in poor CBA results for projects that were required to ensure water security. So then, even where CBA is extensively applied and used to filter out ‘bad’ projects, there is a demand for systematic consideration of other important policy factors in the actual prioritization of investments.

Also, in the last couple of years, some road projects that were traditionally appraised using the CBA methodology have been appraised instead by the CEA approach, due to political pressures and considerations, because for those low traffic flows were not allowing the project to satisfy the positive NVP criteria. These kinds of projects were not considered in the prioritization exercise, as the main economic efficiency index (IVAN) values were not possible to get. This can be seen as a signal of how economic efficiency is losing influence in the investment and prioritization decision by the authorities, as the objectives are politically driven.

IPF recognizes that, in addition to economic considerations, projects may be valued by governments and other stakeholders due to non-economic considerations such as reducing income disparity or territorial inequality, promoting social cohesion, safeguarding the environment, preserving culture, or managing disaster and climate risks. CBA analyses are often unable to capture such developmental policy goals, which are inherently difficult to valuate. While SCBA should certainly be maintained as the gold standard in project appraisal and a key input for decision making, IPF can help prioritize projects under a variety of analytical conditions and according to multiple policy goals.

40
Recognizing the importance of these additional facets to decision making related to infrastructure development, there is an expressed demand to shift away from purely efficiency-based approaches to approaches that can consider strategic and social goals, that are often undervalued in traditional CBA-like assessments. For example, the Governments of the United Kingdom, Australia, and Chile, and many U.S. state governments have published notes and guidance on the application of multi-criteria decision analysis (MCDA), expanding the discourse to suggest structured ways of employing MCDA to incorporate key policy criteria. Some countries, such as Ireland, have imposed thresholds to guide when government should apply SCBA, multi-criteria analysis, or more simple assessments, depending on the size of the proposed investment.

In relation to the PIM framework, the IPF tool is meant to be one possible option in a basket of analytical tools that can be used to gather the best appraisal data available and organize them to support prioritization. In Chile, where multi-criteria approaches to project selection have already been developed and institutionalized, this benefit is clearly recognized.

41