EUROPEAN COMMISSION Brussels, 21.2.2020 SWD(2020) 43 final COMMISSION STAFF WORKING DOCUMENT



EVALUATION Ex post evaluation of major projects in environment financed by the European Regional Development Fund and the Cohesion Fund between 2000 and 2013 {SWD(2020) 44 final}



Table of Contents 1.Introduction Large‑scale environmental projects Purpose and scope 2.Background to the intervention Goals and intervention logic of cohesion policy Shared management — a key feature of cohesion policy Large infrastructure projects: technical aspects Environmental infrastructures 3.Implementation / state of play Description of the current situation Short description of the methodology Review and selection of the case studies Limitations and specificity of the methodology applied 4.Analysis 4.1 Effectiveness 4.1.1 Project objectives were generally well defined 4.1.2 Projects were effective, but only few fully achieved their objectives 4.1.3 Benefits are maximised by accompanying/synergic investments and adequate response to individuals’ needs

4.1.4 Environmental projects generate long-term effects through a variety of mechanisms 4.1.5 Forecasting future demand is the main and most problematic determinant of project performance 4.2 Efficiency 4.2.1 Most projects were efficient, delivering social benefits that exceeded the costs 4.2.2 Most projects were not as efficient as expected: cost overruns and time delays affected their performance 4.3 Relevance 4.3.1 The projects contributed to integrate sustainability concerns in programme design and implementation 4.3.2 All projects addressed real and significant environmental needs 4.3.3 Projects address not only environmental, but also economic needs 4.4.4 Projects are aligned with EU and national priorities 4.5 Coherence 4.5.1 Projects were coherent with the EU environmental framework 4.5.2 Project components were generally consistent with the project objectives 4.5.3 Project components were well placed in the wider policy context 4.6 EU added value 4.6.1 EU financial support 4.6.2 EU strategic support 4.6.3 EU Triggering transboundary effects and fostering knowledge sharing 4.6.4 EU technical support and institutional capacity‑building

5. Conclusions Annex 1: Procedural information Annex 2: Stakeholder consultation Annex 3: Methods and analytical models Annex 4: Case studies Glossary LIST OF ABBREVIATIONS B/C ratio Benefit/cost ratio CBA Cost-benefit analysis BWD Bathing Water Directive CF Cohesion Fund CPR Common Provisions Regulation DG REGIO Directorate-General for Regional and Urban Policy DWD Drinking Water Directive EC European Commission EIA Environmental Impact Assessment EIB European Investment Bank ENPV Economic net present value ERDF European Regional Development Fund ESIFs European Structural and Investment Funds ERR Economic rate of return EU European Union FNPV/C Financial net present value of the investment FNPV/K Financial net present value of national capital FRR/C Financial rate of return on investment GDP Gross domestic product GHG Greenhouse gas GNI Gross national income ISPA Structural Pre-Accession Instrument JASPERS Joint Assistance to Support Projects in European Regions MFF Multiannual financial framework NPV Net present value NUTS Nomenclature of Territorial Units for Statistics O&M Operating and maintenance P&R Protection and remediation SCF Standard conversion factor SDR Social discount rate UWWTD Urban Wastewater Treatment Directive WFD Water Framework Directive 1.Introduction Cohesion policy is the EU’s key investment tool. Enshrined in Articles 174-178 of the Treaty on the Functioning of the European Union (TFEU), it aims to strengthen economic, social and territorial cohesion by reducing disparities in the level of development between regions. Cohesion policy relies notably on three Funds: -the European Regional Development Fund (ERDF); -the Cohesion Fund (CF); -the European Social Fund (ESF). Environmental infrastructure projects are financed by the ERDF and the Cohesion Fund, as follows: Øthe ERDF invests in growth-enhancing sectors to foster competitiveness and create jobs in EU regions and cities. ERDF actions are designed to address territorial, economic, environmental and social challenges, with a focus on sustainable urban development; and Øthe Cohesion Fund invests in environment and transport networks in Member States with a Gross national income per inhabitant below 90% of the EU average (‘cohesion countries’). It aims to reduce economic and social disparities and to promote sustainable development. In 1997, the Treaty of Amsterdam introduced a requirement to integrate environmental protection provisions into all the Union’s policies and activities. Following the entry into force of the Treaty of Lisbon (1 December 2009), this requirement was incorporated in the TFEU, Article 11 which provides that: ‘Environmental protection requirements must be integrated into the definition and implementation of the Union’s policies and activities, in particular with a view to promoting sustainable development.’

As the EU’s main investment tool, cohesion policy contributes to and acts in synergy with several EU sectoral policies. The European Structural and Investment Funds (ESIFs) strongly support infrastructure projects that contribute to compliance with the environmental acquis communautaire and its broader sustainable development goals. In return, by reducing economic, social and territorial disparities between Member States and regions, and by supporting growth and competitiveness, environmental infrastructures contribute to the achievement of the core goals of cohesion policy. Cohesion policy is implemented on the basis of programming periods that generally last seven years in parallel with the EU’s multiannual financial framework (MFF). This report covers the 2000-2006 and 2007-2013 periods. Large‑scale environmental projects Large-scale environmental infrastructure projects often involve significant investments and can account for a substantial proportion of the resources allocated to operational programmes (OPs). In both programming periods, projects for an eligible cost of over €50 million were subject to specific rules for ‘major’ projects, including an assessment procedure and specific approval from the European Commission. In 2000‑2006, cohesion policy support was granted to 844 major projects in the environmental sector. Of 945 major projects in 2007‑2013, 234 were environmental in nature. Purpose and scope The Financial Regulation requires the Commission to carry out ex post evaluations of all programmes and activities that entail significant spending, in order to improve future decision‑making. This requirement is explicitly reflected in the general regulations for the ERDF/Cohesion Fund programmes for each programming period. Ex post evaluations for the periods in question were finalised in 2010 and 2016. While both covered environmental projects, neither focused specifically on major environmental infrastructure projects. Given the long‑term nature of such projects and the fact that their effects take longer to materialise, it was in fact too early to evaluate them. This staff working document aims to fill that knowledge gap. It presents an ex post evaluation of major environmental projects implemented in 2000‑2006 and 2007‑2013 with ERDF and Cohesion Fund co‑financing. A similar document assessing the impact of large infrastructure projects in the transport sector has just been adopted. This document follows the same logic and methodology, adjusted to the specificities of the environmental sector. As regards the scope of the assessment, 10 major projects from the two programming periods were selected. In order to be able to capture long‑term effects as well as possible, only projects that had been in operation for at least five years were considered for selection. The selected projects are not meant to be statistically representative. The choice of projects followed an analysis of all major projects undertaken in both periods, taking account of data availability, available project documentation, interviews with managing authorities and web/desk research. The aim was to select illustrative examples that could provide interesting and generally valid insights into the long‑term effects of large‑scale environmental infrastructure projects. In this respect, the analysis provides an opportunity to draw lessons for future ERDF and Cohesion Fund support for such projects. Overall, the 10 cases represent more than € 689.73 million of total (national plus cohesion policy) investments, out of which the ERDF and the Cohesion Fund accounted for € 291.79 million of support. Overall, the list of selected projects is balanced across: -sectors (drinking water, wastewater and waste treatment, remediation); -countries (Bulgaria, Croatia, France, Italy, Malta, Poland, Romania, Slovenia, Slovakia and Spain) – despite a prevalence of projects from EU‑13 countries, a good coverage of EU‑15 countries is also ensured; and -programming periods (five projects for each period). Table 1 gives an overview of the case studies. They are presented in more detail in Annex 4. PROJECT MS SUB-SECTOR FUND PERIOD Sofia integrated water project BG Water and wastewater Cohesion Fund (ISPA) 2000-2006 Malta South sewage treatment plant MT Wastewater Cohesion Fund 2007-2013 Sochaczew sewage management PL Wastewater Cohesion Fund (ISPA) 2007-2013 Craiova sewerage network RO Water and wastewater Cohesion Fund (ISPA) 2000-2006 Water supply and sewerage system HR Water and wastewater Cohesion Fund (ISPA) 2007-2013 Favara di Burgio aqueduct IT Water ERDF 2000-2006 Aguilas desalination plant ES Water ERDF 2000-2006 Purchase of a multifunctional ship EE Waste management / risk reduction ERDF 2007-2013 Celje waste management centre SI Waste management Cohesion Fund 2000-2006 Sète-Marseillan lido protection FR Protection and remediation ERDF 2007-2013 2.Background to the intervention

Goals and intervention logic of cohesion policy The objectives of cohesion policy are enshrined in the Treaty. Article 174 of the Treaty on the functioning of the EU stipulates: ‘[i]n order to promote its overall harmonious development, the Union shall develop and pursue actions leading to the strengthening of its economic, social and territorial cohesion. The Union shall also aim at reducing disparities between the levels of development of the various regions and the backwardness of the least favoured regions’. Source: SWD (2016)318 final. Ex post evaluation of the ERDF and Cohesion Fund 2007-2013 The objectives of cohesion policy can be ordered on three levels (see Figure 1): -General – achieving economic, social and territorial cohesion. Disparities in levels of development can be seen, inter alia, in terms of: oeconomic goals (e.g. innovation, entrepreneurship); osocial goals (e.g. inclusion, health); and oterritorial goals (e.g. access to quality environmental services); -Strategic – achieving smart, sustainable and inclusive goals, as defined in the Lisbon Strategy and Europe 2020. These are not just a link between economic, social and territorial cohesion on the one hand and individual investment objectives on the other, but also a link to the Union’s overriding priorities and goals; and -Operational – individual policy themes, which contribute to cohesion by: oreducing social, economic and territorial disparities (see above); and ostrengthening social, economic and territorial cohesion overall. Shared management — a key feature of cohesion policy The ERDF and Cohesion Fund are delivered under shared management. Programmes are not run directly by the Commission; instead they are implemented in partnership with the Member States. The principles and priorities of cohesion policy are distilled through a process of discussion between the Commission and Member States. However, the day-to-day management of the policy, including selection of investment projects, is carried out by managing authorities (a national ministry, regional authority or local council) appointed by the Member States. Large infrastructure projects: technical aspects The 2000-2006 Regulation specified that major projects are those ‘which comprise an economically indivisible series of works fulfilling a precise technical function and which have clearly identified aims and whose total cost taken into account in determining the contribution of the Funds exceeds EUR 50 million’. In the 2007-2013 period, major projects were defined along the same lines. In this period, the threshold for environmental major projects was initially set at €25 million, before being aligned to €50 million which was the threshold in all other fields. During the 2000-2006 and 2007‑2013 periods, major projects had to undergo specific approval procedures. The Commission had to appraise and approve major projects that were co-financed within the operational programmes. When submitting a major project for approval to the Commission, the competent authority presented a cost-benefit analysis of the project including: (i) an assessment of financial costs and benefits; (ii) a risk assessment; (iii) information on the economic viability of the project; (iv) an assessment of the feasibility of obtaining full or partial private financing for the project; and (v) an indication of how far the Funds’ contribution would influence whether the projects would be implemented. In the case of investment in major projects, an analysis of the project’s social costs and benefits was indispensable, including an indication of the foreseeable impact on the development or conversion of the region concerned, and of the application of EU rules on public contracting. With this information, the Commission then assessed the project and took its decision on the basis of the following factors:

·the type of investment planned and, where applicable, the revenue expected; ·the results of the cost-benefit analysis, where the project had to have an overall positive marginal equity (i.e. society would be better off with the project than without the project); ·the result of the evaluation of the impact on the environment; ·consistency with the priorities in the corresponding assistance; ·compliance with other EU policies; ·a breakdown of the main sources of the expected economic and social benefits, particularly in terms of employment, value of time saved and value of accidents saved, having regard to the financial resources deployed; ·the coordination of the financial instruments and the combination of assistance and loans. In 2007-2013, requirements were similar. Member States were requested to submit: ·an analysis of the forecast impact on the sector concerned; ·an analysis of the forecast impact on the socio-economic situation in the Member State and/or the region concerned and, where possible and appropriate, in other relevant regions. In 2005, facing the challenge of preparing and approving large infrastructure projects from the Member States that joined the EU in 2004 (the ‘EU-10’), the European Commission joined forces with the European Investment Bank (EIB) in a new initiative known as ‘Joint Assistance to Support Projects in European Regions’ (JASPERS). The aim was to provide Member States with independent advice to help them prepare quality proposals for large investment projects for funding through the EU’s Cohesion Fund and ERDF. The initiative was financed through the Commission’s technical assistance budget. Since 2006 JASPERS has assisted more than 500 projects (mainly major projects). Support from JASPERS is initiated at the request of a Member State, on the basis of an annual action plan. The action plan identifies a number of project tasks that JASPERS will carry out for the Member State in the year in question. Environmental infrastructures Environmental infrastructures have traditionally been a cohesion policy priority, geared to preserving and improving the environment in line with Treaty obligations. The 2000-2006 General Provisions Regulation provided that ‘the Community shall contribute to […] the protection and improvement of the environment’. Similarly, the 2007-2013 Regulation stated that ‘the objectives of the Funds shall be pursued in the framework of sustainable development and the Community promotion of the goal of protecting and improving the environment’. Environmental projects are instrumental in ensuring economic, social and territorial cohesion, in line with the EU’s overall development strategies. The environment is a key dimension of the integrated approach promoted by cohesion policy (European Commission 2005a), which requires that the most pressing socioeconomic issues be tackled through integrated strategies for the renewal, regeneration and development of urban and rural areas. Moreover, infrastructure investments in the environment and risk prevention support cohesion policy efforts to trigger economic development by strengthening the competitiveness and attractiveness of EU regions. Environmental investments are expected to contribute to the economy by: Øensuring the long-term sustainability of economic growth; Øreducing external environmental costs to the economy (e.g. health, clean-up and damage recovery); and Østimulating innovation and job creation.

Environmental policy objectives can be pursued: ·as a horizontal principle across the range of sectors covered by cohesion policy throughout the programming cycle (horizontal integration); or ·by funding specific investments aimed at protecting or improving the environment and meeting the objectives generally (vertical integration). Cohesion policy investments in the environmental sector can be direct or indirect: -the direct investments are those in environmental infrastructures that contribute directly to compliance with EU legislation in areas such as drinking water supply, wastewater treatment, solid waste management and ‘green infrastructure’ (contributing to the achievement of EU targets in fields such as biodiversity and the protection of ecosystems); and -indirect investments in ‘green’ energy, transport and production systems contribute more broadly to the transition to a low‑carbon, resource‑efficient, safe and sustainable economy. This report concerns only direct investments. 3.Implementation / state of play Description of the current situation Allocations for direct environmental investments remained fairly stable in each programming period, at around €40 billion, while those for indirect environmental investments rose sharply. This reflects a ‘greening’ of cohesion policy investments in key sectors, including energy and transport. The largest share of financing in both periods went to drinking and wastewater, followed by waste management. Allocations to large-scale environmental infrastructure projects rank second after those for transport. In 2000-2006, cohesion policy supported 116 major environmental projects, for a total EU contribution of € 7.6 billion. The main beneficiary countries were Spain (ERDF, Cohesion Fund), France (ERDF), Poland (Cohesion Fund), Portugal (Cohesion Fund), Greece (Cohesion Fund) and Italy (ERDF). Cohesion Fund mainly supported wastewater treatment projects (46%) and the ERDF mainly drinking water supply projects (84%). In 2007-2013, the ERDF and the Cohesion Fund supported 167 environmental major projects for a total of €9.4 billion. Most of the support went to Romania, Poland, Hungary, Spain and Portugal. In terms of sub‑sectors funded, most of the funds went to wastewater management projects, followed by drinking water management and distribution projects, and projects supporting the management of household and industrial waste. Support for air‑quality, pollution‑control and biodiversity projects was much less significant. Short description of the methodology This staff working document is largely based on a study by an independent consultant. The analysis is complemented by internal Commission data on fund management, analytical reports and past evaluations. The evaluation follows the principles set out in the Commission’s Better Regulation Guidelines and addresses the five standard evaluation criteria: relevance, coherence, effectiveness, efficiency and EU added value. The methodology follows that used for the evaluation of large transport infrastructure projects, adjusted to the specific features of environmental projects. It is therefore based on an extensive review of the relevant theoretical and empirical literature and ex post cost‑benefit analysis (CBA) of the individual projects, complemented by qualitative assessment techniques (site visits, interviews with stakeholders, press articles, reviews, etc.) in such a way as to produce a ‘narrative’ for each project. As a first step, the impacts of large environmental infrastructure projects were mapped, measured and quantified. A comprehensive set of parameters and unit values for the most common direct effects was developed and applied consistently to all cases. Counterfactual scenarios were used as baseline for comparison to assess the performance of the projects. From an ex post perspective, the counterfactual were produced on the basis of ‘what would have happened in the absence of the project?’. The CBA methodology followed the Commission Guide to Cost-Benefit Analysis of Investment projects, adapted to the ex post perspective. It included financial, economic and risk analysis. Where quantification and monetisation were not possible, qualitative assessments of the effects were carried out. As the next step, the analysis looked into the factors that determined the observed chain of effects. Field visits were carried out for each case study and an extensive interview plan allowed for the collection of primary data and the views of a broad range of stakeholders. A total of 217 people were interviewed, mainly face to face. They represented various interest groups: civil servants (Commission, national ministries, managing authorities), experts (engineers and planners), project managers, policymakers (mayors, regional and municipal councillors), users’ and citizens’ associations, and journalists. Review and selection of the case studies

The 10 case studies were selected from a preliminary screened list of 30 major projects supported by the ERDF and the Cohesion Fund in the two programming periods (for details of the selection process, see Annex 3). As the objective was to analyse the long-term effects of large environmental infrastructure projects, the projects needed to have been in operation for at least five years at the time of the evaluation. The case studies were selected on the basis of three broad criteria (weighted for relative importance): Østrategic relevance for evaluation purposes (40%); Øavailability and quality of data from existing sources (30%); Østakeholders’ availability and willingness to cooperate (30%). The second and third criteria were key. In shared management, managing authorities are responsible for collecting and monitoring data at project level (except for the Cohesion Fund projects in 2000-2006, the Commission collected data only at programme level). Project‑level data availability and quality may therefore be challenging; it was essential that the evaluators could rely on quality data and stakeholders’ willingness to provide them. Cohesion policy data monitoring and collection have improved in recent years. In particular, the introduction of the Open Data Platform has radically boosted transparency. The choice of projects was also intended to be representative in terms of geographical and sectoral coverage (taking account of the relative scale of expenditure on the various sub‑sectors), financing periods, types of project and types of financing. It includes five projects financed in 2000-2006 and five in 2007-2013, as follows: ·seven water sector projects, of which: otwo on wastewater collection and/or treatment (Poland and Malta); otwo on water (Italy and Spain); othree relating to water and sanitation (Bulgaria, Croatia and Romania); ·two waste management projects (Estonia and Slovenia); ·one protection and remediation project (France). Figure 2. Case studies

Limitations and specificity of the methodology applied Effects produced by large scale environmental infrastructures take more or less time to materialise and achieve full potential. Also, projects differ in terms of the spatial scale of their effects. The evaluation captures changes that the projects brought over time, comparing the situations before and after their implementation. Some of the environmental effects were assessed only qualitatively, due to a lack of data and/or valuation difficulties (e.g. as regards biodiversity preservation). Similarly, wider effects (e.g. impacts on socioeconomic structures, health and social effects) were difficult to isolate and attribute to an individual project. These effects were also analysed in a qualitative way and assessed conservatively. Finally, as stressed in the staff working document on transport, it is difficult to separate the impact of an individual project from a whole set of factors influencing growth, jobs and other long-term outcomes. Individual (even large‑scale) projects rarely produce impacts of a magnitude that could be reflected in macroeconomic indicators. Since the projects are in operation, it was necessary to cover ex ante and ex post perspectives (i.e. past and future values). Ex ante values serve as a starting point for the analysis and the deviations from initial assumptions; ex post values are examined to find the reasons for any differences. The objective of comparing ex ante and ex post assumptions is therefore not to produce an updated CBA, but rather to complement the analysis of the projects’ performance. The sample was not meant to be statistically representative. The decision to focus on 10 major projects was the result of an internal analysis. It took into account the financial and timing constraints associated with assessing a wider sample and the need for quality data. It may have led to an under‑estimate of effectiveness. However, the ten projects selected are considered as examples illustrating a wide range of experience, suitable for developing project narratives and capable of producing policy lessons. The objective was to capture long-term contributions to economic development, quality of life and environmental sustainability. Therefore, only projects that had been in operation for at least five years at the time of the evaluation and therefore ‘finalised’ and mature enough to produce stable outcomes were chosen. This factor per se may have created a positive bias. 4.Analysis 4.1 Effectiveness A project’s effectiveness is the extent to which it achieves its stated objectives and delivers the expected effects. The expected long-term effects of major environmental projects identified and analysed in the report relate to: üeconomic growth; üquality of life and wellbeing; üenvironmental sustainability; and üdistributional aspects. Because effects differ according to the environmental focus area, they were analysed in three groups (see Table 2): a)water supply and wastewater; b)solid waste management; and c)environment remediation and protection / risk prevention.

Effects a)water supply / wastewater b)solid waste management c)protection / risk prevention Economic growth Variations in: -quantity of water supplied and wastewater treated; -reliability of water sources and water supply; -water quality; -resource savings (water preserved for other uses); -operating costs Variations in: -waste to landfill; -recovery of materials; -energy recovery; -reliability of waste collection; -deployment cost for utility services Variations in: -value of assets; -interruption of economic activity; -tourism; -fishing and hunting yields; -yields from timber and other raw materials Indirect: -variation in agricultural yields (indirect ecosystem services); -economies of agglomeration; -institutional learning Quality of life and wellbeing Variations in: -number of consumers served by water supply and treatment services; -quality of water supply Variations in: -number of consumers served by waste management services; -exposure to negative environmental effects of waste disposal; -household income; -human health and hygiene Variations in: -health; -recreational opportunities Environmental sustainability Variations in: -contamination of air, water and soil; -protection and resilience of natural resource systems; -GHG emissions Variations in: -GHG emissions; -contamination of air, water and soil Preservation of species or ecosystems Variations in: -carbon sequestration; -hazard risks Distributional Change in socio-geographical distribution by boosting growth and productivity in areas where they might have been constrained in the past. Certain types of user may be highly affected by changes in drinking water and wastewater pricing. Projects improved living conditions of low‑income groups. Not identified The most typical effects of the projects relate to: Øeconomic growth – these are effects resulting in more productive economic activities. They are not the main drivers behind environmental investments (where the main aims are to protect the environment and restore ecosystems), but they do materialise, in terms of costs avoided (risk prevention investments) and costs saved (waste and wastewater treatment facilities). Also, high‑quality water and wastewater infrastructures can stimulate other activities, such as tourism, in previously unserved areas. ‘Natural capital’ is also an input into economic activity, so it has an impact on economic returns; Øquality of life and wellbeing – water quality is fundamental for people’s wellbeing. Investments in water and waste management facilities reduce health risks in the long term, improve hygiene and make regions more attractive; Øenvironmental sustainability – the most relevant expected effects of environmental infrastructure investments are those that improve the condition of natural resources (water, air, soil and biodiversity) and help to preserve them in the long term. This includes making ecosystems more resistant to potentially harmful phenomena (e.g. floods, droughts) and human activities; and Ødistributional impacts – new infrastructures may change socio‑geographical distribution by boosting economic growth and productivity in (typically rural and more sparsely populated) areas where they might previously have been constrained. Projects leading to changes in water, wastewater or waste management tariffs may affect the economic situation of specific groups (e.g. low‑income households) or high water consuming industries. Main findings üProject objectives were generally well defined. üProjects were effective, but only few fully achieved their objectives. üBenefits are maximised by accompanying or synergic investments. However, projects addressing multiple phases of the water cycle are more exposed to risk. üEnvironmental projects generate long-term effects through a variety of mechanisms. üForecasting future demand is one of the most problematic determinants of projects performance. 4.1.1 Project objectives were generally well defined Setting well-defined, measurable and realistic objectives makes it possible to monitor and quantify benefits, and to evaluate a project’s performance. The case studies also highlight the importance of such objectives in enabling us to identify and secure all potential benefits. Mechanisms for monitoring long‑term achievements are fundamental to assessing whether projects have actually been effective. All but one of the selected projects had clearly structured objectives (primarily driven by local needs and compliance with EU legislation), which in turn brought improved living conditions and clear environmental benefits (access to clean water, limited pollution, etc.). The objectives were generally in line with what could be expected from such investments, given the relevant EU directives and strategies. The exception was the Slovenian project (phase II of the Celje regional waste management centre), where the objectives were not well defined and lacked quantitative targets, thus preventing proper quantitative assessment. Also, some objectives overlapped and were more relevant for the waste management centre as a whole. Better objective‑setting would have made decision‑making more transparent and accountable. The Bulgarian project achieved its goal of contributing to compliance with EU directives. However, it could have been more effective if the initial set of objectives had not been reduced during implementation, thus preventing it from also contributing to flood prevention in Sofia. The narrower scope represented a missed opportunity, as not all potential benefits were secured. The project was therefore only partially effective.

In the Maltese case, updated values were unavailable for some of the monitored parameters and no evidence could be found on the degree to which some expected benefits were achieved. 4.1.2 Projects were effective, but only few fully achieved their objectives According to the ex post CBAs, nine of the projects delivered net benefits, but only two fully achieved their intended objectives. Table 3 summarises the main findings: Sector Case study Objectives (as in project application) Score Motivation Waste-water Bulgaria – Sofia integrated water project Compliance with Drinking Water Directive (DWD) and Urban Wastewater Treatment Directive (UWWTD). Stimulate local and regional economic activities and development. 3 The project largely achieved its goals (contributing to compliance with Directives, improving reliability of water distribution, reducing losses, reducing pollution in rivers). A sewage collection system was developed, the number of households whose wastewater is treated increased, tertiary treatment was provided and water supply security improved. However, the project could have been more effective. If its scope had not been reduced during implementation, it could have improved flood prevention in the centre of Sofia and the supply of treated water to two smaller settlements. Waste-water Malta South – sewage treatment Compliance with UWWTD, Bathing Water Directive (BWD) and Water Framework Directive (WFD). Secondary objectives: enhance tourism potential thanks to cleaner seawater and thus trigger economic development. 3 The project achieved its primary objective (treating wastewater collected by sewerage networks in the south of Malta before discharge), restored bathing water quality and ensured Malta’s compliance with the BWD. However, it is not yet in full compliance with the UWWTD and farm waste illegally discharged into sewers negatively affected the plant’s operational phase. Fishing activity has not increased and there is no evidence of better drinking water. There are concerns over the impact of population growth in the long run. Waste-water Poland – Sochaczew sewage management Improve wastewater management in the area by increasing connection rate to sewerage network and enhancing quality and effectiveness of wastewater treatment. 4 The project achieved its objectives of providing a sewerage service for new households and ensuring the capacity of the municipal wastewater treatment plant (WWTP), but the generation of benefits was delayed due to slow connection of households to the network. Waste-water Romania – Craiova sewerage network Treat wastewater in line with Romanian law transposing UWWTD. 4 The project achieved its objectives (reduction of river pollution, transboundary pollution and water losses in distribution network). However, it had a minor negative impact related to odour emissions around the WWTP. Operation of the WWTP resulted in increased energy consumption and it is currently not used at full capacity. Water Croatia – water supply and sewerage system Reduce water pollution, improve living conditions and ensure compliance with EU legislation. 3 The project improved the sewerage system and enabled wastewater treatment. It increased the number of inhabitants connected to the drainage system, raising living standards and protecting the Sava River. However, it led to an increase in GHG emissions. At present, the WWTP operates at half capacity due to depopulation resulting from the economic crisis. Mitigating measures are being taken, as full capacity is expected thanks to implementation of a new project (Slavonski Brod II) to connect two neighbouring agglomerations with compliance obligations. Technical issues also limited the project’s impacts. Water Italy – Favara di Burgio aqueduct Restore effective water captation, collection and adduction at aqueduct, reduce leaks from around 20% to 5% of total input and ensure continuity in provision of water to head tanks for inhabited areas. 4 The project achieved its goal (water provision to municipal head tanks) with minor delays. It has further potential to sustain local development, but accompanying investments were limited. Water Spain – Anguilas desalination plant Address over‑exploitation of water resources in the region and ensure reliable water service to inhabitants and farmers. 4 The project achieved its goal (reliable water provision for households and farms) and end‑users are benefiting from its implementation. New projects are planned to take advantage of it. The generation of benefits was delayed by issues during implementation. Water Estonia – purchase of a multifunctional ship Prevent and respond rapidly to pollution incidents to avoid contamination and disturbance of habitats and maintain their favourable status, ensure functioning of critical areas and reduce risk to human health and life. 5 The project achieved the objectives based on HELCOM recommendations. It achieved its main target of improving offshore sea‑pollution control capacity and reduced the risk of damage from maritime pollution. Waste Slovenia – Celje waste management centre Ensure mechanic biological treatment (MBT) (61 500 t/yr) and thermal treatment (TT) (25 000 t/yr) of municipal waste. 2 The objectives were poorly defined and lacked quantitative indicators, so it is difficult to assess effectiveness. On the whole, the objectives were achieved (though later than scheduled). However, the expected benefits were achieved only partially, mainly because of demand under‑estimation. Selection of a different technology could have improved effectiveness and further reduced GHG emissions, e.g. as in the case of the Ljubljana waste management centre, which used cogeneration of electricity. Others France – Sète‑Marseillan lido protection Restore normal functioning of coastline and ensure better protection against erosion. Protect environment, in particular natural area of ecological, fauna and floristic interest and Natura 2000 perimeter. Maintain mobility function of coast road. Maintain local economic activities on lido. 5 The project’s main objectives were achieved. Source: CSIL, Ramboll (2019). The Estonian and French projects, in the area of risk prevention and remediation, fully achieved their objectives and therefore scored highest. In the water and wastewater sector, the Polish, Romanian, Italian and Spanish projects were effective. They achieved their stated objectives and delivered the expected effects, despite minor reservations in each case. Three projects (in Poland, Italy and Spain) produced benefits later than planned, due to implementation issues that postponed the start of their operation phase. This was particularly relevant in the Spanish and Polish cases. In the Romanian case, the WWTP is still not working at full capacity as a result of over‑optimistic estimates at the project design stage. Even after being revised, estimates of capacity remained too high due to a drastic decrease in water demand, because of reduced water losses in the networks and a diminishing population connected to the water and wastewater services. There are minor negative effects due to odours and increased GHG emissions. The Favara di Burgio aqueduct project (Italy) achieved its primary goal (providing water to the municipal head tanks), but did not deliver all its potential benefits because the requisite accompanying investments were not implemented. The lower effectiveness scores for the four other projects (in Bulgaria, Malta, Croatia and Slovenia) are due to various factors, with no identifiable unifying pattern. In the case of the Bulgarian project, the expected effects were delivered later than expected as a result of implementation issues. Also, the scope of the project was further reduced during implementation, thus preventing it from delivering all potential benefits. In Malta, full compliance with the UWWTD has yet not been achieved, due to illegal dumping of farm waste into the sewers, and there is no evidence that some of the benefits expected ex ante (e.g. better‑quality drinking water) have been secured. The Croatian project achieved its primary goal of improving the sewerage system and enabled wastewater treatment, but it also led to an increase in GHG emissions. Also, the WWTP is operating only at half capacity. The low score for the Slovenian project stems from its poorly defined objectives and a lack of quantitative indicators. As a result, it is difficult to assess effectiveness. While the general/strategic objectives in the project application can be considered as having been achieved, there are significant differences between the benefits that were expected and those that were achieved, due to a failure to take account of changes in recycling rates (thanks to the implementation of EU legislation), which led to a decrease in the volume of mixed municipal waste. In the cases of the Maltese and Croatian projects, the capacity to forecast demographic trends proved to be an important factor influencing effectiveness. In both cases, this affected project performance. 4.1.3 Benefits are maximised by accompanying/synergic investments and adequate response to individuals’ needs EU waste and water legislation reflects the importance of co-investment in maximising the benefits of waste and water management systems. For example, the WFD aimed to bring about a shift from fragmented policies and investment in the water sector towards integrated management of water resources. The project sample shows that in the case of investments following an integrated approach (water and waste), benefits can be maximised through accompanying or synergic investments and/or measures. In fact, impacts can be limited by a lack of accompanying measures. It also shows the complexity and difficulty of such an approach, as projects addressing multiple phases of the water cycle are inevitably more exposed to risk. A trade-off emerges in this regard: while projects with the most integrated approach allow for highly consistent planning and comprehensive design, more risk factors can affect overall project performance, as forecasting is multi-faceted and considerably more demanding. The Spanish desalination plant project is an example of this. Addressing only one phase of the water cycle, it met all set objectives (reducing water deficit, reducing pressure on aquifers), but could have had a greater impact if accompanying investments (e.g. in water storage facilities) had been made to enable the plant to operate at a more efficient scale. Similarly, the Polish project did not involve any incentive mechanisms to facilitate physical connection between individuals and the new wastewater network. This took longer than expected and affected the project’s effectiveness. Likewise, in the Italian project, a lack of investment in local water distribution networks prevented the reconstructed aqueduct from achieving all potential benefits. On the other hand, in the Maltese case, the design of the WWTP allowed for future upgrades, i.e. the construction of a polishing plant providing treated sewage effluent for irrigation, which was instrumental in putting an end to the illegal dumping of farm waste into sewers, thus succeeding where law enforcement had failed. Individuals may react in various ways to policy incentives and their response may affect projects’ effectiveness. The effectiveness of an environmental project depends not only on infrastructure investments and legal provisions, but also on how choices are shaped and what practical alternatives to negative paths are offered. In this respect, other non‑infrastructure accompanying measures can play a positive role.

The Slovenian waste management project included a campaign to promote waste separation. This delivered good results. Although a causal link is difficult to prove, the subsequent slowing‑down of the campaign and related educational activities coincided with a reduction in the quantity and quality of separate waste collection. 4.1.4 Environmental projects generate long-term effects through a variety of mechanisms The analysis shows that large environmental infrastructure projects generate non‑environmental benefits by improving quality of life and contributing to economic development. This is in line with the rationale of environmental and cohesion policy legislation, where interventions are based on both socioeconomic and environmental considerations. The Water Framework Directive and the Waste Framework Directive promote a market‑based approach: an efficient use of water resources and the use of market‑based instruments to provide incentives for the application of the waste hierarchy, respectively. Member States have to apply the ‘cost recovery’ and ‘polluter pays’ principles in achieving the Directives’ environmental objectives. These factors have clear economic implications for operators and consumers. Large‑scale environmental investment projects improve the territorial distribution of key infrastructures (e.g. wastewater treatment plant (WWTPs)) and citizens’ and firms’ access to services (e.g. water supply) and ecosystems (e.g. a restored lido). In this way, they make territories more attractive for both the population and economic operators, thus providing a necessary (but not sufficient) precondition for territorial development in line with cohesion policy objectives. Environmental interventions are multidimensional and give rise to multiple changes. For instance, the Maltese project was aimed at treating wastewater before discharge into the sea, thus contributing to seawater quality. In turn, this contributed to the preservation of seawater ecosystems (environmental impact), odour reduction and the recreational value of beaches (impacts on human wellbeing). The additional benefits generated by the projects differ according to the field of intervention. The main ones – both measurable (included in the CBA) and non‑measurable (not included in the CBA) – were: ·the water supply projects generated mainly benefits of economic growth (Italy and Spain); ·the projects combining wastewater collection and treatment (in Bulgaria, Poland, Romania and Croatia) had a big impact in terms of quality of life and environmental sustainability; ·the Maltese project, which focused mainly on wastewater treatment with only a minor component of wastewater collection, generated benefits mainly in terms of environmental sustainability; and ·in the three other cases, environmental benefits were not the most significant effects. The Slovenian and Estonian projects positively influenced economic development, while the French project mainly improved the quality of life. These considerations remain largely true if measurable effects (economic growth, quality of life, environmental sustainability), covered in the ex post CBAs, are taken into consideration (see Figure 3). Source: CSIL, Ramboll (2019). All the selected major projects show benefits across the three categories, with the exception of the Italian project, which shows a negative impact in terms of environmental sustainability. This is due to the counterfactual scenario used for the analysis and the lack of quantified ex post data. The project consisted of replacing an old aqueduct that was characterised by high levels of leakage and required high amounts of energy to operate. However, the analysis did not capture the positive effect in terms of energy savings, as the counterfactual scenario used was ‘no operation’ rather than ‘business as usual’ (which was considered unrealistic). For the purpose of the CBA, the new aqueduct is therefore considered as a greenfield investment that generated new GHG emissions not compensated by other positive environmental effects. The varying degrees of environmental impact reflect two important features of environmental infrastructure projects in the cohesion policy framework: ·the development of water supply infrastructures involves integrating economic considerations (efficient use of natural resources) into water management in order to make efficiency gains and minimise costs. Investments in water supply, sanitation and waste management are aimed at improving the availability, reliability and quality of water and waste services. Accordingly, these projects addressed not only environmental protection, but also human wellbeing and economic development; · even where a project achieves environmental objectives (e.g. the French case, where the objective was to counteract coastal erosion), it may give rise to much greater non‑environmental benefits. In some cases, there may be a trade-off between environmental benefits and environmental costs. For example, the modernisation of the WWTP in Poland had a positive impact in terms of avoiding the contamination of surface water and soil, but also a negative impact in terms of increased GHG emissions. Where investments entail substantial energy usage, either in construction or operation, the negative environmental implications thus need to be considered, since they may reduce the net environmental benefits of a project. 4.1.5 Forecasting future demand is the main and most problematic determinant of project performance

Several factors were found to affect project performance: ·the quality of ex ante analysis, in particular forecasting and management capacity; ·the quality of the selection process; and ·the governance structure. ‘Forecasting and management capacity’ is understood as the ability to predict future trends and react to unpredicted challenges by estimating and adapting resource requirements. It relates to technical effort in the ex ante (project preparation) phase and the professional capacity to manage the project in the operational phase to deliver the expected level of service. The forecasting exercise (including data collection and modelling) is the foundation for sound project performance. The case studies show that ex ante forecasts are often over‑optimistic and under‑estimate completion time. This may affect the project’s design, overall timeline and financial sustainability, and the actual delivery of long‑term effects. The exercise is also the core of the CBA, the quality of which is thus affected by deficiencies in the forecasts. The main areas where proper forecasting is essential and forecasting weaknesses were identified were demand, costs and completion time. The analysis echoes the findings of the ex post evaluation of major transport projects in 2000-2006 and 2007-2013, which highlighted forecasting capacity as a critical factor in the implementation of large infrastructure projects. In several cases, a failure to predict demand accurately can be attributed to official statistical data that do not factor in changes in macro-trends and to local administrators’ difficulty in anticipating and integrating territorial dynamics in available data. Demand for the planned environmental infrastructures was over‑estimated in six of the reviewed cases, leading to overcapacity. However, whether this leads to operational inefficiencies depends on the chosen technology. For example, the promoters of the Slovenian project did not adequately forecast the effects of changes in waste management legislation on trends in municipal waste. The technology chosen for the waste treatment plant was not fit for the actual amount of waste collected. Consequently, the unit cost of processing waste is higher than expected. Moreover, the aerobic biological treatment of such lower quantities of waste is not efficient, so electricity for the plant’s operation needs to be sourced from the grid, causing higher GHG emissions than when higher quantities were processed. In contrast, in the Romanian case, where demand for treated wastewater was also over‑estimated, the choice of technology enables the operator to adapt to different quantities without an impact on efficiency. Final demand was under‑estimated in the case of the Maltese sewage treatment plant, which was designed for a resident population of 500 000. However, due to strong population growth (17% nationwide between 2007 and 2018), which is expected to continue in the next decade, the plant will have reached maximum capacity by 2023 and will have to be scaled up. As regards completion‑time forecasting, seven projects took longer to implement than initially forecast. Under‑estimation of the complexity and duration of administrative procedures and over‑estimation of contractors’ capacity were the two main factors contributing to delays. In the Bulgarian case, Sofia Municipality had unanticipated difficulties in expropriating properties so that the water pipelines could be built; these contributed to a 3‑year delay in project implementation. In the Italian project, authorisations from local authorities took longer than expected and contributed to a 12-month delay. Three of the projects were delivered on budget. The overruns in the other cases can be explained by the volatility of material and labour costs during and after the financial crisis. The examples above highlight that careful planning and forecasting of project demand, time and costs are essential in preventing project underperformance. At the same time, where a project deviates from a plan, its success depends on the response of those involved. An appropriate response to implementation or operational difficulties can turn a project around. Similarly, where a project is going better than expected, the situation can be further improved by exploiting unexpected opportunities. For example, when it became clear that the construction costs for the Estonian multifunctional ship would come in under budget, the project promoters included additional project management services (e.g. juridical and reporting services) in the budget. This ensured a high level of construction quality. The quality of the selection process relates to the institutional and legislative framework in which public investment decisions (especially those co-financed by ESIFs) are taken. In particular, it concerns the processes in place and the tools used to choose between projects. Overall, the selection process made a positive or slightly positive contribution to the performance of the projects. Selection went smoothly and there were no major complications, partly thanks to the fact that the projects were included in Operational Programmes and sector‑specific plans, for which planning and selection are governed by well‑established regulatory and administrative frameworks. In the examined cases, the steps usually included a preliminary needs-based assessment, followed by a feasibility and options analysis. In general, once an option is selected, it undergoes stakeholder consultation, usually in the form of a public hearing where complaints and suggestions are taken into account. The project design is then completed and the project is implemented. The analysis shows that selection processes can be lengthy. Major environmental projects take a lot of time; many years may pass from the first idea to actual implementation. For strategic, financial and technical reasons, projects may remain in the pipeline for a significant period, with a clear negative impact on effectiveness, as the generation of benefits is significantly delayed. Postponements, delays, administrative inertia and long tendering procedures are among the main reasons for the longest selection processes. In Italy, Malta and Poland, the selection process involved stakeholder consultations. These helped to ensure that the concerns of local communities were addressed and no evidence was found that they contributed to delays in the selection process. Evidence shows that, regardless of the length of the selection process, the project assessment should be updated and revised to take account of changed circumstances.

The evaluation showed that, in some cases, a CBA was carried out only in the framework of the funding request (attached to the application form), rather than as a basis for decision-making. The Maltese project is an example of good planning practice. The construction of a new WWTP in the south of Malta was part of a 1992 sewerage masterplan for Malta and Gozo, which set out the need for three new WWTPs. The first feasibility study (2000) identified the components of the future investment and featured an options analysis, which was updated in 2006, 2007 and 2010 with three project alternatives: phasing out, maintaining or upgrading the existing plant. The preferred option was selected in 2010 on the basis of cost, energy consumption, CO2 footprint, the impact of construction works and technology. Stakeholders were consulted as part of the selection process and a monitoring committee ensured that the concerns of the local community were duly addressed. In the context of this document, ‘project governance’ concerns: ·the number and type of stakeholders involved during the project cycle; ·the degree to which they cooperate effectively; and ·the way roles and responsibilities are shared. Project governance has major implications for the partners’ financing arrangements, autonomy and responsibilities. The most important determinant of success was leadership by a single entity with the responsibility and capacity to lead all stakeholders. The involvement of a large number of stakeholders in project implementation is not necessarily a sign of a weak governance structure, if the leadership and coordination tasks are carried out effectively. However, some cases seemed to show a correlation between a higher number of stakeholders and a less positive impact from the governance structure. In some cases, the Commission supported the formation of an efficient governance structure. In the EU‑15 projects, it had discussions with project promoters at the application stage and then followed the projects more passively. With the EU‑13 projects, it took a more active role in steering project implementation by interacting with stakeholders at various levels of governance. In the Italian case, a multi‑faceted and complex set of institutional actors and stakeholders were involved in project implementation, due to a multi-level governance structure in which no institutional stakeholder played a strong supervisory role over the whole water cycle. In addition, institutional conflicts significantly hampered the implementation of indispensable complementary investments, preventing the benefits generated by the new aqueduct from being maximised. A lack of clarity in the assignment of responsibilities within the governance structure might put at risk the future financial sustainability of the French project. Initially, the costs of maintaining the wave attenuators were to be split between the urban community of Sète and the towns of Sète and Marseillan. However, the related maintenance costs have proved to be higher than expected and the dredging costs for Marseillan have risen in the course of project implementation. As a result, the costs will probably need to be shared differently in future and it is not clear that the three stakeholders will be able to meet them. 4.2 Efficiency ‘Project efficiency’ gauges the relationship between resources used and changes generated, i.e. value for money. It expresses how input resources (especially time and costs) were used in order to produce the desired effects. üMost of the projects were efficient, delivering social benefits that exceeded the costs. üMost of the projects were not as efficient as expected ex ante: cost overruns and time delays affected their performance. Main findings Sector Case study Score Motivation Wastewater Bulgaria – Sofia water cycle 2 Due to pitfalls in the planning stage, the project was severely delayed, over‑estimated final demand and its scope was in the end reduced. Nevertheless, the socioeconomic benefits outweigh the costs. Malta South – WWTP 3 The project was completed on budget. Some differences were reported between costs and benefits forecast ex ante and those that can be observed ex post: an expected increase in fishing and aquaculture activities and an expected increase in the quality of drinking water did not materialise. Nevertheless, the project delivered a positive socioeconomic return on investment. Poland – Sochaczew sewage management 3 The project was completed with a large cost underrun due to the macroeconomic situation. Demographic trends meant that demand was not as high as anticipated. Nevertheless, the project delivered high net socioeconomic benefits. Romania – Craiova sewerage and wastewater treatment 4 Despite an over‑estimation of demand, the project was completed on budget and provides socioeconomic benefits that exceed costs. Water Croatia – Slavonski Brod water supply, sewerage and wastewater treatment 4 Final costs were lower than in the original CBA, as services, equipment and materials prices changed due to the global economic crisis. Implementation was in line with the schedule. Overall, the project generated net benefits. Italy – Favara di Burgio aqueduct 4 The project was implemented on budget. The operating costs ended up lower than predicted, contributing to a net socioeconomic benefit. Spain – Anguilas desalination plant 3 The project cost was above the initial budget due to an increase in scope. However, the socioeconomic benefits outweigh the costs, as forecast. Others Estonia – multifunctional ship 4 Good project planning, design and implementation, due to a competent team and strong cooperation between stakeholders, ensured that the project was finished below anticipated cost. The socioeconomic benefits are roughly equal to the costs. France – Sète lido protection 4 The project is on track to be delivered within budget, with the socioeconomic benefits outweighing the costs. Slovenia – Celje waste management 1 No proper assessment was possible, as no data were available on project design and technology (thus justifying the low score). Costs increased in the construction phase. The demand forecast was over‑estimated, causing the plant to operate below minimum efficient capacity. Both effects led to socioeconomic costs exceeding socioeconomic benefits.

Source: CSIL, Ramboll (2019). 4.2.1 Most projects were efficient, delivering social benefits that exceeded the costs The CBAs show that all but one of the analysed projects delivered benefits that exceeded the costs. The projects generated value for money, independently of whether they were completed on time and on budget. At the same time, most were not as efficient as expected ex ante: in some cases, costs exceeded the ex ante forecasts and benefits were affected by factors such as over‑estimated demand. In some cases, delays were a cause of less-than-expected efficiency in project implementation. There was insufficient data to look into the unit costs of comparable investment (or alternative technical solutions) to determine whether the same benefits could have been secured with fewer resources / at lower cost. However, evidence from projects suffering from overcapacity suggests that resources could have been allocated more efficiently. The two lowest B/C ratios are caused by different factors. In the Slovenian case (B/C ratio < 1), final construction costs are higher and actual demand is lower than in the ex ante forecasts. Waste treatment suffers from overcapacity (causing excessive operating costs) and inflexible technology for energy sourcing. In the Estonian case (B/C ratio = 1), the low ratio is the result of conservative estimates of net benefits: the ship’s impact in terms of reducing the number of deliberate spills has not been quantified, due to the lack of data. Source: Ex post evaluation of major projects supported by the European Regional Development Fund (ERDF) and the Cohesion Fund between 2000 and 2013. Financial sustainability ‘Financial sustainability’ is a measure of a project’s capacity to cover its costs through the investment and operating phases. In line with the rationale of EU funding for major projects, none - except one - of the selected projects were financially profitable: their financial net present values were negative or zero and they required funding. For these, the EU grant was decisive in ensuring financial sustainability. The exception was the Italian project, which turned out to be financially profitable after the event. Ex ante, it was not assessed as financially viable for the private sector. Ex post, however, the reductions in operating costs were greater than expected. Due to strong political will, the project would probably have been implemented even without EU support (see below). Project funding came from various sources in addition to the EU. As shown in Table 5, all projects received national contributions or equivalents. Case study Funding Loans National contribution or equivalent Infrastructure manager’s own resources ERDF / Cohesion Fund EIB Private loan / bonds Bulgaria – Sofia water cycle Malta South – wastewater treatment Poland – Sochaczew sewage management Romania – Craiova sewerage and wastewater treatment Croatia – Slavonski Brod water supply, sewerage and wastewater treatment Italy – Favara di Burgio aqueduct Spain – Anguilas desalination plant Estonia – multifunctional ship France – Sète lido protection Slovenia – Celje waste management Source: CSIL, Ramboll (2019). All but one of the projects generated revenues that formed the basis of their financial sustainability in the operational phase. In seven cases, the revenues covered the operational costs and ensured long-term financial sustainability. Three projects required public funding from local, regional or national sources to cover the operating and maintenance costs.

All water and waste projects collected revenues from end‑users, in line with the ‘polluter pays’ principle. The Water Framework Directive promotes the sustainable use of water as a scarce resource by requiring that the price of water reflect its true cost, in terms both of infrastructure provision and external costs to the environment (European Environment Agency 2013). Similarly, it applies the ‘polluter pays’ principle in the waste sector. In three cases, tariffs ensured financial sustainability over the lifetime of the project. In the Italian case, the company managing local water distribution imposed a tariff on final household and industrial users. Part of this tariff covered the service of water transit, which is paid by the local distributor to the company that operates the aqueduct and supplies water to the municipal head tankers. The revenues from the tariffs exceeded the operating costs over the project lifetime and so ensured the financial sustainability of the aqueduct. Likewise, in the Bulgarian project, the tariffs were set by a national commission at a level that is expected to be sufficient to generate a net revenue throughout the project’s operational phase. The Polish project was not expected to experience cash flow problems, as revenues from the wastewater tariff completely cover operating, maintenance and depreciation costs. Also, due to unexpected reductions in investment costs, the depreciation of the project is lower than expected, which has allowed for a reduction of the tariff. Some projects became financially sustainable only after a transition period, due to end‑users’ low willingness to pay and the existence of alternatives. In the Maltese case, there were concerns that users might not be able to pay tariffs at the level required to guarantee financial sustainability immediately after project completion. Therefore, the tariff is scheduled to increase gradually until 2032, when costs will have been completely recovered. In the meantime, the operating company (WSC) will receive subsidies from the government, which (together with collected revenues) will ensure the project’s sustainability. In the Spanish case, the tariff for water from the desalination plant was set at cost‑recovery levels from the beginning. Because this was nearly double the tariff for water from aquifers (which arguably did not cover external environmental costs), demand for the desalinated water was initially low. Therefore, the operator initially failed to cover operating costs and national public contributions were required to sustain operations. Later, however, farmers experienced difficulties in obtaining water from other sources and had to resort to the desalination plant; eventually, demand picked up and the desalination plant became financially sustainable. Even if financial sustainability is secured in the short term, it may be threatened in the long run as a result of political factors or demographic trends. For instance, the Croatian project was financially sustainable at the time of the evaluation (2018), as the water price covered operational and administrative costs. However, due to strong depopulation in the area, water consumption may decrease in the future, in which case the revenues collected via the water tariff may not be sufficient to cover operating, maintenance and depreciation costs, and operations may become financially unsustainable. For this reason, the national authorities have prepared a phase II of the Slavonski Brod project whereby it will take in two neighbouring agglomerations (Grcin and Brodski Stupkin). In addition, a consolidation of water utility companies throughout Croatia is currently under way. Similarly, in the Slovenian case, the volume of municipal waste treated by the plant is already (by 2018) less than predicted ex ante. This is a result of updated waste legislation and the introduction of separate waste collection and recycling in Slovenia. Increased recycling rates created financial problems for the project in the first few years of its operation, but this was compensated by taking in waste from a wider area. At the time of writing, this has ensured cost recovery and the financial sustainability of the project. Unlike water and waste projects, risk‑reduction projects rely exclusively on government subsidies for their financial sustainability. The operations of Estonia’s multifunctional ship were funded purely by the government. In France, the costs of maintaining the wave attenuator were covered by the local municipality and all other operating costs (e.g. relating to the maintenance of the beach and car parks) were covered by the two towns nearest the beach. Although the project generated revenues in the form of concessions paid by businesses operating on the beach, these were not earmarked to cover the operating costs of the project infrastructure. 4.2.2 Most projects were not as efficient as expected: cost overruns and time delays affected their performance As discussed extensively in the literature, cost and time overruns in project construction are common in major projects, including those relating to environmental infrastructure. Investment costs are often under‑estimated. Delays postpone benefits and may result in cost overruns as well. Seven out of ten selected projects experienced time overruns, while only three experienced cost overruns. Table 6 summarises these factors for each of the case studies. Case study Was the project completed on schedule? Was the project completed within budget? Bulgaria – Sofia water cycle Delayed by 3 years due to prolonged expropriation procedures that were not predicted. Substantial cost overrun (+32%) mainly from price increases due to a construction boom and extension of public procurement procedures. Malta South – wastewater treatment Completed on time. Slight cost underrun (-2%) due to non‑occurrence of planned contingencies and favourable project management contract. Poland – Sochaczew sewage management Delays of over a year due to a slow procurement process. Significant cost underrun (-51%) due to price reductions from economic crisis and competitive tendering process. Romania – Craiova sewerage and wastewater treatment Slightly delayed implementation due to underperformance of a contractor and re-launch of tendering process. Slight cost underrun (-2%) due to savings achieved in tendering process and works contract. Croatia – Slavonski Brod water supply, sewerage and wastewater treatment Finished with 1‑month delay due to bad weather conditions. Significant cost underrun (-22%) due to price reductions following financial crisis. Italy – Favara di Burgio aqueduct Delay of 1 year due to bad weather, administrative procedures and third‑party supplier delays. Significant cost underrun (-17%) due to discount offered by winning consortium of contractors. Spain – Anguilas desalination plant Delayed by 3 years due to lack of complementary investments and change in project design. Cost overrun (+63%) due to changes in project design. Estonia – multifunctional ship Completed 4 months earlier than expected due to experienced contractors and good stakeholder cooperation. Slight cost underrun (-0.6%) due to changes in project design (elements excluded). France – Sète lido protection Delayed by 3 years due to introduction of a testing phase for innovative technology. Cost underrun (-7%) due to good project management (e.g. introduction of cost‑saving test phase). Slovenia – Celje waste management Delay of 1.5 years from changes in selected contractors due to bankruptcy and limited experience. Cost overrun (6%) due to multiple changes in contractors, which resulted in design changes. Low score also influenced by lack of appropriate data. Source: CSIL, Ramboll (2019). Final project completion time and costs compared to data contained in major project applications Delays Three of the projects were completed on schedule. In two cases, project preparation was crucial for smooth implementation. For example, during the planning of the Estonian multifunctional ship, a team of sectoral experts mapped the possible risks associated with the implementation phase. This risk‑mapping, combined with the high expertise of the construction and design companies, resulted in a smooth and timely construction of the ship. In the Croatian case, thanks partly to the technical assistance provided to the project promoters, the project was well prepared in terms of technical documentation, a feasibility study and CBA. In addition, it received strong political support at local and national levels and was completed on time. The other seven projects experienced delays. In three cases, the delays were significant and can be partly attributed to contractors’ underperformance. This was the case with the Romanian project, where the original contractor had to be replaced through a new tendering procedure. For the Slovenian project, a full assessment of cost and time overruns was hampered by a lack of supporting data for the technology and design decisions (hence the very low score). In addition, the first two selected contractors filed for bankruptcy and the third had little experience with the type of technology selected for a solid waste treatment plant of such a scale. This led to delays in the building and installation of equipment, changes in the design of the plant and thus to increased costs. In addition, a fire damaged the MBT plant, so reconstruction and further investment were needed. In two cases, the delay was caused partly by unpredicted exogenous factors: adverse weather conditions (Italian and Polish projects) and unpredictable events. In the case of the Polish project, the builders came up against underground infrastructure that had probably been built over 50 years previously, was not included in existing registers and maps, and required additional works and changes to project design. However, exogenous factors do not necessarily cause delays if they are properly predicted in the planning phase. The Italian project was interrupted due to archaeological findings requiring additional excavations. However, the project was not delayed, as this possibility had been factored in during the planning phase. In contrast, a lack of experience led to significant delays to the Bulgarian water supply network. The project promoters under‑estimated the complexity of (and time involved in) concluding expropriation procedures to carry out the construction work. The impact was amplified by the lack of a geographical information system, which meant that the properties to be expropriated were initially identified manually – a highly time‑consuming activity. Slow procurement procedures caused delays in both the Romanian and the Polish projects.

The French project was deliberately delayed. It involved using two innovative technologies to reduce coastal erosion: -an on-land drainage system that accelerates the infiltration of water into sand on the beach to stabilise it; and -a soft (as opposed to hard) wave attenuator placed 300 m out to sea. Although both technologies had been successfully tested elsewhere, the French authorities required a three-year trial period in which both were tested on a limited scale. The wave attenuator was then selected as the best solution. The decision to test the technologies ensured the project’s financial sustainability, as full deployment of both would have resulted in a net loss. This is an example of prudent project management, where delay was the result of an active strategic choice. Cost overruns For seven of the projects, the final costs were within estimated budgets. Two projects experienced significant cost overruns, while one ran a slight overrun. In general, where benefits exceed costs by a sufficient margin, projects may remain efficient (and still have a good B/C ratio) even in the event of cost overruns and delays. For example, the Bulgarian and Spanish projects suffered from significant cost overruns and delays, but were still efficient, in the sense of socioeconomic benefits outweighing costs. Discounts provided by contractors during the procurement processes played an important role in budget control in several of the analysed projects. For instance, the Polish project was split into smaller contracts, so that small companies could take part in the tender procedure. This increased competition among contractors and put downward pressure on the prices of submitted tenders. In the Italian case, interest charged due to delayed payments and additional payment requests by the construction company increased expenditure by about €2 million. However, due to the discount offered by the winning consortium at the end of the tendering procedure, the total project costs were still 16.8% below those initially envisaged. The tendering strategy had encouraged the participation of local engineering firms with good knowledge of the construction area. Similarly, a well-executed tendering procedure led to total cost savings of about €1.5 million in the Romanian case. Severe macroeconomic shocks can heavily influence the construction costs of major projects. As the cases below show, these may have a major impact on the financial sustainability of project implementation, but are difficult to foresee. In the Polish case, the onset of the financial crisis increased competition among contractors, putting downward pressure on prices in the sector. Also, the crisis caused a devaluation of the Polish currency, reducing project expenditure in euro terms. The crisis therefore contributed to significant cost underruns of about half the predicted total cost. It had a similar effect on the Croatian project, albeit resulting in a smaller cost underrun. The macroeconomic situation had the opposite effect in Bulgaria. A boom in the construction sector at the time of implementation (between 2006 and 2009) led to an increase in construction prices and a cost overrun amounting to about a third of the envisaged total costs. Changes to project design influenced the implementation costs in two projects. In Spain, this was due to a decision to construct pipeline connections to the irrigation community dams, to facilitate the full operation of the desalination plant, which gave rise to a request from Spain to modify substantially the initial eligible costs of the project. In contrast, the scope of the Bulgarian project was reduced due to under‑estimated difficulties in carrying out expropriation procedures. This led to a reduction in the overall expected costs and, as a result, in the grant from the EU. The Slovenian project suffered a slight cost overrun due to adjustments in the design of the planned TT plant for waste. In contrast, the promoters of the Estonian project decided to exclude an on‑board laboratory for the chemical testing of oil spill samples from the project design, because the Police and Border Guard Board (the operating organisation) did not have a laboratory technician. This change of specifications allowed bidders to reduce their prices in the procurement procedure and the project was completed with a slight cost underrun. In the Maltese case, the choice of technology and design for the sewerage treatment plant best suited to the context, and good management, resulted in cost underruns. No technical issues or modifications to the original project plans were reported during implementation. Therefore, none of the €1.5 million budgeted for contingencies was disbursed. As underlined in the effectiveness section, forecasting capacity proved to be a critical factor and in some cases affected the final efficiency of the project. In the ten analysed case studies, ex-ante forecasts were often too optimistic in terms of demand, time, and costs estimation thus affecting the design, implementation timeline, financial sustainability as well as the actual delivery of long-term effects of major projects. Forecasts were not always adjusted over time in line with the evolving context of the projects, such as demographic trends, institutional or legislative framework and behavioural patterns. Demand was for example overestimated in the case of the Sochaczew sewage management plant and in the Slovenian case, causing the plants to operate below their minimum efficient capacity and suggesting a sub-optimal allocation of resources. Administrative costs Managing and implementing ESIF programmes demands financial and personal investments from all involved. ‘Administrative costs’ (costs linked to the administrative tasks of any fund‑programming body) relate to administrative workload and the costs for the purchase of services (e.g. expertise) and goods. The ex post evaluation did not study the overall administrative cost of preparing and managing these specific projects. A recent study established a new baseline for the administrative costs and burden of the current ESIF programming period. It concluded that the administrative costs linked to managing the ESIFs are reasonable overall and decrease as a proportion of the budget with the size of the project. 4.3 Relevance ‘Project relevance’ is a twofold concept relating to a project’s: -alignment with existing needs; and

-consistency with priorities established in the field at various levels (local/regional, national and EU). Main findings üThe projects contributed to integrate sustainability concerns in programme design and implementation. üAll projects address real and significant environmental needs. üThe projects remain relevant over time. üProjects address not only environmental, but also economic needs. üCompliance is a significant driver for project implementation. üProjects are aligned with the EU and national priorities. 4.3.1 The projects contributed to integrate sustainability concerns in programme design and implementation Attention to sustainable development as a horizontal theme has increased over the years, which is reflected in the legislative changes of the three programming periods. In 2000-2006, project relevance from an environmental point of view was not always a prime concern of cohesion policy. The ex post evaluation of 2000-2006 cohesion policy programmes in the environmental sector found that: ‘none of the analysed OPs contained a comprehensive analysis of how environmental issues may interact with the economic and social development of the regions. Decisions on the allocation of funds were driven by sectoral approaches rather than addressing the regional needs from a more integrated perspective. The interventions in the environmental sector were designed to address the main environmental issues within the framework of sectoral policies and were not integrated into a regional perspective. This was particularly true for the major sectoral programmes covering environment that were oriented towards environmental objectives.’ The evaluation also concluded that little attention was paid to sustainable development (linking three pillars: economic, social and environmental) as a horizontal theme in programme design and implementation. Efforts in this area were mostly seen as a question of fulfilling regulatory requirements rather than developing the concept. In 2007-2013, integrating sustainability concerns into programme implementation proved challenging, particularly when it came to developing, appraising and selecting projects for financing. In particular, the newer Member States struggled to transfer the complexity of sustainable development into actual project development and on the ground. The Common Provisions Regulation (CPR) for 2014-2020 puts greater emphasis on sustainable development by requiring Member States and the Commission to ensure that environmental requirements are promoted in the preparation and implementation of partnership agreements and programmes. Sector Case study Score Motivation Waste water Bulgaria – Sofia integrated water project 5 The project addressed a clear bottleneck in infrastructure endowment and service provision, and severe environmental concerns, caused by a lack of sewage system in part of the city, illegal sewer connections and sewage discharge into watercourses. The project was aligned with priorities established at local, national and EU levels. Malta South – sewage treatment infrastructure 5 The project addressed the issue of untreated wastewater discharge into the sea, which hampered the use of coastal waters and damaged ecosystems. It was in line with EU directives in the water sector (especially UWWTD) and directly addressed EU and national priorities (e.g. in the Malta and Gozo sewerage masterplan). Poland – Sochaczew sewage management The project addressed a real need (lack of wastewater infrastructure in part of the city), which meant that rivers and soil (and indirectly groundwater) were affected by sewage discharge. It was consistent with strategic priorities set at European, national and local levels. Romania – Craiova sewerage network 5 The project aimed to address the lack of an operating WWTP, an incomplete sewer system and water leaks in the water distribution network. It was aligned with the strategic objectives of the Danube pollution reduction programme and with EU directives. Water Croatia – water supply and sewerage system 5 The project addressed a low connection rate to the water supply system, water leaks in the water distribution network, an insufficient wastewater network and the lack of a WWTP. It was in line with national, regional and local priorities, as well as EU directives. Italy – Favara di Burgio aqueduct 5 The project was highly relevant in addressing serious water management problems caused by an old and dysfunctional aqueduct that did not ensure a reliable water supply to municipal head tanks. It was in line with EU directives and directly addressed priorities established at national level (reorganisation of the water sector) and regional level (countering local water shortages). Spain – Aguilas desalination plant 5 The project tackled serious problems in water management: the existing plant was not operating at full efficiency and there was a severe water deficit and aquifer over‑exploitation. Water provision for irrigation was hampered, leading to land abandonment and economic slowdown. The project was in line with EU directives in the water sector and directly addressed national and regional priorities. Others Estonia – purchase of a multifunctional ship 5 The project addressed the lack of pollution‑removal capacity in Estonia’s Baltic waters. It was consistent with EU directives, national strategies and Baltic cooperation under the Convention on the Protection of the Marine Environment of the Baltic Sea Area. Slovenia – Celje waste management centre 4 Despite poorly defined goals, the project addressed real needs (lack of landfill space, failure to comply with the Landfill Directive and national law, lack of separate waste collection). It was in line with Slovenia’s 2004-2006 single programming document, which included the priority measures of national waste policy and was funded by the CF. Not all objectives were relevant for the phase II project, as they were more important for the waste management centre as a whole. France – Sète‑Marseillan lido protection 5 The project addressed natural risks (erosion, unregulated parking and camping, pollution linked to waste, noise pollution, accidents and congestion) and took account of ecological and economic factors. It was in line with EU and national recommendations on strategies to deal with erosion issues. Source: CSIL, Ramboll (2019).



4.3.2 All projects addressed real and significant environmental needs All 10 projects responded to a clearly identified need for intervention and addressed real and significant environment-related needs, while delivering a range of benefits. As regards the two water supply projects (Italy’s Favara di Burgio aqueduct and Spain’s desalination plant in Aguilas), new infrastructure was necessary to address water deficits resulting from resource‑management problems. When the Italian project was conceived, the existing aqueduct was in poor condition and suffered from high water losses, so a reliable water supply to municipal head tanks could not be ensured. The Sicilian water crisis (1999‑2004) exacerbated the need for immediate action. Before the Spanish project was implemented, water supply was not consistently ensured due to permanent water scarcity. This led to land abandonment and over‑exploitation of aquifers. An existing desalination plant was underperforming due to operational problems. In other cases, it was necessary to intervene to address severe environmental concerns and avoid potential future damage or destruction of the natural environment. A compromised environmental situation characterised the five projects (in Bulgaria, Malta, Poland, Romania and Croatia) featuring wastewater treatment infrastructure, all of which dealt primarily with the discharge of raw sewage into water bodies. The aim of the Estonian and French projects was to deal with environmental hazards that threatened to cause irreversible damage to natural resources. Seven projects addressing a water cycle, together with the Slovenian waste management project, were designed to ensure the sustainable provision of key services of general interest (e.g. water supply, wastewater collection and treatment, and waste treatment), which were necessary for civil and economic activities. In these cases, the ex ante situation differed depending on the sub-sector. In the water supply sector (projects in Italy and Spain, project components in Bulgaria, Romania and Croatia), services generally existed but were not satisfactory. As regards wastewater collection (projects in Bulgaria, Poland, Romania and Croatia and, to some extent, the Maltese project), no or only incomplete services were provided in the project areas. Wastewater treatment services were lacking in Malta and Croatia, and they needed to be upgraded in Bulgaria, Poland and Romania. The ten selected major projects remained on the whole relevant over time. All the projects addressed basic human needs, and the use and preservation of natural resources, that remain relevant over time. Evolving needs and consumption patterns were to a large extent taken into consideration during planning and implementation, partly through stakeholder consultation. This also guaranteed relevance over time. The Estonian project and the Slovenian waste management project both appear to be characterised by a trade-off between the requisite intervention and long-term relevance. While they were highly relevant at the moment of the financing decision (in the absence of other solutions), they may become less so over time with the adoption of measures to comply with further requirements. 4.3.3 Projects address not only environmental, but also economic needs While environmental concerns were the main drivers of all 10 projects, they made equally important contributions to economic development. However, unlike transport investments, which directly trigger economic growth, environmental projects contribute to economic development mainly by managing natural resources efficiently and providing basic environmental services in previously unserved areas. In most of the assessed cases, considerations relating to the efficient and sustainable management of natural and socioeconomic resources were never separate from purely environmental concerns. For instance, water is a key input for various economic activities and aquatic ecosystems deliver goods and services used by certain sectors (e.g. water‑related tourism, professional fishing). Also, specific projects aimed to deliver services to households. Accordingly, economic considerations can relate to the optimisation (in terms of efficiency, sustainability and quality) of the management of natural resources and the preservation of ecosystems. In several cases, economic goals featured among the projects’ primary objectives. For example, the area around Aguilas, where the Spanish desalination plant was built, suffered from land abandonment and general economic slowdown as a result of irregular water supply. The project was designed to address this problem in the long term. The Maltese WWTP was partly designed to promote tourism and economic development on the south-east coast, where they had been hampered by seawater contamination. Similarly, the protection of local economic activities was part of the rationale for the lido protection project in France. On a different note, the primary goal of the Italian project was to enhance water supply to municipal tanks. However, the project also generated economic effects, as the increased availability of water proved a significant incentive for entrepreneurs to establish their firms in the area. The selected projects did not involve zero‑sum trade-offs between environmental and economic considerations (e.g. the protection of natural resources, on the one hand, and the protection of jobs, on the other). As such, they were generally not controversial and enjoyed the support of economic operators and (most) citizens. 4.3.4 Compliance is a significant driver of project implementation The analysis reveals that compliance was a significant driver behind the implementation of major environmental projects analysed in this ex post evaluation, particularly as regards drinking water, wastewater treatment and waste management. However, it was never an objective per se. Rather, compliance with environmental standards was instrumental in the prioritisation of needs and investments, and the recognition of some as more pressing than others. ERDF/Cohesion Fund co‑funding for major projects is a means of ensuring the implementation of EU environmental legislation, particularly in EU‑13 Member States. The role of compliance as a driver for project prioritisation is a peculiarity of the environmental sector; it was not so evident in the transport sector, for instance (see EC, 2018). Environmental legislation is characterised by standards and requirements that are stricter than those in other infrastructure sectors. All five wastewater treatment projects (in Romania, Malta, Poland, Croatia and Bulgaria) were driven by the need to comply with EU accession criteria and the environmental acquis. However, it is important to stress that compliance considerations simply reflected the countries’ urgent need for such investments. Incomplete compliance with the requirements of the Landfill Directive (due to a lack of landfill space and separate waste collection to ensure the suitable treatment of municipal waste) was one of the main drivers for the Slovenian waste management project. As implied above, compliance with EU requirements was less a driver in the EU‑15. For example, it was not among the lead drivers for the Italian and Spanish projects (the two investments dealing only with water supply), which aimed to address failures in the provision of water for civil and agricultural uses respectively. However, the projects still had to be in line with the WFD and the DWD.

The Estonian project was slightly different, in that a major driver was compliance with HELCOM recommendations, in line with the Convention on the Protection of the Marine Environment of the Baltic Sea Area. Further legal parameters stemmed from national strategies and EU directives (Marine Strategy Framework Directive, Birds Directive and Habitats Directive). 4.3.5 Projects are aligned with EU and national priorities As required by the applicable legal provisions, all the projects had to be included in an OP and screened against the OP’s objectives before acceptance. Most were part of wider strategic sectoral plans. Major projects must be strategically aligned with national and EU objectives to be eligible for EU funding. This requirement has been further strengthened for the 2014-2020 period, with the introduction of specific ex ante conditions. Analysis confirms that most of the water and wastewater projects were developed as part of an integrated network, so they were covered by wider local, regional or national plans. For example, the Italian aqueduct was part of a wider strategic approach aimed at countering the water emergency through the joint management of all major Sicilian aqueducts. Similarly, the WWTP in Malta was part of a broader strategy to enhance wastewater connection and treatment in the country. All 10 projects were found to be consistent with EU and national and priorities. Alignment with regional priorities played a role in the two projects (in Italy and Spain) dealing only with water supply, while local priorities were particularly relevant for the projects (in Bulgaria, Poland and Croatia) dealing with wastewater treatment. Specific policy frameworks apply to the two projects with cross-border (including extra‑EU) effects, which had to comply with an additional layer of provisions: -the Craiova sewerage network project in Romania was designed and implemented in alignment with the Danube pollution reduction programme and the strategic action plan for the Danube river basin, develop