EUROPEAN COMMISSION Brussels, 28.4.2021 SWD(2021) 86 final COMMISSION STAFF WORKING DOCUMENT Synthesis of the findings of the evaluations of European Structural and Investment Funds Programmes 2014-2020 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS European Structural and Investment Funds 2014-2020



2020 Summary report of the programme annual implementation reports







covering implementation in 2014-2019 {COM(2021) 213 final}



1. INTRODUCTION 2. COHESION POLICY PROGRAMMES 2.1. Evaluations completed by the Member States 2.1.1. General overview 2.1.2. Main findings of evaluations 2.1.3. Support to Member States 2.2. Evaluation work by the Commission 2.2.1. ERDF/CF 2.2.2. ESF/YEI 3. EUROPEAN AGRICULTURAL FUND FOR RURAL DEVELOPMENT (EAFRD) 3.1. Evaluations completed by the Member States 3.1.1. General overview 3.1.2. Main findings of evaluations

3.1.3. Support to Member States 3.2. Evaluation work by the Commission 4. EUROPEAN MARITIME AND FISHERIES FUND (EMFF) 4.1. Evaluations completed by the Member States 4.2. Evaluation work by the Commission 5. FINAL REMARKS



1.INTRODUCTION This staff working document provides an overview of the evaluations carried out by the Member States and the Commission on the programmes supported by the European Structural and Investment (ESI) Funds and implemented under shared management to pursue the objectives of the EU strategy for smart, sustainable and inclusive growth. It accompanies the 2020 summary report of the programme annual implementation reports covering implementation in 2014-2019. The Common Provisions Regulation for 2014-2020 requires that evaluations are carried out to improve the quality of the design and implementation of programmes, as well as to assess their effectiveness, efficiency and impact. Member States are required to carry out evaluations for each programme and ensure that each evaluation is duly followed up. At least once during the programming period, an evaluation must assess how support from the ESI Funds has contributed to the objectives for each priority. Evaluation is therefore one of the main tools to measure how the respective policies have performed. Its findings add to the evidence on the investments supported in order to improve the policy and ensure that the actions supported deliver the best possible results. The evaluations carried out by the Member States may assess whether targets have been achieved (i.e. monitoring/progress-oriented evaluations), how funding has been managed, how projects have been selected and run (i.e. procedure/implementation-oriented evaluations), or the outcome and effects of interventions (impact-oriented evaluations). Some of the above-mentioned parts could be combined in a single evaluation. Impact evaluations are carried out later on in the programming period, once the projects and measures implemented have produced enough results to be assessed. Their number has been growing at a faster rate in the past year. However, given the broad thematic and territorial scope of the measures supported by the ESI Funds, Member State evaluations are designed to address local context and evaluation needs. As a result, the reported findings are mostly specific to the context of the analysis. Thus, in many cases the findings cannot be aggregated at the EU level and do not allow for cross-cutting conclusions. This document synthesises the main elements of the findings from all types of evaluations. This can lead to better implementation of the current programmes and provide useful insights for policy learning, also for helping prepare the programmes for 2021-2027. The following sections present an overview of the main findings from the evaluations of the ESI Funds programmes carried out by the Member States and the Commission. These focus mostly on the past year, organised by policy area. Section 2 covers cohesion policy programmes, Section 3 is devoted to rural development policy supported by the European agricultural fund for rural development (EAFRD), and Section 4 focuses on the European maritime and fisheries fund (EMFF). Section 5 contains some concluding remarks. 2.COHESION POLICY PROGRAMMES Evaluation is key to measure how cohesion policy is performing. Cohesion Policy is one of the most evaluated EU policies. This is thanks to the combined effort of the Member States and the Commission to monitor and analyse the results of the policy under the rules of shared management and thanks to the long-established importance of evidence for policy learning. This section provides an overview of the evaluations of actions funded by operational programmes supported by the European Regional Development Fund (ERDF), the Cohesion Fund (CF) and the European Social Fund (ESF) including the Youth Employment Initiative (YEI). 0.

1. 1.0.Evaluations completed by the Member States 1.0.0.General overview The evaluation systems across the Member States differ in terms of maturity. Where there was no established and systematic use of evaluation to support national policies, cohesion policy has helped develop a culture of evaluation, starting from the planning and implementation of operational programmes. Participation in cohesion policy has helped evaluation propagate to national legislation and policies, increasing the efforts of the Member States to pursue an evidence-based approach to policy-making. To help the Member States comply with the regulatory requirements on the evaluation of cohesion policy, the Commission has provided considerable evaluation guidance and many other forms of support to Member States since the start of the programming period, as detailed in Section 2.1.3. As a result, more than 1 400 evaluations carried out by Member States have been published since January 2015, of which over 1 000 referred to the 2014-2020 programming period. This represents the great efforts and commitment of the Member States to generate evidence for policy learning in line with the stronger result orientation of the policy. There has been a material increase in the evaluation effort compared to the 2007-2013 programming period, when around 800 evaluations were produced within a comparable timeframe. The increase in the number of completed evaluations has generated valuable knowledge about the implementation of the programmes and their effectiveness. Many of the completed evaluations mostly focus on finding out what the outcome of an intervention was rather than on developing an understanding of how and why it was achieved. For a fully-fledged impact evaluation, this is a crucial element to investigate in order to improve the way the intervention works and its effectiveness and to exploit the full potential of evaluation for policy learning. This reflects the complexity of the task of evaluating the impact of cohesion policy, especially when trying to disentangle the effects of the measures funded from those of other things happening at the same time. These include changes in the socio-economic context in which the measures are implemented or other policy measures in operation at the same time – not only in the area concerned, but also in other areas that can affect this. They also include spontaneous changes in behaviour, which by their nature may be hard to observe. Taking into account all the relevant factors is extremely challenging, especially in a non-experimental context where data on these factors and on how they interact with each other are imperfect and, in some cases, do not exist – not only because of a failure to set up the means to collect them, but also because a lot of information are intangible. It is therefore important to have realistic expectations of evaluations and to focus on the lessons learned for further improvement. Impact-oriented evaluations are well designed in general, and some apply advanced techniques, e.g. counterfactual methods, to distinguish the effects of the measures examined from other factors. Case studies are frequently used to investigate the effects of specific instances of the measures supported and their outcomes. While the majority of evaluations make serious efforts to go beyond the data readily available (such as data from monitoring systems or official statistics), the lack of adequate data and particularly access to administrative data remains a common challenge faced by evaluators. While impact evaluations are very important, process and monitoring evaluations carried out during implementation provide evidence that can be used immediately in programme management in order to steer operations in the right direction. They can also be used to supplement the data available from local monitoring systems. Completed evaluations Cohesion policy evaluations completed and published since January 2015 relate either to the 2014-2020 programming period or to the previous one (2007-2013), or in a small number of cases to both. While the majority of evaluations refer to 2014-2020, those covering 2007-2013 programmes can still provide useful findings that are relevant for the managing authorities of the ongoing programmes. Overall, the number of evaluations has continued to increase in the past year at a faster rate than in previous years, up from less than 1 000 to over 1 400. The distribution of the evaluations across the funds has not changed in the past year, with a slight prevalence of ERDF/CF over ESF/YEI and around 20% of the evaluations covering both. Funds 2007-2013 Both periods 2014-2020 Total ERDF+CF 202 24 407 633 ESF+YEI 82 12 439 533 Multi-fund 62 10 192 264 Total 346 46 1 038 1 430 of which in the past year ERDF+CF 19 10 176 205 ESF+YEI 3 3 173 179 Multi-fund 1 4 85 90 Total in the past year 23 17 434 474 Source: Commission evaluation helpdesk for cohesion policy The number of evaluations completed so far varies a lot across the Member States. However, the number of evaluations alone may not be representative of the evaluation effort made or of evidence collected, since evaluations vary widely in terms of their scale and scope (e.g. number of operational programmes covered, priorities, themes, etc.). For example, more than 20 evaluations have already been carried out in Poland on the Smart Growth operational programme, many of them very narrowly focused on support for particular sectors. This contrasts with the situation in other countries such as Portugal, where sectoral evaluations in many cases cover all support across multiple programmes and over both programming periods. Source: Commission evaluation helpdesk for cohesion policy

Types of evaluations Around one third of the evaluations are impact-oriented, i.e. they aim to assess the outcomes and results of the programmes, while the others look into procedural aspects or progress towards achieving the targets set. The overall number of evaluations aimed at assessing impact is smaller than implementation-oriented and progress-oriented evaluations. This is mainly because impact evaluations are normally carried out later in the programme cycle. These evaluations aim to analyse the effectiveness, efficiency and impact of the action supported and should be undertaken only once a critical number of projects have been completed and have produced results. Among impact-oriented evaluations, only one third assess exclusively the impact of the interventions, while the other two thirds also look at how the actions were implemented and/or monitor progress towards the targets. As expected, the number of impact evaluations has continued to rise. In the past year there were 161 impact evaluations, the highest value recorded in a year since systematic collection started in 2015 and more than twice as many as in the previous year. For 2014-2020, the share of impact-oriented evaluations reached 20% and is expected to grow further when more results emerge following implementation. There is significant variation between the funds, with a higher share of impact-oriented evaluations for ESF/YEI (26%) than for ERDF/CF (18%) and multi-fund programmes (12%). This is mostly due to the different types of interventions supported by the different funds and their respective time for completion. Source: Commission evaluation helpdesk for cohesion policy Most evaluations combine relatively simple qualitative methods with quantitative ones. In some cases, more sophisticated methods are used in order to assess the results and outcomes of the programmes. Around half of impact evaluations use more advanced research methods, such as counterfactual evaluation techniques, theory-based techniques and cost-benefit analysis. Theory-based methods are more frequently applied in ERDF/CF evaluations, while counterfactual techniques are more often used for ESF/YEI and multi-fund evaluations. Thematic distribution The distribution of the evaluations is spread unevenly across thematic objectives because of the different nature of the actions funded and their financial weight. Figure 3 below shows the thematic coverage of evaluations for the 2014-2020 programming period and impact-oriented evaluations referring to the previous period. The next section summarises the main findings. More than half of the evaluations cover multiple thematic objectives, which are reported under each of them. However, the chart does not include evaluations that broadly address a large number of thematic objectives. Source: Commission evaluation helpdesk for cohesion policy The largest number of evaluations are on the thematic objectives (TOs) related to inclusive growth, i.e. employment (TO8), social inclusion (TO9) and education (TO10), where operations supported by the ESF and YEI are typically implemented faster and generate results in a relatively short time. 1.0.1.Main findings of evaluations The findings summarised below are from the evaluations of 2014-2020 programmes that assessed impact or procedural aspects, with the results likely to be more robust depending on the information on the data used and the methods applied. They also include findings from impact evaluations referring to the 2007-2013 period, where they are still relevant for the current programmes. ·Smart growth: research & innovation (R&I), information and communication technology (ICT) and SME competitiveness For the 2014-2020 programming period, the ERDF is allocating €135.8 billion to smart objectives covering research, technological development and innovation (TO1), ICT (TO2) and SME competitiveness (TO3). Close to 40% of the allocation had already been paid to project beneficiaries by the end of 2019, against around 25% in 2018. This rapid increase, which is set to continue in 2020, is reflected in the change in the number of evaluations focused on the impact of TO1 and TO3 investments over the past year. This shows that R&I projects and support to small and medium-sized enterprises (SMEs) are now producing noticeable results on the ground in many EU regions. The number of TO2 evaluations remains low, as implementation progress is slower than for other TOs (27% in 2019) and funding is comparatively smaller. ØThematic Objective 1: Research & innovation In 2019, investments under TO1 made significant progress towards reaching their targets on new cooperation between enterprises and research institutions and on the commercialisation of new products by supported enterprises. Evaluations provide detailed insights into the performance of programmes and projects in these domains. In Tuscany (IT), the success rate of projects is high, with around 90% of them resulting in the production of prototypes ready to be sold. Evaluations also show that projects that aim to step up cooperation between enterprises and research institutions contribute to overarching objectives. In Friuli-Venezia Giulia (IT) for example, EU-funded projects have helped increase R&D expenditure relative to GDP. TO1 investments also help improve research infrastructure, as in Finland, and to expand research areas in supported research centres, as is the case in Italy.

In some Member States, the manufacturing sector and knowledge-intensive services predominate in the areas where TO1 investments are undertaken, with the effect of directing substantial support towards large firms (especially in Poland and Italy). However, while EU support in Italy contributed to a broader scheme also funded from the national budget, in Poland this led to some deadweight costs, which is also the case when SMEs already undertaking research are given funding. ØThematic Objective 2: Information and Communication Technology Investments in ICT cover various intervention fields. Although the overall implementation of TO2 objectives is lagging behind other thematic objectives, significant progress has already been made in the specific field of broadband coverage, with implementation reaching 42% of the target by the end of 2019. Evaluations covering both 2007-2013 and 2014-2020 analyse in detail the tangible outcome of this intervention field. In Poland, broadband coverage has clearly helped young people and households in difficult socio-economic situations to acquire digital skills. In Spain however, large differences remain between the main urban centres and rural areas, despite a considerable increase in broadband coverage. This demonstrates that broadband coverage alone is not always sufficient to close the digital gap. Evaluations in Poland and Sweden have shown that the prior level of digital skills on the ground is a major factor in the success or failure of certain ICT-related projects. Evaluations of other TO2 investment types show that funded projects have increased the use and quality of e-services in the public sector and helped develop new ones. In Italy, new digital services addressed at enterprises have also helped reduce administrative costs. In Czechia, the introduction of ICT in enterprises has improved workers’ skills and has made it easier to access new markets and partners. ØThematic Objective 3: SME competitiveness Implementation reports show a rapid increase in the number of supported companies, from 40% of the planned achievement in 2018 to 55% by the end of 2019. Evaluations have confirmed that most of them either have grown in terms of employment, productivity and market share or consolidated their level of activity, also thanks to the use of financial instruments. Supported SMEs have boosted their internationalisation and increased their range of services and products. For instance, business incubators in Romania have helped firms strengthen their position on the market, diversify products and improve their quality. Business creation interventions in France have also led to a marginally higher survival rate of the firms supported, despite implementation delays. In Poland, support under the Smart Growth operational programme seems to have benefited micro-enterprises more than other SMEs and helped promote innovation, although the allocation of resources could be improved with different selection criteria for the projects. Under regional operational programmes, EU support for Polish SMEs led to employment growth 4 percentage points higher than that of unsuccessful applicants. Nevertheless, the length of the assessment procedure, together with the administrative burden, are reported to be common obstacles to a broader take-up of support. ·Sustainable growth: low-carbon economy, climate change, environment, transport and energy networks The ERDF and the Cohesion Fund are allocating €171.2 billion to sustainable growth, covering investments in the low-carbon economy (TO4), climate change adaptation and risk prevention (TO5), environment protection and resources efficiency (TO6) and in network infrastructures in transport and energy (TO7). By the end of 2019, spending amounted to over €57 billion (about 33% of the planned total, against 22% in 2018). By the end of 2019, implementation progress was particularly advanced for investments under TO7 (network infrastructure), while it remained below 30% for the other TOs. Nevertheless, in many cases, the results of investments do not appear immediately after their implementation, leading to a limited number of impact evaluations at this stage. ØThematic Objective 4: Low-carbon economy In 2014-2020, 25% of the ESIF must be spent on projects with climate action objectives (27% for the Cohesion Fund and 17% for the ERDF). With €24.1 billion allocated to climate action, investments under TO4 contribute the most to these objectives. Funded projects tackle energy supply or energy efficiency. Evaluations review the different types of action that help realise those objectives on the ground and show the difficulties and opportunities that each of them entail. Projects aimed at reducing the CO2 emissions of companies, besides indisputable environmental benefits, have led to long-term cost savings and increased energy efficiency. However, the overall impact has been limited because of the small proportion of firms supported, especially in Italy and Germany. Likewise, energy efficiency measures for public buildings generally proved to be successful in Germany, Czechia and France. The large scale of the projects, which makes it easier to manage administrative costs, partly explains these positive results. In terms of implementation issues, one evaluation found that financial instruments under the Polish infrastructure and environment operational programme have been an appropriate form of support for increasing both the renewable energy supply and energy efficiency. However, other evaluations in Germany, France and Czechia found that complex and evolving regulations, in particular for housing renovations, lead to difficulties in project implementation. For example, in Czechia the inability to combine the ERDF with other support for renewable production limited the number of projects implemented. In Austria, access to credit remains an obstacle for small firms investing in projects to reduce CO2 emissions. ØThematic Objective 5: Climate change adaptation and risk prevention Comparatively smaller amounts allocated to this theme, together with a relatively slow implementation rate, limited the number of TO5 impact evaluations focused on 2014-2020 (the lowest number of all thematic objectives). However, investments in flood protection measures were already showing tangible results by the end of 2019, with 7.5 million people seeing their exposure to flood risks decrease thanks to ERDF/CF investments (representing 45% of the EU target). Evaluations show that flood protection measures included in particular strengthened cooperation between services to increase preparedness and ensure sufficient capability for when disasters strike, particularly in cross-border areas. Continued collaboration between Member States has led to increased capacity to plan and implement measures on climate change adaptation. Likewise, evaluations in Italy and Hungary showed that EU support focused not only on building infrastructure, but also helped to improve monitoring and control, notably through the implementation of ICT tools (particularly in the 2007-2013 programming period). ØThematic Objective 6: Environment protection and resource efficiency Rapid progress between 2018 and 2019 in the implementation of key indicators such as the number of people benefiting from an improved water supply or habitat areas supported to achieve a better conservation status indicates that TO6 investments are now being completed on the ground in many EU regions. In many evaluations, investments that aim to improve wastewater treatment were found to be successful in preserving natural habitats, promoting biodiversity and creating jobs. This demonstrates the virtuous circle that investments in resource efficiency measures can trigger. EU-funded interventions related to cultural facilities and natural heritage have also helped increase visits to the sites concerned in Poland, France and Germany, even though substantial funds were directed in some cases to areas, where the tourism potential was already high.

Evaluations in Austria and France highlighted that the nature of TO6 investments require the involvement of many organisations and authorities, which has helped increase cooperation between them. Nevertheless, delays in programme implementation have been reported by evaluations in different Member States. These are often linked to the governance of the measures concerned. ØThematic Objective 7: Network infrastructure in transport and energy TO7 is the thematic objective that benefits from the highest share of ERDF/CF support. Evaluations have drawn lessons from the 2007-2013 period and identified preliminary findings for the 2014-2020 period. They go beyond the amounts invested in the Trans-European Transport Network (TEN-T) or in urban or maritime transport by showing the positive side effects and the challenges of TO7 interventions. TEN-T investments often rely heavily on ERDF and CF funding, as is the case in Romania, demonstrating the EU added value in major transport projects. Investment in the rail network has helped reduce travel time and increase the number of passengers, sometimes getting them off the road like in Spain, despite declining ERDF support for transport over the years. Similarly, TEN-T investments in the motorway network in Poland have led to the development of an integrated transport system, increased accessibility and reduced air pollution and road noise. It has also improved road safety, with at least a 50% reduction in road accidents on the sections targeted by the investment. From an economic point of view, estimates show that it only takes 3 to 5 years to recoup investment costs in the shape of benefits for businesses. However, core road and rail links often fail to cover rural areas and secondary cities, as in Czechia. It is worth noting that robust findings on TEN-T investments during the 2014-2020 period remain limited so far, as the concrete results of these projects often take years to emerge. Investments in urban areas in Italy, Poland and Romania have made connected territories more attractive, which has benefited the businesses located there. In Italy, businesses’ financial gains from new transport connections were found to compensate for the losses they had suffered during construction. ·Inclusive growth: employment, social inclusion and education Cohesion policy funds, together with the Youth Employment Initiative (YEI), plan to invest over €152.5 billion in intervention fields related to inclusive growth, with the bulk of funding coming from the ESF. Inclusive growth investments target employment (TO8), social inclusion (TO9) and education (TO10). By the end of 2019, overall Cohesion Policy spending was close to €62.1 billion and reached about 41% of the planned total (around 29% in 2018). This rapid increase continued in 2020. Moreover, preliminary results of investments related to inclusive growth, and in particular investments in quality employment, can be observed as early as a few months after implementation, making it easier to assess their effects and impact earlier in the programming period. ØThematic Objective 8: Sustainable and quality employment By the end of 2019, 4.5 million people had found a job after receiving ESF and YEI support. A large number of evaluations analyse the implementation and impact of investments that support sustainable and quality employment in order to provide a detailed picture of what this number means in practice. In Italy, Portugal and the United Kingdom, evaluation findings have shown a strong correlation between traineeships and increased employability. Project participation, especially in cases where long-term and personalised support is offered, as in Germany, correlates with increased personal and job-related skills. There is also evidence of EU added value as the ESF contributes to the sustainability of services, which would be financially unstable otherwise. However, there is also evidence from evaluations in Italy and Lithuania that employment found after participating in ESF-funded projects is not always stable or remunerated at the average national level. In some regions, differences between the employment levels of participants and non-participants seem to dissipate over time, which indicates that positive outcomes may have a limited duration. In a small number of cases, interventions suffered from communication gaps, reaching out to participants and keeping them engaged can be a challenge, as can be the creation of effective networks for all organisations involved in a project. ØThematic Objective 9: Social inclusion By the end of 2019, 2.5 million participants with disabilities, 5.6 million migrants, participants with a foreign background or minorities and 6.5 million other disadvantaged people had received help from TO9 interventions to improve their employment opportunities and develop the right skills for the jobs market. In addition, ERDF support allowed 39 million people to benefit from improved health services across the EU. The primary focus of TO9 evaluations for 2014-2020 is on labour market integration and the social inclusion of disadvantaged people. Notable positive results are from projects in which there is an emphasis on regular contact between coach and participant. In Germany and the United Kingdom, evaluations demonstrate that assessments of individual needs tend to produce better results both in terms of soft skills developed and hard outcomes related to employment. In Germany, the need to organise and coordinate TO9 interventions has been a reason for forming collaborations between organisations and services, which has led to the creation of networks of organisations previously not in contact. Evaluation findings also highlight increased engagement – the availability of funding and learning opportunities has encouraged first-time social entrepreneurs to create ventures and has made some people more proactive in terms of local policy-making and volunteering. In France, the complementarity of TO9 interventions funded by the ESF with actions funded by the Asylum, Migration and Integration Fund (AMIF) is ensured via regular and structured exchange of information between services. AMIF support helped orientate migrants towards services funded by the ESF for their integration. On the other hand, many actions continue to focus on social inclusion through employment; while this is in itself helpful, it does not always address other social barriers faced by participants. In addition, a German evaluation showed that some measures aimed to higher target achievement rates excluded those who were hardest to reach and help, leading to ‘creaming’ effects (i.e. selection of participants with higher potential rather than those in higher need). ØThematic Objective 10: Education and vocational training By the end of 2019, 17.8 million low-skilled people were helped, 5.5 million had gained a qualification and 1.8 million were in education and training. In addition, ERDF investments in education infrastructure had allowed more than 15 million schoolchildren and students to benefit from an improved studying environment by the end of 2019. TO10 evaluations in the current period mainly focus on early school leaving, general education, continuing vocational training, and the employment results following training. Evaluations of TO10 interventions in Germany, Poland and the United Kingdom have found that practical forms of education such as traineeships and job counselling have a clear positive impact on the employment prospects of participants. There were further positive results in Austria, where promoting communication networks that include students, teachers and families can reduce education dropout rates. Language classes and social counselling offered to migrants can improve the rates at which they integrate in their local community, as in Germany. Evaluations have shown that the courses provided are generally in line with the needs of the local labour markets. However, some groups have lower access to education and training interventions, such as learners with special needs in Estonia and rural dwellers in Lithuania. Service provision is also more challenging in regions with very heterogeneous participant groups, as highlighted in one evaluation in Germany. Moreover, some interventions in Hungary were perceived as being brief and not providing sufficient knowledge to make a lasting impact on participant skillsets.

·Strengthening institutional capacity and efficient public administration ØThematic Objective 11: Efficient public administration With €6.2 billion allocated under the ESF and the ERDF, interventions in efficient public administration benefit from less funding than other TOs. By the end of 2019, over 2 300 projects targeting public administrations or public services at national, regional or local levels had received support from the ESF. TO11 evaluations underline the – often-intangible – results that such actions bring up. They show that TO11 interventions have increased awareness among public authorities on good governance principles, such as the exchange of information, transparency, coherence of interventions and prevention of conflicts of interest during project selection. In Latvia, investment in public administration has helped public authorities better understand how to promote entrepreneurship and the sustainability of SMEs, and the training supported has strengthened inter-institutional cooperation in investigating complex corruption, economic and financial crime cases. In Slovenia, TO11 investments have helped strengthen non-governmental organisation (NGO) networks and connect them with public authorities. However, results have been affected by the limited capacity of NGOs to absorb funding and a lack of coherence between projects and NGO needs. An assessment of the evaluation function of cohesion policy in central and eastern European countries shows that evaluation results are not made sufficiently accessible and that the effects of the measures evaluated should be disseminated better. In addition, evaluation units are often constrained by limited resources. ·Cross-cutting instruments ØTerritorial instruments: ITIs, CLLD and sustainable urban development In the current period, Member States have produced 42 evaluations covering integrated territorial development approaches: 18 focused on integrated territorial investments (ITIs), 6 on community-led local development (CLLD), 3 on both ITIs and CLLD, and 15 on sustainable urban development (SUD) carried out under a specific operational programme or priority axis. The evaluations have mainly examined the governance, procedures and implementation aspects of the instruments. They are based on a mix of quantitative and qualitative methods and most of them include case studies. Evaluations have highlighted positive findings. In general, ITIs seem in line with the needs identified at local level – they favour the development of bottom-up approaches, facilitating the exchange of information, knowledge and experience. In Poland, ITIs have benefited small municipalities partnering with larger ones, helping them overcome their financial and organisational limitations. Cooperation between different regional and local interest groups and public participation in local development strategies was a key success factor of CLLD in Austria, Poland and Romania; in the latter, they could be even more effective if concentrated in specific neighbourhoods and communities concerned. In Sweden, SUD strategies have focused on job creation and innovation, with collaboration between various stakeholders in the projects a key factor for success. On the other hand, managing and monitoring the contributions of different funds and programmes has proved to be difficult and burdensome in terms of administrative capacity in Czechia and the Netherlands. Too many organisations were involved at the different levels in Italy and Hungary, with unclear allocation of tasks at local, regional and national level. Estonia experienced difficulties in integrating projects and different types of measures in a specific territory. Evaluations have also identified a recurring need for timely preparation and adoption of strategies; in addition, they have emphasised that the authorities implementing projects under the SUD strategies need to have staff with administrative, legal and managerial skills adapted to the types of actions undertaken. ØFinancial instruments Several evaluations have addressed the use of financial instruments. They show that these can be used to help companies grow in the early phases of their development. In Germany, financial instruments helped strengthen business competitiveness and increase production and profits. Similarly, under the Polish Smart Growth operational programme, a vast majority of beneficiaries considered that such support was a significant factor in encouraging them to innovate. However, for this operational programme, it was also observed that restrictions on eligible expenditure limited the applications for financing. While the diversity of financial instruments available should constitute an opportunity, evaluations found that in some cases it led to increased complexity that affected the take-up of the instrument. This was the case in Czechia, where the use of financial instruments to fund cohesion policy expenditure is fragmented due to the high number of instruments available at EU, national and regional level. In addition, there are sometimes overlaps with other types of funding, such as grants. ØEuropean territorial cooperation The vast majority of the evaluations carried out under the European territorial cooperation (ETC) goal cover multiple thematic objectives, although they may not necessarily represent the themes in which Interreg programmes invest most. Positive outcomes are reported in the area of cooperation where stakeholders, especially those pertaining to the category of public authorities, are expected to have a long-lasting collaboration even after the project has ended. This is also confirmed by the findings of some evaluations, which point to the fact that partnerships formed during the 2007-2013 programming period also applied for financing in 2014-2020. Partnerships in environmental projects tend to become permanent networks, as observed for example in the Interreg V-A Sweden-Denmark-Norway programme. There is also evidence that the use of the small projects fund to boost social integration across borders under the Interreg V-A Poland-Germany/Saxony programme strengthened cross-border local communities and cooperation among partners. On the other hand, some evaluations report that there is still some room for improving cooperation between the bodies involved in implementing cooperation programmes (managing authorities and joint secretariats). In general, ETC programmes help create a specific framework that makes collaboration easier between stakeholders across countries, which would otherwise be more difficult to attain. The most frequent topics where cooperation has improved are on environmental issues, management of maritime areas and management of natural and cultural heritage. The exchange of knowledge and good practices (via training activities, events, networking platforms etc.) remains an important and successful activity in cooperation programmes. While the evaluations point the fact that ETC programmes helped reduce cultural barriers, the impact of the programmes so far in terms of language barriers is less visible, especially for programmes where the languages of participating countries belong to different etymological backgrounds. Another finding put forward by the evaluations relates to the simplification measures implemented in the 2014-2020 programming period, the most notable ones being those on the electronic exchange of data for project application and monitoring. While there has been a positive impact in several programmes, the conclusion of the evaluations indicate that there are still improvements to be made, especially in terms of user-friendliness and interoperability of the systems in place. There were also positive outcomes for the tourism sector, especially in the maritime areas covered by the cooperation programmes, as reported for example by the Interreg V-A Italy-France programme.

Positive outcomes were also reported in the area of research and innovation. By the end of 2019, 17 500 enterprises and 3 600 research institutions had participated in cross-border, interregional or transnational research projects. For example, the impact evaluation of the Northern Periphery and Arctic Programme shows that projects have achieved closer cooperation between SMEs and R&D centres, increased skills in SMEs and improved the capacity for digital service delivery.



·Horizontal findings on implementation Besides assessing the effects of the actions supported, reported above by thematic objectives and cross-cutting instruments, the largest share of evaluations aim to assess progress and implementation. These evaluations provide valuable evidence on the way that programmes are being carried out, their consistency with the initial objectives and relevance in respect of development or social needs, as well as issues which they are encountering. They ultimately help managing authorities identify and correct the weaknesses that might emerge during implementation. Implementation-oriented evaluation carried out in the Member States were found to have clear and simple objectives. Their underlying evaluation questions are relatively straightforward, as is the evaluation design, and they generally rely on multiple sources (surveys, monitoring data) to ensure that findings are robust. They mostly confirm the strategic nature of the projects selected and progress in line with targets, considering that they were undertaken before the pandemic. However, some difficulties emerged from evaluations that investigated the processes behind programme implementation. Several evaluations reveal significant delays in programme implementation. This is due to a number of different factors, including uneven capacity or limited stability within regional and local authorities in Estonia and Italy, long tendering processes in Czechia and Slovakia and difficult cooperation among major stakeholders in Bulgaria. Delays are also related to the implementation of complex projects, especially infrastructure ones, that caused unexpected difficulties. Other difficulties observed include inadequacy of funding and lack of cooperation between the bodies involved in implementation. Overlap with national policies could produce displacement effects and limit the take-up because of more favourable alternative measures. According to many evaluations, administrative issues can limit potential beneficiaries from participating in the actions funded by the programmes as well as outcomes. The most common issues are overly complicated and lengthy application procedures, changes in management procedures, difficulties in understanding and applying the simplified cost method, and lack of adequate information and sufficient financial incentives. In some cases, the demanding nature of innovative projects deterred potential applicants. A number of evaluations reported issues with project selection procedures – sometimes unclear and too complex for applicants, or too long to complete in the case of some support schemes for enterprises. In a limited number of cases, the selection criteria were deemed as not suitable for selecting the best projects. In a few cases, weaknesses were identified in monitoring and information systems, which are a primary source of data for evaluations as well as for tracking progress in implementation. These range from the timeliness of data collection to incompleteness or inadequacy of the data collected. 1.0.2.Support to Member States The Commission is committed to raising awareness of the importance of evidence-based policy implementation and design within Member States. This will help them improve their knowledge and administrative capacity in the area of cohesion policy evaluation. It manages the evaluation network and the evaluation partnership, which are platforms to discuss cohesion policy evaluation with the Member States on ERDF/CF and ESF/YEI respectively. Topics include methodological guidance, data and indicators, progress on ongoing evaluations and evaluation results and their use. On these topics, Member States exchange experience and good practice to strengthen evaluation capacity throughout the EU. The Commission’s cohesion policy evaluation helpdesk is a dedicated service that helps Member States with evaluation issues. The evaluations carried out in the Member States since 2015 on programmes supported by ERDF/CF and ESF/YEI programmes are systematically collected, analysed and summarised. Summaries of the main findings of the Member States’ evaluations, including a description of the data and methodologies used, as well as the full evaluation studies, are available in the evaluation library published on the Inforegio website. The ESI Funds open data platform also publishes structured data on all the evaluations, together with their summaries. The evaluation helpdesk has also strengthened its support to the Member States on specific methodological issues, following periodic calls for interest launched by the Commission. Expert reviews have provided valuable feedback on the evaluations carried out by national and regional authorities. They can involve assessing full evaluation dossiers (terms of reference, consultation strategies, data collection and use, interim reports, final reports, etc.) or be limited to one or more parts of dossiers. Evaluations do not need to be completed in order to be reviewed. This expert review scheme allows the Commission to provide for timely and tailor-made methodological support to the Member States. In addition, the Commission organises summer schools for managing authorities and other public bodies involved in the evaluation of ERDF/CF programmes, with practical -training on topics selected by the Member States ahead of the training. The latest edition was held online and used concrete examples of evaluations provided by participants on research, technological development and innovation (TO1), support to SMEs (TO3) and low-carbon economy (TO4). As evaluations depend on good monitoring systems, the Commission established the ESF Data Support Centre in March 2020. This centre provides a forum for expert discussions and mutual learning between managing authorities on monitoring issues. It is also a point of contact for Member States to submit methodological queries related to monitoring and reporting requirements for ESF+ programmes for the 2021-2027 programming period. A searchable database with methodological questions and their corresponding answers is available in all EU official languages, under the Shared Fund Management Common System (SFC). This database is updated with new questions on an ongoing basis in order to facilitate programme negotiations. 1.1.Evaluation work by the Commission 1.1.0.ERDF/CF In February 2020, the Commission published two staff working documents presenting the ex post evaluation of major projects supported by the ERDF and the CF between 2000 and 2013. The first document focused on transport and the second on environmental infrastructure projects. The two evaluations confirmed the importance of EU support for these projects in achieving EU objectives. Project quality at entry, project governance and project management proved to be the main determinants of success, while forecasting capacity emerged as a problematic determinant of performance. By supporting large-scale transport projects, cohesion policy has made a significant contribution to improving connectivity at EU and national level and has encouraged the development of sustainable transport. Investments have improved accessibility in the countries and regions concerned and paved the way for increased trade with the rest of the EU, which is vital for economic development. Large-scale environmental infrastructure projects have made a major contribution to compliance with EU law and broader EU sustainable development strategies. The staff working documents also highlighted the EU added value of these projects – the availability of a significant proportion of EU funding was critical for enabling or accelerating implementation, and therefore achieving the objectives. The Commission also made significant progress on its ex post evaluation of investment projects in research and technological development (RTD) infrastructures, competence centres and activities financed by the ERDF in 2007-2013, where €17 billion was invested during that period. The evaluation analyses the factors that contributed to the success or failure of the actions and investigates the mechanisms of change under different socio-economic conditions. Its methodology is based on the use of detailed data on projects and beneficiaries, which helps provide a better view of the impact of the funds and will also serve as a model for the 2014-2020 ex post evaluation. Project- and beneficiary-level data collected during the evaluation are published on the ESI Funds open data platform. The aim is to reach the wider public and for the data be freely available for further analysis and research. The final report of the evaluation is expected in the second half of 2021.

In 2020, the Commission launched an evaluation of e-Cohesion, the electronic data exchange system to make it easier to exchange documents and data between ESI Funds beneficiaries and the relevant authorities, in place for 2014-2020. To allow the managing authorities to incorporate any changes into their systems, the Commission aims to deliver the findings of this evaluation at an early stage of implementation of the 2021-2027 operational programmes. Under Article 57 of the Common Provision Regulation, the Commission will carry out an ex post evaluation of the ESI Funds, to be completed by the end of 2024. At this stage, it is currently considering how best to shape the exercise. It has launched a preparatory study that will explore this option. This study will also identify potential shortcomings in the collection of data, as data quality issues can have an impact on the cost of the evaluation. The Commission is also preparing the 9th evaluation conference on cohesion policy under the auspices of the Portuguese Presidency of the EU Council. The conference will focus on the evaluations carried out by the Member States and the Commission, and will provide a platform to discuss the methodologies used to assess the role of the policy under the different thematic objectives. 1.1.1.ESF/YEI Four thematic evaluations on ESF support were initially planned as studies. They were then upgraded to evaluations to pave the way for the comprehensive ex post evaluation and to feed the upcoming programme negotiations: -evaluation of ESF/YEI support to youth employment ; -evaluation of ESF support to employment and labour mobility (TO8, excluding support to youth employment) ; -evaluation of ESF support to social inclusion (TO9) ; -evaluation of ESF support to education and training (TO10) . They cover the implementation of ESF interventions from 2014 to 2018 and are now completed and published. The findings of the youth employment evaluation support the new Youth Employment Package, in particular the need to reach out to inactive/vulnerable/low-skilled groups and to extend the guarantee to those up to 29 years of age. It also confirms that work experience or vocational training is key to increasing employability. As regards design, an integrated pathway still seems to be the best approach to youth employment and individuals furthest away from the labour market who benefit greatly from tailored guidance and support. It has also been found that a too narrow definition of target groups may pose problems when adapting to specific situations or to changes in the context later on Adopting innovative communication approaches helps overcome the challenge of reaching disadvantaged participants. Finally, embedded communication strategies would help increase the visibility of EU support, such as making wider use of social media and forging partnerships with schools, community organisations and frontline services. Increased visibility would also boost outreach. The evaluation of ESF support to employment and labour mobility (TO8, excluding youth) has shown that the net effects of active labour market policies are particularly strong for women – who are at a certain distance from the labour market – through integrated/individual-based approaches. One of the main policy findings has been that there is a need to better fight embedded gender stereotypes and strive for more integrated approaches to promote active ageing. On labour mobility and support to structural changes more data and evidence will be crucial. In the field of ESF support to social inclusion (TO9), the thematic evaluation has found that the provision of personalised support is costly and means that providers require more training. Yet sufficient time and personalised support for participants is crucial to ensure needs are met and to achieve the desired results. A participatory approach to designing and implementing social inclusion interventions can also improve the provision of individualised support. Cross-sectoral partnerships facilitate effective engagement with the target group and generate greater effectiveness. Moreover, more inclusive partnerships and outreach strategies promote the recruitment of participants, making the intervention more relevant. Finally, the evaluation focussing on ESF support to education and training (TO10) provides evidence on early school leaving and lifelong learning operations as having been the most successful ones, both in terms of direct results and the potential for longer-term systemic change. The relative success of these priorities appears to relate to their stronger alignment with national and regional strategic priorities. Although support to education and training has effectively addressed the needs of young people in general, success has been more mixed for low-qualified adults, those not in education, employment or training and the hardest-to-reach disadvantaged groups. It calls for maintaining a strong focus on disadvantaged learners (and to increase it in particular for tertiary education). Also, there is a need to focus more on other skills that are increasingly relevant, such as digitalisation of learning and distance learning. The results of the three thematic evaluations of the support to TO8 (excluding support to youth), TO9 and TO10 are presented in a single Commission’s staff working document. The ex post evaluation of the ESF to be completed by the end of 2024 will build on these four thematic evaluations and the study carried out on TO11 (Progress assessment of ESF support to public administration). The Common Provisions Regulation requires the funds to be examined in terms of their effectiveness and efficiency and their contribution to the EU priorities of smart, sustainable and inclusive growth. The ex post evaluation will also assess the other evaluation criteria of the Better Regulation Guidelines – relevance, added value and coherence. As in the past, the ex post evaluation will be carried out in close cooperation with the Member States and managing authorities. It will be launched in 2022. Also, with a view to better understanding possible ways of simplification in data collection, DG EMPL is planning a study to examine the legal issues related to access to administrative data for evaluation purposes. 3.EUROPEAN AGRICULTURAL FUND FOR RURAL DEVELOPMENT (EAFRD) 0. 0.0.Evaluations completed by the Member States

0.0.0.General overview Overall, the Member States made considerable progress in their evaluation activities, particularly compared to the last programming period. They have drawn lessons from the evaluation activities for the enhanced annual implementation reports (AIRs) in 2019, with clear action points: closing data gaps, making better use of existing data, and putting more effort into quantifying complementary result indicators and impact indicators. These lessons are frequently addressed in the reported follow-up activities from 2020 and will guide the evaluation efforts until the ex post evaluation. The examples of reported evaluation findings show that Member States have a broad range of evaluation outcomes at their disposal in the meantime that they increasingly use and feed into the policy design process. The findings are being used to optimise the technical implementation of the current rural development programmes (RDPs) – e.g. in terms of adapting selection criteria, speeding up the uptake, providing capacity – and also to detect areas where there are still weaknesses in terms of uptake. The reported data collection efforts have been impressive – environmental monitoring in particular received a lot of attention by the Member States in 2020. It is still a major challenge in view of the ex post evaluation of the current programming period, but also in terms of preparing the future monitoring and evaluation system and measuring how the common agricultural policy contributes to the European Green Deal targets. In terms of assessing the quantified effects of RDPs, further efforts are needed in the Member States to credibly establish the link between the implemented RDP interventions and the actual impacts and to base it less on assumptions and more on clear quantitative links. The European Evaluation Helpdesk for Rural Development has also analysed specific lessons from the 2019 evaluation exercise in the ‘Synthesis of the Evaluation Components of the Enhanced AIRs 2019’, the ‘Assessment of the Progress in Implementing the Evaluation Plans of RDPs 2014-2020’, and the Good Practice Workshop No 12. Against this background, a specific thematic working group was set up in 2020 to address remaining challenges, e.g. on understanding and methodologies for calculating complementary result indicators. For the 113 AIRs submitted in 2020, the amount and quality of the information reported in Chapter 2 of the AIRs in 2020 markedly increased overall, in particular on completed evaluations, their findings, and in the follow-up actions to evaluation results. More than 90 AIRs included completed evaluations and their summaries, compared to around 75 in 2019. The number of follow-up actions has tripled. This trend reflects the advanced stage of the programmes within the evaluation cycle and explains the increasing availability of evaluation results not only on rural development programme (RDP) delivery and monitoring, but also on rural development achievements and impacts. The number of completed evaluations reported in the AIRs in 2020 (285) increased by almost 25% compared to 2019. Among these, 23% were related to common agricultural policy (CAP) objective 2 (Environment), 12% to objective 3 (Territorial balance), 5% to CAP objective 1 (Competitiveness), and 5% to horizontal priority 1 (Knowledge transfer and innovation). However, the vast majority of completed evaluations (33%) covered multiple RDP priorities and objectives. This can be explained by the fact that many of the reported evaluations were still closely related to the enhanced AIRs submitted in 2019 (e.g. background evaluations, studies), which by their very nature focus on all RDP priorities. A large share of completed evaluations were related to RDP governance, implementation, monitoring and evaluation (9%), whereas the evaluation of technical assistance and national rural networks was specifically addressed in 5% of the completed evaluations. Figure 4: Completed EAFRD evaluations across main topics Source: European Evaluation Helpdesk for Rural Development (2020) Modifications to the RDP evaluation plan fell markedly by 37% compared to 2019. Modifications mostly concerned the update of information on governance and coordination arrangements for evaluation, the evaluation topics and the adjustments of resources for evaluation. In total, 26 modifications were mentioned in 8 AIRs. The types of evaluation activities reported indicate that most RDPs have shifted from the planning and preparation phase of the evaluation cycle to actually conducting, disseminating and following up on evaluations. Different strategies to coordinate the generation of evaluation results (e.g. working groups, evaluation units, outsourcing) and ensure their quality (e.g. manuals, capacity building etc.) have become apparent as part of preparing the evaluation inputs for the AIRs. Figure 5: Evaluation activities in relation to the main phases of the evaluation cycle Source: European Evaluation Helpdesk for Rural Development (2020) As for the activities related to the provision and management of data for evaluations, they increased slightly by 9% compared to 2019. They focused on agreements with data providers, improving the governance of data management and data collection from application forms, as well as on additional data needed for quantitative analysis. In total, 357 data-related activities were reported in the AIRs in 2020, most of which concerned the processing/analysis phase. The highest number of data-related activities (49%) dealt with processing and analysing data as well as preparing the data systems (32%).

As for communication activities used to publicise evaluation findings, a substantial number (372) were reported by the RDPs in 2020. Five million stakeholders have been reached, mostly through websites (4.6 million), but also through a combination of different communication channels, i.e. newsletters, social media, evaluation reports, as well as internal and open meetings, workshops, seminars, etc. In terms of target groups, most of the communication activities targeted selected groups, such as RDP monitoring committees, local action groups, national/regional authorities, researchers and thematic experts, evaluators, national rural networks, rural networks and associations. In addition, 28% of the reported communication activities addressed the general public. Figure 6: Number of communication activities by type and stakeholders reached Source: European Evaluation Helpdesk for Rural Development (2020) Follow-ups on evaluation results have been reported to a much higher extent than in previous years, and show that the use of evaluation findings is becoming increasingly important. In 2020, the AIRs contained 1195 follow-ups, which is three times more than in the AIRs submitted in 2019. Most of these relate to improving the RDP delivery mechanism (36%), while 27% are already directly related to using evaluation results to prepare the CAP strategic plans for the post-2020 period. Adaptation of the RDP monitoring and evaluation system is addressed by 17% of the reported activities. Figure 7: Reported follow-up activities given to evaluation results Source: European Evaluation Helpdesk for Rural Development (2020) 0.0.1.Main findings of evaluations Distribution of the evaluation findings across evaluation topics follows one of the completed evaluations shown above. However, the types of evaluation findings vary considerably. The reported findings not only concern RDP results and impacts, but also relate to a much broader range of evaluation outcomes. The largest share (42%) of findings concerned RDP results and impacts (24% of them reporting clearly positive impacts/results). A considerable share of findings (31%) concerned the delivery mechanism of rural development programmes (assessment of selection criteria, budget, communication activities etc.). 18% related to monitoring progress against the objectives/targets (e.g. uptake under measures, progress in monitoring indicators etc.). The rest (9%) concerned other aspects (e.g. update of SWOT analysis, strategic environmental assessment findings, concrete recommendations of evaluations etc.). Evaluation findings were reported by 99 RDPs, otherwise cross-references to AIR Chapter 7 from the previous year were included. In fact, the findings linked to the assessment of RDP achievements and impacts had been reported in detail in the previous year by providing detailed answers to the 30 common evaluation questions. These findings were presented in the last staff working document and are not repeated here. Nevertheless, some complementary evaluation findings reported in 2020 in relation to the three CAP objectives and the horizontal priority are depicted below. These show the range of findings in relation to assessing delivery, monitoring progress and quantifying the RDP results and impacts. ·CAP objective 1 – Fostering the competitiveness of agriculture A number of RDPs have assessed delivery of the measures under the related rural development priorities 2 (farm viability and competitiveness) and 3 (food chain organisations). For example, Piedmont (Italy) analysed the selection criteria for investments in agriculture and for operations related to the setting up of young farmers. It showed that the quality of projects has increased overall. Positive contributions had also been found for example by the RDP Croatia in relation to improving the economic performance of all farms (Focus Area 2A), notably because there are strong synergies between the contributing measures. Moreover, despite the overall negative trend, the share of young managers joining the agricultural sector has increased. In Flanders (Belgium), an evaluation of the economic impact of the aid showed that it helped make enterprises more resilient and increased their gross added value. In the RDP Hungary, the average agricultural family income has increased more rapidly in supported farms (32.5%) than in the control group (16.9%). The RDP Cyprus noted a significant RDP contribution to strengthen cattle breeding but still marginally to improve the competitiveness in sheep and goat farming. ·CAP objective 2 – Ensuring the sustainable management of natural resources and climate actions Many Member States reported on findings stemming from their environmental monitoring (e.g. of farmland birds in Flanders, Slovenia, Castile-La Mancha (Spain) and also from the German RDP on Saxony, where a vegetation survey and plant-ecological evaluation was carried out). The RDP Estonia was able to find evidence that its average indicators of bumblebees per census track were higher in farms that received support for organic farming or more environmentally friendly management (agri-environment climate payments). Nevertheless, some RDPs report that the effects on the environment have still been rather limited in certain areas (e.g. Marche Italy, Greece, Croatia) or that they had issues in quantifying some of the related indicators (e.g. in the RDP Cyprus, the impact indicator on energy savings was difficult to calculate). Still, the RDP Cyprus also found evidence for a significant RDP contribution to improving biodiversity as reflected in the Farmland Bird Index, together with a positive contribution to improving water quality. Croatia found that its RDP measures have provided a significant contribution to maintaining and improving biodiversity in areas with natural constraints as well as contributions to conserving endangered native and protected breeds of domestic animals. ·CAP objective 3 – Achieving a balanced territorial development The delivery of measures under this CAP objective has in many cases been assessed with the main focus on the implementation of LEADER/CLLD. Beyond LEADER, the RDP Cyprus found that its programme only made a marginal contribution to creating employment, increasing GDP and reducing poverty. This is explained by the limited implementation of the measures under the related focus areas. While some RDPs still report a too low number of completed projects to properly estimate the impact, positive results and impacts have been reported for example by the German RDP on Hessen, where the case studies on the support of rural infrastructures demonstrated that the competitiveness of agricultural holdings had been strengthened. In the Spanish RDP on Castile-La Mancha, the assessment carried out by local action groups helped demonstrate the added value of LEADER, in terms of better results, social capital and local governance. With the help of case studies on its support to rural infrastructures, the RDP on Hessen found that the routes supported are multifunctional and are therefore not only used by agriculture, but also by the local population. This helped strengthen the competitiveness of agricultural holdings and at the same time increased the quality of life of the local population, while avoiding negative environmental impacts. ·Rural development priority 1 (fostering knowledge transfer and innovation in agriculture)

Delivery of the interventions related to this priority has been assessed for example by the RDP Estonia. It found that the change to the advisory system and the conditions of advisory support negatively affected the use of counselling and the functioning of the advisory system. In the German RDP on Saxony, an evaluation of advisory services in the field of nature conservation showed that consultants in some areas are reaching their capacity limits and that too few new stakeholders have ‘grown up’. The RDP Hungary reports on the positive trend in a steady increase in the share of qualified farm managers (highly skilled and secondary skilled). Looking at the period of available data, the share of qualified farm managers increased from 17.8% to 21.4% between 2013 and 2016. Bretagne (France) found in its initial results that the RDP contributes through some European Innovation Partnership projects to the development of the knowledge base in rural areas, notably the forest sector. In fact, RDP interventions have encouraged stronger links between agriculture, food production, research and innovation, including the improvement of environmental management and performance. 0.0.2.Support to Member States In 2020, the European Evaluation Helpdesk continued to help Member States improve their evaluation capacities. Examples of specific activities included: -providing Yearly Capacity Building events for the Member States; -organising ‘Good practice’ workshops to discuss lessons and experiences on selected topics -publishing guidance documents and tools; -collecting good practices and publishing them on webpages, in newsletters and in factsheets issued by the European Evaluation Helpdesk (e.g. on the impact of RDPs on soil, animal welfare, ex ante evaluation). 0.1.Evaluation work by the Commission On the evaluation work of the Commission, specific thematic evaluations are also being used to assess the effects of rural development programmes, supported by external studies financed by the Commission. The evaluations assess the effectiveness, efficiency, relevance, coherence and EU added value of all relevant CAP instruments. The study that supports the evaluation of the impact of the CAP measures towards the general objective of viable food production was published in November 2020. It provides a generally positive assessment of the effectiveness, efficiency, relevance, coherence and EU added value of the relevant measures, but points to room for improvement in the effectiveness of coupled payments, decoupled direct payments and price stability tools, as well as the administrative and management efficiency of the greening payment and the active farmer clause. The support study shows that CAP measures that address viable food production have strongly supported farm income and that market measures have helped limit the volatility of domestic prices for most agricultural products. It is important to highlight that the study was subject to considerable limitations due to the short observation period and resulting lack of accurate, detailed and homogenous data. More recent observations and complementary analysis will be added. In January 2021, the Commission published a staff working document on the evaluation of the impact of the EU agricultural promotion policy in internal and third country markets. It deems the EU agri-food promotion policy to boost competitiveness and consumption of EU products in and outside the EU to be effective and efficient. In particular, activities funded under the promotion policy help increase the sales and consumption of EU products promoted in target markets, although the quantitative extent of this is difficult to measure. The objectives of the policy are relevant to stakeholders’ needs and address market challenges. There are no major inconsistencies with other EU policies, including health, climate, environmental and development policies. This evaluation will feed into an overall policy review planned for 2021, which aims to boost its contribution to sustainable production and consumption in line with the Commission’s Farm to fork strategy’s objectives. The Commission published a staff working document on the evaluation of the CAP measures applicable to the wine sector in October 2020. The evaluation concluded that the national support programmes have played a key role in improving the competitiveness of EU wine producers and products, in particular increasing demand from third countries. The measures in the national support programmes are generally relevant to the needs of the sector, especially because they offer a range of tools that can be adapted to the various levels of development of EU local supply chains (i.e. restructuring and conversion, investment, and promotion). The needs not sufficiently addressed by the EU’s wine policy include survival or adaptation of the smallest operators, the need for a better-trained workforce, renewal of businesses between generations, environmental issues, and adaptation to market demand for lower alcohol wines and sustainable products. While there could also be greater coherence between the EU’s wine policy and EU public health objectives, overall the EU’s wine policy is fully consistent with EU economic, social and CAP objectives. The evaluation of the CAP’s impact on biodiversity, water and soil is supported by a number of different studies. The first one, on the CAP’s impact on habitats, landscape and biodiversity, published in March 2020, concluded that the presence of the CAP has increased the Member States’ ambition to address biodiversity objectives as well as the level of funding. However, the CAP’s contribution and benefits are highly dependent on Member States’ implementation choices and priorities. For certain Member States, their biodiversity priorities are not well reflected in their CAP decisions. The most effective measures for conserving semi-natural habitats are agri-environment climate schemes, particularly tailored and targeted to higher-level schemes, and the Natura 2000 measure, which can compensate land managers for the costs of mandatory conservation protection, particularly within Natura 2000 areas. However, the impacts of agri-environment climate schemes are often constrained by limited budgets and farmer uptake, and the Natura 2000 measure has not been used enough by Member States. It is worth noting that the study outcomes were limited due to the absence of suitable monitoring data, where it makes further recommendations. The second study, on the CAP’s impact on water, also published in March 2020, assessed the CAP framework as being effective in maintaining minimum practices beneficial for water quality; however, its effects on the quantitative aspects of water are rather contrasted. The effects of CAP instruments and measures on water were often difficult to assess due to the lack of data. The analysis highlighted the importance of implementation choices by Member States, as only a few measures actually supported operations that directly target water quality and quantity issues. The third study, on the CAP’s impact on sustainable soil management, published in February 2021, recognised that CAP instruments and measures introduced provisions on the use of plant protection products and fertilisers on a significant share of EU arable land. The CAP also contributed to lasting changes in farming practices by helping introduce the use of catch-, cover- and nitrogen-fixing crops. The analysis suggests that efforts to reduce soil erosion need to be strengthened, in particular in hotspots. The efficiency of the policy varies depending on the measures considered. Payment rates under support for activities in forests, the environmental measures of fruit and vegetable operational programmes and support for organic farming were generally sufficient to encourage farmers and forest holders to apply. While the relevant rules address the need to limit erosion, increase carbon content in mineral soils, protect grasslands and ensure that their carbon content is maintained, they are considered not very ambitious and comprehensive. However, the EU regulations seem to have raised the level of ambition on soil protection. The study that supports the evaluation of the CAP’s impact on knowledge exchange and advisory activities was published in February 2021. It concluded that the CAP measures are generally effective, efficient, relevant, coherent and offer value added in creating knowledge exchange and promoting advisory activities and innovation in the agricultural and forestry sectors and in rural areas. The main drawback of the policy is that the latter results reach only around 10% of EU farm holdings and 20% of CAP beneficiaries. Furthermore, CAP instruments and measures contribute to Member States’ Agricultural Knowledge and Innovation Systems (AKIS), as shown in all case studies in Member States. However, CAP support only concerns a small part of Member States’ AKIS. The increased budget (compared to 2007-2013) allocated by rural development programmes to knowledge transfer and information actions (M1), advisory services (M2) and innovation (M16) suggests a growing interest in using these means and activities, in particular to achieve CAP objectives. The study that supports the evaluation of the socioeconomic aspects of CAP impacts on territorial development of rural areas was also published in February 2021. The overall conclusion from the study is that CAP support to balanced territorial development, which aims to improve socioeconomic and social inclusion aspects, is generally positive. However, the impacts tend to vary considerably depending on the characteristics of the rural region analysed and the policy mix applied. CAP funding is shown to be especially helpful in developing highly rural, remote and/or agriculturally-dependent areas. Pillar I is found to be efficient overall, but Pillar II is considered less efficient due to the application of rules on public funding, reducing their accessibility to socially and economically disadvantaged beneficiaries. There is an overall level of consistency between the various measures and instruments, although separate planning of the interventions weaken their complementarity. By focusing significant resources on remote and very marginal rural areas, the CAP helps address needs that national or regional policy instruments may struggle to meet on their own. The evaluation will feed into the design of a long-term vision for rural areas, which the Commission has committed to develop together with all its stakeholders. 4.EUROPEAN MARITIME AND FISHERIES FUND (EMFF)

1. 1.0.Evaluations completed by the Member States EMFF programme implementation advanced significantly in 2018 and 2019. Member States had committed €3.24 billion of the available €5.7 billion in the fund by the end of 2019. During this time, a total of 41 evaluation operations were conducted on the 27 Member State programmes, with €4.17 million committed to support this work. While the focus and maturity of evaluations across the Member States are wide ranging, Member States evaluation capacity for the EMFF has been boosted by the Commission by providing practical support to Managing Authorities, which helped to enrich the scope and strength of evaluations conducted. The evaluations have the effect of providing evidence to improve programme and policy decisions. According to information provided in their Annual Implementation Reports (AIR), several Member States undertook Operational Programme (OP) modifications as a result of recommendations arising from their evaluations (Austria, Belgium, Bulgaria, France, Slovakia, United Kingdom). Changes were based on evaluation findings including the identification of delays or programme objectives that were not optimised. Nevertheless, the evaluations found that progress towards objectives was satisfactory. Improvement of administrative capacity and communication to the beneficiaries to simplify administrative procedures were identified as common areas to be improved. In many Member States, evaluations also made specific reference to preparations for the next programming period. Most Member States outsourced their evaluation tasks. Only Sweden and Slovakia mentioned that external evaluations were used with internal evaluations. The most frequent type of evaluations conducted by the Member States are process evaluations (implemented by Cyprus, Czechia, Spain, France, Malta, Portugal, United Kingdom). Most evaluations were structured to address effectiveness and efficiency at operational programme level as well as specific objectives and measures (Cyprus, Czechia, Spain, France, Latvia, Malta, Croatia, Portugal). Specific evaluations include: -implementation of Community Led Local Development (Estonia, Ireland, Croatia, Portugal, Romania); -achievement of indicators (Austria, Slovakia); -cost/benefit analysis of decommissioning scheme (Ireland); -evaluation of lobster v-notching scheme (Ireland); -evaluation of sustainable fisheries scheme (Ireland); -recirculating aquaculture systems (Finland); -impact assessment of the ‘blue economy’ sectors (Latvia); -ex ante assessment of financial instruments (Czechia, Ireland). Detailed information summarising the evaluations of each Member State based on information provided in AIRs is available in Section 7 of the EMFF implementation report for 2019, published in December 2020. 1.1.Evaluation work by the Commission A mid-term evaluation of the direct management portion of the EMFF was conducted. The staff working document was published in 2020. The evaluation found that the investments made had delivered value for money and recommended a framework of indicators compatible with shared management in the future. The agreement of co-legislators on the European Maritime Fisheries and Aquaculture Fund for 2021–2027 has enshrined this in the text of the fund regulation.