Table of contents List of abbreviations2 1.Introduction3 2.Background on the intervention logic of the SRSP5 3.Implementation / state of Play9 4.Methodology16 5.Analysis and answers to the evaluation questions20 5.1. Relevance 20 5.2. Effectiveness 25 5.3. Efficiency 32 5.4. Coherence 42 5.5. EU added value 50 6. Conclusions55 Annex 1: Procedural information59 Annex 2: Stakeholder consultation61 Annex 3: Evaluation framework66 List of abbreviations

Term or acronym Meaning or definition CSP Cooperation and support plan CSR Country-specific recommendation DG Directorate-General DG REFORM Directorate-General for Structural Reform Support ESIF European Structural and Investment Funds EU European Union EUR Euro ISSG Interservice Steering Group PA Preparatory Action SRSP Structural Reform Support Programme SRSS Structural Reform Support Service SWD Staff working document TAIEX Technical Assistance and Information Exchange instrument 1.Introduction Well-defined reforms are crucial for modernising economies, encouraging investment, creating jobs and raising living standards. To facilitate these reforms across the European Union, Regulation (EU) 2017/825 on the establishment of the Structural Reform Support Programme (‘the SRSP Regulation’) was adopted in May 2017. The objective of the Structural Reform Support Programme’s (the ‘programme’) is to contribute to institutional, administrative and growth-sustaining reforms in Member States by providing technical support to national authorities. The SRSP has a budget of EUR 222.8 million for the period 2017-2020. In accordance with Article 16 of the SRSP Regulation, the programme is subject to a mid-term evaluation. The overall goal of this evaluation is to assess the programme’s relevance, effectiveness, efficiency, coherence and EU added value, in line with the Better Regulation guidelines, and to draw lessons that can serve possible future improvements. The mid-term evaluation also ensures the transparency and accountability of the Commission’s activities. To support the mid-term evaluation exercise, an external contractor was engaged to perform an evaluation report. This report was carried out over a period of 12 months, starting on 20 November 2018. It covers the technical support provided under the SRSP’s 2017 and 2018 budgetary cycles, as well as projects funded through the voluntary transfers of additional resources to the SRSP budget by Greece and Bulgaria in 2018. It also covers technical support projects funded under the Preparatory Action of 2016. In total, 303 technical support projects in 24 Member States fall within the scope of this evaluation. In this staff working document (SWD), the Commission’s staff presents and reflects on the main outcomes and findings of the mid-term evaluation and provides evidence and data which could serve as a basis for further improving the functioning of the SRSP where relevant. The SWD also draws on other sources of information from the Directorate-General for Structural Reform Support, including its internal IT system and the annual monitoring reports, annual activity reports, and ex ante evaluation of the SRSP. It should be underlined that the evaluation was performed when the programme’s implementation was still at an early stage. This SWD outlines exploratory conclusions, proportionate to the scope of the evaluation. The SWD has the following structure. Firstly, it describes the background of the SRSP and summarises the programme’s state of play. Subsequently, the SWD provides a brief overview of the methodology used for the evaluation, including associated limitations to the robustness of the findings. The main results of the evaluation report are presented in the next section and structured according to the five Better Regulation criteria. The last section of the SWD provides the overall conclusion and lessons learned.

2.Background on the intervention logic of the SRSP The effective and efficient design and implementation of administrative and institutional reforms is essential for flexible economies that are able to adapt to changing circumstances. Fruitful and timely implemented reforms contribute to a favourable regulatory and institutional environment. This, in turn, reinforces competitiveness, supports job creation, and enhances investment and sustainable growth. In the aftermath of the economic crisis, many EU Member States have been facing low potential growth, significant macroeconomic imbalances and obstacles to investment. Reforms were needed to address these challenges and enable governments to continue adapting to changing circumstances. It is an EU political priority to advance the implementation of growth-enhancing reforms in order to strengthen the capacity of Member States to adjust their economies. Not only does this help to boost economic growth, job creation and social welfare, but it also contributes to the process of convergence across the EU. The process of designing and implementing institutional and growth enhancing reforms is complex and requires a high degree of knowledge and skills. Member States have different levels of in-house expertise and capacity to prepare and carry out much-needed reforms. Furthermore, they have to constantly keep up with changes, whether political or from the world around us. Often, the Member States most in need of reform are at the same time the ones least equipped. Some EU countries do not have sufficient capacity to initiate and implement reforms in response to the reform challenges that lie ahead. Others have the capacity, but their administrations are still struggling to adapt to constantly changing circumstances arising from globalisation and technological and international developments. Experience in economic policy coordination through the European Semester framework shows that the implementation of some reforms is lagging behind. Although in the multiannual assessment Member States achieved at least ‘some progress’ with the implementation of around two thirds of the country-specific recommendations (CSRs), the implementation of the annual CSRs has a slower pace overall. Furthermore, Member States sometimes struggle with the adequate and timely implementation of EU legislation. Given that the economies of EU Member States are strongly intertwined, especially those that take part in the Economic and Monetary Union, inadequate implementation of reforms can have negative effects beyond the borders of the Member State concerned. To address these challenges, the Commission decided to engage in closer cooperation with Member States to help them implement comprehensive reforms that foster common goals. The Commission already had positive experiences with supporting reforms in Greece and Cyprus, in light of the euro crisis that started in 2009. At the request of the national authorities, the Commission created the Task Force for Greece (2011) and the Support Group for Cyprus (2013). The objective was to help these countries make a series of reforms in a wide spectrum of public policy areas. These reforms formed part of their financial assistance packages from the EU. Given the success of these task forces, and seeing the benefits that reforms could have in making economies more flexible and competitive, the Commission decided to broaden the scope of the support provided. In 2015, the Structural Reform Support Service (SRSS) was set up to roll out support for reforms to all EU countries. In 2017, a dedicated financing instrument was adopted: the Structural Reform Support Programme (SRSP). The SRSS, and subsequently, the Directorate-General for Structural Reform Support (DG REFORM), have been managing the programme since. As laid down in the SRSP Regulation, the SRSP’s general objective is to contribute to institutional, administrative and growth-sustaining reforms in the Member States by providing support to national authorities. The support may also help Member States who are preparing for participation in the euro area. In order to achieve this general objective, the SRSP has set the following specific objectives: a)support the initiatives of national authorities to design their reforms according to their priorities, taking into account initial conditions and expected socio-economic impacts; b)support the national authorities in enhancing their capacity to formulate, develop and implement reform policies and strategies and in pursuing an integrated approach ensuring consistency between goals and means across sectors; c)support the efforts of national authorities to define and implement appropriate processes and methodologies by taking into account good practices of and lessons learned by other countries in addressing similar situations; d)assist the national authorities in enhancing the efficiency and effectiveness of human-resource management, inter alia, by strengthening professional knowledge and skills and setting out clear responsibilities. The programme’s general and specific objectives are also described in the intervention matrix ( Figure 1 ), visualising the relations between the different components of the intervention.

The SRSP is the main funding programme for projects providing technical support for reforms to EU countries. Support under the SRSP is available to any Member State facing challenges in designing and implementing reforms. This includes reforms to address challenges identified in the European Semester process of economic policy coordination (e.g. the CSRs), reforms to address EU priorities or reforms undertaken at the Member State’s own initiative. Furthermore, technical support can be provided to help Member States fully implement EU law. EU countries can also request technical support for economic adjustment programmes. Moreover, since the amendment to the SRSP Regulation in 2018, the SRSP can also finance reform-related actions and activities to help Members States prepare for their entry to the euro area. The technical support can be provided to help design and implement reforms that relate to a broad range of policy domains, including the following public policy areas: ·revenue administration and public financial management; ·governance and public administration; ·growth and business environment (including climate, energy and environment); ·labour market, education, health and social policy; ·financial services and access to finance. The support provided through the programme is demand-driven and tailor-made to suit the specific situation of the country concerned. Actions eligible for funding include the formulation of policy strategies or roadmaps, the provision of long- and short-term experts, and capacity-building activities, such as workshops, training and working visits. More examples of eligible actions are presented in the logic of intervention (Figure 1). The technical support to Member States is intended to help successfully design and/or implement the relevant reforms by tackling obstacles in the reform process. The results expected in the short term include the provision of strategies, reform roadmaps or drafts of new legislation. Other short-term results expected are new or improved procedures and methodologies, as well as organisational change and enhanced human resource management. The programme’s long-term impacts are expected to be: (a) strengthened institutions, governance and public administration; (b) enhanced sustainable growth, job creation and social inclusion; and (c) enhanced cohesion, competitiveness, productivity and investment (see Figure 1). Technical support projects can cover any part of the reform cycle – from identifying the needs to monitoring and evaluating the outcomes. At the same time, the effective follow-up of the actions, in terms of actual implementation of the reforms, remains the prerogative of the recipient Member State. Ownership of the reforms by the Member State is essential for their successful implementation. As the SRSP is a newly established funding programme, the current mid-term evaluation is the first major evaluation exercise for the programme as a whole. In order to ensure that the technical support to Member States is effective and cost-efficient, the Commission monitors the programme’s performance, based on the indicators that are set out in Annex 1 of the SRSP Regulation. In addition to the mid-term evaluation, the Commission also uses other mechanisms to monitor the programme’s progress, including annual monitoring reports. 3.Implementation / state of Play 3.1.How the SRSP is implemented The submission, analysis and selection of technical support requests under the SRSP is organised in annual cycles, in accordance with the provisions of the SRSP Regulation. The Member States have until the end of October to submit their technical support requests for the budgetary cycle of the upcoming year. Once they have done so, the Commission assesses these requests, based on the criteria set out in Article 7 of the SRSP Regulation. To avoid double funding and ensure complementarity, the Commission verifies whether requests overlap with actions implemented under other EU instruments or programmes. The Commission then selects the requests that are best suited for funding. Subsequent to this, the Commission agrees with the Member State concerned on the priority areas, the objectives, an indicative timeline, the scope of the support measures and the estimated financial contribution. This agreement between the Commission and the individual Member State is the cooperation and support plan. The cooperation and support plans are multiannual documents, signed and amended each year (if need be) by the Commission and Member States requesting support from the SRSP. The technical support projects financed through the programme do not require co-financing by the Member State concerned. The SRSP is implemented on the basis of annual work programmes. The annual work programmes are adopted every year and serve as a Financing Decision. They set out the support measures to be implemented for the year concerned, the dedicated annual budget, and the expected results. The annual work programmes include the priority actions for the programme, as agreed between the Commission and the individual Member States in the respective cooperation and support plans.

To facilitate efficient communication with the Commission and the coordinated submission of technical support requests, each Member State has designated a national coordinating authority within its government structures. This coordinating authority functions as the Commission’s interlocutor for implementing the cooperation and support plans, and the SRSP in general. It is responsible for collecting all requests for the Member State in question, and subsequently submitting the requests to the Commission. The coordinating authority can also be responsible for prioritising the requests before submitting them to the Commission. The national beneficiary authorities are the actual recipients of the technical support, for example the thematic ministries that have requested assistance. The SRSP never gives money to the authorities, only technical support, such as strategic and legal advice, studies, training and expert visits on the ground. The coordinating authority is expected to coordinate the ensuing implementation of the support measures in the country and is also encouraged to take appropriate action if challenges arise during the technical support’s implementation. 3.2.Implementation of SRSP 2017 and SRSP 2018 The budget for SRSP 2017 and SRSP 2018 was EUR 53 million ( Figure 2 ). Under SRSP 2017, 159 requests from 16 Member States were selected to receive funding, out of the 271 requests submitted for a total of EUR 83 million – an amount almost four times the EUR 22.5 million budget available for 2017. In 2018, the SRSP’s second year of implementation, there was an increase in demand from Member States. In total, 146 requests from 24 Member States were selected for funding under SRSP 2018, out of the 444 support requests submitted. The submitted requests amounted to an estimated total value of EUR 152 million. This amount was five times the budget available for SRSP 2018 (EUR 30.5 million). Funding source Number requests submitted Number of requests selected Available budget SRSP 2017 271 159 EUR 22.5 million SRSP 2018 444 146 EUR 30.5 million Total 715 305 EUR 53 million Figure 3 shows the number of requests submitted per Member State under SRSP 2017 and SRSP 2018. Figure 3: Number of requests submitted for funding under SRSP 2017 and 2018 per Member State (source: SRSP Annual Monitoring Report, 2018) The high demand from Member States in both 2017 and 2018 indicates a significant interest in the instrument. In line with the principle of sound financial management, the Commission prioritised requests allowing for quick delivery of support and rapid implementation of reforms on the ground, and requests addressing defined objectives, with strong results expected on the ground. Figure 4 shows the number of requests selected per Member State under SRSP 2017 and SRSP 2018.

Figure 5 shows the circumstances of the requests that have been selected under SRSP 2017 and SRSP 2018. Over half of the selected requests were related to implementing challenges identified in the context of economic governance processes, in particular to CSRs. Circumstances of the selected requests SRSP 2017 SRSP 2018 Economic governance process, including CSRs 51% 55% EU priorities 27% 29% EU law 7% 6% Member States’ own initiative 11% 7% Economic adjustment programmes 4% 3% When the Commission decides to fund a request for technical support, the request becomes a project. In the selection process, the Commission can decide to implement one large request through multiple projects. Similarly, the Commission can also decide to implement multiple related requests through one single project. Therefore, the number of selected requests can differ from the number of projects (i.e. support measures) that are implemented. The number of projects approved under SRSP 2017 and SRSP 2018 is provided in Figure 6 . Funding source Number of approved projects SRSP 2017 144 SRSP 2018 143 Total 287 In total, 246 technical support projects funded under SRSP 2017 and SRSP 2018 were evaluated as part of the mid-term evaluation, covering 90% of the allocated financial resources under the two budgetary cycles. More than 50% of the evaluated projects under SRSP 2017 and SRSP 2018 were implemented in 7 Member States (Romania, Croatia, Malta, Cyprus, Lithuania, Slovenia, and Poland). A detailed overview of the distribution of evaluated projects among EU countries is provided in Figure 7 . Technical support under SRSP 2017 and SRSP 2018 covered all five thematic areas. As demonstrated in Figure 8 , out of the evaluated projects funded under SRSP 2017 and SRSP 2018 the largest share of the approved budget was allocated to projects in area of growth, business environment and sectoral issues (EUR 13.5 million), followed by labour market, education, health and social services (EUR 11.4 million) and revenue administration and public financial management (EUR 9.1 million). In line with the SRSP Regulation, the measures of the programme are implemented either directly by the Commission or indirectly in accordance with the Financial Regulation. Under direct management this includes public procurement contracts, grants and reimbursement of costs incurred by external experts. The underlying procedures for concluding contracts/agreements with technical support providers under the SRSP are in line with the Financial Regulation and thereby in accordance with the principle of sound financial management. In line with the Financial Regulation, some activities are implemented under the indirect management mode in view of the type of the implementing entity. Entities, such as international organisations, which have undergone an ex ante (/pillar-) assessment of their procedures and systems, guaranteeing a level of protection of the EU financial interests equivalent to that guaranteed by the Commission in direct management, can act under indirect management for the tasks they undertake to implement the actions. The annual work programme contains an overview of the actions and the related budget that are implemented under direct (public procurement and grants) and indirect management (agreements with implementing organisations). For SRSP 2017 and SRSP 2018, around 98% of the evaluated budget was implemented through grants, procurement, indirect management and TAIEX ( Figure 9 ).

For the projects evaluated in the mid-term evaluation, the types of providers most involved in implementing projects under SRSP 2017 and SRSP 2018 were: ·international organisations (37%) ·private companies (22%) ·government bodies (8%) ·public law bodies (7%) ·bodies governed by private law with a public service mission (4%). These providers accounted for 78% of the evaluated budget allocated under SRSP 2017 and 2018. Looking at the specific measures financed, the largest proportion of the support measures implemented concern the following sectors/actions: SRSP 2017 SRSP 2018 ·revenue administration (11%) ·tax policy (13%) ·capital markets union (9%) ·education and training (9%) ·healthcare system (9%) ·digital public administration (8%) ·energy, including climate change (6%) ·governance (8%) ·education and training (5%) ·healthcare system (7%) ·better regulation (5%) ·Capital Markets Union (5%) Given that the evaluation took place in the middle of the implementation period of SRSP 2017 and SRSP 2018, the majority of the technical support projects were still in progress. At the time of the mid-term evaluation, 22% of the projects had been completed.

4.Methodology 4.1.Fact-finding The mid-term evaluation was carried out by an external contractor and coordinated by the Directorate-General for Structural Reform Support (DG REFORM), with the support of an inter-service steering group of other Commission departments. The contractor delivered an evaluation report, which served as a basis for this staff working document. The mid-term evaluation assessed projects funded under the SRSP’s budgetary cycles of 2017 and 2018, projects funded through the voluntary transfers of additional resources to the SRSP budget by Greece and Bulgaria in 2018, and projects funded under the Preparatory Action of 2016. Figure 10 provides an overview of the number of evaluated projects per funding source. The mid-term evaluation covered 89.4% of the allocated budget for 2016-2018, representing 303 implemented projects and 409 means of delivery. The staff working document is focused primarily on the two budgetary cycles of the SRSP and refers to the Preparatory Action and the Article 11 only where relevant. The findings for evaluated projects funded under Article 11 and the Preparatory Action are similar to evaluated projects funded under SRSP 2017 and SRSP 2018. Funding source Number of projects evaluated Approved financial resources (EUR) % out of the allocated financial resources PA (2016) 23 2 733 223 91.1% SRSP out of which: 246 47 709 027 90.0% ·SRSP (2017) 120 19 608 927 87.2% ·SRSP (2018) 126 28 100 100 92.1% Article 11 (2018) 34 18 830 000 87.6% Total 303 69 272 250 89.4%

The mid-term evaluation was divided into three main phases: (1) the inception phase, in which the structure of the evaluation was defined, based on a conceptual framework; (2) the data collection phase, in which the data was collected and structured, based on the data collection strategy defined in the first phase; and (3) the finalisation and feedback phase, in which the intermediary and final evaluation results were reported to stakeholders. The evaluation was carried out in line with the principles commonly applied for the evaluation of EU initiatives, as laid down in the Better Regulation guidelines. The evaluation comprised different fact-finding tools and techniques, such as desk research, scoping interviews, targeted interviews, online consultation and case studies. Overall, the evaluation relied to a large extent on stakeholder consultation. The fact-finding exercises built on the evaluation framework that was designed during the inception phase of the report. The evaluation framework, including the evaluation questions and the corresponding sources of information, is provided in Annex 3 of this staff working document. The desk research involved the review and analysis of relevant documentation. These can be grouped into six main types: (1) EU treaties and legal acts, i.e. EU legislation underpinning the SRSP, the Preparatory Action and Article 11; (2) Commission preparatory documents, i.e. staff working documents, cooperation and support plans; (3) other Commission resources, i.e. internal DG REFORM documentation; (4) online resources, including web pages of informative nature from official sources; and (5) other resources, including literature and studies. A major source of information for the mid-term evaluation was the Commission (DG REFORM) internal IT system for monitoring technical support projects. Developed in 2018, this system provided a broad range of quantitative and qualitative information about the requests and projects under the SRSP. Data extracts from this system were provided to the contractor for the analyses. In addition to the desk research and information from the Commission’s internal IT system, 16 semi-structured scoping interviews with DG REFORM officials were carried out (heads of unit/cluster, advisers and country coordinators). The aim of these interviews was to get a deeper understanding of the relevant issues for the different stakeholders. The intention was also to set up the detailed methodology and data collection tools. The scoping interviews were not project-specific. As a follow-up to the scoping interviews, 31 semi-structured targeted interviews were carried out. This involved interviewing a sample of 25 DG REFORM project managers and a sample of staff members from six other Commission departments involved in SRSP assessment/implementation or in managing complementary EU funding programmes or policies. To ensure that the samples were representative, different elements were taken into consideration when selecting the interviewees, including the thematic areas of the technical support projects and their geographical coverage. Furthermore, the evaluation included an online consultation targeting 185 stakeholders. Feedback was collected from 9 coordinating authorities, 107 beneficiary authorities and 69 technical support providers. The questions in the questionnaires covered all the five Better Regulation evaluation criteria. In order to optimise the results of this consultation, the questions were adapted to the types of stakeholders, according to the level and nature of engagement in the technical support funded. Both the targeted interviews and the online consultation were project-specific. Additionally, the Commission carried out a public consultation between March and June 2019 on its official website. The goal was to collect information from the general public, businesses, trade associations and interest groups. However, given the very low number of respondents (11), the extent to which these results can be used is limited. The number of respondents was low due to the fact that the SRSP specifically targets public administrations and bodies – not individuals – and that targeted interviews, targeted consultations and scoping interviews had already been carried out. Public administrations and bodies were largely covered by the external contractor’s evaluation report. Moreover, it is presumed that the programme was not widely known among the general public, given that it had only recently been established. Because the limited response to the public consultation cannot serve as solid evidence, the staff working document refers to the public consultation results only for illustrative and transparency purposes. Lastly, the evaluation included case studies at Member State level. The case studies provided an analysis of how the SRSP projects were carried out on the ground in four countries (Croatia, Greece, Italy and Romania). Later in the process, the geographical coverage of the interviews was extended to five additional Member States (Ireland, Latvia, Malta, Portugal and Slovakia) so that the findings would be more relevant. The relevant Member States were selected on the basis of the number of approved projects, the allocated budget and the geographical coverage. The main purpose of the case studies was to assess in more detail the SRSP’s relevance, effectiveness, efficiency, coherence and EU added value in the particular Member States. The case studies included interviews with coordinating authorities (8), beneficiary authorities (22) and technical support providers (21). These interviews related to 28 evaluated projects, covering the five thematic areas of the SRSP. The case studies functioned as a method for gathering additional information in order to have more robust answers to the evaluation questions (i.e. triangulation). 4.2.Limitations and methodological issues As required, the external evaluator contracted for this assignment carried out all tasks under the scrutiny of an inter-service steering group and the steer of DG REFORM. Overall, the quality and representativeness of the collected evidence (data, documentation, interviews, online consultation and case studies) are considered to be satisfactory. The contractor was able to collect extensive and meaningful data that allowed conclusions to be drawn. Nevertheless, there were some limitations and methodological issues. These were mitigated to the extent possible, with any limitations given due consideration. Firstly, the most important constraint to the robustness of the findings was the fact that the SRSP had only been very recently established when the mid-term evaluation took place. The Regulation was adopted in May, and the first call was issued immediately after that. The fact that the mid-term evaluation was performed at a very early stage in the programme’s implementation limits the extent to which fixed conclusions on the SRSP’s success can be drawn. This staff working document provides first conclusions and lessons learned, which might evolve during the SRSP’s continuing implementation. Secondly, in particular, the Commission’s (DG REFORM) new internal IT system was not in place yet for SRSP 2017 but was implemented only afterwards. The system was evolving significantly, and many questions and categories of data were added only at later stages and not necessarily retroactively completed. Therefore, information was to some extent incomplete. This shortcoming was overcome by triangulating the information based on other data sources, including the interviews, online consultation and case studies. Furthermore, the response rate to the online consultation was sufficient and ensured a balanced representation of stakeholders. Thirdly, the breadth and diversity of the programme’s activities posed an important methodological challenge, in that it would not have been possible within the available resources and timeframe to cover all activities in the detail needed to draw robust conclusions. Moreover, much of the programme’s support plays a contributing role alongside other factors, such as the action of national administrations. This contributing role is difficult to assess without in-depth qualitative research. Lastly, given the nature of a mid-term evaluation, many of the projects evaluated were still ongoing at the time of the exercise. Therefore, results and impact can only be measured to a limited extent. A certain share of the projects’ expected results were not yet delivered. In those cases, the likelihood of achieving the results was assessed only to the extent that information was available. The case studies confirmed that it is too early to expect considerable long-term impacts at this stage of the programme. 5.Analysis and answers to the evaluation questions The mid-term evaluation assessed the performance of the SRSP against the following five criteria: relevance, effectiveness, efficiency, coherence and EU added value. The external study consisted of 22 specific evaluation questions, which were subsequently grouped according to their topic in order to provide a coherent framework. The overall analysis is based on evidence from both the external evaluation report and the Commission’s own sources. This chapter presents the analysis and provides the answers to the general evaluation questions. 5.1.Relevance The overall assessment of the programme’s relevance focused on two aspects:

·the relationship between the needs of the Member States at the time of their request for technical support and the objectives and actions of the technical support provided; ·the continued suitability of the programme objectives. 5.1.1.Member State needs and the objectives of technical support The representation of the programme’s logic of intervention ( Figure 1 ) illustrates the alignment of the SRSP design and specific objectives with the needs of the Member States at the time of their request for technical support. The programme’s logic of intervention identifies three challenges facing Member States: 1)Limited administrative and institutional capacity; 2)Challenges in designing and implementing growth-sustaining reforms in line with the EU’s economic and social goals; 3)Limited application and implementation of EU law. Both the beneficiary authorities and providers of the technical support consulted during the mid-term evaluation confirmed these as the main challenges that Member States were facing at the time of the request for technical support. Figure 11 shows that ‘limited administrative/institutional capacity’ and ‘challenges in the design and implementation of structural reforms’ were the challenges most frequently mentioned by beneficiary authorities consulted. With the case studies it was possible to specify these broad challenges. The case studies analysis shows that often the institutions responsible for designing and implementing structuralreforms in the relevant Member States had a limited capacity to implement reforms due to less efficient and effective human resource management. This in turn weakened the organisation, affecting also its capacity to implement enforcement measures. Moreover, these institutions face bottlenecks due to an institutional setting where competencies of sectoral and non-sectoral ministries overlap and responsibilities are unclear. Finally, they lack a systematic, analytical, and integrated approach to implementing reforms that would ensure a regular assessment of priorities, contextual conditions, expected socio-economic impacts and costs and benefits of different policy options. Analysis of the actions and activities funded under the SRSP revealed that the Member States’ needs, as requested under the SRSP, have all been addressed to a large extent by the projects, and that on average, each project covered almost two of the needs mentioned above. The distribution of the projects according to the Member States’ needs is provided in Figure 12 . Respondents to the public consultation were also positive about the extent to which the SRSP responds to the needs of Member States. Project funding source # of projects related to strengthening administrative and institutional capacity # of projects related to supporting the design and implementation of growth-sustaining structural reforms # of projects related to application / implementation of EU law SRSP 2017 82 88 37 SRSP 2018 83 102 51 Total 165 190 88 % (out of 246) 67.1% 77.2% 35.8% By pursuing its specific objectives, the programme provided technical support to national authorities to develop the administrative and institutional capacity they were lacking and that is necessary for them to design, formulate, develop and implement the structural reforms they need, based on their priorities, conditions and expected socioeconomic impacts. The support also aimed at helping Member States to adopt an integrated approach to reforms, which ensures consistency across sectors, as well as coherence with EU priorities, programmes and legislation. Furthermore, the funded actions aimed at supporting national authorities to put in place new, more robust institutional processes and methodologies to enhance the efficiency and effectiveness of human-resource management, strengthening professional knowledge and skills and setting out clear responsibilities.

5.1.2.Continued suitability of the programme’s objectives The SRSP’s objectives and actions were and continue to be relevant for Member States’ needs. The analysis showed that the SRSP is perceived as a suitable and appropriate instrument for supporting Member States in strengthening their administrative and institutional capacity. Almost all (97.3%) of the stakeholders consulted agreed that the SRSP is a suitable instrument for providing technical support to design and implement growth-sustaining structural reforms ( Figure 13 ). Moreover, the the majority of the beneficiary authorities consulted (65%) consider that technical support is still needed. The findings from the case studies suggest that this is due to the long-term and broad nature of the Member States’ needs, their interconnectedness and the willingness of institutions to expand the application of the newly developed skills and capacity to implement reforms in other policy areas. The implementation of institutional, administrative and growth-sustaining reforms in the Member States is a long-term process linked to long-term needs. While the SRSP has supported the launch of this process, reforms require sustained effort over several years for economies to become more modern, flexible and competitive. From the case studies it emerged that, due to the SRSP’s flexibility, the simple contracting and implementing procedures, and the absence of co-financing requirements, Member States have a tendency to consider SRSP as a relevant instrument to help design and implement national reform agendas. Moreover, the implementation of institutional, administrative and growth-sustaining reforms in the Member States is linked to wider needs. Therefore, the beneficiaries still need support for designing reforms both within the same thematic areas and in different ones. Furthermore, the incremental value of the skills and capacity developed by the beneficiary of technical support should also be considered. In other words, once the beneficiary gains experience with the instrument, they can identify other needs and potential areas of action, and thus plan to use the instrument to enlarge the scope and breadth of the reforms to implement. In conclusion, the challenges faced by Member States as defined in the programme’s logic of intervention are aligned with the main challenges expressed by the consulted beneficiary authorities and providers. Almost all stakeholders consulted (beneficiary authorities, coordinating authorities and providers) indicated that the SRSP is a suitable instrument for providing technical support. The programme is perceived as a significant tool for improving the Member States’ capacity to define more efficient processes and methodologies and/or progress towards more effective human resource management. Since its second year of implementation, the SRSP has seen a regular increase in demand for the programme from Member States. This is the first indication that the support provided by the Commission continues to be needed and is still relevant.

5.2.Effectiveness For the criterion of effectiveness, the mid-term evaluation examined to what extent the objectives set were achieved and how this was linked to the EU intervention. The effectiveness of an intervention is measured in terms of both results and impacts. However, the mid-term evaluation of the SRSP was carried out while the programme’s implementation was still at a very early stage. Considering that impacts are expected only in the long-term, sometimes years after completion of the intervention, a solid assessment of the SRSP’s impacts was not possible at this stage. Therefore, the evaluation focused mainly on the SRSP’s effectiveness in terms of results. The overall assessment of the programme’s effectiveness was composed of the following three sub-analyses: ·an examination of the extent to which the technical support projects were well-designed; ·an examination of the extent to which the planned activities and actions were feasible and expected results had been achieved;

·a preliminary exploration of the programme’s achieved impact on the ground. In addition to these three specific areas of analysis, the evaluation also aimed to identify contextual factors that either encouraged or hindered progress towards the results and impact. 5.2.1.Design of the technical support The evidence collected during the evaluation shows that the objectives defined at project level corresponded with the specific objectives of the SRSP. The majority of projects evaluated (around 60%) had links with more than one specific objective. These outcomes demonstrate that the technical support projects are designed in such a way as to contribute to the objectives set out in the SRSP Regulation. Furthermore, the evaluation shows that in most cases (85%), initial challenges faced by the beneficiary authorities when preparing the support requests were taken into account when the projects were designed and implemented. At the same time, the findings show that when the SRSP actions were designed, the relevant socio-economic impacts were largely taken into account as well (72% of the cases evaluated). The socio-economic impacts mostly related to social inclusion, sustainable growth, competitiveness, productivity, investment, job creation and enhanced cohesion. In addition, the evaluation suggests that SRSP actions are often designed in a comprehensive manner, by paying attention to accompanying measures and/or the appropriate sequencing of reform measures – i.e. multiple projects that aim to address the same reform challenge, for instance from different angles or at different points in the reform design or implementation timeline. In Member States such as Cyprus, Czechia, Croatia, Greece, Romania and Slovenia, between 40% and 60% of the projects were implemented in connected reform areas, encompassing in-built synergies. This indicates that different SRSP-funded projects are linked to each other in order to achieve a comprehensive approach towards the reform challenge. Generally, in Member States where the total number of projects is higher, the degree of interconnection between projects also tends to be higher. Lastly, the evidence collected shows that in the majority of the SRSP projects evaluated, the successful experiences and lessons learned from other EU countries are included in the design of the activities. Almost 60% of the projects evaluated included activities allowing for the exchange of best practices, such as short-term expert missions, study visits, workshops, seminars and conferences. In addition to the exchange of EU Member State expertise, workshops and other training activities were also organised by experts from international organisations or other experts. Overall, these findings suggest that the technical support projects are designed with satisfactory consideration of the specific situation of the individual beneficiary authority and the related socio-economic impacts. Technical support is generally provided in a comprehensive manner, while often incorporating in the project’s design the exchange of best practices between EU countries. All of these aspects can contribute to effective and targeted technical support activities. 5.2.2.Feasibility and achieving the expected results To examine the feasibility of the programme’s activities, the evaluation focused on the extent to which the implementation of technical support projects is in accordance with the planned schedule. The collected evidence shows that the greatest share of the projects under SRSP 2017 and SRSP 2018 are on track (84.1%), i.e. the project’s implementation is aligned with the planned start and end date. This indicates that the needs of the Member State and subsequent activities are very often translated into realistic time schedules. Some projects (around 5%) were faced with delays. The targeted interviews with DG REFORM project managers suggested that these delays were often caused by political changes or changes in the beneficiary authority’s needs. Delays also occurred due to challenges in identifying the most suitable technical support provider.

The evaluation also assessed to what extent the projects have achieved their results. In those cases where projects had not yet achieved their expected results, which was common given that many projects were still in progress at the time of evaluation, the likelihood of achieving the results was measured. As outlined in Figure 14 , the evaluation findings show that most of the projects completed under the SRSP have achieved their expected results (70.4%). The majority of the completed projects that had not yet achieved their results had a high likelihood of still doing so (63.6%). As anticipated, only a small number of ongoing projects has already achieved their results (4.2%). The evidence shows, too, that most of the ongoing projects have a high likelihood of achieving their results (86.4%). The likelihood of achieving the results is low for 13.2% of the completed and ongoing projects. The evaluation also assessed the contextual factors influencing the delivery of project outputs. These contextual factors can either be positive or negative (see Figure 15 ). The most common positively influencing factors are: the level of commitment from all stakeholders (45.6%); the provider’s level of expertise (15.2%); and the political circumstances (15.2%). The most common negatively influencing factors are: the level of collaboration among all actors involved (29.7%); resource constraints (29.7%); and political circumstances (21.7%). In conclusion, the evidence from the mid-term evaluation shows that the SRSP is progressing towards achieving its results, taking into account the evaluation’s limitations as many activities are still in progress. The Member States’s needs are to a good extent translated into feasible support actions under the SRSP. The majority of the projects adhere to the planned schedules, which implies that the designed activities are realistic. Meanwhile, the number of projects with delays is low. Furthermore, most of the projects are likely to achieve their results, or have already done so. A small number of completed projects did not achieve their results. A lack of collaboration among stakeholders is a challenging factor for delivering the projects’ outputs. Changes in the political structures of the beneficiary authority also pose an evident risk for the effective implementation of technical support. These and other negative factors should be avoided and mitigated to the extent possible. Optimising the exchange of good project management practices among DG REFORM project managers, for instance, through dedicated training or knowledge-sharing platforms, could help to further increase the number of projects that achieve their results. At the same time, it should be underlined that technical support for reforms is often most needed in institutions that are embedded in challenging political and governmental structures. Therefore, some degree of risk is inevitable. Reinforcing the commitment from all stakeholders in combination with effective collaboration and adequate technical expertise can increase the likelihood of achieving the projects’ results. 5.2.3.Achievement of impacts As stated earlier, it is too early to measure the long-term impacts of the SRSP. The same goes for examining the programme’s unintended or unexpected effects. Nevertheless, the stakeholders’ perception at this early stage was used to assess the extent to which technical support projects helped the Member States to implement the relevant reforms, as planned in the cooperation and support plans. The evaluation also explored the contextual factors under which the results of SRSP actions would produce impacts on the ground.

The evidence collected from the targeted online consultation shows that stakeholders see signs of improvement with respect to the needs that the beneficiary authorities are facing ( Figure 16 ). The highest perception of improvement is with ‘increased administrative and institutional capacity’ (73.7%). Perceived improvements are the lowest for ‘improved application and implementation of EU law’ (42.1%). The majority of the stakeholders consider that these improvements are related to the technical support received through the programme. The respondents to the public consultation were also positive in their view on the extent to which technical support has helped to achieve the SRSP’s specific objectives. The evaluation also looked closely into the specific case studies. Concrete examples of impact on the ground include: ·Support to the spending review process in Slovakia: the spending review resulted in the adoption of legislation, and Slovakia achieved EUR 83 million in savings in 2017 (0.09% of GDP); ·Education (curricular) reform in Croatia: the pilot phase of curricular reform was implemented in 74 schools in the 2018/2019 school year, and 400 tutors were trained under a follow-up project; ·A strategy for internal auditing and training internal auditors at local government level in Romania: the strategy for internal auditing was produced and adopted, and 524 city halls, prefectures and county councils were subject to an internal audit. The evidence collected through the consultation of stakeholders shows that the most commonly mentioned factor positively influencing the achievement of impacts is the beneficiary authority’s commitment. A lack of political support and stability is the most commonly mentioned negative factor for achieving the results. In conclusion, taking into account the limitations of stakeholders’ perceptions and individual case study assessment, the achieved impacts are considered proportionate to the level of completion of the programme. However, it would be premature to draw vast conclusions on the SRSP’s impact when most of the projects are still in progress. Even achieving the project’s expected results does not guarantee that there will be an impact on the ground. Thus, a comprehensive evaluation at a later stage of the programme is essential to provide better substantiated evidence for measuring the extent to which an impact has been achieved in the long term. 5.3.Efficiency Efficiency considers the relationship between the resources used and the changes generated by the programme. Differences in the way the support is designed, implemented and monitored can have a significant influence on the effects. Therefore, assessing the programme’s efficiency is also related to verifying whether other choices could have maximised the use of resources for the greater benefit of the beneficiaries. The overall assessment of the SRSP’s efficiency focused on three aspects: ·the operational efficiency of the SRSP governance, in terms of the time-effectiveness of the SRSP cycle, including its monitoring tools, as well as the quality of the requests and the cooperation with other Commission departments;

·the cost-effectiveness of the support provided, in terms of the correspondence between the results achieved and the budget allocated; ·the time-efficiency of the budget execution. In parallel, the analysis looked at aspects of simplification, pinpointing areas where there is potential to reduce inefficiencies and simplify the intervention. 5.3.1.Operational efficiency of the SRSP governance The assessment of the efficiency of the SRSP governance focused on the time-effectiveness of the programme cycle, including the monitoring mechanisms in place, cooperation with other Commission departments and the improved quality of the requests for technical support. Time-effectiveness was analysed by looking at the duration of the different phases of the programme and verifying whether their duration was aligned with the timing of the needs of the target group, i.e. the beneficiary authorities in the Member States. An important methodological limitation must be underlined. The analysis considered only technical support projects funded under SRSP 2018 (126 in total); due to the timing of the SRSP Regulation’s adoption, SRSP 2017 followed a considerably shorter cycle. Therefore, 2018 was the first year in which it was possible to adopt the planned timetable. Figure 17 shows the SRSP annual cycle. Based on the analysis of the Commission’s (DG REFORM) internal IT system and the results of the targeted interviews, it emerged that for the SRSP 2018 cycle, it took on average 5 months for the selection process to be completed: from the October 2017 deadline for submitting the technical support requests to the approval in March 2018 of the Financing Decision and related SRSP annual work programme containing the full list of projects selected for funding. This period comprised the phases of submission, analysis and selection of the technical support requests, as well as the adoption of the annual work programme and the Financing Decision and the conclusion of the cooperation and support plans. Once the Financing Decision was adopted, for most of the projects (81%) it took at least 6 months before the planned technical support started to be delivered. Figure 18 shows the distribution of evaluated projects under SRSP 2018 by starting date. The average planned duration of the projects under SRSP 2018, from the procedures for concluding contracts/agreements with technical support providers to the completion of the project, was 13 months. As demonstrated in Figure 19 , the planned duration of most projects (71%) was between 6 and 18 months. Therefore, on average, the whole process could take 24 months, from the submission of the request by the Member States to the completion of the technical support project. The targeted online consultations of both beneficiaries and providers confirmed the alignment of the SRSP timeline described above with the timing of the target group’s needs: 74.1% of the beneficiaries consulted and 66.6% of the national coordinating authorities consulted agreed entirely or to a large extent that the period between the submission of the technical support requests and the start of the delivery of the technical support was timely and efficient. Moreover, 84.6% of the beneficiaries, all coordinating authorities, and 86.2% of the providers agreed entirely or to a large extent that the implementation and delivery of the technical support was delivered on time and in accordance with the needs of the beneficiaries. Respondents of the public consultation also positively assessed the programme’s efficiency. Finally, 82.6% of the providers consulted agreed and strongly agreed that the duration of the the procedures for concluding contracts/agreements was aligned to the needs of the beneficiaries. Although in general, stakeholders agreed that the current programme cycle is in line with the beneficiaries’ time needs, with the targeted interviews and case studies it was possible to identify some areas where there is room for improvement. For example, some of the providers considered that the duration of the discussions on the details of the contracts/agreements was not always proportionate to the duration of the contracts/agreements. Some beneficiaries signalled that they would have wanted DG REFORM to involve them more in the process of selecting the providers, even though most beneficiaries were pleased with the interaction with the provider. One in four considered that the alternatives offered were limited and did not represent the most adequate choice for their needs. A specific area where it emerged that there could be room for improvement was the monitoring of the SRSP. Being a legal requirement, the systematic and regular review of progress and quality was continuously performed throughout the annual SRSP cycle, thus helping to ensure the programme’s efficiency. The majority of stakeholders had a largely positive perception of the procedures and tools used by the Commission to monitor the technical support’s implementation and saw them as effective. This was true for 79.8% of the beneficiaries, 66.6% of the coordinating authorities and 83% of the providers who took part in the targeted online consultation. However, a more in-depth analysis of both the Commission’s practical monitoring arrangements (based mainly on the data stored in the Commission’s internal IT system) and Member State practices revealed that all the coordinating authorities consulted wanted additional support from DG REFORM in order to understand their monitoring prerogatives and to adapt their review capacity. The existence of different national approaches to monitoring activities showed the need for coordination and a procedural framework that could help coordinating authorities and beneficiaries to have a clear, strategic overview of the overall support received as well as progress and results achieved. This, in turn, would help to maximise the effects of the technical support, allowing national authorities to draw lessons and better define future needs. It is important to underline that since the SRSP 2018 cycle, the Commission design of the monitoring mechanism has evolved significantly and continues to do so. Analysis of the information available in April 2019 on the Commission’s internal IT system revealed that improvements were needed, in particular to have more complete information on expected activities, expected results, state of play of activities and results. Hence, as a result of the internal feedback processes in DG REFORM, the Commission’s working methods have been continuously developed, reviewed and improved. Furthermore, additional tools have been used to monitor the progress of the projects. Progress reports have been used by the providers as a formal mechanism to inform the beneficiaries and DG REFORM of the progress of projects. At the same time, meetings and events, as well as calls and emails have served as informal tools to ensure that all stakeholders were updated on the progress of projects. From the targeted consultations, it also emerged that the good collaboration with and continuous support from the DG REFORM staff were strong determinants of the quality of the monitoring system. Similarly, the stakeholders had a positive perception of the coordination between DG REFORM and the rest of the Commission. All the other Commission departments interviewed saw their cooperation with DG REFORM as efficient.

The coordination mechanism was structured in such a way that potential overlaps and double financing were identified and avoided both at the technical level, through the feedback provided by the Commission departments on technical support requests, and at strategic level, through the High-Level Steering Committee meetings of representatives of different Commission departments. Dialogue between DG REFORM’s policy officers and their counterparts in the relevant Commission departments has also been strengthened. This system is a determining factor in making the SRSP action efficient and coherent with other EU programmes (for more information on the coherence of the SRSP, see Section 5.4 ). While the coordination mechanism was effective (i.e. no double funding), it was also pointed out that sharing information on the actual implementation of the projects, their outcomes and lessons learned could lead to more efficient cooperation between DG REFORM and the Commission departments. This, in turn, would result in better assessment of overlaps and complementarities and make it easier to assess progress with respect to the implementation of country-specific recommendations. One of the most prominent factors that mitigates such considerations to a large extent is the use and continuous development of the Commission’s internal IT system, which fostered the exchange of information between DG REFORM and the other Commission departments, as pointed out by most of the Commission departments consulted (4 Directorates-General out of 6). The IT system facilitated the cooperation mechanisms because it enabled easier monitoring (i.e. a centralised approach, extraction of reports) and communication (i.e. direct contact at technical level between DG REFORM and other Commission departments). The efficiency of the governance system, the SRSP cycle and cooperation with the other Commission departments can be reflected in the the growing number and increasingly good quality of the technical support requests and in the increasing number of Commission departments taking part in the procedures for the analysis and selection of technical support requests. Figure 20 shows an increase in the number of requests of better quality analysed by DG REFORM under SRSP 2018 compared to SRSP 2017 and an increased number of Commission departments. The analysis showed that the quality of technical support requests has considerably increased from one SRSP annual cycle to another. SRSP 2017 SRSP 2018 Budget EUR 22.5 million EUR 30.5 million Requests received 271 (worth EUR 80 million) 444 (worth EUR 150 million ) Directorate-General that commented 15 Directorates-General 25 Directorates-General Average score of selected requests 2.68 2.91 Average score of requests not selected 1.87 2.08 Requests selected 159 146 Member States 16 Member States 24 Member States Average size of project EUR 140,000 EUR 207,000 Figure 20: Comparison between SRSP 2017 and SRSP 2018 cycles To conclude, the findings suggest that the governance of SRSP and the cooperation between DG REFORM and other Commission departments have been efficient. The duration of the design of the technical support, meaning both requests and projects, seems to have corresponded to the needs of the target group. At the same time, the quality of the technical support requests increased from one SRSP annual cycle to another. Furthermore, the programme’s implementation, including its monitoring, was aligned with the needs of the target groups. The analysis and the stakeholder feedback highlighted that the Commission’s current internal IT system is largely adequate for monitoring. However, there is potential for improvement to make the system more complete. Common standards for national monitoring practices could, for example, be considered in the future. Furthermore, that the governance system and cooperation between DG REFORM and the other Commission departments were efficient was reflected in the increased quality of the technical support requests and in the positive perception of the Commission departments interviewed. Overall, the governance of the SRSP has been continuously developing, duly reflecting the changing needs of the stakeholders involved and the learning curve that goes with the creation and management of a new programme. 5.3.2.Cost-effectiveness of the support The cost-effectiveness of the SRSP projects was assessed by verifying whether the level of achievement of the expected results was commensurate with the budget already allocated. Cost-effectiveness analyses usually look at the relationship between the costs of an action and the benefits obtained. While costs are in general more easily measurable in monetary terms, the benefits tend to emerge over a longer timeframe and are generally harder to measure objectively. Considering that the object of the mid-term evaluation was the implementation of a newly developed programme for the first 2 years, and given that only the results and not the benefits could be monitored at this point in time, the analysis considered project results as a proxy to benefits. The mid-term evaluation assessed the correlation between financial resources allocated and the project status. From the data available in the Commission’s internal IT system, it emerged that on average, all projects were on track, as several steps had already been taken towards achieving the expected results. Figure 21 shows that the average score for the degree of achievement of the expected results for the completed projects was 3.8 out of 4. The projects that have not yet achieved their expected results were mainly ongoing projects (110 out of 121). These ongoing projects had a high likelihood that they would still achieve their expected results by the end of their expected implementation period (score 2.8 out of 3) . Funding source / Project status Degree of achievement of the expected results (score 1 to 4) Number of projects that have already achieved the expected results ( score = 4) Number of projects that have not yet achieved the expected results (score =1, 2 or 3) Total number of project subject to this analysis Completed 3.8 38 11 49 Ongoing 2.7 8 110 118 Figure 21: Degree of achievement of the expected results of evaluated projects under SRSP 2017 and SRSP 2018 (source: Commission’s internal IT system and targeted interviews, EY 2020) When comparing the status of implementation and achievement of results to the cost associated to the projects, it emerged that the financial resources of those projects having already achieved their results (30%) amounted to EUR 7.4 million, i.e. around 16% of the total budget allocated to the entire sample of projects evaluated. The cost associated to the projects that had not yet achieved the expected results was EUR 39.5 million, representing around 84% of the budget allocated to the entire sample of projects evaluated. All the results have been achieved for almost 30% of the evaluated projects that correspond to 16% of the approved financial resources, most of which are linked to completed projects. At the same time, for the remaining 70% of the projects, there is a high likelihood that the results will be achieved, most of which are linked to ongoing projects. Therefore, the achieved results are considered commensurate to the resources allocated. In conclusion, the SRSP actions can be considered cost-effective so far, while acknowledging the limited scope of the analysis. The achieved results are commensurate with the budget used. All the results have been achieved for almost 30% of the evaluated projects, which correspond to 16% of the approved financial resources, most of which are linked to completed projects. At the same time, for the remaining 70% of the projects (i.e. 84% of the amounts), there is a high likelihood that the results will be achieved, most of which are linked to ongoing projects. 5.3.3.Time-efficiency of the budget execution The time efficiency of the budget execution refers to the capacity to contract the financial resource allocated (i.e. absorption capacity ) and the capacity to make payments on time for the contracts/agreements that are signed. Given that information on payments made under the SRSP was not available at the time of the evaluation, the analysis focused on the alignment of the absorption capacity to the status of the projects, as well as on the status of the means of delivery and the existence of possible delays. The analysis of the absorption capacity determined what percentage of the approved financial resources for 267 projects under SRSP 2017 and SRSP 2018 had been contracted. It emerged that 71% of the overall approved budget for the evaluated projects had already been contracted, i.e. EUR 37 million out of EUR 47 million. The findings suggest that the absorption rate is aligned with the two-year duration of the SRSP project cycle, as well as with the status of the contracting procedure under each funding source. Procedures under SRSP 2017 are at a more advanced stage of implementation (almost 34% have been completed), hence the higher absorption rate (95.8%). At the same time, most of the contracting procedures under SRSP 2018 are ongoing (more than 90%), which is correlated with a lower contracting rate, i.e. 67% ( Figure 22 ).

Funding source / Project status # of contracting procedures evaluated Approved amount EUR (in million) Contracted amount EUR ((in million) % of approved amount already contracted amount SRSP 2017 146 (100%) 20 19.2 95.8% Completed 49 (33.6%) 5.3 5.1 96.5% Ongoing 97 (66.4%) 14.8 14.1 95.6% SRSP 2018 158 27 17.9 66.5% Completed 15 (9.5%) 1.2 1 84.1% Ongoing 143 (80.5%) 25.9 17 65.7% Total 303 47 37.1 81.1% Although the absorption rate is aligned with the SRSP cycle, there are signs that some delays might occur , particularly as the share of means of delivery with risk of delays increased from 9% under SRSP 2017 to 17% under SRSP 2018. The distribution of the number of means of delivery with risks of delays is presented in Figure 23 . Of these, indirect management, specific contracts under framework contracts and grant direct awards are the means of delivery with the highest shares of risks of delays. The rise in the share of procedures with risks of delays could be a symptom for the lower contracting rate registered under SRSP 2018 compared to SRSP 2017. Figure 23: Means of delivery with the highest shares of risks of delays (source: Commission’s internal IT system and targeted interviews, EY 2020). To conclude, although some minor risks of delays were noted, the financial resources were contracted in a time-efficient manner, as shown by the alignment of the absorption rate with the status of the projects.



5.4.Coherence In line with the Better Regulation guidelines, the assessment of the programme’s coherence is divided into two parts: internal and external coherence. Internal coherence covers the design of an EU intervention. It looks at how the different components of the intervention interact, and whether they are logically linked to achieve the objectives set. The external coherence is part of an evaluation that examines how an EU intervention interacts (contradicts, overlaps, complements) with other programmes, instruments, policies, and actions. Specifically, the SRSP evaluation comprised the following sub-analyses: ·an explanation of the rationale of the SRSP’s logic of intervention (internal coherence); ·an analysis of how the SRSP is integrated in the EU’s economic governance process (external coherence); ·an analysis of the extent to which the SRSP interacts efficiently with other EU interventions and Commission Directorates-General (DGs), particularly in preventing double funding (external coherence); ·an analysis of the extent to which beneficiary authorities are aware of other programmes and policies with similar objectives as the SRSP (external coherence). 5.4.1.Rationale of the logic of intervention The rationale of the SRSP intervention is visualised in the programme’s logic of intervention ( Figure 1 ). The causal relations between the different components of the programme can be clearly identified. All the projects under the evaluation cover all types of eligible actions. These eligible actions delivered the expected outputs or are expected to still do so. Subsequently, the outputs are expected to deliver the results, which, in turn, are expected to trigger the defined impacts. For example, the experts in a specific policy area (eligible action) are expected to deliver a reform strategy (expected output). The strategy is then expected to be adopted by the relevant Member State (expected result). Subject to contextual factors, the correct adoption and implementation of the strategy, in turn, is expected to trigger enhanced sustainable growth, job creation and social inclusion, or greater cohesion, competitiveness, productivity and investment (expected impact).

At the level of individual projects, the expected activities, outputs and results/outcomes are defined in the description of the action or the terms of reference, depending on the means of delivery. These are drafted following extensive consultation with the beneficiary authority about the needs and expected results. 5.4.2.EU economic governance process Naturally, the SRSP exists in the wider structures of EU policies and interventions. DG REFORM is also closely involved in the European Semester process, where it interacts with other Commission departments. The European Semester is a framework for the coordination of economic policies across the EU. Each year, the Commission carries out a detailed analysis of each Member State’s plans for budget, macroeconomic and structural reforms. It also provides EU governments with country-specific recommendations for the following 12-18 months. The projects under the SRSP are supporting the European Semester. Both the SRSP and the European Semester are organised in annual cycles, which feed into one another ( Figure 24 ). As visualised in the figure above, European Semester country reports (including the CSRs) and the national reform programmes in a given year can serve as a guiding document for defining Member States’ requests for technical support under the SRSP in the following year. The DG REFORM country coordinators participate in the European Semester country team meetings to ensure that the knowledge and experiences gained from the SRSP feed into the European Semester. These meetings have replaced the part of the coordination mechanism involving the technical support country teams. The DG REFORM country coordinators inform the European Semester country teams about the requests submitted and about the state of implementation of the projects linked to the European Semester. In addition, SRSP projects can provide the European Semester country teams with information from the ground on the state of play of certain reforms in Member States and reasons behind obstacles in specific policy areas. This input optimises the quality of the country analysis. Moreover, the progress on the implementation of SRSP projects is included in the country report of each Member State. The majority of the projects selected under SRSP 2017 (51%) and SRSP 2018 (55%) were related to the implementation of reforms in the context of the economic governance process, in particular to CSRs. Of the projects evaluated that were linked to European Semester recommendations, 21.6% have already achieved their expected result . Of the remaining projects that did not achieve their expected results, 82.8% have a high likelihood of doing so . The fact that a significant share of the SRSP projects are related to the implementation of CSRs, as do the aligned governance structures, demonstrates that the SRSP contributes to the economic governance process. A potential for further improvement and even stronger synergies to optimise the integration in the economic governance process can be explored. 5.4.3.Interaction between DG REFORM and other Commission departments and instruments As stated above, the European Semester is a process that allows for a comprehensive approach towards structural reforms and the coordination of different actions and policies, in which DG REFORM plays an integral role. DG REFORM has put in place a coordination mechanism in order to ensure that the technical support provided is aligned with the activities and policies of other Commission departments and to prevent any financing overlap. DG REFORM has the mandate to steer and coordinate the technical support provided by the Commission to the Member States. The coordination mechanism has the following horizontal structure: ·A ‘High-Level Steering Committee’, which meets at least once a year (or per SRSP funding round) at senior management level, and makes recommendations and provides guidance on all aspects of the technical support provided to Member States, including funding priorities. The High-Level Steering Committee also endorses the list of requests proposed for funding under a funding round; ·A ‘Technical Support Working Group’, which meets quarterly to exchange information on the provision of technical support, optimise cooperation between Commission departments, and prepare the process of consultation on requests for support received under the SRSP. As stated above, the Commission’s policy departments are consulted for the assessment of the technical support requests. Given that DG REFORM is itself not a policy department, the appropriate design and implementation of projects requires interaction with departments that are responsible for defining the Commission’s policies and that potentially fund actions with similar objectives. The consultation with policy departments allows these departments to inform DG REFORM of links between the request and their policy priorities. Moreover, this gives these departments the opportunity to signal if a request overlaps with projects funded under their programmes or instruments, potentially leading to double funding. Lastly, departments can identify synergies between the requests for technical support and the actions or projects that they themselves fund . Hence, the consultation between departments allows the Commission as an organisation to timely identify synergies and overlaps. The findings from scoping interviews and targeted interviews with departments confirm that the complementarities between the SRSP and other programmes are ensured to a large extent. The interviews point out that this is mainly due to DG REFORM’s efforts and to the high level of collaboration and communication between the Commission’s different departments. These findings are in line with the outcomes of the 2018 Annual Activity Report, which show that all Commission departments are satisfied with the efficiency of the coordination mechanism. The interviews with Commission departments also confirm DG REFORM’s efforts to avoid funding overlaps through the checking system mentioned above. However, respondents also indicated that approving the SRSP in the middle of the programming period creates challenges for the other EU programmes with similar objectives. SRSP support is directly complementary to many other Commission support programmes, most notably the European Structural and Investment Funds (ESIF) and Fiscalis 2020. To ensure close coherence and complementarity between the technical support and assistance provided under the ESIF funds and that provided under the SRSP, different terminology is used (See box below). Box: Technical assistance versus technical support

Technical support is provided to Member States under the Structural Reform Support Programme to help Member States implement structural reforms. Technical assistance is funded from the European Structural and Investment Funds (ESIF) to help Member States implement ESIF-funded programmes and projects. The scope of ESIF techical assistance to policy reforms is geared to the policy areas falling within the remit of ESIF. For instance, in a number of Member States, the European Social Fund (ESF) supports in-depth reforms or elements of reforms in employment, education and social policy areas, as well as public administration reforms. These reforms are aligned, similarly as in the case of SRSP support, to country-specific challenges identified under the European Semester. The SRSP is active in a vast array of policy areas also going beyond those covered by the ESIF, and the SRSP technical support can take place in any stage or phase of the reform process, also using quite pervasive tools of assistance (such as continuous support on the ground). There is anyhow a range of various EU instruments is at the disposal of Member States to draw upon. Until 2019 (but particularly before the SRSP’s adoption in 2017), technical assistance to tax administrations was also funded through the Fiscalis 2020 programme. The programme aims to ensure that the taxation systems in the internal market work better by promoting closer cooperation between participating countries, their tax authorities and their officials. Some overlaps with the SRSP are possible in the areas of public administration, growth (ESIF), and labour markets (ESF). Technical assistance under the structural funds is not, in principle, provided for support in the area of financial services and tax administration / public financial management. Technical support under the SRSP helps Member States to design projects to implement structural reforms linked to the EU governance processes and priorities, including national priorities in line with EU priorities; this technical support can be provided in any stage or phase of the reform process, using quite pervasive tools of assistance as well (such as continuous support on the ground). This is why the scope of technical support is much broader and deeper than the scope of ESIF technical assistance. At the time of writing this staff working document, there were a number of checks in place to ensure that synergies and funding overlaps were being timely identified at Member State level. By submitting their technical support requests under the SRSP, Member States are obliged to inform the Commission immediately of other related ongoing actions financed out of the EU budget . Subsequently, if a request is selected for funding, the relevant Member State is required to confirm to the Commission that there is no overlap between the selected request under the SRSP and concrete actions funded under other EU instruments, and that there is no double funding. The coordinating authority of a Member State is also obliged to send a signed letter to the Commission, confirming the absence of double funding for all approved technical support requests under the SRSP of that specific annual cycle, after having consulted the relevant national managing authorities of ESIF funds. The optimisation of synergies with other actions and projects and the declaration that there is no double funding and no duplication of efforts are integral parts of the guiding principles of the cooperation and support plans. By signing the plan, the Member State commits to comply with these principles. At the time of the mid-term evaluation, the verification systems mentioned above were gradually being put in place. The case studies indicate that at the time of evaluation, the verification systems for double funding were relatively informal. Given this context, the general perception of the stakeholders consulted is that the process could be improved by further formalising the procedures for checking for double funding before the submission of requests and after their selection. The evaluation also suggests that Member States would need to increase their capacity in order to perform more in-depth checks. As described above, many improvements have already been made since then. 5.4.4.Awareness of other policies and programmes Lastly, the evaluation assessed the extent to which the beneficiary authorities were aware of other policies and programmes whose objectives were similar to the SRSP’s. As indicated in Figure 25 , 25.2% of the responding beneficiary authorities stated that they were indeed aware of these programmes and policies . The EU programme similar to the SRSP that is most commonly known among beneficiary authorities is, according to the evidence from the targeted online consultation, the European Social Fund . Two other EU programmes identified in the public consultation were the European Regional Development Fund and the Cohesion fund. One of the reasons for requesting support under the SRSP could be the relatively low administrative burden, as DG REFORM carries out the contractual procedures . The findings suggest, however, that beneficiary authorities sometimes lack an overview of all funding alternatives. From the targeted online consultation and case studies it also emerged that for some respondents the SRSP was appreciated because the entirety of the support is covered by the SRSP and that there is no need for co-financing at national level. Figure 25: 'Are you aware of other policies or programmes implemented with European/national funds that have similar objectives as the SRSP? (source: targeted online consultation of beneficiary authorities, EY 2020). In conclusion, the SRSP can so far be considered internally and externally coherent to a high degree, taking into account the limited scope of the mid-term evaluation. Firstly, the different components of the logic of intervention have clear causal links. Furthermore, the SRSP is considered to be supportive of the EU economic governance process. The SRSP and the European Semester are adequately feeding into each other, providing in both directions input to enhance the coordinated approach towards achieving the objectives. The fact that a significant share of the SRSP projects are related to the implementation of CSRs demonstrates that the SRSP contributes to the economic governance process. In addition, the evaluation evidence shows that complementarities between the SRSP and other programmes are ensured to a large extent. DG REFORM has continued to optimise its coordination mechanism, which allows for the coordination of technical support with other Commission departments. Synergies and the absence of double funding are best ensured through consultation with other Commission departments during the assessment of the requests submitted under the SRSP, and through the declaration of absence of double funding for the selected request for support. At the level of the Member State, double funding checks are carried out in a relatively informal manner. These procedures have been further developed and formalised in subsequent years, in line with the recommendations of the SRSP mid-term evaluation. Lastly, the findings from the targeted online consultation suggest that beneficiary authorities are sometimes aware of other programmes and policies with objectives similar to the SRSP’s but lack an all-encompassing overview of these funding alternatives.

5.5.EU added value

In accordance with the principle of subsidiarity in areas of non-exclusive competence, the EU should only intervene when the objectives can be better achieved at EU level than at the level of the Member State. In this light, the evaluation assessed what resulted from the SRSP-funded activities in value that was above what could reasonably have been expected from Member States acting at national or regional levels. Given that the programme was only set up recently, there are limitations as to how much EU added value can be measured in terms of impact. Taking this into consideration, the overall assessment of the programme’s EU added value was composed of the following two sub-analyses: ·an explanatory assessment of why SRSP support is managed at EU level and to what extent this support is requested by Member States; ·an examination of good practices exchange, cooperation among institutions from different Member States and cross-border technical support facilitated by the programme. 5.5.1.Rationale of the SRSP and Member States’ demand All Member States should benefit from a resilient and flexible European economy. A strong European economy relies on Member State economies that function well and are adaptable. However, Member States sometimes lack the required expertise or administrative and institutional capacity to carry out reforms that the European Union recommends to them. The same can be observed for the application of EU law. This inadequacy to reform could jeopardise sustainable growth across the EU. By providing EU-level technical support for reforms, it is possible to tackle knowledge and capacity constraints in Member States. Highly needed reforms could otherwise remain unaddressed at national level. In other words, in order to achieve the objective of economic growth shared by all Member States, technical support provided at Union level is needed to help those Member States and public institutions that cannot address the reform challenges with their own national means. In addition, the relatively low bureaucratic and administrative burden of the SRSP for beneficiary authorities, together with the absence of co-financing, contributes to Member States making use of the available support. The evidence from the evaluation shows that this is particularly relevant for beneficiary authorities coping with limited financial resources or administrative capacity. Often, these institutions are the ones most in need of reforms. The evidence on the submitted requests suggests a high demand for technical support managed at EU level. As for the period evaluated, every year, an increasing number of Member States request technical support under the SRSP. Moreover, 80% of the Member States benefiting from support under SRSP 2017 and SRSP 2018 had increased their number of submitted requests from one year to the next. In total, 25 Member States received technical support under the Preparatory Action (2016), SRSP 2017 and SRSP 2018. 5.5.2.Exchange of good practices and cross-border cooperation The SRSP’s EU added value also lies in its ability to facilitate the exchange of good practices among different Member States. The evidence collected shows that 52.5% of the projects evaluated made use of good practices and lessons learned. As outlined in Figure 26 , the main types of outputs that include good practice sharing are: (1) recommendations, (2) analyses and reports and (3) workshops, training and training materials. Furthermore, from the evaluated means of delivery that used good practices and lessons learned, 28.2% was provided by a public sector entity, such as government bodies or public servant experts. Naturally, support provided from one public entity to another enables the sharing of good practices among EU countries. Evidence from targeted online consultation shows that most beneficiary authorities (89%) perceive an increase of knowledge sharing between their institution and institutions from other Member States ( Figure 27 ). Likewise, 91% of the technical support providers perceived an increase in knowledge exchange. The respondents of the public consultation were positive, too, about the programme’s EU added value in terms of sharing good practices and promoting cooperation.

Technical support under the SRSP can also address reform challenges, which are common to several Member States. This can take two forms. In the first type, several Member States may make similar requests for support to address a common reform challenge, if they realise that their reform should be best tackled across national borders. All the interested Member States still need to submit their SRSP requests individually; one Member State cannot apply for support on behalf of another Member State. Second, if different Member States submit requests to tackle a similar issue, even though the relevant countries might be geographically far apart, the Commission can identify synergies between these individual support requests. In cases deemed feasible, the Commission can decide to implement requests from different Member States which are related to one another, through one project, with a view to achieve efficiency gains in terms of contracting. Instead of an individual technical support project for each request, one project would cover the requests of several Member States. Each Member State would in any case still receive individualised support, applicable to their local situation. The findings from the evaluation demonstrate that the extent to which individual SRSP projects tackle cross-border challenges in multiple countries is low. Only one out of all the projects evaluated (0.3%) combine independently requests submitted from different Member States into one multi-country project. The reason for the limited joint technical support actions in different Member States is that the requests for support are country-specific, related to national priorities in line with EU goals. The needs of Member States often have their own unique characteristics, as challenges are embedded in the specific national context. The exchange of information among beneficiary authorities from different Member States during the design of the requests is also not advanced. This could be further enhanced in those cases where it is deemed feasible and of added value. The findings suggests that the Commission needs to optimise further the identification of synergies between individual requests from different Member States, to ensure that technical support efforts are complementary and closely related where relevant and practically achievable. Lastly, the sharing of solutions and knowledge after the project’s implementation among Member States seems to be limited. Although there are individual cases in which knowledge sharing took place after the closure of the project, this is not yet a common practice. Nevertheless, these individual cases suggest that the programme can provide technical support for cross-border cooperation. A large share of the respondents of the public consultation positively assessed the programme’s added value in terms of addressing cross-border or Union-wide challenges. In conclusion, the technical support provided under the SRSP is intended to target precisely those reform challenges that Member States cannot address with their own means and for which they seek expertise beyond the national level. The increasing number of Member States that requested support under the programme, as well as the increase in requests submitted, implies a high demand for technical support at EU level. Furthermore, many projects include activities that allow for the exchange of knowledge and good practices among Member States.The evidence suggests that the Commission is not yet sufficiently facilitating information exchange between beneficiary authorities in different Member States during the design phase of the project. Considering that the programme’s internal processes and procedures have by now taken their final shape, the exchange of information among beneficiary authorities should be reinforced. The Commission could also build on experiences from successful cases of cross-border cooperation, including by optimising the identification of synergies between individual SRSP requests. At the same time, Member States could seek cooperation for requests under SRSP that cope with common challenges. 6. Conclusions The Structural Reform Support programme was adopted in 2017 to help Member States in the process of designing and implementing institutional, administrative and growth-sustaining reforms, by providing technical support to national authorities. This mid-term evaluation assessed the SRSP 2017 and 2018 budgetary cycles. The overall goal of the evaluation was to examine how the SRSP is performing against the evaluation criteria (relevance, effectiveness, efficiency, coherence and EU added value) and to draw lessons that can serve possible future improvements. Overall, the quality and representativeness of the collected evidence is considered satisfactory. The evaluation covered over 300 technical support projects and included different evidence collecting techniques. The most considerable limitation to the robustness of the findings was the fact that the programme had only very recently been adopted at the time of the evaluation. Many projects were still in their initial phase, posing constraints on measuring the SRSP’s results, as well as impacts. At the same time, some internal procedures and processes, and the information system for monitoring projects, were only gradually being implemented. This limits the extent to which fixed conclusions on the SRSP’s success can be drawn in this mid-term evaluation. Nevertheless, the Commission’ staff considers that the findings have an adequate level of robustness to provide preliminary conclusions and lessons learned on the SRSP. Regarding the relevance of the programme, the findings of the evaluation show that the design of the SRSP is aligned with Member States’s actual needs. The evidence demonstrates that the Member States’ needs, as identified in their request for support, have all been addressed to a large extent by the programme. Stakeholders indicate that the SRSP is a suitable instrument for providing technical support to Member States to design and implement growth enhancing reforms. Additionally, Member States especially appreciated the high degree of flexibility, the simplicity of the contracting and implementing procedures managed by the Commission and the absence of co-financing requirements. The findings suggest that the SRSP remains a relevant instrument for Member States. Technical support continues to be needed, given that reforms are linked to long-term and wide needs. The continuous need for support is also connected to the experience and capacity that the beneficiary authorities gain with the instrument, as the beneficiary authority might identify other needs and potential areas for support under the programme, after finishing one project. These outcomes correspond with the Commission’s expectations. The analysis of the programme’s effectiveness shows that the projects funded under the SRSP are generally well designed. The way in which the Commission takes into account the specific needs of Member States, as well as the relevant socio-economic impacts, is considered a good and common practice according to the stakeholders. This makes it also possible to translate the needs into feasible actions with realistic timelines; only a few projects were delayed. The fact that technical support is generally provided in a comprehensive manner, with satisfactory consideration of the situation of the individual beneficiary authority, is a good practice and meets the expectations of delivering tailor-made technical support. Furthermore, the findings demonstrate that the SRSP is progressing towards achieving its objectives, taking into account that many projects were still ongoing at the time of the evaluation. Most of the technical support projects are likely to achieve their results, or have already done so. However, the exchange of good practices in terms of project management among Commission project managers could be reinforced in order to increase further the number of successful projects. Evidence also shows that changes in the political structure of the beneficiary authority, as well as a lack of collaboration among stakeholders, are important obstacles to the delivery of the projects’ outputs. These and other negative factors should be avoided and mitigated to the extent possible. As for the SRSP’s impact, it is premature to draw solid conclusions. Although the stakeholders’ perception of the impact is considered proportionate to the programme’s level of completion, a comprehensive evaluation of the long-term impacts of the supported interventions should be carried out at a later stage, as part of the ex-post evaluation. As regards the programme’s efficiency, the evaluation findings indicate that governance of the SRSP is efficient. Overall, the programme’s cycles, including the design and implementation of the technical support, are aligned with the beneficiary authorities’ needs. The analysis also showed that the current Commission’s internal IT system is largely adequate as a monitoring arrangement, although it still needed further improvements at the time of evaluation. Additionally, all departments indicated that their cooperation with DG REFORM was efficient. Strong points highlighted by the stakeholders interviewed were: the efficiency of the governance system, the SRSP cycle a