Table of contents Glossary 1Introduction 2Background to the intervention 2.1 Rationale for setting up the TFGR 3Objectives and intervention logic 4Implementation 4.1 Setting up and organisation of the TFGR 4.2 Resources allocation and distribution 4.3 Identification of the needs 4.4 Identification of technical assistance expertise 1) For specific policy areas: selection of Domain Leaders 2) For all policy areas: selection of providers 5Methodological approach 5.1 The external evaluation carried out in 2018 5.2 Limitations of the external evaluation 6Analysis and answers to the evaluation questions 6.1 Relevance

6.2 Effectiveness 6.3 Efficiency 6.4 Coherence 6.5 EU added value 7Conclusions 7.1 Lessons learned Glossary Term or acronym Meaning or definition AROP Administrative Reform Operational Programme CF Cohesion Fund DG ECFIN Directorate General for Economic and Financial Affairs DG EMPL Directorate General for Employment, Social Affairs and Inclusion DG TAXUD Directorate General for Taxation and Customs Union EC European Commission ECA European Court of Auditors ECB European Central Bank EOPYY National Organization for Healthcare Provision EQ Evaluation Question ERDF European Regional Development Fund ESF European Social Fund EU European Union HCA Hellenic Court of Audit IMF International Monetary Fund ITC Information Technology and Communications JC Judgment Criteria MoU Memorandum of Understanding MS Member State OECD Organisation for Economic Co-operation and Development OAED Hellenic Manpower Employment Organization SF Structural Funds SRSS Structural Reform Support Service TA Technical Assistance TFGR Task Force for Greece UNECE United Nations Economic Commission for Europe WB World Bank 1.Introduction This Staff Working Document (SWD) presents the results of the ex-post evaluation of the actions carried out by the Task Force Greece (TFGR) during the 2011-2015 period. It follows a recommendation from the European Court of Auditors (ECA) issued in the context of the Special Report no 19/2015: “More attention to results needed to improve the delivery of Technical Assistance (TA) to Greece”. The ECA report raised two main questions: 1.To what extent was the EU technical assistance provided to Greece under the coordination of the TFGR managed effectively? 2.To what extent has the EU technical assistance provided to Greece under the coordination of the TFGR contributed positively to the process of reforms in Greece? This SWD provides a reply to these questions. It is based on the evidence gathered through two external evaluations: I)the “Preliminary evaluation of the Technical Assistance provided to Greece in 2011-2013 in the areas of Tax Administration and Central Administration Reform” ; and II)the external study on “Evaluation of the Technical Assistance provided by the Task Force for Greece 2011-2015”. The three economic adjustment programmes for Greece are out of the scope of this evaluation. 2.Background to the intervention

2.1Rationale for setting up the TFGR ad hoc TFGR in order to coordinate and provide the requested technical assistance. Acting upon request of the Greek authorities, the mandate of the TFGR was twofold: 1) To provide technical assistance to the Greek authorities to support the delivery of commitments undertaken under the economic adjustment programme; and 2) To provide technical assistance to the Greek authorities for the acceleration in the absorption of EU funds. The TFGR regularly reported on its activities through quarterly reports. It concluded its mandate at the end of June 2015. 3.Objectives and intervention logic The specific mandate of the TFGR was structured around four specific objectives: 1)To identify, coordinate and agree on the details of the technical assistance needed, upon request and in close coordination with the Greek authorities; 2)To provide support regarding the legislative, regulatory and administrative framework; 3)To make available to the relevant Greek authorities the needed technical assistance; and 4)To increase the stakeholders’ awareness of the progress of TFGR activities. The intervention logic of the TFGR was based on the expectation that the technical assistance activities and outputs would be defined and agreed in detail with the Greek authorities, based on their requests, in order to respond to the identified needs. The TFGR identified the needs of the Greek Authorities through study visits and expert missions. Following the assessment of the needs, the TFGR planned and organised the appropriate technical assistance with the view to formulate recommendations, guidelines, roadmaps and other guiding documents for implementing specific milestones of the economic adjustment programme, upon request of the Greek Authorities. Figure 1 below shows the intervention logic of TFGR. Figure 1: The TFGR intervention logic Source: Ernst&Young, Analysis 2018

4.Implementation This section includes a description of selected elements of the TFGR operations.

4.1Setting up and organisation of the TFGR The President of the European Commission established the TFGR in July 2011, as an organisational entity initially embedded in the Commission’s Director-General of Human Resources and Security. The Head of the Task Force, appointed at a level equivalent to the rank of Director-General Figure 2 ensure the highest possible levels of synergies, coordination and coherence of the Commission’s work to support the economic adjustment programme of Greece .







Figure 2: The TFGR organisational structure Source: Ernst&Young, Analysis 2018 4.2Resources allocation and distribution . The total budget spent by the TFGR for the technical assistance to Greece during 2011- 2015 was EUR 12.85 million. The areas of intervention covered the Public Administration and different economic sectors, in particular: 1)Administrative reform at central level, e-Government; 2)Administrative reform at regional and local level; 3)Anti-corruption, anti-money laundering, anti-fraud; 4)Business environment and reform of product and service markets, public procurement, better regulation; 5)Financial sector and access to finance; 6)Healthcare; 7)Judicial reform, migration, asylum, state aid, infringements;

8)Labour Market, social security, innovation & education (R&D&I); 9)Privatisation, land use and land registry; 10)Revenue administration and public financial management; 11)Structural Funds and sectoral issues. Figure 3 below presents the distribution of this amount per policy area in descending order. The 39% of the resources have been devoted to interventions in the field of “Revenue Administration and Public Financial Management” Figure 3: Financial resources deployed by the TFGR, by policy area Source: Ernst&Young, Analysis 2018 based on technical assistance data of the TFGR, from the European Social Fund (ESF), the European Regional Development Fund (ERDF) (art. 45 Regulation 1083/2006) and the FISCALIS programme. 4.3Identification of the needs by the TFGR 4.4Identification of technical assistance expertise The identification of the relevant technical support expertise was done in one or two main waves, depending on the policy area: 1)For specific policy areas: selection of Domain Leaders The responsibility of Domain Leader was undertaken by a national government with a strong track record in a policy field of particular complexity (reform of the public administration, for example). This role had varying characteristics depending on the phase of technical assistance. In the first phase, its role was to assist the Greek authorities in preparing the global architecture and the strategy to implement the necessary reforms. More specifically, the Domain Leader, in close relation with Greek authorities and the TFGR, prepared a draft roadmap, in coordination with TFGR, which would be submitted to the Greek authorities for final adoption. In the second phase, the Domain Leader consulted the Greek authorities for defining which actions and/or sub-actions Greece needed technical assistance and what kind of technical assistance should be provided. 2)For all policy areas: selection of providers (

Table 1: Mapping of Domain Leaders and Member States providing support during the TFGR intervention Policy areas supported by TFGR Domain Leader(s) Member States providing support through public officials, private experts and public agencies Acceleration of Cohesion Policy projects No Domain Leader Austria, Belgium, Estonia, France, Germany, Hungary, Ireland, Italy, Lithuania, Netherlands, Poland, Spain, United Kingdom Financial institutions / Access to finance No Domain Leader Austria, Czech Republic, Denmark, France, Germany, Ireland, Italy, Netherlands, Portugal, Sweden, United Kingdom Administrative reform France on central administration; Germany on regional and local administration Austria, Belgium, Estonia, Finland, Ireland, Netherlands, Portugal, Spain, Sweden, United Kingdom Revenue administration and public financial management Belgium on debt collection; France on high wealth individuals; Netherlands on dispute resolution and on the Hellenic Court of Audit (HCA) Austria, Denmark, Estonia, Finland, Germany, Italy, Lithuania, Luxembourg, Poland, Portugal, Spain, Sweden, United Kingdom Anti-money laundering and anti-corruption Netherlands on anti-money laundering Austria, Belgium, Germany, Norway Business environment Netherlands for export promotion Austria, Bulgaria, Cyprus, Finland, Germany, Ireland, Italy, Lithuania, Netherlands, Poland, Portugal, Sweden, United Kingdom Healthcare reform Germany Belgium, Estonia, France, Lithuania, Netherlands, Portugal, Sweden, United Kingdom Judicial reform Austria Estonia, France, Germany, Italy, Netherlands, Poland, Slovenia, Spain, United Kingdom Judicial reform, central State Aid Unit and infringements, migration, asylum and borders No Domain Leader Austria, Estonia, France, Germany, Ireland, Italy, Netherlands, Poland, Portugal, Slovenia, Spain, United Kingdom Labour market, social security, innovation and education No Domain Leader Austria, Belgium, Czech Republic, France, Germany, Ireland, Netherlands, Portugal, Sweden, United Kingdom Privatisation and land registry Netherlands on land registry Austria, Czech Republic, Estonia, France, Germany, Italy, Netherlands, Romania, Spain, Sweden, United Kingdom Industry sectors and services No Domain Leader Austria, Belgium, Estonia, Finland, France, Germany and Italy Source: Ernst&Young, Analysis 2018 - Assessment based on the TFGR quarterly progress reports (2011 to 2014, listed individually in the bibliography), minutes of the high-level meetings, European Commission, TFGR (2012-2014). The timeline of the TFGR operations is shown in more details in Figure 4 .

Figure 4: Milestones and the timeline of the TFGR Source: Ernst &Young, Analysis 2018 5.Methodological approach This Staff Working Document assesses the relevance, effectiveness, efficiency, coherence and added value of the TA activities coordinated and provided by the TFGR. . Because of the extensive consultation carried out on targeted stakeholders by the external evaluator, no open public consultation took place. 5.1The external evaluation carried out in 2018 The methodology applied in the above-mentioned evaluation for gathering data refers to three main sources of data: 1)Desk research of relevant documents: such as internal assessments of the Commission services, as well as external evaluation studies, including the results of the preliminary evaluation carried out in 2013. 2)A structured electronic survey; 3)In-depth semi-structured interviews. 1)Strategic and management – institutions and individuals involved in the set-up, overall strategic steering and day to day management of TFGR interventions; 2)Beneficiaries – institutions benefiting from technical assistance interventions coordinated by TFGR; 3)Providers – international organisations, national agencies and individual experts providing technical assistance.

The structured electronic survey targeted 278 stakeholders who were distributed among the three categories as follows: 1)Strategic and management: 4% 2)Beneficiaries: 33 % 3)Providers – including individual experts from the Domain leaders: 63% Out of the 278 individuals consulted for the survey, 101 responded to the online questionnaire, representing a response rate of 36%. Source: Ernst & Young Questionnaire for targeted stakeholder consultation (2018) 1) 2) 3) 4) 5) 6) 7)

8) 9) 10) 11) 12) 13) Figure 6



Figure 6: Respondents to the survey by policy field (Number of respondents: 101) Source: Ernst & Young Questionnaire for targeted stakeholder consultation (2018) 5.2Limitations of the external evaluation This evaluation focuses on the results of the technical assistance. It is conditioned by the fact that the follow-up of the technical assistance actions was to be implemented by the beneficiaries of the assistance. This was an external factor influencing the outcomes of the TFGR activities, which was not under the TFGR responsibility. In addition, a certain delay generally occurs between the implementation of a reform and its social and economic impacts. Therefore, results and impacts of the TFGR technical assistance can only be observed in the longer term.



6.Analysis and answers to the evaluation questions 6.1Relevance This criterion has replied to the following evaluation questions:

1)To what extent were the objectives of identifying, coordinating TA needs and assisting Greek authorities relevant for achieving the two specific objectives and namely: a.Supporting the delivery of the commitments within the economic adjustment programme; and b.Accelerating the absorption of the Structural Funds. 2)To what extent were the TFGR actions and target groups relevant for achieving its overall and specific objectives? Based on the results of the desk analysis, the TFGR supported a significant number of Greek authorities, upon their request. The TFGR focused on key, central institutions responsible for the reform planning and implementation, as defined in the conditionalities of the economic adjustment programmes. To substantiate this finding, the external evaluation has addressed to the stakeholders an operational question: “To what extent the TFGR assisted all the relevant institutions that needed support for the implementation of commitments undertaken as part of the economic adjustment programme?” The majority of stakeholders (60%) stated that the TFGR assisted to a high extent or very high extent all institutions needing support for the implementation of commitments undertaken as part of the economic adjustment programme. 10% of respondents considered that the TFGR was of little or no assistance to institutions, some of them calling for more direct cooperation with local authorities. Figure 7: The extent to which the TFGR assisted the relevant institutions in implementing the commitments taken under the economic adjustment programme

(Number of respondents: 89) Source: Ernst & Young Questionnaire for targeted stakeholder consultation (2018) 6.2Effectiveness The following evaluation questions were addressed in order to assess the effectiveness of the activities of TFGR: 1)To what extent have the specific objectives of the TFGR – a) supporting the delivery of the commitments within the economic adjustment programme and b) accelerating the absorption of the SF - been achieved? 2)What have been the qualitative and quantitative effects of the technical assistance provided? 3)To what extent have the actions financed under the TFGR contributed to achieving its specific objectives? According to the external evaluation, the SF absorption of the programming period 2007-2013 improved and in the current programming period 2014-2020, clearly the implementation was triggered by the TFGR involvement, but also by the continuous support of the relevant DGs in the EC, namely DG Employment, Social Affairs & Inclusion and DG Regional and Urban Policy. By the end of 2015 Greece had the highest absorption rate in the EU (99.5%) for Structural Funds allocated under the Cohesion Fund, European Regional Development Fund and European Social Fund. This performance was driven particularly by the increase in the maximum EU co-financing rates to 100% for the 2007-2013 programmes but also by the focused efforts of all stakeholders to use EU funds under the challenging circumstances determined by the financial crisis.

TFGR has been the main direct technical assistance provider for the Greek authorities at a central-coordination level as in Greece the resources available under the technical assistance Operational Programme 2007-2013 were spent on the reimbursement of employment costs and additional operational costs (84%). A very small proportion (approximately 1% or roughly €2 million) was spent on developing systems and tools and only 0.1% on capacity-building activities. Based on the results of the desk research, the conclusions of the preliminary evaluation of 2014, and the external evaluation carried out in 2018, the main findings are the following: a)The TFGR focused on the identification of detailed needs, planning of reforms and the design of solutions; b)The detailed needs identification process was extensive and covered a large number of areas, sub-areas, and responsible institutions. Therefore, the TFGR was effective in fulfilling its mission “of integrating the TA in the adjustment programmes as an effective way to push wide-ranging changes through in a short space of time, but, as such is not conducive to achieving sustainable reform. By definition, this model of “independent intervention of TA” is not applicable outside of situations of extreme distress”. The results of the targeted stakeholder consultation are reported below: Figure 8: The extent to which the Greek Authorities’ need of support was identified in detail, as reported by the stakeholder consultation (Number of respondents: 68)

Source: Ernst & Young Questionnaire for targeted stakeholder consultation (2018) The large majority of respondents (80%) found that the needs of the Greek authorities were identified in detail to a high or very high extent. Those that disagreed considered that this was due to uncertainties in the political framework or to lack of administrative capacity from the Member State. These findings were corroborated by both Greek and international respondents. key determinants . The TFGR provided adequate support in terms of improving legislative and regulatory frameworks where requested/needed. For example in the area of absorption of EU funds, the rules relating to the speeding up of procedures for environmental licensing, expropriations, bidding and contracting procedures were prepared with support from the TFGR. The TFGR supported substantially the strategic preparation for 2014-2020 programming period for Greece including key simplification measures introduced in the relating implementing law. The support for the restructuring of administrative frameworks was provided in most policy areas, e.g. the reorganisation and consolidation of tax and customs administrations. Moreover, the TFGR extensively supported the efforts undertaken by different ministries to implement the recommendations of the Greek central administration. The role of several central institutions such as the Hellenic Court of Audit (HCA), the Single Public Procurement Authority, the General Secretariat for Public Revenue, the Labour Inspectorate and Hellenic Manpower Employment Organization (OAED) was strengthened by the TFGR technical assistance, achieving therefore the specific objective of “providing support on the legislative, regulatory and administrative framework”. 6.3Efficiency In order to assess the efficiency of the TFGR activities, the following questions have been raised: 1)To what extent did the available financial envelopes adequately cover the TA needs? 2)To what extent did the coordination, management and administrative structures in place ensure an economic and efficient use of resources in the achievement of TFGR outputs, results and impacts?

3)To what extent were the actions and outputs delivered at a reasonable cost? Efficiency has been analysed by taking into account two variables: the timely delivery of the support and the budget spent for carrying out those different activities. According to the data gathered by the evaluator, the TFGR adequately supported the technical assistance needs expressed by the Greek authorities by ensuring in a timely manner i) detailed needs identification; ii) matching with needed expertise; iii) implementation of the support measures. The TFGR was set up less than two months after the request of the Greek authorities. The needs of the Greek authorities were identified within less than a month from the moment the Greek government submitted its request for support, and high-level meetings were organised to further specify these needs. The needs identification process continued throughout the lifetime of TFGR as new policy areas, projects and work-streams were constantly added to the TFGR intervention. The efficient use of resources for the achievement of TFGR outputs, results and impacts was ensured through a timely deployment of financial, human and ICT resources, as well as by a management mode that enabled decision-making focused on the timeliness principle and by the procurement of services at a reasonable cost. Coordination mechanisms were put in place to ensure operational consistency of TFGR actions, while the administrative structures and procedures induced a reasonable level of administrative burden to the extent that they depended on the TFGR. Based on the stakeholders’ answers, the administrative processes of TFGR were assessed as satisfactory. Providers and experts were less satisfied because of the administrative burden associated to the reimbursement of expenditures. This aspect was acknowledged to be the effect of the EU general spending rules to be applied also to the financial sources available to the TFGR. Nevertheless, the overall perception of the stakeholders was that the administrative burden was reasonable and, consequently, that the TFGR administrative procedures could not have been further simplified. Some contextual factors influenced – positively or negatively - the efficiency of the TFGR interventions. According to the evidence collected by the evaluator, the factors positively influencing the efficiency of the interventions were the following: a)the solidarity of other Member States; b)the ownership of the Greek authorities; and c)the dedication and ongoing involvement of the TFGR staff. Factors limiting the efficiency of intervention were, according to the stakeholders’ opinion: a)the political instability; b)the existing administrative capacity; and c)the exiting budgetary options/constraints. The overall conclusion, based on the data analysis, is that the TFGR intervention was efficient. It supported the technical assistance needs expressed by the Greek authorities in a timely manner and within the budgetary constraints. It did so despite the initial period of funding uncertainty and the lack of a dedicated budget . 6.4Coherence In order to assess the coherence of the TFGR, the following questions were raised: 1)By design, were the TFGR actions coherent with the policies of the Greek government?

2)Was an internal coordination mechanism put in place in order to avoid overlaps with Structural Funds interventions implemented in Greece? 3)Were complementarities with Structural Funds under shared management fully exploited? According to the data gathered through the desk research and the targeted stakeholder consultation carried out by the external evaluator, the TFGR activities were in line with the commitments undertaken by the Greek Government under the economic adjustment programmes, therefore coherent with the policies. As for the internal mechanism coordination, it was successfully put in place to avoid funding overlaps with Structural Funds interventions implemented in Greece. The avoidance of overlaps between the TFGR actions and Structural Funds interventions implemented in Greece was conducted through an internal procedure applied in the case of contracts financed by the centrally managed technical assistance budgets of ESF/ ERDF. The stakeholders responding to the survey likewise confirmed the coherence between the TFGR interventions with the Greek policies. An internal coordination mechanism was put in place within the Commission to avoid overlaps of EU funds implemented in Greece. In particular, the analysis of potential overlaps between the TFGR actions and Structural Funds interventions implemented in Greece (ESF/ERDF) was conducted through an internal procedure applied by the dedicated financial and operational teams in the DG for Employment, Inclusion and Social Affairs. 6.5EU added value The EU added value has been assessed against the following sub-criteria: 1)The contribution of the TFGR activities to the development and implementation of solutions that address national challenges which have an impact on cross-border or Union-wide challenges; 2)The contribution of the TFGR activities to EU values, including solidarity; 3)The contribution of the TFGR activities to a consistent and coherent implementation of EU laws and policies. According to the data gathered by the external evaluator, through desk research and targeted stakeholder consultation, the TFGR was the catalyst for the mobilisation of tools and resources and its involvement served to trigger or accelerate reforms in Greece . The solidarity of Member States was essential for a timely identification and mobilization of expertise. This happened not only in the initial phase of needs identification, when certain Member States promptly offered to become Domain Leaders, but also in the subsequent phases. The creation of the TFGR is considered as an “ad-hoc” instrument – built under strict deadlines – for supporting structural reforms. It mobilised at the same time at the European Commission, Member States’ and international organisations’ levels, in order to address the national challenges with Union-wide effects. According to 29% of the stakeholders’ opinion, a similar national action for the provision of technical assistance would not have been possible without the EU and the TFGR. Almost half (43%) of those who responded to the question were of the view that a similar national action for the provision of technical assistance had a low probability to be implemented. This has been highlighted in particular by considering the scale of the needed reforms, the particular tight deadlines and the budgetary constraints, i.e. lacking of a specific budget line dedicated to this type of intervention. On the other end, approximately 19% of stakeholders were of the view that the technical assistance activities were likely or very likely to be implemented without the TFGR.



Figure 9: The extent to which the TA activities implemented could have been carried out without the TFGR (Number of respondents: 70)

Source: Ernst & Young Questionnaire for targeted stakeholder consultation (2018) 7.Conclusions The purpose of the evaluation of activities carried out by the TFGR was to reply to the two main questions raised by the Court of Auditors, 1)To what extent was the EU TA provided to Greece under the coordination of the TFGR managed effectively? 2)To what extent has the EU TA provided to Greece under the coordination of the TFGR contributed positively to the process of reforms in Greece? Based on the external study findings and by the data gathered in the context of the evaluation process, the following conclusions were drawn: 1)The TFGR intervention was overall relevant in satisfying the needs, given the particular situation in Greece where immediate and wide-ranging of actions were necessary to redress the financial and socio-economic situation. The TFGR intervened swiftly, mobilised support from the Member States and the European Commission and adequately coordinated the requests from the Greek authorities for assistance with available expertise. 2). The specific objectives of the TFGR were largely achieved. The TFGR succeeded to a high extent to detail the needs of the Greek authorities and to define concrete actions for reform implementation. The high-level meetings played an important role in elaborating and matching Greek authorities’ needs with the technical assistance. The TFGR involved an extensive network of international organisations, European Commission representatives and experts for this purpose in order to be able to offer to Greek authorities the needed support. 3)The detailed technical assistance needs of the Greek authorities were timely identified, given the swift operationalisation of the TFGR. Furthermore, identification and mobilisation of expertise was done in a very short period thanks to the direct support provided by the TFGR and by the European Commission staff, as well as thanks to the solidarity demonstrated by the Member States. The TFGR human resources and organisational set-up were efficiently deployed, making use of flexibility within the existing European Commission rules, specifically in order to enhance time efficiencies. The pace of implementation of the support for structural reforms in Greece was adequate, even if marked by contextual and institutional factors such as the political instability. 4)The size of TFGR’s budget was appropriate and proportional to the type of intervention it was set out to achieve – integrating several funding sources to finance the actions of the TFGR. Administrative burden was reasonable. 5)Based on its mandate, the TFGR intervention was in line with the commitments undertaken by the Greek Government under the economic adjustment programmes. The coordination mechanism put in place at the level of European Commission services and in particular between the TFGR and DG EMPL (in the latter, through a dedicated financial team and the geographical unit in charge of Greece) focused primarily on avoiding potential funding overlaps. The TFGR actions contributed then to a coherent implementation of EU law and policies. 6)The TFGR provided significant EU added value by achieving better results than what could have been achieved solely by the actions of the supported Member State, by leveraging on the high-level and senior expertise of various DGs of the European Commission and of the involved Member States. 7)The TFGR contributed to the development and implementation of solutions that addressed national challenges that have an impact on cross-border or Union-wide challenges. In addition, the TFGR supported the promotion of EU values, in particular solidarity within the European Union. Both the European Commission and EU Member States provided expertise and financed the cost of such expertise. Good practices were exchanged extensively in almost all policy areas both to support the planning of needed reforms, as well as to develop and/or pilot specific solutions for reforms. Out of the solutions implemented in Greece, several could be identified as transferable good practices (e.g. trade facilitation roadmap in business environment, the realisation of job descriptions with respect to administrative reform at central level). Finally, most of the stakeholders recognized the EU added value of the TFGR intervention, either by its direct or coordinated technical contribution or by its contribution to triggering some organisational and functional changes at the level of Greek authorities. 7.1Lessons learned 1)Future similar interventions should be clearly framed from the start. An in-depth problem analysis, the key elements of the intervention and the identification of the objectives to be achieved, the definition of the expected results with the respective indicators constitute the necessary programming phase to be performed before any action. Although, in case of economic and financial crisis - when the quick deployment of assistance is required – this is hard to be implemented. 2)An adequate monitoring system would also need to be put in place from the outset of the intervention. 3)The coherence with other EU/national interventions implemented by the Member State requesting the support should be set as a pre-condition for the award of support. It would benefit the recipient country both in terms of impacts as well as sustainability of reform actions. 4)The creation of a dedicated budget upon the set-up of similar instruments could lead to more predictability in the planning of the support for all relevant stakeholders and enable a swift delivery of support on the ground. 5)The request of support received in the framework of similar interventions need to be substantiated by a comprehensive needs identification based on which the objectives of the needed support can be drawn, as well as the nature, volume, length and sequence of activities can be determined. 6)Attention should be paid to the project management and technical capacity available at the level of the entity requesting the support.

7)The partnership between the beneficiary institution and providers of technical support should be strengthened, to ensure ownership and endorsement/adoption of outputs and to facilitate the reform implementation process. 8)The technical assistance should focus on operational activities related to the implementation of reforms and building, as needed, the capacity of the requesting institutions to implement such reforms. 9)A structured database of technical assistance providers should be developed to facilitate rapid identification of required expertise. 10)The role of the institution managing the intervention should be clearly defined in alignment with the tasks and responsibilities of other stakeholders involved. The extent to which the beneficiary institution applies/uses the outputs of the technical support should be monitored. 11)A mechanism should be put in place to clarify the obligations of supported beneficiaries with a view to ensuring that expected results are obtained. 12)A communication strategy should be developed and implemented. Annex 1: Procedural information 1.Lead DG, Decide Planning/CWP references Secretariat General/Structural Reform Support Service - Decide planning reference: PLAN/2017/652 2.Organisation and timing The evaluation was lunched in the last quarter of 2017 and was concluded in May 2019. 3.Exceptions to the better regulation guidelines A public consultation has not been carried out, because of limited extra value, considering the comprehensive targeted consultation of all involved stakeholders. 4.Consultation of the RSB (if applicable) : NA 5.Evidence, sources and quality The evaluation is based on the conclusions of the external evaluation as well as on the including the results of the preliminary evaluation carried out in 2013. The external evaluation study conducted in 2018 gathered qualitative and quantitative data using a variety of methods that include: desk research of relevant documents; structured electronic survey tailored to each stakeholder category; in-depth semi-structured interviews with stakeholders. Limitations of the current evaluation: 1)The data collected through the electronic survey were not based on a pre-identified statistically representative sample of respondents as the overall population of stakeholders was not known prior to the launch of the survey. However, all stakeholders for whom contact data were made available to the evaluator by the various Commission services were invited to participate in the survey in order to allow the widest possible coverage. Furthermore, all stakeholder categories provided inputs into the electronic survey and additional in-depth interviews were conducted to balance the representation of stakeholder category by the main policy areas; and

2)The partial availability of information from secondary sources (i.e. detailed data regarding the outputs and results of the actions) was compensated by the use of in-depth interviews conducted with stakeholders at the level of the main policy areas and other information available. Despite the above-mentioned limitations, the evaluation findings are based on evidence as triangulated by the data collected through desk research, by the targeted survey and by the in-depth interviews with all relevant stakeholder categories, with an “a posteriori” balanced representation at the level of the main policy areas targeted by the TFGR and, therefore, they can be considered conclusive. Annex 2 (Stakeholder consultation activities) - Synopsis Report 1.Outline of the consultation The synopsis report presents the results of all stakeholder consultation activities conducted in the framework of the Evaluation of the TA provided by the TFGR by the external contractor. Thus, it presents the opinions of the stakeholders, as reflected through the two data collection tools, the interviews and the online survey. The triangulation of the information hereby collected with the evidence extracted through desk research, and the corresponding objective conclusions taken on this basis are the subject of the final evaluation report itself. The targeted consultation included the following categories of stakeholders with enhanced interest and influence over TFGR activities: 1)strategic and management – institutions and individuals involved in the set-up, overall strategic steering and day-to-day management of TFGR interventions (i.e. EC officials, members of the High-Level Group (HLG), TFGR members of staff); 2)beneficiaries – institutions benefiting from TA actions coordinated / delivered by TFGR; and 3)providers – international organizations, national development agencies, and individual experts providing TA support. The following data collection tools were used in order to reach out to the relevant audience and to collect the appropriate information: 1)targeted consultation through an electronic survey based on questionnaires customised by stakeholder category; and 2)in-depth, semi-structured interviews based on interview guidelines customised by stakeholder category. The electronic survey was launched in English and Greek on 2 May 2018 and sent out to 397 stakeholders, after having asked for the consent for using contact details (i.e. email addresses). Out of the 278 individuals reached by the survey, 101 individuals responded, representing a response rate of 36%. It can also be pointed out that, at times, respondents answered “I don’t know” to the survey questions, which can be attributed on a case-by-case basis to issues like a limited involvement in scope or in duration during the TFGR intervention. Following the completion of the electronic survey, a total of 36 in-depth, semi structured interviews (i.e. with some predefined questions and guidelines to follow) have been conducted, beyond the 30 interviews planned in the inception phase of the evaluation. The aim of the interviews was three-fold: 1)investigating specific issues raised through the survey that merit being explored in detail; 2)understanding issues that received contradictory and inconsistent feedback by different respondents to the survey; and 3)collecting information to fill any gap from the survey. The approach to the stakeholder consultation, both through the survey and the interviews, ensured that all minimum standards of the Commission regarding stakeholder consultation were respected.

2.Consulted stakeholders’ profiles The analysis reveals that the joint use of the electronic survey and the in-depth interviews ensured an overall balanced representation of stakeholders participating in the targeted consultation, both from the perspective of representation of stakeholder categories, and from the perspective of representation at policy level. The following figures present the stakeholder categories consulted during the electronic survey and respectively, in-depth interviews.



Source: Ernst & Young, Analysis 2018 Figure 11: Number of interviews by stakeholder category (Number of interviewees: 36) Source: Ernst & Young, Analysis 2018 At the same time, there was a balanced representation among policy areas, with more emphasis on those having the largest share in the budget. When a policy area was not well represented in the survey, they were complemented by interviews.



Figure 12: Respondents to the survey by policy field (Number of respondents: 101) Source: Ernst & Young, Analysis 2018 Figure 13: Number of interviews by policy field (Number of interviewees: 36)